Bug 1088463 - ImageMagick fails test suite when no ghostscript-fonts are installed (e.g. when built against ghostscript 9.23)
ImageMagick fails test suite when no ghostscript-fonts are installed (e.g. wh...
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Other
Current
All openSUSE Factory
: P5 - None : Normal (vote)
: ---
Assigned To: Petr Gajdos
E-mail List
https://build.opensuse.org/package/vi...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-04-06 11:26 UTC by Dominique Leuenberger
Modified: 2018-04-18 13:45 UTC (History)
2 users (show)

See Also:
Found By: Development
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dominique Leuenberger 2018-04-06 11:26:35 UTC
We received an update request for ghostscript to version 9.23 in Tumbleweed:
  https://build.opensuse.org/request/show/590297

The staging group with this ghostscript update shows reproducible build failure of ImageMagick, in the test suite:

[  410s] Testsuite summary for ImageMagick 7.0.7
[  410s] ============================================================================
[  410s] # TOTAL: 86
[  410s] # PASS:  79
[  410s] # SKIP:  0
[  410s] # XFAIL: 0
[  410s] # FAIL:  7
[  410s] # XPASS: 0
[  410s] # ERROR: 0

This is currently all in Staging:M and can be reproduced, analysed there:

Full build log of ImageMagick:
https://build.opensuse.org/build/openSUSE:Factory:Staging:M/standard/x86_64/ImageMagick/_log
Comment 1 Johannes Meixner 2018-04-06 12:47:02 UTC
More precisely what exact ImageMagick tests fail:
---------------------------------------------------------------------
$ osc rbl openSUSE:Factory:Staging:M ImageMagick standard x86_64
...
[  270s] FAIL: tests/wandtest.tap 1
[  271s] FAIL: Magick++/tests/tests.tap 10
[  272s] FAIL: Magick++/demo/demos.tap 2
[  272s] FAIL: Magick++/demo/demos.tap 3
[  272s] FAIL: Magick++/demo/demos.tap 5
[  272s] FAIL: Magick++/demo/demos.tap 6
[  272s] FAIL: Magick++/demo/demos.tap 7
---------------------------------------------------------------------
Comment 2 Johannes Meixner 2018-04-06 12:51:22 UTC
Petr Gajdos,
I had a quick look at the *.tap files of the failing tests
but from that files I do not understand what actually
is done by those tests (I am not a ImageMagick user).
Comment 3 Petr Gajdos 2018-04-06 12:57:43 UTC
Thanks, I will look at it next week.
Comment 4 Petr Gajdos 2018-04-06 13:03:27 UTC
No, upstream issue #1058 does not seem to be related.
Comment 5 Johannes Meixner 2018-04-06 13:32:57 UTC
I mentioned upstream issue #1058 only to indicate that
ImageMagick test failures seem to happen more often recently,
in particular ImageMagick.spec contains
# do not run tests on i586 at all
# https://github.com/ImageMagick/ImageMagick/issues/1019
# https://github.com/ImageMagick/ImageMagick/issues/1058
# https://github.com/ImageMagick/ImageMagick/issues/1059
Comment 6 Johannes Meixner 2018-04-06 14:47:55 UTC
I did a
$ osc copypac -e graphics ImageMagick Printing ImageMagick
and in Printing ImageMagick I changed
----------------------------------------------------------------------------
--- ImageMagick.spec (revision 352)
+++ ImageMagick.spec (revision 2)
@@ -364,15 +364,15 @@
 # https://github.com/ImageMagick/ImageMagick/issues/1059
 rm PerlMagick/t/montage.t
 %endif
-make %{?_smp_mflags} check
+make %{?_smp_mflags} check || cat test-suite.log
 export MAGICK_CODER_MODULE_PATH=$PWD/coders/.libs
 export MAGICK_CODER_FILTER_PATH=$PWD/filters/.libs
 export MAGICK_CONFIGURE_PATH=$PWD/config
 cd PerlMagick
 %if 0%{?suse_version} >= 1315
-make %{?_smp_mflags} test
+make %{?_smp_mflags} test || cat test-suite.log
 %else
-make test_dynamic
+make test_dynamic || cat test-suite.log
 %endif
 cd ..
----------------------------------------------------------------------------
and now
$ osc rbl Printing ImageMagick openSUSE_Tumbleweed x86_64
shows the ImageMagick test-suite.log
but again the messages in test-suite.log like
------------------------------------------------------------------------
tests/wandtest.c main 5321 non-conforming drawing primitive
 definition `text' @ error/draw.c/DrawImage/3401
...
Caught exception: shapes: unable to read font `helvetica'
------------------------------------------------------------------------
do not really tell me what the root cause is.

Regarding "unable to read font `helvetica'"
I guess this might be because since ghostscript version 9.23
the ghostscript RPM does no longer require any ghostscript-fonts, see 
https://build.opensuse.org/package/view_file/Printing/ghostscript/ghostscript.changes?expand=1
that reads (excerpt):
------------------------------------------------------------------------
Fri Mar 16 12:39:36 CET 2018 - jsmeix@suse.de
...
- Do no longer require any fonts packages in particular
  neither require ghostscript-fonts-std because the PostScript
  Base35 fonts are provided by Ghostscript (in 'Resource')
  nor require ghostscript-fonts-other (provides Bitream Charter,
  Adobe Utopia, URW Antiqua, URW Grotesq and Hershey fonts where
  all but the last are also provided by texlive-<name>-fonts) and
  those fonts are not required for PostScript compliance, see
  https://bugzilla.opensuse.org/show_bug.cgi?id=1082896#c13
------------------------------------------------------------------------

Accordingly when building graphics/ImageMagick
ghostscript-fonts-std and ghostscript-fonts-other
get installed in the build system
------------------------------------------------------------------------------
$ osc rbl graphics ImageMagick openSUSE_Tumbleweed x86_64 | grep ghostscript-
...
[   50s] ghostscript-fonts-std-9.06-10.2
[   50s] ghostscript-fonts-other-9.06-10.2
[   53s] ghostscript-9.22-2.1
[   60s] ghostscript-devel-9.22-2.1
------------------------------------------------------------------------------
while in contrast building Printing/ImageMagick
only ghostscript gets installed in the build system
------------------------------------------------------------------------------
$ osc rbl Printing ImageMagick openSUSE_Tumbleweed x86_64 | grep ghostscript-
...
[   52s] ghostscript-9.23-103.1
[   57s] ghostscript-devel-9.23-103.1
------------------------------------------------------------------------------

Because ghostscript-fonts is outdated and will be replaced
by new urw-base35-fonts package see bug #1074266
the ImageMagick tests should probably work generically without
the specific ghostscript-fonts-std and ghostscript-fonts-other.
Comment 7 Johannes Meixner 2018-04-06 15:08:15 UTC
Now I build Printing/ImageMagick with this additional change
--------------------------------------------------------------------------
--- ImageMagick.spec (revision 352)
+++ ImageMagick.spec (revision 3)
@@ -86,6 +86,8 @@
 BuildRequires:  pkgconfig(freetype2)
 %if 0%{?suse_version} > 1315
 BuildRequires:  pkgconfig(ijs)
+BuildRequires:  ghostscript-fonts-other
+BuildRequires:  ghostscript-fonts-std
 %else
 BuildRequires:  ghostscript-fonts-other
 BuildRequires:  ghostscript-fonts-std
--------------------------------------------------------------------------
to get ghostscript-fonts-std and ghostscript-fonts-other
installed in the build system in any case and
now the build log shows
--------------------------------------------------------------------------
$ osc rbl Printing ImageMagick openSUSE_Tumbleweed x86_64
...
[  677s] Testsuite summary for ImageMagick 7.0.7
...
[  677s] # TOTAL: 86
[  677s] # PASS:  86
[  677s] # SKIP:  0
[  677s] # XFAIL: 0
[  677s] # FAIL:  0
[  677s] # XPASS: 0
[  677s] # ERROR: 0
--------------------------------------------------------------------------

Accordingly as a SUSE-specific short-term fix ImageMagick.spec
should BuildRequire ghostscript-fonts-std and ghostscript-fonts-other
in any case to make the ImageMagick tests no longer fail.

The real solution at ImageMagick upstream should be
to change the tests to be more generic so that
no longer the specific ghostscript-fonts are needed.
Comment 8 Johannes Meixner 2018-04-06 15:32:12 UTC
I think I got building ImageMagick fixed in Printing/ImageMagick
----------------------------------------------------------------------
$ osc results -v Printing ImageMagick
...
openSUSE_Tumbleweed  i586       succeeded(unpublished)
openSUSE_Tumbleweed  x86_64     succeeded(unpublished)
openSUSE_Leap_15.0   x86_64     succeeded(unpublished)
openSUSE_Factory_PowerPC ppc64      succeeded(unpublished)
openSUSE_Factory_PowerPC ppc64le    succeeded(unpublished)
...
SLE_12_SP2           x86_64     succeeded(unpublished)
SLE_12_SP2           i586       succeeded(unpublished)
SLE_12_SP2           ppc64le    succeeded(unpublished)
----------------------------------------------------------------------
by only this change
----------------------------------------------------------------------
$ osc rdiff graphics ImageMagick Printing ImageMagick
...
--- ImageMagick.spec (revision 352)
+++ ImageMagick.spec (revision 4)
@@ -86,6 +86,8 @@
 BuildRequires:  pkgconfig(freetype2)
 %if 0%{?suse_version} > 1315
 BuildRequires:  pkgconfig(ijs)
+BuildRequires:  ghostscript-fonts-other
+BuildRequires:  ghostscript-fonts-std
 %else
 BuildRequires:  ghostscript-fonts-other
 BuildRequires:  ghostscript-fonts-std
----------------------------------------------------------------------
that BuildRequires ghostscript-fonts-std and ghostscript-fonts-other
in any case.

I guess "BuildRequires: ghostscript-fonts-other" is not needed.
I will test that next Monday.

Have a nice weekend!
Comment 9 Petr Gajdos 2018-04-09 09:30:48 UTC
Johannes,

I had a very nice weekend and I hope you too.

Thanks for your analysis.

Helvetica font family is hardcoded on several places in ImageMagick. For example, it is requesting 'helvetica' shape alternative as a fallback in MagickCore/annotate.c:

[...]
  if (draw_info->font == (char *) NULL)
    args.pathname=ConstantString("helvetica");
  else
[...]

This is causing the exception during run of 4 referenced demo files in comment 1:

Caught exception: button: unable to read font `helvetica' @ error/annotate.c/RenderFreetype/1432

So I think that not only tests of ImageMagick will be affected by removal of ghostscript-fonts. Question is why FT_Open_Face() is failing.
Comment 10 Petr Gajdos 2018-04-09 10:32:17 UTC
Hmm, according to https://www.freetype.org/freetype2/docs/reference/ft2-base_interface.html#FT_Open_Args

args.pathname should be existing path name:

     pathname      A pointer to an 8-bit file pathname.

So 'helvetica' string would never worked. This moves the issue in different location it seems -- the fallback was not used before but it is now.
Comment 11 Petr Gajdos 2018-04-09 11:24:15 UTC
On the system with ghostscript fonts installed:

$ convert /home/abuild/rpmbuild/BUILD/ImageMagick-7.0.7-28/images/logo.jpg -annotate +0+10 'logo' logo-annotated.jpg
$ identify logo-annotated.jpg
logo-annotated.jpg JPEG 123x118 123x118+0+0 8-bit sRGB 18999B 0.000u 0:00.000
$

On the system without ghostscript fonts installed (just dejavu-fonts there per corresponding build requires):

$ convert /home/abuild/rpmbuild/BUILD/ImageMagick-7.0.7-28/images/logo.jpg -annotate +0+10 'logo' logo-annotated.jpg
convert: unable to read font `helvetica' @ error/annotate.c/RenderFreetype/1432.
$

So some sort of fallback is used, which is not translated to correct font file, it seems. Obviously FT_Open_Face() is not supposed to handle such sort of fallback.

Just note, on both systems obviously:

$ fc-match helvetica
DejaVuSans.ttf: "DejaVu Sans" "Book"
$
Comment 12 Johannes Meixner 2018-04-09 11:28:04 UTC
Petr Gajdos,

I am not talking about that ImageMagick should not require
one of the fonts in the PostScript Core Font Set, cf.
https://en.wikipedia.org/wiki/PostScript_fonts#Core_Font_Set

It is perfectly right when a program requires a PostScript Core Font
like Helvetica or Helvetica-Bold or Helvetica-BoldOblique.
Note the exact spelling of a PostScript Core Font name.

I am talking about that in the future ImageMagick
should not require the specific ghostscript-fonts-std
or ghostscript-fonts-other RPM packages.

It should not matter by which specific RPM package
the PostScript Core Font Set is provided.

I know basically nothing at all about font packaging.

I guess perhaps something might be missing in
our current ghostscript-9.23 package because
it provides the PostScript Core Font Set in
/usr/share/ghostscript/9.23/Resource/Font/
but it seems somehow ImageMagick cannot use them.

In
/usr/share/ghostscript/9.23/Resource/Init/Fontmap.GS
there is this font name mapping for Helvetica fonts
-------------------------------------------------------------------------
/Helvetica-Bold                 /NimbusSans-Bold ;
/Helvetica-BoldOblique          /NimbusSans-BoldItalic      ;
/Helvetica-Narrow-Bold          /NimbusSansNarrow-Bold    ;
/Helvetica-Narrow-BoldOblique   /NimbusSansNarrow-BoldOblique        ;
/Helvetica-Narrow               /NimbusSansNarrow-Regular    ;
/Helvetica-Narrow-Oblique       /NimbusSansNarrow-Oblique        ;
/Helvetica                      /NimbusSans-Regular ;
/Helvetica-Oblique              /NimbusSans-Italic      ;
-------------------------------------------------------------------------

I guess this font name mapping is only for Ghostscript
but other programs like ImageMagick won't know that
e.g. a Helvetica-BoldOblique font is provided in
/usr/share/ghostscript/9.23/Resource/Font/NimbusRoman-BoldItalic
Comment 13 Petr Gajdos 2018-04-09 12:23:17 UTC
And I know basically nothing about postscript :).

I think there is at least following behaviour in ImageMagick which does not fit to what I would expect:

System A
--------
$ rpm -qa | grep 'fonts\|ghostscript'
ghostscript-fonts-std-9.06-10.2.noarch
ghostscript-devel-9.22-2.1.x86_64
ghostscript-9.22-2.1.x86_64
ghostscript-fonts-other-9.06-10.2.noarch
dejavu-fonts-2.37-1.5.noarch
$ convert /home/abuild/rpmbuild/BUILD/ImageMagick-7.0.7-28/images/logo.jpg -annotate +0+10 'logo' logo-annotated.jpg
$

logo-annotated.jpg seem to be correctly rendered (logo + 'logo' text in it). convert -list font see ghostscript fonts and dejavu ones.

System B
--------
$ rpm -qa | grep 'fonts\|ghostscript'
dejavu-fonts-2.37-1.5.noarch
ghostscript-9.23-104.1.x86_64
ghostscript-devel-9.23-104.1.x86_64
$ convert /home/abuild/rpmbuild/BUILD/ImageMagick-7.0.7-28/images/logo.jpg -annotate +0+10 'logo' logo-annotated.jpg
convert: unable to read font `helvetica' @ error/annotate.c/RenderFreetype/1432.
$

convert -list font see dejavu fonts only, I would expect it uses them.
Comment 14 Petr Gajdos 2018-04-09 13:41:19 UTC
--with-urw-base35-font-dir configure option looks promising, will try.
Comment 15 Petr Gajdos 2018-04-09 14:36:08 UTC
It seems that the key is in incorrect values of

/etc/ImageMagick-7_Q16HDRI5/type-urw-base35.xml
and
/etc/ImageMagick-7_Q16HDRI5/type-ghostscript.xml

xml files. Will try to investigate more tomorrow.
Comment 16 Petr Gajdos 2018-04-10 08:53:26 UTC
Johannes, could you please accept sr#595143 and report results back?
Comment 17 Petr Gajdos 2018-04-10 17:42:23 UTC
Johannes,

I had seen the ImageMagick package built (finally!) correctly in Printing, so I moved on and submitted it to Factory.

I have disabled the support of urw-base35-fonts now:
 
 %define urw_base35_fonts 0

but the spec should be prepared for drop of ghostscript-fonts in favor of urw-base35-fonts.

Now it is basically your proposed change, that means package just buildrequires ghostscript-fonts.

Dominique, please check it.
Comment 18 Swamp Workflow Management 2018-04-10 18:00:06 UTC
This is an autogenerated message for OBS integration:
This bug (1088463) was mentioned in
https://build.opensuse.org/request/show/595350 Factory / ImageMagick
Comment 19 Johannes Meixner 2018-04-11 09:08:39 UTC
Petr Gajdos,
many thanks for your work, in particular for preparing ImageMagick
for drop of ghostscript-fonts in favor of urw-base35-fonts.
It is much appreciated!

FYI:
Yesterday I watched the ImageMagick build results in Printing
but because in Printing by default the packages are not
used for building other packages (I can explain you the
reason behind if you are interested in the details)
I had to set urw-base35-fonts to be used for building
other packages (for building ImageMagick in this case)
and that delayed the ImageMagick build results in Printing.
Comment 20 Swamp Workflow Management 2018-04-11 12:00:10 UTC
This is an autogenerated message for OBS integration:
This bug (1088463) was mentioned in
https://build.opensuse.org/request/show/595596 Factory / ImageMagick
Comment 22 Petr Gajdos 2018-04-18 13:45:05 UTC
Both ImageMagick change and ghostscript 9.23 are in factory, so I assume this issue is fixed.