Bug 1206322 - [Build 20221211] graphicsMagick fails test suite on i586
[Build 20221211] graphicsMagick fails test suite on i586
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Other
Current
i586 Other
: P5 - None : Normal (vote)
: ---
Assigned To: Petr Gajdos
E-mail List
https://openqa.opensuse.org/tests/294...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2022-12-12 11:15 UTC by Dominique Leuenberger
Modified: 2023-01-12 13:19 UTC (History)
3 users (show)

See Also:
Found By: openQA
Services Priority:
Business Priority:
Blocker: Yes
Marketing QA Status: ---
IT Deployment: ---


Attachments
increase the tolerance value for the compare of images created by montage (764 bytes, patch)
2023-01-10 06:31 UTC, Dirk Weber
Details | Diff
updated patch to use higher tolerance only if uname -m =~ i.86 (1.15 KB, patch)
2023-01-10 18:10 UTC, Dirk Weber
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dominique Leuenberger 2022-12-12 11:15:34 UTC
## Observation


The graphicsMagick test has been identified as failing on i586 (32bit)
Unfortunately, the QA test does not reveal which two tests out of the suite failed. I'll try to collect this information

openQA test in scenario opensuse-Tumbleweed-LegacyX86-DVD-i586-gnome@32bit fails in
[graphicsMagick](https://openqa.opensuse.org/tests/2949140/modules/graphicsMagick/steps/19)

## Test suite description
Revert schedule to working settings - dleuenberger, 20211117


## Reproducible

Fails since (at least) Build [20221207](https://openqa.opensuse.org/tests/2942785)


## Expected result

Last good: (unknown) (or more recent)


## Further details

Always latest result in this scenario: [latest](https://openqa.opensuse.org/tests/latest?arch=i586&distri=opensuse&flavor=LegacyX86-DVD&machine=32bit&test=gnome&version=Tumbleweed)
Comment 1 Dominique Leuenberger 2022-12-19 14:23:09 UTC
REproduced this locally in a VM, the two tests returning KO are:

85 - KO - gm montage degradation.png logo-primary.png ./85-1.png;compare PAE ./85-1.png montage1.png 0
86 - KO - gm montage -texture noise.gif degradation.png logo-primary.png ./86-1.png;compare PAE ./86-1.png montage2.png 0
Comment 2 Petr Gajdos 2022-12-22 10:21:40 UTC
Let's try to remove GraphicsMagick again: bug 1206322
Comment 3 Dominique Leuenberger 2022-12-22 10:24:51 UTC
(In reply to Petr Gajdos from comment #2)
> Let's try to remove GraphicsMagick again: bug 1206322

That's ambitious:

> osc whatdependson openSUSE:Factory GraphicsMagick standard x86_64

 lists 87 packages depending on GraphicsMagick during build; not sure yet how many might have runtime deps in place
Comment 4 John Paul Adrian Glaubitz 2023-01-06 08:38:35 UTC
(In reply to Dominique Leuenberger from comment #1)
> REproduced this locally in a VM, the two tests returning KO are:
> 
> 85 - KO - gm montage degradation.png logo-primary.png ./85-1.png;compare PAE
> ./85-1.png montage1.png 0
> 86 - KO - gm montage -texture noise.gif degradation.png logo-primary.png
> ./86-1.png;compare PAE ./86-1.png montage2.png 0

How do you run this test? This doesn't seem to be part of the normal testsuite that is being at build time, is it?
Comment 5 Dominique Leuenberger 2023-01-06 08:51:14 UTC
(In reply to John Paul Adrian Glaubitz from comment #4)

> How do you run this test? This doesn't seem to be part of the normal
> testsuite that is being at build time, is it?

As comment#0 gives away, this is a test running in openQA

Always latest test run accessible under
https://openqa.opensuse.org/tests/latest?arch=i586&distri=opensuse&flavor=LegacyX86-DVD&machine=32bit&test=gnome&version=Tumbleweed

The test code is:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/x11/graphicsMagick.pm

Specifically:

>    assert_script_run "wget --quiet " . data_url('graphicsmagick/test.sh') . " -O test.sh";
>    assert_script_run "chmod +x test.sh";
>    assert_script_run("./test.sh " . data_url('graphicsmagick'), 3 * 60);

data_url gets stuff from
https://github.com/os-autoinst/os-autoinst-distri-opensuse/tree/master/data/graphicsmagick

=> essentially starting this script
https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/graphicsmagick/test.sh
Comment 6 John Paul Adrian Glaubitz 2023-01-06 09:24:30 UTC
(In reply to Dominique Leuenberger from comment #5)
> (In reply to John Paul Adrian Glaubitz from comment #4)
> 
> > How do you run this test? This doesn't seem to be part of the normal
> > testsuite that is being at build time, is it?
> 
> (...)
>
> => essentially starting this script
> https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/
> graphicsmagick/test.sh

Thanks. Very interesting to see that openQA discovered an issue that the upstream testsuite didn't cover as their testsuite passes without problems.
Comment 7 Dirk Weber 2023-01-06 09:38:01 UTC
(In reply to Dominique Leuenberger from comment #5)
> 
> data_url gets stuff from
> https://github.com/os-autoinst/os-autoinst-distri-opensuse/tree/master/data/
> graphicsmagick
> 
> => essentially starting this script
> https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/
> graphicsmagick/test.sh

thanks for providing these links.

I am trying to reproduce the test execution locally.

gm montage degradation.png logo-primary.png ./85-1.png
runs successful and produces 85-1.png

$ compare PAE ./85-1.png montage1.png 0
compare: unable to open image 'PAE': No such file or directory @ error/blob.c/OpenBlob/3570.
compare: no decode delegate for this image format `' @ error/constitute.c/ReadImage/741.

$ rpm -qf $(which compare)
ImageMagick-7.1.0.55-1.1.i586

$ rpm -q GraphicsMagick
GraphicsMagick-1.3.38-1.6.i586


if I use "gm compare" - the compare functionality of GraphicsMagick
$ gm compare PAE ./85-1.png montage1.png 0 |echo $?
0

the test passes.

$ rpm -ql  GraphicsMagick |grep compare
/usr/share/doc/packages/GraphicsMagick/www/api/compare.html
/usr/share/doc/packages/GraphicsMagick/www/compare.html

Could it be the problem is "just" that there is no "compare" script or binary in the (i586?) GraphicsMagick package?
Comment 8 Dirk Weber 2023-01-06 09:50:04 UTC
(In reply to Dirk Weber from comment #7)
> $ gm compare PAE ./85-1.png montage1.png 0 |echo $?
> 0
> 
sorry: 
$ gm compare PAE ./85-1.png montage1.png 0 ;echo $?
0


also the second test:

 gm montage -texture noise.gif degradation.png logo-primary.png ./86-1.png ; gm compare PAE ./86-1.png montage2.png 0; echo $?
0
Comment 9 Dominique Leuenberger 2023-01-06 09:51:05 UTC
(In reply to Dirk Weber from comment #7)
> $ rpm -qf $(which compare)
> ImageMagick-7.1.0.55-1.1.i586
> 
> $ rpm -q GraphicsMagick
> GraphicsMagick-1.3.38-1.6.i586
> 
> 
> if I use "gm compare" - the compare functionality of GraphicsMagick
> $ gm compare PAE ./85-1.png montage1.png 0 |echo $?
> 0
> 
> the test passes.
> 
> $ rpm -ql  GraphicsMagick |grep compare
> /usr/share/doc/packages/GraphicsMagick/www/api/compare.html
> /usr/share/doc/packages/GraphicsMagick/www/compare.html
> 
> Could it be the problem is "just" that there is no "compare" script or
> binary in the (i586?) GraphicsMagick package?


/usr/bin/compare comes from ImageMagick (as your rpm -qf $(which compare) shows too) and ImageMagick is installed on the VM (as can be confirmed in the textinfo.tzt on the asset tab of the failing tests)
Comment 10 Dirk Weber 2023-01-06 09:58:21 UTC
(In reply to Dominique Leuenberger from comment #9)
> (In reply to Dirk Weber from comment #7)
> > $ rpm -qf $(which compare)
> > ImageMagick-7.1.0.55-1.1.i586
> 
> 
> /usr/bin/compare comes from ImageMagick (as your rpm -qf $(which compare)
> shows too) and ImageMagick is installed on the VM (as can be confirmed in
> the textinfo.tzt on the asset tab of the failing tests)

yes I agree, but it seems ImageMagick's "compare" does not know PAE - it thinks it is an input file which does not exist. While GraphicsMagick's compare seems to understand PAE as a switch.

Is the same test run on x86_64?
Comment 11 Dominique Leuenberger 2023-01-06 10:06:11 UTC
(In reply to Dirk Weber from comment #10)
> 
> Is the same test run on x86_64?

yes, on x86_64 it passes:
https://openqa.opensuse.org/tests/3016273#step/graphicsMagick/15
Comment 12 Dirk Weber 2023-01-06 10:14:17 UTC
(In reply to Dominique Leuenberger from comment #11)
> (In reply to Dirk Weber from comment #10)
> > 
> > Is the same test run on x86_64?
> 
> yes, on x86_64 it passes:
> https://openqa.opensuse.org/tests/3016273#step/graphicsMagick/15

strange, when I call it the same way on x86_64 it fails in the same way:

$ rpm -q GraphicsMagick ImageMagick
GraphicsMagick-1.3.38-1.6.x86_64
ImageMagick-7.1.0.55-1.1.x86_64

$ gm montage degradation.png logo-primary.png ./85-1.png;compare PAE ./85-1.png montage1.png 0
compare: unable to open image 'PAE': No such file or directory @ error/blob.c/OpenBlob/3570.
compare: no decode delegate for this image format `' @ error/constitute.c/ReadImage/741.


and succeeds when called like this:
$ gm montage degradation.png logo-primary.png ./85-1.png;gm compare PAE ./85-1.png montage1.png 0 ; echo $?
0

On my machine:
$ rpm -qf $(which compare)
ImageMagick-7.1.0.55-1.1.x86_64
Comment 13 Dominique Leuenberger 2023-01-06 10:20:09 UTC
(In reply to Dominique Leuenberger from comment #11)
> (In reply to Dirk Weber from comment #10)
> > 
> > Is the same test run on x86_64?
> 
> yes, on x86_64 it passes:
> https://openqa.opensuse.org/tests/3016273#step/graphicsMagick/15

d'oh

https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/cabcdede86a76359a3a8711046cbd68f4b2f45a0/data/graphicsmagick/test.sh#L82

compare is also function in the test.sh script - so test.sh might not call out to the actual compare command
Comment 14 Petr Gajdos 2023-01-06 10:35:20 UTC
(In reply to Dominique Leuenberger from comment #13)
> compare is also function in the test.sh script - so test.sh might not call
> out to the actual compare command

You can try `magick compare` instead of `compare` from ImageMagick 7 on.

Btw. not sure combining both GraphicsMagick and ImageMagick in one test is a good idea in general.
Comment 15 Petr Gajdos 2023-01-06 10:43:40 UTC
(In reply to Dirk Weber from comment #12)
> $ gm montage degradation.png logo-primary.png ./85-1.png;compare PAE
> ./85-1.png montage1.png 0
> compare: unable to open image 'PAE': No such file or directory @
> error/blob.c/OpenBlob/3570.
> compare: no decode delegate for this image format `' @
> error/constitute.c/ReadImage/741.

PAE file does not exist, nothing to compare.
Comment 16 Dominique Leuenberger 2023-01-06 10:46:50 UTC
compare is the function in test.sh, not the command /usr/bin/compare

function compare {
  metric=$1
  image1=$2
  image2=$3
  tolerance=$4
  inversion=$5
  channel=$6

  if [ "$metric" = "PSNR" ]; then
    script="check_compare_PSNR.pl"
  else
    script="check_compare.pl"
  fi

  if [ -z "$channel" ]; then
    channel="Total"
  fi

  if [ ! -f $image1 ]; then
    echo "KO"
    return
  fi

  if [ ! -f $image2 ]; then
    echo "KO"
    return
  fi

  gm compare -metric $metric $image1 $image2 | grep $channel | perl $script $tolerance $inversion
}

So calling 'compare manually' without the function defined in the scope of executing compare is not the same as test.sh does

PAE does not reference a file when calling the compare function, but to the first parameter, metrics.

The name is kinda unfortunate, considering that ImageMAgick delivers a binary with this name
Comment 17 Dirk Weber 2023-01-06 10:47:33 UTC
(In reply to Dominique Leuenberger from comment #13)
> 
> compare is also function in the test.sh script - so test.sh might not call
> out to the actual compare command

that's tricky.
The compare function of the script calls gm compare like this:
function compare {
  metric=$1
  image1=$2
  image2=$3
...

gm compare -metric $metric $image1 $image2 | grep $channel | perl $script $tolerance $inversion

It uses only the first 3 parameters for "gm compare", called like this the output really differs between x86_64 and i586:

x86_64:
$ gm compare -metric PAE ./85-1.png montage1.png 
Image Difference (PeakAbsoluteError):
           Normalized    Absolute
          ============  ==========
     Red: 0.0000000000        0.0
   Green: 0.0000000000        0.0
    Blue: 0.0000000000        0.0
   Total: 0.0000000000        0.0

   
   
i586:
$ gm compare -metric PAE ./85-1.png montage1.png
Image Difference (PeakAbsoluteError):
           Normalized    Absolute
          ============  ==========
     Red: 0.3425650416    22450.0
   Green: 0.2915693904    19108.0
    Blue: 0.2654612039    17397.0
   Total: 0.3425650416    22450.0

and this could be caused by lower floating point precision on i586.
Comment 18 Dirk Weber 2023-01-06 11:10:05 UTC
(In reply to Dirk Weber from comment #17)
> 
> x86_64:
> $ gm compare -metric PAE ./85-1.png montage1.png 
> Image Difference (PeakAbsoluteError):
>            Normalized    Absolute
>           ============  ==========
>      Red: 0.0000000000        0.0
>    Green: 0.0000000000        0.0
>     Blue: 0.0000000000        0.0
>    Total: 0.0000000000        0.0
> 
>    
>    
> i586:
> $ gm compare -metric PAE ./85-1.png montage1.png
> Image Difference (PeakAbsoluteError):
>            Normalized    Absolute
>           ============  ==========
>      Red: 0.3425650416    22450.0
>    Green: 0.2915693904    19108.0
>     Blue: 0.2654612039    17397.0
>    Total: 0.3425650416    22450.0
> 
> and this could be caused by lower floating point precision on i586.

The created image 85-1.png is slightly bigger on i586 (85363 bytes) than on x86_64 (71117 bytes). But they "look" the same visually to me.

The used channel in this case - as not provided - is "Total".
Comment 19 Dirk Weber 2023-01-06 18:45:22 UTC
(In reply to Dirk Weber from comment #17)
> i586:
> $ gm compare -metric PAE ./85-1.png montage1.png
> Image Difference (PeakAbsoluteError):
>            Normalized    Absolute
>           ============  ==========
>      Red: 0.3425650416    22450.0
>    Green: 0.2915693904    19108.0
>     Blue: 0.2654612039    17397.0
>    Total: 0.3425650416    22450.0
> 
> and this could be caused by lower floating point precision on i586.

The i586 results given in comment 17 were obtained in a x86_32 VM.

On x86_32 hardware I got:
$ gm compare -metric PAE ./85-1_tw.png montage1.png
Image Difference (PeakAbsoluteError):
           Normalized    Absolute
          ============  ==========
     Red: 0.2149691005    14088.0
   Green: 0.1889066911    12380.0
    Blue: 0.1703669795    11165.0
   Total: 0.2149691005    14088.0
$ gm compare -metric PAE ./86-1_tw.png montage2.png
Image Difference (PeakAbsoluteError):
           Normalized    Absolute
          ============  ==========
     Red: 0.1409323262     9236.0
   Green: 0.1148699168     7528.0
    Blue: 0.0963302052     6313.0
   Total: 0.1409323262     9236.0

ImageMagick-7.1.0.55-1.1.i586
GraphicsMagick-1.3.38-1.6.i586

I am afraid I can not judge these results. 
I do not know whether these are precise operations which always have to 
give the same results or whether they are fuzzy operations and the result 
can e.g. depend on the used floating point unit,
like x87 or SSE2, and it is normal to have a deviation when an image "montaged" on one type
of fp-unit is compared to one "montaged" on another type of fp-unit even
when both are actually valid results.
Comment 20 Petr Gajdos 2023-01-09 15:40:04 UTC
(In reply to Dominique Leuenberger from comment #3)
> (In reply to Petr Gajdos from comment #2)
> > Let's try to remove GraphicsMagick again: bug 1206322
> 
> That's ambitious:

Yes, it was ambitious, even if I think only five packages really BuildRequires it: inkscape, scribus, octave, wt and pdf2djvu and requirements are optional in all of them. I think it is doable, but I am not sure how much time would this require in case we want to keep all the functionality [bug 1206322 comment 20].

(In reply to Dirk Weber from comment #19)
> $ gm compare -metric PAE ./85-1_tw.png montage1.png
> Image Difference (PeakAbsoluteError):
>            Normalized    Absolute
>           ============  ==========
>      Red: 0.2149691005    14088.0
>    Green: 0.1889066911    12380.0
>     Blue: 0.1703669795    11165.0
>    Total: 0.2149691005    14088.0
> $ gm compare -metric PAE ./86-1_tw.png montage2.png
> Image Difference (PeakAbsoluteError):
>            Normalized    Absolute
>           ============  ==========
>      Red: 0.1409323262     9236.0
>    Green: 0.1148699168     7528.0
>     Blue: 0.0963302052     6313.0
>    Total: 0.1409323262     9236.0
> 
> ImageMagick-7.1.0.55-1.1.i586
> GraphicsMagick-1.3.38-1.6.i586
> 
> I am afraid I can not judge these results. 
> I do not know whether these are precise operations which always have to 
> give the same results or whether they are fuzzy operations and the result 
> can e.g. depend on the used floating point unit,
> like x87 or SSE2, and it is normal to have a deviation when an image
> "montaged" on one type
> of fp-unit is compared to one "montaged" on another type of fp-unit even
> when both are actually valid results.

Thank you Dirk for your evaluation.

I really do not think testing ImageMagick with GraphicsMagick and vice versa is a good idea; the problem, if any at all (reading previous comments), can be in either of them and I am almost convinced who will have to tackle these potential issues down. But ok, I understand you guys want me to have a nice hackweek, right ;)?
Comment 21 Petr Gajdos 2023-01-09 15:43:32 UTC
(In reply to Petr Gajdos from comment #20)
> functionality [bug 1206322 comment 20].

1206620 comment 20
Comment 22 Dirk Weber 2023-01-09 16:08:49 UTC
(In reply to Petr Gajdos from comment #20)

> I really do not think testing ImageMagick with GraphicsMagick and vice versa
> is a good idea; the problem, if any at all (reading previous comments), can
> be in either of them and I am almost convinced who will have to tackle these
> potential issues down. But ok, I understand you guys want me to have a nice
> hackweek, right ;)?

Sorry, the "compare" used in the test is not the compare binary from ImageMagick but the compare function of the script test.sh which correctly uses 
gm compare ...

I was on the wrong track but Dominique corrected me, thanks for that.

Therefore no mixing of ImageMagick and GraphicsMagick in these test cases.

I think whether the NOK test result of i586 is a valid finding and not a false positive depends on the question if the "montage" operation is precise or not.
I almost tend to: it is not (if floating point values are involved and therefore limited precision and rounding).

The compare function of the test.sh script can accept a value for a tolerance. 
So maybe this tolerance value should be used in order to allow the test to pass as long as an image is created and the program does not crash. 
A value like 0.4.
Comment 23 Dirk Weber 2023-01-10 06:24:48 UTC
(In reply to Dirk Weber from comment #22)
> So maybe this tolerance value should be used in order to allow the test to
> pass as long as an image is created and the program does not crash. 
> A value like 0.4.

To be specific, the suggested change would be:
diff test.sh test_new.sh 
353c353
<   "gm montage degradation.png logo-primary.png __1.png;compare PAE __1.png montage1.png 0"
---
>   "gm montage degradation.png logo-primary.png __1.png;compare PAE __1.png montage1.png 0.4"
356c356
<   "gm montage -texture noise.gif degradation.png logo-primary.png __1.png;compare PAE __1.png montage2.png 0"
---
>   "gm montage -texture noise.gif degradation.png logo-primary.png __1.png;compare PAE __1.png montage2.png 0.4"
Comment 24 Dirk Weber 2023-01-10 06:31:25 UTC
Created attachment 863962 [details]
increase the tolerance value for the compare of images created by montage

The images created by montage seem to depend on the HW they are created on. Therefore tolerate deviating values in the output of gm compare which compares the image created during the test with a given image possibly created on an other HW type.
Comment 25 Dominique Leuenberger 2023-01-10 08:33:01 UTC
(In reply to Dirk Weber from comment #24)
> Created attachment 863962 [details]
> increase the tolerance value for the compare of images created by montage
> 
> The images created by montage seem to depend on the HW they are created on.
> Therefore tolerate deviating values in the output of gm compare which
> compares the image created during the test with a given image possibly
> created on an other HW type.

I'm not objecting that change per se, but the statement is kinda broad (a bit too broad):

The test passes as-is on
* x86_64
* aarch64
* ppc64le
(granted, they are all 64bitarches)

I think I'd rather want to change test.sh to only accept that 0.4 variation on i586 arches, not allowing for variation on the arches that are stable already (alternatively, exclude those two tests on i586)
Comment 26 Petr Gajdos 2023-01-10 09:11:50 UTC
(In reply to Dirk Weber from comment #24)
> The images created by montage seem to depend on the HW they are created on.
> Therefore tolerate deviating values in the output of gm compare which
> compares the image created during the test with a given image possibly
> created on an other HW type.

This is interesting and I will look at it during hackweek (tbh I have no other idea so far).

(In reply to Dirk Weber from comment #22)
> Sorry, the "compare" used in the test is not the compare binary from
> ImageMagick but the compare function of the script test.sh which correctly
> uses 
> gm compare ...
> 
> I was on the wrong track but Dominique corrected me, thanks for that.
> 
> Therefore no mixing of ImageMagick and GraphicsMagick in these test cases.

Ah, now I understand, thanks. I was misled by

(In reply to Dirk Weber from comment #19)
> ImageMagick-7.1.0.55-1.1.i586
> GraphicsMagick-1.3.38-1.6.i586

Thank you Dirk for the evaluation again.

By the way: do you know why we are testing GraphicsMagick and not ImageMagick? Or is there similar test for ImageMagick?

TL;DR:
GraphicsMagick is just an old fork of ImageMagick. ImageMagick is more developed, but less stable. They broke API in IM7, GM keeps the API same as IM6, that's perhaps one of reasons why some projects prefer GraphicsMagick (octave is one of examples, used to build also with IM6). But ImageMagick feels little bit like a mainstream.
Comment 27 Petr Gajdos 2023-01-10 09:14:34 UTC
(In reply to Dominique Leuenberger from comment #0)
> ## Test suite description
> Revert schedule to working settings - dleuenberger, 20211117
> 
> 
> ## Reproducible
> 
> Fails since (at least) Build
> [20221207](https://openqa.opensuse.org/tests/2942785)
> 
> 
> ## Expected result
> 
> Last good: (unknown) (or more recent)

So this is a regression or not? Does it mean that it started to fail after <revert> and (before <revert> and before before <revert>) not? This sounds fishy a bit.
Comment 28 Dominique Leuenberger 2023-01-10 10:52:25 UTC
(In reply to Petr Gajdos from comment #27)
> > ## Expected result
> > 
> > Last good: (unknown) (or more recent)
> 
> So this is a regression or not? Does it mean that it started to fail after
> <revert> and (before <revert> and before before <revert>) not? This sounds
> fishy a bit.

Not really. The i586 version was just never tested and after splitting out i586 from openSUSE:Factory to :LegacyX86 (and having it roll independently) the test suite was slightly increased, where this popped up (hence: last good: unknown)
Comment 29 Dominique Leuenberger 2023-01-10 13:15:25 UTC
(In reply to Petr Gajdos from comment #26)
> By the way: do you know why we are testing GraphicsMagick and not
> ImageMagick? Or is there similar test for ImageMagick?

There is a test in the os-autoinst-distri repository, but this test seems not to be running as part of Tumbleweed's test suite

https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/cae6400303ef59c58de143fbaf3050bf5e22457e/tests/x11/ImageMagick.pm

Might be something to look at besides this issue
Comment 30 Dirk Weber 2023-01-10 18:10:16 UTC
Created attachment 863992 [details]
updated patch to use higher tolerance only if uname -m =~ i.86

Updated patch to use tolerance 0.4 only if CPU type matches i.86
Comment 31 Dirk Weber 2023-01-10 18:21:50 UTC
(In reply to Dominique Leuenberger from comment #25)
> (In reply to Dirk Weber from comment #24)
> > Created attachment 863962 [details]
> > increase the tolerance value for the compare of images created by montage
> > 
> > The images created by montage seem to depend on the HW they are created on.
> > Therefore tolerate deviating values in the output of gm compare which
> > compares the image created during the test with a given image possibly
> > created on an other HW type.
> 
> I'm not objecting that change per se, but the statement is kinda broad (a
> bit too broad):
> 
> The test passes as-is on
> * x86_64
> * aarch64
> * ppc64le
> (granted, they are all 64bitarches)
> 
> I think I'd rather want to change test.sh to only accept that 0.4 variation
> on i586 arches, not allowing for variation on the arches that are stable
> already (alternatively, exclude those two tests on i586)
You are right, I modified the patch to use the higher tolerance value only for i.86 - my VM identifies as i686.



(In reply to Petr Gajdos from comment #26)
> (In reply to Dirk Weber from comment #24)
> By the way: do you know why we are testing GraphicsMagick and not
> ImageMagick? Or is there similar test for ImageMagick?

I do not know why the test is like it is. 
I mainly tried to find out what causes the failure because I want to support keeping the i586 build around and avoid the impression that nobody cares about failing tests.
Comment 32 Dominique Leuenberger 2023-01-11 06:48:28 UTC
(In reply to Dirk Weber from comment #30)
> Created attachment 863992 [details]
> updated patch to use higher tolerance only if uname -m =~ i.86
> 
> Updated patch to use tolerance 0.4 only if CPU type matches i.86

Submitted this patch into the QA repo:
  https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/16210
Comment 33 Dominique Leuenberger 2023-01-12 12:38:25 UTC
(In reply to Dominique Leuenberger from comment #32)
> (In reply to Dirk Weber from comment #30)
> > Created attachment 863992 [details]
> > updated patch to use higher tolerance only if uname -m =~ i.86
> > 
> > Updated patch to use tolerance 0.4 only if CPU type matches i.86
> 
> Submitted this patch into the QA repo:
>   https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/16210

Was merged and the test passes on i586:

  https://openqa.opensuse.org/tests/3034552#step/graphicsMagick/24

I'd say we are safe to accept this as FIXED
Comment 34 Petr Gajdos 2023-01-12 13:19:58 UTC
Thanks!