Bug 1077145 - [regression] osc build uses wrong changes file
[regression] osc build uses wrong changes file
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Basesystem
Current
All openSUSE Factory
: P5 - None : Normal (vote)
: ---
Assigned To: Adrian Schröter
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-01-23 06:57 UTC by Bernhard Wiedemann
Modified: 2022-06-13 12:45 UTC (History)
2 users (show)

See Also:
Found By: Development
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bernhard Wiedemann 2018-01-23 06:57:51 UTC
I found this when working on reproducible builds for openSUSE.
Regression occurring since at least 2018-01-19
Using the /usr/lib/build/changelog2spec script from
build-20171128-278.1 from openSUSE:Tools repo

When building packages like antlr openssh qemu python-pandas
that have multiple .changes files
sometimes the wrong one will end up in the resulting .src.rpm
and binary rpms

Steps To Reproduce:
osc co openSUSE:Factory/antlr ; cd $_
time osc build --keep-pkg=test.2/ --noservice antlr.spec
rpm -qp --changelog test.2/antlr-2.7.7-0.src.rpm|head -2

Actual Result:
* Thu Dec 07 2017 dimstar@opensuse.org
- Escape the usage of %{VERSION} when calling out to rpm.

Expected Result:
* Fri Jan 12 2018 tchvatal@suse.com
- Add condition about python2 module, the rewrite happened in antlr4

Reproducible: Sometimes

on another build host I got build-20171027 and could not reproduce the problem.
Comment 1 Bernhard Wiedemann 2018-01-23 09:38:46 UTC
I looked at the diff and this might have broken it:

commit 393c490671d24ea188acfbbec5f62946aba8d12f
Author: Stefan Seyfried <seife+dev@b1-sytems.com>
Date:   Wed Sep 6 16:31:44 2017 +0200

    changelog2spec: prefer _service:.*:.*.changes
    
    without this,. "foo.changes" will be preferred over
    "_service:obs_scm:foo.changes" which means --generatechanges in obs_scm
    has no effect.


but then I wonder why we even consider antlr-bootstrap.changes
when building antlr.spec
just because the 'antlr' string appears in the file name?
because the name might have been _service:...:antlr.changes?

I guess, we should use a more specific regexp in line 86
Comment 2 Bernhard Wiedemann 2018-01-23 10:02:08 UTC
and now I noticed that the randomness comes from perl's randomized hash
in line 81 as explained in
https://github.com/bmwiedemann/theunreproduciblepackage/tree/master/hash
so we need to have a sort after such hash (or not use hashes)
Comment 3 Bernhard Wiedemann 2018-01-23 10:34:40 UTC
There is already a partial fix in git:

commit 797b060ca5020bb0dbc49682c082262a4bd340ad
Author: Stefan Seyfried <seife+dev@b1-sytems.com>
Date:   Thu Nov 23 10:12:40 2017 +0100

    changelog2spec: fix "multiple .changes files" case


but nevertheless the moved sort is useless
because map introduces randomness after it.
Comment 4 Stefan Seyfried 2018-01-23 12:24:11 UTC
it "worked for me", and tbh, I do not know too much about perl's internals, and when I read line 81 of current changelog2spec, my brain goes into suspend mode, and I turn to more rewarding problems.

Anyway, something to this effect needs to be implemented in changelog2spec:

if (just one .changes file is present) {
  use it and be done.
} else if there_is("{*:,}$specname.changes") {
  if (more than one result) {
    prefer ("_service:*:$specname.changes")
  } else {
    use (result)
  }
} else {
  # no $specname.changes, but maybe $specname-master.changes
  # more than one result, because already checked first
  prefer (service:*...)
}

maybe if it is really coded as verbose as my pseudocode, it would actually be maintainable in the future by someone besides mls :-)

My two commits probably did not make anything worse than before, maybe they do not fix every case.

well, without them it was reproducibly broken (wrong changes file used).
Now it might be sometimes broken ;-)
Comment 5 Bernhard Wiedemann 2018-01-23 12:58:07 UTC
(In reply to Stefan Seyfried from comment #4)
> My two commits probably did not make anything worse than before, maybe they
> do not fix every case.

actually, the first commit effectively dropped the sort,
making things worse than before

and the 2nd commit just fixed the case of foo-something.changes
but not something-foo.changes (e.g. python-rpm / rpm)

and also the intended effect of the first commit was about
service generated .changes file and you only sometimes get that
depending on hash randomization.


In a certain sense 'always broken' is better than 'randomly broken'
because you can catch it in testsuites.
Comment 6 Stefan Seyfried 2018-01-23 13:34:58 UTC
ok, then  just revert my two commits.
I'm not going to touch this code again :-)
Comment 7 Stefan Seyfried 2018-01-23 13:36:11 UTC
server side source services are not really wanted or even working on public obs anyway, so the loss of functionality is not significant. I will carry local patches for my build service.
Comment 8 Bernhard Wiedemann 2018-01-23 16:04:18 UTC
I think, this fix should cover your use-case as well
=> see the included test coverage
https://github.com/openSUSE/obs-build/pull/424
Comment 9 Swamp Workflow Management 2018-01-24 16:40:07 UTC
This is an autogenerated message for OBS integration:
This bug (1077145) was mentioned in
https://build.opensuse.org/request/show/569349 Factory / build
Comment 10 Swamp Workflow Management 2018-02-01 01:40:18 UTC
This is an autogenerated message for OBS integration:
This bug (1077145) was mentioned in
https://build.opensuse.org/request/show/571362 15.0 / build
Comment 11 Bernhard Wiedemann 2018-02-09 16:57:33 UTC
should be fixed now
Comment 13 Swamp Workflow Management 2019-02-13 17:18:23 UTC
SUSE-RU-2019:0368-1: An update that has two recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1077145,1122895
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP4 (src):    build-20190128-9.6.1
SUSE Linux Enterprise Software Development Kit 12-SP3 (src):    build-20190128-9.6.1
Comment 14 Swamp Workflow Management 2019-02-14 17:28:16 UTC
SUSE-RU-2019:13959-1: An update that has two recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1077145,1122895
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 11-SP4 (src):    build-20190128-8.10.1
Comment 15 Swamp Workflow Management 2019-02-21 14:09:58 UTC
openSUSE-RU-2019:0227-1: An update that has two recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1077145,1122895
CVE References: 
Sources used:
openSUSE Leap 42.3 (src):    build-20190128-8.1