Bug 1133809 - Enable bit-reproducible packages
Summary: Enable bit-reproducible packages
Status: IN_PROGRESS
Alias: None
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Basesystem (show other bugs)
Version: Current
Hardware: Other openSUSE Factory
: P5 - None : Normal (vote)
Target Milestone: ---
Assignee: Bernhard Wiedemann
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on: 1148824 1140896
Blocks: 1081754
  Show dependency treegraph
 
Reported: 2019-04-30 13:59 UTC by Bernhard Wiedemann
Modified: 2019-09-10 15:06 UTC (History)
4 users (show)

See Also:
Found By: Development
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bernhard Wiedemann 2019-04-30 13:59:29 UTC
To allow people to locally build bit-identical packages to those coming from OBS, 3 macros are needed in prjconf, but only the 1st is already enabled
in Factory (and Leap 15.0)

Macros:
%source_date_epoch_from_changelog Y
%clamp_mtime_to_source_date_epoch Y
%use_source_date_epoch_as_buildtime Y
# optional:
%_buildhost reproducible


I hope, we can discuss/test/enable these until 2020
so that it gets enough coverage before Leap 16

Normalizing rpm metadata was known to have some negative effects.
For python3, this should be good now with https://www.python.org/dev/peps/pep-0552/ in python3.7 that reached Tumbleweed in 2019-02.
For python2 we still carry my older patch that solves it by normalizing the .pyc file content.

There is no hurry at the moment, because the benefit of bit-identical over mostly-the-same is small and some risks and trade-offs are involved.

https://build.suse.de/project/show/SUSE:Factory:Head
already has 3 macros enabled, and allows for bit-identical local builds.

https://en.opensuse.org/openSUSE:Reproducible_Builds has more details.
Comment 1 Max Lin 2019-06-17 08:32:31 UTC
As pinged on IRC, the first stage would like to have %clamp_mtime_to_source_date_epoch to be enabled in the staging project, I enabled %clamp_mtime_to_source_date_epoch Y on Factory staging O[1], this staging got re-bootstrapped and everything rebuilt, there are 38 build failures(ignore obs-build ones) so far.

[1] https://build.opensuse.org/project/show/openSUSE:Factory:Staging:O
Comment 2 Bernhard Wiedemann 2019-06-26 10:47:03 UTC
Found an unrelated issue with Leap 15's rpm:
rpm --delsign $RPM
left several hundred zeroes in the output.
rpm master and the rpm in SLE-12-SP4 work fine though.

git bisect pointed to this fix:
commit 5c279fb149a44a1bc4d19e11c3c01942732b8486
Author: Panu Matilainen <pmatilai@redhat.com>
Date:   Fri Mar 15 11:43:08 2019 +0200

    Simplify RPMSIGTAG_RESERVEDSPACE shrinking


and the issue might have been originally introduced in
commit 90833a57c523c1a5a074cae81c0c84fb3726fb6d
Author: Lubos Kardos <lkardos@redhat.com>
Date:   Thu May 15 10:15:27 2014 +0200

    Reserve space for gpg signature during building of package.
Comment 3 Max Lin 2019-07-05 11:38:28 UTC
Just an update about Staging:O status,

all packages in staging:O build succeeded[1], and openQA looks good as well[2].

[1] https://build.opensuse.org/project/show/openSUSE:Factory:Staging:O
[2] https://openqa.opensuse.org/tests/overview?groupid=2&version=Staging%3AO&build=56.1&distri=opensuse
Comment 4 Bernhard Wiedemann 2019-08-22 11:51:56 UTC
Dominique added to openSUSE:Factory
%clamp_mtime_to_source_date_epoch Y

Let's wait for a month to see if it causes trouble
and then we can try %use_source_date_epoch_as_buildtime

That might confuse some scripts around OBS.