Bugzilla – Bug 1140896
lto causes minor variations in games/warsow binaries
Last modified: 2019-12-18 12:26:24 UTC
While working on reproducible builds for openSUSE, I found that building the games/warsow package with https://build.opensuse.org/request/show/714250 applied still gave minor variations from variations in KVM CPU count. These are ignored by build-compare but prevent us from reaching the goal of bit-identical reproducibility: /usr/bin/warsow: only difference was in build-id, gnu_debuglink or gnu_debugdata , GOOD. /usr/bin/warsow-server: only difference was in build-id, gnu_debuglink or gnu_de bugdata, GOOD. /usr/bin/warsow-tv-server: only difference was in build-id, gnu_debuglink or gnu _debugdata, GOOD. /usr/lib64/warsow/libs/libangelwrap_x86_64.so: only difference was in build-id, gnu_debuglink or gnu_debugdata, GOOD. [snipped more .so files] These variations went away with %define _lto_cflags %{nil} or %define _lto_cflags -flto=2 To reproduce (untested): osc co games/warsow && cd $_ for N in 1 4 ; do osc build --keep-pkg=.rpms$N --vm-type=kvm -j$N \ --define='%clamp_mtime_to_source_date_epoch Y' \ --define='%use_source_date_epoch_as_buildtime Y' done md5sum .rpms*/*.rpm
somehow -flto=4 appears twice in the build call: + [ 22%] Linking CXX shared library ../libs/libsteamlib_x86_64.so + cd /home/abuild/rpmbuild/BUILD/warsow_21_sdk/source/source/build/steamlib && /usr/bin/cmake -E cmake_link_script CMakeFiles/steamlib.dir/link.txt --verbose=1 + /usr/bin/c++ -fPIC -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -flto=4 -DNDEBUG -pipe -Wall -Wno-unused-function -fvisibility=hidden -Winvalid-pch -std=c++0x -O2 -g -DNDEBUG -flto=4 -Wl,--as-needed -Wl,--no-undefined -Wl,-z,now -Wl,--as-needed -shared -Wl,-soname,libsteamlib_x86_64.so -o ../libs/libsteamlib_x86_64.so CMakeFiles/steamlib.dir/steamlib.cpp.o CMakeFiles/steamlib.dir/steamlib_main.cpp.o CMakeFiles/steamlib.dir/steamlib_syscalls.cpp.o
games/Veraball is also affected: filterdiff 'rpm -qp --qf %{OPTFLAGS}' */Veraball-20160127-0.0.no*rpm|less --- filter/RPMS.2017/Veraball-20160127-0.0.noarch.rpm +++ filter/RPMS/Veraball-20160127-0.0.noarch.rpm @@ -1 +1 @@ --O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -flto=4 \ No newline at end of file +-O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -flto=1 Do we need to store the OPTFLAGS in rpm headers? Would always using -flto=1 really be worse? or could we invent a -flto=max which is constant but could do the right thing in gcc/binutils
-flto seems to affect many/all packages. atftp is nicely small+fast to build for experimenting. I workarounded it for now in https://build.opensuse.org/project/prjconf/home:bmwiedemann:reproducible
(In reply to Bernhard Wiedemann from comment #3) > -flto seems to affect many/all packages. > atftp is nicely small+fast to build for experimenting. > > I workarounded it for now in > https://build.opensuse.org/project/prjconf/home:bmwiedemann:reproducible The workaround is not desirable as we want to use -flto=N with maximum possible parallelism. The original difference comes from: $ objdump -s ./1/usr/lib/debug/.dwz/atftp-0.7.2-0.x86_64 > /tmp/1 $ objdump -s ./2/usr/lib/debug/.dwz/atftp-0.7.2-0.x86_64 > /tmp/2 $ diff -u /tmp/1 /tmp/2 --- /tmp/1 2019-07-10 12:40:26.746758800 +0200 +++ /tmp/2 2019-07-10 12:40:28.978729823 +0200 @@ -1,10 +1,10 @@ -./1/usr/lib/debug/.dwz/atftp-0.7.2-0.x86_64: file format elf64-x86-64 +./2/usr/lib/debug/.dwz/atftp-0.7.2-0.x86_64: file format elf64-x86-64 Contents of section .note.gnu.build-id: 0000 04000000 14000000 03000000 474e5500 ............GNU. - 0010 0689672c 6a05b2ea 038e57e3 713069d4 ..g,j.....W.q0i. - 0020 a8f0fa4d ...M + 0010 b3c21a53 324c14ca 20e85ee3 b5f389da ...S2L.. .^..... + 0020 23ceccae #... Contents of section .debug_info: 0000 3b000000 04000000 00000811 00000000 ;............... 0010 63000000 45040799 15000045 01080510 c...E......E.... @@ -810,7 +810,7 @@ 02e0 726f6e6f 75732d75 6e77696e 642d7461 ronous-unwind-ta 02f0 626c6573 202d6673 7461636b 2d636c61 bles -fstack-cla 0300 73682d70 726f7465 6374696f 6e202d66 sh-protection -f - 0310 6c746f3d 31202d66 676e7538 392d696e lto=1 -fgnu89-in + 0310 6c746f3d 32202d66 676e7538 392d696e lto=2 -fgnu89-in 0320 6c696e65 202d6650 4945005f 494f5f46 line -fPIE._IO_F 0330 494c4500 49505052 4f544f5f 47524500 ILE.IPPROTO_GRE. 0340 49505052 4f544f5f 5544504c 49544500 IPPROTO_UDPLITE. Where .note.gnu.build-id HASH is different due to differences in .debug_info section. Let me investigate that.
Which is cause by: $ eu-readelf --debug-dump=info a.out | less ... Compilation unit at offset 144: Version: 4, Abbreviation section offset: 91, Address size: 8, Offset size: 4 [ 9b] compile_unit abbrev: 1 producer (strp) "GNU C17 9.1.1 20190611 [gcc-9-branch revision 272147] -mtune=generic -march=x86-64 -g -flto=1234567890" language (data1) C99 (12) ... So it's encoded in debug info in producer ;) Let me investigate that..
I've got a patch candidate for it: https://gcc.gnu.org/ml/gcc-patches/2019-07/msg00786.html
So it seems, Factory's gcc9 now supports -flto=auto but it is not used yet.
(In reply to Bernhard Wiedemann from comment #7) > So it seems, Factory's gcc9 now supports -flto=auto > but it is not used yet. I see it being used: https://build.opensuse.org/build/games/openSUSE_Tumbleweed/x86_64/warsow/_log Can we close this issue now?
I was looking at Factory and -flto=auto still is not used there: https://build.opensuse.org/package/live_build_log/openSUSE:Factory/libuv/standard/x86_64 I just dont know why.
The rpm patch for flto=auto was merged back into Factory on 2019-10-21 https://build.opensuse.org/request/show/732635