Bug 1105283 - dwz fails to compress debug info for various packages
dwz fails to compress debug info for various packages
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Basesystem
Current
Other Other
: P5 - None : Normal (vote)
: ---
Assigned To: Tom de Vries
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-08-19 20:17 UTC by Martin Liška
Modified: 2020-11-23 14:07 UTC (History)
1 user (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Liška 2018-08-19 20:17:37 UTC
I've taken a staging project and extracted which packages have issues with dwz:

WARNING:kernel-vanilla.log:[ 4429s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
WARNING:rubygem-nokogiri.log:[   76s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/nokogiri/nokogiri.so-1.8.4-3.6.x86_64.debug: DWARF version 0 unhandled
WARNING:go1.7.log:[  173s] dwz: .dwz/go1.7-1.7.5-10.3.x86_64: .debug_line reference above end of section
WARNING:texlive.log:[ 1639s] dwz: ./usr/bin/pmxab-2018.20180414-55.3.x86_64.debug: Invalid DW_AT_decl_file file number 2
WARNING:texlive.log:[ 1651s] dwz: ./usr/bin/scor2prt-2018.20180414-55.3.x86_64.debug: Invalid DW_AT_decl_file file number 2
WARNING:inkscape.log:[ 1322s] dwz: ./usr/lib64/libinkscape_base.so-0.92.3-5.6.x86_64.debug: Couldn't find DIE referenced by DW_OP_GNU_parameter_ref
WARNING:gcc8.log:[ 4227s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
WARNING:rubygem-byebug.log:[   79s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/byebug-10.0.2/byebug/byebug.so-10.0.2-2.5.x86_64.debug: DWARF version 0 unhandled
WARNING:java-1_8_0-openjdk.log:[ 1724s] dwz: ./usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/amd64/server/libjvm.so-1.8.0.171-2.17.x86_64.debug: Couldn't find DIE referenced by DW_OP_GNU_parameter_ref
WARNING:rubygem-unf_ext.log:[   48s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/unf_ext-0.0.7.5/unf_ext.so-0.0.7.5-2.9.x86_64.debug: DWARF version 0 unhandled
WARNING:valgrind.log:[  211s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
WARNING:pkg-config.log:[   77s] dwz: Too few files for multifile optimization
WARNING:rubygem-ruby-augeas.log:[   93s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/ruby-augeas-0.5.0/_augeas.so-0.5.0-4.24.x86_64.debug: DWARF version 0 unhandled
WARNING:libnumbertext.log:[   62s] dwz: .dwz/libnumbertext-1.0.2-3.6.x86_64: .debug_info section not present
WARNING:uuid.log:[   68s] dwz: .dwz/uuid-1.6.2-21.79.x86_64: .debug_info section not present
WARNING:MozillaFirefox.log:[ 3682s] dwz: ./usr/lib64/firefox/libxul.so-61.0.1-2.11.x86_64.debug: Too many DIEs, not optimizing
WARNING:ruby2.5.log:[  522s] dwz: ./usr/lib64/libruby2.5.so.2.5.0-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
WARNING:ruby2.5.log:[  522s] dwz: ./usr/lib64/ruby/2.5.0/x86_64-linux-gnu/racc/cparse.so-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
WARNING:ruby2.5.log:[  523s] dwz: ./usr/lib64/ruby/2.5.0/x86_64-linux-gnu/dbm.so-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
WARNING:ruby2.5.log:[  523s] dwz: Too few files for multifile optimization
WARNING:rrdtool.log:[  149s] dwz: ./usr/lib64/ruby/site_ruby/2.5.0/x86_64-linux-gnu/RRD.so-1.7.0-7.10.x86_64.debug: DWARF version 0 unhandled
WARNING:qt4-qtscript.log:[  444s] dwz: ./usr/lib64/qt4/plugins/script/libqtscript_gui.so.1.0.0-0.2.0-14.5.x86_64.debug: Couldn't find DIE referenced by DW_OP_GNU_parameter_ref
WARNING:gavl.log:[  104s] dwz: .dwz/gavl-1.4.0rsvn4256-3.5.x86_64: .debug_info section not present
WARNING:MozillaThunderbird.log:[ 4660s] dwz: ./usr/lib64/thunderbird/libxul.so-52.9.1-2.10.x86_64.debug: Too many DIEs, not optimizing
WARNING:libb64.log:[   60s] dwz: .dwz/libb64-1.2.1-3.5.x86_64: .debug_info section not present
WARNING:containerd.log:[   94s] dwz: dwz.c:1985: checksum_die: Assertion `((!op_multifile && !rd_multifile && !fi_multifile) || cu != die_cu (ref)) && (!op_multifile || cu->cu_chunk == die_cu (ref)->cu_chunk)' failed.
WARNING:kernel-debug.log:[ 4316s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
WARNING:cmake.log:[  684s] dwz: Too few files for multifile optimization
WARNING:libmypaint.log:[   70s] dwz: .dwz/libmypaint-1.3.0-2.5.x86_64: .debug_info section not present
WARNING:go1.8.log:[  266s] dwz: .dwz/go1.8-1.8.7-2.3.x86_64: .debug_line reference above end of section
WARNING:kernel-default.log:[ 2342s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
WARNING:virtualbox.log:[ 3394s] dwz: ./usr/lib/virtualbox/VirtualBox.so-5.2.16-2.4.x86_64.debug: Couldn't find DIE referenced by DW_OP_GNU_parameter_ref
WARNING:mono-core.log:[ 3234s] dwz: .dwz/mono-core-5.10.1-2.17.x86_64: .debug_line reference above end of section
WARNING:docker.log:[  308s] dwz: dwz.c:1985: checksum_die: Assertion `((!op_multifile && !rd_multifile && !fi_multifile) || cu != die_cu (ref)) && (!op_multifile || cu->cu_chunk == die_cu (ref)->cu_chunk)' failed.
WARNING:gcc7.log:[ 2716s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
Comment 1 Martin Liška 2019-04-23 07:12:48 UTC
Assigning to Tom ;)
Comment 2 Tom de Vries 2019-07-08 12:46:49 UTC
> WARNING:rubygem-nokogiri.log:[   76s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/nokogiri/nokogiri.so-1.8.4-3.6.x86_64.debug: DWARF version 0 unhandled
> WARNING:rubygem-byebug.log:[   79s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/byebug-10.0.2/byebug/byebug.so-10.0.2-2.5.x86_64.debug: DWARF version 0 unhandled
> WARNING:rubygem-unf_ext.log:[   48s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/unf_ext-0.0.7.5/unf_ext.so-0.0.7.5-2.9.x86_64.debug: DWARF version 0 unhandled
> WARNING:rubygem-ruby-augeas.log:[   93s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/ruby-augeas-0.5.0/_augeas.so-0.5.0-4.24.x86_64.debug: DWARF version 0 unhandled
> WARNING:ruby2.5.log:[  522s] dwz: ./usr/lib64/libruby2.5.so.2.5.0-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
> WARNING:ruby2.5.log:[  522s] dwz: ./usr/lib64/ruby/2.5.0/x86_64-linux-gnu/racc/cparse.so-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
> WARNING:ruby2.5.log:[  523s] dwz: ./usr/lib64/ruby/2.5.0/x86_64-linux-gnu/dbm.so-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
> WARNING:ruby2.5.log:[  523s] dwz: Too few files for multifile optimization
> WARNING:rrdtool.log:[  149s] dwz: ./usr/lib64/ruby/site_ruby/2.5.0/x86_64-linux-gnu/RRD.so-1.7.0-7.10.x86_64.debug: DWARF version 0 unhandled

Fixed upstream: https://sourceware.org/bugzilla/show_bug.cgi?id=24174 
Known issue: https://sourceware.org/bugzilla/show_bug.cgi?id=24725


> WARNING:kernel-vanilla.log:[ 4429s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
> WARNING:gcc8.log:[ 4227s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
> WARNING:valgrind.log:[  211s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
> WARNING:kernel-debug.log:[ 4316s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
> WARNING:kernel-default.log:[ 2342s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity
> WARNING:gcc7.log:[ 2716s] dwz: Multi-file optimization not allowed for differentpointer sizes or endianity

Known issue: https://sourceware.org/bugzilla/show_bug.cgi?id=24328


> WARNING:go1.7.log:[  173s] dwz: .dwz/go1.7-1.7.5-10.3.x86_64: .debug_line reference above end of section
> WARNING:go1.8.log:[  266s] dwz: .dwz/go1.8-1.8.7-2.3.x86_64: .debug_line reference above end of section
> WARNING:mono-core.log:[ 3234s] dwz: .dwz/mono-core-5.10.1-2.17.x86_64: .debug_line reference above end of section

Maybe a debug info problem.


> WARNING:texlive.log:[ 1639s] dwz: ./usr/bin/pmxab-2018.20180414-55.3.x86_64.debug: Invalid DW_AT_decl_file file number 2
> WARNING:texlive.log:[ 1651s] dwz: ./usr/bin/scor2prt-2018.20180414-55.3.x86_64.debug: Invalid DW_AT_decl_file file number 2

Maybe a debug info problem.


> WARNING:inkscape.log:[ 1322s] dwz: ./usr/lib64/libinkscape_base.so-0.92.3-5.6.x86_64.debug: Couldn't find DIE referenced by DW_OP_GNU_parameter_ref
> WARNING:java-1_8_0-openjdk.log:[ 1724s] dwz: ./usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/amd64/server/libjvm.so-1.8.0.171-2.17.x86_64.debug: Couldn't find DIE referenced by DW_OP_GNU_parameter_ref
> WARNING:qt4-qtscript.log:[  444s] dwz: ./usr/lib64/qt4/plugins/script/libqtscript_gui.so.1.0.0-0.2.0-14.5.x86_64.debug: Couldn't find DIE referenced by DW_OP_GNU_parameter_ref
> WARNING:virtualbox.log:[ 3394s] dwz: ./usr/lib/virtualbox/VirtualBox.so-5.2.16-2.4.x86_64.debug: Couldn't find DIE referenced by DW_OP_GNU_parameter_ref

Fixed upstream: https://sourceware.org/bugzilla/show_bug.cgi?id=24204


> WARNING:pkg-config.log:[   77s] dwz: Too few files for multifile optimization

The package has two executables, but identical, so find-debuginfo.sh makes one a hardlink to the other, after which there's only one executable left.
...
[   60s] + /usr/lib/rpm/find-debuginfo.sh -j3 --build-id-seed 0.29.2-65.9 --unique-debug-suffix -0.29.2-65.9.x86_64 --unique-debug-src-base pkg-config-0.29.2-65.9.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 50000000 -S debugsourcefiles.list /home/abuild/rpmbuild/BUILD/pkg-config-0.29.2
[   60s] extracting debug info from /home/abuild/rpmbuild/BUILDROOT/pkg-config-0.29.2-65.9.x86_64/usr/bin/pkg-config
[   60s] hard linked /home/abuild/rpmbuild/BUILDROOT/pkg-config-0.29.2-65.9.x86_64/usr/lib/debug/usr/bin/pkg-config-0.29.2-65.9.x86_64.debug to /home/abuild/rpmbuild/BUILDROOT/pkg-config-0.29.2-65.9.x86_64/usr/lib/debug/usr/bin/x86_64-suse-linux-gnu-pkg-config-0.29.2-65.9.x86_64.debug
[   60s] dwz: Too few files for multifile optimization
...
This warning should be fixed in find-debuginfo.sh. The multifile optimization is only triggered if dwz_files > 0, but it shouldn't count hardlinks. 


> WARNING:cmake.log:[  684s] dwz: Too few files for multifile optimization

The executables are larger than the low-mem limit, so they get selected out for multifile optimization, and not enough files are left for multifile optimization.


> WARNING:libnumbertext.log:[   62s] dwz: .dwz/libnumbertext-1.0.2-3.6.x86_64: .debug_info section not present
> WARNING:uuid.log:[   68s] dwz: .dwz/uuid-1.6.2-21.79.x86_64: .debug_info section not present
> WARNING:gavl.log:[  104s] dwz: .dwz/gavl-1.4.0rsvn4256-3.5.x86_64: .debug_info section not present
> WARNING:libb64.log:[   60s] dwz: .dwz/libb64-1.2.1-3.5.x86_64: .debug_info section not present
> WARNING:libmypaint.log:[   70s] dwz: .dwz/libmypaint-1.3.0-2.5.x86_64: .debug_info section not present

Fixed upstream: https://sourceware.org/bugzilla/show_bug.cgi?id=24341
 

> WARNING:MozillaFirefox.log:[ 3682s] dwz: ./usr/lib64/firefox/libxul.so-61.0.1-2.11.x86_64.debug: Too many DIEs, not optimizing
> WARNING:MozillaThunderbird.log:[ 4660s] dwz: ./usr/lib64/thunderbird/libxul.so-52.9.1-2.10.x86_64.debug: Too many DIEs, not optimizing

A DIE counting script counted 78417159 (78 million DIEs) in leap 15.0 libxul.so, which exceeds the 50 million default limit.


> WARNING:containerd.log:[   94s] dwz: dwz.c:1985: checksum_die: Assertion `((!op_multifile && !rd_multifile && !fi_multifile) || cu != die_cu (ref)) && (!op_multifile || cu->cu_chunk == die_cu (ref)->cu_chunk)' failed.
> WARNING:docker.log:[  308s] dwz: dwz.c:1985: checksum_die: Assertion `((!op_multifile && !rd_multifile && !fi_multifile) || cu != die_cu (ref)) && (!op_multifile || cu->cu_chunk == die_cu (ref)->cu_chunk)' failed.

Fixed upstream: https://sourceware.org/bugzilla/show_bug.cgi?id=24170
Comment 3 Tom de Vries 2019-07-12 14:18:14 UTC
(In reply to Tom de Vries from comment #2)
> > WARNING:rubygem-nokogiri.log:[   76s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/nokogiri/nokogiri.so-1.8.4-3.6.x86_64.debug: DWARF version 0 unhandled
> > WARNING:rubygem-byebug.log:[   79s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/byebug-10.0.2/byebug/byebug.so-10.0.2-2.5.x86_64.debug: DWARF version 0 unhandled
> > WARNING:rubygem-unf_ext.log:[   48s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/unf_ext-0.0.7.5/unf_ext.so-0.0.7.5-2.9.x86_64.debug: DWARF version 0 unhandled
> > WARNING:rubygem-ruby-augeas.log:[   93s] dwz: ./usr/lib64/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0/ruby-augeas-0.5.0/_augeas.so-0.5.0-4.24.x86_64.debug: DWARF version 0 unhandled
> > WARNING:ruby2.5.log:[  522s] dwz: ./usr/lib64/libruby2.5.so.2.5.0-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
> > WARNING:ruby2.5.log:[  522s] dwz: ./usr/lib64/ruby/2.5.0/x86_64-linux-gnu/racc/cparse.so-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
> > WARNING:ruby2.5.log:[  523s] dwz: ./usr/lib64/ruby/2.5.0/x86_64-linux-gnu/dbm.so-2.5.0-5.61.x86_64.debug: DWARF version 0 unhandled
> > WARNING:ruby2.5.log:[  523s] dwz: Too few files for multifile optimization
> > WARNING:rrdtool.log:[  149s] dwz: ./usr/lib64/ruby/site_ruby/2.5.0/x86_64-linux-gnu/RRD.so-1.7.0-7.10.x86_64.debug: DWARF version 0 unhandled
> 
> Fixed upstream: https://sourceware.org/bugzilla/show_bug.cgi?id=24174 
> Known issue: https://sourceware.org/bugzilla/show_bug.cgi?id=24725

Also, the lack of support for handling zlib-compressed debug info in debugedit breaks the ruby build ( https://bugzilla.opensuse.org/show_bug.cgi?id=1012533#c4 ), so compressed debug sections are disabled for ruby in tumbleweed and we shouldn't run into these atm.
Comment 4 Tom de Vries 2020-11-23 14:07:06 UTC
Latest run as in comment 0 doesn't show any dwz crashes:
...
obs-dwz-analysis.py /tmp/C/standard/x86_64/
WARNING:weave.log:[  194s] dwz: Too few files for multifile optimization
WARNING:virtualbox:kmp.log:[  254s] dwz: Too few files for multifile optimization
WARNING:valgrind.log:[  317s] dwz: Multi-file optimization not allowed for different pointer sizes or endianity
WARNING:rook.log:[  148s] dwz: Too few files for multifile optimization
WARNING:mozjs78.log:[ 1225s] dwz: Too few files for multifile optimization
WARNING:mozjs68.log:[ 1090s] dwz: Too few files for multifile optimization
WARNING:nodejs14.log:[  882s] dwz: ./usr/bin/node14-14.15.0-3.1.x86_64.debug: Too many DIEs, not optimizing
WARNING:metallb.log:[  198s] dwz: Too few files for multifile optimization
WARNING:kubic-control.log:[   47s] dwz: Too few files for multifile optimization
WARNING:kernel-vanilla.log:[ 2886s] dwz: Multi-file optimization not allowed for different pointer sizes or endianity
WARNING:kernel-debug.log:[ 3165s] dwz: Multi-file optimization not allowed for different pointer sizes or endianity
WARNING:kernel-default.log:[  825s] dwz: Multi-file optimization not allowed for different pointer sizes or endianity
WARNING:go1.4.log:[   55s] dwz: .dwz/go1.4-1.4.3-12.16.x86_64: .debug_line reference above end of section
WARNING:gcc10.log:[ 9813s] dwz: Multi-file optimization not allowed for different pointer sizes or endianity
WARNING:gcc7.log:[ 2683s] dwz: Multi-file optimization not allowed for different pointer sizes or endianity
WARNING:etcd-for-k8s1.18.log:[   96s] dwz: Too few files for multifile optimization
WARNING:etcd-for-k8s1.19.log:[   93s] dwz: Too few files for multifile optimization
WARNING:envoy-proxy.log:[ 1404s] dwz: ./usr/bin/envoy-proxy-1.14.4-2.21.x86_64.debug: Too many DIEs, not optimizing
WARNING:docker.log:[  232s] dwz: Too few files for multifile optimization
WARNING:docker:kubic.log:[  185s] dwz: Too few files for multifile optimization
WARNING:MozillaFirefox.log:[ 7891s] dwz: ./usr/lib64/firefox/libxul.so-82.0.3-2.5.x86_64.debug: Too many DIEs, not optimizing
WARNING:cmake:mini.log:[ 2128s] dwz: Too few files for multifile optimization
WARNING:cmake:full.log:[ 1326s] dwz: Too few files for multifile optimization
WARNING:cilium-proxy.log:[ 1425s] dwz: ./usr/bin/cilium-envoy-20200813-2.21.x86_64.debug: Too many DIEs, not optimizing
...

With all the relevant problems dealt with in separate PRs, closing this PR.