Bug 1168735 - [Tumbleweed] [20200402] RPM segfaults when installing glibc-locale (and some other pkgs)
[Tumbleweed] [20200402] RPM segfaults when installing glibc-locale (and some ...
Status: RESOLVED WORKSFORME
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: libzypp
Current
x86-64 Other
: P1 - Urgent : Critical (vote)
: ---
Assigned To: E-mail List
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2020-04-06 13:47 UTC by Dominik Heidler
Modified: 2020-06-25 10:03 UTC (History)
4 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
zypper logfile (152.98 KB, text/x-log)
2020-04-06 13:47 UTC, Dominik Heidler
Details
rpm debug log (352.57 KB, text/x-log)
2020-04-06 13:48 UTC, Dominik Heidler
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dominik Heidler 2020-04-06 13:47:57 UTC
Created attachment 834966 [details]
zypper logfile

zypper in -f glibc-locale
Loading repository data...
Reading installed packages...
Forcing installation of 'glibc-locale-2.31-3.2.x86_64' from repository 'Haupt-Repository (OSS)'.
Resolving package dependencies...

The following package is going to be upgraded:
  glibc-locale

1 package to upgrade.
Overall download size: 0 B. Already cached: 12.7 MiB. No additional space will be used or freed after the operation.
Continue? [y/n/v/...? shows all options] (y): 
In cache glibc-locale-2.31-3.2.x86_64.rpm                                                                                                                                    (1/1),  12.7 MiB (212.7 MiB unpacked)

Checking for file conflicts: ...............................................................................................................................................................................[done]
(1/1) Installing: glibc-locale-2.31-3.2.x86_64 ............................................................................................................................................................[error]
Installation of glibc-locale-2.31-3.2.x86_64 failed:
Error: Subprocess failed. Error: RPM failed: Command was killed by signal 11 (Segmentation fault).
Abort, retry, ignore? [a/r/i] (a):




When manually downloading the binaries from OBS and executing RPM like zypper would, I get the same result:

'rpm' '--root' '/' '--dbpath' '/var/lib/rpm' '-U' '--percent' '--noglob' '--force' '--nodeps' '--' ~dheidler/Downloads/glibc-locale-2.31-3.2.x86_64.rpm 
%% 0.000000
%% 0.000000
%% 50.000000
Segmentation fault (core dumped)
Comment 1 Dominik Heidler 2020-04-06 13:48:46 UTC
Created attachment 834968 [details]
rpm debug log

This logfile was created by adding the "-vv" argument to rpm.
Comment 2 Michael Schröder 2020-04-06 14:06:23 UTC
But where does it crash? Can you please run rpm with gdb?
Comment 3 Michael Schröder 2020-04-06 14:08:28 UTC
Could you also please archive the contents of the /usr/lib/sysimage/rpm dir in case that this is some problem with the database?
Comment 4 Dominik Heidler 2020-04-07 08:03:32 UTC
https://w3.suse.de/~dheidler/rpm.txz


Program received signal SIGSEGV, Segmentation fault.
fpHashFunction (fp=0x5555558493d0) at fprint.c:265
265         hash ^= ((unsigned)fp->entry->dev);
Missing separate debuginfos, use: zypper install krb5-debuginfo-1.18-1.1.x86_64 libacl1-debuginfo-2.2.53-3.7.x86_64 libbz2-1-debuginfo-1.0.8-2.2.x86_64 libcap2-debuginfo-2.32-1.1.x86_64 libcom_err2-debuginfo-1.45.5-1.4.x86_64 libgcrypt20-debuginfo-1.8.5-2.3.x86_64 libgpg-error0-debuginfo-1.37-1.1.x86_64 libkeyutils1-debuginfo-1.6-1.4.x86_64 liblua5_3-5-debuginfo-5.3.5-2.2.x86_64 liblzma5-debuginfo-5.2.5-1.1.x86_64 libnsl2-debuginfo-1.2.0-2.15.x86_64 libnss_nis2-debuginfo-3.1-1.2.x86_64 libopenssl1_1-debuginfo-1.1.1f-1.1.x86_64 libpopt0-debuginfo-1.16-32.1.x86_64 libselinux1-debuginfo-3.0-1.1.x86_64 libtirpc3-debuginfo-1.2.5-2.1.x86_64 libz1-debuginfo-1.2.11-12.2.x86_64 libzstd1-debuginfo-1.4.4-1.2.x86_64
(gdb) bt
#0  fpHashFunction (fp=0x5555558493d0) at fprint.c:265
#1  0x00007ffff7f6dfc5 in rpmFpHashGetEntry (tableKey=0x0, dataCount=0x7fffffffdb4c, data=0x7fffffffdb60, key=0x5555558493d0, ht=0x5555556c0330) at ../lib/rpmhash.C:259
#2  fpCacheGetByFp (cache=<optimized out>, fp=<optimized out>, ix=<optimized out>, recs=0x7fffffffdb60, numRecs=0x7fffffffdb4c) at fprint.c:465
#3  0x00007ffff7f4a866 in checkInstalledFiles (fpc=0x55555560d250, fileCount=<optimized out>, ts=<optimized out>) at transaction.c:1140
#4  rpmtsPrepare (ts=<optimized out>) at transaction.c:1470
#5  rpmtsRun (ts=<optimized out>, okProbs=<optimized out>, ignoreSet=<optimized out>) at transaction.c:1723
#6  0x00007ffff7f51aa9 in rpmcliTransaction (ts=ts@entry=0x5555555dc2e0, numPackages=1, ia=<optimized out>, ia=<optimized out>) at rpminstall.c:291
#7  0x00007ffff7f53026 in rpmInstall (ts=0x5555555dc2e0, ia=<optimized out>, fileArgv=<optimized out>) at rpminstall.c:632
#8  0x0000555555556948 in main (argc=13, argv=<optimized out>) at rpm.c:265
(gdb)
Comment 5 Fabian Vogt 2020-04-07 08:53:27 UTC
Can't reproduce locally, but crashes with the db from comment 4. Valgrind has this to say:

==481== Invalid read of size 8
==481==    at 0x4895A70: fpHashFunction (fprint.c:265)
==481==    by 0x4898FC4: UnknownInlinedFun (rpmhash.C:259)
==481==    by 0x4898FC4: fpCacheGetByFp (fprint.c:465)
==481==    by 0x4875865: UnknownInlinedFun (transaction.c:1140)
==481==    by 0x4875865: UnknownInlinedFun (transaction.c:1470)
==481==    by 0x4875865: rpmtsRun (transaction.c:1723)
==481==    by 0x487CAA8: rpmcliTransaction.isra.0 (rpminstall.c:291)
==481==    by 0x487E025: rpmInstall (rpminstall.c:632)
==481==    by 0x10A947: main (rpm.c:265)
==481==  Address 0x5e3da40 is 130,736 bytes inside a block of size 262,144 free'd
==481==    at 0x48399AB: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==481==    by 0x48DDC69: rpmDoDigest (rpmfileutil.c:46)
==481==    by 0x487A843: rpmfileContentsEqual (rpmfi.c:996)
==481==    by 0x487638B: UnknownInlinedFun (transaction.c:488)
==481==    by 0x487638B: UnknownInlinedFun (transaction.c:1154)
==481==    by 0x487638B: UnknownInlinedFun (transaction.c:1470)
==481==    by 0x487638B: rpmtsRun (transaction.c:1723)
==481==    by 0x487CAA8: rpmcliTransaction.isra.0 (rpminstall.c:291)
==481==    by 0x487E025: rpmInstall (rpminstall.c:632)
==481==    by 0x10A947: main (rpm.c:265)
==481==  Block was alloc'd at
==481==    at 0x483877F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==481==    by 0x48D6644: rmalloc (rpmmalloc.c:44)
==481==    by 0x48DDBCE: rpmDoDigest (rpmfileutil.c:28)
==481==    by 0x487A843: rpmfileContentsEqual (rpmfi.c:996)
==481==    by 0x487638B: UnknownInlinedFun (transaction.c:488)
==481==    by 0x487638B: UnknownInlinedFun (transaction.c:1154)
==481==    by 0x487638B: UnknownInlinedFun (transaction.c:1470)
==481==    by 0x487638B: rpmtsRun (transaction.c:1723)
==481==    by 0x487CAA8: rpmcliTransaction.isra.0 (rpminstall.c:291)
==481==    by 0x487E025: rpmInstall (rpminstall.c:632)
==481==    by 0x10A947: main (rpm.c:265)
Comment 6 Michael Schröder 2020-04-07 13:45:14 UTC
Ok, there is a bad entry in the "Basename" index database. The header part is correct, but the offset part is too high (31487 instead of something around 4850).

A rebuilddb fixes this, unfortunately, so it's completely unclear what caused that bad entry.
Comment 7 Dominik Heidler 2020-04-07 13:55:44 UTC
rpm --rebuilddb fixed it.
Comment 8 Michael Schröder 2020-04-07 15:39:40 UTC
It's really weird, the index database is in a perfect state nevertheless 3 entries have the wrong file offset:

85f12: LC_CTYPE 4114 4722  -> 53362
863f7: LC_CTYPE 4114 4737  -> 3201
88baf: LC_CTYPE 4114 4863  -> 31487
Comment 9 Michael Schröder 2020-04-08 09:07:48 UTC
Dominik, can you please copy your /var/log/zypp/history file into your Export dir?
Comment 10 Dominik Heidler 2020-04-08 09:50:03 UTC
https://w3.suse.de/~dheidler/history
Comment 11 Michael Schröder 2020-04-09 16:42:14 UTC
This is so weird. An entry in the index database consists of 8 bytes. A couple of entries have their second byte replaced by something random.

I'm currently out of ideas how something like this is possible at all.
Comment 13 Michael Andres 2020-06-25 09:44:30 UTC
@Michael: Is this fixed by the rpm submission in c#12 or do we need to keep it open?
Comment 14 Michael Schröder 2020-06-25 10:03:34 UTC
It's not fixed. It was most likely caused by some memory issue on the host. I can't explain the weird bytes switches that happened by anything else.

As nobody else has reported anything like this let's close the bug.