Bug 1075803 - virt-manager crashes in PyDict_SetItem () - libguestfs
virt-manager crashes in PyDict_SetItem () - libguestfs
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Virtualization:Tools
Current
Other Other
: P5 - None : Major (vote)
: ---
Assigned To: Charles Arnold
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-01-12 12:57 UTC by Goldwyn Rodrigues
Modified: 2021-10-28 16:38 UTC (History)
8 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
gdb output (140.66 KB, text/plain)
2018-02-03 17:01 UTC, Stefan Seyfried
Details
virt-inspector output (822.29 KB, text/plain)
2018-02-06 20:06 UTC, Stefan Seyfried
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Goldwyn Rodrigues 2018-01-12 12:57:48 UTC
While trying to create a new VM using virt-manger, virt-manager crashes.

#0  0x00007f27d8452c6f in PyDict_SetItem () at /usr/lib64/libpython3.6m.so.1.0
#1  0x00007f27d845498c in PyDict_SetItemString ()
    at /usr/lib64/libpython3.6m.so.1.0
#2  0x00007f27c0c092f5 in guestfs_int_py_put_application ()
    at /usr/lib64/python3.6/site-packages/libguestfsmod.cpython-36m-x86_64-linux-gnu.so
#3  0x00007f27c0c0b1d1 in guestfs_int_py_put_application_list ()
    at /usr/lib64/python3.6/site-packages/libguestfsmod.cpython-36m-x86_64-linux-gnu.so
#4  0x00007f27c0be256f in guestfs_int_py_inspect_list_applications ()
    at /usr/lib64/python3.6/site-packages/libguestfsmod.cpython-36m-x86_64-linux-gnu.so
#5  0x00007f27d845f61b in _PyCFunction_FastCallDict ()
    at /usr/lib64/libpython3.6m.so.1.0
#6  0x00007f27d84b31fa in  () at /usr/lib64/libpython3.6m.so.1.0
#7  0x00007f27d84ac0ba in _PyEval_EvalFrameDefault ()
    at /usr/lib64/libpython3.6m.so.1.0
#8  0x00007f27d84b358a in  () at /usr/lib64/libpython3.6m.so.1.0
#9  0x00007f27d84b32b6 in  () at /usr/lib64/libpython3.6m.so.1.0
#10 0x00007f27d84ac0ba in _PyEval_EvalFrameDefault ()
    at /usr/lib64/libpython3.6m.so.1.0
#11 0x00007f27d84b358a in  () at /usr/lib64/libpython3.6m.so.1.0
#12 0x00007f27d84b32b6 in  () at /usr/lib64/libpython3.6m.so.1.0
#13 0x00007f27d84ac0ba in _PyEval_EvalFrameDefault ()
    at /usr/lib64/libpython3.6m.so.1.0
#14 0x00007f27d84b3bc5 in  () at /usr/lib64/libpython3.6m.so.1.0
#15 0x00007f27d84b32b6 in  () at /usr/lib64/libpython3.6m.so.1.0
#16 0x00007f27d84ac0ba in _PyEval_EvalFrameDefault ()
    at /usr/lib64/libpython3.6m.so.1.0
#17 0x00007f27d84b358a in  () at /usr/lib64/libpython3.6m.so.1.0
#18 0x00007f27d84b32b6 in  () at /usr/lib64/libpython3.6m.so.1.0
#19 0x00007f27d84ac0ba in _PyEval_EvalFrameDefault ()
    at /usr/lib64/libpython3.6m.so.1.0
#20 0x00007f27d84b4d72 in _PyFunction_FastCallDict ()
    at /usr/lib64/libpython3.6m.so.1.0
#21 0x00007f27d8428a0e in _PyObject_FastCallDict ()
    at /usr/lib64/libpython3.6m.so.1.0
#22 0x00007f27d84292cf in _PyObject_Call_Prepend ()
    at /usr/lib64/libpython3.6m.so.1.0
#23 0x00007f27d8428d6b in PyObject_Call () at /usr/lib64/libpython3.6m.so.1.0
#24 0x00007f27d84ad7b2 in _PyEval_EvalFrameDefault ()
    at /usr/lib64/libpython3.6m.so.1.0
#25 0x00007f27d84b358a in  () at /usr/lib64/libpython3.6m.so.1.0
#26 0x00007f27d84b32b6 in  () at /usr/lib64/libpython3.6m.so.1.0
#27 0x00007f27d84ac0ba in _PyEval_EvalFrameDefault ()
#28 0x00007f27d84b358a in  () at /usr/lib64/libpython3.6m.so.1.0
#29 0x00007f27d84b32b6 in  () at /usr/lib64/libpython3.6m.so.1.0
#30 0x00007f27d84ac0ba in _PyEval_EvalFrameDefault ()
    at /usr/lib64/libpython3.6m.so.1.0
#31 0x00007f27d84b4d72 in _PyFunction_FastCallDict ()
    at /usr/lib64/libpython3.6m.so.1.0
#32 0x00007f27d8428a0e in _PyObject_FastCallDict ()
    at /usr/lib64/libpython3.6m.so.1.0
#33 0x00007f27d84292cf in _PyObject_Call_Prepend ()
    at /usr/lib64/libpython3.6m.so.1.0
#34 0x00007f27d8428d6b in PyObject_Call () at /usr/lib64/libpython3.6m.so.1.0
#35 0x00007f27d8532bf2 in  () at /usr/lib64/libpython3.6m.so.1.0
#36 0x00007f27d8119558 in start_thread () at /lib64/libpthread.so.0
#37 0x00007f27d7e506df in clone () at /lib64/libc.so.6
Comment 2 Goldwyn Rodrigues 2018-01-18 22:02:58 UTC
Charles: I am no longer able to reproduce this with the latest repos on opensuse TW. Feel free to close it.
Comment 3 Charles Arnold 2018-01-18 22:06:06 UTC
(In reply to Goldwyn Rodrigues from comment #2)
> Charles: I am no longer able to reproduce this with the latest repos on
> opensuse TW. Feel free to close it.

Okay, thanks for the update.
Comment 4 Stefan Seyfried 2018-02-03 17:01:32 UTC
Created attachment 758723 [details]
gdb output

But I can reproduce it.

Attaching typescript of

gdb --args /usr/bin/python3 /usr/share/virt-manager/virt-manager --debug
...wait...
...segv...
bt
bt full
cont
quit
Comment 5 Stefan Seyfried 2018-02-03 17:02:26 UTC
I can reproduce it.
Seems to be a guestfs issue.
Comment 6 Stefan Seyfried 2018-02-03 17:23:55 UTC
structs.c:67 (libguestfs) is:
  65   PyDict_SetItemString (dict, "app_summary",
  66                         guestfs_int_py_fromstring (application->app_summary));
  67   PyDict_SetItemString (dict, "app_description",
  68                         guestfs_int_py_fromstring (application->app_description));
  69   return dict;

gdb:
(gdb) print application->app_description
$7 = 0x7fffc4053220 "Based on xteddy -- really cute ;-)\n\n\n\nAuthors:\n--------\n    Stefan Gustavson <stefang@isy.liu.se>\n    Jens P\366nisch <J.Poenisch@wirtschaft.tu-chemnitz.de>"
(gdb) print guestfs_int_py_fromstring(application->app_description)
$8 = (PyObject *) 0x0

guestfs_int_py_fromstring (handle.c):
361 PyObject *
362 guestfs_int_py_fromstring (const char *str)
363 {
364 #ifdef HAVE_PYSTRING_ASSTRING
365   return PyString_FromString (str);
366 #else
367   return PyUnicode_FromString (str);
368 #endif
369 }

(gdb) print PyUnicode_FromString(application->app_description)
$9 = (PyObject *) 0x0

looks really like a libguestfs fsckup, adding libguestfs maintainers to CC.
Comment 7 Stefan Seyfried 2018-02-03 17:46:42 UTC
uninstalling libguestfs0 (and depenencies guestfs-tools perl-Sys-Guestfs python3-libguestfs) works around the crash.
Comment 8 Cédric Bosdonnat 2018-02-06 15:53:54 UTC
(In reply to Stefan Seyfried from comment #7)
> uninstalling libguestfs0 (and depenencies guestfs-tools perl-Sys-Guestfs
> python3-libguestfs) works around the crash.

Could you please help us reproduce here? Could you readd libguestfs and try add isolate which of your VMs is causing the problem. It seems to be the one listed after qemu:///system:train-leap423-templ2.

then on that VM, could you run one of

virt-inspector -a /path/to/the/disk/image -v -x
virt-inspector -c qemu://system -d guest_name -v -x

and provide the output?
Comment 9 Stefan Seyfried 2018-02-06 20:06:33 UTC
Created attachment 759093 [details]
virt-inspector output

I ran these commands:
virt-inspector -a /vmspace/img/factory.qcow2 -v -x | tee /dev/shm/virt-inspector-image.txt
virt-inspector -c qemu:///system -d factory -v -x | tee /dev/shm/virt-inspector-libvirt.txt

Both outputs are identical, I attached one of them.
seife@strolchi:~> sha256sum /dev/shm/virt-inspector-*
1214e07d9f8d4a301caad7c569c6403bf8d2ba46cf2e846ac1c33943879569c1  /dev/shm/virt-inspector-image.txt
1214e07d9f8d4a301caad7c569c6403bf8d2ba46cf2e846ac1c33943879569c1  /dev/shm/virt-inspector-libvirt.txt
Comment 10 Stefan Seyfried 2018-02-07 18:22:42 UTC
The description of package xpinguin is:
----
        <description>Based on xteddy -- really cute ;-)



Authors:
--------
    Stefan Gustavson &lt;stefang@isy.liu.se&gt;
    Jens P�nisch &lt;J.Poenisch@wirtschaft.tu-chemnitz.de&gt;</description>
----
seife@strolchi:~> file /dev/shm/delme 
/dev/shm/delme: ISO-8859 text

The � is actually an 'ö' umlaut (but latin1 encoded).

https://docs.python.org/3/c-api/unicode.html defines

PyObject *PyUnicode_FromString(const char *u)
    Create a Unicode object from a UTF-8 encoded null-terminated char buffer u.

Well, "const char *u" in this case is *not* UTF-8 encoded, and thus returning NULL is stupid but probably "working as designed".

=> IMNSHO libguestfs' python/structs.c code is broken. Or, to say the least, "overly optimistic" ;-)
Comment 11 Stefan Seyfried 2018-02-10 20:07:01 UTC
looks much better with libguestfs 1.38.o from Virtualization repo:

[Sa, 10 Feb 2018 21:05:06 virt-manager 12025] DEBUG (inspection:251) qemu:///system:factory: detected operating system: linux opensuse 42.3 (openSUSE Leap 42.3)
[Sa, 10 Feb 2018 21:05:06 virt-manager 12025] DEBUG (inspection:252) hostname: factory
[Sa, 10 Feb 2018 21:05:06 virt-manager 12025] DEBUG (inspection:256) # apps: 1678

no crash
Comment 12 Cédric Bosdonnat 2018-02-12 09:12:59 UTC
(In reply to Stefan Seyfried from comment #11)
> looks much better with libguestfs 1.38.o from Virtualization repo:
> 
> [Sa, 10 Feb 2018 21:05:06 virt-manager 12025] DEBUG (inspection:251)
> qemu:///system:factory: detected operating system: linux opensuse 42.3
> (openSUSE Leap 42.3)
> [Sa, 10 Feb 2018 21:05:06 virt-manager 12025] DEBUG (inspection:252)
> hostname: factory
> [Sa, 10 Feb 2018 21:05:06 virt-manager 12025] DEBUG (inspection:256) # apps:
> 1678
> 
> no crash

Good news! I honestly have no idea how comes this magically fixes that bug, the encoding isn't better handled in this one.. Let's close for the while and feel free to reopen if that happens again.
Comment 13 Stefan Seyfried 2018-02-12 20:28:00 UTC
This is actually what happens now:

[Mo, 12 Feb 2018 21:19:11 virt-manager 16138] ERROR (inspection:170) qemu:///system:factory: exception while processing
Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/inspection.py", line 161, in _process_vm
    data = self._inspect_vm(conn, vm)
  File "/usr/share/virt-manager/virtManager/inspection.py", line 243, in _inspect_vm
    apps = g.inspect_list_applications(root)
  File "/usr/lib64/python3.6/site-packages/guestfs.py", line 5131, in inspect_list_applications
    r = libguestfsmod.inspect_list_applications(self._o, root)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 108: invalid start byte

So it does handle it better now.

Note that the reason for this failure is (i guess), that rpm is called with non-UTF-8 locale:

seife@strolchi:~> LC_ALL=en_US rpm -qi xpinguin|tail -2|hexdump -C
00000000  20 20 20 20 4a 65 6e 73  20 50 f6 6e 69 73 63 68  |    Jens P.nisch|
00000010  20 3c 4a 2e 50 6f 65 6e  69 73 63 68 40 77 69 72  | <J.Poenisch@wir|
00000020  74 73 63 68 61 66 74 2e  74 75 2d 63 68 65 6d 6e  |tschaft.tu-chemn|
00000030  69 74 7a 2e 64 65 3e 0a  44 69 73 74 72 69 62 75  |itz.de>.Distribu|
00000040  74 69 6f 6e 3a 20 6f 70  65 6e 53 55 53 45 20 54  |tion: openSUSE T|
00000050  75 6d 62 6c 65 77 65 65  64 0a                    |umbleweed.|
0000005a
seife@strolchi:~> LC_ALL=en_US.utf8 rpm -qi xpinguin|tail -2|hexdump -C
00000000  20 20 20 20 4a 65 6e 73  20 50 c3 b6 6e 69 73 63  |    Jens P..nisc|
00000010  68 20 3c 4a 2e 50 6f 65  6e 69 73 63 68 40 77 69  |h <J.Poenisch@wi|
00000020  72 74 73 63 68 61 66 74  2e 74 75 2d 63 68 65 6d  |rtschaft.tu-chem|
00000030  6e 69 74 7a 2e 64 65 3e  0a 44 69 73 74 72 69 62  |nitz.de>.Distrib|
00000040  75 74 69 6f 6e 3a 20 6f  70 65 6e 53 55 53 45 20  |ution: openSUSE |
00000050  54 75 6d 62 6c 65 77 65  65 64 0a                 |Tumbleweed.|
0000005b

So setting the locale to utf8 for listing the rpm details might be an easy fix.
Comment 14 Cédric Bosdonnat 2018-02-13 11:17:58 UTC
(In reply to Stefan Seyfried from comment #13)
> So setting the locale to utf8 for listing the rpm details might be an easy
> fix.

Thanks so much for your deep investigations. The description of xpinguin is indeed latin-1. However the fix won't be that easy since libguestfs doesn't call rpm to dump the data but reads the database directly.
Comment 15 Cédric Bosdonnat 2018-02-14 17:55:19 UTC
Reopening since I still see the issue even with 1.38.

Fix submitted for upstream review: https://www.redhat.com/archives/libguestfs/2018-February/msg00097.html

Will wait for inclusion to add to our packages
Comment 17 Cédric Bosdonnat 2019-07-12 08:42:30 UTC
Larry, could you have a look at this guestfs bug?
Comment 18 Charles Arnold 2021-10-28 16:38:56 UTC
I have tested on Tumbleweed 20211025 and do not see a segmentation fault.
Feel free to reopen if crash is reproduced on this or a newer version.