Same error shows up in this case:
https://openqa.opensuse.org/tests/2375666#step/usr_sbin_dnsmasq/1

This test should be simpler, no swtpm involved

I'm seeing the same errors. Had to roll back to 20220523.

(In reply to Mathias Homann from comment #2)
> I'm seeing the same errors. Had to roll back to 20220523.

Can you verify if it is 'just' the qemu packages that need a rollback? The change there in that snapshot looks quite harmless, but if we can confirm that, i will put a fever into snapshot 0527 and possibly into the update channel

(In reply to Dominique Leuenberger from comment #3)
> (In reply to Mathias Homann from comment #2)
> > I'm seeing the same errors. Had to roll back to 20220523.
> 
> Can you verify if it is 'just' the qemu packages that need a rollback? The
> change there in that snapshot looks quite harmless, but if we can confirm
> that, i will put a fever into snapshot 0527 and possibly into the update
> channel

I went back up to 20220524, rebooted, verified that vms dont start.
Then I switched back to 20220523 with tumbleweed-cli and rolled back only all installed qemu packages, restarted libvird, and vms start again.

Thanks.

I just added qemu rev 227 (the one from snapshot 0523 into the Update channel.

It should appear there shortly (build in Progress)
This should then allow you to be fully up-to-date again without locking to older snapshots.

If confirmed, snapshot 0526 should get the guy intree instead of the update channel again

(In reply to Dominique Leuenberger from comment #5)
> Thanks.
> 
> I just added qemu rev 227 (the one from snapshot 0523 into the Update
> channel.
> 
> It should appear there shortly (build in Progress)
> This should then allow you to be fully up-to-date again without locking to
> older snapshots.
> 
> If confirmed, snapshot 0526 should get the guy intree instead of the update
> channel again

I don't think that worked - switched back up to 0524, and did "zypper dup" - and I got the bad build of qemu again.

You did get the qemu packages from the update repo?

https://download.opensuse.org/repositories/openSUSE:/Factory:/Update/standard/

(In reply to Dominique Leuenberger from comment #7)
> You did get the qemu packages from the update repo?
> 
> https://download.opensuse.org/repositories/openSUSE:/Factory:/Update/
> standard/

I did, they're the same packages as from the snapshot 20220524 itself.

It would be useful to rollback the qemu version

Version that breaks: qemu-x86-6.2.0-48.1.x86_64.rpm

$ sudo virsh start VM1
error: Failed to start domain 'VM1'
error: internal error: qemu unexpectedly closed the monitor: qxl_send_events: spice-server bug: guest stopped, ignoring
*** buffer overflow detected ***: terminated

Downgrading just the qemu-x86 package to the previous version works as temporary fix:

http://download.opensuse.org/tumbleweed/repo/oss/x86_64/qemu-x86-6.2.0-47.2.x86_64.rpm

$ sudo zypper in --oldpackage ./qemu-x86-6.2.0-47.2.x86_64.rpm
$ sudo virsh start VM1
Domain 'VM1' started

If it helps, I also have an error that prevents me from launching my VM after the latest update.

Fehler beim Starten der Domain: Interner Fehler: qemu unexpectedly closed the monitor: *** buffer overflow detected ***: terminated

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1407, in startup
    self._backend.create()
  File "/usr/lib64/python3.8/site-packages/libvirt.py", line 1352, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: Interner Fehler: qemu unexpectedly closed the monitor: *** buffer overflow detected ***: terminated


A lot of things went wrong this snapshot and the last one. Spectacle doesnt launch, fonts in KDE are blurry because of wrong hinting with Noto Sans and now this. Whats going on..? :)

Can you guys try qemu-6.2.0-48.2 from the update channel please?

(In reply to Dominique Leuenberger from comment #12)
> Can you guys try qemu-6.2.0-48.2 from the update channel please?

Together with the other qemu packages from the update repo, all same version of course

(In reply to Dominique Leuenberger from comment #13)
> (In reply to Dominique Leuenberger from comment #12)
> > Can you guys try qemu-6.2.0-48.2 from the update channel please?
> 
> Together with the other qemu packages from the update repo, all same version
> of course

still the same error.

(In reply to Mathias Homann from comment #14)
> (In reply to Dominique Leuenberger from comment #13)
> > (In reply to Dominique Leuenberger from comment #12)
> > > Can you guys try qemu-6.2.0-48.2 from the update channel please?
> > 
> > Together with the other qemu packages from the update repo, all same version
> > of course
> 
> still the same error.

That's bad - it means a revert of qemu does not fix the issue and it's rather something that came in through the backdoor and with a rebuild ...

So anything that changed between may 17 (last build of qemu that worked) and may 24, when a new qemu patch was added. But the old sources would have caused the same issue in rebuild only

Based on an apparent unrelated bug, my best guess is gnutls.

A testbuild to confirm this theory should appear soon in
https://download.opensuse.org/repositories/home:/dimstar:/boo1199924/standard

Build can be monitored at https://build.opensuse.org/project/show/home:dimstar:boo1199924

(In reply to Dominique Leuenberger from comment #15)
> (In reply to Mathias Homann from comment #14)
> > (In reply to Dominique Leuenberger from comment #13)
> > > (In reply to Dominique Leuenberger from comment #12)
> > > > Can you guys try qemu-6.2.0-48.2 from the update channel please?
> > > 
> > > Together with the other qemu packages from the update repo, all same version
> > > of course
> > 
> > still the same error.
> 
> That's bad - it means a revert of qemu does not fix the issue and it's
> rather something that came in through the backdoor and with a rebuild ...
> 
Yeah, I have the same feeling.

> So anything that changed between may 17 (last build of qemu that worked) and
> may 24, when a new qemu patch was added. But the old sources would have
> caused the same issue in rebuild only
>
I think GCC 12 happened within that window, didn't it?

> Based on an apparent unrelated bug, my best guess is gnutls.
> 
> A testbuild to confirm this theory should appear soon in
> https://download.opensuse.org/repositories/home:/dimstar:/boo1199924/standard
>
Right, good guess (FWIW), it could very well be it...

(In reply to Dominique Leuenberger from comment #15)
> That's bad - it means a revert of qemu does not fix the issue and it's
> rather something that came in through the backdoor and with a rebuild ...
> 
In fact...

(In reply to Dominique Leuenberger from comment #0)
> 2022-05-25T16:30:05.742354Z qemu-system-x86_64: warning: host doesn't
> support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
> 2022-05-25T16:30:05.742369Z qemu-system-x86_64: warning: host doesn't
> support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
> 2022-05-25T16:30:05.743989Z qemu-system-x86_64: warning: host doesn't
> support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
> 2022-05-25T16:30:05.744050Z qemu-system-x86_64: warning: host doesn't
> support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
> *** buffer overflow detected ***: terminated
> 
I've seen reports of various packages starting to fail to start with this exact same error (most recent one I've heard about is GNOME Calculator)

(In reply to Dario Faggioli from comment #17)
> (In reply to Dominique Leuenberger from comment #15)
> > So anything that changed between may 17 (last build of qemu that worked) and
> > may 24, when a new qemu patch was added. But the old sources would have
> > caused the same issue in rebuild only
> >
> I think GCC 12 happened within that window, didn't it?

GCC 12 happened may 10th, qemu worked for two weeks post that

(In reply to Dario Faggioli from comment #18)
> (In reply to Dominique Leuenberger from comment #15)
> > That's bad - it means a revert of qemu does not fix the issue and it's
> > rather something that came in through the backdoor and with a rebuild ...
> > 
> In fact...
> 
> I've seen reports of various packages starting to fail to start with this
> exact same error (most recent one I've heard about is GNOME Calculator)

Yep, that's http://bugzilla.opensuse.org/show_bug.cgi?id=1199929 which also points at gnutls.

(In reply to Dominique Leuenberger from comment #19)
> (In reply to Dario Faggioli from comment #17)
> > (In reply to Dominique Leuenberger from comment #15)
> > > So anything that changed between may 17 (last build of qemu that worked) and
> > > may 24, when a new qemu patch was added. But the old sources would have
> > > caused the same issue in rebuild only
> > >
> > I think GCC 12 happened within that window, didn't it?
> 
> GCC 12 happened may 10th, qemu worked for two weeks post that
>
Ok, I had the dates mixed up, my bad.

In fact, it does seem that it really could be gnutls, as it's causing similar issues in quite a few other places, e.g.: bug 199929

(In reply to Dario Faggioli from comment #21)
> In fact, it does seem that it really could be gnutls, as it's causing
> similar issues in quite a few other places, e.g.: bug 199929
>
Err.. https://bugzilla.opensuse.org/show_bug.cgi?id=1199929 (not sure why it was converted to that url... :-O)

(In reply to Dario Faggioli from comment #22)
> Err.. https://bugzilla.opensuse.org/show_bug.cgi?id=1199929 (not sure why it
> was converted to that url... :-O)
>
Ah, ok, see it now. Missing a "1". Well, sorry for the noise (and quite a bit of it :-( )

*** Bug 1199967 has been marked as a duplicate of this bug. ***

I just tested https://download.opensuse.org/repositories/home:/dimstar:/boo1199924/standard/ and on my laptop the issue is still present. All packages from this repo are installed (qemu-* and gnutls).

> phoenix@racetrack-7290:~> virsh start tumbleweed
> error: Failed to start domain 'tumbleweed'
> error: internal error: qemu unexpectedly closed the monitor: qxl_send_events: spice-server bug: guest stopped, ignoring
> *** buffer overflow detected ***: terminated

AFAICS this build does not resolve it.

Sure it's (only) gnutls? Reverting back to snapshot 20220523 using tumbleweed-cli didn't work for me, vms still didn't start.
So I rolled back to a snapshot recorded 05/23/2022 using snapper and got snapshot 20220521. I locked all gnutls packages (gnutls, libgnutls30 and libgnutls-dane0, all at version 3.7.4-2.1), then called "zypper dup" and updated to snapshot 20220525. But again vms didn't start, although gnutls was still at 3.7.4.

So I used snapper again to roll back to snapshot 20220521. vms start and also Spectacle works. Guess I will stay at this snapshot until most problems are sorted out.

(In reply to Oliver Schwabedissen from comment #26)
> So I used snapper again to roll back to snapshot 20220521. vms start and
> also Spectacle works. Guess I will stay at this snapshot until most problems
> are sorted out.
>
Mmm... yes, Spectacle was also broken, apparently, but that was, AFAIUI, a different issue (see bug 1199931).

Can you, by any chance try GNOME Calculator? (or another app that was suffering from the "*** buffer overflow detected ***" issue... GNOME Calculator is just one that I'm sure it does, and that might be fairly quick to install and test, I guess)

(In reply to Oliver Schwabedissen from comment #26)
> Sure it's (only) gnutls? Reverting back to snapshot 20220523 using
> tumbleweed-cli didn't work for me, vms still didn't start.


ok now that is ...weird. For me, rolling back to 20220523 is what makes VMs work again - and replacing the qemu-x86 package with the one from 20220523 is actually the only thing that matters...

Right now I am fully on 20220525 except for that one package which i got from 20220523 and locked.
VMs work fine.

(In reply to Mathias Homann from comment #28)
> (In reply to Oliver Schwabedissen from comment #26)
> > Sure it's (only) gnutls? Reverting back to snapshot 20220523 using
> > tumbleweed-cli didn't work for me, vms still didn't start.
> 
> 
> ok now that is ...weird. For me, rolling back to 20220523 is what makes VMs
> work again - and replacing the qemu-x86 package with the one from 20220523
> is actually the only thing that matters...
> 
> Right now I am fully on 20220525 except for that one package which i got
> from 20220523 and locked.
> VMs work fine.
>
Mmm... can I see a:

> rpm -q --changelog qemu|head -20

?

So, the problem we see is qemu, same source, that built on may 17 worked, built on may 24 fails.

FORTIFY_SOURCE=3 has happened in this timeframe, which might be a source here

(In reply to Dario Faggioli from comment #29)
> (In reply to Mathias Homann from comment #28)
> > (In reply to Oliver Schwabedissen from comment #26)
> > > Sure it's (only) gnutls? Reverting back to snapshot 20220523 using
> > > tumbleweed-cli didn't work for me, vms still didn't start.
> > 
> > 
> > ok now that is ...weird. For me, rolling back to 20220523 is what makes VMs
> > work again - and replacing the qemu-x86 package with the one from 20220523
> > is actually the only thing that matters...
> > 
> > Right now I am fully on 20220525 except for that one package which i got
> > from 20220523 and locked.
> > VMs work fine.
> >
> Mmm... can I see a:
> 
> > rpm -q --changelog qemu|head -20
> 
> ?
Ok, I checked myself.

20220525, which you say does not work, has qemu-6.2.0-47.2.x86_64.rpm. Last change was:

> * Wed May 11 2022 Martin Liška <mliska@suse.cz>
> - Filter out rpmlint error that is valid for qemu, but will
>   have its badness increased in the future.

Now, 20220523, which apparently is working, has, as far as I can tell, qemu-6.2.0-47.2.x86_64.rpm, i.e., _the_same_ package. Last change was:

> * Wed May 11 2022 Martin Liška <mliska@suse.cz>
> - Filter out rpmlint error that is valid for qemu, but will
>   have its badness increased in the future.

So, the same change.

Can you confirm the versions?

(In reply to Dominique Leuenberger from comment #30)
> So, the problem we see is qemu, same source, that built on may 17 worked,
> built on may 24 fails.
> 
> FORTIFY_SOURCE=3 has happened in this timeframe, which might be a source here
>
If someone tells me how to turn this off (or back to something else), I can try a build

I set my branch to fortify source= 2, qemu is rebuilding - rel 50.3, once ready, should be worth a test again

> Can you confirm the versions?

Works: 6.2.0-47.2
Fails: 6.2.0-48.1
Fails: 6.2.0-48.2

Tumbleweed fully updated (zypper dup) to openSUSE-release-20220525-1634.1.x86_64.

Only the qemu-x86 needs downgraded to get VMs working.

(In reply to Dominique Leuenberger from comment #33)
> I set my branch to fortify source= 2, qemu is rebuilding - rel 50.3, once
> ready, should be worth a test again
>
Ok, I'm doing the same (but with the new QEMU version) here:

https://build.opensuse.org/project/show/home:dfaggioli:boo1199924

(I've copied the project config from here: https://build.opensuse.org/projects/home:dimstar:boo1199924/prjconf )

Ok, packages from here work for me:

https://download.opensuse.org/repositories/home:/dfaggioli:/boo1199924/openSUSE_Tumbleweed/home:dfaggioli:boo1199924.repo

BEWARE that, in there, you'd find QEMU 7.0, not QEMU 6.2. And the reason is I've already submitted that (i.e., QEMU 7.0) to Factory, and, considering that Dominique was already building 6.2 himself anyway, I wanted to check the new one.

So:
- QEMU 7.0, built with FORTIFY_SOURCE=2
- GNUtls 3.7.4

Seems fine to me.

Can confirm, Dario's repo with qemu 7 works also fine here on my side.

@Dominique, in your project, I think I still see -D_FORTIFY_SOURCE=3, in the latest available build:

https://build.opensuse.org/package/live_build_log/home:dimstar:boo1199924/qemu:testsuite/standard/x86_64
https://build.opensuse.org/build/home:dimstar:boo1199924/standard/x86_64/qemu:testsuite/_log

> [  171s] + /home/abuild/rpmbuild/BUILD/qemu-6.2.0/configure --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --docdir=/usr/share/doc/packages --firmwarepath=/usr/share/qemu-testsuite --python=/usr/bin/python3 '--extra-cflags=-O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables [...]

For mine, I re-triggered the build manually, after changing the prjconf. Do you maybe need to do the same?

In any case, I think we're coming to the conclusion that FORTIFY=3 has problems, and that FORTIFY=2 could be a solution.

So, what's the way forward here? Will Factory be reverted to 2 entirely? Will this happen only for the QEMU package?

Specifically, do I need to do anything, in the package and/or in the devel project?

osc rbl home:dimstar:boo1199924 qemu standard x86_64 | grep FORTIFY looks good

That build resulted in 6.2.0-50.3 which should be working. Any Testers?

As we seem to be on track there we will need this as part of the qemu.spec instead of the prjconf:

The fortify_source=2 should be injected into the extra cflags passed to configure

> That build resulted in 6.2.0-50.3 which should be working. Any Testers?

Downloaded https://download.opensuse.org/repositories/home:/dimstar:/boo1199924/standard/x86_64/qemu-x86-6.2.0-50.3.x86_64.rpm

Works for me.

$ sudo virsh start VM1
Domain 'VM1' started

Works: 6.2.0-47.2
Fails: 6.2.0-48.1
Fails: 6.2.0-48.2
Works: 6.2.0-50.3 (from home:dimstar)

From the qemu download page:
Version numbering

Since version 3.0.0, QEMU uses a time based version numbering scheme:

major
    incremented by 1 for the first release of the year
minor
    reset to 0 with every major increment, otherwise incremented by 1 for each release from git master
micro
    always 0 for releases from git master, incremented by 1 for each stable branch release 

So I think should be enough to go to version 7 as we are in the middle of 2022 (version 7.x.y branch)

(In reply to Paul Fee from comment #41
> Works: 6.2.0-47.2
> Fails: 6.2.0-48.1
> Fails: 6.2.0-48.2
> Works: 6.2.0-50.3 (from home:dimstar)

Ok, thanks! With this we an say conclusively that fortify 3 is the culprit for qemu.

I injected fortify 2 into openSUSE:Factory:Update - qemu packages in there should fall out in 1-2hours and work too, without home repos 

The gnutls error from gnome-calculator only put us on the wrong path as it seems:

Dario: can you please prepare a 6.2 submission with only the cflags adjusted, do i can take that into snapshot 0527 without staging blocks?

(In reply to Dominique Leuenberger from comment #43)
> Dario: can you please prepare a 6.2 submission with only the cflags
> adjusted, do i can take that into snapshot 0527 without staging blocks?>
>
Working on it!

(In reply to Dario Faggioli from comment #44)
> (In reply to Dominique Leuenberger from comment #43)
> > Dario: can you please prepare a 6.2 submission with only the cflags
> > adjusted, do i can take that into snapshot 0527 without staging blocks?>
> >
> Working on it!
>
@Dominiqu, does this make sense?

https://build.opensuse.org/request/show/979490

(In reply to Dominique Leuenberger from comment #43) 
> The gnutls error from gnome-calculator only put us on the wrong path as it
> seems:
> 
Right. Symptoms were the same there too (in GNOME Calculator, I mean), though... Are we sure it's gnutls and not FORTIFY_SOURCES in that as well?

Well, anyway, that's off-topic here...

> Dario: can you please prepare a 6.2 submission with only the cflags
> adjusted, do i can take that into snapshot 0527 without staging blocks?
>
Done. Actually, I reinstated the ARM build fix as well, the one that you reverted,  when thinking it was the problem.

This was in released snapshot and working well already, so I think it's not a problem.

Just wanted to confirm that 6.2.0-48.3 from today's update solved the issue. No more buffer overflow.
Thank you all! 

virsh start win10
Domain 'win10' started

(In reply to Dario Faggioli from comment #45)
> (In reply to Dario Faggioli from comment #44)
> > (In reply to Dominique Leuenberger from comment #43)
> > > Dario: can you please prepare a 6.2 submission with only the cflags
> > > adjusted, do i can take that into snapshot 0527 without staging blocks?>
> > >
> > Working on it!
> >
> @Dominiqu, does this make sense?
> 
> https://build.opensuse.org/request/show/979490

To me that looks fine. And yes, the arm fix can go back in. It just seemed most obvious that the PKG change might be responsible. We have proven this to be wrong by now

(In reply to Nik Kai from comment #47)
> Just wanted to confirm that 6.2.0-48.3 from today's update solved the issue.
> No more buffer overflow.
> Thank you all! 
> 
> virsh start win10
> Domain 'win10' started

same here, all is well with 6.2.0-48.3

(In reply to Dominique Leuenberger from comment #48)
> (In reply to Dario Faggioli from comment #45)
> > 
> > https://build.opensuse.org/request/show/979490
> 
> It just seemed
> most obvious that the PKG change might be responsible. We have proven this
> to be wrong by now
>
Sure! I wasn't blaming or complaining (sorry in case it sounded like that!).

However strange it looked, the decision of reverting it was the one that made the most sense (given the data coming from this bug). :-)

Happy that it they can go in at once, so we don't have to transition toward the "aarch64 is broken" again :-D

For everyone, once this is released, please, let us know here if things still work, so we can close the bug.

(In reply to Dario Faggioli from comment #50)
> 
> For everyone, once this is released, please, let us know here if things
> still work, so we can close the bug.

$ sudo zypper dup
$ rpm -q qemu-x86
qemu-x86-6.2.0-48.3.x86_64
$ sudo virsh start VM1
Domain 'VM1' started

Working as expected using latest Tumbleweed official package.

(In reply to Paul Fee from comment #51)
> (In reply to Dario Faggioli from comment #50)
> > 
> > For everyone, once this is released, please, let us know here if things
> > still work, so we can close the bug.
> 
> $ sudo zypper dup
> $ rpm -q qemu-x86
> qemu-x86-6.2.0-48.3.x86_64
> $ sudo virsh start VM1
> Domain 'VM1' started
> 
> Working as expected using latest Tumbleweed official package.
>
Wow... Already?

Ok, perfect, closing it them.

Thanks everyone for all the information and all the testing!

Final testing now that extra updates have arrived...

Full update to openSUSE-release-20220527-1638.1 via "sudo zypper dup".

Now running qemu-x86-6.2.0-50.1.

$ rpm -q qemu-x86 --changelog | head
* Fri May 27 2022 Dario Faggioli <dfaggioli@suse.com>
- It has been observed that building QEMU with _FORTIFY_SOURCE=3
  causes problem (see bsc#1199924). Force it to =2 for now, while
  we investigate the issue.

VMs start up as expected without issue.

(In reply to Dario Faggioli from comment #52)
> (In reply to Paul Fee from comment #51)
> > (In reply to Dario Faggioli from comment #50)
> > > 
> > > For everyone, once this is released, please, let us know here if things
> > > still work, so we can close the bug.
> > 
> > $ sudo zypper dup
> > $ rpm -q qemu-x86
> > qemu-x86-6.2.0-48.3.x86_64
> > $ sudo virsh start VM1
> > Domain 'VM1' started
> > 
> > Working as expected using latest Tumbleweed official package.
> >
> Wow... Already?
> 
> Ok, perfect, closing it them.
> 
> Thanks everyone for all the information and all the testing!

May I please ask where will you track the debugging attempt as -D_FORTIFY_SOURCE=3 very likely exposes a real issue in qemu?

(In reply to Martin Liška from comment #54)
> May I please ask where will you track the debugging attempt as
> -D_FORTIFY_SOURCE=3 very likely exposes a real issue in qemu?
>
Right. We've added it to our team backlog, but it's not something we have the time/manpower to deal with right away.

In any case, it would be best if this is addressed upstream. In fact, I've QEMU's community about it:
https://lore.kernel.org/qemu-devel/6a6dbfb53f2ea5a9740249c2fdf480be183e6ee8.camel@suse.com/

Let's see if anyone picks it up/has time for helping us with that.

(In reply to Dario Faggioli from comment #55)
> (In reply to Martin Liška from comment #54)
> > May I please ask where will you track the debugging attempt as
> > -D_FORTIFY_SOURCE=3 very likely exposes a real issue in qemu?
> >
> Right. We've added it to our team backlog, but it's not something we have
> the time/manpower to deal with right away.

I see.

> 
> In any case, it would be best if this is addressed upstream. In fact, I've
> QEMU's community about it:
> https://lore.kernel.org/qemu-devel/6a6dbfb53f2ea5a9740249c2fdf480be183e6ee8.
> camel@suse.com/

You did well. One question: would it be possible to get a backtrace from the abort. Note the 'buffer overflow detected' write to stderr is following by call of abort and a backtrace would really help us?

> 
> Let's see if anyone picks it up/has time for helping us with that.

Btw. the issue happens here:

Starting program: /usr/bin/qemu-system-x86_64 -name guest=opensuse-factory,debug-threads=on -S -machine pc-q35-6.2,usb=off,vmport=off,dump-guest-core=off,memory-backend=pc.ram -accel kvm -cpu host,migratable=on -m 2048 -object \{\"qom-type\":\"memory-backend-ram\",\"id\":\"pc.ram\",\"size\":2147483648\} -overcommit mem-lock=off -smp 2,sockets=2,cores=1,threads=1 -uuid b1375f79-0900-4ee9-b033-c7fd4ba6a748 -no-user-config -nodefaults -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=16,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=17,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=18,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=19,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=20,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=21,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=22,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device pcie-root-port,port=23,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 -device pcie-root-port,port=24,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 -device pcie-root-port,port=25,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 -device pcie-root-port,port=26,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 -device pcie-root-port,port=27,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 -device pcie-root-port,port=28,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 -device pcie-root-port,port=29,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 -device virtio-scsi-pci,id=scsi0,bus=pci.6,addr=0x0 -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -blockdev \{\"driver\":\"file\",\"filename\":\"/var/lib/libvirt/images/opensuse-factory.qcow2\",\"node-name\":\"libvirt-1-storage\",\"auto-read-only\":true,\"discard\":\"unmap\"\} -blockdev \{\"node-name\":\"libvirt-1-format\",\"read-only\":false,\"discard\":\"unmap\",\"driver\":\"qcow2\",\"file\":\"libvirt-1-storage\",\"backing\":null\} -device ide-hd,bus=ide.0,drive=libvirt-1-format,id=sata0-0-0,bootindex=1 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0,index=0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -audiodev \{\"id\":\"audio1\",\"driver\":\"spice\"\} -spice port=5901,addr=127.0.0.1,disable-ticketing=on,seamless-migration=on -device virtio-vga,id=video0,max_outputs=1,bus=pcie.0,addr=0x1 -device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x1b -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0,audiodev=audio1 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 -object \{\"qom-type\":\"rng-random\",\"id\":\"objrng0\",\"filename\":\"/dev/urandom\"\} -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.5,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
...

*** buffer overflow detected ***: terminated

Thread 1 "qemu-system-x86" received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff642c380 (LWP 121307)]
0x00007ffff71ff55c in __pthread_kill_implementation () from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff71ff55c in __pthread_kill_implementation () at /lib64/libc.so.6
#1  0x00007ffff71ac6f6 in raise () at /lib64/libc.so.6
#2  0x00007ffff7195814 in abort () at /lib64/libc.so.6
#3  0x00007ffff71f279e in __libc_message () at /lib64/libc.so.6
#4  0x00007ffff729767a in __fortify_fail () at /lib64/libc.so.6
#5  0x00007ffff7295c36 in  () at /lib64/libc.so.6
#6  0x00007ffff72957f5 in __snprintf_chk () at /lib64/libc.so.6
#7  0x0000555555b1c1fd in pcibus_get_fw_dev_path ()
#8  0x0000555555f2bde4 in qdev_get_fw_dev_path_helper.constprop ()
#9  0x0000555555f2bd86 in qdev_get_fw_dev_path_helper.constprop ()
#10 0x00005555559a6e5d in get_boot_device_path ()
#11 0x00005555559a712c in get_boot_devices_list ()
#12 0x0000555555b1a3d0 in fw_cfg_machine_reset ()
#13 0x0000555555bf4c2d in pc_machine_reset ()
#14 0x0000555555c66988 in qemu_system_reset ()
#15 0x0000555555a6dff6 in qdev_machine_creation_done ()
#16 0x0000555555c79186 in qmp_x_exit_preconfig.part ()
#17 0x0000555555c7b459 in qemu_init ()
#18 0x0000555555960a29 in main ()

I have a patch for that.

The abort comes from the following line:

https://gitlab.com/qemu-project/qemu/-/blob/master/hw/pci/pci.c#L2650

        snprintf(path + off, sizeof(path) + off, ",%x", PCI_FUNC(d->devfn));

where the second argument should be sizeof(path) - off

Let's come up with a proper fix, reopenning.

I cooked up an upstream patch, queued now by the maintainer,

https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg06183.html

Thanks Martin, Dario for the find!

(In reply to Claudio Fontana from comment #60)
> I cooked up an upstream patch, queued now by the maintainer,
> 
> https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg06183.html
> 
> Thanks Martin, Dario for the find!

Great, can you please cherry-pick that to Virtualization/qemu package and push it to Factory?

Fixed in devel project.

SUSE-SU-2022:2254-1: An update that solves three vulnerabilities and has three fixes is now available.

Category: security (important)
Bug References: 1197084,1198035,1198037,1198712,1199018,1199924
CVE References: CVE-2021-4206,CVE-2021-4207,CVE-2022-26354
JIRA References: 
Sources used:
openSUSE Leap 15.3 (src):    qemu-5.2.0-150300.115.2, qemu-linux-user-5.2.0-150300.115.2, qemu-testsuite-5.2.0-150300.115.4
SUSE Linux Enterprise Module for Server Applications 15-SP3 (src):    qemu-5.2.0-150300.115.2
SUSE Linux Enterprise Module for Basesystem 15-SP3 (src):    qemu-5.2.0-150300.115.2
SUSE Linux Enterprise Micro 5.2 (src):    qemu-5.2.0-150300.115.2
SUSE Linux Enterprise Micro 5.1 (src):    qemu-5.2.0-150300.115.2

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.

SUSE-SU-2022:2260-1: An update that solves four vulnerabilities and has 5 fixes is now available.

Category: security (important)
Bug References: 1197084,1198035,1198037,1198711,1198712,1199015,1199018,1199625,1199924
CVE References: CVE-2021-4206,CVE-2021-4207,CVE-2022-26353,CVE-2022-26354
JIRA References: 
Sources used:
openSUSE Leap 15.4 (src):    qemu-6.2.0-150400.37.5.3, qemu-linux-user-6.2.0-150400.37.5.1, qemu-testsuite-6.2.0-150400.37.5.5
SUSE Linux Enterprise Module for Server Applications 15-SP4 (src):    qemu-6.2.0-150400.37.5.3
SUSE Linux Enterprise Module for Basesystem 15-SP4 (src):    qemu-6.2.0-150400.37.5.3

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.