Bug 947816 - Failed to load kdump on openSUSE Leap
Summary: Failed to load kdump on openSUSE Leap
Status: REOPENED
Alias: None
Product: openSUSE Distribution
Classification: openSUSE
Component: Kernel (show other bugs)
Version: Leap 15.1
Hardware: Other Other
: P5 - None : Normal (vote)
Target Milestone: ---
Assignee: Joey Lee
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-28 14:31 UTC by Takashi Iwai
Modified: 2019-12-16 16:43 UTC (History)
6 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
kdump service log (609 bytes, text/plain)
2019-08-01 20:04 UTC, Ralf Kölmel
Details
verbose traces (1.00 KB, text/plain)
2019-08-02 15:21 UTC, Ralf Kölmel
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Takashi Iwai 2015-09-28 14:31:18 UTC
Loading kdump on openSUSE Leap beta1 failed.

journal shows:

kdump[7917]: Loaded kdump kernel: /sbin/kexec -p /boot/vmlinuz-4.1.6-10-desktop --append="quiet elevator=deadline sysrq=yes reset_devices acpi_no_memhotplug cgroup_disable=memory irqpoll nr_cpus=1 root=kdump disable_cpu_apicid=0   panic=1" --initrd=/boot/initrd-4.1.6-10-desktop-kdump  -s, Result: kexec_file_load failed: Key was rejected by service
load.sh[7861]: kexec_file_load failed: Key was rejected by service
Comment 1 Takashi Iwai 2015-09-28 14:43:26 UTC
Installing FACTORY kernel-exec and kdump seems to make things working.  So we need some updates in the user-space stack.

I added openSUSE_Leap_42.1 repo to Kernel:kdump project now...
Comment 2 Takashi Iwai 2015-09-28 14:52:20 UTC
It seems that kdump package on Leap that is derived from SLE12-SP1 doesn't work.

Petr, would you work on this?  Or should we just copy FACTORY packages to Leap?
Comment 3 Joey Lee 2015-09-28 15:05:18 UTC
(In reply to Takashi Iwai from comment #0)
> Loading kdump on openSUSE Leap beta1 failed.
> 
> journal shows:
> 
> kdump[7917]: Loaded kdump kernel: /sbin/kexec -p
> /boot/vmlinuz-4.1.6-10-desktop --append="quiet elevator=deadline sysrq=yes
> reset_devices acpi_no_memhotplug cgroup_disable=memory irqpoll nr_cpus=1
> root=kdump disable_cpu_apicid=0   panic=1"
> --initrd=/boot/initrd-4.1.6-10-desktop-kdump  -s, Result: kexec_file_load
> failed: Key was rejected by service
> load.sh[7861]: kexec_file_load failed: Key was rejected by service

openSUSE doesn't support kernel module verification, so kernel didn't embedded key. I think that's why the crash kernel binary doesn't pass the verification.

I will set CONFIG_KEXEC_VERIFY_SIG=n then check the status of loading crash kernel by kexec.
Comment 4 Takashi Iwai 2015-09-28 15:14:55 UTC
(In reply to Joey Lee from comment #3)
> (In reply to Takashi Iwai from comment #0)
> > Loading kdump on openSUSE Leap beta1 failed.
> > 
> > journal shows:
> > 
> > kdump[7917]: Loaded kdump kernel: /sbin/kexec -p
> > /boot/vmlinuz-4.1.6-10-desktop --append="quiet elevator=deadline sysrq=yes
> > reset_devices acpi_no_memhotplug cgroup_disable=memory irqpoll nr_cpus=1
> > root=kdump disable_cpu_apicid=0   panic=1"
> > --initrd=/boot/initrd-4.1.6-10-desktop-kdump  -s, Result: kexec_file_load
> > failed: Key was rejected by service
> > load.sh[7861]: kexec_file_load failed: Key was rejected by service
> 
> openSUSE doesn't support kernel module verification, so kernel didn't
> embedded key. I think that's why the crash kernel binary doesn't pass the
> verification.

Thanks, this is what I expected from the journal message.
 
> I will set CONFIG_KEXEC_VERIFY_SIG=n then check the status of loading crash
> kernel by kexec.

OK, that should work.  But this made me wonder whether SLE12 kdump package would ever work with other kernels than SLE's standard one?
Comment 5 Joey Lee 2015-09-29 09:57:59 UTC
(In reply to Takashi Iwai from comment #4)
> (In reply to Joey Lee from comment #3)
> > (In reply to Takashi Iwai from comment #0)
> > > Loading kdump on openSUSE Leap beta1 failed.
> > > 
> > > journal shows:
> > > 
> > > kdump[7917]: Loaded kdump kernel: /sbin/kexec -p
> > > /boot/vmlinuz-4.1.6-10-desktop --append="quiet elevator=deadline sysrq=yes
> > > reset_devices acpi_no_memhotplug cgroup_disable=memory irqpoll nr_cpus=1
> > > root=kdump disable_cpu_apicid=0   panic=1"
> > > --initrd=/boot/initrd-4.1.6-10-desktop-kdump  -s, Result: kexec_file_load
> > > failed: Key was rejected by service
> > > load.sh[7861]: kexec_file_load failed: Key was rejected by service
> > 
> > openSUSE doesn't support kernel module verification, so kernel didn't
> > embedded key. I think that's why the crash kernel binary doesn't pass the
> > verification.
> 
> Thanks, this is what I expected from the journal message.
>  

I just changed the CONFIG_KEXEC_VERIFY_SIG in config files of x86_64. Waiting merge to openSUSE-42.1 kernel.

> > I will set CONFIG_KEXEC_VERIFY_SIG=n then check the status of loading crash
> > kernel by kexec.
> 
> OK, that should work.  But this made me wonder whether SLE12 kdump package
> would ever work with other kernels than SLE's standard one?

hm... Currently the logic in kdump package is checking x86_64 architecture then direct call kexec_file_load() syscall.

From the viewpoint of security, kernel should blocks non-secure interfaces or enable the verification mechanism. I think kdump package should try 2 interfaces, kexec_load and kexec_file_load, even kernel may blocked them because security.

I will send patch to modify kdump logic.
Comment 6 Takashi Iwai 2015-09-29 10:07:03 UTC
(In reply to Joey Lee from comment #5)
> (In reply to Takashi Iwai from comment #4)
> > (In reply to Joey Lee from comment #3)
> > > (In reply to Takashi Iwai from comment #0)
> > > > Loading kdump on openSUSE Leap beta1 failed.
> > > > 
> > > > journal shows:
> > > > 
> > > > kdump[7917]: Loaded kdump kernel: /sbin/kexec -p
> > > > /boot/vmlinuz-4.1.6-10-desktop --append="quiet elevator=deadline sysrq=yes
> > > > reset_devices acpi_no_memhotplug cgroup_disable=memory irqpoll nr_cpus=1
> > > > root=kdump disable_cpu_apicid=0   panic=1"
> > > > --initrd=/boot/initrd-4.1.6-10-desktop-kdump  -s, Result: kexec_file_load
> > > > failed: Key was rejected by service
> > > > load.sh[7861]: kexec_file_load failed: Key was rejected by service
> > > 
> > > openSUSE doesn't support kernel module verification, so kernel didn't
> > > embedded key. I think that's why the crash kernel binary doesn't pass the
> > > verification.
> > 
> > Thanks, this is what I expected from the journal message.
> >  
> 
> I just changed the CONFIG_KEXEC_VERIFY_SIG in config files of x86_64.
> Waiting merge to openSUSE-42.1 kernel.

Now pulled, thanks.  I guess we need to fix the same for stable and master, too?

> > > I will set CONFIG_KEXEC_VERIFY_SIG=n then check the status of loading crash
> > > kernel by kexec.
> > 
> > OK, that should work.  But this made me wonder whether SLE12 kdump package
> > would ever work with other kernels than SLE's standard one?
> 
> hm... Currently the logic in kdump package is checking x86_64 architecture
> then direct call kexec_file_load() syscall.
> 
> From the viewpoint of security, kernel should blocks non-secure interfaces
> or enable the verification mechanism. I think kdump package should try 2
> interfaces, kexec_load and kexec_file_load, even kernel may blocked them
> because security.
> 
> I will send patch to modify kdump logic.

It sounds good.
Comment 7 Joey Lee 2015-09-29 16:35:17 UTC
(In reply to Takashi Iwai from comment #6)
> (In reply to Joey Lee from comment #5)
> > (In reply to Takashi Iwai from comment #4)
> > > (In reply to Joey Lee from comment #3)
> > > > (In reply to Takashi Iwai from comment #0)
> > > > > Loading kdump on openSUSE Leap beta1 failed.
> > > > > 
> > > > > journal shows:
> > > > > 
> > > > > kdump[7917]: Loaded kdump kernel: /sbin/kexec -p
> > > > > /boot/vmlinuz-4.1.6-10-desktop --append="quiet elevator=deadline sysrq=yes
> > > > > reset_devices acpi_no_memhotplug cgroup_disable=memory irqpoll nr_cpus=1
> > > > > root=kdump disable_cpu_apicid=0   panic=1"
> > > > > --initrd=/boot/initrd-4.1.6-10-desktop-kdump  -s, Result: kexec_file_load
> > > > > failed: Key was rejected by service
> > > > > load.sh[7861]: kexec_file_load failed: Key was rejected by service
> > > > 
> > > > openSUSE doesn't support kernel module verification, so kernel didn't
> > > > embedded key. I think that's why the crash kernel binary doesn't pass the
> > > > verification.
> > > 
> > > Thanks, this is what I expected from the journal message.
> > >  
> > 
> > I just changed the CONFIG_KEXEC_VERIFY_SIG in config files of x86_64.
> > Waiting merge to openSUSE-42.1 kernel.
> 
> Now pulled, thanks.  I guess we need to fix the same for stable and master,
> too?
> 

Thanks for your reminding, I just push changes to stable and master in my home branch.

> > > > I will set CONFIG_KEXEC_VERIFY_SIG=n then check the status of loading crash
> > > > kernel by kexec.
> > > 
> > > OK, that should work.  But this made me wonder whether SLE12 kdump package
> > > would ever work with other kernels than SLE's standard one?
> > 
> > hm... Currently the logic in kdump package is checking x86_64 architecture
> > then direct call kexec_file_load() syscall.
> > 
> > From the viewpoint of security, kernel should blocks non-secure interfaces
> > or enable the verification mechanism. I think kdump package should try 2
> > interfaces, kexec_load and kexec_file_load, even kernel may blocked them
> > because security.
> > 
> > I will send patch to modify kdump logic.
> 
> It sounds good.
Comment 8 Dominique Leuenberger 2015-10-15 10:09:44 UTC
From going through the history, it sounds like all activities are done here and that bug would be closeable?
Comment 9 Takashi Iwai 2015-10-15 10:22:02 UTC
There are some other odd things that prevent kdump working.  Someone needs to retest with the current state...

This is one thing we might be able to cover via openQA, I think.
Comment 10 Petr Tesařík 2015-10-15 11:04:58 UTC
For one thing, many things have been fixed in kdump for SLE12 SP1. These should go into Leap; IMO there's little sense in re-discovering them on Leap right now.
Comment 11 Dominique Leuenberger 2015-10-15 11:07:42 UTC
(In reply to Petr Tesařík from comment #10)
> For one thing, many things have been fixed in kdump for SLE12 SP1. These
> should go into Leap; IMO there's little sense in re-discovering them on Leap
> right now.

osc rdiff openSUSE:Leap:42.1 kdump SUSE:SLE-12-SP1:GA
=> no diff

latest entry in kdump.changes is from:
-------------------------------------------------------------------
Mon Oct 12 09:08:57 UTC 2015 - ptesarik@suse.cz
Comment 12 Joey Lee 2015-10-15 11:28:07 UTC
I didn't finish the work to modify kdump package to try 2 interfaces, kexec_load and kexec_file_load yet. But I think that should not block kexec_file_load interface used by kdump package because the CONFIG_KEXEC_VERIFY_SIG was turned off.
Comment 13 Takashi Iwai 2015-10-17 10:17:20 UTC
I confirmed that kdump worked on Leap RC1.  So let's close this bug for now.

Joey, if you want to continue working on kexec_load stuff, please clone the bug for FACTORY.
Comment 14 Joey Lee 2015-10-20 08:15:01 UTC
(In reply to Takashi Iwai from comment #13)
> I confirmed that kdump worked on Leap RC1.  So let's close this bug for now.
> 
> Joey, if you want to continue working on kexec_load stuff, please clone the
> bug for FACTORY.

I created a new bug boo#951144 reflect to kdump tool's change.

Thanks
Comment 15 Mike Galbraith 2016-02-09 08:02:01 UTC
(In reply to Takashi Iwai from comment #13)
> I confirmed that kdump worked on Leap RC1.  So let's close this bug for now.

It must have been reborn, because when running an SLES kernel, kdump says key rejected by service in up to the minute leap here.  Thanks to finding this bug, installing kdump from factory fixed me up.
Comment 16 Mike Galbraith 2016-09-28 05:56:46 UTC
(In reply to Mike Galbraith from comment #15)
> (In reply to Takashi Iwai from comment #13)
> > I confirmed that kdump worked on Leap RC1.  So let's close this bug for now.
> 
> It must have been reborn, because when running an SLES kernel, kdump says
> key rejected by service in up to the minute leap here.  Thanks to finding
> this bug, installing kdump from factory fixed me up.

And it is still broken:

homer:/root # rpm -q kdump
kdump-0.8.15-24.1.x86_64
homer:/root # systemctl status kdump
kdump.service - Load kdump kernel on startup
   Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled)
   Active: active (exited) since Wed 2016-09-28 07:41:32 CEST; 9min ago
  Process: 4061 ExecStop=/lib/kdump/unload.sh (code=exited, status=0/SUCCESS)
  Process: 4066 ExecStart=/lib/kdump/load.sh (code=exited, status=0/SUCCESS)
 Main PID: 4066 (code=exited, status=0/SUCCESS)

Sep 28 07:41:32 homer load.sh[4066]: kexec_file_load failed: Key was rejected by service
Sep 28 07:41:32 homer systemd[1]: Started Load kdump kernel on startup.

(/me builds factory kdump for leap-42.1)

homer:/root # rpm -q kdump
kdump-0.8.15-99.x86_64
homer:/root # systemctl status kdump
kdump.service - Load kdump kernel on startup
   Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled)
   Active: active (exited) since Wed 2016-09-28 07:53:18 CEST; 53s ago
  Process: 12280 ExecStop=/etc/init.d/boot.kdump stop (code=exited, status=0/SUCCESS)
  Process: 12288 ExecStart=/etc/init.d/boot.kdump start (code=exited, status=0/SUCCESS)
 Main PID: 12288 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/kdump.service

Sep 28 07:52:16 homer boot.kdump[12288]: Loading kdump
Sep 28 07:53:18 homer boot.kdump[12288]: ..done
Sep 28 07:53:18 homer systemd[1]: Started Load kdump kernel on startup.
homer:/root #
Comment 17 Mike Galbraith 2016-09-28 07:04:15 UTC
(In reply to Mike Galbraith from comment #16)
> (In reply to Mike Galbraith from comment #15)
> > (In reply to Takashi Iwai from comment #13)
> > > I confirmed that kdump worked on Leap RC1.  So let's close this bug for now.
> > 
> > It must have been reborn, because when running an SLES kernel, kdump says
> > key rejected by service in up to the minute leap here.  Thanks to finding
> > this bug, installing kdump from factory fixed me up.
> 
> And it is still broken:
> 
> homer:/root # rpm -q kdump
> kdump-0.8.15-24.1.x86_64
> homer:/root # systemctl status kdump
> kdump.service - Load kdump kernel on startup
>    Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled)
>    Active: active (exited) since Wed 2016-09-28 07:41:32 CEST; 9min ago
>   Process: 4061 ExecStop=/lib/kdump/unload.sh (code=exited, status=0/SUCCESS)
>   Process: 4066 ExecStart=/lib/kdump/load.sh (code=exited, status=0/SUCCESS)
>  Main PID: 4066 (code=exited, status=0/SUCCESS)
> 
> Sep 28 07:41:32 homer load.sh[4066]: kexec_file_load failed: Key was
> rejected by service
> Sep 28 07:41:32 homer systemd[1]: Started Load kdump kernel on startup.
> 
> (/me builds factory kdump for leap-42.1)
> 
> homer:/root # rpm -q kdump
> kdump-0.8.15-99.x86_64
> homer:/root # systemctl status kdump
> kdump.service - Load kdump kernel on startup
>    Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled)
>    Active: active (exited) since Wed 2016-09-28 07:53:18 CEST; 53s ago
>   Process: 12280 ExecStop=/etc/init.d/boot.kdump stop (code=exited,
> status=0/SUCCESS)
>   Process: 12288 ExecStart=/etc/init.d/boot.kdump start (code=exited,
> status=0/SUCCESS)
>  Main PID: 12288 (code=exited, status=0/SUCCESS)
>    CGroup: /system.slice/kdump.service
> 
> Sep 28 07:52:16 homer boot.kdump[12288]: Loading kdump
> Sep 28 07:53:18 homer boot.kdump[12288]: ..done
> Sep 28 07:53:18 homer systemd[1]: Started Load kdump kernel on startup.
> homer:/root #

And with this kdump version, kdump/crash works fine through 4.6 kernels, SLE, SLERT or homebrew.  After that, makedumpfile ceases to function, and taking makedumpfile from factory helps not at all.. but at least things work properly until 4.7.
Comment 18 Takashi Iwai 2016-09-28 10:55:23 UTC
(In reply to Mike Galbraith from comment #16)
> (/me builds factory kdump for leap-42.1)

Are you sure that the result is from your TW version kdump?

The kdump included in Leap contains the patches verifying signature. Are you testing with your own build kernel or the official SLE kernel with the SUSE key signed?

If you're using the self-made kernel, turning off CONFIG_KEXEC_VERIFY_SIG should make things working.  Or use TW version kdump that has no signature check.

The kdump really should be more robust in this regard.  Joey, how is the status of bug 951144, to allow the old method as fallback?  I guess this issue bytes many developers.
Comment 19 Mike Galbraith 2016-09-28 11:31:08 UTC
(In reply to Takashi Iwai from comment #18)
> (In reply to Mike Galbraith from comment #16)
> > (/me builds factory kdump for leap-42.1)
> 
> Are you sure that the result is from your TW version kdump?

If you mean 'factory', yes.
 
> The kdump included in Leap contains the patches verifying signature. Are you
> testing with your own build kernel or the official SLE kernel with the SUSE
> key signed?

All kernels run here are locally built.
 
> If you're using the self-made kernel, turning off CONFIG_KEXEC_VERIFY_SIG
> should make things working.  Or use TW version kdump that has no signature
> check.

My local mainline/stable configs do not contain CONFIG_KEXEC_VERIFY_SIG, only SLE12/SLE12-RT kernels do.
Comment 20 Mike Galbraith 2016-09-28 11:57:41 UTC
(In reply to Mike Galbraith from comment #19)
> (In reply to Takashi Iwai from comment #18)
> > (In reply to Mike Galbraith from comment #16)
> > > (/me builds factory kdump for leap-42.1)
> > 
> > Are you sure that the result is from your TW version kdump?
> 
> If you mean 'factory', yes.
>  
> > The kdump included in Leap contains the patches verifying signature. Are you
> > testing with your own build kernel or the official SLE kernel with the SUSE
> > key signed?
> 
> All kernels run here are locally built.
>  
> > If you're using the self-made kernel, turning off CONFIG_KEXEC_VERIFY_SIG
> > should make things working.  Or use TW version kdump that has no signature
> > check.
> 
> My local mainline/stable configs do not contain CONFIG_KEXEC_VERIFY_SIG,
> only SLE12/SLE12-RT kernels do.

Perhaps relevant...

/etc/sysconfig/kdump:
KDUMP_KERNELVER="4.4.20-smp"

That's a local config kernel.
Comment 21 Mike Galbraith 2016-09-28 12:05:40 UTC
(In reply to Takashi Iwai from comment #18)
> (In reply to Mike Galbraith from comment #16)
> > (/me builds factory kdump for leap-42.1)
> 
> Are you sure that the result is from your TW version kdump?

> homer:/root # rpm -q kdump
> kdump-0.8.15-99.x86_64

BTW, it says .99 because when I build packages with osc, I use --release=99, and squirrel the resulting rpms away for possible reuse.
Comment 22 Takashi Iwai 2016-09-28 12:32:11 UTC
(In reply to Mike Galbraith from comment #19)
> (In reply to Takashi Iwai from comment #18)
> > (In reply to Mike Galbraith from comment #16)
> > > (/me builds factory kdump for leap-42.1)
> > 
> > Are you sure that the result is from your TW version kdump?
> 
> If you mean 'factory', yes.

It's strange because the FACTORY / TW version doesn't invokes /lib/kdump/load.sh, but it invokes /etc/init.d/boot.kdump start, instead.
I doubt you're using Leap or SLE version of kdump.
Comment 23 Mike Galbraith 2016-09-28 13:00:02 UTC
(In reply to Takashi Iwai from comment #22)
> (In reply to Mike Galbraith from comment #19)
> > (In reply to Takashi Iwai from comment #18)
> > > (In reply to Mike Galbraith from comment #16)
> > > > (/me builds factory kdump for leap-42.1)
> > > 
> > > Are you sure that the result is from your TW version kdump?
> > 
> > If you mean 'factory', yes.
> 
> It's strange because the FACTORY / TW version doesn't invokes
> /lib/kdump/load.sh, but it invokes /etc/init.d/boot.kdump start, instead.
> I doubt you're using Leap or SLE version of kdump.

If copy/paste in comment 16 isn't good enough, how can I convince you?
Comment 24 Takashi Iwai 2016-09-28 13:07:32 UTC
OK, after checking the comment 16, I see the point.  Your TW version does work while Leap version doesn't?  I was confused by mixed comments, sorry ;)

Basically the situation of kdump itself is same as of the original bug report.  We "fixed" it by disabling CONFIG_KEXEC_VERIFY_SIG for openSUSE kernels.

So the question is whether you still get this particular error even with the local kernels without CONFIG_KEXEC_VERIFY_SIG.  Please clarify it: whether the same error "kexec_file_load failed: Key was rejected by service" appears even for kernels without that kconfig.
Comment 25 Mike Galbraith 2016-09-28 14:30:45 UTC
  
> So the question is whether you still get this particular error even with the
> local kernels without CONFIG_KEXEC_VERIFY_SIG.  Please clarify it: whether
> the same error "kexec_file_load failed: Key was rejected by service" appears
> even for kernels without that kconfig.

Yeah, it's SLE/SLERT kernels, and my mainline/stable/tip -rt trees, which I also configure with an enterprise derived config.

With maintenance lag time being what it is, if I want tools that can handle all of the kernels I work on, I'm gonna have to fix them up myself anyway, so it's not a big hairy deal, it just seems to that you should be able to use a stock suse workstation to work on any suse kernel, which you can't.
Comment 26 Takashi Iwai 2016-09-28 14:35:14 UTC
OK, thanks for confirmation.  Yes, it sucks, I fully agree.

The workaround shouldn't be too hard, but it seems that upstream didn't like the ideal to fall back.  Nevertheless, we may fix it inside kdump load.sh instead of kexec, too.
Comment 27 Joey Lee 2016-09-28 15:01:30 UTC
(In reply to Takashi Iwai from comment #26)
> OK, thanks for confirmation.  Yes, it sucks, I fully agree.
> 
> The workaround shouldn't be too hard, but it seems that upstream didn't like
> the ideal to fall back.  Nevertheless, we may fix it inside kdump load.sh
> instead of kexec, too.

Yes, the kexec upstream doesn't not take my patch on boo#951144 to fallback to old kexec system call. As you said, the kdump script needs to implement the fallback.
Comment 28 Joey Lee 2016-09-28 15:06:45 UTC
(In reply to Joey Lee from comment #27)
> (In reply to Takashi Iwai from comment #26)
> > OK, thanks for confirmation.  Yes, it sucks, I fully agree.
> > 
> > The workaround shouldn't be too hard, but it seems that upstream didn't like
> > the ideal to fall back.  Nevertheless, we may fix it inside kdump load.sh
> > instead of kexec, too.
> 
> Yes, the kexec upstream doesn't not take my patch on boo#951144 to fallback
> to old kexec system call. As you said, the kdump script needs to implement
> the fallback.

I simply traced the kdump code, my plan is to add the logic to init/rc.kdump.functions.
Comment 29 Joey Lee 2016-09-28 15:23:28 UTC
(In reply to Joey Lee from comment #28)
> (In reply to Joey Lee from comment #27)
> > (In reply to Takashi Iwai from comment #26)
> > > OK, thanks for confirmation.  Yes, it sucks, I fully agree.
> > > 
> > > The workaround shouldn't be too hard, but it seems that upstream didn't like
> > > the ideal to fall back.  Nevertheless, we may fix it inside kdump load.sh
> > > instead of kexec, too.
> > 
> > Yes, the kexec upstream doesn't not take my patch on boo#951144 to fallback
> > to old kexec system call. As you said, the kdump script needs to implement
> > the fallback.
> 
> I simply traced the kdump code, my plan is to add the logic to
> init/rc.kdump.functions.

hm... the kdump in openSUSE:Factory looks different with the kdump in openSUSE:42.1/42.2.
As Taskshi mention, the init/load.sh should takes the logic to call kexec-file.
Comment 30 Joey Lee 2016-09-28 17:50:26 UTC
(In reply to Mike Galbraith from comment #25)
>   
> > So the question is whether you still get this particular error even with the
> > local kernels without CONFIG_KEXEC_VERIFY_SIG.  Please clarify it: whether
> > the same error "kexec_file_load failed: Key was rejected by service" appears
> > even for kernels without that kconfig.
> 
> Yeah, it's SLE/SLERT kernels, and my mainline/stable/tip -rt trees, which I
> also configure with an enterprise derived config.
> 
> With maintenance lag time being what it is, if I want tools that can handle
> all of the kernels I work on, I'm gonna have to fix them up myself anyway,
> so it's not a big hairy deal, it just seems to that you should be able to
> use a stock suse workstation to work on any suse kernel, which you can't.

hm... I tried to build and install a self-signed SLE12-SP2 kernel on openSUSE 42.1. I found the result of loading kernel through kexec-file result that it doesn't like SLE12-SP2.

On SLE12-SP2, the self-signed kernel works fine to load through kexec-file. But on openSUSE 42.1, it already returns "Key was rejected by service" then I found this log in dmesg:

[    3.392659] PKCS7: Sig 1: X.509 chain contains auth-skid nonmatch (1->1)

The same kernel, the same signing process but it got different result on openSUSE with SLE12 SP2. I guess that it is about the pesign,  mozilla-nss or openssl.

Hi Mike, 

Did you see the same PKCS7 error message? If yes, then I prefer create another bug to reflect this situation. 

Thanks
Comment 31 Mike Galbraith 2016-09-29 00:51:52 UTC
> hm... I tried to build and install a self-signed SLE12-SP2 kernel on
> openSUSE 42.1. I found the result of loading kernel through kexec-file
> result that it doesn't like SLE12-SP2.
> 
> On SLE12-SP2, the self-signed kernel works fine to load through kexec-file.
> But on openSUSE 42.1, it already returns "Key was rejected by service" then
> I found this log in dmesg:
> 
> [    3.392659] PKCS7: Sig 1: X.509 chain contains auth-skid nonmatch (1->1)
> 
> The same kernel, the same signing process but it got different result on
> openSUSE with SLE12 SP2. I guess that it is about the pesign,  mozilla-nss
> or openssl.
> 
> Hi Mike, 
> 
> Did you see the same PKCS7 error message?

No, that's not in the journal.  (I whack it regularly though...)
Comment 32 Mike Galbraith 2016-09-29 03:22:01 UTC
 
> > (/me builds factory kdump for leap-42.1)
> > 
> > homer:/root # rpm -q kdump
> > kdump-0.8.15-99.x86_64
... 
> And with this kdump version, kdump/crash works fine through 4.6 kernels,
> SLE, SLERT or homebrew.  After that, makedumpfile ceases to function, and
> taking makedumpfile from factory helps not at all.. but at least things work
> properly until 4.7.

For the record, the reason I continued to have trouble dumping kernels > 4.6 was that I forgot to whack the existing kdump initrd after installing makedumpfile, so it would be rebuilt (install order matters).

I took kexec-tools and crash as well on general principal, and put the lot into a local directory repository, and am green across the board.. at least until the merge window opens in a couple days ;-)
Comment 34 Tomáš Chvátal 2018-04-13 14:58:39 UTC
This is automated batch bugzilla cleanup.

The openSUSE 42.1 changed to end-of-life (EOL [1]) status. As such
it is no longer maintained, which means that it will not receive any
further security or bug fix updates.
As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
openSUSE, or you can still observe it under openSUSE Leap 15.0, please
feel free to reopen this bug against that version (see the "Version"
component in the bug fields), or alternatively open
a new ticket.

Thank you for reporting this bug and we are sorry it could not be fixed
during the lifetime of the release.

[1] https://en.opensuse.org/Lifetime
Comment 39 Ralf Kölmel 2019-08-01 20:04:23 UTC
Created attachment 812509 [details]
kdump service log

On Leap 15.1 (Kernel 4.12.14-lp151.28.10-default, no CONFIG_KEXEC_VERIFY_SIG in kernel config) kdump service doesn't activate the panic kernel.
The trace "kexec_file_load failed: Key was rejected" is present in the kdump log, but the return value of the kexec (in the script /lib/kdump/load.sh) seams 0 and the service doesn't go into a failure state. My kernel crash tests are showing that the panic kernel is not activated and kdump is not working.
In this script the kexec is called with "output=$(eval "$KEXEC_CALL -s" 2>&1)"
If i remove the fix parameter -s or set -c, kdump is working and activates the panic kernel.
Why is "-s" used on Leap, when it is not working with the Leap kernels ?
Comment 40 Ralf Kölmel 2019-08-01 20:05:55 UTC
I reopen this bugreport for Leap 15.1, but i see this error trace (s. my last comment) also on the latest Leap 15.0.
Comment 41 Joey Lee 2019-08-02 07:27:28 UTC
Hi Ralf,

(In reply to Ralf Kölmel from comment #40)
> I reopen this bugreport for Leap 15.1, but i see this error trace (s. my
> last comment) also on the latest Leap 15.0.

Looks that the CONFIG_KEXEC_BZIMAGE_VERIFY_SIG diddn't enable on openSUSE, so arch_kexec_kernel_verify_sig() can not find the verify_sig() function.

Could you please enable the dynamic debug log?

Command line to enable the dynamic debug log:

echo -n 'file arch/x86/kernel/kexec-bzimage64.c +p' > /sys/kernel/debug/dynamic_debug/control

Or put the following kernel parameter to grub2 config:

ddebug_query=file arch/x86/kernel/kexec-bzimage64.c +p

Could you please replace issue then checking the "kernel loader does not support signature verification." debug log be exposed in dmesg?

Thanks
Comment 42 Joey Lee 2019-08-02 07:30:48 UTC
(In reply to Joey Lee from comment #41)
> Hi Ralf,
> 
> (In reply to Ralf Kölmel from comment #40)
> > I reopen this bugreport for Leap 15.1, but i see this error trace (s. my
> > last comment) also on the latest Leap 15.0.
[...snip]
> 
> Could you please replace issue then checking the "kernel loader does not
                   ^^^^^^^ reproduce

Sorry for my typo.
Comment 43 Ralf Kölmel 2019-08-02 15:21:00 UTC
Created attachment 812638 [details]
verbose traces

Dear Joey,
the message "kernel loader does not support signature verification." is not present after activating the debug log and restarting kdump service with the original kexec_file_load method. I've given the traces in the appended file.
Comment 44 Matwey Kornilov 2019-12-16 16:43:18 UTC
Currently, I see the same issue with 4.12.14-lp151.28.36-default