Bug 904987

Created attachment 613254 [details]
Screenshot - List of disks by-uuid

In this screenshot you can see dracut looking for the root volume, however, because it has not unlocked the encrypted luks (on the screenshot shown as 3c8dd99d-28c7-4258-a19f-c75e0a067076), the boot process can't continue.

There was a similar report already?
As the 13.2 medium is out already I guess all that can be done is writing an article on the opensuse wiki page...

Federico: Would you mind doing so, please.
Please take care to have key words like 13.1 to 13.2 update/upgrade crypt/whatever included, so that others can find it quickly.

Then just paste in your workaround as described in this bug.
A reference to this bug may also be nice if we still find some more that can be done.

This would help others with a similar setup a lot.
Afaik Julian tried an update and it worked, but the setup may have been slightly different?

I found a mention of this problem in this thread:

https://www.mail-archive.com/initramfs@vger.kernel.org/msg03622.html

The actual thread mentions a few things but you will find the same particular issue happened. Here is an excerpt:

================
> This doesn't work with full disk encryption. Unit file does not get
> generated and I see this message:
> 
> [    7.142993] testhost dracut-initqueue[202]: Failed to start 
> systemd-cryptsetup@luks\x2d342d2414\x2d159c\x2d48d7\x2da0b7\x2d5b59fa6e27a5.service:
>  Unit 
> systemd-cryptsetup@luks\x2d342d2414\x2d159c\x2d48d7\x2da0b7\x2d5b59fa6e27a5.service
>  failed to load: No such file or directory.

Note that I don't have an entry for this luks device in /etc/crypttab. I
always thought that this is not necessary since all needed options
are in cmdline.
=================

If the best is to write a wiki article about it, I surely can give that a try.

Regards,

Federico

I didn't experience that problem I have in laptop mSATA with boot and encrypted LVM containing root and swap, and large HDD contains encrypted home. root and home use ext4. So that in 13.1 I was asked for two passwords during the boot. I on-line upgraded to 13.2 and there was no problem in the boot. The only think is I'm asked only for one password during the boot. I have no idea how his is possible. But it works. All partitions are correctly mounted.

In my case I did not have the entry for my encrypted volumes in crypttab. That is what I think caused the issue.

Do you have an entry for yours?

Yes, I do

$ cat /etc/crypttab 
cr_home         /dev/disk/by-id/ata-ST1000LM014-1EJ164_W3802Q1Y-part1 none       none
cr_ata-KINGSTON_SMS200S360G_50026B72420708CE-part3 /dev/disk/by-id/ata-KINGSTON_SMS200S360G_50026B72420708CE-part3 none       none

$ cat /etc/fstab
UUID=63dab9f4-dbe1-44be-a93d-b7a6e141b44d       /       ext4    noatime,defaults,noacl,user_xattr 1 1 
/dev/mapper/cr_home     /home   ext4    acl,user_xattr,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,nofail 0 2 
/dev/disk/by-id/ata-KINGSTON_SMS200S360G_50026B72420708CE-part2 /boot   ext4    acl,user_xattr 1 2 
UUID=cb0a2ff8-b112-4eb4-af0b-549c8ca05697       swap    swap    defaults 0 0 
/dev/disk/by-id/ata-KINGSTON_SMS200S360G_50026B72420708CE-part1 /boot/efi       vfat    umask=0002,utf8=true 0 0

I haven't touched those files.

OK, so the issue remains. Without an entry on crypttab, dracut will not provide support for the encrypted volumes (even if they are currently mounted and grub add sorry for them).

In the past it was not really necessary to have the entry in crypttab because grub would take care of them. Now, without that entry, an upgrade to 13.2 will render the system unbootable.

Regards

I didn't create cryptab. I encrypted the system during installation of 13.1, so the installer had to create it, I guess.

Created attachment 709050 [details]
dracut-shell

Hey,

I can reproduce it with kernel 4.9.0-2.gfd5379c-pae (i686).

After entering passphrase system froze for 2 mins. After that we get kernel-message every second:
dracut-initqueue[131]: Warning dracut-initqueue timeout - starting timeout stripts 

After 1 min we get dracut-shell...

I can't mount USB stick (for copying /run/initramfs/rdsosreport.txt), kernel doesn't see new devices.
Kernel 4.8.13-1-default works well.

Update kernel to 4.10 (from /Kernel:/HEAD/standard) fixes this problem.

@Alexander Naumov Please describe your upgrade path. When did this start to fail exactly? Was it related to a distro upgrade?

Created attachment 712432 [details]
Unit systemd-cryptsetup@luks.service not found

I am getting

> Failed to start systemd-cryptsetup@luks.service: Unit systemd-cryptsetup@luks.service not found.

This is the result of a default/suggested LUKS+LVM install using the 30-Jan Tumbleweed snapshot ISO.

I also believe it occurred after when doing a `zypper up` ~2 days ago.

Created attachment 712434 [details]
lsinitrd outputs

I just setup tumbleweed from 04022017 with lvm-based cryptoroot. I cannot reproduce this. Do you have a crypttab in your initrd? Specifically, please provide the output of

lsinitrd -f etc/crypttab.

Hi everybody, sorry, found this only after placing another bug, please have a look here as my issue seems similar:

https://bugzilla.opensuse.org/show_bug.cgi?id=1024240

Bye, 

Joost

*** Bug 1024240 has been marked as a duplicate of this bug. ***

Please provide the output of the following:

cat /etc/crypttab
sudo lsinitrd -f etc/crypttab
/usr/lib/systemd/system-generators/systemd-cryptsetup-generator && ls /tmp/*.service

(In reply to Daniel Molkentin from comment #18)
> Please provide the output of the following:
> 
> cat /etc/crypttab
> sudo lsinitrd -f etc/crypttab
> /usr/lib/systemd/system-generators/systemd-cryptsetup-generator && ls
> /tmp/*.service

Hi, thanks a lot for taking this up.

crypttab in initrd is a 0 byte file (empty).


Chrooting to manually mounted system partition with:

cryptsetup luksOpen /dev/nvme0n1p3 system
--- PW entry
lvm_scan
lvm vgchange -ay
mkdir /mnt
mount /dev/mapper/system-root /mnt
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys
chroot /mnt


system /etc/crypttab:
cr_nvme-200080d030005f1a7-part3 /dev/disk/by-id/nvme-200080d030005f1a7-part3 none none

Resulting file after generator run: please see attachment (as I can not boot anymore, it's typing or taking a picture for me).

Created attachment 713413 [details]
Output of systemd-cryptsetup-generator

One additional thought: is the unit file dependent on the /dev/disk/by-id devnames (I get this impression from looking at the file).

I ran ls -l /dev/disk/by-id in my initrd, where I get a completely different id for the luks encrypted partition than what is written in the crypttab and in the unit file (please see attachment). Might that be a lead?

Created attachment 713424 [details]
output of ls -l /dev/disk/by-id

Ok, some progress. I managed to boot into multiuser.target (not into graphical.target) with 

- chrooting from dracut rescue shell into manually mounted system partition (see above)
- changing system /etc/crypttab, using one line/entry with the /dev/disk/by-uuid/ name of the encrypted partition, options "none none"
- mount /boot & dracut -f in the chroot

Now, the crypttab gets copied into the initrd and the password prompt appears on next boot.

Though my boot is still not completely restored, as a boot into graphical.target hangs indefinitely after "Switching root" at the end of the systemd boot sequence, without X / SDDM coming up and without dropping to a shell.
Work around for now is to boot into multi-user.target (add "3" to the kernel boot command in GRUB), and then log in, afterwards run

systemctl isolate graphical.target.

Then X comes up. I have to 

systemctl restart NetworkManager

to get networking, then things seem to work ok.

I ran into the same problem. openSUSE Tumbleweed with Kernel 4.9.10 is not starting anymore. At boot, the plymouth splash screen doesn't show the password prompt. The system message from dracut-initqueue is: Failed to connect to the bus. No such file or directory.

Can you tell me, how to enter the dracut shell from grub?

(In reply to Steffen Golle from comment #24)
> I ran into the same problem. openSUSE Tumbleweed with Kernel 4.9.10 is not
> starting anymore. At boot, the plymouth splash screen doesn't show the
> password prompt. The system message from dracut-initqueue is: Failed to
> connect to the bus. No such file or directory.
> 
> Can you tell me, how to enter the dracut shell from grub?
The dracut shell should come up automatically if booting fails, though it has a delay of 1-2minutes; try to boot and come back 5 minutes later.
At least your error message is different from the one I had, perhaps you have more luck in fixing it. I had to give up and do a complete reinstall.

I have just attempted to install Tumbleweed on a new hardware.

 - Intel NUC
 - Corsair Force NVME SSD
 - OpenSUSE-Tumbleweed-DVD-x86_64-Snapshot20170304-Media.iso (4.10.1-1-default)

*all* installs have been booted through UEFI.

* First install attempt using the LUKS+LVM option resulted in 3 partitions. efi(vfat), boot(ext4), and system(lvm).

  On boot no password entry was requested, and was not present on any alternate TTYs. System dropped to a dracut shell.

* Second install attempt, the installer initially wanted to install to the USB stick that was installing it. When changing the target disk and selecting the LUKS+LVM option, only two partitions were created. efi(vfat) and system(lvm). 

  On boot Grub requested the LVM password, and the subsequent graphical boot also requested the password. System booted flawlessly.


* Third install attempt matched the third. No password prompt on any TTY.

  In dracut shell:
   * /etc/crypttab was empty

  In chrooted mount (from dracut shell):
   * /etc/crypttab contains `cr_-Force-part3 /dev/disk/by-id/-Force-part3 none none` There is no /dev/disk/by-id matching that name.
   * `lsinitrd -f etc/crypttab` returns no output. But `lsinitrd | grep crypttab` lists a 0 byte file.
   * `/usr/lib/systemd/system-generators/systemd-cryptsetup-generator` creates `cryptsetup.target.requires`, `dev-disk-by-id\x2did-\x2dForce\x2dpart3.device.wants`, `dev-mapper-cr_\x2dForce\x2dpart3.device.d`, `dev-mapper-cr_\x2dForce\x2dpart3.device.requires`, `systemd-cryptsetup@cr_\x2dForce\x2dpart3.service`. 


Running `dracut -f` did not change the size of crypttab in `lsinitrd`. 
Fixing the crypttab to point to somewhere that does exist, and then running `dracut -f` fixed the size of the crypttab in the initrd. On boot the password was requested, but did not progress past the loading throbber.

(In reply to Chris Targett from comment #26)
> I have just attempted to install Tumbleweed on a new hardware.
> 
>  - Intel NUC
>  - Corsair Force NVME SSD
>  - OpenSUSE-Tumbleweed-DVD-x86_64-Snapshot20170304-Media.iso
> (4.10.1-1-default)
> 
> *all* installs have been booted through UEFI.
> 
[...]
> * Third install attempt matched the third. No password prompt on any TTY.
> 
>   In dracut shell:
>    * /etc/crypttab was empty
> 
>   In chrooted mount (from dracut shell):
>    * /etc/crypttab contains `cr_-Force-part3 /dev/disk/by-id/-Force-part3
> none none` There is no /dev/disk/by-id matching that name.

Please open a new bug report against YaST2, as it puts a wrong udev id in there and attach the file generated by "save_y2logs" and the output of "hwinfo --disk".

Had same problem (boot only after manually entering cryptsetup luksOpen <device> <name> and then exiting) because of empty /etc/crypttab.

Updated /etc/crypttab and am able to boot properly (at least with own kernels).

Is there any update to this issue?

I stumbled across when applying updates to Leap 42.2. The default installation works well but after installing all updates, the system does not show the password prompt.

The system did not fall into the dracut shell but I can access it with the rescue system from the installation medium.

- /etc/crypttab is correct
- etc/crypttab in the initrd was empty
- Running dracut -f fixed the crypttab in the initrd but still no password prompt after reboot.

Is there anything else I need to run on the rescue system?

Hello,

I have the same issue but on Leap 42.2. My setup is LVM+LUKS, after doing zypper up to the latest version system is unbootable. I've tried to reproduce this issue on VM but it was not possible, so I guess it is somehow related to my configuration which is mSATA drive.

Link to the duplicated issue:
https://bugzilla.opensuse.org/show_bug.cgi?id=1049359

I'm running into the exact same problem as everyone else on 42.2 with latest updates applied using an NVMe.  I was able to restore a previous kernel/initrd from offline backup (4.4.70-18.9-default) and get back into a usable system but the latest updates never prompt for the password.  In case it helps at all:

delder:~ # cat /etc/crypttab
cr_nvme-200080d0200127f94-part4 /dev/disk/by-id/nvme-200080d0200127f94-part4 none       none
cr_ata-Samsung_SSD_850_EVO_2TB_S2HCNWAG702239M-part1 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_2TB_S2HCNWAG702239M-part1 none       none
cr_swap         /Slow/swap           none       noauto

delder:~ # lsinitrd -f etc/crypttab
cr_nvme-200080d0200127f94-part4 /dev/disk/by-id/nvme-200080d0200127f94-part4 none       none

delder:~ # /usr/lib/systemd/system-generators/systemd-cryptsetup-generator && ls /tmp/*.service
Failed to create unit file /tmp/systemd-cryptsetup@cr_nvme\x2d200080d0200127f94\x2dpart4.service: File exists
/tmp/systemd-cryptsetup@cr_ata\x2dSamsung_SSD_850_EVO_2TB_S2HCNWAG702239M\x2dpart1.service  /tmp/systemd-cryptsetup@cr_swap.service
/tmp/systemd-cryptsetup@cr_nvme\x2d200080d0200127f94\x2dpart4.service

Hi,

I hit the same issue after applying the latest patches to OpenSuse 42.2 two weeks ago on my system (HP Spetre 13 w/ NVME SSD). The system is configured with LUKS encrypted system LVM, only /boot is unencrypted. The encrypted LVM setup was done by the OpenSuse installer when I provisioned the system.

The symptom for me was exactly the same as in the other bug reports: no password prompt for unlocking and mounting the rootfs logical volume during boot.

The root cause that I found on the system was that the OpenSuse patch update automagically renamed the partitions in /dev/disk/by-id/ . However, the update neither rebuilt the /etc/crypttab with the new partition names nor rebuilt the initrd.

To get the system working again, I used the following procedure:

* Boot into emergency shell from grub menu (add '1' to end of kernel boot line), the shell with the boot output just errors out after not finding the rootfs and tells me to rebuild the initrd... 
* Selected emergency shell with ALT+F7 (shouldn't the emergency shell prompt open on the same shell as the boot messages?)
* dracut emergency shell /etc/crypttab contained old /dev/disk/by-id/ partition ...

* Booted rescue system from USB stick and setup chroot environment. Did the following in the chroot environment:

ls -l /dev/disk/by-id/
(find partition with the encrypted physical volume for the system LVM)
cryptsetup luksOpen /dev/nvme0n1p9 
lvscan
cp /etc/crypttab /etc/crypttab.old
vi /etc/crypttab
(replaced the wrong /dev/disk/by-id/... path with the absolute partition path /dev/nvme0n1p9)
mkinitrd
(rebuilt initrd, copies the new /etc/crypptab to initrd filesystem)
exit chroot
reboot

==> system now asks for password to unlock encrypted partition and boots as expected after password entry.

Created attachment 733768 [details]
/dev listing with changed devi/by-id/ names and old /etc/crypttab

This looks a lot like bug 1048679 (udev regression).

I agree with Fabian, this bug is not describing a single issue. I am therefore closing it. Please reopen new issues where applicable.