Bug 1169783 - Passphase prompt for full disk encryption dissappear after TW upgrade from 20200411-0 to 20200414-0
Passphase prompt for full disk encryption dissappear after TW upgrade from 20...
Status: RESOLVED FIXED
: 1169719 1169794 1169891 1169927 (view as bug list)
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Upgrade Problems
Current
64bit Other
: P2 - High : Normal with 46 votes (vote)
: ---
Assigned To: Cliff Zhao
Jiri Srain
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2020-04-17 15:11 UTC by andy great
Modified: 2022-09-06 05:46 UTC (History)
22 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---
qzhao: needinfo? (andythe_great)
qzhao: needinfo? (kv)


Attachments
Cannot boot to desktop, stuck here. (4.09 MB, image/jpeg)
2020-04-17 15:11 UTC, andy great
Details
Tumbleweed boot with an encrypted partition (3.66 MB, video/webm)
2020-04-22 15:22 UTC, Cliff Zhao
Details
Support log from Andy (1.97 MB, application/x-xz-compressed-tar)
2020-04-24 22:08 UTC, andy great
Details
Screenshot for Gparted and YaST Partition. (294.00 KB, image/png)
2020-04-24 22:17 UTC, andy great
Details
supportconfig (2.37 MB, application/x-xz)
2020-04-25 06:35 UTC, Michael K
Details
supportconfig K. Voinov (1.70 MB, application/x-xz-compressed-tar)
2020-08-23 09:20 UTC, Konstantin Voinov
Details
plymouth dbg K. Voinov (118.24 KB, text/x-log)
2020-08-23 09:21 UTC, Konstantin Voinov
Details

Note You need to log in before you can comment on or make changes to this bug.
Description andy great 2020-04-17 15:11:46 UTC
Created attachment 836048 [details]
Cannot boot to desktop, stuck here.

With the latest TW update, I could not boot to the desktop.

Normally, my PC will ask for my password for encrypted drive on Btrfs that I have setup according to https://en.opensuse.org/SDB:Encrypted_root_file_system which also avoid typing password twice.

Everything have been working well until the latest upgrade from 20200411-0 to 20200414-0.
The latest update did not ask me for the password, it seems to just skip it and would get stuck as the image shown.

I roll back with snapper rollback and try to upgrade again but fail the same way.

Any suggestion to how to get more info on this issue?

System info (not broken)

Operating System: openSUSE Tumbleweed 20200411

KDE Plasma Version: 5.18.4

KDE Frameworks Version: 5.68.0

Qt Version: 5.14.1

Kernel Version: 5.6.2-1-default

OS Type: 64-bit

Processors: 8 × Intel® Core™ i7-3770 CPU @ 3.40GHz

Memory: 7.5 GiB of RAM
Comment 1 andy great 2020-04-17 16:02:41 UTC
I lock all the plymouth in YaST2 and zypper dup, it work now. I guess it is plymouth bug.
Comment 2 Antoine Belvire 2020-04-17 18:59:31 UTC
*** Bug 1169794 has been marked as a duplicate of this bug. ***
Comment 3 Antoine Belvire 2020-04-17 19:02:40 UTC
cc plymouth maintainer
Comment 4 Antoine Belvire 2020-04-17 19:07:25 UTC
*** Bug 1169719 has been marked as a duplicate of this bug. ***
Comment 5 IT wrx 2020-04-17 22:23:21 UTC
In my case, it successfully prompts for the luks passphrase for both partitions on the root drive, but fails when it comes to a third, auxiliary internal sata drive. The first two are btrfs and the third is ext4, if that might matter.
Comment 6 Bug Reporter 2020-04-18 12:17:37 UTC
Can confirm. Booting into recovery mode, i.e. without plymouth, is also broken.
Comment 7 andy great 2020-04-18 22:26:51 UTC
@Bug Reporter
Try boot into working snapshot.
Lock all plymouth packages and upgrade the rest. That should fix the issue if plymouth were the problem, if not then it might be different issue.
Comment 8 Marcel Kuehlhorn 2020-04-20 14:23:21 UTC
*** Bug 1169891 has been marked as a duplicate of this bug. ***
Comment 9 Cliff Zhao 2020-04-20 15:42:59 UTC
Hi andy:
could you show me the support-log? 
Thank you so much!
Comment 10 IT wrx 2020-04-20 15:50:28 UTC
correction for comment #5: the root partition is btrfs, the home is xfs and the data drive that didn't ask for luks pass is ext4.
Comment 11 Bug Reporter 2020-04-20 16:03:49 UTC
(In reply to andy great from comment #7)

Yes, your workaround does alleviate this, so plymouth likely is the culprit. Although recovery mode does not display a splash screen, it apparently still relies on plymouth.
Comment 12 Javier Llorente 2020-04-20 17:03:27 UTC
I am also affected by this bug; I am no longer asked for my encrypted /home passphrase after I upgraded to 20200415 from a month-old snapshot.

Quick workaround: after the system goes automatically into rescue mode, remove /var/run/plymouth/pid and exit. The system will continue booting up.
Comment 13 Antoine Belvire 2020-04-20 18:29:27 UTC
*** Bug 1169927 has been marked as a duplicate of this bug. ***
Comment 14 Robert Kaiser 2020-04-20 20:03:22 UTC
I found I can work around this by waiting for the timeout and enter the root password to get into the recovery console, then running:

systemctl start cryptsetup.target

The prompt appears and I enter the password for my /home partition (which is the one encrypted disk I have). Then I run this to get back to the normal system:

systemctl default

And then the system comes up correctly.


Of course, this is an ugly workaround but at least the system can come up in some form in the mean time.
Comment 15 andy great 2020-04-20 22:53:43 UTC
@Cliff Zhao
What support-log?
Comment 16 t neo 2020-04-21 03:27:39 UTC
(In reply to Robert Kaiser from comment #14)
> I found I can work around this by waiting for the timeout and enter the root
> password to get into the recovery console, then running:
> 
> systemctl start cryptsetup.target
> 
> The prompt appears and I enter the password for my /home partition (which is
> the one encrypted disk I have). Then I run this to get back to the normal
> system:
> 
> systemctl default
> 
> And then the system comes up correctly.
> 
> 
> Of course, this is an ugly workaround but at least the system can come up in
> some form in the mean time.

You can fix it permanently: 

In the file /usr/lib/systemd/system/systemd-ask-password-plymouth.service 

Change FROM:
ConditionPathExists=/run/plymouth/pid

TO:
ConditionPathExists=/var/run/plymouth/pid
Comment 17 Keks Dose 2020-04-21 10:23:46 UTC
I ran into this as well, openSuse TW.

Machine stopped booting for no apparent reason and opened emergency prompt. 

After snapper rollback I locked all plymouth packages and made zypper dup again. Machine booted. So from my point ov fiew the new plymouth version is buggy. (Version: 0.9.5+git20191224+d7c737d as announced by DimStar April 16.)
Comment 18 Thomas Zimmermann 2020-04-21 12:42:34 UTC
FTR: This bug also happens on new installs.
Comment 19 Cliff Zhao 2020-04-21 15:28:01 UTC
(In reply to Thomas Zimmermann from comment #18)
> FTR: This bug also happens on new installs.

Okay Thomas, thank you so much for the confirmation. I will switch back to this issue when I finished the openSUSE:Leap gap problem.
Comment 20 IT wrx 2020-04-21 18:13:39 UTC
(In reply to Cliff Zhao from comment #19)
> (In reply to Thomas Zimmermann from comment #18)
> > FTR: This bug also happens on new installs.
> 
> Okay Thomas, thank you so much for the confirmation. I will switch back to
> this issue when I finished the openSUSE:Leap gap problem.

Forgive my ignorance, but isn't the openSUSE:Leap gap problem more of a longer term improvement/feature/project? while this *bug* breaks booting(!) for installs using encrypted filesystems. It seems to me, this would be more urgent. 

Also, as an end user who is largely ignorant in regards to openQA, i would expect openQA to catch something that breaks booting, so i wonder why it doesn't.  If there is something needed, maybe that can be highlighted, so people can know what needs to be done. 

I've used arch linux for years (prior to my recent ongoing move to TW) and it hasn't broken my ability to boot the OS, and TW is supposed to be stabilized.  Maybe plymouth could fail gracefully into text mode, instead of just hanging the whole booting process? I think i purposely didn't install plymouth on arch machines, b/c of problems with it in the past. Maybe i'll try removing all the plymouth packages in my TW and see if that lets me enter my luks passphrases without any trouble. I don't *need* my luks password entry themed anyways... Also, plymouth has 0.9.x version numbers. Maybe they mean it, and distros just aren't taking it seriously? Maybe the TW installer should just not install plymouth, unless/until it's fully tested and supported under all supported config conditions (disk encryption, etc).

BTW, my TW never went into any rescue mode. I'm guessing, b/c plymouth worked for my root drive partitions, but not an aux drive that was in fstab without "nofail" option.
Comment 21 Thomas Zimmermann 2020-04-22 06:10:46 UTC
(In reply to Cliff Zhao from comment #19)
> (In reply to Thomas Zimmermann from comment #18)
> > FTR: This bug also happens on new installs.
> 
> Okay Thomas, thank you so much for the confirmation. I will switch back to
> this issue when I finished the openSUSE:Leap gap problem.

Well, I don't know, but this bug breaks many users' systems. I'd prioritize this one. Simply reverting Plymouth back to the old version would be an improvement.
Comment 22 Cliff Zhao 2020-04-22 09:44:01 UTC
(In reply to andy great from comment #15)
> @Cliff Zhao
> What support-log?

1, $ zypper install supportutils
2, $ supportconfig
3, Upload the tar bar from your system "/var/log/scc_...txz" to here as an
attachment.
Comment 23 Cliff Zhao 2020-04-22 15:22:14 UTC
Created attachment 836470 [details]
Tumbleweed boot with an encrypted partition

(In reply to Thomas Zimmermann from comment #21)
> (In reply to Cliff Zhao from comment #19)
> > (In reply to Thomas Zimmermann from comment #18)
> > > FTR: This bug also happens on new installs.
> > 
> > Okay Thomas, thank you so much for the confirmation. I will switch back to
> > this issue when I finished the openSUSE:Leap gap problem.
> 
> Well, I don't know, but this bug breaks many users' systems. I'd prioritize
> this one. Simply reverting Plymouth back to the old version would be an
> improvement.
Okay, I'll come up with some remedies if I can not deal with them in a short term.
the attachment is the test for the latest Tumbleweed installed with encrypted partition but it seems works fine. I don't know if there has any difference between us. Do you have any suggestions?
Thank you!
Comment 24 Cliff Zhao 2020-04-24 07:32:11 UTC
Hi andy:
I can not reproduce this bug, Could you please show me detailed steps?
How did you partition your disk? what is your LVM status? and which partition did you encrypted?
Could you show me your support log now?

Thank you so much for the feedback.
Comment 25 t neo 2020-04-24 15:52:21 UTC
I can't speak for Andy, but the issue is in the update to 2020-04-16 for me. The path in the plymouth.service file was not updated correctly. A fresh install with the same snapshot gave the same error. 

Thus a way to replicate is installing snapshot 2020-04-10 and upgrade to 2020-04-16. Or install snapshot 2020-04-16 fresh. Once the path in plymouth.service is fixed a new upgrade with zypper dup did not throw an issue.
Comment 26 t neo 2020-04-24 15:55:00 UTC
Oh and when installing fresh. Keep your partitioning scheme, as that might be a trigger for this issue. That a new schema with a new encrypted partition breaks Plymouth. In my situation home is separated and encrypted and I have a data drive encrypted, both using XFS.
Comment 27 Bug Reporter 2020-04-24 21:47:06 UTC
In my case, the problem occurs with an extraneous encrypted drive that is unlocked at boot (via /etc/crypttab), but not mounted. Hence, filesystem and partition layout do not seem to matter.
Comment 28 andy great 2020-04-24 22:03:23 UTC
I could not upload support log at the moment, I got server error.
I tried to use sfdisk and parted to give partition layout. Not sure if it help.

andy@linux=:~> sudo sfdisk -l
Disk /dev/sda: 465.78 GiB, 500107862016 bytes, 976773168 sectors
Disk model: CT500MX500SSD1  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 2DD3C9B4-4604-4A48-85B0-FEE0645CC722

Device         Start       End   Sectors   Size Type
/dev/sda1       2048     18431     16384     8M BIOS boot
/dev/sda2      18432  16795647  16777216     8G Linux swap
/dev/sda3   16795648 121653247 104857600    50G Linux filesystem
/dev/sda4  121653248 976766975 855113728 407.8G Linux filesystem


Disk /dev/mapper/cr_ata-CT500MX500SSD1_1914E1F81E8B-part4: 407.77 GiB, 437816131584 bytes, 855109632 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
andy@linux:~> sudo parted -l
Model: ATA CT500MX500SSD1 (scsi)
Disk /dev/sda: 500GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: pmbr_boot

Number  Start   End     Size    File system     Name  Flags
 1      1049kB  9437kB  8389kB                        bios_grub
 2      9437kB  8599MB  8590MB  linux-swap(v1)        swap
 3      8599MB  62.3GB  53.7GB  btrfs                 legacy_boot
 4      62.3GB  500GB   438GB
Comment 29 andy great 2020-04-24 22:08:39 UTC
Created attachment 836774 [details]
Support log from Andy
Comment 30 andy great 2020-04-24 22:09:26 UTC
I upload the support log.
Just noticed that the file owner have to change to user first before I can upload.
Comment 31 Robert Kaiser 2020-04-24 22:12:38 UTC
(In reply to t neo from comment #16)
> In the file /usr/lib/systemd/system/systemd-ask-password-plymouth.service 

FWIW, that file does not exist in my installation or in the plymouth package.
Comment 32 andy great 2020-04-24 22:16:22 UTC
(In reply to Robert Kaiser from comment #31)
> (In reply to t neo from comment #16)
> > In the file /usr/lib/systemd/system/systemd-ask-password-plymouth.service 
> 
> FWIW, that file does not exist in my installation or in the plymouth package.

I do have that file on my TW installation.
Comment 33 andy great 2020-04-24 22:17:09 UTC
Created attachment 836775 [details]
Screenshot for Gparted and YaST Partition.
Comment 34 Michael K 2020-04-25 06:35:41 UTC
Created attachment 836779 [details]
supportconfig

Need another supportconfig? Just in case..
Comment 35 Robert Kaiser 2020-04-25 12:25:30 UTC
(In reply to andy great from comment #32)
> (In reply to Robert Kaiser from comment #31)
> > (In reply to t neo from comment #16)
> > > In the file /usr/lib/systemd/system/systemd-ask-password-plymouth.service 
> > 
> > FWIW, that file does not exist in my installation or in the plymouth package.
> 
> I do have that file on my TW installation.

Ugh, yes, my fault, I looked for plymouth-* when it clearly isn't.

But this looks strange:
> grep "pid" /usr/lib/systemd/system/*plymouth*
/usr/lib/systemd/system/plymouth-start.service:ExecStart=/usr/sbin/plymouthd --mode=boot --pid-file=/var/run/plymouth/pid --attach-to-session
/usr/lib/systemd/system/systemd-ask-password-plymouth.path:ConditionPathExists=/run/plymouth/pid
/usr/lib/systemd/system/systemd-ask-password-plymouth.service:ConditionPathExists=/var/run/plymouth/pid

Note that the middle one, in .path, points to /run and not to /var/run as the other two...
Comment 36 Robert Kaiser 2020-04-25 12:49:52 UTC
Hmm, and even more:
> grep "plymouth/pid" /usr/lib/systemd/system/*.{path,service}
/usr/lib/systemd/system/systemd-ask-password-console.path:ConditionPathExists=!/run/plymouth/pid
/usr/lib/systemd/system/systemd-ask-password-plymouth.path:ConditionPathExists=/run/plymouth/pid
/usr/lib/systemd/system/plymouth-start.service:ExecStart=/usr/sbin/plymouthd --mode=boot --pid-file=/var/run/plymouth/pid --attach-to-session
/usr/lib/systemd/system/systemd-ask-password-console.service:ConditionPathExists=!/run/plymouth/pid
/usr/lib/systemd/system/systemd-ask-password-plymouth.service:ConditionPathExists=/var/run/plymouth/pid

FWIW, as I'm using the proprietary NVidia driver here, I do not get a graphical screen for password entry but always fall back to the console, which may play a part as well.
Comment 37 Robert Kaiser 2020-04-25 13:39:05 UTC
That said, switching all those paths to /var/run did not fix it for me, but then, I also do not run into the timeout any more that ended up giving me the possibility to work around the issue, so my system doesn't boot at all any more atm and I have to use the rescue system from a Leap 15.1 DVD to get to anything.
This new problem happened after upgrading from Tumbleweed 20200416 to 20200422, and looking through the emails on packages updates in between those I see systemd 245, pointing to https://github.com/openSUSE/systemd/blob/SUSE/v245/NEWS whioch mentions an "UPGRADE ISSUE" connected to plymouth which lead me to https://bugzilla.redhat.com/show_bug.cgi?id=1807771 and https://gitlab.freedesktop.org/plymouth/plymouth/-/merge_requests/99 - which could also be connected.

Now applying that on the system before rebooting does not fix this for me yet, though I see the prompt for a very short time on the console output screen. I did reboot and try to enter the password when that was visible, but not not complete that (3-4 stars appeared there), but then when the switch to the three-dot-text screen of plymouth comes up, now it actually switches to the text-screen plymouth prompt and that works correctly...

So not completely fixed but getting closer...
Comment 38 Robert Kaiser 2020-04-25 13:48:03 UTC
OK, and to make this more clear: I need to press Enter on the prompt on the console just before the plymouth three-dot screen comes up, and then the plymouth prompt appears, and then I can enter the passphrase for the disk just fine. May take pressing enter on the console prompt a few times to make sure it's not prompting at the time of the switch.
Comment 39 Robert Kaiser 2020-04-25 13:57:56 UTC
Sorry for yet another comment, but it seems I can enter it there but it may not work. Entering it on the console prompt before the switch happens does work though. Thankfully my passphrase is not too long to be entered in that short time.
Comment 40 Konstantin Voinov 2020-04-27 04:24:51 UTC
I'm stuck with this bug too. Is there "official" workaround?
If it needs logs/diags from my suffering system - please tell.
Comment 41 andy great 2020-04-27 15:38:01 UTC
(In reply to Konstantin Voinov from comment #40)
> I'm stuck with this bug too. Is there "official" workaround?
> If it needs logs/diags from my suffering system - please tell.

You could roll back to working snapshot and lock all plymouth packages from updating and just update the rest.
Comment 42 andy great 2020-04-27 15:39:06 UTC
(In reply to Konstantin Voinov from comment #40)
> I'm stuck with this bug too. Is there "official" workaround?
> If it needs logs/diags from my suffering system - please tell.

Also more system logs will be beneficial. Do it as comment 22 says.
Comment 43 Jacob W 2020-04-27 21:47:54 UTC
I had this issue too, but my setup is more standard and oddly enough, I had it *much* worse than others. I only have /home encrypted via LUKS. Paritions are standard for TW (meaning / is btrfs with default TW setup).

So while I could not get the password prompt to work at all, even if disabling plymouth via grub, I also had the "pleasure" of having all my snapshots removed! I could not rollback because all snapshots disappeared from grub and from /.snapshots in recovery mode.

After much cursing (which helps a lot in these types of situations & I highly recommend it), many reboots and trial and error, including editing crypttab because the man page has wrong info re the password entry, by some miracle, I finally got my snapshots to show up in grub again. I took no further risks, and immedietly proceeding to rollback, locked every package that has anything to do with plymouth, and zypper dup again. Now my system works fine.

I hate plymouth with a passion. Years ago I had a similar situation but without the disappearing snapshots (I reported a bug, but plymouth maintainer decided to ignore it after some time).

There was talk awhile back of removing plymouth from openSUSE. This needs to be done, not just talked about.

I'm most definitely not risking more horrors & decided not to "retry" to get logs.
Comment 44 andy great 2020-04-27 23:41:38 UTC
(In reply to Jacob W from comment #43)
> I had this issue too, but my setup is more standard and oddly enough, I had
> it *much* worse than others. I only have /home encrypted via LUKS. Paritions
> are standard for TW (meaning / is btrfs with default TW setup).
> 
> So while I could not get the password prompt to work at all, even if
> disabling plymouth via grub, I also had the "pleasure" of having all my
> snapshots removed! I could not rollback because all snapshots disappeared
> from grub and from /.snapshots in recovery mode.
> 
> After much cursing (which helps a lot in these types of situations & I
> highly recommend it), many reboots and trial and error, including editing
> crypttab because the man page has wrong info re the password entry, by some
> miracle, I finally got my snapshots to show up in grub again. I took no
> further risks, and immedietly proceeding to rollback, locked every package
> that has anything to do with plymouth, and zypper dup again. Now my system
> works fine.
> 
> I hate plymouth with a passion. Years ago I had a similar situation but
> without the disappearing snapshots (I reported a bug, but plymouth
> maintainer decided to ignore it after some time).
> 
> There was talk awhile back of removing plymouth from openSUSE. This needs to
> be done, not just talked about.
> 
> I'm most definitely not risking more horrors & decided not to "retry" to get
> logs.

Plymouth have ability to delete .snapshots? Maybe this have to be filed in a separate bug/feature request to limit plymouth permission?
Comment 45 andy great 2020-04-28 03:09:47 UTC
Tumbleweed snapshot 20200425 with new plymouth update release, have anyone try it?
Comment 46 Konstantin Voinov 2020-04-28 04:34:57 UTC
>>Tumbleweed snapshot 20200425 with new plymouth update release, have anyone try it?

the same
Comment 47 IT wrx 2020-04-29 04:43:23 UTC
(In reply to andy great from comment #45)
> Tumbleweed snapshot 20200425 with new plymouth update release, have anyone
> try it?

i removed all plymouth packages and am mounting the partition plymouth (or something) was having trouble with, with a bash script and systemd service instead.
Comment 48 Thomas Zimmermann 2020-04-29 06:26:21 UTC
(In reply to andy great from comment #45)
> Tumbleweed snapshot 20200425 with new plymouth update release, have anyone
> try it?

It actually got worse. There is a systemd update as part of the snapshot. And now it doesn't even seem to t go into emergency mode (within a few minutes?) automatically. So booting and repairing just got harder.
Comment 49 J Merkel 2020-04-29 09:55:25 UTC
A question for clarification, hoping some of you could answer.
I'm not (yet) affected by this bug, because I've stopped updating when I saw this bug report on the mailing list.
All 3 partitions on my TW installation are encrypted, and I've configured the system i.e. grub to ask only once for the passphrase, all three partitions are then unlocked by a keyfile provided in cryptab later in the process.
I do not see plymouth involved in this case so, in case I would do an upgrade would the configuration I've described be affected at all by the bug ?
Comment 50 Alexander Linkenbach 2020-04-29 22:22:08 UTC
Same problem, after update system hangs where it should ask for encryption password of the encrypted XFS /home partition. Slightly different behavior: never times out and goes into emergency mode, just hangs without prompt. I seem to remember with older versions it did. Disabling plymouth successfully used as work around.
Comment 51 J Merkel 2020-04-30 20:09:06 UTC
(In reply to J Merkel from comment #49)
> A question for clarification, hoping some of you could answer.
> I'm not (yet) affected by this bug, because I've stopped updating when I saw
> this bug report on the mailing list.
> All 3 partitions on my TW installation are encrypted, and I've configured
> the system i.e. grub to ask only once for the passphrase, all three
> partitions are then unlocked by a keyfile provided in cryptab later in the
> process.
> I do not see plymouth involved in this case so, in case I would do an
> upgrade would the configuration I've described be affected at all by the bug
> ?

To answer myself for the sake of others in similar situation:
My assumption was right, I did an upgrade to 20200428, my config boots without problem.
Plymouth is showing the three dots while decrypting the drives but is not screwing the boot process.

Why on earth is plymouth being used on TW - a (b)leading edge rolling release where probably most users want to see the diagnostics at boot and push the ESC key anyhow???
Comment 52 Hans-Peter Jansen 2020-05-01 11:15:10 UTC
(In reply to J Merkel from comment #51)
> 
> Why on earth is plymouth being used on TW - a (b)leading edge rolling
> release where probably most users want to see the diagnostics at boot and
> push the ESC key anyhow???

I beg to differ. 

My users and especially me enjoys my custom grub2/plymouth boot screens **a lot**. Using openSUSE everywhere (desktops, notebooks, servers, set top boxes (eg. vdr)), plymouth is making a whole lot of difference during boot. I even learned to make nvidia driver driven systems behave in this regard (nvidia-drm.modeset=1). 

BTW, on servers, I don't install plymouth at all, remove quiet and any splash args from kernel cmdline. Why do you bother with plymouth, if you hate it? But you shouldn't infer from yourself to others!
Comment 53 t neo 2020-05-01 15:24:24 UTC
Moving forward to the latest Tumbleweed the issue is still here. I was not easily able to fix it this time around and rolled back to my previous working snapshot. 

@Cliff Zhao do you have an update on the fix for the upgrade path with Plymouth?
Comment 54 Cliff Zhao 2020-05-03 14:57:16 UTC
(In reply to t neo from comment #53)
> Moving forward to the latest Tumbleweed the issue is still here. I was not
> easily able to fix it this time around and rolled back to my previous
> working snapshot. 
> 
> @Cliff Zhao do you have an update on the fix for the upgrade path with
> Plymouth?

Right now, I have 2~3 priorities beyond this problem on my to-do list, and if I take it upon myself to prioritize it, I will be criticized. sorry.
but don't worry, I will do it as soon as possible.
Comment 55 Ricardo Klein 2020-05-04 08:27:47 UTC
I am just wondering how many users are moving to another distro after *2 weeks with an update that can simply make your machine useless*, and how that affects Leap and even SLE in the long run...
Comment 56 Konstantin Voinov 2020-05-04 09:02:24 UTC
I'm not gonna switch. But no any official statement makes things worse.
Comment 57 Michael K 2020-05-04 09:12:44 UTC
They should either stop those plymouth updates or at least show a warning.
Comment 58 t neo 2020-05-04 14:20:36 UTC
I'm not going to switch, but it is worrisome that this issue is not given priority.
Comment 59 Dominique Leuenberger 2020-05-04 16:33:46 UTC
As the plymouth maintainer seems not to have capacity to actually solve this issue, I have reverted plymouth back to the version of the TW snapshot 0411 (0.9.5+git20190908+3abfab2)

This does not fix the issue, but at least gets our users back on track to not having to worry. It would be great if some could stay around and test future versions Cliff might be providing - before they hit Tumbleweed.

The reverted version will be part of Snapshot 20200504
Comment 60 Andreas Schwab 2020-05-05 09:43:26 UTC
This has also caused all userspace boot messages to disapper from the console after Starting Show Plymouth Boot Screen.
Comment 61 Daniel Molkentin 2020-05-06 07:43:15 UTC
I think I have a fix for the crypto partition unlock problem: The run directory, which is critical to the process is incorrect after dropping a patch that hard codes it, in favor of letting configure auto guess. Unfortunately, due to variations in the build environment, this goes wrong (which is consistent with #c16).

If I am correct, the package from https://software.opensuse.org//download.html?project=home%3Admolkentin%3Abranches%3ABase%3ASystem&package=plymouth fixes the Problem. Could someone please try that?

Andreas: Clearing the console is another issue, and it probably should be in a separate report.
Comment 62 t neo 2020-05-07 13:49:22 UTC
Thanks @Dominique Leuenberger I have without issues moved forward to the latest snapshot.
Comment 63 Konstantin Voinov 2020-05-08 03:46:53 UTC
(In reply to Daniel Molkentin from comment #61)
> I think I have a fix for the crypto partition unlock problem: The run
> directory, which is critical to the process is incorrect after dropping a
> patch that hard codes it, in favor of letting configure auto guess.
> Unfortunately, due to variations in the build environment, this goes wrong
> (which is consistent with #c16).
> 
> If I am correct, the package from
> https://software.opensuse.org//download.
> html?project=home%3Admolkentin%3Abranches%3ABase%3ASystem&package=plymouth
> fixes the Problem. Could someone please try that?
> 
> Andreas: Clearing the console is another issue, and it probably should be in
> a separate report.

I've tried. Switched to your repo, but plymouth fails on start:

-- Logs begin at Fri 2020-05-08 13:38:13 +10, end at Fri 2020-05-08 13:42:19 +10. --
мая 08 13:38:14 kot-sony systemd[1]: Starting Show Plymouth Boot Screen...
мая 08 13:38:14 kot-sony plymouthd[345]: /usr/sbin/plymouthd: symbol lookup error: /usr/sbin/plymouthd: undefined symbol: ply_kernel_command_line_get_key_value
мая 08 13:38:14 kot-sony plymouthd[343]: failed to read status from child immediately after starting to daemonize: Success
мая 08 13:38:14 kot-sony systemd[1]: plymouth-start.service: Control process exited, code=exited, status=1/FAILURE
мая 08 13:38:14 kot-sony systemd[1]: plymouth-start.service: Failed with result 'exit-code'.
мая 08 13:38:14 kot-sony systemd[1]: Failed to start Show Plymouth Boot Screen.
мая 08 13:38:16 kot-sony systemd[1]: Starting Show Plymouth Boot Screen...
мая 08 13:38:16 kot-sony plymouthd[448]: /usr/sbin/plymouthd: symbol lookup error: /usr/sbin/plymouthd: undefined symbol: ply_kernel_command_line_get_key_value
мая 08 13:38:16 kot-sony plymouthd[447]: unexpectedly exited with status 127 immediately after starting to daemonize
мая 08 13:38:16 kot-sony systemd[1]: plymouth-start.service: Control process exited, code=exited, status=1/FAILURE
мая 08 13:38:16 kot-sony systemd[1]: plymouth-start.service: Failed with result 'exit-code'.
мая 08 13:38:16 kot-sony systemd[1]: Failed to start Show Plymouth Boot Screen.
мая 08 13:38:18 kot-sony systemd[1]: Starting Show Plymouth Boot Screen...
мая 08 13:38:19 kot-sony systemd[1]: Started Show Plymouth Boot Screen.

At least text console with password prompt appears.

zypper se -si plymouth
Загрузка данных о репозиториях...
Чтение установленных пакетов...

С  | Имя                        | Тип   | Версия                          | Архитектура | Репозиторий
---+----------------------------+-------+---------------------------------+-------------+-------------------------------------
i+ | plymouth                   | пакет | 0.9.5+git20200418+14e91cc-290.1 | x86_64      | home:dmolkentin:branches:Base:System
i+ | plymouth-branding-openSUSE | пакет | 84.87.20191004-3.7              | noarch      | openSUSE:Tumbleweed
i+ | plymouth-branding-openSUSE | пакет | 84.87.20191004-3.7              | noarch      | openSUSE:Factory
i+ | plymouth-branding-openSUSE | пакет | 84.87.20191004-3.7              | noarch      | openSUSE-Tumbleweed-Oss
i+ | plymouth-dracut            | пакет | 0.9.5+git20200418+14e91cc-290.1 | x86_64      | home:dmolkentin:branches:Base:System
i+ | plymouth-plugin-label      | пакет | 0.9.5+git20200418+14e91cc-290.1 | x86_64      | home:dmolkentin:branches:Base:System
i+ | plymouth-plugin-label-ft   | пакет | 0.9.5+git20200418+14e91cc-290.1 | x86_64      | home:dmolkentin:branches:Base:System
i+ | plymouth-plugin-script     | пакет | 0.9.5+git20200418+14e91cc-290.1 | x86_64      | home:dmolkentin:branches:Base:System
i+ | plymouth-plugin-two-step   | пакет | 0.9.5+git20200418+14e91cc-290.1 | x86_64      | home:dmolkentin:branches:Base:System
i+ | plymouth-scripts           | пакет | 0.9.5+git20200418+14e91cc-290.1 | noarch      | home:dmolkentin:branches:Base:System
i+ | plymouth-theme-bgrt        | пакет | 0.9.5+git20200418+14e91cc-290.1 | noarch      | home:dmolkentin:branches:Base:System
i+ | plymouth-theme-spinner     | пакет | 0.9.5+git20200418+14e91cc-290.1 | noarch      | home:dmolkentin:branches:Base:System
Comment 64 Javier Llorente 2020-05-11 10:41:15 UTC
(In reply to Dominique Leuenberger from comment #59)
> As the plymouth maintainer seems not to have capacity to actually solve this
> issue, I have reverted plymouth back to the version of the TW snapshot 0411
> (0.9.5+git20190908+3abfab2)
> 
> This does not fix the issue, but at least gets our users back on track to
> not having to worry. It would be great if some could stay around and test
> future versions Cliff might be providing - before they hit Tumbleweed.
> 
> The reverted version will be part of Snapshot 20200504

I have upgraded to 20200509 and I can also confirm that it works like before.
Thanks Dominique!
Comment 65 Ricardo Klein 2020-05-18 08:03:44 UTC
I can confirm that upgrading went well:
grep VERSION_ID /etc/os-release 
VERSION_ID="20200511"


Just one question, can we include the scenario with root FS using btrfs and /home encrypted and using XFS (and maybe ext4) on the tests we run?
That will probably prevent errors like this one in the future.
Comment 66 Cliff Zhao 2020-07-24 09:26:30 UTC
Hi Andy and all bug followers:
Could you please help to test the latest update of Plymouth? if there's no problem, we will release it.
Thank you in advance.
(https://build.opensuse.org/package/binaries/Base:System/plymouth/openSUSE_Factory)
Comment 67 Konstantin Voinov 2020-08-05 13:02:52 UTC
Hi,

Sadly no. I updated to that version and was unable to boot again.
Can I provide some debug info?
Comment 68 Cliff Zhao 2020-08-05 14:38:17 UTC
(In reply to Konstantin Voinov from comment #67)
> Hi,
> 
> Sadly no. I updated to that version and was unable to boot again.
> Can I provide some debug info?

It's strange. 
You are using a virtual-machine or a real workstation?
what do you mean by "unable to boot"? you can not login to desktop, or you can't get the TTY console?
please provide the debuginfo, supportlog and plymouth.log.
thank you so much for the feedback!
Comment 69 Konstantin Voinov 2020-08-06 12:35:03 UTC
> It's strange. 

Should I update all packages to this repo or just plymouth?

> You are using a virtual-machine or a real workstation?

This is real laptop with two ssd, one ssd is encrypted /home

> what do you mean by "unable to boot"? you can not login to desktop, or you
> can't get the TTY console?

Plymouth's animation rotates forever. When I press ESC I get console with these last lines:

[ OK ] Reached target Initrd Root File System.
       Starting Reload Configuration from the Real Boot...

(I can upload a photo if needed)
> please provide the debuginfo, supportlog and plymouth.log.

Hmmm, how can I get it in such situation?
Comment 70 Cliff Zhao 2020-08-09 14:37:43 UTC
(In reply to Konstantin Voinov from comment #69)
> > It's strange. 
> 
> Should I update all packages to this repo or just plymouth?
> 
Just plymouth is fine.

> > You are using a virtual-machine or a real workstation?
> 
> This is real laptop with two ssd, one ssd is encrypted /home
> 
Thanks, understand.

> > what do you mean by "unable to boot"? you can not login to desktop, or you
> > can't get the TTY console?
> 
> Plymouth's animation rotates forever. When I press ESC I get console with
> these last lines:
> 
> [ OK ] Reached target Initrd Root File System.
>        Starting Reload Configuration from the Real Boot...
> 
> (I can upload a photo if needed)
OK, by the way, you use gdm or sddm?

> > please provide the debuginfo, supportlog and plymouth.log.
> 
> Hmmm, how can I get it in such situation?
for supportlog:
1, $ zypper install supportutils
2, $ supportconfig
3, Upload the tar file from your system "/var/log/scc_...txz" to here as an
attachment.

for plymouthlog:
1, Edit /boot/grub2/grub.cfg, find the line of kernel paramter (which would looks like: "linux /boot/vmlinuz-5.3.18-lp152.19-default root=UUID=3a6c732b-0ae4-4950-853f-217548143fe2  splash=silent mitigations=auto quiet")
2, Add "plymouth:debug" in the tail of this line.
3, reboot.
4, Log into the system. You will find the log in "/var/log/plymouth-debug.log"
5, Upload the log as an attachment to this bug.

Thank you very much!
Comment 71 Cliff Zhao 2020-08-17 02:05:48 UTC
Hi Konstantin:
please see the latest comment. thanks!
Comment 72 Konstantin Voinov 2020-08-17 02:31:57 UTC
Sorry for delay. I use this laptop everyday, so not much time for experiments.
I cannot boot in this btrfs snapshot at all. All I can is the change boot option in grub directly during boot then try to get log from other working boot config.
Comment 73 Konstantin Voinov 2020-08-17 02:33:38 UTC
> OK, by the way, you use gdm or sddm?

SDDM
Comment 74 Cliff Zhao 2020-08-17 06:07:08 UTC
(In reply to Konstantin Voinov from comment #72)
> Sorry for delay. I use this laptop everyday, so not much time for
> experiments.
> I cannot boot in this btrfs snapshot at all. All I can is the change boot
> option in grub directly during boot then try to get log from other working
> boot config.

You can generate the support-log from the snapshot which works for you. and upload it first.
Comment 75 Cliff Zhao 2020-08-21 08:33:15 UTC
Hi Konstantin:
Are you still there?
Comment 76 Konstantin Voinov 2020-08-21 09:46:53 UTC
(In reply to Cliff Zhao from comment #75)
> Hi Konstantin:
> Are you still there?

being busy, i'll do tests and report in this weekend
Comment 77 Konstantin Voinov 2020-08-23 09:20:59 UTC
Created attachment 840945 [details]
supportconfig K. Voinov
Comment 78 Konstantin Voinov 2020-08-23 09:21:43 UTC
Created attachment 840946 [details]
plymouth dbg K. Voinov
Comment 79 Konstantin Voinov 2020-08-23 09:22:44 UTC
I've added files from healthy config. Will try to get plymouth log from broken.
Comment 80 Cliff Zhao 2020-09-24 11:55:07 UTC
(In reply to Konstantin Voinov from comment #79)
> I've added files from healthy config. Will try to get plymouth log from
> broken.

I am checking the support log and debug log these days, Thanks for the corrperate.
May I ask what will be the result when you run: $ plymouth-set-default-theme?
Comment 81 Konstantin Voinov 2020-09-29 23:38:47 UTC
(In reply to Cliff Zhao from comment #80)
> (In reply to Konstantin Voinov from comment #79)
> > I've added files from healthy config. Will try to get plymouth log from
> > broken.
> 
> I am checking the support log and debug log these days, Thanks for the
> corrperate.
> May I ask what will be the result when you run: $ plymouth-set-default-theme?

It means, changing the theme then upgrade plymouth?
Comment 82 Cliff Zhao 2020-10-07 15:45:45 UTC
(In reply to Konstantin Voinov from comment #81)
> (In reply to Cliff Zhao from comment #80)
> > (In reply to Konstantin Voinov from comment #79)
> > > I've added files from healthy config. Will try to get plymouth log from
> > > broken.
> > 
> > I am checking the support log and debug log these days, Thanks for the
> > corrperate.
> > May I ask what will be the result when you run: $ plymouth-set-default-theme?
> 
> It means, changing the theme then upgrade plymouth?

Without any paramenters, it will show the theme name which you are using.
Comment 83 Cliff Zhao 2020-10-07 15:56:41 UTC
and could you please have a try with https://api.opensuse.org/package/binaries/Base:System/plymouth/openSUSE_Factory? I only update the source with upstream.
and don't forget to use the latest edition of plymouth-branding, if not do so, plymouth will quit unexpectedly.
Thanks a lot!
Comment 84 Cliff Zhao 2020-10-16 02:28:04 UTC
Hi Konstantin:
Could you please try the update of plymouth in https://api.opensuse.org/package/binaries/Base:System/plymouth/openSUSE_Factory?
Comment 85 Cliff Zhao 2020-10-16 02:29:52 UTC
Hi Andy:
Could you please try the update of plymouth in https://api.opensuse.org/package/binaries/Base:System/plymouth/openSUSE_Factory?
Comment 86 Konstantin Voinov 2020-10-25 04:23:27 UTC
(In reply to Cliff Zhao from comment #84)
> Hi Konstantin:
> Could you please try the update of plymouth in
> https://api.opensuse.org/package/binaries/Base:System/plymouth/
> openSUSE_Factory?

Hi,
I've tried today and as i can see the plymouth is updated in main repo:

v  | plymouth | пакет                  | 0.9.5+git20200921+20778f2-307.22 | x86_64      | Base:System Factory Devel Project (openSUSE_Factory)
i+ | plymouth | пакет                  | 0.9.5+git20200921+20778f2-1.1    | x86_64      | openSUSE:Tumbleweed

So, i don't know should i change repo or is the bug resolved?
Comment 87 Hans-Peter Jansen 2020-10-25 11:22:13 UTC
Cliff, I'm using the latest B:S/plymouth on several systems and harvest bootsplash regressions (filed upstream) on 3 (very different) systems. In those cases, plymouthd crashes.

In home:frispete:Tumbleweed/plymouth, I've updated to latest GIT, but see no difference. An attempt to go back to some 0.9.5-2019 version (rebuilt) fails in similar ways, which points to some other issue. Might be glibc, dracut, kernel, whatever...

Want me to file another bugzilla here?
Comment 88 Cliff Zhao 2020-10-29 09:27:33 UTC
(In reply to Hans-Peter Jansen from comment #87)
> Cliff, I'm using the latest B:S/plymouth on several systems and harvest
> bootsplash regressions (filed upstream) on 3 (very different) systems. In
> those cases, plymouthd crashes.
> 
> In home:frispete:Tumbleweed/plymouth, I've updated to latest GIT, but see no
> difference. An attempt to go back to some 0.9.5-2019 version (rebuilt) fails
> in similar ways, which points to some other issue. Might be glibc, dracut,
> kernel, whatever...
> 
> Want me to file another bugzilla here?

Could you please check the branding package first? have you updated it already?
Comment 89 Cliff Zhao 2020-10-29 09:35:07 UTC
(In reply to Konstantin Voinov from comment #86)
> (In reply to Cliff Zhao from comment #84)
> > Hi Konstantin:
> > Could you please try the update of plymouth in
> > https://api.opensuse.org/package/binaries/Base:System/plymouth/
> > openSUSE_Factory?
> 
> Hi,
> I've tried today and as i can see the plymouth is updated in main repo:
> 
> v  | plymouth | пакет                  | 0.9.5+git20200921+20778f2-307.22 |
> x86_64      | Base:System Factory Devel Project (openSUSE_Factory)
> i+ | plymouth | пакет                  | 0.9.5+git20200921+20778f2-1.1    |
> x86_64      | openSUSE:Tumbleweed
> 
> So, i don't know should i change repo or is the bug resolved?

I have dropped many modifications to make the update with limited side effects.
Currently, Base:System and openSUSE:Tumbleweed are the same, you can use either repo.
Comment 90 Cliff Zhao 2020-10-29 09:41:31 UTC
(In reply to Hans-Peter Jansen from comment #87)
> Cliff, I'm using the latest B:S/plymouth on several systems and harvest
> bootsplash regressions (filed upstream) on 3 (very different) systems. In
> those cases, plymouthd crashes.
> 
> In home:frispete:Tumbleweed/plymouth, I've updated to latest GIT, but see no
> difference. An attempt to go back to some 0.9.5-2019 version (rebuilt) fails
> in similar ways, which points to some other issue. Might be glibc, dracut,
> kernel, whatever...
> 
> Want me to file another bugzilla here?

A little question: How do you update? with "zypper" or "rpm"?
Comment 91 Hans-Peter Jansen 2020-10-29 10:47:23 UTC
I always use zypper.

Meanwhile, the problem was disclosed. Since some time and up to plymouth GIT HEAD, plymouthd could crash, if DeviceTimeout is undefined (iow. 0). Defining 

[Daemon]
[...]
DeviceTimeout=5

solved it for *all* crashing systems. Upstream doesn't want to apply a simple fix to prevent a DeviceTimeout=0 setting, hence we might want to deal with this in %post or apply a local patch. Probably we just missed to install a properly adjusted plymouthd.defaults. But I haven't fully grokked the plymouthd.conf and plymouthd.defaults handling, yet, sorry.
Comment 92 Cliff Zhao 2020-10-29 11:57:14 UTC
(In reply to Hans-Peter Jansen from comment #91)
> I always use zypper.
> 
...
> 
> solved it for *all* crashing systems. Upstream doesn't want to apply a
> simple fix to prevent a DeviceTimeout=0 setting, hence we might want to deal
> with this in %post or apply a local patch. Probably we just missed to
> install a properly adjusted plymouthd.defaults. But I haven't fully grokked
> the plymouthd.conf and plymouthd.defaults handling, yet, sorry.

This has been discussed. option "DeviceTimeout" could not be pre-defined in the program because a lot of people install Linux in a server without display server. In this case the server will wait for an additional meaningless time when booting to OS. Only the administrator knows how many seconds is appropriate. and the recommended time is already been set in the config file. so it could not be modified.
But you said that "plymouthd could crash, if DeviceTimeout is undefined" because currently, Plymouth config file is migrated to branding package, I will confirm if it is correct. if there has any problem, I will update.
Thank you!
Comment 93 Cliff Zhao 2022-09-06 05:46:43 UTC
Close the bug as there's no other complainers.