Bug 1148477

Summary: [Build 20190824] installation fails on setting uuid of btrfs
Product: [openSUSE] openSUSE Tumbleweed Reporter: Rodion Iafarov <riafarov>
Component: AutoYaSTAssignee: YaST Team <yast-internal>
Status: VERIFIED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Major    
Priority: P5 - None CC: ancor, aschnell, dgonzalez, emiura, jeriveramoya
Version: Current   
Target Milestone: ---   
Hardware: Other   
OS: Other   
URL: https://trello.com/c/XibWb5sb
See Also: http://bugzilla.suse.com/show_bug.cgi?id=1152535
Whiteboard: https://openqa.opensuse.org/tests/1018539/modules/installation/steps/5
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: YaST logs
autoinst.xml used in testing (created from a fresh sle15sp1 install)

Description Rodion Iafarov 2019-08-28 07:03:36 UTC
Created attachment 815950 [details]
YaST logs

## Observation

When we clone system and reinstall it with default partitioning (btrfs for root), we get this error.
We have same issue on SLE 15 SP2.

Relevant part of the logs:
2019-08-28 02:55:11 <1> install(3714) [libstorage] BlkDeviceImpl.cc:914 name:/dev/vda2 exists:true
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:69 constructor SystemCmd("/sbin/mkfs.btrf
s --force '/dev/vda2'")
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:188 SystemCmd Executing:"/sbin/mkfs.btrfs
 --force '/dev/vda2'"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:189 timestamp [120.475764], 2019-08-28 06
:55:11 GMT, 2019-08-28 02:55:11 EDT
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 1 "btrfs-progs v5.2.1 "
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 2 "See http://btrfs.wiki.
kernel.org for more information."
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 3 ""
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 4 "Label:              (n
ull)"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 5 "UUID:               52
25f348-086d-4b5c-a96d-dc265f911e37"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 6 "Node size:          16
384"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 7 "Sector size:        40
96"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 8 "Filesystem size:    37
.99GiB"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 9 "Block group profiles:"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 10 "  Data:             s
ingle            8.00MiB"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 11 "  Metadata:         D
UP             256.00MiB"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 12 "  System:           D
UP               8.00MiB"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 13 "SSD detected:       n
o"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 14 "Incompat features:  e
xtref, skinny-metadata"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 15 "Number of devices:  1
"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 16 "Devices:"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 17 "   ID        SIZE  PATH"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 18 "    1    37.99GiB  /dev/vda2"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:663 Adding Line 19 ""
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:629 pid:5495 added lines:19 stderr:false
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:419 stopwatch 0.083570s for "/sbin/mkfs.btrfs --force '/dev/vda2'"
2019-08-28 02:55:11 <1> install(3714) [libstorage] SystemCmd.cc:439 system() Returns:0
2019-08-28 02:55:11 <1> install(3714) [libstorage] ActiongraphImpl.cc:633 Commit Action "Setting UUID of btrfs on /dev/vda2 (37.99 GiB) to 5225f348-086d-4b5c-a96d-dc265f911e37" [sid:130, last]
2019-08-28 02:55:11 <3> install(3714) [libstorage] BlkFilesystemImpl.cc:784 THROW:       stub do_set_uuid called
2019-08-28 02:55:11 <3> install(3714) [libstorage] CallbacksImpl.cc:62 CAUGHT:   stub do_set_uuid called
2019-08-28 02:55:11 <1> install(3714) [Ruby] callbacks/libstorage_callback.rb:73 libstorage-ng reported an error, asking the user whether to continue
2019-08-28 02:55:11 <1> install(3714) [Ruby] callbacks/libstorage_callback.rb:74 Error details. Message: Setting UUID of btrfs on /dev/vda2 (37.99 GiB) to 5225f348-086d-4b5c-a96d-dc265f911e37. What: stub do_set_uuid called.
2019-08-28 02:55:11 <1> install(3714) [Ruby] callbacks/libstorage_callback.rb:89 Setting UUID of btrfs on /dev/vda2 (37.99 GiB) to 5225f348-086d-4b5c-a96d-dc265f911e37

Unexpected situation found in the system.

Click below to see more details (English only).

Continue despite the error?


openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-autoyast_reinstall_gnome@64bit fails in
[installation](https://openqa.opensuse.org/tests/1018539/modules/installation/steps/5)

## Test suite description



## Reproducible

Fails since (at least) Build [20190731](https://openqa.opensuse.org/tests/997491)


## Expected result

Last good: [20190730](https://openqa.opensuse.org/tests/996251) (or more recent)


## Further details

Always latest result in this scenario: [latest](https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=opensuse&flavor=DVD&machine=64bit&test=autoyast_reinstall_gnome&version=Tumbleweed)
Comment 2 Arvin Schnell 2019-08-28 07:35:27 UTC
libstorage-ng supports setting the UUID only for swap. FMPOV cloning
should never try to clone any UUIDs since it would make UUIDs not unique.
Comment 3 Ancor Gonzalez Sosa 2019-08-28 07:44:48 UTC
(In reply to Arvin Schnell from comment #2)
> libstorage-ng supports setting the UUID only for swap. FMPOV cloning
> should never try to clone any UUIDs since it would make UUIDs not unique.

Well, setting the UUID via AutoYaST was possible in SLE-12 (and likely 11), as far as I can see in the documentation. So no matter whether it's a good practice or not, it should keep working in SLE-15. We can of course enhance the documentation to warm about the risks, but we must not fail.
Comment 4 Arvin Schnell 2019-08-28 07:59:30 UTC
Also in libstorage I can only find code to set the UUID for swap.
Comment 5 Lukas Ocilka 2019-08-28 08:22:19 UTC
There are actually two use-cases:

1. Cloning

   As already stated UUID is, well, unique, so the default behavior should
   not be to export configuration using UUID. While comparing the probability,
   it's not that likely that the same UUID will be present. Of course, it may
   happen. And for that reason ...

2. AutoInstallation

   All ways of (supported) defining a disk / partition should be available
   also for AutoYaST. In other words, Import must be able to accept UUID.
   For instance, rules/classes could be used to identify the hardware and
   thus the customer is actually able to know which UUID is there. It's even
   easier in VMs.
Comment 7 Rodion Iafarov 2019-09-20 10:31:28 UTC
Bug reached SLE 15 SP2:
https://openqa.suse.de/tests/3381986#
Comment 8 José Iván López González 2019-10-01 11:35:01 UTC
*** Bug 1152535 has been marked as a duplicate of this bug. ***
Comment 9 Ancor Gonzalez Sosa 2019-10-03 16:33:18 UTC
Looking at the full picture, there are several problems here.

1) The AutoYaST documentation about the uuid attribute is confusing, not to say plain wrong.

2) Cloning the system should never include the uuid field in the <partition> sections (since that value is only useful to reuse existing partitions). This is only broken in TW and the upcoming 15.2. In 15.1 the uuid field is not exported to the profile.

3) The process should not fail if the profile includes the uuid field for a new device (i.e. a device that is not being reused). That's what the bug report was about originally. This was already broken in 15.1 (it also fails if a surplus uuid field is included), but it has not been observed before because (2) works ok in 15.1, so we were not autogenerating wrong profiles.

4) While checking this, we found out that reusing devices by UUID (the only thing for which the uuid field is useful) is not working. That was also already broken in 15.1.

I'm already working in fixing the four problems in all the affected branches.
Comment 10 Ancor Gonzalez Sosa 2019-10-03 17:12:26 UTC
(In reply to Ancor Gonzalez Sosa from comment #9)
> Looking at the full picture, there are several problems here.

Fixing (1):
https://github.com/SUSE/doc-sle/pull/483

Fixing (3) and (4) in 15.1:
https://github.com/yast/yast-storage-ng/pull/976

More to come.
Comment 11 Ancor Gonzalez Sosa 2019-10-04 10:00:56 UTC
(In reply to Ancor Gonzalez Sosa from comment #10)
> Fixing (3) and (4) in 15.1:
> https://github.com/yast/yast-storage-ng/pull/976

Corresponding maintenance request:
https://build.suse.de/request/show/202407
Comment 12 Ancor Gonzalez Sosa 2019-10-04 11:17:29 UTC
Fixing (2), (3) and (4) in the master branch (that is, 15.2 and TW):
https://github.com/yast/yast-storage-ng/pull/977
Comment 13 Ancor Gonzalez Sosa 2019-10-04 14:07:03 UTC
(In reply to Ancor Gonzalez Sosa from comment #12)
> Fixing (2), (3) and (4) in the master branch (that is, 15.2 and TW):
> https://github.com/yast/yast-storage-ng/pull/977

Corresponding SR to TW:
https://build.opensuse.org/request/show/735289

Corresponding SR to SLE-15-SP2 (and Leap 15.2):
https://build.suse.de/request/show/202431

With that, we can consider the bug as fixed (for all images including yast2-storage-ng >= 4.2.46).
Comment 14 Ancor Gonzalez Sosa 2019-10-04 14:25:02 UTC
(In reply to Ancor Gonzalez Sosa from comment #9)

Just some final notes for extra fun. That's what I managed to find via digital archeology. Maybe I'm wrong at some point.

> 1) The AutoYaST documentation about the uuid attribute is confusing, not to
> say plain wrong.

The uuid attribute was added to the documentation (with a misleading description) many years ago (6 at least). But as far as I was able to discover, it was never part of the AutoYaST schema definition and was never used for anything in the AutoYaST code. Neither for setting the UUID of the created device or for searching for devices to reuse.

So, for years, the attribute only existed in the documentation!

> 4) While checking this, we found out that reusing devices by UUID (the only
> thing for which the uuid field is useful) is not working. That was also
> already broken in 15.1.

As said before, reusing by UUID was actually never implemented before (I just did it for the first time in my mentioned requests). The first reference to the possibility of using the UUID to indicate the device to reuse was introduced in the documentation in Dec 2017 (https://github.com/SUSE/doc-sle/commit/6e1294c1e1ad3ffb4fc270138c1d7bf9a0133988).

And after talking to Imobach I just found it was kind of an accident. Likely he meant "label" instead. :-)

Anyway, since we introduced that in the documentation by mistake... now we support it. And at least the attribute "uuid" is used for something now, after years of being in the documentation for no reason. :-)
Comment 16 Rodion Iafarov 2019-10-23 10:50:12 UTC
Fixed in 20191009, thanks for the patch!
Comment 18 Ednilson Miura 2019-10-24 19:30:54 UTC
Hello. Would you have an example autoinst.xml for this test? I was using the attached autoinst.xml, but it didn't succeeded on keeping the uuid after the installation (no messages complaining about the uuid tag, though).\
I have loaded the updated yast2-storage-ng and yast2-installer using a dud.
Comment 19 Ednilson Miura 2019-10-24 19:31:32 UTC
Created attachment 822445 [details]
autoinst.xml used in testing (created from a fresh sle15sp1 install)
Comment 20 Ancor Gonzalez Sosa 2019-10-25 08:25:57 UTC
(In reply to Ednilson Miura from comment #18)
> Hello. Would you have an example autoinst.xml for this test? I was using the
> [...] but it didn't succeeded on keeping the uuid after the
> installation (no messages complaining about the uuid tag, though).

That's because cloning the UUID has never been a real possibility. That defeats the purpose of the UUID (being unique).

As explained in previous comments, the presence of the uuid field in the AutoYaST profile is kind of an historical accident and it has never worked to enforce the UUID in the final system.

After the latest changes, the uuid field can be used to look for a device in order to reuse it. But that's it.
Comment 21 Swamp Workflow Management 2019-12-17 17:47:07 UTC
SUSE-RU-2019:3322-1: An update that has 8 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1130988,1136463,1139808,1141006,1148477,1149011,1151075,1152535
CVE References: 
Sources used:
SUSE Linux Enterprise Module for Basesystem 15-SP1 (src):    yast2-installation-4.1.47-3.6.18, yast2-storage-ng-4.1.90-3.16.11
SUSE Linux Enterprise Installer 15-SP1 (src):    yast2-installation-4.1.47-3.6.18, yast2-storage-ng-4.1.90-3.16.11

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 22 Swamp Workflow Management 2019-12-22 23:12:37 UTC
openSUSE-RU-2019:2705-1: An update that has 8 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1130988,1136463,1139808,1141006,1148477,1149011,1151075,1152535
CVE References: 
Sources used:
openSUSE Leap 15.1 (src):    yast2-installation-4.1.47-lp151.2.6.1, yast2-storage-ng-4.1.90-lp151.2.9.1