I hit the submit button a bit to early.

After enabling multipathd.service on the the spare blade, it also boots to emergency mode. Disabling multipathd.service and rebooting makes the system boot again. The same applies to the updated system.

I have no clue why multipathd prevents the system from assembling md2. In emergency mode I checked "multipath -ll" and "dmsetup ls --tree" and both do not have the disk devices used for md2 in use.

I'll attach the journalctl -b output both for a failed (multipathd enabled) boot as well as for a successfull (multipathd disabled) boot. Disabling multipathd is no option as the system is equipped with a FC LUN from our SAN.

Created attachment 626158 [details]
Output of journalctl -b

Output of journalctl -b for both a failed an successful boot on the upgraded and the spare host

the failed one has
kernel: device-mapper: multipath: version 1.7.0 loaded
kernel: device-mapper: multipath service-time: version 0.2.0 loaded
kernel: device-mapper: table: 253:0: multipath: error getting device
kernel: device-mapper: ioctl: error adding target to table
kernel: device-mapper: table: 253:0: multipath: error getting device
kernel: device-mapper: ioctl: error adding target to table

These messages do net seem to be the cause. When I disable the multipathd systemd unit and start it manually after a reboot, the messages are also shown:

Mär 10 17:16:34 testhost kernel: device-mapper: multipath: version 1.7.0 loaded
Mär 10 17:16:34 testhost kernel: device-mapper: multipath service-time: version 0.2.0 loaded
Mär 10 17:16:34 testhost kernel: device-mapper: table: 253:0: multipath: error getting device
Mär 10 17:16:34 testhost kernel: device-mapper: ioctl: error adding target to table
Mär 10 17:16:34 testhost kernel: device-mapper: table: 253:0: multipath: error getting device
Mär 10 17:16:34 testhost kernel: device-mapper: ioctl: error adding target to table

But the devices are avalable:
testhost:~ # multipath -ll
360050768018085377800000000000084 dm-0 IBM,2145
size=1000G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 1:0:6:0 sdd 8:48 active ready running
| `- 2:0:6:0 sdf 8:80 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 1:0:1:0 sdc 8:32 active ready running
  `- 2:0:0:0 sde 8:64 active ready running
testhost:~ # dmsetup ls --tree
360050768018085377800000000000084-part1 (253:1)
 └─360050768018085377800000000000084 (253:0)
    ├─ (8:64)
    ├─ (8:32)
    ├─ (8:80)
    └─ (8:48)


I can see the device-mapper lines an all systems (also 13.1), were multipathd is enabled, so they look harmless to me.

We have scheduled a multipath-tools and dracut update for 13.2 which might help here.

I've just installed the dracut update, but that didn't help. md2 is still not assambled when multipathd.service is enabled. I'm now waiting for the multipath-tools update.

I just wanted to ask when the scheduled multipath-tools update will arrive.

This issue prevents me from updating a lot of bare metal servers depending on multipath for over half a year now. The issue should be easy to reproduce: I just had to enable multipathd.service and md2 was missing.

Could you please again have a look or provide the updated multipath-tools package?

Can you try with the version from:

https://build.opensuse.org/package/show/home:morbidrsa:branches:openSUSE:13.2:Update/multipath-tools

It's the version used in  Leap and SLE12-SP1.

Dear Johannes,

I wasn't able to find a package to download with the provided link. "Download package" says "no data".

I've found http://download.opensuse.org/distribution/leap/42.1-Current/repo/oss/suse/x86_64/multipath-tools-0.5.0-48.3.x86_64.rpm but this one requires libdevmapper.so.1.02(DM_1_02_97)(64bit). So I've also fetched http://download.opensuse.org/distribution/leap/42.1-Current/repo/oss/suse/x86_64/device-mapper-1.02.97-61.4.x86_64.rpm. I'll try this out and report back wether this fixes the issue.

With both updates applied, system still boots to emergency mode. I'll try changing from multipathd.service to multipathd.socket.

Forgot to activate the download repo sorry.

But yes, the Leap 42.1 is the same.

I'll check if there's any difference to upstream

If you were to enable multipath on a non-multipathed root system you need to blacklist the root filesystem.

Also, due to the timing involved multipath might claim the device for MD, so if MD references the devices as raw block devices (ie using /dev/sdX) it won't be able to start.

(In reply to Hannes Reinecke from comment #12)
> If you were to enable multipath on a non-multipathed root system you need to
> blacklist the root filesystem.
I don't know why I should have to do that. OpenSUSE 13.1 works fine with Swap, / and /srv each on RAID1 md devices and additional multipathed FC LUN's. OpenSUSE 13.2 also assembles SWAP and /, but misses /srv if multipathd.service is enabled.


> Also, due to the timing involved multipath might claim the device for MD, so
> if MD references the devices as raw block devices (ie using /dev/sdX) it
> won't be able to start.
If already written that I have checked that point. While in emergency mode, dmsetup does not report /dev/sd[a,b]3 to be claimed (see #c4). I can manually assemble the missing array and continue booting.

The issue does not exist in 42.1. We will skip 13.2. You can close this issue.

Closing as it's fixed on 42.1