User:NeddySeagoon/Root Inside LVM On Raid

From Gentoo Wiki
Jump to:navigation Jump to:search

Overview

There is nothing new in Root Inside LVM On Raid, so why another document?

Conventional wisdom says to use an an initrd to start the raid, then poke about inside the raid to bring up LVM so the kernel can actually see the root logical volume. That's well documented elsewhere.

Lets do it without an initrd.

Warning
Don't Try This at Home! - Its a wonderful example of just because you can do something does not mean that you should!

"In theory there is no difference between theory and practice - in practice there is"

This process is for users that have some experience of managing both raid sets and logical volumes.

Consider it like driving the Monte Carlo rally. You must have participated as a navigator before you can drive it. Accordingly, the instructions are somewhat abbreviated.

Fails

The kernel runs DM_INIT before autodetecting raid sets.

[    7.862511] device-mapper: table: 253:0: linear: Device lookup failed
[    7.881161] device-mapper: ioctl: error adding target to table
[    7.891245] Waiting 10 sec before mounting root device...
[   18.070621] md: Waiting for all devices to be available before autodetect
[   18.078804] md: If you don't use raid, use raid=noautodetect
[   18.086746] md: Autodetecting RAID arrays.
[   18.156238] md: autorun ...
[   18.166807] md: running: <sda5><sda4>
[   18.178665] md127: WARNING: sda4 appears to be on the same physical disk as sda5.
[   18.187634] True protection against single-disk failure might be compromised.

There is a rootdelay=10 in the gap and some USB messages that are not relevant.

Works

Root on RAID

Root in LVM

Partitioning

Use a partitioning tool that you are comfortable with. Parted and fdisk both work. Set the RAID partition type to Linux RAID. The kernel will use this as a flag for raid auto assembly.

Important
GPT and MSDOS partition tables use different numbers for the Linux RAID partition type.

Making The Raid Set

As there is no initrd, we will use kernel raid autoassembly.

That's conventional but raid --metadata=0.90 must be used.

Note
--metadata=0.90 is old and depreciated but still around.

Root on kernel auto assembled raid5 works on GPT and MSDOS.

Root on LVM without an initrd works.

Next, combine the two.

Making the Volume Group and Logical Volumes

As the kernel is going to poke around for vg-root, there are some constraints on how the physical volume is constructed. This proof of concept will use linear on top of raid. It will be a single /dev/mdX.

Keep that in mind. Everything else is usual LVM commands.

The Kernel

The kernel needs to support the raid level(s) in use, built in. The required device mapper options, built in. In particular, DM_INIT, as this allows the kernel to parse the kernel command line to know how to find root.

Kernel Command Line

The kernel won't read the LVM metadata to find out where root is, it must be added to the kernel command line, in the form of a table, which you can extract with dmsetup table.

root #dmsetup table
arm64-root: 0 25165824 linear 8:6 2048

Its possible to have the PV spread over several devices, in which case a table per device is required.

The LV name arm64-root
0 the first block of the LV
25165824 the number of 512B blocks in the LV
linear how the components of the LV go together
8:6 kernel major:minor device numbers (/dev/sda6) of the PV
2048 the offset (number of 512B blocks) to the start of the LV

dm-mod.create="arm64-root,,0,rw,0 25165824 linear /dev/sda6 2048" root=/dev/dm-0 ro net.ifnames=0 earlyprintk console=tty0 console=ttyS0,115200n8

Summary

This process is extremely fragile. So other that the novelty value of root in LVM on raid, is not recommended.

If the root filesystem is grown, the kernel command line needs to be updated.

Kernel device names are allocated on a first come, first served basis, so are not fixed.

In other words, while its possible, its not maintainable.