GRUB2/AdvancedStorage

From Gentoo Wiki
< GRUB2
Jump to: navigation, search
This article is a stub. You can help by expanding it.

This section is based on converting a non-UEFI, MBR partition based system to boot from a GPT RAID enabled disk. It is currently incomplete, and partially edited from the previous version.

Note
In BIOS - MBR disks, leave a space at the start of the disk for GRUB2 to be installed. The conventional 32 kilobytes (63 sectors of 512 bytes) is not enough for GRUB2 when RAID is enabled, the space should be at least 1MB (the first primary partition should start on or after sector 2048). Most disk partition utilities automatically perform this operation.


On BIOS - GPT disks, explicit allocation of space for the GRUB2 BIOS boot partition must be provided. Create a disk at least 1 MB in size, type EF02 (BIOS boot partition). (This will not be used in a UEFI system, so you could use the same space for the EFI partition (type EF00) when swapping the disk between systems)
For the following three code blocks, first observe the output of GRUB2 installing on the first few sectors (only 63 sectors of 512 bytes) of a disk. Installing with limited sectors will cripple GRUB2's operation making it incapable of supporting complex disk setups such as RAID and ultimately unable to boot the system. GRUB2's output on the second block (a disk with 2048 sectors free before the first primary partition) displays adequate output. The output from GRUB2 in the third block shows the EF02 partition (without which GRUB2 will refuse to install).

CODE Code block 1
Disk /dev/sda: 160.0 GB, 160000000000 bytes, 312500000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000cfe45
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    48821534    24410736   8e  Linux LVM
CODE Code block 2
Disk /dev/sda: 320.1 GB, 320072933376 bytes, 625142448 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009d633
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      499711      248832   8e  Linux LVM
CODE Code block 3
GPT fdisk (gdisk) version 0.8.8
 
Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present
 
Found valid GPT with protective MBR; using GPT.
Disk /dev/sdb: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 47915407-BA7E-4869-8D3E-3CB44F5FDA12
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 1050590 sectors (513.0 MiB)
 
Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   EF02  BIOS boot partition
   2         1312768        68421631   32.0 GiB    8200  Linux swap
   3        68683776       270010367   96.0 GiB    8300  root
   4       270272512       471599103   96.0 GiB    8300  var
   5       471861248      5860533134   2.5 TiB     8300  home

Booting from LVM logical volumes

GRUB2 supports booting from an LVM partition however the device-mapper USE flag must be set in order to activate this feature.

FILE /etc/portage/package.useEnabling the device-mapper USE flag for GRUB2
sys-boot/grub:2 device-mapper

If GRUB2 is currently installed, re-emerge it using the --newuse option:

root #emerge --ask --newuse sys-boot/grub:2

Next tell GRUB2 to pre-load the lvm module:

FILE /etc/default/grubPreloading LVM module
GRUB_PRELOAD_MODULES=lvm

If you get an 'invalid root device' when you boot and are using an initramfs (especially the one made by genkernel) you need to pass dolvm (for both lvm1 and lvm2) to the kernel:

FILE /etc/default/grubAdding default kernel parameters
GRUB_CMDLINE_LINUX_DEFAULT="dolvm"

Finally (re)generate a GRUB2 grub.cfg file using the grub2-mkconfig command.

Booting from RAID array

In this section, we assume a system has three hard drives. /dev/sda is the original MBR Windows Vista disk, untouched apart from installing GRUB2 onto it. /dev/sdb and /dev/sdc are configured identically.

CODE
GPT fdisk (gdisk) version 0.8.10
 
Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present
 
Found valid GPT with protective MBR; using GPT.
Disk /dev/sdc: 1953525168 sectors, 931.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): CB232664-F98D-45C0-8111-2AC526D5EA61
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 509918 sectors (249.0 MiB)
 
Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048        67110911   32.0 GiB    8200  Linux swap
   2        67110912        67127295   8.0 MiB     EF02  BIOS boot partition
   3        67373056       402917375   160.0 GiB   8300  Linux filesystem
   4       403179520      1953525134   739.3 GiB   8300  Linux filesystem

/dev/sdb3 and /dev/sdc3 are raided together to make the / partition.

/dev/sdb4 and /dev/sdc4 are raided together to make the /home partition.

FILE /boot/grub/grub.cfg
menuentry 'Gentoo GNU/Linux, with Linux x86_64-3.14.14-gentoo' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-x86_64-3.14.14-gentoo-advanced-8bb0e52c-d524-4af8-ac08-f3c9238c6040' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_msdos
	insmod part_gpt
	insmod diskfilter
	insmod mdraid1x
	insmod ext2
	set root='mduuid/660afb13150e817a0cdd36476d5b2c51'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint='mduuid/660afb13150e817a0cdd36476d5b2c51'  8bb0e52c-d524-4af8-ac08-f3c9238c6040
	else
	  search --no-floppy --fs-uuid --set=root 8bb0e52c-d524-4af8-ac08-f3c9238c6040
	fi
	echo	'Loading Linux x86_64-3.14.14-gentoo ...'
	linux	/boot/kernel-3.14.14-gentoo domdadm root=UUID=8bb0e52c-d524-4af8-ac08-f3c9238c6040 ro  
	echo	'Loading initial ramdisk ...'
	initrd	/boot/initramfs-genkernel-x86_64-3.14.14-gentoo
}
 
menuentry 'Gentoo GNU/Linux mirror 2' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-8bb0e52c-d524-4af8-ac08-f3c9238c6040' {
        load_video
        insmod gzio
        insmod part_msdos
        insmod part_gpt
	insmod mdraid1x
        insmod ext2
        set root=(mduuid/660afb13:150e817a:0cdd3647:6d5b2c51)
        if [ x$feature_platform_search_hint = xy ]; then
	          search --no-floppy --fs-uuid --set=root --hint-bios=hd1,msdos2 --hint-efi=hd1,msdos2 --hint-baremetal=ahci1,msdos2 --hint='hd1,msdos2'  8bb0e52c-d524-4af8-ac08-f3c9238c6040
          else
		  search --no-floppy --fs-uuid --set=root 660afb13:150e817a:0cdd3647:6d5b2c51
        fi
	echo    'Loading Linux 3.14.14-gentoo ...'
	linux   /boot/kernel-3.14.14-gentoo domdadm root=/dev/disk/by-id/md-uuid-660afb13:150e817a:0cdd3647:6d5b2c51 ro
	echo    'Loading initial ramdisk ...'
	initrd  /boot/initramfs-genkernel-x86_64-3.14.14-gentoo
}

insmod part_gpt (or insmod part_msdos) is needed otherwise GRUB2 will not be able to read the partition table.

insmod mdraid1x is required for RAID v1.1 or higher. If using RAID v0.9 or v1.0 the RAID module might not be needed (mdraid09 for v0.9) because the RAID information is stored at the end of the partition. Any references to insmod raid are obsolete.

insmod ext2 is required for ext partitions. For a other file systems substitute the appropriate module.

set root='mduuid/660afb13150e817a0cdd36476d5b2c51' tells GRUB2 what root device Linux will be using.

At this point, GRUB2 is ready to hand over control to Linux. Any errors up to this point (missing modules, missing boot partition, etc.) are problems with GRUB2 not the Linux kernel. Onward from this point, it is safe to narrow errors down to Linux.

Linux is unable to assemble RAID assemblies in the kernel. It must call out to user-space tools; if the root partition is a RAID set, an initramfs must be used.

Warning
Do not use genkernel utility to generate a kernel when using a RAID set for the root partition.

Make sure the kernel is configured correctly. Especially make sure that CONFIG_FHANDLE variable is set (this is why genkernel must not be used - genkernel will reset the value of CONFIG_FHANDLE variable and reap havoc upon the RAID set).

Compile the kernel manually by running the following commands as root:

root #cd /usr/src/linux
root #make
root #make modules_install
root #make install

At this point it is probably a good idea to emerge udev - and make sure all warnings are fixed!

Use genkernel utility to generate an initramfs:

root #genkernel --mdadm --install initramfs
Note
The --mdadm option is needed to make the initramfs RAID-aware.

The linux and initrd lines in /boot/grub/grub.cfg should now load the Linux kernel. Add the domdadm parameter to the Linux kernel command line in GRUB2. Without it the RAID set will not assemble and the Linux kernel will not find the root device.

Note
The second boot option contains a deliberate error. Linux will be unable to find root device by itself and will drop into the GRUB2 shell. This is needed.

Discussion on the Linux RAID mailing list implies that the root partition is not being passed correctly from the initramfs to the running system.

Note also that it is confusing whether to use UUIDs or mduuids. I think you need to pass mduuids to grub, and UUIDs to linux. Suck it and see.

The first boot option correctly assembles the root RAID set at boot time, and boots successfully. Any attempt to access the root device from user space will fail as if the RAID set does not exist. This breaks GRUB2, and making the mistake of trying to run grub2-install on top of these errors will break GRUB boot altogether. Make sure Rescue CD and a copy of the Gentoo handbook are easily available before attempting to use a RAID set on the root disk.

The second option drops into a GRUB2 shell. Running ls /dev/md* will show that initrd has found the RAID set devices, and giving it the device, e.g. /dev/md127, will boot into a fully working system with userspace access to the boot device.

Put these into a /etc/grub.d script at the first available opportunity.

Further documentation on this subject, along with booting from encrypted physical volumes, is close to null.

Any further documentation from a user that has successfully booted a system from a RAID array welcome to complete add a RAID boot section to this article.

Note
Booting from RAID is only relevant for BIOS [boot]. EFI users shall roll out their custom initramfs or use an initramfs package available out there. Depending on your setup, you may use either a BIOS - MBR or a BIOS - GPT combination. Here is an example to a BIOS - GPT setup with /boot and root / in a RAID-1 configuration [1] and Xen on RAID-5.
Note
The grub2-mkdevicemap command and UUID re-setting have been removed and are no longer distributed with GRUB2. The UUID problem GRUB2 once had no longer appears to be an issue. Installing to two devices in a RAID-1 configuration seemed to work with no problems; This is the command that was successful grub2-install /dev/sda && grub2-install /dev/sdb (after creating the expected EF02 BIOS boot partition).
Important
If this is a new installation of Gentoo, be sure to emerge sys-fs/mdadm in the chrooted environment before attempting to install GRUB2 as described in the note above, otherwise it will fail with "Path /boot/grub is not readable by GRUB on boot. Installation is impossible. Aborting." After installing the mdadm tool, GRUB2 can correctly read the RAID devices. The grub2-install /dev/sda and grub2-install /dev/sdb should work as expected on a RAID-1 array comprised of /dev/sda2 and /dev/sdb2 for boot.

Booting from a RAID array is very similar to booting from a LVM Logical Volume aside from RAID specific terminology and syntax of RAID partitioned volume.

CODE
insmod raid
#    and load the related `mdraid' module `mdraid09' for RAID arrays with version 0.9 metadata, and `mdraid1x' for arrays with version 1.x metadata.
     insmod mdraid09
     set root=(md0p1)
#    or the following for an unpartitioned RAID array
     set root=(md0)
Note
A bleeding edge RAID user might emerge the live ebuild to get the latest bit of fixes and updates.
Note
A NVidia (dm_nv) specific module for RAID 3/5, so something like `insmod dm_nv' is necessary for NVidia related hardware.

This should work with a simple software RAID setup. However, the author has no idea or what command, if any exit at the moment of writing, to use to assemble an array such as mdadm --assemble --scan /dev/md0 that an initramfs could take care of.

Booting from LUKS

Tell GRUB2 to look for cryptodisks:

root #echo "GRUB_CRYPTODISK_ENABLE=y" >> /etc/default/grub

(Re)generate the grub.cfg with the grub2-mkconfig utility.

Important
In GRUB 2.02 and later, this parameter was renamed to GRUB_ENABLE_CRYPTODISK