ZFS/root

Root on ZFS
This is set of tips why and how to have ZFS on root, making your Gentoo/Linux life much easier.

'''This is not step-by-step guide. Also it is considered for experience users, and users with some ZFS experience.'''

Initramfs & BE & Grub
Many of the scripts mentioned in this text come from my github repository. See https://github.com/robertek/root-scripts. Please be sure to check these scripts before you start using them, there are many hardcoded paths and you would probably need to change them (mainly the ones that are not beadm). Feel free to update these script to more generic way and share with me.

I have cloned this in /root/bin.

Initramfs
Since zfs module is already compiled in the kernel, initramfs is here to do only one job and it is import the root pool and switch to correct dataset.

The initramfs than needs to be only very minimal. So I have developed easy script for genrating such initramfs.

composing initramfs in /tmp/initramfs.AmH backing up /boot/initramfs-0.7.1 creating new /boot/initramfs-0.7.1
 * 1) make_initramfs

What does it do. It will collect all the zfs userspace tools and its libs, busybox and some other extras. Adds this to some temporary directory. Create init file from template and than pack this to cpio archive.

The most important is the init script. Check the https://raw.githubusercontent.com/robertek/root-scripts/master/init.tpl what it is doing.

The main work-flow is this: import pool -> mount root -> switch_root.

You may have defined root= or rpool= variables on kernel command line. If none is defined than "rpool" will be imported and what is in its bootfs variable will be root.

beadm
Under my scripts you may find slightly updated "beadm" command.

This command is the way how to manage boot environments (BE). It has similar functionality as the original beadm command in Solaris.

usage: beadm activate  beadm create [-e nonActiveBe | -e beName@snapshot]  beadm create  beadm destroy [-F]  beadm list [-a] [-s] [-D] [-H] beadm rename   beadm mount  [mountpoint] beadm { umount | unmount } [-f] 
 * 1) beadm

To have this tool working correctly you need to have correct layout of zfs root datasets. It needs to be in format "pool/somedataset/root" and the pool needs to have the bootfs set.

Example:

NAME  PROPERTY  VALUE                       SOURCE rpool bootfs    rpool/GENTOO/root-20171004  local
 * 1) zpool get bootfs rpool
 * 1) zpool set bootfs=rpool/GENTOO/root-20171004 rpool

If this is set correctly you will have something like this in the beadm output:

BE           Active Mountpoint  Space Created root         -      /            0.0K root-20170418 -     -          855.0M 2017-04-18 12:01 root-20170509 -     -          351.7M 2017-05-09 15:56 root-20170530 -     -          808.3M 2017-05-30 12:33 root-20170925 -     -          709.0M 2017-09-26 17:39 root-20171004 NR    /           14.2G 2017-10-04 14:55
 * 1) beadm list

Using beadm activate you can activate new BE after reboot. Other options are self explanatory.

Grub
I have found out updating grub the old way is easier and more convenient when I want some non standard expectation.

I have generated static grub2 image with all the needed modules loaded, which than can be put anywhere what efi boots.


 * 1) grub-mkimage -O x86_64-efi -o bootx64.efi -p "" part_gpt fat normal boot configfile linux gzio xzio gfxterm
 * 2) mount /boot
 * 3) mkdir -p /boot/EFI/boot
 * 4) mv bootx64.efi /boot/EFI/boot

And sample of my config. Nothing special, rest of the config file are only defined another kernels or BE. Note the second menu entry, this define boot to specific BE.

loadfont /EFI/boot/unicode.pf2 set gfxmode="auto" terminal_output gfxterm
 * 1) head -n 25 /boot/EFI/boot/grub.cfg
 * 2) variables
 * 1) variables

set default="0" set timeout="5"

set root=(hd0,gpt1)

set splash_par="splash=verbose,theme:natural_gentoo console=tty1" set crash_par="crashkernel=64M"

menuentry 'Gentoo Linux systemd 4.9.52 zfs-bootfs' --class gnu-linux --class os { linux /vmlinuz-4.9.52+ $splash_par $crash_par quiet initrd /initramfs-systemd-0.7.1 } menuentry 'Gentoo Linux systemd 4.9.52 20171101' --class gnu-linux --class os { linux /vmlinuz-4.9.52+ oot=rpool/GENTOO/root-20171101 $splash_par $crash_par quiet initrd /initramfs-systemd-0.7.1 }
 * 1) menu
 * 1) menu

System update
In the old days system update (and specific Gentoo linux) took a long time, it improved a bit on latest hw, but not much. It is time you keep your computer do its work and on many systems (Windows, Linuxes) you are not allowed to work at all. On Gentoo if you run update in the background you may expect some inconveniences during updated, or if it fails you may have screwed system.

With BE you can let the history behind. The simple idea (something that Solaris do for years) is to clone current root and update on the clone. After the clone is updated successfully you reboot to the clone. If anything goes wrong you still has working system in the previous BE (until you delete it). This is something that is not ZFS specific (you may implement this on top of LVM or btrfs), but I find this the easiest and most convenient way.

Prerequisites
It is good to keep the portage tree somehow local to the BE. If you return to the older BE you will have consistent portage tree with the contents of BE.

I have found the easiest way to keep portage tree under squashfs. Nowadays it is much easier, since portage tree is already delivered also as squashfs images.

I have sqfs image in /var/lib/portage/gentoo/gentoo.xz.sqfs mounted as /var/lib/portage/gentoo/gentoo.

/var/lib/portage/gentoo.xz.sqfs /var/lib/portage/gentoo squashfs defaults 0 0 [DEFAULT] main-repo = gentoo
 * 1) grep gentoo /etc/fstab
 * 1) cat /etc/portage/repos.conf/gentoo

[gentoo] location = /var/lib/portage/gentoo

But you may left the portage tree unpacked normally under /usr/portage and have it on the root dataset.

On the other side it is good to have something shared between BE. Perfect example is distfiles. I have special dataset for distfiles and point the portage to this.

rpool/GENTOO_data/distfiles       2.43G  44.6G  2.43G  /var/lib/portage/distfiles DISTDIR="/var/lib/portage/distfiles"
 * 1) zfs list | grep distfiles
 * 1) grep DISTDIR /etc/portage/make.conf

Chroot update
Simple use-case how to update using boot environment. In the chroot do what you are used to do when updating (other than syncing portage tree).

(chroot) # emerge -avuDN @world (chroot) # ..... (chroot) # exit
 * 1) beadm create root-updated
 * 2) beadm mount root-updated /mnt
 * 3) wget http://distfiles.gentoo.org/snapshots/squashfs/gentoo-current.xz.sqfs
 * 4) mv gentoo-current.xz.sqfs /mnt/var/lib/portage/gentoo/gentoo.xz.sqfs
 * 5) chroot_start /mnt
 * 6) chroot /mnt /bin/bash --login
 * 7) PS1="(chroot) $PS1"
 * 1) chroot_stop /mnt
 * 2) beadm umount root-updated
 * 3) beadm activate root-updated
 * 4) reboot

As was noted in GRUB section you may add new GRUB entry to reflect the old BE. Otherwise you will always boot to the active BE.

Something like: menuentry 'Gentoo Linux 4.9.30 root-updated' --class gnu-linux --class os { linux /vmlinuz-4.9.30+ root=rpool/GENTOO/root-updated initrd /initramfs } EOF
 * 1) cat <>/boot/EFI/boot/grub.cfg