User:Ali3nx/Installing Gentoo Linux EFISTUB On ZFS

Initial notes
This guide is currently an incomplete work in progress. This guide when finished intends to detail installing Gentoo Linux on zfs without using grub bootloader

Install Gentoo Linux on OpenZFS using EFIStub Boot
Author: Michael Crawford (ali3nx) Contact: mcrawford@eliteitminds.com

Preface
This guide will show you how to install Gentoo Linux on AMD64 with:

* UEFI-GPT (EFI System Partition) - This will be on a FAT32 unencrypted partition as per UEFI Spec. * /, /var, /home, on segregated ZFS datasets * swap on regular partition * OpenZFS 0.8.3 * efistub boot without Grub * genkernel initramfs * systemd * Gentoo Stable (amd64)

Download the System Rescue CD + ZFS ISO
You will need to download Fearedbliss modified System Rescue CD that includes ZFS from one of the mirrors on the front page.

Linux
We will be creating a UEFI Bootable USB since this guide will be showing you how to install Gentoo Linux on ZFS with UEFI Enabled.

For the following commands, we will assume that your USB is /dev/sdg.

Create the FAT32 Filesystem on the USB
We will now create the FAT32 filesystem on the USB. This needs to be FAT32 since this is the filesystem used in the UEFI Specification. The label we will use for this partition will be in the following format SYSRCDXYZ, where XYZ is the version number of the System Rescue CD you downloaded.

For example, if you are using System Rescue CD 6.0.7, the label will be SYSRCD607.

Copy files over from ISO to USB
And that's it! You now have a Bootable UEFI USB.

Windows
Balena Etcher is the USB Utility I recommend when on Windows. You can Download Balena Etcher here.


 * 1) Start Balena Etcher
 * 2) Select your USB Device from the Device drop down.
 * 3) Select your ISO by clicking SELECT.
 * 4) Click START.

This should be all that's necessary to have a Bootable UEFI USB.

Assumptions

 * Only installing Gentoo on one drive called /dev/sda (or /dev/nvme0n1, etc)
 * Fearedbliss System Rescue CD + ZFS iso is being used.
 * genkernel is being used as your initramfs.
 * gentoo-sources is being used as your kernel.

You are free to substitute any of the above for whatever you want if you know what you are doing (alternate kernels, initramfs generators, partition layouts (RAIDZ2?), etc). However, support will only be provided from me when the above configuration is used. Also, this guide is the way I install Gentoo, not exactly the way the handbook has it.

Boot your system into the ISO
Since this is highly computer dependent, you will need to figure out how to boot your USB on your system and get to the live environment. You may need to disable Secure Boot if that causes your USB to be rejected. Make sure your system BIOS/UEFI is set up to boot UEFI devices, rather than BIOS devices (Legacy).

Confirm that you booted in UEFI Mode
After you booted into the Live CD, make sure that you booted into UEFI mode by typing the following:

If the above directory is empty or doesn't exist, you are not in UEFI mode. Reboot and boot into UEFI mode.

Partition
We will now partition the drive and aim to create the following layout:

/dev/sda1  | 512 MB        |   EFI System Partition                | /efi /dev/sda2  | 32768 MB      |   swap                                | swap /dev/sda3  | Rest of Disk  |   ZFS                                 | /, /home/username ...

Open up your drive in GNU parted and tell it to use optimal alignment:

Create GPT partition layout
This will delete all partitions and create a new GPT table.

Larger swap will accommodate hibernation should that be desired. 32GB swap is used in the below example to accommodate many different hardware configurations.

Final View
Exit the application

Determine disk/by-id identifier
Using traditional block device identifiers such as /dev/sda or /dev/nvme0n1 with zfs can work but can also be undesirable due to the possibility of a block device name changing. Should this ever happen zfs pools are unaware of the change having occurred which can render a zfs pool inoperable. Use of non generic device specific disk identifiers which are also identified by disk serial number is more desirable for use with zfs as a result of this complication. This also provides added utility advantages for identifying a faulty disk in larger zfs pools.

Generally using /dev/disk/by-id/ata-disk or /dev/disk/by-id/nvme-disk is more desirable to ensure the disk block device is more specific.

There may be dev/disk/by-id/wmm or /dev/disk/by-id/nvme-eui. The latter should be avoided if possible for use with this guide.

lrwxrwxrwx 1 root root 10 Mar 2 11:28 wwn-0x5002538e40aba28d-part1 -> ../../sda1 lrwxrwxrwx 1 root root 10 Mar 2 11:28 wwn-0x5002538e40aba28d-part2 -> ../../sda2

Lrwxrwxrwx 1 root root 13 Mar 2 11:28 nvme-eui.0025385971b064dd -> ../../nvme0n1 lrwxrwxrwx 1 root root 15 Mar 2 11:28 nvme-eui.0025385971b064dd-part1 -> ../../nvme0n1p1

To determine the non generic disk identifier id type the following

Nvme storage devices would resemble this example

Create your zpool
Create your zpool which will contain your drives and datasets:

xattrs, relatime and posixacl are enabled to provide support for modern filesystem security features and relative atime updates which are a global default in ext4. xattrs is necessary for proper functionality of systemd-journald

Create your zfs datasets
Create the dataset container structure and dataset mount necessary for /. ZFS is extremely flexible and you can easily add or remove datasets in the future.

Creation of several unmounted dataset containers is necessary to provide dataset contents structure for the pool for. Creation of these containers after install is complete can be disruptive, involved vand best completed before a filesystem is written to disk to ensure the system will boot. Dataset container mountpoints for /usr and /var especially benefit from this having been completed in advance. rpool/home dataset is created to segregate user home directory dataset contents from the rootfs dataset for improved rootfs dataset incremental snapshot size management.

Verify everything looks good
You can verify that all of these things worked by running the following:

Now we are ready to install Gentoo!

Set your date and time
Let's say it's August 24, 2014 @ 2:48 PM (will be 14:48 in 24 hour time), we would do the following:

Preparing to chroot
First let's mount our efi boot partition in our chroot directory:

and now download the systemd amd64 image here and extract it:

Edit fstab
Everything is on zfs so we don't need anything in here except for the boot and swap zvol entries. My fstab looks as follows:

Modify make.conf
Let's modify our /etc/portage/make.conf so we can start installing stuff with a good base (Change it to what you need):

Get the portage tree
Copy the default example portage config

Using the 'bliss-kernel' configuration
If you don't want to use the pre-compiled binaries for 'bliss-kernel', you can use the 'bliss-kernel' kernel configurations as a base for your kernel. The configuration is ready to go for general desktop, laptop, and server use, and it is also configured to work with ZFS. The 'bliss-kernel' configuration is simply a slightly tweaked Fedora config.

You can download the 'bliss-kernel' configurations here.

Using the pre-compiled 'bliss-kernel' binaries
Install the kernel

Install required applications
Unmask latest ZFS versions

Enable ZFS support in GRUB

Now install the apps:

Installing the bootloader onto your drive
We will need to install the bootloader onto the drive. Before we do that however, let's see if GRUB can detect our /boot and /boot/efi filesystem types:

This should say 'zfs'. If it doesn't, then something is wrong and your system will not boot!

This should say 'fat'. If it doesn't, then something is wrong and your system will not boot!

Before we install the bootloader, we will need to have read/write access to the efi nvram variables. Let's remount our efivars now:

Now run the following to install the bootloader to the drive:

The above command will install the grub bootloader files into /boot and the efi files into /boot/efi. It should return a "Installation finished. No error reported." message. If it doesn't, then something is wrong and your system will not boot!

Make the GRUB 2 configuration file
You can use the following configuration file as a basis for your system:

Generating new zpool.cache file before/after reboot
ZFS is very sensitive about the data that is contained in the zpool.cache file and at this point, when we reboot, the information in it might not be completely accurate. To ensure we have a good cache file, we have instructed bliss-initramfs in the bootloader config above, to ignore the current cachefile on the system, and make a new one that is up-to-date. We only need to do this once.

Remove zpool.cache refresh flag from bootloader configuration
Open up your grub.cfg and remove the 'refresh' flag from the kernel line.

Take a snapshot of your new system
Since we now have a working system, we will snapshot it in case we ever want to go back or recover files:

You can view the contents of these snapshots by checking their respective and hidden .zfs directories:

Limiting the ARC size
If you want to cap the ZFS ARC from growing past a certain point, you can put the number of bytes inside the /etc/modprobe.d/zfs.conf file, and then remake your initramfs. When the system starts up, and the module is loaded, these options will be passed to the zfs kernel module.

(Temporary) Change the ARC max for the running system to 4 GB

(Permanent) Save the 4 GB ARC cap as a loadable kernel parameter

Once we have the above file created, let's regenerate the initramfs. bliss-initramfs will automatically detect that this file exists and copy it into the initramfs. When you reboot your machine, the initramfs will load up the zfs kernel module with the parameters found in the file.

Bliss ZFS Scripts
The following scripts allow you to automatically:


 * Take snapshots of your pool
 * Replicate the pool to another pool (Full and Incremental Backups)
 * Clean the old snapshots on your pools.

You can download, customize, and install the scripts into your /usr/local/sbin directory. Github.

And that's it. Enjoy!