User:Monsieurp/Gentoo Linux on ZFS

Introduction
This is yet another guide to install Gentoo Linux on a ZFS-formatted disk. This guide has been greatly inspired by Jonathan Vasquez' Gentoo Linux ZFS guide and Funtoo's ZFS Install guide.

Why yet another guide?
 * I find Jonathan's disk layout overly complicated (no offence if you read that mate!).
 * The Funtoo ZFS install guide is actually pretty well written but unfortunately has been written with Funtoo in mind.
 * I took some notes during the installation and decided to put them online for others to read and learn from.

Throughout this guide, I will often link to the official Oracle ZFS documentation, a well written piece of document that gets to the point. Although it has been written with Solaris in mind, you can adapt most commands for Linux.

Required Tools

 * Jonathan Vasquez' modified System Rescue CD with ZFS support.

Download the ISO file and burn it to a CD or USB key. At the time of this writing, the latest ISO file available is:.

There are many ways to go about burning an ISO file to a USB key,  and   to name a few. This part won't be covered in this guide. You should look it up in your favourite search engine :-)

Booting the SR CD
Once you've burnt the ISO, power on the computer and boot off the CD player or USB key. The System Rescue CD boots and you should land in the multi-selection entry screen on the right hand side. Here, pick either the 1st or 2nd entry, which are default boot options or all files cached to memory. On my server, I had to pick the 2nd one. Your mileage may vary.

After hitting enter, the system will then boot into a Gentoo Linux LiveCD and automatically log you into a zsh shell as.

Let's get cracking!

Storage pool
We are creating a basic ZFS storage pool named  that contains only one disk. In this configuration, ZFS takes up the whole disk. There's no need to create additional partitions with the  or   commands since GRUB 2 now supports reading ZFS-formatted disks and booting from them just fine.

Creation
Here's the command to create the storage pool:

What just happened? Let's try to break down the command:
 * : create a new pool.
 * : force the creation of the pool.
 * : the selected value for the alignment shift, 12 in this case, which corresponds to 2^12 Bytes or 4 KiB. This value be set once at the pool creation. You can find out more about the alignment shift here.
 * : create a pool cache file in . This file is extremely important and MUST be copied over in the chroot directory. We will get back to it later.
 * : the selected normalisation algorithm to use when comparing two Unicode file names on the file system,  in this case. I found this pretty interesting link on the unicode.org website that explains at length the gory details of the different Unicode normalisation forms.
 * : the selected compression algorithm to use when writing files to disk,  in this case. This link explains the different compression algorithms built in ZFS much better than I could.
 * : do not set a mountpoint for this storage pool.
 * : the alternate root directory, which is actually just a temporary mount point for the installation.
 * : the name of this storage pool. You can customise this value if you want but we will use  throughout this guide.
 * : the path to the physical disk, also known as a  in ZFS lingo. It must point to the actual disk you want to format.

All in all,  is the equivalent of the gool old. I encourage you to read two links from the Oracle website here and here. They go over what a storage pool is about and the wide range of scenarios they can be used in.

Verification
Let's check to see whether the pool was created successfully:

Datasets
In ZFS jargon, a dataset is, as its name suggest, a bunch of data. A dataset has several attributes which can be enabled or disabled when creating the dataset. Datasets can be nested, that is, a dataset can be created out of a dataset. When doing so, child datasets will inherit properties from their parents. You can create as many datasets as you wish. However, too many datasets will slow down the execution of the  command as well as the startup of the system (over 100 according to the FreeBSD ZFS Guide)

In our layout, all datasets are mounted (except the root one) and created out of the root dataset. For the sake of keeping a concise guide and in order not to confuse the reader, our file system layout is as simple as it gets:


 * one dataset dedicated for the system aka
 * one dataset dedicated for the kernel, initramfs and GRUB 2 files aka

Feel free to customise this layout and create additional datasets for directories you think might need have their own dedicated datasets (/usr/portage, /var, /var/log, etc).

Creation
Here are the commands to create the datasets according to the layout described above:

We have to mark the  dataset as bootable otherwise the system won't boot at all:

We are creating a Swap dataset for the system to use according to the ZFS On Linux official guide:

Have a look at the link above for an explanation of each option.

Verification
Let's check to see whether everything looks good:

Getting the installation directory ready
Now that our datasets are created and mounted, let's prepare the installation directory. First, fetch the latest stage3 and extract it in the installation directory:

Remember the  file I told you about? We are going to copy it over in the installation directory:

Let's not forget the  file to get out on the Interwebz from within the chroot:

It's time to chroot into the installation directory:

Configuring the system
Let's start off by configuring the system locales:

Set the timezone we are in:

Configure the hostname:

Configure :

Configure :

Copy Portage repositories configuration file:

Synchronise the tree and read the news:

Re-emerge @world, install vi and a DHCP client:

Unmask ZFS packages
Some of these packages haven't been marked stable yet. We need to add manually the list of packages to the  file:

We must unmask the  USE flag for GRUB 2 and add it to the   directory:

Unmask genkernel and gentoo-sources
For the time being, only kernel versions greater than or equal to 4.13.x are compatible with ZFS. You must also install the latest version of genkernel to embed ZFS in your kernel:

genkernel needs util-linux to be compiled with the  USE flag:

Let's emerge the genkernel and gentoo-sources packages:

At this point, you are all set to compile ZFS in your kernel. Generate a generic kernel config file:

Exit and save.

Compile the kernel with genkernel
Take a deep breath and type this in:

What's going on here? I must admit it is a convoluted way to compile the kernel as well everything needed to have a working ZFS system. But hey, it works!

GRUB 2 compilation, with the  USE flag turned on, ultimately fails during configure on the first try (you can try on your own before launching this command). Indeed, the configure script scouts for a whole lot of ZFS include files in the /usr/src/linux directory and unless we compile the kernel at least once before, these files won't be present on the system. We need to generate them in some fashion with genkernel. However, at the end of the compilation process, genkernel calls ZFS commands, which are not installed on the system yet, and fails. We catch this failure with the  shell command. In the second part of the command, we emerge GRUB 2 and launch again the genkernel command. This time around, GRUB 2 will install just fine. Note that GRUB 2 with the  USE flag installs the required ZFS dependencies i.e. sys-fs/zfs, sys-fs/zfs-mkod and sys-kernel/spl.

Install GRUB 2
We've got to install the GRUB 2 bootloader in order to boot our system. Let's get GRUB 2 to probe the  partition first:

When zpool created our storage pool, it created partitions under a GPT scheme. In order to boot Gentoo Linux on a GPT partition under legacy boot (BIOS), GRUB 2 requires a BIOS boot partition. By design, ZFS left a very small unpartitioned space at the beginning of the disk. We will use, which is part of sys-apps/gptfdisk, to format this free space into a BIOS boot partition:

Let's now run  to refresh the list of partitions recognised by the kernel for the current disk:

We've got to let GRUB 2 know that our kernel lives in a ZFS partition. Edit the file  and add the following two lines: