User:Monsieurp/Gentoo Linux on ZFS

Introduction
This is yet another guide to install Gentoo Linux on a ZFS-formatted disk. This guide has been greatly inspired by Jonathan Vasquez' Gentoo Linux ZFS guide and Funtoo's ZFS Install guide.

Why yet another guide?
 * I find Jonathan's disk layout overly complicated (no offence if you read that mate!).
 * The Funtoo ZFS install guide is actually pretty well written but unfortunately has been written with Funtoo in mind.
 * I took some notes during the installation and decided to put them online for others to read and learn from.

Throughout this guide, I will often link to the official Oracle ZFS documentation, a well written piece of document that gets to the point. Although it has been written with Solaris in mind, you can adapt most commands for Linux.

Required Tools

 * Jonathan Vasquez' modified System Rescue CD with ZFS support.

Download the ISO file and burn it to a CD or USB key. At the time of this writing, the latest ISO file available is:.

There are many ways to go about burning an ISO file to a USB key,  and   to name a few. This part won't be covered in this guide. You should look it up in your favourite search engine :-)

Booting the SR CD
Once you've burnt the ISO, power on the computer and boot off the CD player or USB key. The System Rescue CD boots and you should land in the multi-selection entry screen on the right hand side. Here, pick either the 1st or 2nd entry, which are default boot options or all files cached to memory. On my server, I had to pick the 2nd one. Your mileage may vary.

After hitting enter, the system will then boot into a Gentoo Linux LiveCD and automatically log you into a zsh shell as.

Let's get cracking!

Storage pool
We are creating a basic ZFS storage pool named  that contains only one disk. In this configuration, ZFS takes up the whole disk. There's no need to create additional partitions with the  or   commands since GRUB 2 now supports reading ZFS-formatted disks and booting from them just fine.

Creation
Here's the command to create the storage pool:

What just happened? Let's try to break down the command:
 * : create a new pool.
 * : force the creation of the pool.
 * : the selected value for the alignment shift, 12 in this case, which corresponds to 2^12 Bytes or 4 KiB. This value be set once at the pool creation. You can find out more about the alignment shift here.
 * : create a pool cache file in . This file is extremely important and MUST be copied over in the chroot directory. We will get back to it later.
 * : the selected normalisation algorithm to use when comparing two Unicode file names on the file system,  in this case. I found this pretty interesting link on the unicode.org website that explains at length the gory details of the different Unicode normalisation forms.
 * : the selected compression algorithm to use when writing files to disk,  in this case. This link explains the different compression algorithms built in ZFS much better than I could.
 * : do not set a mountpoint for this storage pool.
 * : the alternate root directory, which is actually just a temporary mount point for the installation.
 * : the name of this storage pool. You can customise this value if you want but we will use  throughout this guide.
 * : the path to the physical disk, also known as a  in ZFS lingo. It must point to the actual disk you want to format. Remember to use by-id for the disk.

All in all,  is the equivalent of the gool old. I encourage you to read two links from the Oracle website here and here. They go over what a storage pool is about and the wide range of scenarios they can be used in.

Verification
Let's check to see whether the pool was created successfully:

Datasets
In ZFS jargon, a dataset is, as its name suggests, a bunch of data. A dataset has several attributes which can be enabled or disabled when creating the dataset. Datasets can be nested, that is, a dataset can be created out of another dataset. When doing so, child datasets will inherit properties from their parents. You can create as many datasets as you wish with ZFS. However, too many datasets will slow down the execution of the  command as well as the startup of the system (over 100 according to the FreeBSD ZFS Guide)

In order not to confuse the reader, our file system layout is as simple as it gets:


 * a root dataset, unmounted and out of which the two following datasets are created
 * the  dataset dedicated to the system
 * the  dataset dedicated to the kernel, initramfs and GRUB 2 files

Feel free to customise this layout and create dedicated datasets for directories such as,  ,  , etc. if you deem they each need to live on separate datasets.

Creation
Here are the commands to create the datasets according to the layout described above:

We have to mark the  dataset as bootable otherwise the system won't boot at all:

We are creating a Swap dataset for the system according to the ZFS On Linux official guide:

Have a look at the link above for an explanation of each option.

Verification
Let's check to see whether everything looks good:

Getting the installation directory ready
Now that our datasets are created and mounted, let's prepare the installation directory. First, fetch the latest stage3 and extract it in the installation directory:

Remember the  file I told you about? We are going to copy it over in the installation directory:

Let's not forget the  file to get out on the Interwebz from within the chroot:

It's time to chroot into the installation directory:

Configuring the system
Let's start off by configuring the system locales:

Set the timezone we are in:

Configure the hostname:

Configure :

Configure :

Copy Portage repositories configuration file:

Synchronise the tree and read the news:

Re-emerge @world, install vi and a DHCP client:

Unmask ZFS packages
Some of these packages haven't been marked stable yet. We need to add manually the list of packages to the  file:

We must unmask the  USE flag for GRUB 2 and add it to the   directory:

Unmask genkernel and gentoo-sources
For the time being, only kernel versions greater than or equal to 4.13.x are compatible with ZFS. You must also install the latest version of genkernel to embed ZFS in your kernel:

genkernel needs util-linux to be compiled with the  USE flag:

Let's emerge genkernel, gentoo-sources and gptfdisk:

At this point, you are all set to compile ZFS in your kernel. Generate a generic kernel config file:

Exit and save.

Compile the kernel with genkernel
Take a deep breath and type this in:

What's going on here? I must admit it is a convoluted way to compile the kernel as well everything needed to have a working system with ZFS support enabled. But hey, it works!

GRUB 2 compilation, with the  USE flag turned on, ultimately fails during configure on the first try (you can try on your own before launching this command). Indeed, the configure script scouts for a whole lot of ZFS include files in the  directory. Unless we compile the kernel at least once before, these files won't be present on the system. We need to generate them in some fashion with genkernel. However, at the end of the compilation process, genkernel calls ZFS commands, which are not installed on the system yet, and fails. We catch this failure with the  shell command. In the second part of the command, we emerge GRUB 2 and launch again the genkernel command. This time around, GRUB 2 will install just fine. Note that GRUB 2 with the  USE flag installs the required ZFS dependencies automatically: ,  , and.

Install GRUB 2
We've got to install the GRUB 2 bootloader in order to boot our system. Let's get GRUB to probe the  partition first:

When zpool created our storage pool, it created partitions under a GPT scheme. In order to boot Gentoo Linux on a GPT partition under legacy boot (BIOS), GRUB 2 requires a BIOS boot partition. By design, ZFS left a very small unpartitioned space at the beginning of the disk. We will use the  utility, which is part of , to format this free space into a BIOS boot partition:

Let's now run  to refresh the list of partitions recognised by the kernel for the current disk:

We've got to let GRUB 2 know that our kernel lives in a ZFS partition. Edit the file  and add the following two lines:

Go ahead and install the bootloader:

Let's check whether the ZFS module for GRUB has been installed:

Finally, we've got to generate the  file in the   directory so that the boot loader can boot our freshly generated kernel:

Services
The following services must be enabled for ZFS to properly start and run:

User management
Change the root password:

Add a new user and change his password:

Wrapping it up
Let's exit, unmount everything and reboot into our new system: