User:Fearedbliss/Installing Gentoo Linux On ZFS

From Gentoo Wiki
Jump to: navigation, search

Contents

Install Gentoo Linux on ZFS

Author: Jonathan Vasquez (fearedbliss)
Contact: fearedbliss@gentoo.org

Preface

This guide will show you how to install Gentoo Linux on AMD64 with:

 * UEFI/GPT (EFI System Partition (ESP) will be on a FAT32 unencrypted partition - as per UEFI specifications)
 * Everything on ZFS (/, /home, /boot, swap)
 * ZFS 0.6.5.9
 * GRUB 2 (Bootloader)
 * OpenRC
 * Gentoo Stable (amd64)

Notes on other stuff you can do in the ZFS install can be found here.

Required Tools

Download my modified System Rescue CD that includes ZFS from one of the mirrors on the front page.

Linux

If you have a Linux system, you can create a bootable usb from it by doing the following:

root #mkdir /tmp/iso
root #mount -o loop,ro <system rescue iso> /tmp/iso
root #/tmp/iso/usb_inst.sh

That should open up a screen where you can select your plugged in flash drive.

Windows

You can download "Rufus" and select the ISO by clicking the little image next to the "Create a bootable disk using <>". After that, you can use the default settings and press "Start".

Download Rufus

Assumptions

  • Only installing Gentoo on one drive called /dev/sda.
  • You are using my modified System Rescue CD that includes ZFS
  • bliss-initramfs and bliss-kernel are used. You can also use genkernel and your own custom compiled kernel.

Also, this guide is the way I install Gentoo, not exactly the way the handbook has it.

Partition

We will now partition the drive and aim to create the following layout:

/dev/sda1   | 2 MB         |   BIOS Boot Partition                   | So GRUB 2 can work in GPT
/dev/sda2   | 100 MB       |   EFI System Partition                  | /boot/efi
/dev/sda3   | 500 MB       |   Boot Partition (ZFS Pool Version 28)  | /boot
/dev/sda4   | Rest of Disk |   ZFS (Pool Version 5000)               | /, swap, etc

Open up your drive in GNU parted and tell it to use optimal alignment:

root #parted -a optimal /dev/sda
Warning
Due keep in mind that all of the following operations will affect the disk immediately. GNU parted does not stage changes like fdisk or gdisk.

Set the units to Megabytes (Technically Mebibytes)

(parted)unit mib

Create GPT partition layout

This will delete all partitions and create a new GPT table.

(parted)mklabel gpt

Create and label your partitions

(parted)mkpart primary 1 3
(parted)mkpart primary 3 103
(parted)mkpart primary 103 603
(parted)mkpart primary 603 -1
(parted)name 1 grub
(parted)name 2 esp
(parted)name 3 boot
(parted)name 4 rootfs

Set the bios_grub flag on the grub partition

(parted)set 1 bios_grub on

Set the bootable flag on the ESP partition

(parted)set 2 boot on

Final View

(parted)print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 102400MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
  
Number   Start      End      Size     File system  Name   Flags
 1       1.00MiB    3.00MiB   2.00MiB               grub   bios_grub
 2       3.00MiB    103MiB    100MiB                esp    boot, esp
 3       103MiB     603MiB    500MiB                boot
 4       603MiB     102399MiB 101796MiB             rootfs

Exit the application

(parted)quit

Format your drives

Create your boot partition

root #mkfs.fat -F32 /dev/sda2
Warning
This partition needs to be FAT32 due it being an UEFI requirement. If it isn't, your system will not boot!

Create your zpool

Create your zpool which will contain your drives and datasets:

root #zpool create -f -o ashift=12 -o cachefile= -O compression=lz4 -m none -R /mnt/gentoo rpool /dev/sda4

Create your zfs datasets

We will keep it simple and just create a few datasets for /, and /home. ZFS is extremely flexible and you can easily add or remove datasets in the future.

root #zfs create rpool/ROOT
root #zfs create -o mountpoint=/ rpool/ROOT/gentoo
root #zfs create -o mountpoint=/home rpool/HOME
root #zfs create -o mountpoint=/root rpool/HOME/root

Create your boot pool

It is safer for us to create a separate zpool that has all feature flags disabled. This is because even though grub currently supports the latest feature flags for the 0.6.5 release of ZFS, new feature flags added to ZFS without proper bootloader support can make your system unbootable. Since GRUB 2 has solid support for a zpool with no feature flags, we will create a separate featureless boot pool. However, our main system will still have all features enabled.

root #zpool create -f -d -o ashift=12 -o cachefile= -m /boot -R /mnt/gentoo boot /dev/sda3
Warning
Do not do a 'zpool upgrade' on your boot pool. Doing this will mostly likely leave your system unbootable.

Create your swap zvol

This creates the zfs block device that we will use for swap.

root #zfs create -o sync=always -o primarycache=metadata -o secondarycache=none -b 4k -V 4G -o logbias=throughput rpool/swap
root #mkswap -f /dev/zvol/rpool/swap
root #swapon /dev/zvol/rpool/swap
Note
  • Currently, the performance of the swap zvol isn't that great. This should improve over time.
  • You should have a swap even if it is small. Your system could lock up under high load, especially since ZFS uses your physical RAM for its ARC.

Verify everything looks good

You can verify that all of these things worked by running the following:

root #zpool status
  pool: boot
 state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on software that does not support
        feature flags.
  scan: none requested
config: 

        NAME        STATE     READ WRITE CKSUM
        boot        ONLINE       0     0     0
          sda3      ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: none requested
config: 

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          sda4      ONLINE       0     0     0

errors: No known data errors
root #zfs list
NAME                USED  AVAIL  REFER  MOUNTPOINT
boot                484K   448M   136K  /mnt/gentoo/boot
rpool              4.25G  91.7G    96K  none
rpool/HOME          192K  91.7G    96K  /mnt/gentoo/home
rpool/HOME/root      96K  91.7G    96K  /mnt/gentoo/root
rpool/ROOT          192K  91.7G    96K  none
rpool/ROOT/gentoo    96K  91.7G    96K  /mnt/gentoo
rpool/swap         4.25G  95.9G    64K  -

Now we are ready to install Gentoo!

Installing Gentoo

Check and set your time

root #date

If it's wrong then fix it. Let's say it's August 24, 2014 @ 2:48 PM (will be 14:48 in 24 hour time), we would do the following:

root #date 082414482014
Note
08 24 1448 2014 (Month, Date, 24 Hour Time, Year). It will be UTC at the moment but we will fix it once we are inside our environment.

Preparing to chroot

First let's mount our efi boot partition in our chroot directory:

root #cd /mnt/gentoo
root #mkdir boot/efi
root #mount /dev/sda2 boot/efi

and now download the amd64 image here and extract it:

root #wget <file>
root #tar xpf <file>

Copy zpool cache

root #mkdir etc/zfs
root #cp /etc/zfs/zpool.cache etc/zfs

Copy network settings

root #cp /etc/resolv.conf etc/

Mount required devices

root #mount --rbind /dev dev
root #mount --rbind /proc proc
root #mount --rbind /sys sys

Chroot into your environment

root #env -i HOME=/root TERM=$TERM chroot . bash -l

Inside the chroot

Generate locales

I'm using US English w/ UTF-8, so I will enable that locale on my system:

root #nano /etc/locale.gen
root #locale-gen
root #env-update
root #. /etc/profile

Fix your date

I'm in New York and the time currently is August 24, 2014 @ 2:48 PM (will be 14:48 in 24 hour time)

root #ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime
root #date 082414482014

Set hostname

Open up /etc/conf.d/hostname with your text editor and add your desired hostname in there.

root #nano /etc/conf.d/hostname

For example, my /etc/conf.d/hostname file could be:

hostname="bliss"

Create mtab symlink

root #ln -sf /proc/self/mounts /etc/mtab

Edit fstab

Everything is on zfs so we don't need anything in here except for the boot and swap zvol entries. My fstab looks as follows:

root #nano /etc/fstab

/dev/sda2               /boot/efi       vfat            defaults,noauto        1 2
/dev/zvol/rpool/swap    none            swap            sw                     0 0
Warning
Do not auto-mount the /boot/efi partition. If you do, the init system will create the /boot/efi folder if it doesn't exist, and that will prevent the 'boot' zpool from mounting.

Modify make.conf

Let's modify our /etc/portage/make.conf so we can start installing stuff with a good base (Change it to what you need):

root #nano /etc/portage/make.conf

USE="mmx sse sse2 -branding"

# This should be your number of processors + 1
MAKEOPTS="-j5"

EMERGE_DEFAULT_OPTS="--with-bdeps=y"
LINGUAS="en en_US"
VIDEO_CARDS="intel"
INPUT_DEVICES="evdev synaptics"

Get the portage tree

Copy the default example portage config

root #cp /usr/share/portage/config/repos.conf /etc/portage/repos.conf
root #emerge --sync

Compiling your own kernel

Using the 'bliss-kernel' configuration as a base

If you don't want to use the pre-compiled "bliss-kernel" in "bliss-overlay" below, I've made available the 'bliss-kernel' configs so that you can compile the kernel yourself using that configuration as a base. The configuration is ready to go for general desktop, laptop, and server use, and it is also configured to work with ZFS.

You can download the 'bliss-kernel' configuration here.

Using your own configuration

If you are not going to use the above configuration, then you will need to make sure your kernel has the following options enabled or ZFS will not work:

Note
This list is still being updated, and is using the options listed in the sys-kernel/spl ebuild as a base.

Disable anything marked [ ], Compile in [*], Options marked [M] may either be separate modules or compiled into the kernel:

KERNEL Kernel Settings for ZFS
[ ] DEBUG_LOCK_ALLOC 
[*] EFI_PARTITION
[*] IOSCHED_NOOP
[*] MODULES
[*] BLK_DEV_INITRD
[*] KALLSYMS 
[ ] PAX_KERNEXEC_PLUGIN_METHOD_OR 
[ ] PAX_SIZE_OVERFLOW
[M] ZLIB_DEFLATE
[M] ZLIB_INFLATE

(Optional) Add bliss-overlay and install a pre-compiled/configured ZFS-ready kernel

bliss-kernel is a vanilla kernel (directly from kernel.org, no patches) that fearedbliss maintains. It uses a slightly tweaked fedora config. The kernel is pre-configured and tested for ZFS. Use this if you don't want to spend time tweaking/debugging the Linux Kernel.

Open repos.conf and add bliss-overlay

root #nano /etc/portage/repos.conf

[bliss-overlay]
location = /var/lib/overlays/bliss-overlay
sync-type = git
sync-uri = https://github.com/fearedbliss/bliss-overlay.git
auto-sync = yes

Compile git

root #emerge dev-vcs/git

Sync once more to get bliss-overlay

root #emerge --sync

Install the kernel

root #emerge bliss-kernel
Note

Consider pinning the kernel version that was installed so that when we do a future kernel upgrade, an emerge --depclean won't delete our kernel files from the /boot directory. If the bliss-kernel ebuild that emerge pulled to install was: bliss-kernel-4.1.8, we would do:

root #emerge --noreplace =bliss-kernel-4.1.8

Install required applications

Enable the AMD64 EFI platform for grub

root #echo 'GRUB_PLATFORMS="efi-64"' >> /etc/portage/make.conf

Unmask the latest versions of zfs

root #echo "sys-kernel/spl ~amd64" >> /etc/portage/package.accept_keywords
root #echo "sys-fs/zfs ~amd64" >> /etc/portage/package.accept_keywords
root #echo "sys-fs/zfs-kmod ~amd64" >> /etc/portage/package.accept_keywords

Enable ZFS support in GRUB

root #echo "sys-boot/grub ~amd64" >> /etc/portage/package.accept_keywords
root #echo "sys-boot/grub libzfs" >> /etc/portage/package.use

Now install the apps:

root #emerge bliss-initramfs grub

Update your system

root #emerge -uDNav @world

Installing the bootloader onto your drive

We will need to install the bootloader onto the drive. Before we do that however, let's see if GRUB can detect our /boot and /boot/efi filesystem types:

root #grub-probe /boot

This should say 'zfs'. If it doesn't, then something is wrong and your system will not boot!

root #grub-probe /boot/efi

This should say 'fat'. If it doesn't, then something is wrong and your system will not boot!

Now run the following to install the bootloader to the drive:

root #grub-install --efi-directory=/boot/efi /dev/sda

The above command will install the grub bootloader files into /boot and the efi files into /boot/efi. It should return a "Installation finished. No error reported." message. If it doesn't, then something is wrong and your system will not boot!

Make the GRUB 2 configuration file

You can use the following configuration file as a basis for your system:

root #nano /boot/grub/grub.cfg

set timeout=1
set default=0

insmod part_gpt
insmod fat
insmod efi_gop
insmod efi_uga

menuentry "Gentoo - 4.1.8-FC.01" {
    linux /@/kernels/4.1.8-FC.01/vmlinuz root=rpool/ROOT/gentoo by=id elevator=noop quiet logo.nologo refresh
    initrd /@/kernels/4.1.8-FC.01/initrd
}

Generating new zpool.cache file before/after reboot

ZFS is very sensitive about the data that is contained in the zpool.cache file and at this point, when we reboot, the information in it might not be completely accurate. To ensure we have a good cache file, we have instructed bliss-initramfs in the bootloader config above, to ignore the current cachefile on the system, and make a new one that is up-to-date. We only need to do this once.

Generate the initramfs and move the file to its correct location

root #bliss-initramfs 1 4.1.8-FC.01
root #mv initrd-4.1.8-FC.01 /boot/kernels/4.1.8-FC.01/initrd

Final steps before reboot

root #rc-update add zfs-import boot
root #rc-update add zfs-mount boot
root #rc-update add zfs-share default
root #rc-update add zfs-zed default
root #passwd
root #exit
root #reboot

After you reboot

Remove zpool.cache refresh flag from bootloader configuration

Open up your grub.cfg and remove the 'refresh' flag from the kernel line.

root #nano /boot/grub/grub.cfg

Take a snapshot of your new system

Since we now have a working system, we will snapshot it in case we ever want to go back or recover files:

root #zfs snapshot rpool/ROOT/gentoo@2016-03-25-0000-01-INSTALL
root #zfs snapshot rpool/HOME@2016-03-25-0000-01-INSTALL
root #zfs snapshot boot@2016-03-25-0000-01-INSTALL

You can view the contents of these snapshots by checking their respective and hidden .zfs directories:

root #ls /.zfs/snapshots
root #ls /home/.zfs/snapshots
root #ls /boot/.zfs/snapshots
Note
We are naming the snapshots in this format because it's the format that my 'bliss-zfs-scripts' use, and it is also very readable.

Other stuff you might want to install

Tools in Portage
root #emerge sudo htop gentoolkit eix app-misc/screen app-text/tree
Bliss ZFS Scripts

The following scripts allow you to automatically:

  • Take snapshots of your pool
  • Replicate the pool to another pool (Full and Incremental Backups)
  • Clean the old snapshots on your pools.

You can download, customize, and install the scripts into your /usr/local/sbin directory. Github.

And that's it. Enjoy!