User:Fearedbliss/Installing Gentoo Linux On ZFS

Install Gentoo Linux on ZFS
Author: Jonathan Vasquez (fearedbliss) Contact: jvasquez1011@gmail.com

Preface
This guide will show you how to install Gentoo Linux with:

* /boot, /, and swap on ZFS * ZFS 0.6.5.1 * GPT (Partition Layout) * GRUB 2 (bootloader) * OpenRC (Init) * Gentoo Stable (amd64) * Precompiled Binary Packages

Notes on other stuff you can do in the ZFS install can be found here.

Required Tools
Download my modified System Rescue CD that includes ZFS from one of the mirrors on the front page.

Linux
If you have a Linux system, you can create a bootable usb from it by doing the following:

mkdir /tmp/iso mount -o loop,ro /tmp/iso /tmp/iso/usb_inst.sh

That should open up a screen where you can select your plugged in flash drive.

Windows
You can download "Rufus" and select the ISO by clicking the little image next to the "Create a bootable disk using <>". After that, you can use the default settings and press "Start".

Download Rufus

Assumptions

 * Only installing Gentoo on one drive called /dev/sda.
 * You are using my modified System Rescue CD that includes ZFS
 * Will be using my tools (bliss-kernel, bliss-initramfs).

Also, this guide is the way I install Gentoo, not exactly the way the handbook has it.

Optional: Installing over SSH
If you want to be able to copy links easier into your terminal (I use PuTTY on Windows), you could set the password on your laptop, get the ip, and then connect to your laptop over the network and perform most of the install through it. passwd [Enter your new password]

ifconfig [Retrieve the IP for this machine]

screen

Then go to your other machine where you will be installing from, log in, and then re-attach to your screen:

screen -rd Using screen is optional but convenient.

Wipe Partition Layout
This will delete all partitions and their respective MBR/GPT data structures. No confirmation or prompt will be given.

sgdisk -Z /dev/sda

Create your partitions
gdisk /dev/sda

Once gdisk is opened, use the following layout:

/dev/sda1 = 500 MB      | ZFS Pool Version 28   | /boot Partition | Code: BE00 /dev/sda2 = 4 MB        | BIOS Boot Partition   | GRUB 2 - GPT    | Code: EF02 /dev/sda3 = Rest of Disk | ZFS Pool Version 5000 | /, /home, swap | Code: BF00

The "->" means to just press enter and accept the default.

/boot Partition

Command: n Partition number: -> First sector: -> Last sector: +500M Hex code or GUID: be00

BIOS Boot Partition

Command: n Partition number: -> First sector: -> Last sector: +4M Hex code or GUID: ef02

/ Partition

Command: n Partition number: -> First sector: -> Last sector: -> Hex code or GUID: bf00

Command: p

Number Start (sector)    End (sector)  Size       Code  Name 1           2048         1026047   500.0 MiB   BE00  Solaris boot 2        1026048         1034239   4.0 MiB     EF02  BIOS boot partition 3        1034240      1953525134   931.0 GiB   BF00  Solaris root

Create /boot filesystem
Since GRUB 2 only has ZFS support for pool version 28, and we want our /boot to be on ZFS, we will make a separate pool that is version 28. Our / will be on another pool that is on a higher version that includes all of the new ZFS features.

zpool create -f -o version=28 -o ashift=12 -o cachefile= -m none boot /dev/sda1

zpool export boot
 * 1) Export the pool since we will import it back in in a bit with /dev/disk/by-id tags

'''Do not upgrade your 'boot' pool above version 28, since this will make GRUB 2 not be able to read it. If you ever do, just backup the existing files in /boot, destroy the 'boot' pool, and recreate it back on version 28. Not a big deal and doesn't take much time, but just something to keep in mind.'''

Create your zpool
Create your zpool which will contain your drives and datasets:

zpool create -f -o ashift=12 -o cachefile= -O compression=lz4 -m none -R /mnt/gentoo tank /dev/sda3 zpool export tank

Import your pools using /dev/disk/by-id
zpool import -R /mnt/gentoo -o cachefile= -d /dev/disk/by-id tank zpool import -R /mnt/gentoo -o cachefile= -d /dev/disk/by-id boot

Create your zfs datasets
We will keep it simple and just create two datasets, one for your root filesystem, and one for your home. ZFS is extremely flexible and you can easily shrink or extend it in the future.

zfs create -o mountpoint=/boot boot/gentoo
 * 1) Create your /boot dataset

zfs create tank/gentoo zfs create -o mountpoint=/ tank/gentoo/root zfs create -o mountpoint=/home tank/gentoo/home
 * 1) Create your / and other datasets

You can verify that all of these things worked by running the following:

zpool status zfs list

Now we are ready to install Gentoo!

Create your swap zvol
zfs create -o sync=always -o primarycache=metadata -o secondarycache=none -b 4k -V 4G -o logbias=throughput tank/swap mkswap -f /dev/zvol/tank/swap swapon /dev/zvol/tank/swap


 * Currently, the performance of the swap zvol isn't that great. This should improve over time.
 * You should have a swap even if it is small. Your system could lock up under high load, especially since ZFS uses your physical RAM for its ARC.

Check and set your time
date

If it's wrong then fix it. Let's say it's August 24, 2014 @ 2:48 PM (will be 14:48 in 24 hour time), we would do the following:

date 082414482014

08 24 1448 2014 (Month, Date, 24 Hour Time, Year). It will be UTC at the moment but we will fix it once we are inside our environment.

Preparing to chroot
cd /mnt/gentoo mkdir boot mount /dev/sda1 boot wget tar xpf

If you don't have a stage 3 for your architecture already downloaded, then click here.

Copy zpool cache
mkdir etc/zfs cp /etc/zfs/zpool.cache etc/zfs

Copy network settings
cp /etc/resolv.conf etc/

Mount required devices
mount --rbind /dev dev mount --rbind /proc proc mount --rbind /sys sys

Chroot into your environment
env -i HOME=/root TERM=$TERM chroot. bash -l

Create mtab symlink
ln -sf /proc/self/mounts /etc/mtab

Fix your date
I'm in New York so I will do the following:

ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime date 082412542014

Generate locales
nano /etc/locale.gen (I uncommented the en_US ISO/UTF-8 lines) locale-gen

Set hostname
nano /etc/conf.d/hostname

Edit fstab
nano /etc/fstab

Everything is on zfs so we don't need anything in here except for the swap zvol entry.

My fstab looks as follows: /dev/zvol/tank/swap    none            swap            sw              0 0

Modify make.conf
Let's modify our /etc/portage/make.conf so we can start installing stuff with a good base:

nano /etc/portage/make.conf

Example, change it to what you need:

ACCEPT_LICENSE="*"

CPU_FLAGS_X86="mmx sse sse2" USE="$CPU_FLAGS_X86 -branding"

GENTOO_SOURCES="http://xyinn.org/gentoo/ http://mirrors.rit.edu/gentoo/" PORTAGE_BINHOST="http://xyinn.org/gentoo/packages"

MAKEOPTS="-j5" EMERGE_DEFAULT_OPTS="--jobs=2 --load-average=2 --with-bdeps=y --quiet-build=y" FEATURES="buildpkg" LINGUAS="en en_US" VIDEO_CARDS="intel" INPUT_DEVICES="evdev synaptics"

Configure Portage, get the Portage Tree, and setup bliss-overlay
We will now configure portage, download the Portage Tree, and setup "bliss-overlay" through it. This means that you will be able to receive all bliss-overlay updates through portage.

cp /usr/share/portage/config/repos.conf /etc/portage/repos.conf
 * 1) Copy the example portage config so we can modify it

nano /etc/portage/repos.conf sync-uri = rsync://xyinn.org/portage
 * 1) Open repos.conf and switch the following value for the [gentoo] entry to the following:
 * 1) Add the following entry to the same file:

[bliss-overlay] location = /var/lib/overlays/bliss-overlay sync-type = git sync-uri = https://github.com/fearedbliss/bliss-overlay.git auto-sync = yes

emerge --sync
 * 1) Download the tree (You will get an error when getting bliss-overlay since you don't have git yet)

emerge dev-vcs/git
 * 1) Compile git

emerge --sync
 * 1) Sync once more to get bliss-overlay

Install required applications
At the time of writing, the 'libzfs' use flag is masked for GRUB 2 on the stable platform (It's fine on ~amd64). Let's unmask this so we can compile GRUB with zfs support:

mkdir /etc/portage/profile echo "sys-boot/grub -libzfs" >> /etc/portage/profile/package.use.mask

Unmask the latest versions of zfs: echo "sys-kernel/spl ~amd64" >> /etc/portage/package.accept_keywords echo "sys-fs/zfs ~amd64" >> /etc/portage/package.accept_keywords echo "sys-fs/zfs-kmod ~amd64" >> /etc/portage/package.accept_keywords

Now install the apps: emerge -ag bliss-kernel bliss-initramfs grub

'''Consider pinning the kernel version that was installed so that when we do a future kernel upgrade, an emerge --depclean won't delete our kernel files from the /boot directory. If the bliss-kernel ebuild that emerge pulled to install was: bliss-kernel-3.14.26, we would do:'''

emerge --noreplace =bliss-kernel-3.14.26

Update your system
emerge -uDNavg @world

''' The "g" option in the first emerge command will attempt to pull as many compatible binary packages from my server. This means you will have to compile from source a lot less packages. '''

Installing the bootloader onto your drive
We will need to install the bootloader, copy some files so we can get a menu, and also flash the firmware that will allow us to boot from gpt:

mkdir /boot/extlinux cd /boot/extlinux extlinux --install. cp /usr/share/syslinux/{libutil,menu}.c32. dd if=/usr/share/syslinux/gptmbr.bin of=/dev/sda

Configure bliss-boot
We will now configure bliss-boot.

'''This application isn't necessary, however, it helps you generate your initial bootloader configuration file. You can either use bliss-boot to regenerate future configurations, or you can use the first file it generates as a template for your own configurations.'''

nano /etc/bliss-boot/config.py

Navigate to the "Kernels & Options" section and modify the default entry that's there to look like the following (or similar depending your situation):

('Gentoo', '3.14.26-KS.01', 1, 'vmlinuz', 'initrd', 'root=tank/gentoo/root elevator=noop quiet'),

And switch the bootloader to extlinux so that bliss-boot generates that config type:

bootloader = "extlinux"

Generate the bootloader config
bliss-boot -o /boot/extlinux/extlinux.conf

''' Check your extlinux.conf to make sure that it has the correct information! '''

Generate the initramfs and move the file to its correct location
bliss-initramfs 1 3.14.26-KS.01 mv initrd-3.14.26-KS.01 /boot/kernels/3.14.26-KS.01/initrd

If all went well, extlinux should now be installed and configured.

Final steps before reboot
emerge -ag dhcpcd syslog-ng

rc-update add zfs-mount boot rc-update add dhcpcd default rc-update add sshd default rc-update add syslog-ng default

passwd exit reboot

Ensuring we have our /boot directory files inside our zfs snapshots
Since our /boot directory is not on zfs, any zfs snapshots we take will not contain the files in this directory (kernel, initramfs, bootloader config, etc). This is pretty important considering one of the reasons we've put our rootfs on zfs is so that in the case of a system failure, we could easily rollback to a good state.

What we can do is use the program 'lsyncd' so that it monitors our /boot directory and whenever a file gets added, removed, changed, it will update another location (/boot2). This /boot2 directory will reside on the zfs pool itself, which means our snapshots will contain this folder.

I've added the openrc script and an example config file (Which happens to be a /boot -> /boot2 example) to my overlay (ebuild was forked from Gentoo's portage tree).

emerge -a lsyncd::bliss-overlay rc-config start lsyncd rc-update add lsyncd default

If you check your / directory, you should see that you now have a /boot and /boot2 directory, with /boot2 just mirroring the files that are on /boot.

octopus bliss-overlay # ls /boot* /boot: extlinux kernels  lost+found

/boot2: extlinux kernels  lost+found

Take a snapshot of your new system
We will take a snapshot with zfs of the system in this state in case we ever want to rollback to it:

zfs snapshot tank/gentoo/root@install zfs snapshot tank/gentoo/home@install

Tools in Portage
emerge -ag sudo htop gentoolkit eix app-misc/screen

Bliss ZFS Scripts
The following scripts allow you to automatically take snapshots of your pool, automatically replicate the pool to another pool (Full and Incremental Backups), and clean the old snapshots on your pools.

You can download, customize, and install the scripts into your /usr/local/sbin directory. Github.

And that's it. Enjoy!