User:StefanLangenmaier/Poor man's Cluster/Host

Host

As a host Gentoo with the following features will be used:

lxc/docker
btrfs
kexec
dd-client (dyndns)
ntp-client
iptables
libvirtd
sshd
dmcrypt
snapper

Kexec

With btrfs we can already do root filesystem snapshots which can be used to roll back any updates to the system. There is one missing point - the active kernel. To have a safe way to try a kernel update and to roll back in case of error we're going to use kexec.

With kexec we can load a kernel that will replace the current kernel without doing a full hardware reboot. The running system will restart but only to start with the selected kernel. This way we can test any kernel and in case of a problem we do a hardware reboot and the known working kernel will loaded as usual.

You can find example of how to work with kexec on the Kernel_Crash_Dumps page, as of kernel 4.3 I had problems specifying the command line. (If the command line was specified the new kernel hung on the restart.) My workaround is to compile the whole command line directly into the kernel. This way the execution is as simple as the following to try out safely a new kernel:

$ mount /boot
$ kexec -l /boot/zImage-with-cmdline-builtin
$ umount /boot
$ kexec -e

Dm-crypt

The data will not be on the internal SD-card but external harddisks which are encrypted. The keys will be on the SD-card.

In the file /etc/conf.d/dmcrypt

source='/dev/sda1'
key='/etc/keys/external-usb-disks.key'

SSHD

Nowadays the default configuration no longer allows root login with just a password. This is good so you can only use a certificate/public-key based login. But if you need to login from machines where you haven't uploaded the key from you have a problem.

Here you can use two factor authentication (2FA). There is already documentation for the Google Authenticator.

You can find the repo here: https://github.com/stefan-langenmaier/google-authenticator-overlay

iptables

libvirtd creates the nat network for the containers. To forward an external port to lxc container behind a nat network.

#once for each port that should be forwarded
iptables -I FORWARD -d 192.168.122.71/32 -p tcp -m state --state NEW -m tcp --dport 9091 -j ACCEPT
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 9091 -j DNAT --to 192.168.122.71:9091

Snapper

snapper needs dbus running so don't forget to

/etc/init.d/dbus start
rc-update add dbus default

Snapper uses always the folder /.snapshots to store the snapshots. This is not configurable and it has to be a subvolume itself. This might be a good setup for a Suse system but the btrfs devs propose another setup.[1] This way the original subvolume can be deleted without losing all the created snapshots.

Tips and tricks

Portage

Automatic snapshots before and after the installation of package with portage can be taken with the script from here [3].

[1] : https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Managing_Snapshots

[2] : https://wiki.archlinux.org/index.php/Snapper

[3] : https://rich0gentoo.wordpress.com/2013/11/26/btrfs-and-snapper-with-portage-on-gentoo/