User:Maffblaster/Drafts/bcachefs
bcachefs is a next generation, robust, high performance Linux filesystem supporting CoW, device tiering, compression, and encryption.
Presently bcachefs is not ready to be distributed in the upstream Linux kernel. Those who are interested in using bcachefs must patch the Linux kernel in order to add support.
Installation
Prerequisites
Obtain the patched kernels sources.
Right now it is easier for bcachefs' developer Ken Overstreet to maintain kernel sources with the bcachefs already applied.
Move to the kernel sources directory on Gentoo, then clone the sources (dev-vcs/git is required):
root #cd /usr/src
root #git clone --depth=1 https://evilpiepirate.org/git/bcachefs.git bcachefs.git
Build the kernel
The bcachefs kernel sources will need to be built. With the exceptions of defining any new kernel features and enabling CONFIG_BCACHE_FS, the kernel can be built using the currently running kernel's configuration:
root #zcat /proc/config.gz > /usr/src/bcachefs.git/.config
root #make oldconfig
Enable CONFIG_BCACHE_FS:
root #make menuconfigBuild (and install) the kernel, then update the bootloader (GRUB2 presumed):
root #make && make install
root #grub-mkconfig -o /boot/grub/grub.cfg
Emerge
There are presently no USE flags for the sys-fs/bcache-tools package, so simply emerge the package:
root #emerge --ask sys-fs/bcache-toolsThis is for bcache-tools package which is associated with hybrid disk configuraiton. This package has been requested with a proposed ebuild via bug #657120.
Tutorial by Tuxie
I don't have all the exact commands ready, but here is the gist of it. Very quick and dirty hacked together guide, you'll need to RTFM to fill in the gaps
First, all disks including the cache ssd are formatted with
cryptsetup luksFormat -d /etc/luks.pwd /dev/sdX
where /etc/luks.pwd is the passfile (a sha256 hash of some random data), and they are all added to /etc/crypttab like
ssd01_crypt UUID=xxxxxxxxxxxx /etc/luks.pwd luks,discard hdd01_crypt UUID=xxxxxxxxxxxx /etc/luks.pwd luks hdd02_crypt UUID=xxxxxxxxxxxx /etc/luks.pwd luks
Start them with cryptdisks_start hdd01_crypt etc.
Then the cache SSD is formatted with make-bcache --cache --writeback --discard /dev/mapper/ssd01_crypt and the data HDDs with make-bcache --bdev /dev/mapper/hdd01_crypt etc. Attach each bdev to the cache by echoing the UUID of the cache dev to /sys/block/dm-*/bcache/attach. You'll find the UUID in /sys/fs/bcache/.
Format all the /dev/bcacheN devices with {[c|mkfs.xfs /dev/bcache0}} or your filesystem of choice and add them by their UUID to /etc/fstab. I mount them as /raw/hdd01 etc. My fstab looks like (for the first 3 disks):
UUID=191f1a4d-5885-49dd-8b68-ebdc234e1078 /raw/hdd01 xfs noatime 0 0 UUID=f4e1f378-1253-4dd6-a1f1-fdfa70b258f0 /raw/hdd02 xfs noatime 0 0 UUID=b9537fc9-1a68-41b5-af37-6e1e209522da /raw/hdd03 xfs noatime 0 0
Between all steps, make sure to look at lsblk -f to see which disk belongs to which bdev and crypt dev and to find the UUIDs. You may need to run udevadm trigger some times to make changes visible without rebooting.
In every /raw/hdd*/ I have created a directory named "pool" which is the root of the mergerfs pool (sys-fs/mergerfs, sys-fs/mergerfs-tools). After a few years of tweaking and debugging weird behaviours of some programs when using the pool, I have landed on this mergerfs config (in /etc/fstab), which is optimized for compatibility rather than performance. For me, this config Just Works for everything I use it for and the performance is Good Enough. YMMV. Code:
/raw/hdd*/pool /pool fuse.mergerfs allow_other,use_ino,noatime,moveonenospc=true,ignorepponrename=true,link_cow=true,cache.files=off,dropcacheonclose=true,category.search=newest,xattr=passthrough,noforget,security_capability=true,posix_acl=false,minfreespace=50G,category.create=msplfs,hard_remove,fsname=mergerfs,_netdev 0 0
Then finally create an /etc/snapraid.conf with your /raw disks like:
data hdd01 /raw/hdd01/pool/ data hdd02 /raw/hdd02/pool/ data hdd03 /raw/hdd03/pool/ parity /raw/hdd11/snapraid.parity content /raw/hdd11/snapraid.content
and add some exclude-rules for incomplete downloads, tempfiles and similar. Then schedule snapraid (sys-fs/snapraid) sync in crontab to run however often you want.
BTW, I use disk -> bcache -> luks -> xfs -> mergerfs myself, but this guide is for disk -> luks -> bcache -> xfs -> mergerfs, which is how I would do it if I were to start over. Also make sure that the root filesystem where you put /etc/luks.pwd is encrypted!