User:Pietinger/temp/test

From Gentoo Wiki
Jump to:navigation Jump to:search

Hi @xarvatium,

I am highly interested in a bcachefs-solution for the root partition with two SSDs as RAID-1 (and with encryption) ... but at the moment I am not able to test it ... I have wrote an embedded initramfs which should be able to mount bcachefs correctly ... BUT IT IS NOT TESTED (because I cant). If you want try it ... (If you want you can edit also this page of course)

This is highly experimental and meant only for @xarvatium who want test bcachefs as filesystem for the root partition using a RAID-1 (with two harddisks or SSDs).

This will work only for a manually configured kernel NOT already using an initramfs from dracut (or genkernel).

I have a stable (AMD64) Intel system (last synced yesterday and it is a split-usr system with OpenRC.

You will need a STATIC "busybox" and (of course) "bcachefs-tools-1.4.0"

(USE="-pam static static-libs" emerge -pvD busybox)


Please check with "ldd /sbin/bcachefs" if you have the same bindings as I have - especially GCC ... (if you are on unstable you will have 14) ... maybe some librarys dont exist on your machines ... then delete the line. Please check also the softlinks ... ;-)

# mkdir /usr/src/initramfs
# nano -w /usr/src/initramfs/initramfs_list
=>
dir /bin        755 0 0
dir /dev        755 0 0
dir /etc        755 0 0
dir /lib        755 0 0
dir /lib64      755 0 0
dir /mnt        755 0 0
dir /mnt/root   755 0 0
dir /proc       755 0 0
dir /root       700 0 0
dir /sbin       755 0 0
dir /sys        755 0 0
dir /usr        755 0 0
dir /usr/bin    755 0 0
dir /usr/lib64  755 0 0
dir /var        755 0 0
nod /dev/console 0600 0 0 c 5 1
file    /init                           /usr/src/initramfs/init         755 0 0
file    /bin/busybox                    /bin/busybox                    755 0 0
file    /sbin/bcachefs                  /sbin/bcachefs                  755 0 0
file    /lib64/libblkid.so.1.1.0        /lib64/libblkid.so.1.1.0        755 0 0
file    /lib64/libuuid.so.1.3.0         /lib64/libuuid.so.1.3.0         755 0 0
file    /usr/lib64/liburcu-common.so.8.1.0  /usr/lib64/liburcu-common.so.8.1.0  755 0 0
file    /usr/lib64/liburcu.so.8.1.0     /usr/lib64/liburcu.so.8.1.0     755 0 0
file    /usr/lib64/libsodium.so.26.1.0  /usr/lib64/libsodium.so.26.1.0  755 0 0
file    /lib64/libz.so.1.3              /lib64/libz.so.1.3              755 0 0
file    /usr/lib64/liblz4.so.1.9.4      /usr/lib64/liblz4.so.1.9.4      755 0 0
file    /lib64/libzstd.so.1.5.5         /lib64/libzstd.so.1.5.5         755 0 0
file    /lib64/libudev.so.1.7.7         /lib64/libudev.so.1.7.7         755 0 0
file    /lib64/libkeyutils.so.1.10      /lib64/libkeyutils.so.1.10      755 0 0
file    /lib64/libm.so.6                /lib64/libm.so.6                755 0 0
file    /lib64/libaio.so.1.0.2          /lib64/libaio.so.1.0.2          755 0 0
file    /usr/lib/gcc/x86_64-pc-linux-gnu/13/libgcc_s.so.1   /usr/lib/gcc/x86_64-pc-linux-gnu/13/libgcc_s.so.1   755 0 0
file    /lib64/libc.so.6                /lib64/libc.so.6                755 0 0
file    /lib64/ld-linux-x86-64.so.2     /lib64/ld-linux-x86-64.so.2     755 0 0
file    /lib64/libcap.so.2.69           /lib64/libcap.so.2.69           755 0 0
slink   /sbin/fsck.bcachefs             /sbin/bcachefs                  777 0 0
slink   /sbin/mkfs.bcachefs             /sbin/bcachefs                  777 0 0
slink   /sbin/mount.bcachefs            /sbin/bcachefs                  777 0 0
slink   /lib64/libblkid.so.1            /lib64/libblkid.so.1.1.0        777 0 0
slink   /lib64/libuuid.so.1             /lib64/libuuid.so.1.3.0         777 0 0
slink   /usr/lib64/liburcu-common.so.8  /usr/lib64/liburcu-common.so.8.1.0  777 0 0
slink   /usr/lib64/liburcu.so.8         /usr/lib64/liburcu.so.8.1.0     777 0 0
slink   /usr/lib64/liburcu-common.so.8  /usr/lib64/liburcu-common.so.8.1.0  777 0 0
slink   /usr/lib64/libsodium.so.26      /usr/lib64/libsodium.so.26.1.0  777 0 0
slink   /lib64/libz.so.1                /lib64/libz.so.1.3              777 0 0
slink   /usr/lib64/liblz4.so.1          /usr/lib64/liblz4.so.1.9.4      777 0 0
slink   /lib64/libzstd.so.1             /lib64/libzstd.so.1.5.5         777 0 0
slink   /lib64/libudev.so.1             /lib64/libudev.so.1.7.7         777 0 0
slink   /lib64/libkeyutils.so.1         /lib64/libkeyutils.so.1.10      777 0 0
slink   /lib64/libaio.so.1              /lib64/libaio.so.1.0.2          777 0 0
slink   /lib64/libcap.so.2              /lib64/libcap.so.2.69           777 0 0
<=

# nano -w /usr/src/initramfs/init
=>
#!/bin/busybox sh

### CHANGE THIS !
myrootpartition="/dev/nvme0n1p3:/dev/nvme1n1p3"

abend() {
    echo "$@"
    echo "You are now in a rescue shell."
    busybox --install -s
    exec /bin/sh
}

echo "Mounting proc, sys, and devtmpfs ..."
mount -t devtmpfs none /dev || abend "Error: mount /devtmpfs failed !"
mount -t proc none /proc || abend "Error: mount /proc failed !"
mount -t sysfs none /sys || abend "Error: mount /sysfs failed !"

echo "Mounting root partition ..."
/sbin/mount.bcachefs $myrootpartition /mnt/root || abend "Error mounting root partition !"

echo "Unmounting proc, sys, and devtmpfs ..."
umount /proc /sys /dev

echo "All done. Switching to real root."
exec switch_root /mnt/root /sbin/init
<=

# cd /usr/src/linux
# make menuconfig
=>
General setup  --->
    [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
    (/usr/src/initramfs/initramfs_list) Initramfs source file(s)
    [*]   Support initial ramdisk/ramfs compressed using gzip
    Built-in initramfs compression mode (Gzip)  --->
# make
# make modules-install
(# mount /boot)  (If old system without /efi ?)
# make install OR copy to ESP - depending if you are using grub or booting the kernel directly from UEFI
# IF grub: grub-mkconfig...

-> Add your second harddisk (/partition of second harddisk) to bcachefs ... and try a reboot with this kernel

Please dont shout to me if something is wrong ;-)

(I made the initramfs only theoretical without test)

....arrggss ... I have forgot busybox itself ... now it is in the file-list ...

Jumping in as I'm interested in this too
This looks like it should work, I'll chuck it in a VM as I want to the hook at https://github.com/breavyn/dracut-bcachefs/tree/master for encryption and there is maybe a chance we can hack that to let booting work until grub gets bcachefs support.
Immolo (talk) 01:56, 13 January 2024 (UTC)

Hi @immolo, great you are in !


Preliminary test: What happens if a hard disk of a (encrypted) bcache RAID1 fails ?

I have formatted two unused partitions with bcachefs:

# bcachefs format /dev/sda1 /dev/sda2 --replicas=2 --encrypt --wide_macs --acl --verbose
Enter passphrase: 
Enter same passphrase again:
# mkdir /mnt/bc

After this I had this problem:

# mount.bcachefs /dev/sda1:/dev/sda2 /mnt/bc
ERROR - bcachefs_rust::cmd_mount: Fatal error: Required key not available

I found this: https://lore.kernel.org/all/12360099.O9o76ZdvQC@lichtvoll.de/T/ and did it:

# keyctl link @u @s
# mount.bcachefs /dev/sda1:/dev/sda2 /mnt/bc

Now it worked. I copied a file into it and made a subvolume:

 /mnt/bc # bcachefs subvolume create /mnt/bc/etc
 /mnt/bc # lal
insgesamt 8
drwxr-xr-x 3 root root    0 19. Jan 20:59 .
drwxr-xr-x 6 root root 4096 19. Jan 20:42 ..
-rw-r--r-- 1 root root  342 19. Jan 20:58 BCACHEFS.txt
drwxr-xr-x 2 root root    0 19. Jan 20:59 etc
drwx------ 2 root root    0 19. Jan 20:56 lost+found

THEN I destoyed /dev/sda1

# cd
# umount /mnt/bc/
# mkfs.ext4 /dev/sda1

NOW: Can I mount /dev/sda2 ... and HOW ?

# mount.bcachefs /dev/sda1:/dev/sda2 /mnt/bc
bcachefs (/dev/sda1): error reading default superblock: Not a bcachefs superblock
bcachefs (/dev/sda1): error reading superblock: Not a bcachefs superblock layout
ERROR - bcachefs_rust::cmd_mount: Fatal error: "invalid_sb_layout"

NOPE

# mount.bcachefs -o degraded /dev/sda1:/dev/sda2 /mnt/bc
bcachefs (/dev/sda1): error reading default superblock: Not a bcachefs superblock
bcachefs (/dev/sda1): error reading superblock: Not a bcachefs superblock layout
ERROR - bcachefs_rust::cmd_mount: Fatal error: "invalid_sb_layout"

NOPE

# mount.bcachefs /dev/sda2 /mnt/bc
ERROR - bcachefs_rust::cmd_mount: Fatal error: Invalid argument

NOPE

# mount.bcachefs -o degraded /dev/sda2 /mnt/bc
# cd /mnt/bc
/mnt/bc # lal
insgesamt 8
drwxr-xr-x 3 root root    0 19. Jan 20:59 .
drwxr-xr-x 6 root root 4096 19. Jan 20:42 ..
-rw-r--r-- 1 root root  342 19. Jan 20:58 BCACHEFS.txt
drwxr-xr-x 2 root root    0 19. Jan 20:59 etc
drwx------ 2 root root    0 19. Jan 20:56 lost+found

Aaahh ... :-) ... AND ... there is no second question for my password ... SO ... How do I programm it in my init-file of my initramfs ? Before I will try it on my test partitions:

# nano test.sh
=>
#!/bin/sh

### CHANGE THIS !
myrootpartition="/dev/sda1:/dev/sda2"

abend() {
    echo "$@"
    echo "You are now in a rescue shell."
    busybox --install -s
    exec /bin/sh
}

third_try() {
    echo "There was again a problem. Trying last chance !"
    /sbin/mount.bcachefs -o degraded /dev/sda2 /mnt/bc
    exit
}

second_try() {
    echo "There was a problem. Trying something different !"
    /sbin/mount.bcachefs -o degraded /dev/sda1 /mnt/bc || third_try
    exit
}

#echo "Mounting proc, sys, and devtmpfs ..."
#mount -t devtmpfs none /dev || abend "Error: mount /devtmpfs failed !"
#mount -t proc none /proc || abend "Error: mount /proc failed !"
#mount -t sysfs none /sys || abend "Error: mount /sysfs failed !"

echo "Mounting root partition ..."
/sbin/mount.bcachefs $myrootpartition /mnt/bc || second_try

#echo "Unmounting proc, sys, and devtmpfs ..."
#umount /proc /sys /dev
<=

You see, this is NOT a busybox-script ... you have to change something ;-) Giving it a +x and run it

# ./test.sh 
Mounting root partition ...
bcachefs (/dev/sda1): error reading default superblock: Not a bcachefs superblock
bcachefs (/dev/sda1): error reading superblock: Not a bcachefs superblock layout
ERROR - bcachefs_rust::cmd_mount: Fatal error: "invalid_sb_layout"
There was a problem. Trying something different !
bcachefs (/dev/sda1): error reading default superblock: Not a bcachefs superblock
bcachefs (/dev/sda1): error reading superblock: Not a bcachefs superblock layout
ERROR - bcachefs_rust::cmd_mount: Fatal error: "invalid_sb_layout"
There was again a problem. Trying last chance !
 ~ # 
 ~ # cd /mnt/bc/
 /mnt/bc # ll
insgesamt 4
-rw-r--r-- 1 root root 342 19. Jan 20:58 BCACHEFS.txt
drwxr-xr-x 2 root root   0 19. Jan 20:59 etc
drwx------ 2 root root   0 19. Jan 20:56 lost+found

Okay ... now the last problem: This does not look very nice (Maybe some programmers are dying now ...). For now it was only an alpha test ...