User:Massimo B./NewBtrbkDevice
From Gentoo Wiki
< User:Massimo B.(Redirected from User:Massimo B./Btrbk-setup)
Jump to:navigation
Jump to:search
Migration to a new btrfs for btrbk
A central LUKS-encrypted btrfs on a mobile disk is receiving snapshots from different machines via btrbk. This mobile disk is always with me when working on different machines. Additionally this mobile btrfs also contains a mobile bootable OpenSuse installation, also snapshotting via snapper.
This guide shows how to create a new btrfs and migrating or restoring from a damaged old btrfs (like in User:Massimo_B./Btrfs-repairing-01).
After partitioning, create a new LUKS device:
root #cryptsetup luksFormat --cipher aes-xts-plain64 --key-size 512 --hash sha512 --iter-time 5000 --pbkdf argon2id --pbkdf-memory $((4*1024*1024)) --pbkdf-parallel 4 /dev/sda1Make a backup of the LUKS header:
root #cryptsetup luksHeaderBackup /dev/disk/by-uuid/263b5499-c4bd-422f-8865-c33670de2ca6 --header-backup-file /.rescue/263b5499-c4bd-422f-8865-c33670de2ca6.headerAllowing discard and make that default in the LUKS header:
root #cryptsetup luksOpen --allow-discards --persistent /dev/sda1 newlocaldata_cryptMake a backup of the tablekey:
root #dmsetup table --showkeys | grep 263b5499-c4bd-422f-8865-c33670de2ca6 > /.rescue/263b5499-c4bd-422f-8865-c33670de2ca6.tablekeyMount new data:
root #mount -o compress-force=zstd:19,subvol=/ /dev/mapper/newlocaldata_crypt /mnt/local/newdata/Create directory tree for btrbk:
root #mkdir -vp /mnt/local/newdata/snapshots/{machineA,machineB,machineC,machineD}/{root,home,data,vm} Restoring from a damaged RO btrfs:
Rescue-mount old data read-only:
root #mount -o ro,rescue=all,subvol=/ /dev/mapper/localdata_crypt /mnt/local/data/Btrbk snapshots
Transferring btrbk-snapshots, using btrbk archive for parent relationships. Take care to set archive_preserve in btrbk.conf large enough like 10y to cover all snapshots.
root #for machine in machineA,machineB,machineC,machineD;do for source in root home data vm;do btrbk --progress archive /mnt/local/data/snapshots/$machine/$source/ /mnt/local/newdata/snapshots/$machine/$source/;done;doneCompare counts of subvolumes:
root #for machine in machineA,machineB,machineC,machineD;do echo "$machine:";ls -ald /mnt/local/data/snapshots/$machine/*/* | wc -l;done
root #for machine in machineA,machineB,machineC,machineD;do echo "$machine:";ls -ald /mnt/local/newdata/snapshots/$machine/*/* | wc -l;doneOther snapshots
Send single snapshots not covered by btrbk manually:
root #btrfs send /mnt/local/data/snapshots/machineC/volumes.root/boot.20200810T010500+0200/ | mbuffer -v 1 -m 5% | btrfs receive /mnt/local/data/snapshots/machineC/volumes.root/Other stuff via rsync
Transferring other stuff not inside subvolumes:
root #for mydir in ISO rescue tmp data; do rsync -avz --delete /mnt/local/data/"$mydir"/ /mnt/local/newdata/"$mydir"/; doneOpenSuse
Transferring an OpenSuse mobile installation. btrfs send / receive is not allowed for rw subsolumes, even if the mount is ro (which should be changed in my opinion). I can't make ro snapshots on a ro mounted btrfs either. So the only way is rsync, excluding the insanly nested subs on OpenSuse:
root #btrfs subvolume create -p /mnt/local/newdata{@,@/var,@/usr/local,@/srv,@/opt,@/home,@/.snapshots/1/snapshot} Transfer things not in insane nested subvolumes:
root #for mypath in @/var @/usr/local @/srv @/opt @/home @/.snapshots/1/snapshot;do rsync -avz --delete --one-file-system /mnt/local/data/"$mypath"/ /mnt/local/newdata/"$mypath"/;doneRemove empty directories:
root #for mydir in /mnt/local/data/@/.snapshots/1/snapshot/.snapshots/*/snapshot/;do rmdir -v /mnt/local/newdata/"${mydir#\/mnt\/local\/data\/}";doneSend the read-only snapshots:
root #for mysnap in /mnt/local/data/@/.snapshots/1/snapshot/.snapshots/*/snapshot/;do mysnap2="${mysnap#\/mnt\/local\/data\/}";btrfs send "$mysnap" | mbuffer -v 1 -m 5% | btrfs receive /mnt/local/newdata/"${mysnap2%/snapshot/}";doneConfiguration
FILE
/etc/btrbk/btrbk.confReal-life btrbk config, as an example# vim: set shiftwidth=4 softtabstop=4 expandtab foldlevel=0 foldmethod=indent: # vim-modline, don't delete
#############################################################################
# $HGFile: etc/btrbk/btrbk.conf $
# $HGLocation: / $
# 1st Author: Massimo
# Created: 2017
# $HGChecked in by: massimo $
# $HGDate: 2024-10-10 07:56 +0200 $
# $HGRevision: 1065 $
# $HGBranch: mobalindesk_3 $
# $HGDesc: updates $
#############################################################################
#send_compressed_data no
#send_protocol 1
archive_preserve 10y
archive_preserve_min latest
incremental yes
lockfile /var/log/btrbk/btrbk.lock
preserve_day_of_week monday
snapshot_create onchange
snapshot_preserve 14d 5w 2m
snapshot_preserve_min 1w
ssh_identity /root/.ssh/id_ed25519
stream_buffer 5%
stream_buffer_remote 5%
stream_compress zstd
stream_compress_adapt no
stream_compress_level default
stream_compress_long default
stream_compress_threads 0
target_preserve_min 1w
timestamp_format long-iso
transaction_log /var/log/btrbk/btrbk.log
###################################################################################
volume /mnt/btrfs-top-lvl/
group all,cron,local
#-------------------------------------
subvolume volumes/root
group root
snapshot_dir snapshots/root
target send-receive /mnt/local/data/snapshots/mobalindesk/root
group localdata
target_preserve 30d 8w 12m 3y
target send-receive /mnt/usb/mobiledata/snapshots/mobalindesk/root
group mobiledata
target_preserve 30d 8w 12m 3y
target send-receive ssh://mobalintop/mnt/archive/mobalindesk/root
group archive
target_preserve 4w
#-------------------------------------
subvolume volumes/home
group home
snapshot_dir snapshots/home
target send-receive /mnt/local/data/snapshots/mobalindesk/home
group localdata
target_preserve 30d 24w 12m 8y
target send-receive /mnt/usb/mobiledata/snapshots/mobalindesk/home
group mobiledata
target_preserve 30d 24w 12m 8y
target send-receive ssh://mobalintop/mnt/archive/mobalindesk/home
group archive
target_preserve 4w
#-------------------------------------
subvolume volumes/data
group data
snapshot_dir snapshots/data
target send-receive /mnt/local/data/snapshots/mobalindesk/data
group localdata
target_preserve 6d 8w 12m 2y
target send-receive /mnt/usb/mobiledata/snapshots/mobalindesk/data
group mobiledata
target_preserve 6d 8w 12m 2y
#target send-receive ssh://mobalintop/mnt/archive/mobalindesk/data
# group archive
# target_preserve 4w
#-------------------------------------
subvolume volumes/vm
group vm
snapshot_dir snapshots/vm
target send-receive /mnt/local/data/snapshots/mobalindesk/vm
group localdata
target_preserve 6m 1y
target send-receive /mnt/usb/mobiledata/snapshots/mobalindesk/vm
group mobiledata
target_preserve 6m 1y
#target send-receive ssh://mobalintop/mnt/archive/mobalindesk/vm
# group archive
# target_preserve 4w
###################################################################################
volume ssh://mobalintop/mnt/btrfs-top-lvl/
send_compressed_data yes
send_protocol 2
group all,cron,remote,ssh,mobalintop
#-------------------------------------
subvolume volumes/root
group root
snapshot_dir snapshots/root
target send-receive /mnt/local/data/snapshots/mobalintop/root
group localdata
target_preserve_min latest
target_preserve 30d 8w 12m 3y
target send-receive /mnt/usb/mobiledata/snapshots/mobalintop/root
group mobiledata
target_preserve 30d 8w 12m 3y
#-------------------------------------
subvolume volumes/home
group home
snapshot_dir snapshots/home
target send-receive /mnt/local/data/snapshots/mobalintop/home
group localdata
target_preserve_min latest
target_preserve 30d 24w 12m 8y
target send-receive /mnt/usb/mobiledata/snapshots/mobalintop/home
group mobiledata
target_preserve 30d 24w 12m 8y
#-------------------------------------
subvolume volumes/data
group data
snapshot_dir snapshots/data
target send-receive /mnt/local/data/snapshots/mobalintop/data
group localdata
target_preserve_min latest
target_preserve 6d 8w 12m 2y
target send-receive /mnt/usb/mobiledata/snapshots/mobalintop/data
group mobiledata
target_preserve 6d 8w 12m 2y
#-------------------------------------
subvolume volumes/vm
group vm
snapshot_dir snapshots/vm
target send-receive /mnt/local/data/snapshots/mobalintop/vm
group localdata
target_preserve_min latest
target_preserve 6m 1y
target send-receive /mnt/usb/mobiledata/snapshots/mobalintop/vm
group mobiledata
target_preserve 6m 1y
###################################################################################
# Reverse Backups
###################################################################################
#volume /mnt/usb/mobiledata
# group all,cron,reverse,mobiledata
# snapshot_create no
# #-------------------------------------
# subvolume snapshots/gentoo/root
# target send-receive /mnt/local/data/snapshots/gentoo/root
# target_preserve_min latest
# target_preserve 2w
# #-------------------------------------
# subvolume snapshots/gentoo/home
# target send-receive /mnt/local/data/snapshots/gentoo/home
# target_preserve_min latest
# target_preserve 2w
# #-------------------------------------
# subvolume snapshots/gentoo/data
# target send-receive /mnt/local/data/snapshots/gentoo/data
# target_preserve_min latest
# target_preserve 2w
# #-------------------------------------
# subvolume snapshots/gentoo/vm
# target send-receive /mnt/local/data/snapshots/gentoo/vm
# target_preserve_min latest
# target_preserve 2w
# #-------------------------------------