User:TitanOfOld/Advanced backup using rsnaphot
This article describes an advanced automated remote backup scheme using the tool rsnapshot as a non-root user, which is based on rsync.
rsnapshot makes a specified number of incremental backups of specified file trees from remote servers via ssh with non-user root using sudo, with help of hard links to save space on the backup medium.
The following backup scheme will log in to the source host via SSH, fetch all files with rsync to storage
host, rotate the backups on a daily, weekly, and monthly basis. That means, it will keep a daily snapshot for 7 days, a weekly snapshot for 4 weeks, and a monthly snapshot for 12 months.
Source Host
First, we set up the source host; the host whose files we want to backup. This host could be any machine on the network, Internet, or even the localhost. In the examples that follow, @source refers to this machine.
Requirements
On the source host, only three programs are needed:
Let's make sure they're all emerged:
root@source #
emerge --ask net-misc/openssh net-misc/rsync app-admin/sudo
Ensure that sshd is running and starts automatically:
root@source #
/etc/init.d/sshd start
root@source #
rc-update add sshd
Backup user
It's a good idea to have a non-root, machine (non-person) user specifically meant for the task. Name this user "backup".
root@source #
useradd -m backup
Set a password for the user:
root@source #
passwd backup
Sudo
In order to access a significant portion of files and directories, the backup user must have permissions to run rsync as root. Sudo can delegate root authority to non-root users, such as backup, to run some commands as root.
root@source #
visudo
...
backup ALL=(ALL) NOPASSWD: /usr/bin/rsync
...
This declares that the user backup can run the command /usr/bin/rsync as the root user on all the hosts. NOPASSWD is set so the task can run without user interaction.
Rsync wrapper
The storage host will connect to the source host as the backup user and execute backup@source:~/rsync-wrappper.sh. This let's the storage host connect as a nonroot user, but get just enough root-like permissions to backup the entire filesystem with the rsync command through sudo. Additionally, the logger statements records the action on the source host.
#!/bin/sh
/usr/bin/logger -t backup -- "Backup starting with rsync options: $@"
/usr/bin/sudo /usr/bin/rsync "$@"
/usr/bin/logger -t backup -- "Backup finished"
It needs to be executable, too:
backup@source $
chmod ug+x /home/backup/rsync-wrapper.sh
Storage Host
Now to set up the storage host. This host could be any machine on the network, Internet, or even the localhost. In the examples that follow, @storage refers to this machine, and it's assumed that @storage is the ultimate destination for the files from @source.
storage:/mnt/backup is where rsnapshot will work in and store the files.
An SSH key will be made and shared with the source hosts to enable passwordless and noninteractive access.
Requirements
On the storage host, these three packages need to be emerged:
root@storage #
emerge --ask app-backup/rsnapshot net-misc/openssh net-misc/rsync
Backup user
SSH keys, configurations for backup will be stored in backup user backup@storage
. Lets create those user and group
root@storage #
useradd -m backup
Directories
All backups will be saving to /mnt/backup directory. We will create backup directory
root@storage #
mkdir -p /mnt/backup
root@storage #
chown backup:backup /mnt/backup
root@storage #
chmod 770 /mnt/backup
SSH keys
rsnapshot will log in to remote servers via ssh public keys. Lets generate private/public ssh keys for all next ssh sessions.
root@storage #
sudo -i -u backup
backup@storage $
ssh-keygen -t ed25519
Save ssh keys to default path without password. After this, copy ssh key to remote server with ssh-copy-id:
backup@storage $
ssh-copy-id backup@source
And lets recheck, that everything is fine
backup@storage $
ssh source
No password should be asked and you simply login to backup@source.
rsnapshot
This configuration gives a good starting point. rsnapshot configuration files are tab delimited. Run the syntax checker after each edit to ensure it will run properly.
config_version 1.2
# Required settings.
cmd_cp /bin/cp
cmd_rm /bin/rm
cmd_rsync /usr/bin/rsync
cmd_ssh /usr/bin/ssh
ssh_args -i /home/backup/.ssh/id_ed25519
# If there's space available, this will first complete a successful backup
# before rotating. Uncomment to enable.
#sync_first 1
rsync_short_args -aAX
# Generally a good idea to just enable. This allows rsync to do some of the
# tasks cp would have performed, but with better support for non-Linux systems.
link_dest 1
# When enabled, will cross partitions. Disable (set to 0) to not.
one_fs 1
# Exclude pattern (refer to --exclude-from from rsync man page)
exclude /dev/*
exclude /proc/*
exclude /sys/*
exclude /run/*
exclude /var/tmp/*
exclude /var/run/*
exclude /tmp/*
exclude /var/cache/distfiles/*
exclude /lost+found
# Required. Local filesystem path to save all snapshots.
snapshot_root /mnt/backup/
# Prevent creating snapshot_root. This is especially useful when snapshot_root
# is mounted on a separate file system.
no_create_root 1
# Backup source path to destination path. Destination path is relative to
# snapshot_root.
# backup source destination options
#backup / local_root/ one_fs=0
backup backup@source:/ source/ +rsync_long_args=--rsync-path=/home/backup/rsync-wrapper.sh
# At least one of these must be declared. daily, weekly, and monthly don't
# actually mean anything. It could have been foo, bar, and foobar...but that'd
# be confusing.
retain daily 7
retain weekly 4
retain monthly 12
The file parametes to pay particular attention to are:
- rsync_short_args:
-a
: Archive mode is a brief way to express-rlptgoD
. It tells rsync to recurse into directories, copy symlinks as symlinks, preserve permissions, preserve modification times, preserve group, preserve owner, transfer character and block device files, and transfer special files, such as named sockets and fifos.-A
: preserve ACLs (this is fine to leave enabled even when ACLs aren't used)-X
: preserve extended attributes (this is fine to leave enabled even when extended attributes aren't used)
- +rsync_long_args: The + leaves the default parameters that'd be passed to rsync and adds:
rsync-path
to tell rsync to execute the wrapper script /home/backup/rsync-wrapper.sh on the source host
- ssh_args: Path to the private key created earlier to authenticate the log into backup@source
- snapshot_root: Path to top level directory where all backup files will be stored
- backup specifies a container directory for the backups, usually referring to the machine (in this case, source). This can be changed to any name of your choosing. The final snapshots will be saved under /mnt/backup/{daily,weekly,monthly}.[0-9]*/source/
- exclude: This directory will be excluded from backup
Once finished editing, check for syntax errors by running:
backup@storage $
rsnapshot configtest
rsnapshot will always take the last daily snapshot to create the first weekly snapshot and the last weekly snapshot to create the first monthly one. It will not take the 7th daily snapshot to create the first weekly snapshot. Therefore, it is possible to keep less or more than 7 daily snapshots, but is this case the first weekly snapshot is not one week old.
cron jobs
Add cron job to run backup:
backup@storage $
crontab -e
To have the backups include file owners, groups, permission, and other attributes, add the rsnaphost cron jobs to root's crontab instead:
root@storage #
crontab -e
# Refer to the retain labels defined in /etc/rsnapshot.conf
# 3am each day
0 3 * * * ionice -c 3 nice -n +19 /usr/bin/rsnapshot daily
# 4am each week
0 4 * * 1 ionice -c 3 nice -n +19 /usr/bin/rsnapshot weekly
# and 4am each month
0 4 1 * * ionice -c 3 nice -n +19 /usr/bin/rsnapshot monthly
rsnapshot jobs will run rsnapshot with minimum CPU and I/O priority.
Automate the backups via systemd timer units
Instead of crontab you can use systemd timers and service units. The .timer unit starts the .service unit.
This example timer starts daily at 11 am. Adapt to your needs and create one for daily, weekly, monthly, depending on your rsnapshot.conf
[Unit]
Description=rsnapshot daily backup
[Timer]
OnCalendar=*-*-* 11:00:00
Persistent=true
Unit=rsnapshot-daily.service
[Install]
WantedBy=timers.target
[Unit]
Description=rsnapshot daily backup
[Service]
User=backup
Type=oneshot
Nice=19
IOSchedulingClass=idle
ExecStart=/usr/bin/rsnapshot daily
As root, enable and set the timer unit active
root@storage #
systemctl enable --now rsnapshot-daily.timer
To find out how much time is left until systemd will activate the service the next time or when it ran the last time
root@storage #
systemctl list-timers
MySQL backup
This backup configuration are workable for small non-production databases, that doesn't have too many transactions. For more advanced MySQL backup, see https://dev.mysql.com/doc/refman/5.7/en/backup-methods.html (replication or Binary Log backup)
Login to backup user:
user@backup.example.com $
sudo -i -u backup
Create file .my.cnf with such content
[mysqldump]
host = localhost
port = 3306
user = backup
password = BACKUP_USER_PASSWORD
This file are used every time, when mysqldump tool will be called. Be sure, that only backup user have access to /home/backup/.my.cnf file
Create bash script mysql_dump.bash, that will for backup:
#/bin/bash
/usr/bin/mysqldump --all-databases | bzip2 -c > /backup/mysql/`date +%Y.%m.%d_%H.%M.%S.sql.bz2`
Add executable flag for script:
user@backup.example.com $
chmod +x /home/backup/mysql_dump.bash
Create directory /backup/mysql, that contain all MySQL backups and grand permissions:
root@backup.example.com #
mkdir -p /backup/mysql
root@backup.example.com #
chown backup /backup/mysql
root@backup.example.com #
chmod 700 /backup/mysql
Create MySQL user backup with access to all databases (like root user, but for backup)
root@backup.example.com #
mysql
and type:
mysql>
GRANT ALL PRIVILEGES ON *.* TO 'backup'@'localhost' IDENTIFIED BY 'BACKUP_USER_PASSWORD'
Last step - create cron job, that will call mysql_dump.bash script and dump all databases. Execute from backup user
backup@backup.example.com $
crontab -e
and add such line (run every day at 01:00)
0 1 * * * /home/backup/mysql_dump.bash
PostgreSQL backup
While PostgreSQL data files should be included with the rsnapshot above, it is a good idea to maintain distinct backups of the database for easy restoration upon data entry error, corruption, or other failure (human or otherwise) affecting the database alone.
The Backup and Restore chapter of the official PostgreSQL documentation has a variety of examples available to take a quick snapshot. However, the PostgreSQL wiki article Automated Backup on Linux has a comprehensive backup solution. It follows a similar retention policy as rsnapshot, and will save each database to its own file.
Restoration
To restore the source backups specified above, we would use:
root #
mount /mnt/backup
root #
rsync -aAX /mnt/backup/monthly.0/source/ /mnt/myroot
root #
rsync -aAX /mnt/backup/weekly.0/source/ /mnt/myroot
root #
rsync -aAX /mnt/backup/daily.0/source/ /mnt/myroot
Where /mnt/myroot is the mount point of the fresh root filesystem. In the paths above, the .0 refers to the latest increment.
If backups are on remote server, rsync can be done via ssh:
root #
rsync -aAX root@storage:/mnt/backup/monthly.0/source/ /mnt/myroot
MySQL
Technically, MySQL dumps are just bzipped text files of SQL commands for the MySQL database. This command will unzip archive, and send SQL command to mysql:
root #
bzcat /backup/mysql/some_ziped_archive.bz2 | mysql --user=root
PostgreSQL
See the chapter Restoring the Dump in the official documentation. In general, after following the PostgreSQL/QuickStart guide and recreating the database dbname, restore from the latest backup with:
user $
psql --set ON_ERROR_STOP=on dbname < dbname-dumpfile
Possible improvements
- Use remote device for storing backups /mnt/backup - TO BE DONE
- Use encryption to crypt backups /mnt/backup - TO BE DONE