User:TitanOfOld/Advanced backup using rsnaphot

Warning, this page is a work in progress by TitanOfOld (talk | contribs). Treat its contents with caution.

Resources

This article [[Article description::describes a advanced automated remote backup scheme using the tool rsnapshot as non-root user|Article description::describes an advanced automated remote backup scheme using the tool rsnapshot as non-root user]], which is based on rsync.

rsnapshot makes a specified number of incremental backups of specified file trees from remote servers via ssh with non-user root using sudo, with help of hard links to save space on the backup medium.

The following backup scheme will log in to the source host via SSH, fetch all files with rsync to storagehost, rotate the backups on a daily, weekly, and monthly basis. That means, it will keep a daily snapshot for 7 days, a weekly snapshot for 4 weeks, and a monthly snapshot for 12 months.

Source Host

First, set up the source host. This host could be any machine on the network, Internet, or even the localhost. In the examples that follow, @source refers to this machine.

Requirements

On the source host, only three programs are needed:

Rsync
Sudo
SSH

Rsync and SSH are most likely already installed. Ensure that SSH is running

Backup user

It's a good idea to have a non-root, machine (non-person) user specifically meant for the tasks. Name this user "backup".

root@source #useradd -m backup

Set a password for the user

root@source #passwd backup

Sudo

In order to access a significant portion of files and directories, the backup user must have permissions to run rsync as root. Sudo can delegate root authority to non-root users, such as backup, run some commands as root.

root@source #visudo

FILE root@source:/etc/sudoers

...
backup ALL=(ALL) NOPASSWD: /usr/bin/rsync
...

User backup can run the command /usr/bin/rsync as the root user on all the hosts. NOPASSWD is set so the task can run without user interaction.

Rsync wrapper

The storage host will connect to the source host as backup, and execute backup@source:~/rsync-wrappper.sh. This wrapper requires for sudo. Lets create those dummy wrapper script

FILE backup@source:/home/backup/rsync-wrapper.shScript that rsnapshot will run when it logs into the source host

#!/bin/sh
 
logger -t backup $@
/usr/bin/sudo /usr/bin/rsync "$@";

And give executable flag for those script

backup@source $chmod ug+x /home/backup/rsync-wrapper.sh

Storage Host

Now to set up the storage host. This host could be any machine on the network, Internet, or even the localhost. In the examples that follow, @storage refers to this machine, and it's assumed that @storage is the ultimate destination for the files from @source.

Requirements

rsync

rsnapshot

SSH

Emerge app-backup/rsnapshot on the storage server:

root #emerge --ask app-backup/rsnapshot

Backup server will connect to backup@remote.example.com server via ssh public key. All backup files will be save to /mnt/backup directory.

Backup user

SSH keys, configurations for backup will be stored in backup user backup@backup.example.com. Lets create those user and group

root@storage #useradd -m backup

Directories

All backups will be saving to /mnt/backup directory. We will create backup directory

root@storage #

mkdir -p /mnt/backup

root@storage #

chown backup:backup /mnt/backup

root@storage #

chmod 770 /mnt/backup

SSH keys

rsnapshot will log in to remote servers via ssh public keys. Lets generate private/public ssh keys for all next ssh sessions.

root@storage #sudo -i -u backup

backup@storage $ssh-keygen

Save ssh keys to default path without password. After this, copy ssh key to remote server with ssh-copy-id:

backup@storage $ssh-copy-id backup@source

And lets recheck, that everything is fine

backup@storage $ssh source

No password should be asked and you simply login to remote.example.com

rsnapshot

This configuration gives a good starting point. rsnapshot configuration files are tab delimited. Run the syntax checker after each edit to ensure it will run properly.

FILE root@storage:/etc/rsnapshot.conf

config_version	1.2

# Required settings.
cmd_cp		/bin/cp
cmd_rm		/bin/rm
cmd_rsync	/usr/bin/rsync
cmd_ssh		/usr/bin/ssh

ssh_args		-i	/home/backup/.ssh/id_rsa

# If there's space available, this will first complete a successful backup
# before rotating. Uncomment to enable.
#sync_first	1

rsync_short_args	-aAX

# Generally a good idea to just enable. This allows rsync to do some of the
# tasks cp would have performed, but with better support for non-Linux systems.
link_dest	1

# When enabled, will cross partitions. Disable (set to 0) to not.
one_fs	1

# Exclude pattern (refer to --exclude-from from rsync man page)
exclude		/dev/*
exclude		/proc/*
exclude		/sys/*
exclude		/run/*
exclude		/var/tmp/*
exclude		/var/run/*
exclude		/tmp/*
exclude		/var/cache/distfiles/*
exclude		/lost+found

# Required. Local filesystem path to save all snapshots.
snapshot_root	/mnt/backup/

# Prevent creating snapshot_root. This is especially useful when snapshot_root
# is mounted on a separate file system.
no_create_root	1

# Backup source path to destination path. Destination path is relative to
# snapshot_root.
# backup	source	destination	options
#backup	/	local_root/	one_fs=0
backup	backup@source:/	source/	+rsync_long_args=--rsync-path=/home/backup/rsync-wrapper.sh

# At least one of these must be declared. daily, weekly, and monthly don't
# actually mean anything. It could have been foo, bar, and foobar...but that'd
# be confusing.
retain	daily		7
retain	weekly		4
retain	monthly	12

This files have such params:

rsync_short_args:
- -a: Archive mode is a brief way to express -rlptgoD. It tells rsync to recurse into directories, copy symlinks as symlinks, preserve permissions, preserve modification times, preserve group, preserve owner, transfer character and block device files, and transfer special files, such as named sockets and fifos.
- -A: preserve ACLs (this is fine to leave enabled even when ACLs aren't used)
- -X: preserve extended attributes (this is fine to leave enabled even when extended attributes aren't used)
+rsync_long_args: We don't want to override the defaults Parameters, that will directly be passed to rsync command:
rsync-path Execute on remote server rsync wrapper script /home/backup/rsync-wrapper.sh
ssh_args: Path to public key, that should be used for remote ssh login
snapshot_root: Path to directory, where all backup files will be stored
backup specifies a container directory for the backups, usually referring to the machine (in this case, source). This can be changed to any name of your choosing. The final snapshots will be saved under /mnt/backup/{daily,weekly,monthly}.[0-9]*/source/
exclude: This directory will be excluded from backup

Once you're finished editing, check for syntax errors by running:

backup@storage $rsnapshot configtest

Note
rsnapshot will always take the last daily snapshot to create the first weekly snapshot and the last weekly snapshot to create the first monthly one. It will not take the 7th daily snapshot to create the first weekly snapshot. Therefore, it is possible to keep less or more than 7 daily snapshots, but is this case the first weekly snapshot is not one week old.

cron jobs

Add cron job to run backup

backup@backup.example.com $crontab -e

Note
If you want full backup of remote system, with full backup of file owners, groups, permission, attributes, you should run rsnaphost cron jobs from root user. In other words, command below should be run from root

root@backup.example.com #crontab -e

FILE

#3am each day
0 3 * * *    ionice -c 3 nice -n +19 /usr/bin/rsnapshot daily
#4am each week
0 4 * * 1    ionice -c 3 nice -n +19 /usr/bin/rsnapshot weekly
#4am each month
0 4 1 * *    ionice -c 3 nice -n +19 /usr/bin/rsnapshot monthly

rsnapshot jobs will run rsnapshot with minimum CPU and I/O priority.

Automate the backups via systemd timer units

Instead of crontab you can use systemd timers and service units. The .timer unit starts the .service unit.

This example timer starts daily at 11 am. Adapt to your needs and create one for daily, weekly, monthly, depending on your rsnapshot.conf

FILE /etc/systemd/system/rsnapshot-daily.timer

[Unit]
Description=rsnapshot daily backup

[Timer]
OnCalendar=*-*-* 11:00:00
Persistent=true
Unit=rsnapshot-daily.service

[Install]
WantedBy=timers.target

FILE /etc/systemd/system/rsnapshot-daily.service

[Unit]
Description=rsnapshot daily backup

[Service]
User=backup
Type=oneshot
Nice=19
IOSchedulingClass=idle
ExecStart=/usr/bin/rsnapshot daily

As root, enable and set the timer unit active

root@storage #systemctl enable --now rsnapshot-daily.timer

To find out how much time is left until systemd will activate the service the next time or when it ran the last time

root@storage #systemctl list-timers

MySQL backup

Note
This backup configuration are workable for small non-production databases, that doesn't have too many transactions. For more advanced MySQL backup, see https://dev.mysql.com/doc/refman/5.7/en/backup-methods.html (replication or Binary Log backup)

Login to backup user:

user@backup.example.com $sudo -i -u backup

Create file .my.cnf with such content

FILE /home/backup/.my.cnf

[mysqldump]
host = localhost
port = 3306
user = backup
password = BACKUP_USER_PASSWORD

This file are used every time, when mysqldump tool will be called. Be sure, that only backup user have access to /home/backup/.my.cnf file

Create bash script mysql_dump.bash, that will for backup:

FILE /home/backup/mysql_dump.bash

#/bin/bash
/usr/bin/mysqldump --all-databases | bzip2 -c > /backup/mysql/`date +%Y.%m.%d_%H.%M.%S.sql.bz2`

Add executable flag for script:

user@backup.example.com $chmod +x /home/backup/mysql_dump.bash

Create directory /backup/mysql, that contain all MySQL backups and grand permissions:

root@backup.example.com #

mkdir -p /backup/mysql

root@backup.example.com #

chown backup /backup/mysql

root@backup.example.com #

chmod 700 /backup/mysql

Create MySQL user backup with access to all databases (like root user, but for backup)

root@backup.example.com #mysql

and type:

mysql>

GRANT ALL PRIVILEGES ON *.* TO 'backup'@'localhost' IDENTIFIED BY 'BACKUP_USER_PASSWORD'

Last step - create cron job, that will call mysql_dump.bash script and dump all databases. Execute from backup user

backup@backup.example.com $crontab -e

and add such line (run every day at 01:00)

FILE

0 1 * * * /home/backup/mysql_dump.bash

PostgreSQL backup

To be DONE

Restoration

To restore the remote.example.com backups specified above, we would use:

root #

mount /mnt/backup

root #

rsync -aAX /mnt/backup/remote.example.com/monthly.0/remote.example.com/. /mnt/myroot/

root #

rsync -aAX /mnt/backup/remote.example.com/weekly.0/remote.example.com/. /mnt/myroot/

root #

rsync -aAX /mnt/backup/remote.example.com/daily.0/remote.example.com/. /mnt/myroot/

If backup are on remote server, rsync can be done via ssh

root #rsync -aAX root@remote.example.com/monthly.0/remote.example.com/. /

where /mnt/myroot is the mount point of the fresh root filesystem. In the paths above *.0 refers to the latest increment.

MySQL

Technically, MySQL dump (created from upper section) are just bzipped text file with SQL commands to MySQL database. Those command will unzip archive, and send sql command to mysql

root #bzcat /backup/mysql/some_ziped_archive.bz2 | mysql --user=root

Possible improvements

Use remote device for storing backups /mnt/backup - TO BE DONE
Use encryption to crypt backups /mnt/backup - TO BE DONE

External resources

[1] - Documentation from rsnapshot site with some help about configuring rsnapshot
[2] - HOWTO configure the same, but more general

User:TitanOfOld/Advanced backup using rsnaphot

Contents

Source Host

Requirements

Backup user

Sudo

Rsync wrapper

Storage Host

Requirements

Backup user

Directories

SSH keys

rsnapshot

cron jobs

Automate the backups via systemd timer units

MySQL backup

PostgreSQL backup

Restoration

MySQL

Possible improvements

See also

External resources