rsync

From Gentoo Wiki
Jump to:navigation Jump to:search
This page contains changes which are not marked for translation.


rsync is a powerful file sync program capable of efficient file transfers and directory synchronization.

rsync is included at the system set on all Gentoo profiles has been used as the primary method for syncing the Gentoo ebuild repository since Gentoo was created.

Installation

USE flags

USE flags for net-misc/rsync File transfer program to keep remote files into sync

acl Add support for Access Control Lists
examples Install examples, usually source code
iconv Enable support for the iconv character set conversion library
lz4 Enable support for lz4 compression (as implemented in app-arch/lz4)
rrsync Install rrsync script to setup restricted rsync users via ssh logins
ssl Add support for SSL/TLS connections (Secure Socket Layer / Transport Layer Security)
stunnel Provide helper scripts for using rsync via >=net-misc/stunnel-4
system-zlib Use system zlib instead of bundled one. This is incompatible with older rsync releases!
verify-sig Verify upstream signatures on distfiles
xattr Add support for extended attributes (filesystem-stored metadata)
xxhash Enable dev-libs/xxhash support for hashing
zstd Enable support for ZSTD compression
Data provided by the Gentoo Package Database · Last update: 2025-01-15 22:15 More information about USE flags

Emerge

root #emerge --ask --oneshot net-misc/rsync

Configuration

rsync can run both as a client or a server.

Files

  • /etc/rsyncd.conf - rsync's configuration file for daemon mode.
    • See man 5 rsyncd.conf for more configuration options.

Service

Daemon help can be seen by running:

root #rsync --daemon --help

OpenRC

To enable the service to run at system boot:

root #rc-update add rsyncd default

To start the rsync daemon now:

root #rc-service rsyncd start

systemd

To enable the service to run at system boot:

root #systemctl enable rsyncd.service

To start the rsync daemon now:

root #systemctl start rsyncd.service

Examples

Sharing from a list of files

FILE /etc/rsyncd.confSharing configuration.
...
# use chroot  = no # Needed when crossing filesystem boundaries. - #
...
# hosts allow = 192.168.0.0/16 # Allow local network. - #
# hosts deny  = *              # Deny the rest. - #
...
# uid = root # Needed for files like /etc/ssh/sshd_config. - #
# gid = root #         >>            /etc/sudoers. - #
...
[sync]
    path         = /
    include from = /backup.list # List of files to copy. - #
    exclude      = *            # Exclude the rest. - #

In the file list, all the intermediary paths are necessary, except when the *** wildcard is used.

FILE /backup.listExample file list.
/etc/
/etc/conf.d/
/etc/conf.d/hwclock
/etc/fonts/***

Synchronizing Gentoo's repository between local machines

If several Gentoo systems are in the same network, it spares bandwidth for everyone to define a local server for the main ebuild repository instead of letting all the machines synchronize in turn with the same external mirror.

The version of /etc/rsyncd.conf shipped by Gentoo already provides a valid default configuration. Uncomment the following lines:

FILE /etc/rsyncd.conflocal rsync server
[gentoo-portage]
	path = /var/db/repos/gentoo
	comment = Gentoo ebuild repository
	exclude = /distfiles /packages

Then, start the rsync daemon on the server.

Finally, change the repository configuration on client machines to match the address of the local server. For example, if this address is 192.168.2.24:

FILE /etc/portage/repos.conf/gentoo
[gentoo]
#sync-uri = rsync://rsync.gentoo.org/gentoo-portage 
sync-uri = rsync://192.168.2.24/gentoo-portage

The usual commands emerge --sync or emaint sync -r gentoo function normally and the process is usually much faster. The change can be reversed at any time.

Usage

Invocation

Client invocation:

user $rsync --help

Backuping

Rsync is a good tool for the full system backup (but please use git for specific configs and data), because if only a few files were changes - only them will occupy space inside the next backup folder - thanks to hardlinking.

In order to make a backup:

FILE /root/full-backup/full-backup.sh
BEFORE=$(df -h)

STARTED=$(date)

DATE=`date "+%Y-%m-%d"`

DEST="/mnt/backup/$DATE"

rsync --archive --acls --xattrs --delete --progress --verbose --exclude-from=exclude.txt --link-dest=/mnt/backup/last --mkpath / $DEST

ln --symbolic --force --no-dereference $DATE /mnt/backup/last

echo "Started at:   " $STARTED
echo "Current time: " $(date)

echo "Before:

$BEFORE

Now:
"

df -h

In the same folder you can have exclude list:

FILE /root/full-backup/exclude.txt
/dev/*
/proc/*
/sys/*
/run/*
/var/db/repos/gentoo
/var/db/repos/guru
/tmp/*
/var/tmp
/lost+found
/mnt/*

/home/vitaly/.npm
/home/vitaly/.cache
/home/vitaly/go/pkg/mod/cache

So your backups will looks like this:

user $ls -lh

How to restore from such backup:

root #rsync --archive --acls --xattrs --progress --verbose . /mnt

On the new SSD you probably will need to install the bootloader, or alter UUID values in /etc/fstab.

See more at https://wiki.archlinux.org/title/rsync#Snapshot_backup

Troubleshooting

rsync daemon stops working after glibc build

Occasionally the rsync daemon will stop working after glibc has been rebuilt. The error on the client side (when trying to sync from the server) will be something like the following: 

@ERROR: invalid uid nobody
rsync error: error starting client-server protocol (code 5) at main.c(1657) [Receiver=3.1.3]

Unfortunately simply restarting the rsync daemon will not fix the issue.

The solution, found in bug #414843, is to rebuild the net-misc/rsync package so that it can re-link against the proper version of glibc's libc.so.6 file.

root #emerge --ask --oneshot net-misc/rsync

After the rebuild then restart the daemon.

See also

  • Portage
  • Ssh — the ubiquitous tool for logging into and working on remote machines securely.
  • csync2 — a tool for asynchronous file synchronization in clusters.
Retrieved from "https://wiki.gentoo.org/index.php?title=Rsync&oldid=1301088"