Local distfiles cache

From Gentoo Wiki
Jump to: navigation, search

When several machines are running Gentoo on the same local area network, it is a good idea to cache distfiles on one of them and let the rest download from that one. This saves bandwidth for both the network owner and the public source mirrors.

Setting up the server

Select the machine that is going to serve the local distfiles cache and set it up. It will need to have enough free disk space to store all of the packages that are used by any machines on the network. The server machine will also need to be available whenever any other computer emerges a new package, and other computers on the network will need to be able to reach it using a static IP address or hostname.

There are a number of ways in which the cache can be implemented. This page describes a simple setup using net-misc/apt-cacher-ng.

Installing software

Install net-misc/apt-cacher-ng:

root #emerge --ask net-misc/apt-cacher-ng

Configuring for Gentoo

Apt-cacher-ng includes support for the Gentoo source mirrors, but some configuration is still needed. Since it is designed primarily for use with the Debian package management system, there are some types of files used in Gentoo distfiles which by default it refuses to download. To change this behavior, create the file /etc/apt-cacher-ng/gentoo.conf:

FILE /etc/apt-cacher-ng/gentoo.conf
PfilePattern: .*
Warning
This will most likely cause bad behavior for other distributions. Using the same apt-cacher-ng installation to cache files for Gentoo as well as another distribution is not recommended.

Configuring mirrors

Configure the list of public Gentoo mirrors from which apt-cacher-ng will download source packages:

FILE /etc/apt-cacher-ng/backends_gentoo
http://mirror1...
http://mirror2...

Starting the service

Now start the cache service:

root #rc-service apt-cacher-ng start

To start the service at boot:

root #rc-update add apt-cacher-ng default

Setting up clients

On all machines which are to use the distfiles cache (including the cache server itself), add the following to /etc/portage/make.conf:

FILE /etc/portage/make.conf
http_proxy="http://distfilescache:3142"
Note
Replace distfilescache with the name or IP address of the cache server.

Enhancements

  • Emerge net-misc/apt-cacher-ng with the fuse use flag and mount acngfs on /usr/portage/distfiles on the cache server. This avoids storing two copies of all of the packages installed on that server.

Open issues

  • Apt-cacher-ng installs a cron job to delete unreferenced files from the cache. Since the Gentoo cache contains no index files, this probably deletes either everything or nothing from it.
  • If sync-type is set to webrsync in /etc/portage/repos.conf, apt-cacher-ng will cache the snapshots. This is good if multiple machines are set up to use webrsync, but in the preferable case that a local rsync mirror is being used, it is just a waste of space.

See also

External resources