Ccache

From Gentoo Wiki
Jump to: navigation, search

ccache helps avoid repeated recompilation for the same C and C++ object files by fetching result from a cache directory.

Compiler cache is typically useful for:

  • Developers who rebuild the same/similar codebase multiple times and use /etc/portage/patches to test patches.
  • Users who frequently play with USE-flag changes and end up rebuilding the same packages multiple times.
  • Users who use live ebuilds extensively.
  • Installing very big ebuilds, such as Chromium or LibreOffice, without fear of losing multiple hours of code compilation due to a failure.

Installation

Emerge

root #emerge --ask dev-util/ccache

Configuration

Initial setup

Create the cache directory:

root #mkdir -p /var/cache/ccache
root #chown root:portage /var/cache/ccache
root #chmod 2775 /var/cache/ccache

Set initial configuration file:

FILE /var/cache/ccache/ccache.conf
# Maximum cache size to maintain
max_size = 100.0G

# Allow others to run 'ebuild' and share the cache.
umask = 002

# Preserve cache across GCC rebuilds and
# introspect GCC changes through GCC wrapper.
compiler_check = %compiler% -v

# I expect 1.5M files. 300 files per directory.
cache_dir_levels = 3

Enable ccache support in make.conf:

FILE /etc/portage/make.conf
FEATURES="ccache"
CCACHE_DIR="/var/cache/ccache"

Done! From now on all builds will try to reuse object files from /var/cache/ccache cache.

Compression

ccache has the ability to perform compression on content. To enable it, edit ccache.conf:

FILE /var/cache/ccache/ccache.conf
compression = true
compression_level = 6

Man page

Manual page for dev-util/ccache (see man ccache) is a great source of various knobs to make caching more robust and aggressive.

General notes

ccache works by prepending /usr/lib/ccache/bin to PATH variable:

user $ls -l /usr/lib/ccache/bin
...
c++ -> /usr/bin/ccache
c99 -> /usr/bin/ccache
x86_64-pc-linux-gnu-c++ -> /usr/bin/ccache
...

FEATURES="ccache" triggers the same behavior in portage.

You can also enable ccache for your current user and reuse the same cache directory:

FILE ~/.bashrc
export PATH="/usr/lib/ccache/bin${PATH:+:}$PATH"
export CCACHE_DIR="/var/cache/ccache"

Useful variables and commands

Some variables:

  • Variable CCACHE_DIR points to cache root directory.
  • Variable CCACHE_RECACHE allows evicting old cache entries with new entries:
root #CCACHE_RECACHE=yes emerge --oneshot cat/pkg

See man ccache for many more variables.

Some commands:

  • Command ccache -s shows cache hit statistics:
user $CCACHE_DIR=/var/cache/ccache ccache -s
cache directory                     /var/cache/ccache
primary config                      /var/cache/ccache/ccache.conf
secondary config      (readonly)    /etc/ccache.conf
stats zero time                     Fri Sep  7 07:24:24 2018
cache hit (direct)                114988
cache hit (preprocessed)           38254
cache miss                        246428
cache hit rate                     38.34 %
...
files in cache                    603419
cache size                          16.9 GB
max cache size                     100.0 GB
  • Command ccache -C drops all caches:
user $CCACHE_DIR=/var/cache/ccache/ ccache -C

See man ccache for many more commands.

Gentoo specifics/gotchas

gcc is a wrapper

For sys-devel/gcc-config before 2.0 gcc used to be a wrapper binary. This means by default ccache is not safe to use and leads to errors like bug #640958. sys-devel/gcc-config-2.0 will fix it.

To pass through a binary I suggest using

FILE ccache.conf
compiler_check = %compiler% -v

in all your ccache.conf files. -v also has a nice side-effect of not invalidating your cache if compiler itself was rebuilt without version changes.

Caveats

Before using advanced ccache options make sure you understand what is being used as a cache key by ccache. By default these are:

  • Timestamp and size of a compiler binary (beware of shell and binary wrappers)
  • Compiler options used
  • Contents of a source file
  • Contents of all include files used for compilation

See also