User:Sam/Autoconf cache

From Gentoo Wiki
Jump to:navigation Jump to:search

Background

configure tests can be rather time-intensive, especially on lower-end hardware. It's especially frustrating to see this:

root #./configure
...
checking for size_t... yes
checking for working alloca.h... yes
checking for alloca... yes
checking for dirent.h that defines DIR... yes
checking for library containing opendir... none required
checking for egrep... (cached) /bin/grep -E
checking for sys/wait.h that is POSIX.1 compatible... (cached) yes
checking for an ANSI C-conforming const... yes
checking how to run the C preprocessor... x86_64-pc-linux-gnu-gcc -m32 -E
checking for uid_t in sys/types.h... yes
...

It's possible for configure (autoconf-generated configure, at least) to cache the results of some tests so it doesn't have to re-run them. Note that it's most valuable to cache tests where a small e.g. C program is being compiled and executed as these are often the most expensive.

Implementation within Portage

This was actually implemented within Portage for a period between 2006 and 2008 when it was removed.

It was removed because of various bugs (TODO: links), but the main context is in bug #156308.

There was also a package dev-util/confcache which seems to be on GitHub, but not verified it. I'm curious as to what it did. It ended up being dropped as:

root #git log 62796a056f4480fdbe90ca0d6b0b093a33d1c3a0
commit 62796a056f4480fdbe90ca0d6b0b093a33d1c3a0
Author: Alec Warner <antarus@gentoo.org>
Date:   Mon Oct 30 18:19:35 2006 +0000

    <+ferringb> antarus: just punt the fucking thing, frankly.
[...]

It was hosted at http://gentooexperimental.org/~ferringb/confcache/.

Thoughts

  • Obviously, you can't easily use this across systems and platforms
    • You can kind of do this if you restrict to a subset of tests, but you really want to generate it at least once on each platform, not guess.
  • Need to invalidate cache on major observable changes
    • CHOST
    • Architecture
    • Profile?
    • Toolchain versions changing
  • Implement it per-package instead to speed up rebuilds and upgrades at least, rather than system-wide
  • Would benefit from a shared workaround list like app-portage/no-distcc-env or gentooLTO

Setup

Warning
Please don't file bugs when using this configuration for now. Various tests will fail unexpectedly on some packages.

Create a directory for the cache files to reside in:

root #mkdir /etc/portage/confcache

Then configure /etc/portage as follows:

FILE /etc/portage/bashrc
export CONFIG_SITE="${PORTAGE_CONFIGROOT}/etc/portage/confcache/config.site"
FILE /etc/portage/confcache/config.site
# (From smaeul's repo)
# Load the cache, but don't tell configure where it is, to keep configure
# from overwriting it.
. "${PORTAGE_CONFIGROOT}/etc/portage/confcache/config.cache"
FILE /etc/sandbox.d/50-confcache
# Allow writing to /etc/portage/confcache, not /etc/portage
# out of caution.
# TODO: I doubt PORTAGE_CONFIGROOT works literally here.
SANDBOX_WRITE="${PORTAGE_CONFIGROOT}/etc/portage/confcache"

Verification

TODO: explain how to check if it's working

Links