User:NeddySeagoon/Build In A Chroot

From Gentoo Wiki
Jump to:navigation Jump to:search


A lot of users have a desktop and a far weaker portable system these days. Installing and maintaing Gentoo on a weak system is a challenge for the impatient, so don't do it, at least, not quite by following the handbook. The author has posted bits of pieces of this page on the forums several times over the last few months. It's time it was collected together in one place.

Being lazy, I can post a link on the forums too.


A Gentoo install splits nicely into two parts

  • The things needed to build and maintain the install
  • The things needed to use the install.

Keep that in mind for later.

Some examples, the ::gentoo repo, distfiles and binary packages are not required at run time. The binary files installed as a result of using the ::gentoo repo and distfiles are definitely required.

If your weaker system is storage constrained, that hint may be enough. If your weaker system is storage, RAM and CPU constrained, it's not. Read on.

Most of this page does not apply to users mounting the slow system filesystem on the fast system - skip to the end of the page

First hand Experience

The author uses an AMD 5950x with 128G RAM and 32 threads to maintain a chroot for an AMD E350 system with 8G RAM and two threads. On its own, updates were taking a week to 10d days, by which time, the system was out of date again.

Examples below are from the above arrangement.


Lets call our two systems tortoise and hare. With hare being the one doing all the heavy lifting.

Hare will have a chroot that mimics the install required on tortoise. Some of the bits from hares own install can be shared with the tortoise chroot.

Information about the real CFLAGS and CPU_FLAGS_X86 required by tortoise are needed in the chroot.

Work on hare

Make the chroot

As root make a chroot with

root #mkdir /usr/tortoise

That location was chosen as crossdev does something similar.

Fetch a stage3 and untar it to /usr/tortoise in the normal handbook way. Its probably a good idea to use a script for the rest, as it will be repeated every update.

In file /usr/tortoise/


# Mount the pesudo filesystem inside /usr/tortoise
mount -o bind /dev /usr/tortoise/dev
mount -t sysfs sysfs /usr/tortoise/sys
mount -t proc  proc /usr/tortoise/proc

# Share distfiles and the ::gentoo repo inside /usr/tortoise
# Note that there are in the new default locations
mount -t ext4 /var/cache/distfiles /usr/tortoise/var/cache/distfiles
mount -t ext4 /var/db/repos/gentoo /usr/tortoise/var/db/repos/gentoo

# tmpfs is a very good thing
mount -t tmpfs tmpfs /usr/tortoise/dev/shm

# and fix its permissions or build systems complain
chmod 1777 /usr/tortoise/dev/shm

# Put our build space in RAM too. We don't want wear out the NVMe 
mount -t tmpfs tmpfs /usr/tortoise/var/tmp/portage

# Likewise, tmp in RAM too.
mount -t tmpfs tmpfs /usr/tortoise/tmp

# fix the permissions to keep users separate 
chmod 1777 /usr/tortoise/tmp

# We need devpts in the chroot or we don't have any jobs
# Its picky about permissions too 
mount -t devpts devpts /usr/tortoise/dev/pts -o rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
The host filesystem type and locations of the ::gentoo repo and distfiles are user configurable
root #bash /usr/tortoise/

will run the script.

Enter the chroot

As per the handbook. Don't forget the preamble, /etc/resolv.conf is required.

If tortoise is already installed, copy over its /etc/portage/*. If not, set the chroot /etc/portage/* to suit the target tortoise, then copy it to tortoise before the install.

/etc/portage/* code generation options in both places must be identical

Tweaks to make.conf

Set CFLAGS to whatever app-misc/resolve-march-native outputs on tortoise.

Set CPU_FLAGS_X86 to the output of app-portage/cpuid2cpuflags from tortoise.

A part install may be required on tortoise to get the above information

Add FEATURES=buildpkg to make.conf, or no binary packages will be saved.

Change MAKEOPTS to suit the build host.

The world file

If tortoise is already installed, copy over its world file from /var/lib/portage/world to the chroot.

Hare summary so far

/usr/tortoise now reflects the install on tortoise but without any binary packages.

Build the Binary Packages

root #emerge -eav @world --keep-going

as we want binaries of everything the first time.

For updates

root #emerge -uUDav @world --keep-going

is sufficient.

Clean Up

As normal but add

root #eclean packages

to remove binary packages built by ebuilds that are no longer in the repo.

Share the Binary Packages

Not all of the methods that come to mind are used by the original author.

tortoise needs access to the binary packages and the matching repo.

The ::gentoo repo is updated every 30 minutes. Users need to make their own arrangements to keep it constant across the two systems.

None of the items being shared are required at run time. The author prefers NFS as there is only one copy of all the bits.

Choose one of the following options only ...

Network File System

Requires kernel support on hare to provide the export and on tortoise to do the nfs mount.

Not the easiest to set up but it just works. Share the ::repo(s) used to build the binaries and the binaries.

The author prefers NFS as there is only one copy of all the bits.

/etc/exports On hare, outside of the chroot.

# /etc/exports: NFS file systems being exported.  See exports(5).
# Portage wants it rw!

# share the repo the was used to build the above
# with the entire wired network
# No needed now but there may be other chroots later.


Copy /usr/tortoise/var/cache/binpkgs to tortoise:/var/cache/binpkgs

Copy the ::repo(s) used for the build


Copy/usr/tortoise/var/cache/binpkgs to tortoise:/var/cache/binpkgs

Copy the ::repo(s) used for the build

rsync will be faster unless a secure tunnel is used.


Host /usr/tortoise/var/cache/binpkgs

Host the ::repo(s) used for the build.

Other methods

e.g. Sneakernet.

Work on tortoise

Using the Binary Packages

With the ::repos and binary packages available on/to tortoise, its the emerge command used to build or update on hare ... almost.

For local ::repos and binary packages add -K or -k. NFS mounts are considered local.

For remote ::repos, emerge --sync to update the local copy of the ::repo. Then for remote binary packages add -G or -g to the emerge command.

The uppercase K/G means usepackageonly and fail if a suitable package is not found. The lowercase k/g, means usepackage if available and build it if not.

As no building is planned on tortoise build time only dependencies will not be installed.

Add --with-bdeps=y if build time dependencies are required.

What Goes Wrong

When the CPU instruction set on tortoise is a subset of hare. Nothing - it works as advertised.

When tortoise has CPU instructions that hare does not, illegal instruction exceptions will be raised (look in dmesg) and the offending job will be killed.

Its possible for hare to emit the full instruction set for tortoise but not execute it, which is mostly OK. There are two exceptions.

  • A few packages build some code then run it as a part of the build. They will fail if hare cannot execute the required instructions.
  • Some packages must be executed on hare to carry out the build process. They must run there.

In both cases the fix is per package CFLAGS and CPU_FLAGS_X86 to avoid the problems. If a toolchain package gets built for tortoise then crashes with an illegal instruction exception, picking up the pieces is a little more difficult but hare contains all the bits to fix it. As ever, its not start over, that never fixes anything in Gentoo.

Mounting the slow system filesystem directly on the build system

On the slow system, before doing the mount(s) run app-misc/resolve-march-native and replace its -march=native with the output of

root #resolve-march-native
-march=native will change on the build host, so must not be used

Mount the root filesystem from the slow system onto /mnt/gentoo on the fast system. Both NFS and USB3 work. USB3 with UAS is the fastest.

Mount any other slow system filesystems below /mnt/gentoo in the normal places.

Do not use the slow systems swap or /tmp

Mount the build hosts tmpfs at /mnt/gentoo/tmp.

Optionally, the build hosts copy of the the ::gentoo repo can be shared.

root #mount -t ext4 /var/db/repos/gentoo /mnt/gentoo/var/db/repos/gentoo

Follow the handbook to mount the pseudo filesystems inside /mnt/gentoo then perform the chroot steps.

Any emerges inside the chroot will use the build system resources but install to the slow systems filesystem.