Guida ai pacchetti binari
emerge — configuration — ebuild repository — dispatch-conf
world file — USE flags — ebuilds — profiles
upgrades — using testing packages — binary packages
tools — gentoolkit — eselect
Portage FAQ — cheat sheet — FAQ
all articles
See the binary package quickstart article for information on using pre-built binary packages from the Gentoo binary package host. That article should cover everything needed to easily use the binary packages from the public Gentoo binhost.
This guide covers in-depth binary package creation, distribution, use, and maintenance, and a few more advanced topics near the end.
Oltre al supporto classico alle ebuild, Portage supporta anche la costruzione e l'installazione di pacchetti binari. Questa guida spiega come crearli, installartli, e come configurare un server di pacchetti binari.
Tutti gli strumenti usati in questa guida fanno parte di sys-apps/portage, a meno che non sia specificato diversamente.
Why use binary packages on Gentoo?
Ci sono molte ragioni per cui ad alcuni amministratori di sistemi piace usare pacchetti binari per le installazioni di software su Gentoo:
- Permette agli amministratori di "risparmiare tempo nel momento in cui sistemi simili devono essere aggiornati". Dover compilare tutto da sorgenti può diventare consumare molto tempo. Mantenere numerosi sistemi simili, alcuni dei quali con hardware più datati, può essere molto più facile se solo un sistema deve compilare tutto da sorgente e gli altri usano i pacchetti binari.
- Fare aggiornamenti sicuri. Per sistemi di importanza critica per la produzione è importante rimanere "usabili" il maggior tempo possibile. Ciò si può ottenere tramite un server di staging che effettua prima tutti gli aggiornamenti su di sé. Una volta osservato che il server è in uno stato funzionante, gli aggiornamenti possono essere estesi ai sistemi critici. Una variante di questo approccio consiste nell'applicare gli aggiornamenti in un chroot sullo stesso sistema e poi usare i binari creati qui sul sistema reale.
- Come backup. Spesso i pacchetti binari sono l'unica maniera per ripristinare un sistema non funzionante (per esempio, un compilatore che non funziona). Avere a disposizione dei binari precompilati su un server o localmente può essere di grande aiuto in caso di catena di compilazione danneggiata.
- Permette di aggiornare sistemi molto vecchi. L'attività di aggiornamento di sistemi molto vecchi può essere ampiamente facilitata usando pacchetti binari. È solitamente utile installare pacchetti binari su vecchi sistemi in quanto essi non necessitano l'installazione o l'aggiornamento delle dipendenze necessarie per la compilazione. I pacchetti binari evitano anche errori nel processo di compilazione in quanto sono già precompilati.
Binary package formats
Two binary package formats for use in Gentoo exist, XPAK and GPKG. Starting with v3.0.31, Portage supports the new binary package format GPKG. The GPKG format solves issues with the legacy XPAK format and offers the benefit of new features, however it is not backward compatible with the legacy XPAK format.
System administrators using older versions of Portage (<=v3.0.30) should continue to use the legacy XPAK format, which is Portage's default setting on those versions.
Motivation for the newer GPKG format design can be found in GLEP 78: Gentoo binary package container format. Bugs bug #672672 and bug #820578 also provide helpful details.
To instruct Portage to use the GPKG format, change the BINPKG_FORMAT value in /etc/portage/make.conf. Note that current versions of Portage use gpkg by default.
BINPKG_FORMAT="gpkg"
This guide mostly applies to both formats; where this is not the case it will be noted. See the Understanding the binary package format section for technical details on the binary package formats themselves.
Usare i pacchetti binari
General prerequisites
Affinché i pacchetti binari siano utilizzabili su altri sistemi, essi devono soddisfare alcuni requisiti:
- Le architetture del client e del server e la variabile CHOST devono corrispondere.
- Le variabili CFLAGS e CXXFLAGS usate per compilare i pacchetti binari devono essere compatibili con tutti i client.
- I flag USE per funzionalità di insiemi di istruzioni specifiche del processore (come MMX, SSE, ecc.) devono essere selezionate con attenzione; tutti i client devono supportarli.
Binary packages that are distributed as part of Gentoo's official Binhost project use a minimum instruction set and conservative compiler settings in order to be as widely usable as possible. By way of example, amd64 keyworded packages are built with
-march=x86-64 -mtune=generic
, which works for any machine which runs the x86-64 (amd64) instruction set.Portage non controlla se tali requisiti sono soddisfatti. È responsabilità degli amministratori assicurarsi di controllare tali impostazioni.
Handling *FLAGS in detail
The app-misc/resolve-march-native utility can be used to find a subset of CFLAGS that is supported by both the server and client(s). For example, the host might return:
user $
resolve-march-native
-march=skylake -mabm -mrtm --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=12288
While the client might return:
user $
resolve-march-native
-march=ivybridge -mno-rdrnd --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=3072
In this example CFLAGS could be set to -march=ivybridge -mno-rdrnd
since -march=ivybridge
is a full subset of -march=skylake
. -mabm
and -mrtm
are not included as these are not supported by the client. However, -mno-rdrnd
is included as the client does not support -mrdrnd
. To find which -march
's are subsets of others, check the gcc manual, if there is no suitable subset set e.g. -march=x86-64
.
Optionally, it is also possible to set -mtune=some-arch
or -mtune=native
to tell gcc to tune code to a specific arch. In contrast to -march
, the -mtune
argument does not prevent code from being executed on other processors. For example, to compile code which is compatible with ivybridge and up but is tuned to run best on skylake set CFLAGS to -march=ivybridge -mtune=skylake
. When -mtune
is not set it defaults to whatever -march
is set to.
When changing -march
to a lower subset for using binary packages on a client, a full recompilation is required to make sure that all binaries are compatible with the client's processor, to save time packages that are not compiled with e.g. gcc/clang can be excluded:
user $
emerge -e @world --exclude="acct-group/* acct-user/* virtual/* app-eselect/* sys-kernel/* sys-firmware/* dev-python/* dev-java/* dev-ruby/* dev-perl/* dev-lua/* dev-php/* dev-tex/* dev-texlive/* x11-themes/* */*-bin"
Similarly, app-portage/cpuid2cpuflags can be used to find a suitable subset of processor specific instruction set USE flags. For example, the host might return:
user $
cpuid2cpuflags
CPU_FLAGS_X86: aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3
While the client might return:
user $
cpuid2cpuflags
CPU_FLAGS_X86: avx f16c mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3
In this example CPU_FLAGS_X86 can be set to avx f16c mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3
in /etc/portage/make.conf because these flags are supported by both the client and the host
Oltre a ciò, Portage controllerà se il pacchetto binario viene costruito con gli stessi flag USE attesi dal client. Se un pacchetto viene costruito con una combinazione di flag USE diversa, Portage ignorerà il pacchetto binario (ed userà una compilazione basata su sorgenti) oppure fallirà, a seconda delle opzioni passate al comando emerge al momento dell'invocazione dello stesso (consultare la sezione installazione dei pacchetti binari).
Sui client, sono necessari alcuni cambiamenti di configurazione affinché i pacchetti binari possano essere usati.
Installare i pacchetti binari
Alcune opzioni possono essere passate al comando emerge per informare Portage di usare i pacchetti binari:
Opzione | Descrizione |
---|---|
--usepkg (-k ) |
Tenta di usare il pacchetto binario od i pacchetti binari nella cartella disponibile localmente packages. Utile quando si usano host di pacchetti binari montati su NFS o SSHFS. Se i pacchetti binari non vengono trovati, verrà effettuata un'installazione reolare (basata sui sorgenti). |
--usepkgonly (-K ) |
Simile a --usepkg (-k ) ma fallisce se il pacchetto binario non viene trovato. Questa opzione è utile se devono essere usati solo pacchetti binari pre-costruiti.
|
--getbinpkg (-g ) |
Scarica il pacchetto binario od i pacchetti binari da un host di pacchetti binari remoto. Se i pacchetti binari non vengono trovati, verrà effettuata un'installazione regolare (basata su sorgenti). |
--getbinpkgonly (-G ) |
Simile a --getbinpkg (-g ) ma fallisce se il pacchetto binario od i pacchetti binari non possono essere scaricati. Questa opzione è utile se devono essere usati solo pacchetti pre-costruiti.
|
| --usepkg
(-k
) || Tries to use the binary package(s) in the locally available packages directory. Useful when using NFS or SSHFS mounted binary package hosts. If the binary packages are not found, a regular (source-based) installation will be performed.
|-
| --usepkgonly
(-K
) || Similar to --usepkg
(-k
) but fail if the binary package cannot be found. This option is useful if only pre-built binary packages are to be used.
|-
| --getbinpkg
(-g
) || Download the binary package(s) from a remote binary package host. If the binary packages are not found, a regular (source-based) installation will be performed.
|-
| --getbinpkgonly
(-G
) || Similar to --getbinpkg
(-g
) but will fail if the binary package(s) cannot be downloaded. This option is useful if only pre-built binary packages are to be used.
|-
|}
Per usare automaticamente installazioni di pacchetti binari, si può aggiungere l'opzione appropriata alla variabile EMERGE_DEFAULT_OPTS:
EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --getbinpkg"
There is a Portage feature that automatically implements the equivalent of --getbinpkg
(-g
) without the need for updating the EMERGE_DEFAULT_OPTS variable with the --getbinpkg
value:
FEATURES="getbinpkg"
Verify binary package OpenPGP signatures
OpenPGP signing and verification is only available for the GPKG binpkg format.
Portage will try to verify the binary package's signature whenever possible, but users must first set up trusted local keys. app-portage/getuto can be used to set up a local trust anchor and update the keys in /etc/portage/gnupg. Portage calls getuto automatically with --getbinpkg or --getbinpkgonly.
This configures portage such that it trusts the Gentoo Release Engineering keys as also contained in the package sec-keys/openpgp-keys-gentoo-release for binary installation purposes.
Changes to the configuration can be done as root using gpg
with the parameter --homedir=/etc/portage/gnupg
. This way allows importing additional signing keys (e.g. for non-standard installation sources) and declare them as trusted.
To add a custom signing key:
- Generate (or use an existing) key with signing abilities, and export the public key to a file.
- Run
getuto
if it has never run:root #
getuto
- Use
gpg --homedir=/etc/portage/gnupg --import public.key
to import the public key in portage's keyring. - Trust and sign this key using the key created by
getuto
. In order to do this, first get the password to unlock the key at /etc/portage/gnupg/pass, then use:Typeroot #
gpg --homedir=/etc/portage/gnupg --edit-key YOURKEYID
sign
,yes
, paste (or type) the password. The key is now signed. To trust it, typetrust
, then4
to trust it fully. Finally, typesave
. - Update the trustdb so that GPG considers the key valid:
root #
gpg --homedir=/etc/portage/gnupg --check-trustdb
If you hit any issues, check if a pre-existing /etc/portage/gnupg existed. If it did, move it away and then repeat the above steps.
Congratulations, Portage now has a working keyring!
Trusting the key marginally or less will not work
By default, Portage will only verify GPG signatures when a signature file is found in a package, which allows the user to mix signed and unsigned GPKG binary packages from different sources, and allows to use old XPAK format binary packages.
If the user wishes to force signature verification, the binpkg-request-signature
feature needs to be enabled. This feature assumes that all packages should be signed and rejects any unsigned package. Note that this feature does not support per-binhost configuration.
# Require that all binpkgs be signed and reject them if they are not (or have an invalid sig)
FEATURES="binpkg-request-signature"
Pulling packages from a binary package host
"The PORTAGE_BINHOST variable is deprecated in favor of the /etc/portage/binrepos.conf configuration file" - make.conf(5)
When using a binary package host, clients need to have the PORTAGE_BINHOST variable set in /etc/portage/make.conf or the sync-uri variable in /etc/portage/binrepos.conf. Without this configuration, the client will not know where the binary packages are stored which results in Portage being unable to retrieve them.
[binhost]
sync-uri = https://example.com/binhost
priority = 10
For each binhost, a name can be configured in the brackets. sync-uri must point to the directory in which the Packages file resides. Optionally, priority can be set. When a package exists in multiple binary package repositories, the package is pulled from the binary package host with the highest priority. This way, a preferred binary package host can be set up.
Many Gentoo stages already come with a preinstalled /etc/portage/binrepos.conf file, which points to the corresponding binary packages generated during the stage builds.
The support for multiple binary package servers is somewhat incomplete. If several servers serve a binary package for the same package version, then only the first one will be considered. This can be problematic when these binary packages differ in their USE variable configuration and the USE variable configuration of a later binary package would match the systems configuration.
Reinstalling modified binary packages
Passing the --rebuilt-binaries
option to emerge will reinstall every binary that has been rebuilt since the package was installed. This is useful in case rebuilding tools like revdep-rebuild are run on the binary package server.
A related option is --rebuilt-binaries-timestamp
. It causes emerge not to consider binary packages for a re-install if those binary packages have been built before the given time stamp. This is useful to avoid re-installing all packages, if the binary package server had to be rebuild from scratch but --rebuilt-binaries
is used otherwise.
Additional client settings
Next to the getbinpkg
feature, Portage also listens to the binpkg-logs
feature. It controls if log files for successful binary package installations should be kept. It is only relevant if the PORT_LOGDIR variable has been set and is enabled by default.
Similar to excluding binary packages for a certain set of packages or categories, clients can be configured to exclude binary package installations for a certain set of packages or categories.
To accomplish this, use the --usepkg-exclude
option:
root #
emerge -uDNg @world --usepkg-exclude "sys-kernel/gentoo-sources virtual/* sys-kernel/gentoo-kernel"
To enable such additional settings for each emerge command, add the options to the EMERGE_DEFAULT_OPTS variable in the make.conf file:
EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --usepkg-exclude 'sys-kernel/gentoo-sources virtual/*'"
Updating packages on the binary package host
Do not use
--changed-use
(-U
) when updating packages on the binary package host, doing so will cause packages with added or removed USE flags to be skipped, which will cause their installation from binary package on the client to fail due to non-matching USE between the source ebuild and binary package (if the client's --binpkg-respect-use=y
, the default). Use --newuse
(-N
), which will always rebuild packages even for added or removed USE flags, ensuring the binary package stays in sync with the source ebuild.Creare pacchetti binari
Esistono tre metodi principali per la creazione di pacchetti binari:
- Dopo un'installazione regolare, usando l'applicazione quickpkg.
- In maniera esplicita durante un'operazione emerge usando l'opzione
--buildpkg
(-b
). - In maniera automatica usando il valore
buildpkg
nella variabile FEATURES di Portage.
Tutti e tre i metodi creeranno un pacchetto binario nella directory indicata dalla variabile PKGDIR (che ha un valore predefinito uguale a /usr/portage/packages).
Usare --buildpkg come opzione di emerge
Quando si installa del software usando emerge, si può chiedere a Portage di creare pacchetti binari usando l'opzione --buildpkg
(-b
):
root #
emerge --ask --buildpkg sys-devel/gcc
È anche possibile chiedere a Portage di creare solamente un pacchetto binario ma di non installare il software sul sistema live. Per questo, si può usare l'opzione --buildpkgonly
(-B
):
root #
emerge --ask --buildpkgonly sys-devel/gcc
Quest'ultimo approccio tuttavia richiede che tutte le dipendenze per la compilazione siano state già installate.
Implementare buildpkg come funzionalità di Portage
La maniera più comune per creare automaticamente pacchetti binari ogni qual volta che un pacchetto viene installato da Portage è quella di usare la funzionalità buildpkg
, che può essere impostata in /etc/portage/make.conf come di seguito:
FEATURES="buildpkg"
Con questa funzionalità attivata, ogni volta che Portage installerà del software esso creerà anche un pacchetto binario.
Escludere la creazione di alcuni pacchetti
È possibile dire a Portage di non creare pacchetti binari per una certa selezione di pacchetti o categorie. Per ottnere ciò, si deve passare l'opzione --buildpkg-exclude
ad emerge:
root #
emerge -uDN @world --buildpkg --buildpkg-exclude "virtual/* sys-kernel/*-sources"
Ciò può essere usato per pacchetti che hanno poco o nulla da beneficiare nell'avere disponibili un pacchetto binario. Dei casi possibili sono le sorgenti del kernel Linux o i pacchetti binari upstream (quelli che terminano con -bin come www-client/firefox-bin).
Binary package compression formats
It is possible to use a specific compression type on binary packages. Currently, the following formats are supported: bzip2
, gzip
, lz4
, lzip
, lzop
, xz
, and zstd
. Defaults to zstd
. Review man make.conf and search for BINPKG_COMPRESS for the most up-to-date information.
The compression format can be specified via make.conf.
BINPKG_COMPRESS="lz4"
Note that the compression type used might require extra dependencies to be installed, for example, in this case app-arch/lz4.
Binary package OpenPGP signing
OpenPGP signing and verification is only available for the GPKG binpkg format.
A PGP signature enables Portage to check the creator and integrity of a binary package, and to perform trust management based on PGP keys. The binary package signing feature is disabled by default. To use it, enable the binpkg-signing
feature. Note that whether this feature is enabled does not affect the signature verification feature.
FEATURES="binpkg-signing"
Users also need to set the BINPKG_GPG_SIGNING_GPG_HOME and BINPKG_GPG_SIGNING_KEY variables for Portage to find the signing key.
BINPKG_GPG_SIGNING_GPG_HOME="/root/.gnupg"
BINPKG_GPG_SIGNING_KEY="0x1234567890ABCDEF"
Portage will only try to unlock the PGP private key at the beginning. If the user's key will expire over time, then consider enabling gpg-keepalive
to prevent signing failures.
FEATURES="gpg-keepalive"
gpg-agent by default expires cache entries after 2 hours. This means that, by default, if an emerge session lasts longer than 2 hours, signing of binpkgs will eventually fail regardless of FEATURES="gpg-keepalive". To prevent this problem, set max-cache-ttl to some large value (e.g. 34560000) in $BINPKG_GPG_SIGNING_GPG_HOME/gpg-agent.conf.
Usare quickpkg
L'applicazione quickpkg prende uno o più atom di dipedenza (o insiemi di pacchetti) e crea pacchetti binari per tutti i pacchetti installati corrispondenti a quell'atom.
Occorre fare una precisazione riguardo questo metodo: esso si basa sui file installati, che possono essere un problema nel caso dei file di configurazione. Gli amministratori cambiano spesso i file di configurazione dopo aver installato del software. Poiché ciò potrebbe far trasparire dati importanti (magari anche confidenziali) nei pacchetti, quickpkg non include in maniera predefinita tutti i file di configurazione protetti tramite il metodo CONFIG_PROTECT. Per forzare l'inclusione dei file di configurazione, si usino le opzioni --include-config
o --include-unmodified-config
.
Per esempio, per creare pacchetti binari di tutte le versioni di GCC installate:
root #
quickpkg sys-devel/gcc
To create binary packages for the system set:
root #
quickpkg @system
Per creare pacchetti binari di tutti i pacchetti installati nel sistema, si usi il simbolo *
:
root #
quickpkg "*/*"
Configurare un host di pacchetti binari
Portage supporta un grande numero di protocolli per lo scaricamento di pacchetti binari: FTP, FTPS, HTTP, HTTPS e SSH. Ciò lascia spazio a molte implementazioni di host di pacchetti binari diverse.
Non esiste, tuttavia, un metodo "automatico" fornito da Portage per la distribuzione di pacchetti binari. A seconda dell'installazione desiderata sarà necessario scaricare ulteriore software.
Host di pacchetti binari basato sulla rete
Un approccio comune per la distribuzione di pacchetti binari consiste nella creazione di un host di pacchetti binari basato sulla rete.
HTTPD
Si usi un server di rete come lighttpd (www-servers/lighttpd) e lo si configuri per fornire accesso in lettura al percorso /etc/portage/make.conf di PKGDIR.
# Aggiungere questo alla fine della configurazione standard
server.modules += ( "mod_alias" )
alias.url = ( "/packages" => "/usr/portage/packages/" )
Caddy
To set up the Caddy HTTP server to provide a web-based binary package host, create a Caddyfile
containing:
x.x.x.x:80 { # Replace x.x.x.x with your host's IPv4 address
root * /path/to/binhost/var/cache/binpkgs
file_server browse # Needed to server
}
Once that is created, run Caddy with:
root #
caddy run --config /path/to/Caddyfile
In seguito, sui sistemi client, si configuri di conseguenza la variabile PORTAGE_BINHOST:
PORTAGE_BINHOST="http://binhost.example.com/packages"
Host di pacchetti binari SSH
Per fornire un approccio più autenticato per i pacchetti binari, si può considerare l'uso di SSH.
Quando si usa SSH, è possibile usare la chiave SSH dell'utente root di Linux (senza password in quanto le installazioni devono avvenire in background) per connettersi ad un host di pacchetti binari remoto.
Per questo, è necessario assicurarsi che la chiave SSH dell'utente root sia permessa sul server. Ciò dovrà accadere per ogni macchina che si connetterà all'host SSH:
root #
cat root.id_rsa.pub >> /home/binpkguser/.ssh/authorized_keys
La variabile PORTAGE_BINHOST dovrà quindi essere come di seguito:
PORTAGE_BINHOST="ssh://binpkguser@binhostserver/usr/portage/packages"
If the SSH server is listening to a different port (e.g 25), then it must be specified after the address, like so:
PORTAGE_BINHOST="ssh://binpkguser@binhostserver:25/var/cache/binpkgs"
Non usare i file di configurazione di SSH che si trovano in ~/.ssh/config per configurare le porte od il nome utente. Questo percorso viene ignorato quando Portage tenta di effettuare l'rsync dei pacchetti sul client. Impostare invece correttamente tutte le opzioni nella variabile PORTAGE_BINHOST.
Esportato su NFS
Quando si usano pacchetti binari su una rete interna, può risultare più facile esportare i pacchetti tramite NFS e montarlo sui client.
Il file /etc/exports potrebbe essere così:
/usr/portage/packages 2001:db8:81:e2::/48(ro,no_subtree_check,root_squash) 192.168.100.1/24(ro,no_subtree_check,root_squash)
Sui client, il percorso può poi essere montato. Un esempio di /etc/fstab potrebbe essere:
binhost:/usr/portage/packages /usr/portage/packages nfs defaults 0 0
The NFS share is mounted on the local filesystem, so there is no need to set PORTAGE_BINHOST or use the --getbinpkg
option. Instead, follow the normal procedures for installing binary packages, remembering to point PKGDIR to the NFS share so that portage knows where to find the packages:
PKGDIR="/var/cache/binpkgs"
If PKGDIR is network-mounted, it may be advantageous to enable
FEATURES="pkgdir-index-trusted"
. This feature disables checking the entire PKGDIR for added or removed packages and instead trusts the contents of the Packages
file to be accurate. This significantly improves performance on high-latency networks.Maintaining binary packages
Exporting and distributing the binary packages will lead to useless storage consumption if the binary package list is not actively maintained.
Removing outdated binary packages
In the gentoolkit package an application called eclean is provided. It allows for maintaining Portage-related variable files, such as downloaded source code files, but also binary packages.
The following command will remove all binary packages that have no corresponding ebuild in the installed ebuild repositories:
root #
eclean packages
For more details please read the eclean article.
Another tool that can be used is the qpkg tool from the app-portage/portage-utils package. However, this tool is a bit less configurable.
To clean up unused binary packages (in the sense of used by the server on which the binary packages are stored):
root #
qpkg -c
Maintaining the Packages file
As of portage-3.0.52, Portage defaults to
FEATURES=pkgdir-index-trusted
for performance, which requires an accurate Packages index. This can be disabled if it is an inconvenience to regularly fix up the index with emaint after manual changes.Inside the packages directory exists a manifest file called Packages. This file acts as a cache for the metadata of all binary packages in the packages directory. The file is updated whenever Portage adds a binary package to the directory. Similarly, eclean updates it when it removes binary packages.
If for some reason binary packages are simply deleted or copied into the packages directory, or the Packages file gets corrupted or deleted, then it must be recreated. This is done using emaint command:
root #
emaint binhost --fix
To clear the cache of all binary packages:
root #
rm -r /var/cache/binpkgs/*
Advanced topics
Chrooting
If creating packages for a different Portage profile or system with different USE flags, a chroot can be created.
This example uses /var/chroot/buildenv as the chroot path, but any path can be used.
Creating the directories
First, the directories for this chroot must be created:
root #
mkdir --parents /var/chroot/buildenv
Deploying the build environment
Next, the appropriate stage 3 tarball must be downloaded and extracted, here the desktop profile | openrc tarball is being used:
This can be extracted with the following command:
/var/chroot/buildenv/ #
tar xpvf stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner
Configuring the build environment
{{{1}}}
The build environment should be configured to match that of the system it is building for. The simplest way to do this is to copy the /etc/portage and /var/lib/portage/world files. This can be done with rsync:
This command should be executed on the build target machine, where the remote host has the chroot.
user $
rsync --archive --whole-file --verbose /etc/portage/* larry@remote_host:/var/chroot/buildenv/etc/portage
user $
rsync --archive --whole-file --verbose /var/db/repos/* larry@remote_host:/var/chroot/buildenv/var/db/repos
{{{1}}}
This process should be repeated for the world file:
user $
rsync --archive --whole-file --verbose /var/lib/portage/world larry@remote_host:/var/chroot/buildenv/var/lib/portage/world
/var/lib/portage and /var/lib/portage/world should have the
root:portage
permissions.Configuring the chroot
Once created, mounts must be bound for the chroot to work:
/var/chroot/buildenv #
mount --types proc /proc proc
/var/chroot/buildenv #
mount --rbind /dev dev
/var/chroot/buildenv #
cp --dereference /etc/resolv.conf etc
Entering the chroot
To enter this chroot, the following command can be used:
/var/chroot/buildenv #
chroot . /bin/bash
Optionally, the prompt can be set to reflect the fact that the chroot is active:
/ #
export PS1="(chroot) $PS1"
Performing an initial build
This step assumes this configuration has been completed: Setting portage to use buildpkg.
This step is optional, but rebuilds all packages in the new world:
(chroot) #
emerge --emptytree @world
Building for other architectures
crossdev is a tool to easily build cross-compile toolchains. This is useful to create binary packages for installation on a system whose architecture differs from that of the system used to build the packages. A common example would be building binary packages for a device like an arm64 Raspberry Pi from a more powerful amd64 desktop PC.
An installation guide for sys-devel/crossdev can be found at the crossdev page.
Build a cross compiler
Using crossdev with the following command can build a toolchain for the desired system:
root #
crossdev --stable -t <arch-vendor-os-libc>
For the rest of this section, the example target will be for a Raspberry Pi 4:
root #
crossdev --stable -t aarch64-unknown-linux-gnu
After this has built, a toolchain will have been created in /usr/aarch64-unknown-linux-gnu, and will look like a bare-bones Gentoo install where it is possible to edit Portage settings as normal.
Replacing
aarch64-unknown-linux-gnu
with aarch64-unknown-linux-musl
would build a system with the Musl libc rather than Glibc.Basic setup
Removing the -pam
flag from the USE line in /usr/aarch64-unknown-linux-gnu/etc/portage/make.conf is generally recommended in a setup like this:
CHOST=aarch64-unknown-linux-gnu
CBUILD=x86_64-pc-linux-gnu
ROOT=/usr/${CHOST}/
ACCEPT_KEYWORDS="${ARCH}"
USE="${ARCH}"
CFLAGS="-O2 -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"
FEATURES="-collision-protect sandbox buildpkg noman noinfo nodoc"
# Ensure pkgs from another repository are not overwritten
PKGDIR=${ROOT}var/cache/binpkgs/
#If you want to redefine PORTAGE_TMPDIR uncomment (and/or change the directory location) the following line
PORTAGE_TMPDIR=${ROOT}var/tmp/
PKG_CONFIG_PATH="${ROOT}usr/lib/pkgconfig/"
#PORTDIR_OVERLAY="/var/db/repos/local/"
Profiles
List available profiles for the device by running:
root #
PORTAGE_CONFIGROOT=/usr/aarch64-unknown-linux-gnu eselect profile list
Next, select the profile that best suits:
root #
PORTAGE_CONFIGROOT=/usr/aarch64-unknown-linux-gnu eselect profile set <profile number>
Build a single package
To build a single binary package for use on the device, use the following:
root #
emerge-aarch64-unknown-linux-gnu --ask foo
Build world file
To build every package in the world file, then the following command is needed:
root #
emerge-aarch64-unknown-linux-gnu --emptytree @world
Binary location
By default, all binary packages will be stored in /usr/aarch64-unknown-linux-gnu/var/cache/binpkgs, so this is the location needed to be selected when setting up a binary package host.
Creating snapshots of the packages directory
When deploying binary packages for a large number of client systems it might become worthwhile to create snapshots of the packages directory. The client systems then do not use the packages directory directly but use binary packages from the snapshot.
Snapshots can be created using the /usr/lib/portage/python3.11/binhost-snapshot tool that comes with Portage (note that the path to that tool may need to be adjusted to match the python version with which Portage is installed). It takes four arguments:
- A source directory (the path to the packages directory).
- A target directory (that must not exist).
- A URI.
- A binary package server directory.
The files from the package directory are copied to the target directory. A Packages file is then created inside the binary package server directory (fourth argument) with the provided URI.
Client systems need to use an URI that points to the binary package server directory. From there they will be redirected to the URI that was given to binhost-snapshot. This URI has to refer to the target directory.
Understanding the binary package format
XPAK format
XPAK format binary packages created by Portage have the file name ending with .tbz2. These files consist of two parts:
- A .tar.bz2 archive containing the files that will be installed on the system.
- A xpak archive containing package metadata, the ebuild, and the environment file.
See man xpak for a description of the format.
In app-portage/portage-utils some tools exists that are able to split or create tbz2 and xpak files.
The following command will split the tbz2 into a .tar.bz2 and an .xpak file:
user $
qtbz2 -s <package>.tbz2
The .xpak file can be examined using the qxpak utility.
To list the contents:
user $
qxpak -l <package>.xpak
The next command will extract a file called USE which contains the enabled USE flags for this package:
user $
qxpak -x package-manager-0.xpak USE
GPKG format
GPKG format binary packages created by Portage have the file name ending with .gpkg.tar. These files consist of four parts at least:
- A gpkg-1 empty file that used to identify the format.
- A C/PV/metadata.tar{.compression} archive containing package metadata, the ebuild, and the environment file.
- A C/PV/image.tar{.compression} archive containing the files that will be installed on the system.
- A Manifest file containing checksums to protect against file corruption.
- Multiple optional .sig files containing OpenPGP signature are used for integrity checking and verification of trust.
The format can be extracted by tar without the need for additional tools.
The PKGDIR layout
The currently used format version 2 has the following layout:
PKGDIR
`+- Packages
+- app-accessibility/
| +- pkg1-version.tbz2
| `- pkgN-version.tbz2
+- app-admin/
| `- ...
`- ...
The Packages file is the major improvement (and also the trigger for Portage to know that the binary package directory uses version 2) over the first binary package directory layout (version 1). In version 1, all binary packages were also hosted inside a single directory (called All/) and the category directories only had symbolic links to the binary packages inside the All/ directory.
In portage-3.0.15 and later, FEATURES=binpkg-multi-instance
is enabled by default:
PKGDIR
`+- Packages
+- app-accessibility/
| +- pkg1/
| +- pkg1-version-build_id.xpak
| `- pkgN-version-build_id.xpak
+- app-admin/
| `- ...
`- ...
Unpacking with quickunpkg
Zoobab wrote a simple shell tool named quickunpkg to quickly unpack tbz2 files.
See also
- Binary package quickstart — using the fully-supported Gentoo binary package host for amd64 and arm64 with the settings the packages are compiled with
- Emerge — the main command-line interface to Portage
- Portage — the official package manager and distribution system for Gentoo.
- Project:Binhost — aims to provide readily installable, precompiled packages for a subset of configurations, via central binary package hosting
External resources
quickpkg man page.