Guide sur les paquets binaires

Portage - the heart of Gentoo
emerge — configuration — ebuild repository — dispatch-conf

world file — USE flags — ebuilds — profiles
upgrades — using testing packages — binary packages
tools — gentoolkit — eselect
Portage FAQ — cheat sheet — FAQ
all articles

See also
Voir l'article démarrage rapide des paquets binaires pour plus d'informations pour l'utilisation des paquets binaires pré-compilés depuis l'hôte des paquets binaires de Gentoo.

Ce guide se concentrera sur l'utilisation des paquets binaires, la création, le partage et la maintenance, quelques sujets plus avancés sur la manière de fournir des paquets binaires sera traité près de la fin de celui-ci.

Portage supporte les paquets binaires en plus de son support bien connu pour des ebuilds basés sur le code source (NdT: compilation). Portage peut être utilisé pour créer des paquets binaires, pour les installer, ou mettre à jour ceux déjà présents sur le système depuis des paquets binaires. Portage peut récupérer les binaires depuis les hôtes de paquets binaires.

Remarque
Tous les outils utilisés dans ce guide font partie de sys-apps/portage, sauf mention contraire.

Pourquoi utiliser les paquets binaires sur Gentoo ?

Il y a de nombreuses raisons qui expliquent que les administrateurs système aiment utiliser des paquets binaires sur Gentoo :

• Économiser du temps tout en gardant aussi bien le système à jour. Compiler depuis les sources peut prendre plus de temps qu'en installant depuis un paquet binaire. Une approche différente est de faire ces mises à jours dans un chroot sur le même système et utiliser les binaires crées à cet emplacement pour mettre à jour le vrai système.
• Faire des mises à jour sûres. Pour des systèmes critiques en production il est important de rester utilisable autant que possible. Cela peut être réalisé à travers un serveur intermédiaire pour faire les mises à jours pour lui-même. Une fois que le serveur intermédiaire est fonctionnel les mises à jours peuvent être appliqué vers le système critique avec des paquets binaires pour le système hôte.
• En tant que sauvegarde. Parfois, les paquets binaires sont le seul moyen de réparer un système endommagé (exemple: compilateur cassé). Avoir des paquets pré-compilé dans un coin, soit sur un serveur de binaires ou localement, peut être d'une aide importante dans le cas d'une chaîne de compilation non-fonctionnel.
• Cela peut aider à mettre un jour un système très vieux. Mettre à jour un très vieux système cela peut être facilité via l'utilisation de paquets binaires. Il est généralement utile d'installer des paquets binaires sur des vieux systèmes car ils ne requièrent pas de compiler des dépendances pour l'installation / mise à jours. Les paquets binaires peuvent permettre d'éviter des erreurs lors de la compilation.

Formats d'un paquet binaire

Deux types de formats de paquet existent pour Gentoo, XPAK et GPKG. Depuis v3.0.31, il supporte le nouveau format de paquet binaire GPKG. Le format GPKG résout des problèmes avec le format historique XPAK et offre les bénéfices de nouvelles fonctionnalités, cependant il n'y a pas de rétro-compatibilité avec le format historique XPAK.

Les administrateurs système utilisant une vieille version de Portage (<=v3.0.30) devraient utiliser le format XPAK, qui est le paramétrage par défaut de Portage.

Les raisons d'utiliser le nouveau format GPKG peuvent être trouver dans GLEP 78: Gentoo binary package container format. Les bugs bug #672672 et bug #820578 procurent aussi des détails utiles.

Pour demander à Portage d'utiliser le format GPKG, changer la valeur de BINPKG_FORMAT dans /etc/portage/make.conf.

FILE /etc/portage/make.confSpécifier GPKG en tant que format des paquets binaires

BINPKG_FORMAT="gpkg"

Ce guide s'adapte généralement aux deux formats, cela sera noté où ce n'est pas le cas. Voir la section comprendre le format de paquet binaires pour des détails techniques sur le format des paquets binaires en lui-même.

Utiliser des paquets binaire

Pré-requis généraliste

Pour qu'un paquet binaire puisse être utilisable sur un autre système ils doivent respecter ces pré-requis :

• L'architecture du compilateur et du client doivent avoir la même valeur CHOST.
• Les variables CFLAGS et CXXFLAGS doivent être compatibles avec tout les clients.
• Les USE flags pour les fonctionnalités spécifiques aux instructions du processeur (comme MMX, SSE, etc.) doivent être sélectionnés avec prudence, tout les clients doivent les supporter.

Remarque
Les paquets binaires sont distribués en tant que partie officielle du projet binhost de Gentoo et utilise un jeu d'instructions minimum et des paramètres de conservation du compilateur de manière à être le plus largement utilisable autant que faire se peut. À titre d'exemple, le mot-clé amd64 paramétré pour des paquets construit avec les options -march=x86-64 -mtune=generic et qui fonctionne sur n'importe quelle machine qui tourne avec le jeu d'instructions de x86-64 (amd64).

Important
Portage ne peut pas valider (NdT: ces paquets) si c'est pré-requis ne sont pas correspondants. En cas de doutes, il est de la responsabilités de l'administrateur de s'assurer de la validité de ces paramètres.

Gérer les *FLAGS en détail

L'utilitaire app-misc/resolve-march-native peut être trouvé dans une sous-suite de CFLAGS qui est supporté par à la fois le serveur et le(s) client(s). Par exemple, l'hôte peut retourner :

user $resolve-march-native

 -march=skylake -mabm -mrtm --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=12288 

Pendant ce temps là le client peut retourner :

user $resolve-march-native

 -march=ivybridge -mno-rdrnd --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=3072 

Dans cet exemple CFLAGS peut être paramétré avec -march=ivybridge -mno-rdrnd avec -march=ivybridge qui est une sous-suite de -march=skylake. -mabm et -mrtm ne sont pas supporté par le client. Cependant, -mno-rdrnd est inclus comme le client ne supporte pas -mrdrnd. Pour trouver quel -march est une sous-suite, vérifier le manuel de GCC, s'il n'y a pas de sous-suite comme -march=x86-64.

Optionally, it is also possible to set -mtune=some-arch or -mtune=native to tell gcc to tune code to a specific arch. In contrast to -march, the -mtune argument does not prevent code from being executed on other processors. For example, to compile code which is compatible with ivybridge and up but is tuned to run best on skylake set CFLAGS to -march=ivybridge -mtune=skylake. When -mtune is not set it defaults to whatever -march is set to.

When changing -march to a lower subset for using binary packages on a client, a full recompilation is required to make sure that all binaries are compatible with the client's processor, to save time packages that are not compiled with e.g. gcc/clang can be excluded:

user $emerge -e @world --exclude="acct-group/* acct-user/* virtual/* app-eselect/* sys-kernel/* sys-firmware/* dev-python/* dev-java/* dev-ruby/* dev-perl/* dev-lua/* dev-php/* dev-tex/* dev-texlive/* x11-themes/* */*-bin"

Similarly, app-portage/cpuid2cpuflags can be used to find a suitable subset of processor specific instruction set USE flags. For example, the host might return:

user $cpuid2cpuflags

 CPU_FLAGS_X86: aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3 

While the client might return:

user $cpuid2cpuflags

 CPU_FLAGS_X86: avx f16c mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3 

In this example CPU_FLAGS_X86 can be set to avx f16c mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3 in /etc/portage/make.conf because these flags are supported by both the client and the host

Next to these, Portage can check if the binary package is built using the same USE flags as expected on the client. Unless using --usepkgonly (-K) or --getbinpkgonly (-G), if a package is built with a different USE flag combination, Portage will either ignore the binary package (and use source-based build) or fail, depending on the options passed to the emerge command upon invocation (see Installing binary packages).

On clients, a few configuration changes are needed in order for the binary packages to be used.

Installer des paquets binaires

There are a few options that can be passed on to the emerge command that inform Portage about using binary packages:

Option	Description
--usepkg (-k)	Tries to use the binary package(s) in the locally available packages directory. Useful when using NFS or SSHFS mounted binary package hosts. If the binary packages are not found, a regular (source-based) installation will be performed.
--usepkgonly (-K)	Similar to --usepkg (-k) but fail if the binary package cannot be found. This option is useful if only pre-built binary packages are to be used.
--getbinpkg (-g)	Download the binary package(s) from a remote binary package host. If the binary packages are not found, a regular (source-based) installation will be performed.
--getbinpkgonly (-G)	Similar to --getbinpkg (-g) but will fail if the binary package(s) cannot be downloaded. This option is useful if only pre-built binary packages are to be used.

In order to automatically use binary package installations, the appropriate option can be added to the EMERGE_DEFAULT_OPTS variable:

FILE /etc/portage/make.confAutomatically fetch binary packages and build from source if not available

EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --getbinpkg"

There is a Portage feature that forces emerge to always try to fetch files from the binary package host:

FILE /etc/portage/make.confEnabling getbinpkg in the FEATURES variable

FEATURES="getbinpkg"

Note that this is not equivalent to --getbinpkg (-g) in the EMERGE_DEFAULT_OPTS variable. Especially, --getbinpkg=n on the command line won't override the feature.

Verify binary package OpenPGP signatures

Important
OpenPGP signing and verification is only available for the GPKG binpkg format.

Portage will try to verify the binary package's signature whenever possible, but users must first set up trusted local keys. app-portage/getuto can be used to set up a local trust anchor and update the keys in /etc/portage/gnupg. Portage calls getuto automatically with --getbinpkg or --getbinpkgonly.

This configures portage such that it trusts the Gentoo Release Engineering keys as also contained in the package sec-keys/openpgp-keys-gentoo-release for binary installation purposes.

Changes to the configuration can be done as root using gpg with the parameter --homedir=/etc/portage/gnupg. This way allows importing additional signing keys (e.g. for non-standard installation sources) and declare them as trusted.

To add a custom signing key:

1. Generate (or use an existing) key with signing abilities, and export the public key to a file.
2. Run getuto if it has never run:
   root #getuto
3. Use gpg --homedir=/etc/portage/gnupg --import public.key to import the public key in portage's keyring.
4. Trust and sign this key using the key created by getuto. In order to do this, first get the password to unlock the key at /etc/portage/gnupg/pass, then use:
   root #gpg --homedir=/etc/portage/gnupg --edit-key YOURKEYID
   Type sign, yes, paste (or type) the password. The key is now signed. To trust it, type trust, then 4 to trust it fully. Finally, type save.
5. Update the trustdb so that GPG considers the key valid:
   root #gpg --homedir=/etc/portage/gnupg --check-trustdb

If you hit any issues, check if a pre-existing /etc/portage/gnupg existed. If it did, move it away and then repeat the above steps.

Congratulations, Portage now has a working keyring!

Important
Trusting the key marginally or less will not work

By default, Portage will only verify GPG signatures when a signature file is found in a package, which allows the user to mix signed and unsigned GPKG binary packages from different sources, and allows to use old XPAK format binary packages.

If the user wishes to force signature verification, the binpkg-request-signature feature needs to be enabled. This feature assumes that all packages should be signed and rejects any unsigned package. Note that this feature does not support per-binhost configuration.

FILE /etc/portage/make.confEnabling Portage's binpkg-request-signature feature

# Require that all binpkgs be signed and reject them if they are not (or have an invalid sig)
FEATURES="binpkg-request-signature"

Pulling packages from a binary package host

Attention !
"The PORTAGE_BINHOST variable is deprecated in favor of the /etc/portage/binrepos.conf configuration file" - make.conf(5)

When using a binary package host, clients need to have the PORTAGE_BINHOST variable set in /etc/portage/make.conf or the sync-uri variable in /etc/portage/binrepos.conf. Without this configuration, the client will not know where the binary packages are stored which results in Portage being unable to retrieve them.

FILE /etc/portage/binrepos.confSetting binhost sync-uri

[binhost]
sync-uri = https://example.com/binhost
priority = 10

For each binhost, a name can be configured in the brackets. sync-uri must point to the directory in which the Packages file resides. Optionally, priority can be set. When a package exists in multiple binary package repositories, the package is pulled from the binary package host with the highest priority. This way, a preferred binary package host can be set up.

Many Gentoo stages already come with a preinstalled /etc/portage/binrepos.conf file, which points to the corresponding binary packages generated during the stage builds.

Remarque
The support for multiple binary package servers is somewhat incomplete. If several servers serve a binary package for the same package version, then only the first one will be considered. This can be problematic when these binary packages differ in their USE variable configuration and the USE variable configuration of a later binary package would match the systems configuration.

Réinstaller des paquets binaires modifiés

Passing the --rebuilt-binaries option to emerge will reinstall every binary that has been rebuilt since the package was installed. This is useful in case rebuilding tools like revdep-rebuild are run on the binary package server.

A related option is --rebuilt-binaries-timestamp. It causes emerge not to consider binary packages for a re-install if those binary packages have been built before the given time stamp. This is useful to avoid re-installing all packages, if the binary package server had to be rebuild from scratch but --rebuilt-binaries is used otherwise.

Additional client settings

Next to the getbinpkg feature, Portage also listens to the binpkg-logs feature. It controls if log files for successful binary package installations should be kept. It is only relevant if the PORT_LOGDIR variable has been set and is enabled by default.

Similar to excluding binary packages for a certain set of packages or categories, clients can be configured to exclude binary package installations for a certain set of packages or categories.

To accomplish this, use the --usepkg-exclude option:

root #emerge -uDNg @world --usepkg-exclude "sys-kernel/gentoo-sources virtual/*"

To enable such additional settings for each emerge command, add the options to the EMERGE_DEFAULT_OPTS variable in the make.conf file:

FILE /etc/portage/make.confEnabling emerge settings on every invocation

EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --usepkg-exclude 'sys-kernel/gentoo-sources virtual/*'"

Updating packages on the binary package host

Important
Do not use --changed-use(-U) when updating packages on the binary package host, doing so will cause packages with added or removed USE flags to be skipped, which will cause their installation from binary package on the client to fail due to non-matching USE between the source ebuild and binary package (if the client's --binpkg-respect-use=y, the default). Use --newuse(-N), which will always rebuild packages even for added or removed USE flags, ensuring the binary package stays in sync with the source ebuild.

Créer des paquets binaires

Il y a trois méthodes principales pour créer des paquets binaires :

1. Après une installation standard, utiliser l'application quickpkg.
2. Explicitement, au cours d'une opération emerge en utilisant l'option --buildpkg (-b)
3. Automatiquement, en utilisant la valeur de buildpkg pour la variable FEATURES de Portage.

Les trois méthodes créeront un paquet binaire dans le répertoire indiqué par la variable PKGDIR (qui par défaut prend la valeur /var/cache/binpkgs).

Utilisation de --buildpkg comme option de emerge

Lorsque l'installation du logiciel est opérée par emerge, Portage peut installer des fichiers binaires si l'option --buildpkg (-b) est ajoutée à la ligne de commande :

root #emerge --ask --buildpkg sys-devel/gcc

Il est aussi possible de demander à Portage de créer seulement un paquet binaire mais sans installer le logiciel sur le système à chaud. Dans ce cas, utiliser l'option --buildpkgonly (-B) :

root #emerge --ask --buildpkgonly sys-devel/gcc

Cette dernière approche nécessite toutefois que toutes les dépendances de compilation soient préalablement installées.

Utilisation de buildpkg comme variable FEATURE de Portage

La façon la plus courante de créer automatiquement des paquets binaires lorsqu'un paquet est installé par Portage est d'utiliser la fonctionnalité buildpkg, qui peut être spécifiée comme suit dans /etc/portage/make.conf:

FILE /etc/portage/make.confActiver la fonctionnalité buildpkg de Portage

FEATURES="buildpkg"

Une fois cette fonctionnalité activée, chaque fois que Portage installera un logiciel, il créera un paquet binaire également.

Comment exclure la création de certains paquets

Il est possible de dire à Portage de ne pas créer de paquets binaires pour quelques paquets ou catégories sélectionnés. Cela se fait en passant l'option --buildpkg-exclude à emerge :

root #emerge -uDN @world --buildpkg --buildpkg-exclude "virtual/* sys-kernel/*-sources"

Cette façon de faire peut être retenue pour les paquets dont le déploiement au format binaire ne présente que peu ou pas d'avantage. Des exemples pourraient être les paquets source du noyau Linux ou les packages binaires déployés en amont (ceux se terminant par -bin comme www-client/firefox-bin).

Binary package compression formats

It is possible to use a specific compression type on binary packages. Currently, the following formats are supported: bzip2, gzip, lz4, lzip, lzop, xz, and zstd. Defaults to zstd. Review man make.conf and search for BINPKG_COMPRESS for the most up-to-date information.

The compression format can be specified via make.conf.

FILE /etc/portage/make.confSpecify binary package compression format

BINPKG_COMPRESS="lz4"

Note that the compression type used might require extra dependencies to be installed, for example, in this case app-arch/lz4.

Binary package OpenPGP signing

Important
OpenPGP signing and verification is only available for the GPKG binpkg format.

A PGP signature enables Portage to check the creator and integrity of a binary package, and to perform trust management based on PGP keys. The binary package signing feature is disabled by default. To use it, enable the binpkg-signing feature. Note that whether this feature is enabled does not affect the signature verification feature.

FILE /etc/portage/make.confEnabling Portage's binpkg-signing feature

FEATURES="binpkg-signing"

Users also need to set the BINPKG_GPG_SIGNING_GPG_HOME and BINPKG_GPG_SIGNING_KEY variables for Portage to find the signing key.

FILE /etc/portage/make.confConfiguring Portage's signing key

BINPKG_GPG_SIGNING_GPG_HOME="/root/.gnupg"
BINPKG_GPG_SIGNING_KEY="0x1234567890ABCDEF"

Portage will only try to unlock the PGP private key at the beginning. If the user's key will expire over time, then consider enabling gpg-keepalive to prevent signing failures.

FILE /etc/portage/make.confEnabling Portage's gpg-keepalive feature

FEATURES="gpg-keepalive"

Conseil
gpg-agent by default expires cache entries after 2 hours. This means that, by default, if an emerge session lasts longer than 2 hours, signing of binpkgs will eventually fail regardless of FEATURES="gpg-keepalive". To prevent this problem, set max-cache-ttl to some large value (e.g. 34560000) in $BINPKG_GPG_SIGNING_GPG_HOME/gpg-agent.conf.

Utilisation de quickpkg

L'application quickpkg prend comme argument un ou plusieurs atomes (ou ensembles de paquets) et crée des paquets binaires pour tous les paquets "installés" qui leur correspondent.

Attention, cette méthode doit être utilisée avec prudence, car elle part des fichiers installés, ce qui peut poser problème en cas de fichiers de configurations spécifiques à la machine source. Les administrateurs changent souvent ces fichiers de configuration après avoir installé les logiciels. Dès lors que l'inclusion des fichiers de configuration pourrait poser des problèmes de diffusion de données importantes (et parfois même confidentielles) dans les paquets, par défaut quickpkg n'inclut pas les fichiers de configuration qui sont protégés par la méthode CONFIG_PROTECT. Pour forcer l'inclusion de ces fichiers de configuration, utiliser les options --include-config ou --include-unmodified-config.

Par exemple, pour créer les paquets binaires de toutes les versions installées de GCC :

root #quickpkg sys-devel/gcc

To create binary packages for the system set:

root #quickpkg @system

Pour créer les paquets binaires de tous les paquets installés sur le système, utiliser le glob * :

root #quickpkg "*/*"

Mise en place d'un serveur de paquets binaires

Portage prend en charge un certain nombre de protocoles pour télécharger des paquets binaires : FTP, FTPS, HTTP, HTTPS et SSH/SFTP. Cela permet d'envisager de nombreuses implémentations éventuelles de serveurs de paquets binaires.

Il n'y a cependant aucune méthode "prête à l'emploi" fournie par Portage pour distribuer des paquets binaires. Selon la configuration souhaitée, des logiciels supplémentaires devront être installés.

Un serveur de paquets binaires sur Internet

Une approche courante pour distribuer des packages binaires consiste à créer un hôte de packages binaires raccordé à Internet.

HTTPD

Utilisez un serveur Web tel que lighttpd (www-servers/lighttpd) et configurez-le pour fournir un accès en lecture à l'emplacement de la variable PKGDIR de /etc/portage/make.conf.

FILE /etc/lighttpd/lighttpd.conflighttpd: exemple de configuration

# Ajouter ceci à la fin de la configuration standard
server.modules += ( "mod_alias" )
alias.url = ( "/packages" => "/var/cache/binpkgs/" )

Caddy

To set up the Caddy HTTP server to provide a web-based binary package host, create a Caddyfile containing:

FILE Caddyfile

x.x.x.x:80 { # Replace x.x.x.x with your host's IPv4 address
    root * /path/to/binhost/var/cache/binpkgs
    file_server browse # Needed to server 
}

Once that is created, run Caddy with:

root #caddy run --config /path/to/Caddyfile

Puis, sur les systèmes clients, configurer la variable PORTAGE_BINHOST en conséquence:

FILE /etc/portage/make.confUtilisation d'un serveur de paquets binaires sur Internet

PORTAGE_BINHOST="http://binhost.example.com/packages"

SSH binary package host

To provide an authenticated approach for binary package mirrors, Portage can be configured to use the SSH protocol to access binary packages.

When using SSH, it is possible to use the root Linux user's SSH key (without passphrase as the installations need to happen in the background) to connect to a remote binary package host.

To accomplish this, make sure that the root user's SSH key is allowed on the server. This will need to happen for each machine that will connect to the SSH capable binary host:

root #cat root.id_rsa.pub >> /home/binpkguser/.ssh/authorized_keys

The PORTAGE_BINHOST variable could then look like so:

FILE /etc/portage/make.confSetting PORTAGE_BINHOST for SSH access

PORTAGE_BINHOST="ssh://binpkguser@binhostserver/var/cache/binpkgs"

If the SSH server is listening to a different port (e.g 25), then it must be specified after the address, like so:

FILE /etc/portage/make.confSetting PORTAGE_BINHOST for SSH access on port 25

PORTAGE_BINHOST="ssh://binpkguser@binhostserver:25/var/cache/binpkgs"

Remarque
Do not use the SSH configuration files found in ~/.ssh/config for setting ports or username. This location is ignored when Portage tries to rsync the packages back onto the client. Instead set all the options correctly in the PORTAGE_BINHOST variable.

NFS exported

When using binary packages on an internal network, it might be easier to export the packages through NFS and mount it on the clients.

The /etc/exports file could look like so:

FILE /etc/exportsExporting the packages directory

/var/cache/binpkgs   2001:db8:81::/48(ro,no_subtree_check,root_squash) 192.168.100.0/24(ro,no_subtree_check,root_squash)

On the clients, the location can then be mounted. An example /etc/fstab entry would look like so:

FILE /etc/fstabEntry for mounting the packages folder

binhost:/var/cache/binpkgs      /var/cache/binpkgs    nfs    defaults    0 0

The NFS share is mounted on the local filesystem, so there is no need to set PORTAGE_BINHOST or use the --getbinpkg option. Instead, follow the normal procedures for installing binary packages, remembering to point PKGDIR to the NFS share so that portage knows where to find the packages:

FILE /etc/portage/make.confSetting the package directory for portage

PKGDIR="/var/cache/binpkgs"

Remarque
If PKGDIR is network-mounted, it may be advantageous to enable FEATURES="pkgdir-index-trusted". This feature disables checking the entire PKGDIR for added or removed packages and instead trusts the contents of the Packages file to be accurate. This significantly improves performance on high-latency networks.

Maintaining binary packages

Exporting and distributing the binary packages will lead to useless storage consumption if the binary package list is not actively maintained.

Removing outdated binary packages

In the gentoolkit package an application called eclean is provided. It allows for maintaining Portage-related variable files, such as downloaded source code files, but also binary packages.

The following command will remove all binary packages that have no corresponding ebuild in the installed ebuild repositories:

root #eclean packages

For more details please read the eclean article.

Another tool that can be used is the qpkg tool from the app-portage/portage-utils package. However, this tool is a bit less configurable.

To clean up unused binary packages (in the sense of used by the server on which the binary packages are stored):

root #qpkg -c

Maintaining the Packages file

Conseil
As of portage-3.0.52, Portage defaults to FEATURES=pkgdir-index-trusted for performance, which requires an accurate Packages index. This can be disabled if it is an inconvenience to regularly fix up the index with emaint after manual changes.

Inside the packages directory exists a manifest file called Packages. This file acts as a cache for the metadata of all binary packages in the packages directory. The file is updated whenever Portage adds a binary package to the directory. Similarly, eclean updates it when it removes binary packages.

If for some reason binary packages are simply deleted or copied into the packages directory, or the Packages file gets corrupted or deleted, then it must be recreated. This is done using emaint command:

root #emaint binhost --fix

To clear the cache of all binary packages:

root #rm -r /var/cache/binpkgs/*

Advanced topics

Chrooting

If creating packages for a different Portage profile or system with different USE flags, a chroot can be created.

Remarque
This example uses /var/chroot/buildenv as the chroot path, but any path can be used.

Creating the directories

First, the directories for this chroot must be created:

root #mkdir --parents /var/chroot/buildenv

Deploying the build environment

Next, the appropriate stage 3 tarball must be downloaded and extracted, here the desktop profile | openrc tarball is being used:

/var/chroot/buildenv/ #wget https://bouncer.gentoo.org/fetch/root/all/releases/amd64/autobuilds/20230528T163158Z/stage3-amd64-desktop-openrc-20230528T163158Z.tar.xz

This can be extracted with the following command:

/var/chroot/buildenv/ #tar xpvf stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner

Configuring the build environment

Important
{{{1}}}

The build environment should be configured to match that of the system it is building for. The simplest way to do this is to copy the /etc/portage and /var/lib/portage/world files. This can be done with rsync:

Remarque
This command should be executed on the build target machine, where the remote host has the chroot.

user $rsync --archive --whole-file --verbose /etc/portage/* larry@remote_host:/var/chroot/buildenv/etc/portage

user $rsync --archive --whole-file --verbose /var/db/repos/* larry@remote_host:/var/chroot/buildenv/var/db/repos

Remarque
{{{1}}}

This process should be repeated for the world file:

user $rsync --archive --whole-file --verbose /var/lib/portage/world larry@remote_host:/var/chroot/buildenv/var/lib/portage/world

Remarque
/var/lib/portage and /var/lib/portage/world should have the root:portage permissions.

Configuring the chroot

Once created, mounts must be bound for the chroot to work:

/var/chroot/buildenv #mount --types proc /proc proc

/var/chroot/buildenv #mount --rbind /dev dev

/var/chroot/buildenv #cp --dereference /etc/resolv.conf etc

Remarque
If a tmpfs is being used for portage's temp dir, ensure that is mounted.

Entering the chroot

To enter this chroot, the following command can be used:

/var/chroot/buildenv #chroot . /bin/bash

Optionally, the prompt can be set to reflect the fact that the chroot is active:

/ #export PS1="(chroot) $PS1"

Performing an initial build

Remarque
This step assumes this configuration has been completed: Setting portage to use buildpkg.

This step is optional, but rebuilds all packages in the new world:

(chroot) #emerge --emptytree @world

Building for other architectures

crossdev is a tool to easily build cross-compile toolchains. This is useful to create binary packages for installation on a system whose architecture differs from that of the system used to build the packages. A common example would be building binary packages for a device like an arm64 Raspberry Pi from a more powerful amd64 desktop PC.

An installation guide for sys-devel/crossdev can be found at the crossdev page.

Build a cross compiler

Using crossdev with the following command can build a toolchain for the desired system:

root #crossdev --stable -t <arch-vendor-os-libc>

For the rest of this section, the example target will be for a Raspberry Pi 4:

root #crossdev --stable -t aarch64-unknown-linux-gnu

After this has built, a toolchain will have been created in /usr/aarch64-unknown-linux-gnu, and will look like a bare-bones Gentoo install where it is possible to edit Portage settings as normal.

Conseil
Replacing aarch64-unknown-linux-gnu with aarch64-unknown-linux-musl would build a system with the Musl libc rather than Glibc.

Basic setup

Removing the -pam flag from the USE line in /usr/aarch64-unknown-linux-gnu/etc/portage/make.conf is generally recommended in a setup like this:

FILE /usr/aarch64-unknown-linux-gnu/etc/portage/make.confDisable the pam USE flag

CHOST=aarch64-unknown-linux-gnu
CBUILD=x86_64-pc-linux-gnu
 
ROOT=/usr/${CHOST}/
 
ACCEPT_KEYWORDS="${ARCH}"
 
USE="${ARCH}"
 
CFLAGS="-O2 -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"
 
FEATURES="-collision-protect sandbox buildpkg noman noinfo nodoc"
# Ensure pkgs from another repository are not overwritten
PKGDIR=${ROOT}var/cache/binpkgs/
 
#If you want to redefine PORTAGE_TMPDIR uncomment (and/or change the directory location) the following line
PORTAGE_TMPDIR=${ROOT}var/tmp/
 
PKG_CONFIG_PATH="${ROOT}usr/lib/pkgconfig/"
#PORTDIR_OVERLAY="/var/db/repos/local/"

Profiles

List available profiles for the device by running:

root #PORTAGE_CONFIGROOT=/usr/aarch64-unknown-linux-gnu eselect profile list

Next, select the profile that best suits:

root #PORTAGE_CONFIGROOT=/usr/aarch64-unknown-linux-gnu eselect profile set <profile number>

Build a single package

To build a single binary package for use on the device, use the following:

root #emerge-aarch64-unknown-linux-gnu --ask foo

Build world file

To build every package in the world file, then the following command is needed:

root #emerge-aarch64-unknown-linux-gnu --emptytree @world

Binary location

By default, all binary packages will be stored in /usr/aarch64-unknown-linux-gnu/var/cache/binpkgs, so this is the location needed to be selected when setting up a binary package host.

Creating snapshots of the packages directory

When deploying binary packages for a large number of client systems it might become worthwhile to create snapshots of the packages directory. The client systems then do not use the packages directory directly but use binary packages from the snapshot.

Snapshots can be created using the /usr/lib/portage/python3.11/binhost-snapshot tool that comes with Portage (note that the path to that tool may need to be adjusted to match the python version with which Portage is installed). It takes four arguments:

1. A source directory (the path to the packages directory).
2. A target directory (that must not exist).
3. A URI.
4. A binary package server directory.

The files from the package directory are copied to the target directory. A Packages file is then created inside the binary package server directory (fourth argument) with the provided URI.

Client systems need to use an URI that points to the binary package server directory. From there they will be redirected to the URI that was given to binhost-snapshot. This URI has to refer to the target directory.

Comprendre le format des paquets binaires

XPAK format

XPAK format binary packages created by Portage have the file name ending with .tbz2. These files consist of two parts:

1. A .tar.bz2 archive containing the files that will be installed on the system.
2. A xpak archive containing package metadata, the ebuild, and the environment file.

See man xpak for a description of the format.

In app-portage/portage-utils some tools exists that are able to split or create tbz2 and xpak files.

The following command will split the tbz2 into a .tar.bz2 and an .xpak file:

user $qtbz2 -s <paquet>.tbz2

The .xpak file can be examined using the qxpak utility.

To list the contents:

user $qxpak -l <paquet>.xpak

The next command will extract a file called USE which contains the enabled USE flags for this package:

user $qxpak -x package-manager-0.xpak USE

GPKG format

GPKG format binary packages created by Portage have the file name ending with .gpkg.tar. These files consist of four parts at least:

1. A gpkg-1 empty file that used to identify the format.
2. A C/PV/metadata.tar{.compression} archive containing package metadata, the ebuild, and the environment file.
3. A C/PV/image.tar{.compression} archive containing the files that will be installed on the system.
4. A Manifest file containing checksums to protect against file corruption.
5. Multiple optional .sig files containing OpenPGP signature are used for integrity checking and verification of trust.

The format can be extracted by tar without the need for additional tools.

The PKGDIR layout

The currently used format version 2 has the following layout:

CODE Packages directory layout (version 2)

PKGDIR
`+- Packages
 +- app-accessibility/
 |  +- pkg1-version.tbz2
 |  `- pkgN-version.tbz2
 +- app-admin/
 |  `- ...
 `- ...

The Packages file is the major improvement (and also the trigger for Portage to know that the binary package directory uses version 2) over the first binary package directory layout (version 1). In version 1, all binary packages were also hosted inside a single directory (called All/) and the category directories only had symbolic links to the binary packages inside the All/ directory.

In portage-3.0.15 and later, FEATURES=binpkg-multi-instance is enabled by default:

CODE Packages directory layout (version 2 + FEATURES=binpkg-multi-instance)

PKGDIR
`+- Packages
 +- app-accessibility/
 |  +- pkg1/
 |    +- pkg1-version-build_id.xpak
 |    `- pkgN-version-build_id.xpak
 +- app-admin/
 |  `- ...
 `- ...

Unpacking with quickunpkg

Zoobab wrote a simple shell tool named quickunpkg to quickly unpack tbz2 files.

See also

• Binary package quickstart — using the fully-supported Gentoo binary package host for amd64 and arm64 with the settings the packages are compiled with
• Emerge — la principale interface en ligne de commande de Portage
• Portage — gestionnaire de paquet et le système de distribution officiel de Gentoo.
• Project:Binhost — aims to provide readily installable, precompiled packages for a subset of configurations, via central binary package hosting

External resources

quickpkg man page.