ebuild repository

From Gentoo Wiki
(Redirected from Overlay)
Jump to:navigation Jump to:search
This page contains changes which are not marked for translation.
Other languages:
Deutsch • ‎English • ‎español • ‎français • ‎polski • ‎русский • ‎中文(中国大陆)‎ • ‎日本語 • ‎한국어

This page is a work in progress by kyoreln (talk | contribs). Treat its contents with caution.

An ebuild repository is a file-structure that can provide packages for installation on a Gentoo system. Ebuild repositories contain ebuilds, eclasses, and other types of descriptive metadata files that supply Portage with packages, news items, profile targets, etc.

The Gentoo ebuild repository is Gentoo Linux's primary and official ebuild repository - it contains all the information needed to build and install every package that makes up Gentoo. Additional ebuild repositories can be configured with Portage, to provide extra packages.

An ebuild repository configured with Portage can replace packages from the Gentoo ebuild repository, or from other ebuild repositories, by providing alternative ebuilds for packages. Portage will install the latest available version of a package from any configured ebuild repository, by default. If the latest available version is provided by several ebuild repositories, it will be chosen according to a set order of priority - hence the colloquial name overlay.

Administrators of Gentoo systems can configure additional ebuild repositories with Portage by using the utilities and methods described below.

GURU is an official ebuild repository maintained collaboratively by Gentoo users, with a little help from a few Gentoo developers. It is complimentary to the Gentoo ebuild repository, and the maintainers strive to maintain a reasonable level of quality for the packages provided. There is also a list of public ebuild repositories registered on repos.gentoo.org,

The Gentoo ebuild repository

The Gentoo ebuild repository is the main ebuild repository for a Gentoo Linux system, and is usually where all the packages come from. It is maintained on the gitweb.gentoo.org Gentoo server, and gets synchronized to local machines (/var/db/repos/gentoo), to be available to Portage.

The Gentoo ebuild repository contains ebuild files that tell Portage how to build and install each package that makes up an "official" Gentoo installation. These ebuilds describe all the software available, with metadata, dependency information, and everything else needed to get a package in working order.

The metadata provides the package's name, version, where to get sources from, available USE flags, license, website etc.

Dependency information in ebuilds allows Portage to pull in any other packages required to build and run a package that is to be installed - no more, no less. Dependencies are very granular in Gentoo, they will even vary depending on what use flags are selected, for ultimate selectivity.

Perhaps most importantly, the ebuilds contain the information required to configure, build (compile), install, and test each package - usually from a project's own source code.

In addition to the ebuilds, the Gentoo ebuild repository contains the official profiles, which define the default state of USE flags, default values for most variables found in /etc/portage/make.conf, the set of system packages, etc.

The Gentoo ebuild repository is also the place where the news items are posted, which is why any new news items will be highlighted after a Gentoo ebuild repository synchronization.

The Gentoo ebuild repository, and it's ebuilds, are maintained by the Gentoo developers and other members of the community.

The Gentoo ebuild repository will sometimes be called by shorter, or even colloquial, names, such as the Gentoo repository, the Gentoo repo, ::gentoo, gentoo.git, or occasionally just the "repo". It was historically known within the Gentoo community as the Portage tree, rsync tree, or sometimes just "the tree".

Where do ebuild repositories come from ?

Because an ebuild repository is simply a structure of files and directories, a new ebuild repository can be made available to Portage simply by copying those files and directories to a location known to Portage. The location of repositories configured for Portage is specified in /etc/portage/repos.conf, and the actual ebuild repository and it's files are usually under /var/db/repos/. Ebuild repositories can be configured on any accessible filesystem however, even on an nfs or SSHFS filesystem - allowing them to be stored on a network or Internet server.

As previously discussed, the Gentoo ebuild repository is hosted on gitweb.gentoo.org. That server also hosts other ebuild repositories.

In practice, except for maybe private and personal use, any additional ebuild repositories aren't just copied to a directory by hand and configured for Portage (meaning added to /etc/portage/repos.conf). Generally, new repositories are made available by third parties, and once configured for Portage, are synchronized by Portage. Synchronization copies all the files from a central remote location to the configured location on a locally available filesystem - in the case of the first synchronization - or updates the files from the remote location on subsequent synchronizations.

Because ebuild repositories are just file-structures, many methods can be used to synchronize them, and Portage offers several methods. Rsync is the default synchronization method, git is also popular. The synchronization method is specified in /etc/portage/repos.conf when configuring a repository, along with the information needed to retrieve it, for an rsync or git based repository.

Publicly available repositories can be registered on repos.gentoo.org for easy configuration with the management tools.

Repository management

Use the eselect repository tool to easily add, disable, or remove ebuild repositories configured with Portage. This tool also provides a handy way to list and add repositories available through being registered on repos.gentoo.org.

New ebuild repositories for use with Portage can also be created by the user.

Ebuild repositories can always be configured by editing /etc/portage/repos.conf manually.

Layman can still be used to manage repositories, but it is superseded by eselect repository for most uses.

Installing packages from other repositories

Packages from repositories other than the Gentoo ebuild repository can be installed with the emerge command, just as always:

root #emerge --ask category/atom

For example, once the GURU repository is added, to install the x11-misc/xbanish package from that repository:

root #emerge --ask x11-misc/xbanish
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   #] x11-misc/xbanish-1.7::guru  0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Note that the repository is not specified in the command. The "::guru" appended to the package atom shows what repository the package will be installed from. This works because the x11-misc/xbanish package is present in the GURU repository, but not in the Gentoo repository.

If multiple versions of the same package are available from two or more different ebuild repositories, Portage will install the most recent version.

Beware when performing system updates that if a newly added ebuild repository is providing newer versions of currently installed packages, these packages will be replaced with the newer versions from the overlay. Consider if any updates are desired.

See masking enabled ebuild repositories to avoid an ebuild repository blanket-upgrading packages. Note that the GURU ebuild repository intentionally avoids providing versions that would replace packages from the Gentoo ebuild repository.

If the latest version of a package is available from more than one ebuild repository, the repository with the highest priority will be used. The priority can be set for an ebuild repository in /etc/portage/repos.conf. The Gentoo ebuild repository has a default priority set to -1000, and the default if priority is not set is 0. If several ebuild repositories have the same priority (such as two or more not having a priority set, so having priority 0), the order is undetermined - a package to install will be chosen arbitrarily.

It is possible to instruct Portage to install a package from a specific ebuild repository with the :: version specifier:

root #emerge --ask category/atom::repository-name

The same notation can be used for different emerge instructions, including uninstalling a package through --depclean.

The list of active ebuild repositories with their priorities can be obtained through the output of one of the following commands:

user $emerge --info
user $portageq repos_config /

Repository synchronization

Ebuild repositories should be updated, aka synchronized, so that local copies will reflect a recent state of the parent repositories, to be able to keep the system up to date, and install current software. Regularly synchronizing with the Gentoo repository and updating the system in this way is important, to ensure that all the latest security updates are installed, and that the local system does not get too out of sync with the Gentoo repository, as this can make upgrades complicated.

Synchronization for each repository is configured through the files in /etc/portage/repos.conf.

If local repositories are not very up to date, synchronize the repositories and update the system, before installing packages.

Repository synchronization is managed with the emaint sync command:

root #emaint sync --help
usage: usage: emaint [options] COMMAND

The emaint program provides an interface to system health checks
and maintenance. See the emaint(1) man page for additional
information about the following commands:

  all            Perform all supported commands
  binhost        Scan and generate metadata indexes for binary packages.
  cleanconfmem   Check and clean the config tracker list for uninstalled packages.
  cleanresume    Discard emerge --resume merge lists
  logs           Check and clean old logs in the PORTAGE_LOGDIR.
  merges         Scan for failed merges and fix them.
  movebin        Perform package move updates for binary packages
  moveinst       Perform package move updates for installed and binary packages.
  sync           Check repos.conf settings and sync repositories.
  world          Check and fix problems in the world file.

optional arguments:
  -h, --help            show this help message and exit
  -c, --check           Check for problems (a default option for most modules)
  -f, --fix             Attempt to fix problems (a default option for most modules)
  --version             show program's version number and exit
  -C, --clean           Cleans out logs more than 7 days old (cleanlogs only) module-options: -t, -p
  -t NUM, --time NUM    (cleanlogs only): -t, --time Delete logs older than NUM of days
  -p, --pretend         (cleanlogs only): -p, --pretend Output logs that would be deleted
  -P, --purge           Removes the list of previously failed merges. WARNING: Only use this option if you plan on manually fixing them or do not want them re-installed.
  -y, --yes             (merges submodule only): Do not prompt for emerge invocations
  -r REPO, --repo REPO  (sync module only): -r, --repo Sync the specified repo
  -A, --allrepos        (sync module only): -A, --allrepos Sync all repos that have a sync-url defined
  -a, --auto            (sync module only): -a, --auto Sync auto-sync enabled repos only
  --sync-submodule {glsa,news,profiles}
                        (sync module only): Restrict sync to the specified submodule(s)

To sync all repositories for which auto-sync=true is set, run emaint sync with the --auto switch (-a for short). This is usually the command that should be run regularly, before system updates and package installation (and is equivalent to using the old emerge --sync command):

root #emaint sync --auto

To sync the foo repository (irrespective of the foo auto-sync setting):

root #emaint sync --repo foo

To sync all repositories with a valid sync-type and sync-url defined. (ignoring auto-sync settings):

root #emaint sync --allrepos

For any repositories that should not be synced when running emaint sync --auto, auto-sync = no must be set in the appropriate file in /etc/portage/repos.conf, due to the default being auto-sync = true.
The emerge --sync command is now only a compatibility command. Primary control of all sync operations has been moved from emerge to emaint, and emerge --sync now just calls the emaint sync module with the --auto option. This performs a sync on only those repositories with the auto-sync setting set to yes or true. If the auto-sync option is not set or is absent for the configured repositories, then emerge --sync may sync no repositories at all.
The emerge-webrsync tool can be used to download and install the daily Gentoo Repository rsync snapshot, to help with firewall restrictions, or to speed up the first synchronization, for example.

See man emaint for information on how to use the portage synchronization commands. See the Portage project sync article about migrating to the new modular sync system from Portage version 2.2.16, it contains important information, notably for users of eix-sync, esync -l, and emerge --sync && layman -S .

Best practices

Cache generation

When large ebuild repositories are installed, Portage may take a long time to perform operations like dependency resolution. This is because ebuild repositories do not usually contain a metadata cache.

Generate a local metadata cache by running emerge --regen after syncing the ebuild repositories:

root #emaint sync --allrepos
root #( ulimit -n 4096 && emerge --regen )

Be careful, because emerge --regen takes a lot of time and it's not recommended for rsync users as rsync updates the cache using server-side caches (most of users of portage are rsync users). Rsync users should simply run emerge --sync (or eix-sync) to regenerate the cache. It's probably only users of very large ebuild repositories should try emerge --regen.

Masking enabled ebuild repositories

When using large ebuild repositories or those with unknown/low quality code, it is best practice to hard mask the whole ebuild repository and only accept specific ebuilds on a case-by-case basis. For example, for an overlay named "repository-foobar":

FILE /etc/portage/package.maskMask all packages in an ebuild repository

Then add the specific package(s) from the repository-foobar overlay so that they will be available visible to Portage for installation:

FILE /etc/portage/package.unmaskUnmask a specific package in an ebuild repository

After the above unmask the package named "foo-category/bar" should be available and none of the other packages from the repository-foobar overlay will be available, which is by design.

See also

External resources