Test environment

From Gentoo Wiki
Jump to:navigation Jump to:search
This article has been flagged for not conforming to the wiki guidelines. Please help Gentoo out by starting fixing things.

Testing your ebuilds (see also Package testing) can be a tedious task. Beside a simple re-emerge of the package in question, to see whether it merges successfully, a good testing practice usually needs to take one or more of the following questions into account:

  • Having a clean gentoo installation to test with: Using your day to day desktop, might miss on dependencies, which you happen to have installed already and thus producing false positives, i.e. letting your ebuild successfully install, while on a new system it would have been failed due to missing dependencies.
  • An exhaustive testing of all possible USE flag combinations: For ebuilds with only a few USE flags, this can be done easily by hand. For packages with a lot of USE flags, such an approach is error-prone. You might be better by writing a shell or python script to enumerate all the possibilities.
  • Profile testing: Here testing of default vs. hardened and multilib vs. no-multilib profiles is of interest. It may for example uncover problems with PIC/PIE code in hardened profiles or problems of missing proper multiclassing in the ebuild. Usually you don't want to switch profiles just for the purpose of testing an ebuild.
  • Keyword testing: This requires the proper hardware and is done by the arch projects.

So to run proper and efficient tests for your ebuild, a dedicated test environment seems necessary. There are several options for this, like using a chroot environment, VM's, a containerized enviroment or even dedicated hardware. The table below summarizes some of the pros and cons of these options.

Method Pros Cons
chroot guide
  • fast setup
  • small memory footprint
need to reset the environment after each emerge
chroot using btrfs
  • fast setup
  • small memory footprint
  • needs btrfs
  • need to reset the environment after each emerge
Virtual Machine
  • easy to set up from a live DVD
  • snapshot's can be used to avoid reinstallation
large memory footprint
Container

Using Docker containers for ebuild testing

dev-util/ebuildtester is a python script for testing ebuilds within a Docker container. The script compiles a docker container with the parameters passed at invocation time and either installs the specified package or puts the user into a shell inside the container. Usage of the script is simple (see http://ebuildtester.readthedocs.io for more details):

user $ebuildtester --help
ebuildtester [-h] [--atom ATOM [ATOM ...]] [--manual]
                           --portage-dir PORTAGE_DIR
                           [--overlay-dir OVERLAY_DIR] [--update]
                           [--threads N] [--use USE [USE ...]] [--unmask ATOM]
                           [--gcc-version VER] [--with-X]

optional arguments:
  -h, --help            show this help message and exit
  --atom ATOM [ATOM ...]
                        The package atom(s) to install
  --manual              Install package manually
  --portage-dir PORTAGE_DIR
                        The local portage directory
  --overlay-dir OVERLAY_DIR
                        Add overlay dir (can be used multiple times)
  --update              Update container before installing atom
  --threads N           Use N threads to build packages
  --use USE [USE ...]   The use flags for the atom
  --unmask ATOM         Unmask atom (can be used multiple times)
  --gcc-version VER     Use gcc version VER
  --with-X              Install VNC server to test graphical applications

The --portage-dir option is mandatory, as well as use of either --atom or --manual. You can pass in one or more additional overlays with the --overlay-dir option. The script maps the portage and overlay dirs into the container, so changes to the files inside the container will affect the files outside the container on your file system. A one time use case for this script could look something like

CODE One-time test
ebuildtester --portage-dir /usr/portage \
  --overlay-dir /usr/local/portage \
  --use R boost imaging python qt5 rendering views \
  --unmask =sci-libs/vtk-8.0.1 \
  --with-X \
  --atom =sci-libs/vtk-8.0.1

assuming, the vtk-8.0.1.ebuild file has been modified in a local repository at /usr/local/portage.

Using LXC containers for ebuild testing

These instructions make use of unprivileged LXC containers.

user $lxc-create -t download -n proxy-maint

Then for Distribution, choose gentoo; Release, choose current; and Architecture, choose amd64.

When it completes, start up the container with,

user $lxc-start -n proxy-maint

Then configure a basic development environment using the recommended tools.

FILE /etc/portage/sets/tools
# tools
app-crypt/gnupg
app-editors/vim # or just symlink busybox to `/usr/local/bin/vi` and update your environment variables.
app-portage/tatt
dev-vcs/git
dev-util/pkgcheck
dev-util/pkgdev
user $lxc-console -n proxy-maint
root #emerge --ask @tools

To stop the container,

user $lxc-stop -n proxy-maint

To clone the container for working on a particular ebuild,

user $lxc-copy -n proxy-maint -N "${PN}" # clones the original container

To delete it when finished,

user $lxc-destroy -n "${PN}" # destroy the container when finished

Using Kubler for ebuild testing

See Kubler for more info.

Kubler may be used to develop against any ebuild repository.

Create a new namespace, let's call it edev:

user $kubler new namespace edev

Create a new builder, select kubler/bob as IMAGE_PARENT:

user $kubler new builder edev/bob

Edit the new builder's build.sh and add any additional repositories:

Note
For ::gentoo only there's no need to add the repo, it's already available; binding in the development fork and updating .bashrc is probably desirable, however.
FILE /config/build.sh
configure_builder() {
    # we overwrite this with a local host mount later, but this takes care of the initial overlay setup in the builder for us
    add_overlay kubler https://github.com/edannenberg/kubler-overlay.git
    # just for convenience
    echo 'cd /var/db/repos/kubler' >> ~/.bashrc
}

Create a new image, let's call it bench, use kubler/bash as IMAGE_PARENT:

user $kubler new image edev/bench

Edit the new image's build.conf and configure the builder and ebuild repo to be mounted builder:

FILE /config/build.conf
BUILDER="edev/bob"
BUILDER_MOUNTS=(
    "/data/development/ebuild-repos/gentoo:/var/db/repos/gentoo"
    "/data/development/ebuild-repos/kubler-overlay:/var/db/repos/kubler"
)

Start an interactive build container and get tinkering:

user $kubler build -i edev/bench
root #ebuild dev-lang/foo/foo-0.4.0.ebuild manifest merge

See also