User:Juippis/The ultimate testing system with lxd

The ultimate way of testing ebuild contributions.

Summary: You keep a base container updated and in a tidy condition. You make copies of the base container, use the copies to test ebuild contributions, and delete the copies when you're done. This is automated as much as possible; in an ideal situation all you ever have to do, is give one command.

You can of course use this built environment to test your own ebuild modifications before pushing. Please see some examples how it works.

Installation
We're going to need LXD to be installed in the host system.

You should expect to have at least 2-4 GB space per container, especially if they pull rust-bin, gentoo-kernel-bin etc. Therefore it might be wise to symlink somewhere with more space available. Using an SSD/NVME is heavily suggested, as it makes all the operations happen in an instant compared to HDDs.

Configuring lxd
Add your user to newly created lxd group. your chosen lxd directory properly for this user.

We will use a binhost @ localhost to be used between host/container. Set up subuid & subgid for accessing binpkg repository from both in and outside the container.

Getting the correct container image
Make sure to choose an image suited for your testing needs.

This will download a default gentoo-x86_64 container image, set it up, name it as my-test-container and start it. You can check that it works,

But for now, we'll want to turn it off so we can configure it properly.

default profile
You can edit the 'default' profile which, by default, gets used by containers and contain some necessary options.

Note especially the network part. LXD

Accessing display from container to test GUI runtime
Edit the config file of your container.

Sharing host's disk to container
With the example above, we're sharing our host's distfiles and binpkg directories with the containers.

Launching the container
Now that we've configured the relevant parts, we can launch the container. We'll construct the base container image that is kept up-to-date and clean. We'll use that to create discardable copies, where the testing happens.

Set up git repositories
Or alternatively, the latest portage tree from any of the mirror's.

Edit relevant /etc/portage files
Edit where relevant.

This will install rust-bin instead if a dep pulls it. Note that with binpkgs it does not matter, your system rust should be compatible.

Edit where relevant. These are just "common requirements" I run into all the time. Some packages have especially nasty REQUIRED_USE that stops the automated tests.

Optional. I find that ~ruby causes a lot of problems for automation to deal with.

Always find the latest here.

Binhost / binpkgs
With the above settings we are enabling binpkgs to be generated, and used, for everything other except packages we're actually testing. This will speed up testing process immensively. Make sure your user has correct permissions in your host to be able to write to shared binpkg directory.

Update your container
Start the update.

Let's use the latest gcc.

Set up pkg-testing-tool
Our testing depends heavily on https://github.com/slashbeast/pkg-testing-tools.

Finishing touches for your container
Run to submit the initial list, and from now on running  from a snapshot-container or main container will only submit modified package list when comparing to base image.

Log out, turn off container.

(Ctrl+d works too)

Automatic maintain scripts
Write a new script to your user's  that ideally gets cron job'd 1-4 times a day depending on your needs. Note that running this script will require your main container to be turned off, as it should be unless you're doing manual maintenance. This script can obviously be called manually whenever beginning to work with PRs, if you test packages more sporadically. If you work with your containers daily and update the base image daily, this process is relatively fast. It will delete all inactive copies, as that's the desired action to do.

Optional: If you'd like, now's a good time to make a snapshot of your base image. Depending on your workflow and use cases, this is most likely not needed though.

You can use the snapshot should you accidentally mess with the base image, or if you want a single image to work on. But it requires you to keep the snapshot updated too.

cron job
Use your desired cron system to make the maintenance script a cron job.

PR testing script
Intended to be used with Github pull requests, but it can be modified to work with any git-format patches, such as some bugzilla attachments.

By default the script creates a new discardable snapshot for each PR, but you can obviously do all testing in a single discardable snapshot image to save space and possibly time.

Testing that everything works!
Choose any PR to your liking from https://github.com/gentoo/gentoo/pulls. Do note that it can't be CLOSED, in other words, merged to main tree as that will cause patch merging collision. And it should edit an .ebuild file, ideally adding a new .ebuild to the tree to be tested.

You can always, at any given time log in to the container and inspect / test everything manually. This is especially handy if you spot a mistake in the .ebuild file and need to test that your modification works.

Examples
A couple of real-life examples with full output below.

'''Example 1: Simple PR with no deps. Gets ran with  and  .'''

All looks good. Proceed to merge using pram (or if you share your ::gentoo repository between container/host, just do.

Example 2: PR bumping a package with some USE flags, and a test phase available.

All looks good. Proceed to merge using pram (or if you share your ::gentoo repository between container/host, just do.

Example 3: A PR with multiple commits touching different packages:

Example 4: How it looks like when emerging contribution fails. {{Cmd|test-pr.sh 20348|collapse-output=true|output= Initializing new test environment... this will take a while. Applying: dev-php/pecl-parallel: New package: v1.1.4 Packages to be tested: dev-php/pecl-parallel-1.1.4

[INFO] >>> Following testing jobs will be executed: =dev-php/pecl-parallel-1.1.4   USE: php_targets_php7-3 php_targets_php7-4 php_targets_php8-0 =dev-php/pecl-parallel-1.1.4   USE: -php_targets_php7-3 php_targets_php7-4 -php_targets_php8-0 =dev-php/pecl-parallel-1.1.4   USE: php_targets_php7-3 -php_targets_php7-4 -php_targets_php8-0 =dev-php/pecl-parallel-1.1.4   USE: -php_targets_php7-3 php_targets_php7-4 php_targets_php8-0 =dev-php/pecl-parallel-1.1.4   USE: php_targets_php7-3 -php_targets_php7-4 php_targets_php8-0 =dev-php/pecl-parallel-1.1.4   USE: -php_targets_php7-3 -php_targets_php7-4 php_targets_php8-0 =dev-php/pecl-parallel-1.1.4   USE:, FEATURES: test [INFO] >>> Running (1 of 7) =dev-php/pecl-parallel-1.1.4 with USE: php_targets_php7-3 php_targets_php7-4 php_targets_php8-0

These are the packages that would be merged, in order:

Calculating dependencies \[binary N     ] dev-libs/oniguruma-6.9.6-r1:0/5::gentoo  USE="-crnl-as-line-terminator -static-libs" ABI_X86="32 (64) (-x32)" 0 KiB [binary NS    ] sys-libs/db-5.3.28-r5:5.3::gentoo [6.0.35-r3:6.0::gentoo] USE="-cxx -doc -examples -java -tcl -test" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] app-admin/metalog-20200113-r1::gentoo  USE="(unicode)" 0 KiB [binary N     ] virtual/logger-0-r1::gentoo  0 KiB [binary N     ] dev-libs/apr-1.7.0-r1:1/1.7::gentoo  USE="urandom -doc -older-kernels-compatibility (-selinux) -static-libs" 0 KiB [binary N     ] dev-libs/apr-util-1.6.1-r6:1::gentoo  USE="berkdb gdbm -doc -ldap (-libressl) -mysql -nss -odbc -openssl -postgres -sqlite -static-libs" 0 KiB [binary N     ] app-admin/apache-tools-2.4.46::gentoo  USE="ssl (-libressl)" 0 KiB [binary N     ] mail-mta/nullmailer-2.2-r1::gentoo  USE="ssl -test" 0 KiB [binary N     ] virtual/mta-1-r2::gentoo  0 KiB [binary N     ] www-servers/apache-2.4.46-r6:2::gentoo  USE="gdbm (split-usr) ssl suexec-caps threads -debug -doc -ldap (-libressl) (-selinux) -static -suexec -suexec-syslog" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_core authn_dbm authn_file authz_core authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers http2 include info log_config logio mime mime_magic negotiation rewrite setenvif socache_shmcb speling status unique_id unixd userdir usertrack vhost_alias -access_compat -asis -auth_digest -auth_form -authn_dbd -authn_socache -authz_dbd -brotli -cache_disk -cache_socache -cern_meta -charset_lite -dbd -dumpio -ident -imagemap -lbmethod_bybusyness -lbmethod_byrequests -lbmethod_bytraffic -lbmethod_heartbeat -log_forensic (-lua) -macro -md -proxy -proxy_ajp -proxy_balancer -proxy_connect -proxy_fcgi -proxy_ftp -proxy_html -proxy_http -proxy_http2 -proxy_scgi -proxy_wstunnel -ratelimit -remoteip -reqtimeout -session -session_cookie -session_crypto -session_dbd -slotmem_shm -socache_memcache -substitute -version -watchdog -xml2enc" APACHE2_MPMS="-event -prefork -worker" LUA_SINGLE_TARGET="lua5-1 -lua5-2 -lua5-3 -lua5-4" 0 KiB [binary N     ] app-eselect/eselect-php-0.9.7::gentoo  USE="apache2 -fpm" 0 KiB [ebuild N     ] dev-lang/php-7.3.27-r1:7.3::gentoo  USE="acl berkdb bzip2 cli ctype fileinfo filter flatfile gdbm hash iconv ipv6 jit json opcache phar posix readline session simplexml ssl threads tokenizer unicode xml zlib -apache2 -argon2 -bcmath -calendar -cdb -cgi -cjk -coverage -curl -debug -embed -enchant -exif -firebird -fpm -ftp -gd -gmp -imap -inifile -intl -iodbc -kerberos -ldap -ldap-sasl -libedit (-libressl) -lmdb -mhash -mssql -mysql -mysqli -nls -oci8-instant-client -odbc -pcntl -pdo -phpdbg -postgres -qdbm -recode (-selinux) -session-mm -sharedmem -snmp -soap -sockets -sodium -spell -sqlite -systemd -sysvipc -test -tidy -tokyocabinet -truetype -wddx -webp -xmlreader -xmlrpc -xmlwriter -xpm -xslt -zip -zip-encryption" 11854 KiB [ebuild N     ] dev-lang/php-8.0.3:8.0::gentoo  USE="acl berkdb bzip2 cli ctype fileinfo filter flatfile gdbm iconv ipv6 jit opcache phar posix readline session simplexml ssl threads tokenizer unicode xml zlib -apache2 -argon2 -bcmath -calendar -cdb -cgi -cjk -coverage -curl -debug -embed -enchant -exif -ffi -firebird -fpm -ftp -gd -gmp -imap -inifile -intl -iodbc -kerberos -ldap -ldap-sasl -libedit (-libressl) -lmdb -mhash -mssql -mysql -mysqli -nls -oci8-instant-client -odbc -pcntl -pdo -phpdbg -postgres -qdbm (-selinux) -session-mm -sharedmem -snmp -soap -sockets -sodium -spell -sqlite -systemd -sysvipc -test -tidy -tokyocabinet -truetype -webp -xmlreader -xmlwriter -xpm -xslt -zip" 10504 KiB [ebuild N     ] dev-lang/php-7.4.16:7.4::gentoo  USE="acl berkdb bzip2 cli ctype fileinfo filter flatfile gdbm iconv ipv6 jit json opcache phar posix readline session simplexml ssl threads tokenizer unicode xml zlib -apache2 -argon2 -bcmath -calendar -cdb -cgi -cjk -coverage -curl -debug -embed -enchant -exif -ffi -firebird -fpm -ftp -gd -gmp -imap -inifile -intl -iodbc -kerberos -ldap -ldap-sasl -libedit (-libressl) -lmdb -mhash -mssql -mysql -mysqli -nls -oci8-instant-client -odbc -pcntl -pdo -phpdbg -postgres -qdbm (-selinux) -session-mm -sharedmem -snmp -soap -sockets -sodium -spell -sqlite -systemd -sysvipc -test -tidy -tokyocabinet -truetype -webp -xmlreader -xmlrpc -xmlwriter -xpm -xslt -zip" 10093 KiB [ebuild N     ] dev-php/pecl-parallel-1.1.4::gentoo  USE="-test" PHP_TARGETS="php7-3 php7-4 php8-0" 58 KiB

Total: 15 packages (14 new, 1 in new slot, 11 binaries), Size of downloads: 32508 KiB

The following USE changes are necessary to proceed: (see "package.use" in the portage(5) man page for more details) >=dev-lang/php-7.3.27-r1:7.3 threads >=dev-lang/php-7.4.16:7.4 threads >=dev-lang/php-8.0.3 threads
 * 1) required by dev-php/pecl-parallel-1.1.4::gentoo[php_targets_php7-3,-test]
 * 2) required by =dev-php/pecl-parallel-1.1.4 (argument)
 * 1) required by dev-php/pecl-parallel-1.1.4::gentoo[-test,php_targets_php7-4]
 * 2) required by =dev-php/pecl-parallel-1.1.4 (argument)
 * 1) required by dev-php/pecl-parallel-1.1.4::gentoo[-test,php_targets_php8-0]
 * 2) required by =dev-php/pecl-parallel-1.1.4 (argument)

Autounmask changes successfully written. ... done! [binary N     ] dev-libs/oniguruma-6.9.6-r1:0/5::gentoo  USE="-crnl-as-line-terminator -static-libs" ABI_X86="32 (64) (-x32)" 0 KiB [binary NS    ] sys-libs/db-5.3.28-r5:5.3::gentoo [6.0.35-r3:6.0::gentoo] USE="-cxx -doc -examples -java -tcl -test" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] app-admin/metalog-20200113-r1::gentoo  USE="(unicode)" 0 KiB [binary N     ] virtual/logger-0-r1::gentoo  0 KiB [binary N     ] dev-libs/apr-1.7.0-r1:1/1.7::gentoo  USE="urandom -doc -older-kernels-compatibility (-selinux) -static-libs" 0 KiB [binary N     ] dev-libs/apr-util-1.6.1-r6:1::gentoo  USE="berkdb gdbm -doc -ldap (-libressl) -mysql -nss -odbc -openssl -postgres -sqlite -static-libs" 0 KiB [binary N     ] app-admin/apache-tools-2.4.46::gentoo  USE="ssl (-libressl)" 0 KiB [binary N     ] mail-mta/nullmailer-2.2-r1::gentoo  USE="ssl -test" 0 KiB [binary N     ] virtual/mta-1-r2::gentoo  0 KiB [binary N     ] www-servers/apache-2.4.46-r6:2::gentoo  USE="gdbm (split-usr) ssl suexec-caps threads -debug -doc -ldap (-libressl) (-selinux) -static -suexec -suexec-syslog" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_core authn_dbm authn_file authz_core authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers http2 include info log_config logio mime mime_magic negotiation rewrite setenvif socache_shmcb speling status unique_id unixd userdir usertrack vhost_alias -access_compat -asis -auth_digest -auth_form -authn_dbd -authn_socache -authz_dbd -brotli -cache_disk -cache_socache -cern_meta -charset_lite -dbd -dumpio -ident -imagemap -lbmethod_bybusyness -lbmethod_byrequests -lbmethod_bytraffic -lbmethod_heartbeat -log_forensic (-lua) -macro -md -proxy -proxy_ajp -proxy_balancer -proxy_connect -proxy_fcgi -proxy_ftp -proxy_html -proxy_http -proxy_http2 -proxy_scgi -proxy_wstunnel -ratelimit -remoteip -reqtimeout -session -session_cookie -session_crypto -session_dbd -slotmem_shm -socache_memcache -substitute -version -watchdog -xml2enc" APACHE2_MPMS="-event -prefork -worker" LUA_SINGLE_TARGET="lua5-1 -lua5-2 -lua5-3 -lua5-4" 0 KiB [binary N     ] app-eselect/eselect-php-0.9.7::gentoo  USE="apache2 -fpm" 0 KiB [ebuild N     ] dev-lang/php-7.3.27-r1:7.3::gentoo  USE="acl berkdb bzip2 cli ctype fileinfo filter flatfile gdbm hash iconv ipv6 jit json opcache phar posix readline session simplexml ssl threads tokenizer unicode xml zlib -apache2 -argon2 -bcmath -calendar -cdb -cgi -cjk -coverage -curl -debug -embed -enchant -exif -firebird -fpm -ftp -gd -gmp -imap -inifile -intl -iodbc -kerberos -ldap -ldap-sasl -libedit (-libressl) -lmdb -mhash -mssql -mysql -mysqli -nls -oci8-instant-client -odbc -pcntl -pdo -phpdbg -postgres -qdbm -recode (-selinux) -session-mm -sharedmem -snmp -soap -sockets -sodium -spell -sqlite -systemd -sysvipc -test -tidy -tokyocabinet -truetype -wddx -webp -xmlreader -xmlrpc -xmlwriter -xpm -xslt -zip -zip-encryption" 11854 KiB [ebuild N     ] dev-lang/php-8.0.3:8.0::gentoo  USE="acl berkdb bzip2 cli ctype fileinfo filter flatfile gdbm iconv ipv6 jit opcache phar posix readline session simplexml ssl threads tokenizer unicode xml zlib -apache2 -argon2 -bcmath -calendar -cdb -cgi -cjk -coverage -curl -debug -embed -enchant -exif -ffi -firebird -fpm -ftp -gd -gmp -imap -inifile -intl -iodbc -kerberos -ldap -ldap-sasl -libedit (-libressl) -lmdb -mhash -mssql -mysql -mysqli -nls -oci8-instant-client -odbc -pcntl -pdo -phpdbg -postgres -qdbm (-selinux) -session-mm -sharedmem -snmp -soap -sockets -sodium -spell -sqlite -systemd -sysvipc -test -tidy -tokyocabinet -truetype -webp -xmlreader -xmlwriter -xpm -xslt -zip" 10504 KiB [ebuild N     ] dev-lang/php-7.4.16:7.4::gentoo  USE="acl berkdb bzip2 cli ctype fileinfo filter flatfile gdbm iconv ipv6 jit json opcache phar posix readline session simplexml ssl threads tokenizer unicode xml zlib -apache2 -argon2 -bcmath -calendar -cdb -cgi -cjk -coverage -curl -debug -embed -enchant -exif -ffi -firebird -fpm -ftp -gd -gmp -imap -inifile -intl -iodbc -kerberos -ldap -ldap-sasl -libedit (-libressl) -lmdb -mhash -mssql -mysql -mysqli -nls -oci8-instant-client -odbc -pcntl -pdo -phpdbg -postgres -qdbm (-selinux) -session-mm -sharedmem -snmp -soap -sockets -sodium -spell -sqlite -systemd -sysvipc -test -tidy -tokyocabinet -truetype -webp -xmlreader -xmlrpc -xmlwriter -xpm -xslt -zip" 10093 KiB [ebuild N     ] dev-php/pecl-parallel-1.1.4::gentoo  USE="-test" PHP_TARGETS="php7-3 php7-4 php8-0" 58 KiB

Total: 15 packages (14 new, 1 in new slot, 11 binaries), Size of downloads: 32508 KiB

>>> Verifying ebuild manifests >>> Emerging binary (1 of 15) dev-libs/oniguruma-6.9.6-r1::gentoo >>> Emerging binary (2 of 15) sys-libs/db-5.3.28-r5::gentoo >>> Emerging binary (3 of 15) app-admin/metalog-20200113-r1::gentoo >>> Emerging binary (4 of 15) dev-libs/apr-1.7.0-r1::gentoo >>> Installing (2 of 15) sys-libs/db-5.3.28-r5::gentoo >>> Installing (1 of 15) dev-libs/oniguruma-6.9.6-r1::gentoo >>> Installing (3 of 15) app-admin/metalog-20200113-r1::gentoo >>> Installing (4 of 15) dev-libs/apr-1.7.0-r1::gentoo >>> Emerging binary (5 of 15) virtual/logger-0-r1::gentoo >>> Installing (5 of 15) virtual/logger-0-r1::gentoo >>> Emerging binary (6 of 15) mail-mta/nullmailer-2.2-r1::gentoo >>> Installing (6 of 15) mail-mta/nullmailer-2.2-r1::gentoo >>> Emerging binary (7 of 15) virtual/mta-1-r2::gentoo >>> Emerging binary (8 of 15) dev-libs/apr-util-1.6.1-r6::gentoo >>> Installing (7 of 15) virtual/mta-1-r2::gentoo >>> Installing (8 of 15) dev-libs/apr-util-1.6.1-r6::gentoo >>> Emerging binary (9 of 15) app-admin/apache-tools-2.4.46::gentoo >>> Installing (9 of 15) app-admin/apache-tools-2.4.46::gentoo >>> Emerging binary (10 of 15) www-servers/apache-2.4.46-r6::gentoo >>> Installing (10 of 15) www-servers/apache-2.4.46-r6::gentoo >>> Emerging binary (11 of 15) app-eselect/eselect-php-0.9.7::gentoo >>> Installing (11 of 15) app-eselect/eselect-php-0.9.7::gentoo >>> Emerging (12 of 15) dev-lang/php-7.3.27-r1::gentoo >>> Emerging (13 of 15) dev-lang/php-8.0.3::gentoo >>> Emerging (14 of 15) dev-lang/php-7.4.16::gentoo >>> Installing (14 of 15) dev-lang/php-7.4.16::gentoo >>> Installing (13 of 15) dev-lang/php-8.0.3::gentoo >>> Installing (12 of 15) dev-lang/php-7.3.27-r1::gentoo >>> Emerging (15 of 15) dev-php/pecl-parallel-1.1.4::gentoo >>> Failed to emerge dev-php/pecl-parallel-1.1.4, Log file: >>> '/var/tmp/portage/vbslogs/build/dev-php/pecl-parallel-1.1.4:20210414-114543.log' >>> Jobs: 14 of 15 complete, 1 failed              Load avg: 4.46, 3.44, 1.56 * Package:   dev-php/pecl-parallel-1.1.4 * Repository: gentoo * Maintainer: jaco@uls.co.za proxy-maint@gentoo.org * USE:       abi_x86_64 amd64 elibc_glibc kernel_linux php_targets_php7-3 php_targets_php7-4 php_targets_php8-0 userland_GNU * FEATURES:  network-sandbox preserve-libs sandbox userpriv usersandbox

>>> Unpacking source... >>> Unpacking pecl-parallel-1.1.4.tgz to /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work >>> Source unpacked in /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work >>> Preparing source in /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/parallel-1.1.4 ... * Running '/usr/lib64/php7.3/bin/phpize' ... [ ok ] * Running eautoreconf in '/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php7.3' ... * Running 'libtoolize --install --copy --force' ... [ ok ] * Running 'aclocal' ... [ ok ] * Running 'autoconf --force' ... [ ok ] * Running 'autoheader' ... [ ok ] * Running elibtoolize in: php7.3/ *  Applying portage/1.2.0 patch ... *   Applying sed/1.5.6 patch ... *   Applying as-needed/2.4.3 patch ... * Running '/usr/lib64/php7.4/bin/phpize' ... [ ok ] * Running 'autoconf --force' ... [ ok ] * Running 'autoheader' ... [ ok ] * Running '/usr/lib64/php8.0/bin/phpize' ... [ ok ] * Running 'autoconf --force' ... [ ok ] * Running 'autoheader' ... [ ok ] >>> Source prepared. >>> Configuring source in /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/parallel-1.1.4 ... * econf: updating php7.4/config.guess with /usr/share/gnuconfig/config.guess * econf: updating php7.4/build/config.guess with /usr/share/gnuconfig/config.guess * econf: updating php7.4/build/config.sub with /usr/share/gnuconfig/config.sub * econf: updating php7.4/config.sub with /usr/share/gnuconfig/config.sub * econf: updating php8.0/config.guess with /usr/share/gnuconfig/config.guess * econf: updating php8.0/build/config.guess with /usr/share/gnuconfig/config.guess * econf: updating php8.0/build/config.sub with /usr/share/gnuconfig/config.sub * econf: updating php8.0/config.sub with /usr/share/gnuconfig/config.sub * econf: updating php7.3/config.guess with /usr/share/gnuconfig/config.guess * econf: updating php7.3/config.sub with /usr/share/gnuconfig/config.sub ./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --docdir=/usr/share/doc/pecl-parallel-1.1.4 --htmldir=/usr/share/doc/pecl-parallel-1.1.4/html --with-sysroot=/ --libdir=/usr/lib64 --with-php-config=/usr/lib64/php7.3/bin/php-config ... ... it prints the whole build.log ... ... /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/src/cache.c:276:30: error: incompatible types when assigning to type 'zend_type' from type 'int' /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/src/check.c:129:85: warning: implicit declaration of function 'ZEND_TYPE_IS_CLASS'; did you mean 'ZEND_TYPE_HAS_CLASS'? [-Wimplicit-function-declaration] 129 |    if (ZEND_TYPE_IS_SET(it->type) && (ZEND_TYPE_CODE(it->type) == IS_OBJECT || ZEND_TYPE_IS_CLASS(it->type))) { |                                                                                ^      |                                                                                 ZEND_TYPE_HAS_CLASS /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/src/check.c:139:15: error: 'zend_arg_info' {aka 'struct _zend_arg_info'} has no member named 'pass_by_reference' 139 |        if (it->pass_by_reference) { |              ^~ /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/src/check.c:165:15: error: 'zend_arg_info' {aka 'struct _zend_arg_info'} has no member named 'pass_by_reference' 165 |        if (it->pass_by_reference) { |              ^~ /bin/sh /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/libtool --mode=compile x86_64-pc-linux-gnu-gcc -I. -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0 -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/include -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/main -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0 -I/usr/lib64/php8.0/include/php -I/usr/lib64/php8.0/include/php/main -I/usr/lib64/php8.0/include/php/TSRM -I/usr/lib64/php8.0/include/php/Zend -I/usr/lib64/php8.0/include/php/ext -I/usr/lib64/php8.0/include/php/ext/date/lib -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0 -DHAVE_CONFIG_H  -march=native -O2 -pipe -frecord-gcc-switches   -Wall -DZEND_ENABLE_STATIC_TSRMLS_CACHE=1  -c /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/src/loop.c -o src/loop.lo /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/src/check.c: In function 'php_parallel_check_class_inline': /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/src/check.c:538:13: warning: implicit declaration of function 'ZEND_TYPE_IS_CE'; did you mean 'ZEND_TYPE_IS_SET'? [-Wimplicit-function-declaration] 538 |        if (ZEND_TYPE_IS_CE(info->type)) { |            ^      |             ZEND_TYPE_IS_SET make: *** Error 1 make: *** Waiting for unfinished jobs.... make: *** Error 1 make: *** Error 1 x86_64-pc-linux-gnu-gcc -I. -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0 -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/include -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/main -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0 -I/usr/lib64/php8.0/include/php -I/usr/lib64/php8.0/include/php/main -I/usr/lib64/php8.0/include/php/TSRM -I/usr/lib64/php8.0/include/php/Zend -I/usr/lib64/php8.0/include/php/ext -I/usr/lib64/php8.0/include/php/ext/date/lib -I/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0 -DHAVE_CONFIG_H -march=native -O2 -pipe -frecord-gcc-switches -Wall -DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 -c /var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0/src/loop.c -fPIC -DPIC -o src/.libs/loop.o * ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): *  emake failed * * If you need support, post the output of `emerge --info '=dev-php/pecl-parallel-1.1.4::gentoo'`, * the complete build log and the output of `emerge -pqv '=dev-php/pecl-parallel-1.1.4::gentoo'`. * The complete build log is located at '/var/tmp/portage/vbslogs/build/dev-php/pecl-parallel-1.1.4:20210414-114543.log'. * For convenience, a symlink to the build log is located at '/var/tmp/portage/dev-php/pecl-parallel-1.1.4/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/dev-php/pecl-parallel-1.1.4/temp/environment'. * Working directory: '/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/php8.0' * S: '/var/tmp/portage/dev-php/pecl-parallel-1.1.4/work/parallel-1.1.4' ... ... continues normally to test other USE flag combinations ... ... These are the packages that would be merged, in order:

Calculating dependencies... done! [ebuild  R    ] dev-php/pecl-parallel-1.1.4::gentoo  USE="test*" PHP_TARGETS="php7-3 php7-4* -php8-0" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests >>> Emerging (1 of 1) dev-php/pecl-parallel-1.1.4::gentoo >>> Installing (1 of 1) dev-php/pecl-parallel-1.1.4::gentoo >>> Jobs: 1 of 1 complete                          Load avg: 2.51, 3.04, 1.57 >>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

* GNU info directory index is up-to-date.

* IMPORTANT: 1 news items need reading for repository 'gentoo'. * Use eselect news read to view new items.

[ERROR] >>> Not all runs were successful. atom: =dev-php/pecl-parallel-1.1.4, USE flags: 'php_targets_php7-3 php_targets_php7-4 php_targets_php8-0' atom: =dev-php/pecl-parallel-1.1.4, USE flags: '-php_targets_php7-3 php_targets_php7-4 php_targets_php8-0' atom: =dev-php/pecl-parallel-1.1.4, USE flags: 'php_targets_php7-3 -php_targets_php7-4 php_targets_php8-0' atom: =dev-php/pecl-parallel-1.1.4, USE flags: '-php_targets_php7-3 -php_targets_php7-4 php_targets_php8-0' Grepping through the logs for any errors or QA notices... /var/tmp/portage/vbslogs/build/dev-php/pecl-parallel-1.1.4:20210414-114543.log: * ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): /var/tmp/portage/vbslogs/build/dev-php/pecl-parallel-1.1.4:20210414-114625.log: * ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): /var/tmp/portage/vbslogs/build/dev-php/pecl-parallel-1.1.4:20210414-114637.log: * ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): /var/tmp/portage/vbslogs/build/dev-php/pecl-parallel-1.1.4:20210414-114650.log: * ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): /var/tmp/portage/vbslogs/elog/dev-php/pecl-parallel-1.1.4:20210414-114647.log:ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): /var/tmp/portage/vbslogs/elog/dev-php/pecl-parallel-1.1.4:20210414-114558.log:ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): /var/tmp/portage/vbslogs/elog/dev-php/pecl-parallel-1.1.4:20210414-114655.log:ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): /var/tmp/portage/vbslogs/elog/dev-php/pecl-parallel-1.1.4:20210414-114634.log:ERROR: dev-php/pecl-parallel-1.1.4::gentoo failed (compile phase): Finished grepping. }}

It's easy to spot in here, that a single enabled USE flag causes build failures.

Example 5: QA notices. {{Cmd|test-pr.sh 19702|collapse-output=true|output= Initializing new test environment... this will take a while. Applying: app-misc/rox-filer-2.11 libSM dependency Packages to be tested: app-misc/rox-filer-2.11

[INFO] >>> Following testing jobs will be executed: =app-misc/rox-filer-2.11   USE: [INFO] >>> Running (1 of 1) =app-misc/rox-filer-2.11 with USE:

These are the packages that would be merged, in order:

Calculating dependencies... done! [binary N     ] media-gfx/graphite2-1.3.14::gentoo  USE="-perl -test" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] gnome-base/gsettings-desktop-schemas-3.38.0::gentoo  USE="introspection" 0 KiB [binary N     ] x11-themes/hicolor-icon-theme-0.17::gentoo  0 KiB [binary N     ] dev-libs/fribidi-1.0.9::gentoo  USE="-static-libs" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] dev-libs/lzo-2.10:2::gentoo  USE="(split-usr) -examples -static-libs" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] media-fonts/liberation-fonts-2.1.2::gentoo  USE="X -fontforge" 0 KiB [binary N     ] media-libs/libpng-1.6.37-r2:0/16::gentoo  USE="-apng -static-libs" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="sse" 0 KiB [binary N     ] virtual/ttf-fonts-1-r1::gentoo  0 KiB [binary N     ] x11-themes/gnome-themes-standard-3.28::gentoo  0 KiB [binary N     ] media-libs/freetype-2.10.4:2::gentoo  USE="X adobe-cff bzip2 cleartype_hinting png -bindist -brotli -debug -doc -fontforge -harfbuzz -infinality -static-libs -utils" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] sys-libs/binutils-libs-2.35.2-1:0/2.35.2::gentoo  USE="-64-bit-bfd (-cet) -multitarget -nls -static-libs" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-libs/libXcomposite-0.4.5::gentoo  USE="-doc" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-libs/libXcursor-1.2.0::gentoo  USE="-doc" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-libs/libXdamage-1.1.5::gentoo  ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-libs/libXi-1.7.10::gentoo  USE="-doc" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-libs/gdk-pixbuf-2.42.2:2::gentoo  USE="introspection -gtk-doc -jpeg -tiff" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] dev-libs/atk-2.36.0::gentoo  USE="introspection -gtk-doc" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] dev-util/gtk-update-icon-cache-3.24.26-1::gentoo  0 KiB [binary N     ] media-libs/fontconfig-2.13.1-r2:1.0::gentoo  USE="-doc -static-libs" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] app-eselect/eselect-fontconfig-1.1-r1::gentoo  0 KiB [binary N     ] x11-libs/cairo-1.16.0-r4::gentoo  USE="X glib svg (-aqua) -debug (-gles2-only) -opengl -static-libs -utils -valgrind" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-libs/libXft-2.3.3::gentoo  USE="-doc" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] media-libs/harfbuzz-2.7.4-1:0/0.9.18::gentoo  USE="cairo glib graphite introspection truetype -debug -doc -icu -static-libs -test" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-libs/pango-1.42.4-r2::gentoo  USE="X introspection -test" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] gnome-base/librsvg-2.50.3:2::gentoo  USE="introspection vala" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-themes/adwaita-icon-theme-3.38.0::gentoo  USE="-branding" 0 KiB [binary N     ] x11-libs/gtk+-2.24.32-r1:2::gentoo  USE="introspection (-aqua) -cups -examples -test -vim-syntax -xinerama" ABI_X86="32 (64) (-x32)" 0 KiB [binary N     ] x11-themes/gtk-engines-adwaita-3.28::gentoo  ABI_X86="32 (64) (-x32)" 0 KiB [ebuild N     ] gnome-base/libglade-2.6.4-r3:2.0::gentoo  USE="-static-libs -test" ABI_X86="32 (64) (-x32)" 348 KiB [ebuild N     ] app-misc/rox-filer-2.11::gentoo  USE="-test" 1863 KiB

Total: 30 packages (30 new, 28 binaries), Size of downloads: 2211 KiB

>>> Verifying ebuild manifests >>> Emerging binary (1 of 30) media-gfx/graphite2-1.3.14::gentoo >>> Emerging binary (2 of 30) gnome-base/gsettings-desktop-schemas-3.38.0::gentoo >>> Emerging binary (3 of 30) x11-themes/hicolor-icon-theme-0.17::gentoo >>> Emerging binary (4 of 30) dev-libs/fribidi-1.0.9::gentoo >>> Emerging binary (5 of 30) dev-libs/lzo-2.10::gentoo >>> Emerging binary (6 of 30) media-fonts/liberation-fonts-2.1.2::gentoo >>> Emerging binary (7 of 30) media-libs/libpng-1.6.37-r2::gentoo >>> Emerging binary (8 of 30) sys-libs/binutils-libs-2.35.2::gentoo >>> Installing (8 of 30) sys-libs/binutils-libs-2.35.2::gentoo >>> Emerging binary (9 of 30) x11-libs/libXcomposite-0.4.5::gentoo >>> Installing (9 of 30) x11-libs/libXcomposite-0.4.5::gentoo >>> Emerging binary (10 of 30) x11-libs/libXcursor-1.2.0::gentoo >>> Installing (3 of 30) x11-themes/hicolor-icon-theme-0.17::gentoo >>> Emerging binary (11 of 30) x11-libs/libXdamage-1.1.5::gentoo >>> Installing (4 of 30) dev-libs/fribidi-1.0.9::gentoo >>> Emerging binary (12 of 30) x11-libs/libXi-1.7.10::gentoo >>> Installing (6 of 30) media-fonts/liberation-fonts-2.1.2::gentoo >>> Emerging binary (13 of 30) dev-libs/atk-2.36.0::gentoo >>> Installing (7 of 30) media-libs/libpng-1.6.37-r2::gentoo >>> Emerging binary (14 of 30) virtual/ttf-fonts-1-r1::gentoo >>> Installing (2 of 30) gnome-base/gsettings-desktop-schemas-3.38.0::gentoo >>> Emerging binary (15 of 30) media-libs/freetype-2.10.4::gentoo >>> Installing (1 of 30) media-gfx/graphite2-1.3.14::gentoo >>> Emerging binary (16 of 30) x11-themes/gnome-themes-standard-3.28::gentoo >>> Installing (5 of 30) dev-libs/lzo-2.10::gentoo >>> Emerging binary (17 of 30) x11-libs/gdk-pixbuf-2.42.2::gentoo >>> Installing (10 of 30) x11-libs/libXcursor-1.2.0::gentoo >>> Installing (11 of 30) x11-libs/libXdamage-1.1.5::gentoo >>> Installing (12 of 30) x11-libs/libXi-1.7.10::gentoo >>> Installing (13 of 30) dev-libs/atk-2.36.0::gentoo >>> Installing (14 of 30) virtual/ttf-fonts-1-r1::gentoo >>> Installing (15 of 30) media-libs/freetype-2.10.4::gentoo >>> Emerging binary (18 of 30) media-libs/fontconfig-2.13.1-r2::gentoo >>> Installing (16 of 30) x11-themes/gnome-themes-standard-3.28::gentoo >>> Installing (17 of 30) x11-libs/gdk-pixbuf-2.42.2::gentoo >>> Emerging binary (19 of 30) dev-util/gtk-update-icon-cache-3.24.26::gentoo >>> Installing (18 of 30) media-libs/fontconfig-2.13.1-r2::gentoo >>> Emerging binary (20 of 30) app-eselect/eselect-fontconfig-1.1-r1::gentoo >>> Installing (19 of 30) dev-util/gtk-update-icon-cache-3.24.26::gentoo >>> Installing (20 of 30) app-eselect/eselect-fontconfig-1.1-r1::gentoo >>> Emerging binary (21 of 30) x11-libs/cairo-1.16.0-r4::gentoo >>> Emerging binary (22 of 30) x11-libs/libXft-2.3.3::gentoo >>> Installing (21 of 30) x11-libs/cairo-1.16.0-r4::gentoo >>> Installing (22 of 30) x11-libs/libXft-2.3.3::gentoo >>> Emerging binary (23 of 30) media-libs/harfbuzz-2.7.4::gentoo >>> Installing (23 of 30) media-libs/harfbuzz-2.7.4::gentoo >>> Emerging binary (24 of 30) x11-libs/pango-1.42.4-r2::gentoo >>> Installing (24 of 30) x11-libs/pango-1.42.4-r2::gentoo >>> Emerging binary (25 of 30) gnome-base/librsvg-2.50.3::gentoo >>> Installing (25 of 30) gnome-base/librsvg-2.50.3::gentoo >>> Emerging binary (26 of 30) x11-themes/adwaita-icon-theme-3.38.0::gentoo >>> Installing (26 of 30) x11-themes/adwaita-icon-theme-3.38.0::gentoo >>> Emerging binary (27 of 30) x11-libs/gtk+-2.24.32-r1::gentoo >>> Installing (27 of 30) x11-libs/gtk+-2.24.32-r1::gentoo >>> Emerging binary (28 of 30) x11-themes/gtk-engines-adwaita-3.28::gentoo >>> Installing (28 of 30) x11-themes/gtk-engines-adwaita-3.28::gentoo >>> Emerging (29 of 30) gnome-base/libglade-2.6.4-r3::gentoo >>> Installing (29 of 30) gnome-base/libglade-2.6.4-r3::gentoo >>> Emerging (30 of 30) app-misc/rox-filer-2.11::gentoo >>> Installing (30 of 30) app-misc/rox-filer-2.11::gentoo >>> Recording app-misc/rox-filer in "world" favorites file... >>> Jobs: 30 of 30 complete                        Load avg: 0.86, 0.39, 0.15

* Messages for package media-fonts/liberation-fonts-2.1.2:

* The following fontconfig configuration files have been installed: * *   60-liberation.conf * * Use `eselect fontconfig` to enable/disable them.

* Messages for package media-libs/fontconfig-2.13.1-r2:

* Please make fontconfig configuration changes using `eselect * fontconfig`. Any changes made to /etc/fonts/fonts.conf will be * overwritten. If you need to reset your configuration to upstream * defaults, delete the directory /etc/fonts/conf.d/ and re-emerge * fontconfig. * * (Note: Above message is only printed the first time package is * installed. Please look at /usr/share/doc/fontconfig-2.13.1-r2/README.gentoo* * for future reference)

* Messages for package x11-libs/gtk+-2.24.32-r1:

* Please install app-text/evince for print preview functionality. * Alternatively, check "gtk-print-preview-command" documentation and * add it to your gtkrc. * To make the gtk2 file chooser use 'current directory' mode by default, * edit ~/.config/gtk-2.0/gtkfilechooser.ini to contain the following: * [Filechooser Settings] * StartupMode=cwd * * (Note: Above message is only printed the first time package is * installed. Please look at /usr/share/doc/gtk+-2.24.32-r1/README.gentoo* * for future reference)

* Messages for package app-misc/rox-filer-2.11: * Log file: /var/tmp/portage/vbslogs/build/app-misc/rox-filer-2.11:20210305-174048.log

* QA Notice: Files built without respecting LDFLAGS have been detected * Please include the following list of files in your report: * /usr/bin/rox * QA Notice: Pre-stripped files found: * /usr/bin/rox >>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

* Regenerating GNU info directory index... * Processed 86 info files.

[INFO] >>> All good. Grepping through the logs for any errors or QA notices... /var/tmp/portage/vbslogs/build/app-misc/rox-filer-2.11:20210305-174048.log: * QA Notice: Files built without respecting LDFLAGS have been detected /var/tmp/portage/vbslogs/build/app-misc/rox-filer-2.11:20210305-174048.log: * QA Notice: Pre-stripped files found: /var/tmp/portage/vbslogs/elog/app-misc/rox-filer-2.11:20210305-174058.log:QA Notice: Files built without respecting LDFLAGS have been detected /var/tmp/portage/vbslogs/elog/app-misc/rox-filer-2.11:20210305-174058.log:QA Notice: Pre-stripped files found: Finished grepping. my-test-container-snap-19702 ~ # emacs /var/tmp/portage/vbslogs/build/app-misc/rox-filer-2.11:20210305-174048.log }}

rdep testing script
Test reverse dependencies of a package. Useful to see whether some update causes a breakage. Takes a package list from https://qa-reports.gentoo.org/output/genrdeps/rindex.

TODO: Upload scripts into Github, link here.

Examples
Incoming.

-native-symlinks PR/ebuild tester
Sometimes you see PRs fixing -native-symlinks bugs, but testing those can be annoying or hard.

TODO.

Examples
Example -native-symlinks PR being tested. Todo.

Acknowledgements / TODO

 * The scripts are most likely awful, but after I've gotten them to work, I haven't looked back. https://imgs.xkcd.com/comics/is_it_worth_the_time.png
 * The  itself can be used to build a tinderbox checking all commits, but an automated bug-reporting system would be needed then. However, it might be good for overlays. You'll need to store the git checkout of a previous run, then compare that to.
 * could run  before applying patches, but it's not needed for my own workflow since I usually do all testing once a day.
 * Arch-testing tools don't test with, I usually do it manually. For now, I've disabled ignoring  from pkg-testing-tool directly.
 * I need better error handling with . Take example list from https://github.com/toralf/tinderbox/blob/master/data/CATCH_QA and work with that. Also the rundown could use enhancements.
 * Not every PR needs to be tested with multiple USE flags, ie if there's a simple  fix or something. But you can easily modify the given scripts here to do a single run, I don't feel there's a need to "host" them for you.
 * I'm making this public to hear feedback! Ideally I'd want an infra-hosted service with Github hook that tests PRs on-demand. After a Gentoo developer has reviewed the PR, they could launch a build-test from Github, which would then return PASS or FAIL to the PR with web-browsable logs.
 * Only consecutive duplicate entries gets removed, in other words, if someone makes 4 commits to a, a, b, b, only a and b gets tested. However if someone commits to a, b, a, b, then a and b gets tested twice. First I did go through the array with, but the current method works 99 % of time and is much cleaner.
 * TODO: Study possibility to use SquashFS to reduce container image sizes.
 * TODO: Switch to tatt instead, with custom scripts?
 * TODO: Upload shell scripts / portage/* files to Github for easier distribution, and tracking for changes? Although they don't change that often.