DISCLAIMER: This Article is still WIP, and the software it describes is not yet feature complete (although in an error-free, workable and helpful stage)
Overlay CI with Travis
This is a personal project to provide a useful Travis CI for Gentoo overlays hosted on GitHub. The aim is to catch non-functional ebuilds, uphold QA standards, and detect regressions in a packages test suite. Right now, only the first part is implemented. You can find the project here
What is CI
Continuous Integration is, overly simplified and from the perspective of your average GitHub project, the process of automatically testing changes to development branches, such as push and pull requests, for functionality, coverage and regressions.
What is Travis
How it works
Travis is configured via the travis.yml file to run multiple jobs as declared by the build matrix. In this project, it runs the run.sh script for every package and architecture specified in the travis.yml
Travis itself runs an Ubuntu VM (or container in the case of non-amd64). On this we simply run docker and use my containers to get into a gentoo environment and run the tests there.
The script looks at the pull diff, filters out any modified ebuilds, looks if those are keyworded for the specific arch runner, and then emerges them with FEATURES=test. Currently unimplemented is the ability to filter FEATURES=test on a per package basis and QA checks with pkgcheck.
How to use
Install in your overlay
To install this into your overlay, simply copy the contents of the src/ directory into your repository so that it contains .travis.yml and .travis
Edit the .travis.yml env and arch sections to include the packages and arches you want to test. On your repos travis-ci.com page, be sure to enable "Build pushed pull requests" in settings.
Actually using it
Travis can test on pushes and pulls. Since a push to master equals pushing to the users with overlays, you should put ALL ebuild changes in pull requests to master.
Caveats and Shortcomings
Travis jobs run with two CPU threads and 7.5 (4 on non-amd64) GB of RAM. This should suffice for most software, but I've seen some truly horrifying things on this planet.
The bigger issue is that travis jobs are limited to 50 minutes runtime. If your ebuild needs more than 50 minutes to run for a particular architecture, you'll have to exclude it from the .travis.yml
Profiles and Environment
Currently I only provide docker images for the base profile. If e.g. your ebuild depends on the plasma base, then you'll hit the 50 minute timeout long before your dependencies finish - desktop images are planned at a later stage
Since we run stuff in a docker container, specific operations such as device management and graphical sessions may not work.