User:Jannik2099/Overlay-CI-with-Travis

DISCLAIMER: This Article is still WIP, and the software it describes is not yet feature complete (although in an error-free, workable and helpful stage)

Overlay CI with Travis

This is a personal project to provide a useful Travis CI for Gentoo overlays hosted on GitHub. The aim is to catch non-functional ebuilds, uphold QA standards, and detect regressions in a packages test suite. Right now, only the first part is implemented. You can find the project here

Terminology

What is CI

Continuous Integration is, overly simplified and from the perspective of your average GitHub project, the process of automatically testing changes to development branches, such as push and pull requests, for functionality, coverage and regressions.

What is Travis

Travis is an online CI service for GitHub repositories, free of charge for open repositories. If you've never seen a repository use Travis CI I recommend glimpsing through their documentation

How it works

Travis configuration

Travis is configured via the travis.yml file to run multiple jobs as declared by the build matrix. In this project, it runs the run.sh script for every package and architecture specified in the travis.yml

Test environment

Travis itself runs an Ubuntu VM (or container in the case of non-amd64). On this we simply run docker and use my containers to get into a gentoo environment and run the tests there.

Test script

The script looks at the pull diff, filters out any modified ebuilds, looks if those are keyworded for the specific arch runner, and then emerges them with FEATURES=test. Currently unimplemented is the ability to filter FEATURES=test on a per package basis and QA checks with pkgcheck.

How to use

Install in your overlay

To install this into your overlay, simply copy the contents of the src/ directory into your repository so that it contains .travis.yml and .travis

Configuration

Edit the .travis.yml env and arch sections to include the packages and arches you want to test. On your repos travis-ci.com page, be sure to enable "Build pushed pull requests" in settings.

Actually using it

Travis can test on pushes and pulls. Since a push to master equals pushing to the users with overlays, you should put ALL ebuild changes in pull requests to master.

Caveats and Shortcomings

Build time(out)

Travis jobs run with two CPU threads and 7.5 (4 on non-amd64) GB of RAM. This should suffice for most software, but I've seen some truly horrifying things on this planet.

The bigger issue is that travis jobs are limited to 50 minutes runtime. If your ebuild needs more than 50 minutes to run for a particular architecture, you'll have to exclude it from the .travis.yml

Profiles and Environment

Currently I only provide docker images for the base profile. If e.g. your ebuild depends on the plasma base, then you'll hit the 50 minute timeout long before your dependencies finish - desktop images are planned at a later stage

Since we run stuff in a docker container, specific operations such as device management and graphical sessions may not work.