Catalyst

From Gentoo Wiki
Jump to: navigation, search
Resources
Note
This page is about the release building tool. For the AMD graphics drivers, and the configuration tool that comes with them, see fglrx

Catalyst is a release building tool, used by Gentoo Linux. With catalyst users are able to completely customize a Gentoo install by customizing the very tools that are used to install a system. Official Gentoo Linux release media is built using Catalyst.

catalyst is capable of:

  • Building installation stages.
  • Building bootable LiveCDs.
  • Building GRP (Gentoo Reference Platform) sets. [Note: GRP has been discontinued]
  • Setting up a Tinderbox target for test building.
  • Building netboot images.

Installation

USE flags

USE flags for dev-util/catalyst Release metatool used for creating releases based on Gentoo Linux

ccache Enables ccache support local
doc Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally global

Emerge

To install catalyst, run:

root #emerge --ask dev-util/catalyst

Configuration

After emerging catalyst, the first (and probably only) configuration step is to edit /etc/catalyst/catalyst.conf

Usage

Seed tarball

For creating a stage3 a seed tarball is needed. Catalyst will chroot into the seed and emerge the packages for the new stages inside it, so it does not pollute the build system.

The seed tarball should be placed in the following temporary directory unless the temporary directory location was modified during the Catalyst configuration in the configuration section above. If the temporary directory was modified put the seed in the appropriate directory, otherwise:

root #mkdir -p /var/tmp/catalyst/builds/default

The seed tarball can be a current stage3 tarball downloaded from one of the Gentoo mirrors.

Relatively recent amd64 tarballs can be found here, and x86 tarballs can be found here.

Use wget to download from the command line. For example, to download an amd64 tarball:

root #wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64/stage3-amd64-20150402.tar.bz2 -O /var/tmp/catalyst/builds/default

Snapshot

Local generation

Along with the tarball a snapshot of the Portage tree is required. Catalyst can create a snapshot of the system's current /usr/portage directory with the catalyst -s <date> command.

The snapshot will be saved to /var/tmp/catalyst/snapshots/portage-<date>.tar.bz2

In the example below 2015.04 (April, 2015 in an easy to read command-line format) is the argument passed to the --snapshot (-s) option as the snapshot date:

root #catalyst -s 2015.04

The <date> argument can be any date format that makes sense to the user. To use the current year/month following format above, simply:

root #catalyst -s $(date +%Y.%m)

Remote download

Another option is to use one of the Portage tree snapshots available on the mirrors. Current snapshots can be found here.

As mentioned above, the wget command can be used to download a remote snapshot hosted on a HTTP(S) server:

root #wget -O portage-latest.tar.bz2 http://distfiles.gentoo.org/snapshots/portage-latest.tar.bz2

Specs

Catalyst uses specs files for reading the parameters to create a stage. For building a stage3 a stage1 and stage2 must be built, then the stage3 will be created. Therefore a spec file is needed for each of the stages.

The specs the Gentoo Release team use for autobuilds can be found in the RelEng git repository:

https://gitweb.gentoo.org/proj/releng.git/tree/releases/weekly/specs

This article will use the specs for amd64:

https://gitweb.gentoo.org/proj/releng.git/plain/releases/weekly/specs/amd64/

root #wget -O stage1.spec "https://gitweb.gentoo.org/proj/releng.git/plain/releases/weekly/specs/amd64/stage1.spec"
root #wget -O stage2.spec "https://gitweb.gentoo.org/proj/releng.git/plain/releases/weekly/specs/amd64/stage2.spec"
root #wget -O stage3.spec "https://gitweb.gentoo.org/proj/releng.git/plain/releases/weekly/specs/amd64/stage3.spec"

Download the three files files above. These specs are very simple; all three of them contain many of the same parameters.

Inside a stage1.spec file

Check out the stage1.spec file:

FILE stage1.specamd64 stage1 spec file example
subarch: amd64
target: stage1
version_stamp: 2015.04
rel_type: default
profile: default/linux/amd64/13.0
snapshot: 2015.04
source_subpath: default/stage3-amd64-latest
update_seed: yes
update_seed_command: --update --deep @world
#portage_confdir: /release/releng/releases/weekly/portage/stages

.spec file option list

The following table provides a list of .spec file options and their descriptions.

Option Description
subarch The subarch can be any of the supported Catalyst sub-architectures. Check the available sub-architectures in /usr/lib/catalyst/arch/${ARCH}.py, ${ARCH} being the arch to be built. subarch simply sets the CHOST and CFLAGS/CXXFLAGS accordingly.
target The target specifies what target Catalyst will build. The example above specifies stage1 as the target to be built. For building a CD, livecd-stage1 should be defied as the target.
version_stamp The version stamp is used as an identifier for the build. On the autobuilds the date is utilized for this setting, however it can be anything the user desires. This parameter will be used on the output tarball, (${target}-${ARCH}-${version_stamp}.tar.bz2), temporary directories, etc...
rel_type rel_type defines what kind of build the system will perform. This is merely another identifier that is available to be used in case (more) differentiation of the builds is needed. If a normal, hardened, and uclibc stages were to be built these could be defined here. Changing it will change the sub-directory inside /var/tmp/catalyst/builds from default to whatever has been set for this value.
profile This is the system profile to be used by Catalyst to build this target. It is specified as a relative path and must be set to one of the system profiles available at /usr/portage/profiles. In the example above the default profile is used (default/linux/amd64/13.0).
snapshot This value specifies which snapshot of main Gentoo repository. See the section above on creating a snapshot for more information on snapshots. If 2015.04 is used like in the example above, catalyst will look for a snapshot available at /var/tmp/catalyst/snapshots/portage-2015.04.tar.bz2.
source_subpath This is the relative path which specifies where the seed stage comes from for this target. In the example above, it will use /var/tmp/catalyst/builds/default/stage3-amd64-latest.tar.bz2 as the seed stage. The prefix of this filesystem path (/var/tmp/catalyst/builds) is determined by the value contained in the storedir variable the /etc/catalyst/catalyst.conf file. The suffix (.tar.bz2) is automatically generated by catalyst at build time.
distcc_hosts These are the hosts used as distcc slaves when distcc is enabled in the /etc/catalyst/catalyst.conf file. It follows the same syntax as distcc-config --set-hosts and is entirely optional. This setting is lacking from the example above.
portage_confdir This is an absolute path where a custom Portage configuration directory can be defined. This comes in handy when desiring to include unstable (~) packages or include a specific make.conf. If this line is left out of the .spec file, then Catalyst will default to using its default configuration. If this line is defined make sure it is defined across all targets (.spec files) to minimize problems.
portage_overlay This option specifies the location to a Portage overlay to use when building the target.
pkgcache_path This allows the optional directory containing the output packages for catalyst. Mainly used as a way for different .spec files to access the same cache directory. Default behavior is for this location to be auto-generated by catalyst based on the .spec file.

On stage2.spec and stage3.spec files, source_subpath needs to reference the stage1 and stage2 tarballs respectively. In other words, with exception of the first .spec file in the sequence, source_subpath should be set to the tarball build previously. For an example, if 2015.04 was used as the version_stamp, the source_subpath for stage2 should be: default/stage1-amd64-2015.04, and the source_subpath for stage3 should be: default/stage2-amd64-2015.04.

Invocation

root #catalyst --help
Catalyst, version 2.0.18
Copyright 2003-2008 Gentoo Foundation
Copyright 2008-2012 various authors
Distributed under the GNU General Public License version 2.1

Usage catalyst [options] [-C variable=value...] [ -s identifier]
 -a --clear-autoresume	clear autoresume flags
 -c --config     use specified configuration file
 -C --cli        catalyst commandline (MUST BE LAST OPTION)
 -d --debug      enable debugging
 -f --file       read specfile
 -F --fetchonly  fetch files only
 -h --help       print this help message
 -p --purge      clear tmp dirs,package cache and autoresume flags
 -P --purgeonly  clear tmp dirs,package cache and autoresume flags and exit
 -T --purgetmponly  clear tmp dirs and autoresume flags and exit
 -s --snapshot   generate a release snapshot
 -V --version    display version information
 -v --verbose    verbose output

Usage examples:

Using the commandline option (-C, --cli) to build a Portage snapshot:
catalyst -C target=snapshot version_stamp=my_date

Using the snapshot option (-s, --snapshot) to build a release snapshot:
catalyst -s 20071121

Using the specfile option (-f, --file) to build a stage target:
catalyst -f stage1-specfile.spec

Building

Once the specs have been reviewed, start catalyst by running the following command:

root #catalyst -f stage1.spec && catalyst -f stage2.spec && catalyst -f stage3.spec

If everything went as expected a stage3 should be show up in the /var/tmp/catalyst/builds/default/ directory.

See also