User:SwifT/Gensetup

To automate the installation of Gentoo on KVM guests for testing and development purposes, a script called "gensetup" is created which mimics the installation instructions as documented in the Gentoo Handbook

About gensetup
Allow me to be very clear on this - gensetup is not an official installer. More, it is not scheduled to be extended feature-wise to provide features that I don't need for "seeding" my virtual images. Finally, it will definitely have bugs which I will resolve when I encounter them. However, the script is made flexible enough to have different deployment scenario's even though I constantly use a single scenario, so chances are low that I hit the same bugs that you would if you would choose to use this (hackish, ugly) script.

The purpose of the script is to setup and configure a base Gentoo Linux system without much user intervention, but still according to the documentation in the Gentoo Handbook. By following the instructions rather than creating binary packages or even stage4 builds, I can validate if installations are still valid according to the documentation.

Of course, the script still needs to execute all tasks automatically, so some aspects of the installation are done "blindly" (providing the keys and feedback a user would give) without capturing and interacting with the application. An example is  usage. As such, it is not an exact validation of the installation instructions.

Another aspect is that the instructions often give the users many choices. Since I will not check for every choice a user could give, it will only run an installation for a given set of choices. However, to make sure it is flexible enough, the choices and feedback are all part of a configuration file.

Setup of Virtual Environment
Before starting the virtual images, I prepare my environment by loading the correct modules, setup a device and a virtual ethernet switch (through   ).

Loading the virtual environment prerequisites

The device will be the link between the host operating system and the guests. The  script uses a few   commands to forward packages between the guest operating systems and the host' gateway so that the guests can access Internet.

Creating and Booting Images
The first step is of course to create and boot a virtual image. My virtual guests all use a 20Gbyte disk, but once I have a base Gentoo system, that disk is frozen and all images are copy-on-write images of this base environment. This keeps the storage use sufficiently manageable and allows me to quickly setup additional guests (since the base is already done).

First, create a 20Gbyte disk:

Creating a 20Gbyte Qemu disk

Next, I boot this with a systemrescuecd ISO:

Booting the sysresccd

I can already hear you say ebbeh? here, so a quick explanation of the parameters used:

Of course, this is all scriptable. I use a few scripts that fire up the necessary guests with or without CD, with or without many CPUs / memory, etc. and all in  sessions. I might eventually switch to  later, but for now this seems to work just fine.

Downloading
I keep my sources in a GitHub repository. They are not tagged for releases of any kind, so to make sure you have the "latest" revision, it is advisable to  from the repository.

Downloading the gensetup code

After this command, you will have a directory in which you'll find the  directory. This is the directory you'll need.

Of course, you might not have  available on the live environment. What I do is manage a cloned repository on my workstation, and use  from the live environment to get the necessary files:

Using rsync to download gensetup

Configuring
The  configuration is done in a simple key/value set. An example configuration, which I use to create new images, is available as. An elaborate explanation of the configuration settings is available in the next chapter.

Running
Once the configuration file is edited, you can run  with this configuration file as a parameter. Running gensetup

The steps seen in the output is a feature in the  script that categorizes activities. These steps allow me to repeat a particular part of the installation (or continue after manually fixing an earlier failure).

To get an overview of the available steps, just run  without a configuration file.

{{Code|Getting an overview of the available steps| Usage: ./gensetup.sh  to start from a particular step, and even stop after another step. For instance, to run the configure and tools steps, but not those before or after:
 * 1) ./gensetup.sh

{{Code|Running a part of the installation| }}
 * 1) ./gensetup.sh simple.conf configure tools

BTW, with a binhost available and the kernel distributed as a binary, deployment of Gentoo Linux this way takes a few minutes.

Debugging / Troubleshooting
In case of failures, check the logfile (cfr the  configuration item) for the output of the command. In the file, this logfile is stored in.

The configuration file
All configuration entries are made in a key/value filled configuration file. As said earlier, an example is available too. In the rest of this section, we'll go through the various settings in the configuration file.

Generic Settings
Generic gensetup settings

The  parameter defines where the command output is stored. The  parameter is to tell the script where the installation should take place (mount point).

Disk Settings
Disk settings

This set defines the settings (  is the device name for a virtio-driven virtual image, so the substitute of  for non-virtual or non-virtio devices). These settings are:


 * of the partition, in megabytes, or empty to use the remainder of the disk
 * of the partition, using the hexadecimal notation. 83 is a standard Linux partition, 82 a Linux swap partition and 8e a partition to be used as a physical volume in an LVM2 setup
 * of the partition, with root being the root partition, swap the swap partition, and lvm:vg the LVM volume group called " vg ". You can also use the mount location (like ) as purpose for all locations (except for the root partition).
 * command to be used. The command will be executed with the partition name as last argument (so a  of   for  will result in   )
 * to be used (used by creation)

Logical Volume Settings
Logical volume settings

In this section, the volume group vg (cfr the lvm:vg definition earlier) is defined. In the volume group, logical volumes are defined (the part after  ) and each of these volumes gets the same definition settings again like defined earlier for the disks.

So, a  definition will create a  logical volume, and due to its   this logical volume will be mounted on.

Gentoo Profile Settings
Profile settings

In this section, the files to use for the installation (stage3 file and portage snapshot) are defined. In the listing above, you'll find an example for when you use the official Gentoo mirror system. However, I keep the files local (so I can run some tests during flights).

The last setting defines the Gentoo profile (to be used with   ).

make.conf
make.conf settings

This should be fairly obvious - all settings are defined here. You can use other variables too, everything after  is verbatim copied to the  file during installation.

Portage Directory Settings
Portage directory settings

The portage directory settings will be used to generate the right subdirectories and files. Each line creates a file in the correct subdirectory with the filename being the last identifier. So  creates  with the given content.

For the file, each entry on the line is given on a new line. For, a file will only get a single line.

System Configuration
System configuration settings

The  information is used to create the  file and is reused later when setting the timezone in the configuration file(s).

The  sets the right variable(s) in the file given as third identifier. So,  sets   in.

The other settings map on the Gentoo Handbook installation instructions. The  setting creates the  file. The  setting is to tell   how many lines you have defined - I'm too lazy to code a way to find that out automatically.

Package installation
Package installation settings

The  enumerates which packages to install.

With  you define which services to add to which runlevel.

Finally,  and   allow you to run commands before and after the installation. You can also use  to have certain settings used in the same command:

Example prepend usage

Kernel configuration
Kernel configuration settings

The kernel configuration section defines how the Linux kernel configuration is done on the installation. The  gives the package to install (unless   is given, in which case we tell Gentoo Portage not to install the package but assume that it is installed to allow for dependencies to be matched properly).

Next, we can either have the Kernel configuration built or installed as a binary. The  and   define where to fetch the configuration/binary package from.

To generate a binary package, I use the  in the  location. This results in a tarball that can be distributed and deployed on all virtual guests.