Sakaki's EFI Install Guide/Building the Gentoo Base System Minus Kernel

From Gentoo Wiki
Jump to: navigation, search


In this section, we'll be following along with Chapter 6 of the Gentoo handbook, and installing the base system. However, we'll also be diverging considerably, as we will provide the option of using systemd (not Gentoo's more usual OpenRC) as the init system (since GNOME 3.8+ requires it to work properly[1] unless patched), and bootstrapping the core system files (apart from the kernel, which will be addressed in the next section).

The process we'll be following in this section is:

  1. Performing final preparations for a chroot (setting mirrors, copying DNS, and mounting required system directories);
  2. Entering the chroot environment (after which, the /mnt/gentoo/ location will appear to be the true root directory of the filesystem);
  3. Installing an up-to-date (and authenticated) Portage tree;
  4. Ensuring that the base profile is set;
  5. Setting your timezone and locale;
  6. Setting your (post-boot) keymap;
  7. (Optionally) informing Portage about your CPU's specific features;
  8. Installing a tool to show parallel builds in progress, from the sakaki-tools overlay;
  9. Bootstrapping the toolchain (optional);
  10. Rebuilding everything in the @world set using the new toolchain (optional);
  11. Verifying that all executables and libraries have been rebuilt (optional);
  12. Configuring your system to use authenticated Portage snapshots by default;
  13. Choosing whether to use systemd or OpenRC (some brief guidance is given to help you decide); then:
    • Either, setting the final target profile (for GNOME/systemd);
    • Or, installing the necessary patchset overlays, and then setting the appropriate target profile (for GNOME/OpenRC);
  14. Updating the @world set to reflect the new profile (whether systemd or OpenRC).

Instructions will also be provided for those who don't want to spend the time necessary to perform a full bootstrap. Let's get started!

Final Preparations for the chroot

We begin by setting up appropriate the server URIs, which portage will search when looking for source tarballs.[2] To do this, we use the mirrorselect tool (shipped as part of the minimal install system), which allows you to choose the appropriate mirror(s) from a list, and then save the result to the make.conf file (as an assignment to the GENTOO_MIRRORS variable):

livecd ~ #mirrorselect -i -o >> /mnt/gentoo/etc/portage/make.conf
<navigate through the list, and select all mirrors in your region>
Note
Use the cursor keys and Page Up and Page Down to navigate, and Space to toggle a mirror on and off; then Enter to save and exit when done. You do not need to select all available protocols for all servers in your region; generally, choosing the HTTP protocol entry only (for each desired server) will suffice.

Next, setup a /mnt/gentoo/etc/portage/repos.conf/gentoo.conf file (this specifies information about repositories in the system to Portage; see the earlier overview for a brief primer on the concepts involved). Issue:

livecd ~ #mkdir -p -v /mnt/gentoo/etc/portage/repos.conf
livecd ~ #cp -v /mnt/gentoo/usr/share/portage/config/repos.conf /mnt/gentoo/etc/portage/repos.conf/gentoo.conf
livecd ~ #nano -w /mnt/gentoo/etc/portage/repos.conf/gentoo.conf

and edit the file (removing any sync-uri line, if present, and changing auto-sync value to no) so it reads:

FILE /mnt/gentoo/etc/portage/repos.conf/gentoo.confSetting up repository information for Portage
[DEFAULT]
main-repo = gentoo

[gentoo]
location = /usr/portage
sync-type = rsync
auto-sync = no

Save and exit nano.

The above simply says that:

  • The main repository is set to be gentoo, for all other repositories (such as overlays) that do not specify masters;
  • The repository location is set to be /usr/portage (within the chroot, that is);
  • The repository will not be synced during emerge --sync and emaint sync --auto runs;
  • The repository is set to synchronize using the rsync protocol (this is unauthenticated, but don't worry, in this tutorial, we won't actually call for any syncs to be performed in this manner, and have specified changing the auto-sync value to no in the above, to prevent it happening inadvertently).

Next, we must specify the sync-uri variable, which tells Portage where to look for the rsync server, when bringing your Portage tree of ebuilds up to date.

Note
This replaces the setting of the SYNC variable in make.conf, which is now deprecated (although still present in some versions of the handbook).

Issue:

livecd ~ #mirrorselect -i -r -o | sed 's/^SYNC=/sync-uri = /;s/"//g' >> /mnt/gentoo/etc/portage/repos.conf/gentoo.conf
<navigate through the list, and select the most appropriate mirror from your region>
Note
Unlike the previous invocation, you can only select a single mirror here. Unless you have strong reasons not to, choose the 'Any available mirror' variant in a location close to you.

Check that something appropriate was written to /mnt/gentoo/etc/portage/make.conf and /mnt/gentoo/etc/portage/repos.conf:

livecd ~ #tail /mnt/gentoo/etc/portage/make.conf
livecd ~ #tail /mnt/gentoo/etc/portage/repos.conf/gentoo.conf

Next up, we need to make sure our DNS information, which exists on our target machine's current host system at /etc/resolv.conf, is still available after we chroot; otherwise networking will stop working. Issue:

livecd ~ #cp -v -L /etc/resolv.conf /mnt/gentoo/etc/

The -L option ensures we don't copy a symbolic link by mistake, since the host filesystem will become inaccessible once we chroot.

Next, if you are performing this install over a WiFi network, copy the /etc/wpa.conf configuration file for wpa_supplicant (which we setup earlier) into the chroot filesystem. Issue:

livecd ~ #cp -v /etc/wpa.conf /mnt/gentoo/etc/
Note
If you are installing over wired Ethernet, you should skip this particular command. It isn't relevant to your ability to use WiFi in the final system, only during the install itself.

Then, we need to ensure that the kernel information in /proc, and the special files in /sys and /dev are still available from inside the chroot. We therefore mount /proc on /mnt/gentoo/proc, and then bind-mount /sys and /dev:[3]

livecd ~ #mount -v -t proc none /mnt/gentoo/proc
livecd ~ #mount -v --rbind /sys /mnt/gentoo/sys
livecd ~ #mount -v --rbind /dev /mnt/gentoo/dev

Finally, we ensure that the chroot's /sys and /dev are (recursive) slave mounts (meaning that mounts and unmounts in the 'outer' system's corresponding subtrees will propagate to them, but not vice-versa); issue:

livecd ~ #mount -v --make-rslave /mnt/gentoo/sys
livecd ~ #mount -v --make-rslave /mnt/gentoo/dev

Entering the chroot

We're now ready to change root (chroot), thereby entering our new system. After this is done, although we will still be running the minimal install kernel pro tem, commands issued from within the chroot-ed shell will only be able to 'see' and use files (including, but not limited to, executables, libraries, and configuration files) that lie within the stage 3 file structure we just unpacked (except for the special /proc, /sys and /dev directories, of course).

Per the handbook, we'll perform the chroot in three steps:

  1. Enter the chroot, using bash as the shell;
  2. Reload some settings, using source; and
  3. Redefine the console prompt (to remind us we are in a chroot).

Issue:

livecd ~ #chroot /mnt/gentoo /bin/bash

then:

livecd / #source /etc/profile
livecd / #export PS1="(chroot) $PS1"
Important
Remember that if you subsequently use the target machine directly at its keyboard (rather than through the ssh/screen combination as here), you'll be working outside of the chroot, and all your paths will be incorrect (e.g. the new system will still appear at /mnt/gentoo/). It's an easy mistake to make, hence the renaming of the prompts. Once the kernel is built and the machine rebooted, we'll be 'natively' inside the new system, at which point this path issue will disappear.

Installing an Up-to-Date Portage Tree

The next step is to install a Portage (repository tree) snapshot, a set of files (updated on a daily basis) informing Portage what software is available to install, what profiles are available, and so on.

For security, Portage snapshots are digitally signed by Gentoo Release Engineering. However, because (at the time of writing) gpg is not one of the pre-installed stage 3 binaries, after downloading the latest snapshot, we will have to verify its signature outside of the chroot (where gpg is available), and then (once verified) install it (inside the chroot again).

Warning
The default Handbook flow suggests (and, to be fair, earlier versions of this guide also suggested) that you first perform a 'vanilla' (no-signature-verification) emerge-webrsync, followed optionally by an emerge --sync, to download and install the Portage repository tree, at this stage in the installation process.
However, doing so will update the repo using unauthenticated protocols (viz., an unchecked tarball fetched (usually) via HTTP, and rsync, respectively). Doing so makes it possible for a 'man-in-the-middle' attacker to tamper with the repo without your knowing, and to cause arbitrary code to be executed with root privileges on your machine thereby (see e.g. bug #597804).
As such, in this guide we will stick only to using authenticated snapshots.
If you still wish to use 'vanilla' emerge-webrsync or rsync, do so at your own risk.

Begin by fetching a signed snapshot. To do so, issue:

(chroot) livecd / #PORTAGE_GPG_DIR="/tmp" FEATURES="webrsync-gpg" emerge-webrsync --keep
... additional output suppressed ...
Trying to retrieve YYYYMMDD snapshot from <a_local_mirror> ...
Fetching file portage-YYYYMMDD.tar.xz.md5sum ...
Fetching file portage-YYYYMMDD.tar.xz.gpgsig ...
Fetching file portage-YYYYMMDD.tar.xz ...
Checking digest ...
Checking signature ...
emerge-webrsync: error: cannot check signature: gpg binary not found
Note
You may see some warnings printed about non-existent directories when you run this command. This is normal at this stage of the installation process, and may be ignored.
Note
The error cannot check signature: gpg binary not found is expected, as the gpg binary is not (at the time of writing) part of the minimal stage 3 image. In what follows, we will check the downloaded snapshot's digital signature outside of the chroot, and if it is OK, switch back inside the chroot and install the snapshot there.

This command locates and downloads the latest available Portage snapshot, digest and digital signature from one of your local mirrors (which we set up earlier). Take a note of the last reported snapshot date in emerge-webrsync's output (i.e. the one closest to the Checking digest... line), as it will try 'counting back' from the current date, until it finds an available tarball set. This date is shown as 'YYYYMMDD' in the above — but of course will actually be a real date, such as e.g., '20170911'.

Note
Because we specified the --keep option to emerge-webrsync, the downloaded files will be stored in the /usr/portage/distfiles directory inside the chroot, which is /mnt/gentoo/usr/portage/distfiles outside. If you don't specify this option, the files will be be stored to a temporary directory instead, and deleted as soon as emerge-webrsync exits, making external verification impossible.
Of course, once gpg is installed inside the chroot (which we will do shortly), emerge-webrsync will be able to check signatures itself — provided webrsync-gpg is present in FEATURES — so it is only necessary to specify --keep at this point in the install; it isn't a general requirement for safe use of the tool.
Note
The value of PORTAGE_GPG_DIR is unimportant here (since gpg is currently not installed within the chroot) but we have to specify something, hence the use of /tmp in the above.

Now, hit Ctrla followed by c to start a new virtual console in screen. This will open outside the chroot, which is what we want.

In the new virtual console, fetch the relevant public keys, as specified on the Release Engineering page. You need at least the ID of the key described as "Gentoo Portage Snapshot Signing Key (Automated Signing Key)":

livecd ~ #gpg --keyserver pool.sks-keyservers.net --recv-key 96D8BF6D
Note
Although correct at the time of writing, if it has since been updated, you may need to substitute the relevant key ID (taken from the Release Engineering page) for 96D8BF6D in the above. If in doubt, you may safely import all (unexpired) keys listed on that page.
Note
As before, for this to work you must ensure you have enabled outbound access on your firewall (if you have one installed) for port 11371/tcp to allow HKP communication, along with the usual state-tracking input rule.
Alternatively, you can use the following command, to fetch the key over port 80 (which should be open on most firewalls):
livecd ~ #gpg --keyserver hkp://pool.sks-keyservers.net:80 --recv-key 96D8BF6D
Note
If the above keyserver is unavailable for some reason, you should be able to use any other one, such as pgp.mit.edu for example.

You should next verify that the key's full fingerprint matches that listed on the Release Engineering page:

livecd ~ #gpg --fingerprint 96D8BF6D
pub   rsa4096 2011-11-25 [C] [expires: 2018-07-01]
      DCD0 5B71 EAB9 4199 527F  44AC DB6B 8C1F 96D8 BF6D
uid           [ unknown] Gentoo Portage Snapshot Signing Key (Automated Signing Key)
sub   rsa4096 2011-11-25 [S] [expires: 2018-07-01]
Note
Although correct at the time of writing, the key ID you need to enter in the above command may differ from 96D8BF6D, as may the fingerprint data shown. Always use the Release Engineering page data as your primary reference.

If that looks good, proceed to verify the snapshot. Issue:

livecd ~ #gpg --verify /mnt/gentoo/usr/portage/distfiles/portage-YYYYMMDD.tar.xz{.gpgsig,}
Note
Substitute the snapshot date you just noted, for YYYYMMDD in the above command.
Note
As before, you can ignore gpg output such as:
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.

This is normal since you have just imported the public key, and not yet 'trusted' it.[4]

Assuming that worked (the output reports 'Good signature'), close out this second virtual console using Ctrld, thereby returning to your original virtual console (inside the chroot again).

Note
There is no need for a "two-step" verification here: for Portage snapshots, the signature covers the actual tarball-of-interest itself (rather than covering a manifest file).

We can now proceed to install the verified snapshot. Issue:

(chroot) livecd / #emerge-webrsync --keep --revert=YYYYMMDD
... additional output suppressed ...
Trying to retrieve YYYYMMDD snapshot from <a_local_mirror> ...
Checking digest ...
Getting snapshot timestamp ...
Syncing local tree ...
... additional output suppressed ...
Note
Substitute the same Portage snapshot date you noted earlier, for YYYYMMDD in the above command. Also note that we don't specify PORTAGE_GPG_DIR or FEATURES this time, since we don't want emerge-webrsync attempting to verify the snapshot (using a non-existent gpg) and failing.
Note
Although — as shown in the output above — this prints a message stating that it is trying to retrieve the specified snapshot from a mirror, because we specified the --keep and --revert options, it will actually check the /usr/portage/distfiles cache first. The lack of any following 'Fetching file' lines indicates that the cached files (for the specified datestamp) are actually the ones being used (which is what we want).
Note
emerge-webrsync may complain about non-existent directories or attributes; however, now that bug #484950 has been fixed, it will still complete successfully. There is no need to manually create /usr/portage, /usr/portage/profiles or set /usr/portage/profiles/repo_name before running emerge-webrsync.

The install may take some time to complete. Once done, the emerge-webrsync command may recommend some software be updated, and tell you there is news to read: if so, you can safely ignore these for now (we're about to do a ground-up rebuild anyway, and we'll get to the news items shortly).

You can now remove the cached snapshot files, if you like, to save space (this step is optional). To do so, issue:

(chroot) livecd / #rm -v -f /usr/portage/distfiles/portage-YYYYMMDD*
Note
Substitute the same Portage snapshot date you noted earlier, for YYYYMMDD in the above command.
Important
We will configure your system to use only authenticated Portage snapshots by default later in this chapter — we defer this until after bootstrapping has been completed, to avoid having to rebuild gpg and its dependencies unnecessarily. Do not issue any further ('vanilla') emerge-webrsync commands until default authentication has been set up, or you may end up installing an unauthenticated copy of the Portage tree.
For avoidance of doubt, regular package installs via emerge are safe to do (as the authenticated Portage tree we just downloaded contains, inter alia, precomputed hashes for all package source tarballs), and also, if you follow the instructions in this chapter in the order given, you will be fine.

Now we have a valid Portage ebuild repository, we can use emerge to install (or update, delete etc.) packages. Our first task is to ensure that Portage itself is up-to-date. Issue:

(chroot) livecd / #emerge --ask --verbose --oneshot portage
... additional output suppressed ...
Would you like to merge these packages? [Yes/No] <press y, then press Enter>
... additional output suppressed ...

The emerge command, without special options, asks Portage to pull down (if necessary), build, and install the latest version of the package specified, in this case the Portage system itself. The --oneshot option informs Portage not to add itself to the set of 'explicitly-user-requested' packages in the 'world' file (/var/lib/portage/world) (Portage is already part of the @system set, via the virtual/package-manager package, so it would be redundant to add it here). The --ask --verbose flag set makes emerge ask you before making any changes to your system, and produce verbose output (generally useful, and may be abbreviated to -av).

Now, if that emerge worked OK, (you eventually see "Jobs: 1 of 1 complete" in the output after you pressed y and Enter), congratulations: your Portage system is functional, and you should be able to proceed.

Tip
As a shortcut, you can generally just press Enter, rather than y then Enter, when confirming that you want to proceed with an emerge command.

If you were prompted about news items being available, please take the time to read them now: these are important, short announcements from developers to users, pushed out via the Portage tree. To view the current news items issue:

(chroot) livecd / #eselect news list

You'll notice the items are numbered. To read a news item N, enter:

(chroot) livecd / #eselect news read N
Note
Replace N in the above with the number of the news item you wish to read, such as 1, 2 etc.

You can purge (optional) all read news items with:

(chroot) livecd / #eselect news purge

If you like, you can read more about the Gentoo news system on its manual page (using man news.eselect).

Ensuring that the Base Profile is Set

As was mentioned earlier, Gentoo uses profiles to set important variables (such as USE, CFLAGS, etc.) relevant to your particular architecture and usage scenario (e.g., a machine running the GNOME desktop). Profiles also constrain the versions of packages that can be installed. This is all maintained on your behalf by the Gentoo developers.

At this point, we just need to check that we are running the baseline amd64 profile (we should be). We'll change this to the 'real' profile post-bootstrap. Issue:

(chroot) livecd / #eselect profile list
  [1]   default/linux/amd64/13.0 *
  [2]   default/linux/amd64/13.0/selinux
... additional output suppressed ...
  [5]   default/linux/amd64/13.0/desktop/gnome/systemd
... additional output suppressed ...
  [19]  default/linux/uclibc/amd64
  [20]  hardened/linux/uclibc/amd64
Note
The profile list shown on your machine may differ slightly from the above.

And double-check that the first entry is marked with an asterisk, indicating that it is active. If not, issue:

(chroot) livecd / #eselect profile set 1

to correct this.

You can see the various (cumulative) variables assignments (to USE etc.) associated with a profile by looking at its make.defaults file, and those of its parents. Specifically, the eselect profile set N command creates a symbolic link in /etc/portage/make.profile. If you follow this link, and then look at the make.defaults files in that directory (and in any of the directories listed by the parent text file therein (transitively)), you will see how it works. Note that USE flags accumulate, but a flag with a minus sign in front of it gets removed from the list.

Important
Don't be tempted to change anything inside the /etc/portage/make.profile directory; it'll be overwritten on the next Portage update.

You can also see the net effect of the profile, your make.conf, environment etc. by issuing (this is a useful sanity check):

(chroot) livecd / #emerge --info

Lastly, if you are unsure of the meaning of any use flag, you can look it up as follows:

(chroot) livecd / #grep -i useflag /usr/portage/profiles/use.desc
Note
Replace useflag in the above command with the flag you want to query, for example multilib.

Setting Up Timezone and Locale

We don't have a timezone or locale set yet. So let's fix that now.

Note
If you elect to use systemd as your target init system (the choice comes later on this chapter), then note that some of these configuration options will need to be redone, in an systemd context, later in the tutorial. Nevertheless, we need to specify these basic elements now ahead of the bootstrap (as this will be performed in an OpenRC context).

First up is timezone. You can find the available timezone in /usr/share/zoneinfo (generally, you'll have to check the subdirectory too):

(chroot) livecd / #ls /usr/share/zoneinfo

Choose an appropriate location, and place its relative path into /etc/timezone. For example, to set London, in the UK, you would use:

(chroot) livecd / #echo "Europe/London" > /etc/timezone
Note
Please substitute the appropriate timezone for your own location in the above command.
Warning
Per the handbook's advice, please avoid using the /usr/share/zoneinfo/Etc/GMT* timezones, as their names do not indicate the expected offsets.

Now, we need to reconfigure the sys-libs/timezone-data package; this will pick up the value from /etc/timezone and reflect the correct value in /etc/localtime. The latter is necessary for the system C library.

(chroot) livecd / #emerge -v --config sys-libs/timezone-data

Secondly, we must set an appropriate locale. You must specify the locales you want to use in /etc/locale.gen. Edit this file (using your favourite editor, such as nano), and uncomment those that you want. If necessary, add additional locales to the end of the list. For example, as there are no UK locales on the list (UK = United Kingdom, approximately the same ^-^ as Great Britain = GB) use we would need to add those as follows:

(chroot) livecd / #nano -w /etc/locale.gen

Then append (for this example):

FILE /etc/locale.genAppend the following locales to the file (example only, adjust as needed)
en_GB ISO-8859-1
en_GB.UTF-8 UTF-8

Leave the rest of the file as-is. Save and exit the nano editor.

Warning
Per the handbook, it is strongly recommended that you include at least one UTF-8 locale in the list, since some applications may require it.
Note
The above is just an example. If your desired locales are already in the list (e.g., for systems in the USA, en_US ISO-8859-1 and en_US.UTF-8 UTF-8), simply uncomment them.
Note
More information about locale strings and the like can be found in the Localization Guide, please refer to that if you have any questions.

Next, we must run locale-gen to create the locales, based on the information in the /etc/locale.gen file:

(chroot) livecd / #locale-gen

Assuming that completes successfully, you then must then specify the default locale to use. Find the system list with:

(chroot) livecd / #eselect locale list
  [1]   C
  [2]   en_GB
  [3]   en_GB.iso88591
  [4]   en_GB.utf8
  [5]   POSIX
  [ ]   (free form)
Note
Your output will probably differ from the above, depending on what entries you uncommented or added to /etc/locale.gen.

Choose the appropriate value from the list. At this stage, you should use the 'C' (default) locale (other setups can cause issues with the indirect ssh/screen remote connection); we'll switch to your actual locale later on. Issue:

(chroot) livecd / #eselect locale set N
Note
Replace N in the above command with the appropriate number from the eselect locale list output. For example, to choose the C locale (as recommended), you would use eselect locale set 1 here.

Now, reload your environment:

(chroot) livecd / #env-update && source /etc/profile && export PS1="(chroot) $PS1"

Setting Up (Post-Boot) Keymap

We also need to setup a keymap, both for use post-boot, so that you will be able to type commands correctly when typing directly at your machine's keyboard.

Note
The virtual console keymap here does not replace that which we will subsequently set up for early-boot use in the initramfs (which is necessary to allow correct entry of the LUKS password); see this later comment.

We begin by displaying a list of keymaps, and filtering out those of interest. The Panasonic CF-AX3 has a Japanese layout, but obviously your machine may differ. Issue:

(chroot) livecd / #ls /usr/share/keymaps/i386/qwerty
Note
If you use a non-qwerty layout, search inside the relevant directory instead; for example, /usr/share/keymaps/i386/azerty.

Review the list, and choose a keymap file relevant to your location. In my case for example, there is one appropriate result, jp106.map.gz. Strip off the .map.gz suffix to get the appropriate name: in my case jp106 (yours will obviously vary, depending on your actual keyboard). Now we can setup the (virtual console) keymap. Issue:

(chroot) livecd / #nano -w /etc/conf.d/keymaps

and edit the keymap=... line, to cite the string you just found. For example, in my case:

FILE /etc/conf.d/keymapsSet the keymap as required
keymap="jp106"

Leave the rest of the file as-is (unless you know there are other changes you need to make here), and save and exit nano.

Note
Substitute for jp106 in the above with a value appropriate for your own keyboard! For example, a standard US layout would use keymap="us" here; a standard UK layout, keymap="uk".

Informing Portage of Processor-Specific Features (Optional)

In the last chapter, we left the CPU_FLAGS_X86 as the default ("mmx sse sse2") in /etc/portage/make.conf (you will recall that this variable is used to inform Portage about any processor-specific features (such as MMX, for example) available on your machine). Now, if you like, we will fill it out, using the app-portage/cpuinfo2cpuflags tool to help us.

Note
This is strictly optional: setting CPU_FLAGS_X86 can result in more efficient code in many cases, but, if you wish to build code for redistribution to others in binary form, you should skip this step, and leave it as-is (since, unless those users also have a machine with exactly the same processor feature set as you, your distributed binaries may crash on execution).

If you do want to set CPU_FLAGS_X86, first emerge the /proc/cpuinfo query tool (we'll use --oneshot here, to avoid adding it to your @world set):

(chroot) livecd / #emerge --verbose --oneshot app-portage/cpuid2cpuflags

Now, run the tool and note its output (yours may well differ from the below, which is taken from the Panasonic CF-AX3):

(chroot) livecd / #cpuinfo2cpuflags-x86
CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"

Issue:

(chroot) livecd / #nano -w /etc/portage/make.conf

then copy the flags output by cpuinfo2cpuflags-x86 into the existing CPU_FLAGS_X86="mmx sse sse2" definition (replacing mmx sse sse2); for our example:

FILE /etc/portage/make.confModifying the CPU_FLAGS_X86 for more efficient compiled code
CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"

(Your values will most likely differ.) Leave the rest of the file as-is. Save and exit nano.

Preparing to Run Parallel emerges

We have (intentionally) set our Portage system up to maximize parallelism when emerging. However, a side-effect of this is that, because multi-job support is active, emerge does not show any console build output, just a summary progress display.

This is easily fixed, using a simple script, which will show us the latest lines from both the Portage background tarball download log (/var/log/emerge-fetch.log) and from the most recently updated build log (Portage builds its projects in /var/tmp/portage by default). I have provided this and some other necessary utilities for this tutorial in a GitHub ebuild repository (aka overlay), which makes them straightforward to install and keep up to date within Portage (for more details about ebuild repositories, see the background reading section earlier).

Note
As of version 2.2.16 of sys-apps/portage, a new plug-in sync system is available, which simplifies the use of ebuild repositories, and obviates the need for tools like layman (which was recommended in previous versions of this tutorial).

To begin with, we'll first install the git source code management software, as we'll need this to be able to synchronize the ebuild repository from GitHub. Issue:

(chroot) livecd / #mkdir -p -v /etc/portage/package.use
(chroot) livecd / #touch /etc/portage/package.use/zzz_via_autounmask
(chroot) livecd / #echo -e "# required by asciidoc (pulled in by git)\napp-text/texlive-core xetex" >> /etc/portage/package.use/texlive-core
(chroot) livecd / #emerge --ask --verbose dev-vcs/git
... additional output suppressed ...
Would you like to merge these packages? [Yes/No] <press y, then Enter>
... additional output suppressed ...
Note
This may take some time to complete, as it'll bring in other packages transitively required by dev-vcs/git itself.
Note
The xetex package-specific USE flag adds support for XeTeX, a TeX system with Unicode support. It is required to ensure the successful emerge of app-text/asciidoc, which itself is an emerge dependency of (some versions of) dev-vcs/git.
Note
We create the /etc/portage/package.use/zzz_via_autounmask file as a convenient receptacle for changes that you may, in future, request be automatically made to your package.use configuration to allow an emerge operation to complete successfully. Per the emerge manpage (see particularly the --autounmask and --autounmask-write options), such changes are written to the "lexicographically last file" when /etc/portage/package.use is a directory (as here).

Next, we need to tell Portage about our ebuild repository, by adding an entry into the /etc/portage/repos.conf directory. Issue:

(chroot) livecd / #nano -w /etc/portage/repos.conf/sakaki-tools.conf

and add the following content to that file:

FILE /etc/portage/repos.conf/sakaki-tools.confAdd the following text to specify the sakaki-tools ebuild repository
[sakaki-tools]

# Various utility ebuilds for Gentoo on EFI
# Maintainer: sakaki (sakaki@deciban.com)

location = /usr/local/portage/sakaki-tools
sync-type = git
sync-uri = https://github.com/sakaki-/sakaki-tools.git
priority = 50
auto-sync = yes

Save, and exit nano.

The above simply specifies that:

  • The repository is called sakaki-tools;
  • It should be cloned to the /usr/local/portage/sakaki-tools directory;
  • It is a git archive, which may be synchronized via the URI https://github.com/sakaki-/sakaki-tools.git;
  • It has 'priority' 50 (ebuilds with higher priority override those of lower priority, in the event of a name clash; the default gentoo tree has priority -1000); and
  • It will be automatically synced during during emerge --sync or emaint sync --auto runs.

Next, we'll pull in the ebuild repository itself. Issue:

(chroot) livecd / #emaint sync --repo sakaki-tools
Note
After adding an ebuild repository (overlay), you may find (as in this case) you are prompted that new news items (supplied by that repository) are available. It's generally a good idea to take a look at these before proceeding; for brief instructions, please see these earlier notes.
Incidentally, for avoidance of doubt, there is no need to take action on any of the plymouth news items (2017-08-06 → 2017-08-13 inclusive) at this point, as the underlying issue has been addressed in the main Gentoo tree; nor is there any need to take action on the efivarfs item dated 2017-09-17 (that is already taken care of in this tutorial).

Whenever using a third-party ebuild repository such as this one, remember that it will (in general, due to the 'priority' setting) take precedence over your 'real' Portage tree (which it 'overlays', hence the historical name), in case of conflicting ebuilds. For this reason, it is prudent in such cases to:

  1. mask (blacklist) all packages from the ebuild repository, then
  2. unmask (whitelist) only those packages from the repository you explicitly know you want, then
  3. read the contents of those remaining, unmasked repository ebuilds before emerging (installing) them.
Note
For more background details about ebuilds and other Portage internals, see this earlier primer.

We'll begin by wildcard-masking everything from sakaki-tools. As described earlier, we'll use a file in the /etc/portage/package.mask directory for this. Issue:

(chroot) livecd / #mkdir -p -v /etc/portage/package.mask
(chroot) livecd / #echo '*/*::sakaki-tools' >> /etc/portage/package.mask/sakaki-tools-repo
Note
This qualified atom specifies all versions of all packages in all categories, that are supplied from the sakaki-tools ebuild repository. For more details, refer to the background reading in the previous chapter.

Now we can unmask what we explicitly need (using, as described earlier, files in the /etc/portage/package.unmask directory). Issue:

(chroot) livecd / #mkdir -p -v /etc/portage/package.unmask
(chroot) livecd / #touch /etc/portage/package.unmask/zzz_via_autounmask
(chroot) livecd / #echo "app-portage/showem::sakaki-tools" >> /etc/portage/package.unmask/showem
(chroot) livecd / #echo "sys-kernel/buildkernel::sakaki-tools" >> /etc/portage/package.unmask/buildkernel
(chroot) livecd / #echo "app-portage/genup::sakaki-tools" >> /etc/portage/package.unmask/genup
(chroot) livecd / #echo "app-crypt/staticgpg::sakaki-tools" >> /etc/portage/package.unmask/staticgpg
Note
When /etc/portage/<...> elements are used as subdirectories in this way, the contents are simply processed in alphabetical order - there is no 'magic' to the underlying file names used.
Note
Also, as above, we create a zzz_via_autounmask file as a convenient receptacle for changes that you may, in future, request be automatically made to your package.unmask configuration to allow an emerge operation to complete successfully.

Here is a brief summary of what the above (whitelisted) ebuilds are, and what they do:

Package ebuild Location Description
app-portage/showem /usr/local/portage/sakaki-tools/app-portage/showem/showem-1.0.3.ebuild Provides a simple utility script (showem), which enables you to monitor the progress of a parallel emerge. A manpage is provided. The underlying source is available on GitHub.
sys-kernel/buildkernel /usr/local/portage/sakaki-tools/sys-kernel/buildkernel/buildkernel-1.0.28.ebuild Provides a script (buildkernel) to build a kernel suitable for booting from a USB key in UEFI mode, together with an integral initramfs. Automatically sets the necessary kernel configuration parameters, including the command line, and signs the resulting kernel if possible. Has a interactive and non-interactive (batch) mode. (Will be used later in the tutorial.) On installation, attempts to automatically configure the /etc/buildkernel.conf settings file (to set the two variables EFIPARTUUID and CRYPTPARTUUID, which must be assigned before buildkernel is run). Manpages for the script and the configuration file are provided. The underlying source is available on GitHub.
app-portage/genup /usr/local/portage/sakaki-tools/app-portage/genup/genup-1.0.15.ebuild Provides the genup script, to simplify the process of keeping your Gentoo system up-to-date. genup can automatically update the Portage tree (using gpg authentication, where enabled), all installed packages, and kernel. Has interactive and non-interactive (batch) modes. (Will be used later in the tutorial (systemd, OpenRC).) A manpage is provided. The underlying source is available on GitHub.
app-crypt/staticgpg /usr/local/portage/sakaki-tools/app-crypt/staticgpg/staticgpg-1.4.16.ebuild The buildkernel tool (above) needs to be able to be able to place a copy of the gpg tool into the initramfs, so that the keyfile (which we created earlier) can be decrypted.

Unfortunately, the version of gpg that is emerged by Portage by default is the 2.x variant. This requires a (rather convoluted) service known as pinentry to ask you for your password (even when compiled statically), and currently genkernel's initramfs builder (and init script) does not work correctly with it. Instead, genkernel expects to be using a version 1.x gpg which can query for passphrases itself, without invoking an outside agent.

However, as the current app-crypt/gnupg is not SLOTted, we unfortunately can't install both a 1.x and 2.x version of gpg simultaneously into different SLOTs (as we can for say, dev-lang/python). The nuclear option is to roll our currently installed version of gpg back to 1.x of course (by placing the appropriate atom into /etc/portage/package.mask and re-emerging), but as gpg v2.x and pinentry are tightly integrated with GNOME, that's not a great idea.

To work around this, I have provided the app-crypt/staticgpg ebuild in sakaki-tools. This is derived from the version 1.4.16 ebuild of app-crypt/gnupg (which does not rely on pinentry). However, this ebuild will always compile statically, has none of the standard 1.4.16 gpg's USE-flag driven options, and only installs one program, staticgpg (plus a placeholder manpage) when emerged. It can safely be installed beside a standard 2.x version of app-crypt/gnupg.

Note
If for some reason you don't wish to use these packages directly, then by all means review and adapt their underlying source independently. However, you won't be able to follow along directly with the text in that case: in what follows, I'll be assuming that you have chosen to install the ebuild repository.

Next, note that all user repository ebuilds (by stipulation) must specify that they are on the 'unstable' branch (since not yet fully tested). However, we are allowing (by default) only stable (amd64) packages (see above), we have to specify that for the above packages, the unstable/testing ('tilde') branch is acceptable. We use a file in the /etc/portage/package.accept_keywords directory for this, as described earlier. Issue:

(chroot) livecd / #mkdir -p -v /etc/portage/package.accept_keywords
(chroot) livecd / #touch /etc/portage/package.accept_keywords/zzz_via_autounmask
(chroot) livecd / #echo "*/*::sakaki-tools ~amd64" >> /etc/portage/package.accept_keywords/sakaki-tools-repo
(chroot) livecd / #echo -e "# all versions of efitools currently marked as ~ in Gentoo tree\napp-crypt/efitools ~amd64" >> /etc/portage/package.accept_keywords/efitools
Note
For explanation of the qualified atoms used here, refer to the previous chapter.
Note
The only other keyworded package here is app-crypt/efitools, which will be required to manipulate the secure boot keystores on the target machine later in the tutorial. We're not going to be installing (emerging) efitools yet, but we add the entry here so we don't have to come back to it further on.
Note
Again, as above, we create a zzz_via_autounmask file as a convenient receptacle for changes that you may, in future, request be automatically made to your package.accept_keywords configuration to allow an emerge operation to complete successfully.

Next, if you like, take a look through the ebuild files in the repository (and any patches), read the underlying sources, and satisfy yourself that everything is benign. It is good hygiene to do this, particularly prior to using a third-party ebuild repository for the first time.

Now, with that setup done, we can now install the showem tool using the standard emerge command. Issue:

(chroot) livecd / #emerge --ask --verbose app-portage/showem
... additional output suppressed ...
Would you like to merge these packages? [Yes/No] <press y, then Enter>
... additional output suppressed ...

And that's showem installed!

We're about to do quite a lot of building, so it'll be useful to set up a second virtual console, from which we can view the progress of emerge using our new script. Assuming you are running screen (as discussed earlier), press Ctrla then c to start a new console. Then in that new console (which is back outside the chroot, to begin with) enter:

livecd ~ #chroot /mnt/gentoo /bin/bash

followed by

livecd / #source /etc/profile
livecd / #export PS1="(chroot:2) $PS1"

Now hit Ctrla then p to get back to the original console, and continue.

Bootstrapping the Base System (Optional but Recommended)

We are about to perform the bootstrap proper! In the below, we will follow the Gentoo FAQ's advice on how to recreate a 'stage 1' bootstrap in the modern age ^-^. (However, if you are content to rely on the shipped binaries, click here to skip this step.)

Note that here, bootstrapping refers to the process of:

  1. building (from source) the standard toolchain (GCC compiler, linker, assembler, C library and a number of other items), i.e., the components necessary to build the other software and libraries that make up Gentoo's @world package set; and then
  2. using that newly constructed toolchain to rebuild everything in the @world package set, from source.

And the point of this whole exercise? Simply put, it is to ensure the integrity of the end binaries, as far as we can. To reiterate: we will first build the core tools from source, using (by necessity) the existing toolchain binaries that are shipped as part of the minimal 'stage 3' filesystem (which we're currently chroot-ed inside of). We then use this newly built toolchain to rebuild (from source) all the shipped software in the stage 3 tarball (plus that which we've explicitly emerged, such as dev-vcs/git). Once this has been done, we verify that all libraries and binaries (apart from the kernel, which we haven't built yet) have indeed been regenerated, and that no old binaries or libraries are still lying around.

Note
The GCC compiler suite itself contains a 'bootstrap', which is triggered during this process, but it is of a slightly different nature.[5] In the GCC bootstrap, a pre-existing (shipped system binary) compiler (S) (which in this case actually is also GCC, but need not be) is used, to build a simplified GCC compiler from source (A), which is then used to build the full GCC compiler (for C or C++) (B), and then the full compiler is built again (C) using the compiler just produced (i.e., S builds A, then A builds B, then B builds C). The binaries for the last two compilers (B and C) are then compared to ensure they are identical (modulo build timestamps and the like). Confusingly, this process is also called a '3 stage' bootstrap!
Note
There's only so much comfort you can get with recompilation, of course. Per Ken Thompson's 1983 Turing Award lecture, "Reflections on Trusting Trust", the use of any 'original' binary can potentially expose all 'downstream' code to a backdoor attack.[6] For example, a compromised C compiler could 'infect' all subsequent compiled code - so even if the system were bootstrapped (as here), the malware would remain. Remember too, that you are still running under a kernel you didn't compile (that shipped with the minimal install image), although we'll get to building a new kernel from source shortly! Finally, even an open-source system relies on the probity of a lot of opaque firmware and microcode, not to mention the hardware itself (particularly, the IME). Nevertheless, although such attack vectors exist, they are relatively complex to exploit compared to the simplicity of producing and distributing a tainted high-level system binary (signatures notwithstanding). As such, there is still value to doing a ground-up rebuild, in my view. Should you not wish to do so, but would rather rely on the shipped stage 3 binaries, you can simply skip down to the section "Choosing between systemd or OpenRC Init". (Note though, that with Gentoo, you'll probably end up rebuilding most everything over time anyway!).

In Gentoo parlance, people speak of three 'stages' of bootstrapping (and their corresponding file system tarballs):

  1. Stage 1: When starting from a stage 1 tarball, the base toolchain (GCC, standard C libary etc.) must be built using the existing (binary) host system toolchain, under direction of the /usr/portage/scripts/bootstrap.sh script. This yields a:
  2. Stage 2 system. Here, we still need to emerge (build) the core @world package set, using our new toolchain. This yields a:
  3. Stage 3 system, in which the toolchain has been bootstrapped, and the important system binaries and libraries have been compiled using it. A tarball of such a stage 3 system's directories is now provided as a default part of the Gentoo distribution (stage 1 and stage 2 tarballs are not available to end-users anymore).

Although we have already downloaded a stage 3 tarball, we're going to pretend we haven't, and instead build up from stage 1.

Gentoo Bootstrap Remix: Progressing from Stage 1 to Stage 2

Right, let's get going. First, we'll need to build ourselves a toolchain! We'll switch to the correct directory, and then do a dummy run to see what the supplied bootstrap.sh script wants to do:

(chroot) livecd / #cd /usr/portage/scripts

then:

(chroot) livecd /usr/portage/scripts #./bootstrap.sh --pretend
... additional output suppressed ...
-------------------------------------------------------------------------------
  [[ (0/3) Locating packages ]]
 * Using baselayout : >=sys-apps/baselayout-2
 * Using portage    : portage
 * Using os-headers : >=sys-kernel/linux-headers-4.4
 * Using binutils   : sys-devel/binutils
 * Using gcc        : sys-devel/gcc
 * Using gettext    : gettext
 * Using libc       : virtual/libc
 * Using texinfo    : sys-apps/texinfo
 * Using zlib       : zlib
 * Using ncurses    : ncurses
-------------------------------------------------------------------------------
... additional output suppressed ...
!!! CONFIG_PROTECT is empty
... additional output suppressed ...

The exact versions (and packages) shown in your output may differ.

Note
You can ignore any additional information/warnings about slot conflicts, packages forcing rebuilds etc. at this stage. The bootstrap is an extremely 'low level' build and will almost always go through successfully.
Note
The most direct way to make sense of the reported set of packages (and build order) output by the ./bootstrap.sh --pretend command, is to look at a minimal, ground-up Linux installation like Linux from Scratch - this has an excellent handbook, which runs though the process you'd be going through by hand, if Portage didn't exist![7]

The Gentoo FAQ suggests you may wish to edit the /usr/portage/scripts/bootstrap.sh script after reviewing it - and indeed, we will do so, because there are two 'gotchas' lurking in the above proposed emerge list. The first problem is that the C standard library that the bootstrap intends to rebuild is a virtual (virtual/libc); however, in Portage, emerging a virtual package does not, by default, cause any already-installed package that satisfies that virtual (in our case, sys-libs/glibc) to be rebuilt - which we want.

We'll edit bootstrap.sh so that it builds the underlying C library instead. Issue:

(chroot) livecd /usr/portage/scripts #nano -w bootstrap.sh

Once nano opens, issue Ctrlw to start a search, and then type should never fail and press Enter. Navigate down to the line setting myLIBC and modify it so the block now reads:

FILE /usr/portage/scripts/bootstrap.shEnsuring glibc gets rebuilt during bootstrap
# This stuff should never fail but will if not enough is installed.
[[ -z ${myBASELAYOUT} ]] && myBASELAYOUT=">=$(portageq best_version / sys-apps/baselayout)"
[[ -z ${myPORTAGE}    ]] && myPORTAGE="portage"
[[ -z ${myBINUTILS}   ]] && myBINUTILS="binutils"   
[[ -z ${myGCC}        ]] && myGCC="gcc"
[[ -z ${myGETTEXT}    ]] && myGETTEXT="gettext"
[[ -z ${myLIBC}       ]] ; myLIBC="$(portageq expand_virtual / virtual/libc)"
[[ -z ${myTEXINFO}    ]] && myTEXINFO="sys-apps/texinfo"
[[ -z ${myZLIB}       ]] && myZLIB="zlib"
[[ -z ${myNCURSES}    ]] && myNCURSES="ncurses"

You only need modify the one line, replacing "&&" with ";" as shown above (this makes the following statement execute unconditionally). Leave the rest of the file as-is. When done, save and exit nano.

Note
As this file exists as part of the main Gentoo ebuild repository, your changes will be overwritten next time you sync. As we only want to bootstrap our system now, that's not a problem (but you can of course make a copy of the modified bootstrap.sh file at this point, should you wish).

Ensure that will do what we intended:

(chroot) livecd /usr/portage/scripts #./bootstrap.sh --pretend
... additional output suppressed ...
-------------------------------------------------------------------------------
  [[ (0/3) Locating packages ]]
 * Using baselayout : >=sys-apps/baselayout-2
 * Using portage    : portage
 * Using os-headers : >=sys-kernel/linux-headers-4.4
 * Using binutils   : sys-devel/binutils
 * Using gcc        : sys-devel/gcc
 * Using gettext    : gettext
 * Using libc       : sys-libs/glibc:2.2
 * Using texinfo    : sys-apps/texinfo
 * Using zlib       : zlib
 * Using ncurses    : ncurses
-------------------------------------------------------------------------------
... additional output suppressed ...
!!! CONFIG_PROTECT is empty
... additional output suppressed ...
Note
As before, you can ignore any additional information/warnings about slot conflicts, packages forcing rebuilds etc. at this stage. The bootstrap is an extremely 'low level' build and will almost always go through successfully.

(The exact versions in your output may differ.)

Well, we have addressed the first problem (the output is now showing sys-libs/glibc:2.2 will be built for libc, not the virtual) - but a second remains. Specifically, the line !!! CONFIG_PROTECT is empty is telling you that the bootstrap process will not preserve any configuration files you may have modified, if any of the packages to be installed try to overwrite them. We have modified (as opposed to created from scratch) two such configuration files so far (here and here), so lets use the (supplied) qfile utility to check if either is affected (this shows us which package 'owns' a given file):

(chroot) livecd /usr/portage/scripts #qfile /etc/locale.gen /etc/conf.d/keymaps
sys-apps/openrc (/etc/conf.d/keymaps)
sys-libs/glibc (/etc/locale.gen)

As the bootstrap is proposing to emerge the second of these packages, we will need to stash a temporary copy of the configuration file /etc/locale.gen, so it can be restored afterwards.

Note
In normal use, and as discussed later, Portage does not usually blindly overwrite changed configuration files in this manner, but rather warns you of the conflict, and allows you to merge any changes yourself. This issue only occurs during stage 1 bootstrapping.

Issue:

(chroot) livecd /usr/portage/scripts #cp -v /etc/locale.gen{,.bak}
Note
Although we did modify (rather than create from scratch) /etc/portage/make.conf earlier, this particular file is not owned by any package, as you can verify yourself using qfile, if you like.

OK, now we are good to go, so let's start the bootstrap for real:

(chroot) livecd /usr/portage/scripts #./bootstrap.sh

This will take a while! You can now switch to the second screen console we prepared earlier, and see what is going on (as the files download, build etc.). Hit Ctrla then n to switch to the second console (you can do this while the bootstrap is running), and issue:

(chroot:2) livecd / #showem

You can now switch back and forward between the two windows as you like (using Ctrla then n to cycle forwards, and Ctrla then p to cycle backwards - identical in function here where we only have two windows active), which should give you a good overview of the bootstrap process as it progresses.

Running a Stage 1 Bootstrap in One Console...
...And Watching the Logs with showem in Another
Note
The output you see will most likely be slightly different from the screenshots above, as new versions of software get released etc.

Assuming the bootstrap.sh script completes successfully, you next need to check your GCC configuration (since we just rebuilt the compiler, and possibly upgraded it too). Switch back to the first console (hit Ctrla then 0; screen indexes consoles from zero), and issue:

(chroot) livecd /usr/portage/scripts #gcc-config --list-profiles

If (and only if) the output tells you your current config is invalid, per these notes, issue the following to fix it:

(chroot) livecd /usr/portage/scripts #gcc-config 1
(chroot) livecd /usr/portage/scripts #env-update && source /etc/profile && export PS1="(chroot) $PS1"
(chroot) livecd /usr/portage/scripts #emerge --ask --verbose --oneshot sys-devel/libtool
... additional output suppressed ...
Would you like to merge these packages? [Yes/No] <press y, then Enter>
... additional output suppressed ...

(That's a one as an argument to gcc-config by the way, not an 'ell'!)

Warning
It is very important that you ensure your GCC profile is correct. Failure to do so can break subsequent builds in a mildly spectacular fashion.
Note
If you did find it necessary to update gcc-config, you may then also, at this point, wish to switch to your second virtual console with Ctrla then n, stop any running showem with Ctrlc, then issue:
(chroot:2) livecd / #source /etc/profile && export PS1="(chroot:2) $PS1"
finally restarting showem if desired, then switching back to the first virtual console again with Ctrla then p.
However, provided you are only going to run emerge operations from the first virtual console, using the second to run showem, sourcing /etc/profile in the second virtual console may safely be omitted, as I have done in the main body of the text.

Next, we'll run the bootstrap.sh script again, to ensure that everything in the toolchain (including early dependencies, such as sys-devel/gettext and sys-libs/zlib) have been rebuilt using the new compiler, and are then themselves used in a rebuild of that compiler:

(chroot) livecd /usr/portage/scripts #./bootstrap.sh
 * System has been bootstrapped already!
 * If you re-bootstrap the system, you must complete the entire bootstrap process
 * otherwise you will have a broken system.
 * Press enter to continue or CTRL+C to abort ..
<press Enter>
... additional output suppressed ...

Now, just as above, you can switch back and forth between the two screen consoles, to review the progress.

Once the toolchain bootstrap has completed (for the second time), we're nearly ready to rebuild all the other binaries using it. Firstly however, although you shouldn't need to reset the GCC profile this time around, to be on the safe side, double-check it is valid:

(chroot) livecd /usr/portage/scripts #gcc-config --list-profiles

With that done, restore the configuration file you backed-up above, and regenerate the locale information (as this will also have been purged by the bootstrap). Issue:

(chroot) livecd /usr/portage/scripts #mv -v /etc/locale.gen{.bak,}
(chroot) livecd /usr/portage/scripts #locale-gen

Double-check that your current locale is still set to 'C' (as was specified earlier):

(chroot) livecd /usr/portage/scripts #eselect locale show
LANG variable in profile:
  C

And finally, revert back to the root directory:

(chroot) livecd /usr/portage/scripts #cd /

Gentoo Bootstrap Remix: Progressing from Stage 2 to Stage 3

We now need to build everything in the @world package set, using our shiny new toolchain.

Before we start, we need to write a null 'timestamp' file, which we'll use as a marker when checking later that all our executables and libraries have indeed been rebuilt. Issue:

(chroot) livecd / #touch /tmp/prebuild_checkpoint

Right, with that done, we are finally ready to build everything in the @world package set, including all the libraries upon which they depend, as follows:

(chroot) livecd / #emerge --ask --verbose --emptytree --with-bdeps=y @world
... additional output suppressed ...
Would you like to merge these packages? [Yes/No] <press y, then Enter>
... additional output suppressed ...
Note
The above process may complain about some kernel flags not being set correctly. That's because they aren't - yet! Ignore this for now, as we will build a kernel with the correct flags shortly.
Note
For avoidance of doubt, the @world set includes the @system set.

Here's what those emerge parameters mean (see also the emerge manpage):

Parameter Short Form Meaning
--ask -a Before performing the operation, show what would take place, and then ask whether to confirm or abort. It's usually a good idea to leave this set.
--verbose -v Provide more information about the emerge operation (e.g. USE flags that will be applied for each package, in the information provide by --ask).
--emptytree -e Build and install all the targets (in this case, everything in the world set), and their entire deep dependency tree, as though nothing were currently installed.
--with-bdeps=y N/A Include build-time dependencies that are not strictly required when calculating the deep dependency graph.
@world N/A Specifies what should be emerged, in this case the world set of packages (the '@' prefix indicates that a set of packages is being referred to, rather than a single package). This includes the transitive closure of everything you asked be installed (listed in /var/lib/portage/world) together with the @system set.
Tip
The emerge will take quite some time; as before (when we undertook the stage 1 -> stage 2 bootstrap), you can use Ctrla then n to switch to screen's second virtual console, and review the ongoing build using showem; switch back to the original console with Ctrla then p. If you like (since the build will take several hours), you can even disconnect screen pro tem. To do this, hit Ctrla then d. You are then back in your original ssh session, and you can log out of that too if you like, using Ctrld. The target machine will continue doing its thing in the background. Then, at some later point, you can ssh back in again, and execute
livecd ~ #screen -D -R
to reconnect to screen - your two virtual consoles will still be there. Quite neat! Of course, if you like, there is no need to do any of this, you can simply stay connected all the way through the build.
Note
By default, Gentoo will also write a high-level summary log file to /var/log/emerge.log.
Note
This command will not just rebuild existing binaries; it'll upgrade them too (if a newer version ebuild is available, since the stage 3 image was created).
Note
As this is a regular emerge, we don't need to backup /etc/locale.gen as we did earlier. Portage will handle any file conflicts for us, as we will discuss next.

After some time, the build will hopefully complete successfully (if it does not, please see the troubleshooting section, below).

Tip
If the bootstrap emerge appears to be hung, with no change in the number of jobs completed for many hours (be careful about concluding this though, check your genup output also, in the second screen console window), you can try first killing it, by pressing Ctrlc (in the first screen window, the one showing the "Jobs:..." output), and then issuing:
(chroot) livecd / #emerge --resume
to restart it where it left off.
For avoidance of doubt, it is generally not necessary to restart the @world emerge in this manner.

Assuming you modified your /etc/locale.gen file and/or /etc/conf.d/keymaps file earlier (see here and here), then Portage will probably prompt you with:

... additional output suppressed ...
 * IMPORTANT: 2 config files in '/etc' need updating.
 * See the CONFIGURATION FILES section of the emerge
 * man page to learn how to update config files.
... additional output suppressed ...

in the output from the full emerge run.

Note
Depending upon your version of Portage, you may only be warned about changes to one file — /etc/locale.gen — and not /etc/conf.d/keymaps. This is nothing to worry about: if it happens to you, just cat the file /etc/conf.d/keymaps, to reassure yourself your changes (if any) have been preserved.

This is easily dealt with, using the dispatch-conf tool (and it's good hygiene to run this anyway, after a large build session). This utility exploits the fact that Portage does not install packages directly to the filesystem, but rather uses a sandbox approach. As a result, it will not directly overwrite files in directories protected by the CONFIG_PROTECT variable (includes most files under /etc/ in a standard setup, you can use emerge --info to see the full details). Instead, it writes the new configs to .cfg0000_<name> files. dispatch-conf lets you step through each of these conflicting files in turn, shows you a diff between your current version and what the install wanted to write, and asks you what you want to do.

Enter:

(chroot) livecd / #dispatch-conf

Then follow the prompts to review each proposed change. Assuming only the /etc/locale.gen and/or /etc/conf.d/keymaps need modifying (should be the case at this stage), then you simply need to press z for each file when prompted. This means (rather non-obviously) 'zap-new' - or in other words keep the version of the file that was there before the emerge was initiated. Once this has been done, dispatch-conf should exit automatically.

Note
If no configuration files are affected, then dispatch-conf will simply exit immediately. However, there's no harm in running it.
Note
Once you have got the full @world emerge to work successfully, you can easily redo it again (should you wish, or your paranoia dictate) by issuing
(chroot) livecd / #emerge --depclean && emerge --ask --verbose --emptytree --with-bdeps=y @world
once more.

If everything worked fine, then you can skip ahead to the next section; otherwise, here are a few tips on how to rectify any problems that may have occurred.

Troubleshooting a Failed Build

Generally, emerges, even of the full @world set, go without a hitch. However, if you elected earlier to use the '~amd64' (aka 'testing') phase ebuilds, issues will sometimes pop up. For example, here's a problem that happened during the writing of this guide, when attempting the stage 2 -> stage 3 emerge (I was using the testing branch then, since GNOME 3 had not been stabilized at the time - nevertheless, it's an instructive example):

media-libs/mesa Build Error during emerge

In this case, a build failure occurred while emerging the media-libs/mesa-9.2.5 package. Here are some steps you can take if this kind of thing happens to you (this is not an in-depth guide - for that see this section from the Gentoo handbook):

  • Sometimes, intermittent issues occur due to the very parallel build architecture we have specified. These issues will often resolve themselves if the build is recommenced. Fortunately, there's a flag for that. Issue
    (chroot) livecd / #emerge --resume
    and emerge will try to pick up where it left off, with all the same settings. Alternatively, you can try to resume the emerge with parallel make temporarily switched off; to do so, issue
    (chroot) livecd / #MAKEOPTS="-j1" EMERGE_DEFAULT_OPTS="--jobs=1" emerge --resume
    If that works OK, great, you're done. If not, proceed to the next step.
  • If you are building on a system with little RAM (e.g. an old computer or a virtual machine), you might be running out of memory. On a VM, consider shutting it down, giving it more RAM, starting it again, and chrooting back into your installation if you still don't have a bootable system; you will be able to resume your build with emerge --resume. Alternatively, you might be running too many concurrent build jobs and / or compilations; in this case, go to /root/.bashrc and crank down the --load-average=Y and --jobs=Z parameters on your EMERGE_DEFAULT_OPTS and crank down the -j and -l options on MAKEOPTS. If your build still fails, try creating a paging file on your hard drive and enabling it to have at least some extra virtual memory as follows:
    (chroot) livecd / #dd if=/dev/zero of=/pagefile bs=1M count=1024
    (chroot) livecd / #mkswap /pagefile
    (chroot) livecd / #swapon /pagefile
  • Since you are (at the moment) running on a fairly 'vanilla' system, it's likely that others will have seen the build problem you're experiencing, and have already reported it. Gentoo has a pretty quick turnaround for fixes, which are posted to the Portage (ebuild) tree when signed off. Therefore, the next-lowest-overhead way to fix a build problem is simply to a day or so (if it's a major issue, you shouldn't have to wait long), then resync Portage (using the instructions given above, to ensure authentication), and try the emerge again (NB, but not using --resume here, since we've sync'd in between):
    (chroot) livecd / #emerge --ask --verbose --emptytree --with-bdeps=y @world
    Hopefully, that fixed your problem. If not (or you don't want to wait), keep going...
  • Look at the full build log. As the build has failed, this will still be in /var/tmp/portage somewhere - indeed, in this example, the actual file is at /var/tmp/portage/media-libs/mesa-9.2.5/temp/build.log. Have a read. Having done so, try searching for any appropriate-sounding errors using Google (or, if you want to use an anyonymized wrapper around the Google search engine, Startpage). For example, here the text
    Makefile:603: recipe for target 'all-recursive' failed
    is quite specific and suggestive. A Google search for "Gentoo ~amd64 emerge media-libs/mesa-9.2.5 Makefile:603: recipe for target 'all-recursive' failed" yields a relevant hit: a post on the Gentoo discussion forums: "mesa-9.2.5 fails, egl_gallium/llvm related". A quick perusal shows us we have found the correct problem, and that it is related to bug #488216, a problem with the package sys-devel/llvm. In short, it appears that there is a problem with the default-use-flag build of sys-devel/llvm-3.4, it's just that it shows itself in the build of media-libs/mesa-9.2.5. You can now try to:
  1. Work around the problem, per the information you have found. For example, in this case, you could try appending ~sys-devel/llvm-3.4 -ncurses to the /etc/portage/package.use file (see this earlier discussion of atoms and /etc/portage/package.use if you're unclear as to what this means) and issuing the full @world emerge again. Just don't forget that you have that you have made this change - and do try to come back and see if you can revert the fix later! (were a new ebuild for LLVM to be released, for example).
  2. Try using an earlier version of the affected package (this is, in fact, the approach I adopted at the time, although it was rendered moot by the stabilization of GNOME 3). You can do this using the /etc/portage/package.mask file (discussed earlier), which tells Portage the versions of software not to allow. Here, we could try preventing Portage from using the 3.4 version of sys-devel/llvm, by adding =sys-devel/llvm-3.4 to /etc/portage/package.mask (forcing Portage to reject it), then re-emerging. In this case, from the online package information grid for LLVM, we can see this will drop us back to version 3.3-r3 (remember, I was on the testing branch). This approach generally works, but be warned - you may end up rolling back quite a lot of other packages when you mask in this way. Furthermore, if you adopt this approach, always be sure only to mask the specific version that has the problem (use '=' at the start of the /etc/portage/package.mask line, per our earlier discussion of atoms), as this means when a new (hopefully, fixed) ebuild is released (with a higher version number), Portage will automatically attempt to try to use that instead.
Note
Whatever you do, do not downgrade glibc. Forcing a downgrade will literally break your system the very second you put the older version in service. The official ebuild has actually been specially crafted to never let you downgrade glibc and exit with error status if a downgrade attempt is detected. If you can't build glibc, fix your system instead. For further info, read Downgrade Glibc from the Gentoo Wiki archive.
  • If you have switched your Portage profile and some toolchain-related packages like glibc refuse to build, try building gcc first, as some profiles like hardened require rebuilding the toolchain with different USE variables. Depending on your attention to detail, you can just issue a simple emerge -DNau --with-bdeps=y gcc and then emerge -DNau --with-bdeps=y @world, or you can outright boostrap your system again. Refer to Hardened FAQ: How do I switch to the hardened profile for further info.


Finally, here are some miscellaneous hints that you may find useful:

  1. Check that your gcc configuration is set correctly, especially if you have recently upgraded your GCC installation - if it is invalid, builds will often fail in hard-to-debug ways. To do so, issue:
    (chroot) livecd / #gcc-config -l
    (That's an 'ell', not a 'one'!) If it returns 'Active gcc profile is invalid' (or 'No gcc profile is active!'), then specify an appropriate one from the list presented; for example, to select the first configuration in the list, issue:
    (chroot) livecd / #gcc-config 1
    (That's a 'one' this time!) If you do change your gcc profile in this way, be sure to issue:
    (chroot) livecd / #env-update && source /etc/profile && export PS1="(chroot) $PS1"
    (chroot) livecd / #emerge --verbose --oneshot sys-devel/libtool
    to update sys-devel/libtool's library locations, and to modify the settings in your current shell, before retrying any emerge.
  2. Check that your perl libraries are up-to-date. This will not always be handled properly by Portage, but you can use perl-cleaner to bring things back in line. Issue:
    (chroot) livecd / #perl-cleaner --reallyall
    If your system detects stale Perl libraries and they all fail to build, try rebuilding Perl itself:
    (chroot) livecd / #emerge -a dev-lang/perl
  3. Ensure that ruby is set to the latest version, if it is present in your system (if the following eselect ruby list command returns an error, you do not). Certain things like net-libs/webkit-gtk require an up-to-date version. Issue:
    (chroot) livecd / #eselect ruby list
    to see a list of available versions, and then:
    (chroot) livecd / #eselect ruby set 3
    to choose one.
    Note
    Replace the 3 in the above command with the index of the most recent ruby version, as reported in the output of eselect ruby list, above.

Then re-attempt the failed emerge.

Verifying the Bootstrap (Optional but Recommended)

Congratulations - by this point you have performed a 'stage 1' bootstrap, and so all the (binary) applications and libraries within the current chroot should have been recompiled. Just to make sure that nothing has sneaked through (i.e., an app or library somehow included in the stage 3 tarfile we downloaded earlier, but not present in the deep transitive closure of the @world package set), we'll check the datestamps of all relevant components against the /tmp/prebuild_checkpoint we wrote before commencing the emerge. We aren't worried about any text executables (such as scripts) since, in principle, these can be audited to ensure they are not malicious.

First, remove any installed packages that are not part of the dependency tree, so that the following test is not confused. Issue:

(chroot) livecd / #emerge --depclean

Let this process run through to completion (it may take a little while). Then issue:

(chroot) livecd / #find / -type d -path /boot/efi -prune -o -type f -executable -not -newer /tmp/prebuild_checkpoint -print0 2>/dev/null | xargs -0 file --no-pad --separator="@@@" | grep -iv '@@@.* text'

This command finds all executable files (except beneath the EFI mountpoint at /boot/efi; which will host a FAT filesystem - in which all files appear executable - when mounted (it currently shouldn't be mounted though, assuming you've been following the tutorial flow, but the prune is there for robustness)), which aren't newer than the checkpoint file, then tests their file type using file, and prints them if they are not recognized as some form of text file (script, source code etc.). It should produce no output (as all files of this type should have been recreated, if our bootstrap was complete).

So much for executable binaries - what about libraries (shared or static)?[8] On Gentoo, we'll have already checked most shared ('.so') libraries, since they have their execute bits set. However, this is not true for all flavours of Linux, and furthermore, static ('.a') libraries (and '.o' object files) do not have the execute bits set.[9] Therefore, to be double-sure, we'll run another test (this may take some time to complete):

(chroot) livecd / #find / -type d -path /boot/efi -prune -o -type f -not -executable -not -newer /tmp/prebuild_checkpoint -print0 2>/dev/null | xargs -0 file --no-pad --separator="@@@" | grep '@@@.*\( ELF\| ar archive\)'

This looks for any files which aren't newer than our checkpoint file, and which are recognized by file as ELF or ar archives; it may take some time to complete (as there are a lot of files to check). Again, it should produce no output. If so, and the previous check also worked, then congratulations, you have fully rebuilt all executable binaries and libraries on your machine!

Configuring Default Use of Authenticated Portage Snapshots

Now that the bootstrapping process is out of the way, we can (shadowing these instructions from the official Handbook) set up the target system to use authenticated snapshots by default. As a convenience, we will make use of the app-crypt/gentoo-keys package, which will maintain a gpg keyring of sanctioned Release Engineering keys automatically for us. To install it, together with the app-crypt/gnupg package (which supplies gpg), issue:

(chroot) livecd / #emerge --ask --verbose --noreplace app-crypt/gentoo-keys app-crypt/gnupg
... additional output suppressed ...
Would you like to merge these packages? [Yes/No] <press y, then press Enter>
... additional output suppressed ...
Note
We specify the --noreplace option here, as when you installed dev-vcs/git earlier, this should have already emerged app-crypt/gnupg as a dependency (enabled via the default gpg USE flag). If so, --noreplace will not rebuild it, but simply ensure you have it as a 'top-level' package in your @world set (the file /var/lib/portage/world). If however the package is not installed for some reason, emerge --noreplace will build it as normal. It's a useful idiom to be aware of.

This will install its keys into the /var/lib/gentoo/gkeys/keyrings/gentoo/release directory.

Note
Pace the official instructions, it is not necessary to 'trust' the keys in the /var/lib/gentoo/gkeys/keyrings/gentoo/release keyring, in order to be able to use them for snapshot verification. As such we will not do so.

Next, edit /etc/portage/make.conf, and enable support for validating the signed Portage tree snapshots. Issue:

(chroot) livecd / #nano -w /etc/portage/make.conf

and append:

FILE /etc/portage/make.confAppend lines to use gpg signature checking with webrsync
FEATURES="${FEATURES} webrsync-gpg"
PORTAGE_GPG_DIR="/var/lib/gentoo/gkeys/keyrings/gentoo/release"

Leave the rest of the file as-is. Save and exit the nano editor.

Note
The official instructions also suggest setting sync-type = webrsync (and a blank sync-uri) in /etc/portage/repos.conf.gentoo.conf, but at the time of writing, this does not yet seem to work fully in all situations. As such, I recommend you leave the standard rsync option in place in this file, but with auto-sync turned off, as was specified earlier. That means you will have to use emerge-webrsync (or eix-sync -w, once app-portage/eix is installed, later in the tutorial) to synchronize the main gentoo repo.

Choosing between systemd or OpenRC Init

At this point, you need to choose which init system you want to use on your target machine: systemd or OpenRC.

systemd is the default choice, since a 'vanilla' (unpatched) GNOME 3.8+ system requires it to run properly.[1] It is currently the most widely adopted[10] init system amongst Linux distributions.

However, please be aware that there is now an alternative, for those who would rather not use systemd. This involves installing a patchset provided by Dantrell B., which enables Gentoo users to enjoy the full GNOME experience (including session tracking and power management) under Gentoo's own init system (OpenRC).

Important
Don't stress out about making this choice! If:
  • you want to use Gentoo's standard repositories and profiles for GNOME; or
  • you have no strong preference; or
  • you're not really sure what all this init system stuff is about anyway ^-^,
then you should select systemd, as doing so will leave you on the 'mainstream' track, where you will find it more straightforward to receive timely security updates, obtain support via the forums etc.

If you choose to go the OpenRC route, and so use Dantrell's patchset for GNOME (full instructions for which are provided in this guide), please be aware that, as with any third-party overlay:

  • you may find it more difficult to get support via the normal Gentoo channels (the forums etc.) when using an 'unofficial' patchset such as this one;
  • security fixes etc. may take longer to be released, and the overlays may themselves contain security holes;
  • the patchset's author (Dantrell B.) may cease support for this patchset in the future.

Having said that:

  • the vast majority of the GNOME code is unaffected by the patches;
  • the patchset code itself is largely shared with Funtoo, who use it in their mainline distribution,[11] so you will not be alone;
  • Dantrell B. has stated his intention to support this patchset for Gentoo (at least until either a proper answer for the GNOME/systemd issue is arrived at, he ceases to use GNOME, or the complexity gets out of control).

Ultimately, as with all things Gentoo, the choice is yours ^-^

When you have decided, continue as below:

  • For systemd, follow the "Option 1" text below, and then read the rest of this guide following the 'default' flow. Do not follow "Option 2" or read any of the 'alternative track' chapters (they are clearly marked). Be sure also to follow any specific instructions in chapters 8 and 9 that are marked for the attention of systemd users; or
  • For OpenRC, follow the "Option 2" text below, and then read the rest of this guide, following the 'alternative track' for Chapters 10 onwards (you will see the branch point clearly marked at the end of Chapter 9). Be sure also to follow any specific instructions in chapters 8 and 9 that are marked for the attention of OpenRC users.

Option 1: Setting a systemd Profile

Note
Remember: if you want to use OpenRC as your init system, please do not carry out the instructions in this section, but read "Option 2" instead.

OK, so let's set our Portage profile to the version we actually want to use going forward (we avoided doing so earlier, to minimize the scope of the bootstrap).

Issue:

(chroot) livecd / #eselect profile list
  [1]   default/linux/amd64/13.0 *
  [2]   default/linux/amd64/13.0/selinux
... additional output suppressed ...
  [5]   default/linux/amd64/13.0/desktop/gnome/systemd
... additional output suppressed ...
  [19]  default/linux/uclibc/amd64
  [20]  hardened/linux/uclibc/amd64

The current (default) profile is shown marked with an asterisk. However, the most correct profile for our purposes is the 'desktop' variant that cites GNOME and systemd. In the above list, this is number 5 (but your output may differ, so be sure to check).

Note
The output shown is just an example and will change over time. Note also that the 'developer' profiles are for Gentoo internal development; if you just want to develop regular software on your system, you don't need to select one of these.

To switch, issue:

(chroot) livecd / #eselect profile set N
Note
Replace N in the above command with the desired profile number: 5 in the case of this example.

Now, continue reading at "Updating @world to Reflect the New Profile", below (i.e., please skip the "Option 2" text below, as it does not apply).

Option 2: Adding Dantrell's Ebuild Repositories (Overlays) and Setting an OpenRC Profile

Note
Remember: if you want to use systemd as your init system, please do not carry out the instructions in this section, but read "Option 1" instead.

To use GNOME with OpenRC, you will first need to install a few custom ebuild repositories (aka 'overlays'), to apply Dantrell B.'s patchset for GNOME (see the background reading section earlier for a brief overview of ebuild repositories).

At the time of writing there are seven repos to install: a 'common' one, and one for each of GNOME 3.14, 3.16, 3.18, 3.20, 3.22 and 3.24. We'll create configuration files in /etc/portage/repos.conf for each of these.

Note
It is also possible to install and use layman to manage the repositories, rather than specifying them in /etc/portage/repos.conf. For details of this alternative approach, please see the Gentoo Wiki page GNOME Without systemd; however, in this tutorial, I'm going to assume you want to install the repos directly.

Begin with the 'common' ebuild repository; issue:

(chroot) livecd / #nano -w /etc/portage/repos.conf/dantrell-gnome.conf

and enter the following text:

FILE /etc/portage/repos.conf/dantrell-gnome.confConfiguration for dantrell-gnome repo
[dantrell-gnome]

# Dantrell B.'s Gentoo ebuild repository ('overlay') for GNOME (generic)
# Maintainer: Dantrell B. (email: see CONTRIBUTING.md in main GitHub project)
# Homepage: https://github.com/dantrell/gentoo-project-gnome-without-systemd

location = /usr/local/portage/dantrell-gnome
sync-type = git
sync-uri = https://github.com/dantrell/gentoo-overlay-dantrell-gnome.git
priority = 150
auto-sync = yes

Save, and exit nano.

The above simply specifies that:

  • The (common GNOME) repository is called dantrell-gnome;
  • It should be cloned to the /usr/local/portage/dantrell-gnome directory;
  • It is a git archive, which may be synchronized via the URI https://github.com/dantrell/gentoo-overlay-dantrell-gnome.git;
  • It has 'priority' 150 (ebuilds with higher priority override those of lower priority, in the event of a name clash; the default gentoo tree has priority -1000); and
  • It will be automatically synced during during emerge --sync or emaint sync --auto runs.


Next, the 3.14 ebuild repository. Issue:

(chroot) livecd / #nano -w /etc/portage/repos.conf/dantrell-gnome-3-14.conf

and enter the following text:

FILE /etc/portage/repos.conf/dantrell-gnome-3-14.confConfiguration for dantrell-gnome-3-14 repo
[dantrell-gnome-3-14]
 
# Dantrell B.'s Gentoo ebuild repository ('overlay') for GNOME (3.14)
# Maintainer: Dantrell B. (email: see CONTRIBUTING.md in main GitHub project)
# Homepage: https://github.com/dantrell/gentoo-project-gnome-without-systemd
 
location = /usr/local/portage/dantrell-gnome-3-14
sync-type = git
sync-uri = https://github.com/dantrell/gentoo-overlay-dantrell-gnome-3-14.git
priority = 100
auto-sync = yes

Save, and exit nano. (The contents should be self-explanatory).

We can use this file as a basis for the other (3.16 → 3.24) variants, rather than entering them manually. To do so, issue:

(chroot) livecd / #for ((R=16;R<=24;R+=2)); do cp -v /etc/portage/repos.conf/dantrell-gnome-3-{14,${R}}.conf; sed -e "s/14/${R}/g" -i /etc/portage/repos.conf/dantrell-gnome-3-${R}.conf; done

Note that it is fine to have multiple versions installed, as the profile — which we will select shortly — will decide which takes precedence.

Note
If you are sure you only want to use one version of GNOME, for example 3.22, then it suffices to create the /etc/portage/repos.conf/... file specific to it (in this example, that would be /etc/portage/repos.conf/dantrell-gnome-3-22.conf, in addition of course to the common repository (/etc/portage/repos.conf/dantrell-gnome.conf). However, to allow later flexibility, I recommend setting up all six variants (3.14, 3.16, 3.18, 3.20, 3.22 and 3.24) now, per the main body of the text (the profile, which we will set shortly, will determine which version takes precedence on your system).

Now, pull in the ebuild repositories themselves (in this case, via HTTPS to GitHub):

(chroot) livecd / #emaint sync --auto
Note
You may recall the advice given earlier to apply a global mask / selective unmask strategy to third-party ebuild repositories, as a hygiene measure. Due to the size of the Dantrell repos, we won't do that here, but feel free to cast an eye over the provided ebuilds if you like, before proceeding further.
Note
Assuming you have set up /etc/portage/repos.conf/gentoo.conf as specified earlier, this command will not cause the main Gentoo repo to be sync'd (as it has been marked with auto-sync = no).

The Dantrell ebuild repositories (aka 'overlays') we just installed provide (inter alia) a bundled set of Portage profiles; these allow you to conveniently set USE flags, masks etc. and thereby ensure that subsequent installation of GNOME proceeds smoothly.

Note
For each version of GNOME (3.14, 3.16, 3.18, 3.20, 3.22 and 3.24) a 'standard' (no-suffix) and 'extended' profile are provided. The 'extended' profiles attempt to match many of the standard USE flags that a desktop install of GNOME in Gentoo would have (whereas the 'standard' profiles are more minimal in scope). As such, I recommend that you use an 'extended' profile in what follows.
Incidentally, the use of the dantrell-gnome-reparent profiles (mentioned in earlier versions of this guide) is no longer recommended. Please choose a corresponding Dantrell 'extended' profile instead (as now recommended in the main body of this text), as these perform essentially the same function.

Accordingly, we next set our profile to the version we actually want to use going forward (we avoided doing so earlier, to minimize the scope of the bootstrap).

Issue:

(chroot) livecd / #eselect profile list
  [1]   default/linux/amd64/13.0 *
  [2]   default/linux/amd64/13.0/selinux
... additional output suppressed ...
  [21]  dantrell-gnome:default/amd64/3.14
  [22]  dantrell-gnome:default/amd64/3.14/extended
  [23]  dantrell-gnome:default/amd64/3.16
  [24]  dantrell-gnome:default/amd64/3.16/extended
  [25]  dantrell-gnome:default/amd64/3.18
  [26]  dantrell-gnome:default/amd64/3.18/extended
  [27]  dantrell-gnome:default/amd64/3.20
  [28]  dantrell-gnome:default/amd64/3.20/extended
  [29]  dantrell-gnome:default/amd64/3.22
  [30]  dantrell-gnome:default/amd64/3.22/extended
  [31]  dantrell-gnome:default/amd64/3.24
  [32]  dantrell-gnome:default/amd64/3.24/extended

(Your output may vary.)

Note
Although you will see 'developer' profiles in this list, don't be concerned - these are for Gentoo internal development only, you can still develop regular software on your system under any of the profiles, including the one we will choose.

The current (default) profile is shown marked with an asterisk. However, the safest profile for our purposes is the 'extended' variant whose version matches the current 'stable' version of GNOME in mainline Gentoo. To find out what version this is (version 3.22, at the time of writing), consult the table in gnome-base/gnome, and look for the green box with a plus sign in it, under the amd64 heading (there may be more than one, in which case it is generally best to select the most modern 'plus signed' variant listed).

Note
Users who have elected to use the 'testing' branch (~amd64 rather than amd64) earlier may use any available 'dantrell-gnome' profile. You may wish to take note of the 'grading' status of the available GNOME versions on Dantrell's page here, when making your choice.
Note
If you wish to do so, GNOME will also build fine with the appropriate Dantrell 'unextended' profile selected. However, you may find your experience is impaired due to missing codecs etc., when compared to a 'standard' Gentoo GNOME desktop. In what follows, I'm going to assume you have chosen a 'extended' profile.

Having found the stable version of GNOME, select the appropriate profile; for example, to use GNOME 3.22 (matching one of the 'stable' Gentoo versions at the time of writing; adapt as appropriate), issue:

(chroot) livecd / #eselect profile set "dantrell-gnome:default/amd64/3.22/extended"
Note
Obviously, substitute for the dantrell-gnome:default/amd64/3.22/extended profile name in the above command if necessary; for example, if targeting GNOME 3.20 instead, you would use dantrell-gnome:default/amd64/3.20/extended.
Important
Please do not choose any profile other than an extended (or, should you wish, a standard) dantrell-gnome variant, if you intend to use GNOME with OpenRC: doing so will almost certainly fail later in the install, unless you are very sure of what you are doing.

Check that the new profile is now active:

(chroot) livecd / #eselect profile show
Current /etc/portage/make.profile symlink:
  dantrell-gnome:default/amd64/3.22/extended

The output you see may differ from the above, but should reflect the profile choice you just made.

Note
To remove these repositories (should you wish to do so at some future time), simply switch your profile back (for example, to default/linux/amd64/13.0), move any dantrell-gnome*.conf files out of /etc/portage/repos.conf/, and update @world. That isn't something you should do at this point in time, obviously!

Having now successfully set up the appropriate custom profile, continue reading below.

Updating @world to Reflect the New Profile

Having chosen a target init system (systemd or OpenRC), and selected the appropriate profile, we must update @world set, to pull in (and build from source) any new packages required, and to reflect any USE flag changes to existing packages (by rebuilding them as necessary).

Note
(systemd users only) It is no longer necessary to do a two-stage sys-apps/dbus/sys-apps/systemd emerge (as you will sometimes see reported); the circular dependency on sys-fs/udev-init-scripts-25 was broken by moving it to PDEPEND, as of revision 1.9 of the systemd-208-r2.ebuild.[12]

Issue:

(chroot) livecd / #emerge --ask --verbose --deep --with-bdeps=y --newuse --update @world
... additional output suppressed ...
Would you like to merge these packages? [Yes/No] <press y, then Enter>
... additional output suppressed ...

Here's what those emerge parameters mean (see also the emerge manpage):

Parameter Short Form Meaning
--ask -a Before performing the operation, show what would take place, and then ask whether to confirm or abort. It's usually a good idea to leave this set.
--verbose -v Provide more information about the emerge operation (e.g. USE flags that will be applied for each package, in the information provide by --ask).
--deep -D Consider the full transitive closure of the build dependency graph, not just the immediate dependencies.
--with-bdeps=y N/A Include build-time dependencies that are not strictly required when calculating the deep dependency graph.
--newuse -N Include packages whose use flags have changed since they were last built.
--update -u Update packages to the best version available in the Portage ebuild tree (as of the last time you sync'd), subject to masking etc.
@world N/A Specifies what should be emerged, in this case the world set of packages (the '@' prefix indicates that a set of packages is being referred to, rather than a single package). This includes the (newly expanded, due to your changed profile) @system set, plus any packages you requested be installed.
Note
Do not worry if you are warned about kernel configuration options not being set correctly, we will create an appropriate kernel in the next chapter.
Note
This emerge will take some time. While it is running, as we discussed earlier in the bootstrapping section, you can switch to the second screen virtual console and review progress with showem, if you like.
Note
If your emerge exits with an error, see the section "Troubleshooting a Failed Build" above, for some hints. In particular, make sure you always try restarting the emerge at least once (with emerge --resume) if it does not complete successfully on the first attempt.
Note
(OpenRC users only) If you are having trouble with the emerge, try switching from the 'extended' to the 'standard' dantrell-gnome profile for your chosen version of GNOME (as the 'standard' variants specify fewer USE flags than their 'extended' siblings). For example, if you are having trouble emerging under the dantrell-gnome:default/amd64/3.14/extended profile, try switching to the dantrell-gnome:default/amd64/3.14 profile, and then attempting the emerge again.
Note
Because of the profile change, the emerge command may tell you there are news items to read. Feel free to read any fresh news items now, if you like, following the earlier instructions.
Tip
If you do not want to rebuild packages whose available USE flags have been changed, but where those changes are orthogonal to your currently installed version's configuration (e.g., the addition of a non-default USE flag), then you can specify --changed-use instead of the (more conservative) --newuse in the above command.

Whether or not emerge warns you that you have configuration files that need updating as a result of the above update, it is good hygiene to run:

(chroot) livecd / #dispatch-conf

Then follow the prompts to review each proposed change, if any.

Note
Assuming no modified configuration files have been modified (in a manner that cannot be mechanically reconciled) by the update (the most likely scenario), then dispatch-conf will simply exit immediately, printing nothing. However, there's no harm in running it. The dispatch-conf tool is discussed in more detail in the bootstrapping section above.

Next Steps

Having successfully built the @world set for our new profile, we can now turn our attention to the heart of the system: the Linux kernel itself. Click here to go to the next chapter, "Configuring and Building the Kernel".

Notes

  1. 1.0 1.1 Gentoo Wiki: "GNOME/3.8-upgrade-guide", Systemd section
  2. Gentoo Development Guide: "Mirrors"
  3. SuperUser Forum: "mount dev, proc, sys in a chroot environment"
  4. Information Security Stack Exchange: "Ways to sign gpg public key so it is trusted?"
  5. GNU: "Installing GCC: Building"
  6. Thompson, Ken. "Reflections on Trusting Trust". 1983 Turing Award lecture
  7. Beekmans, Gerard. Linux from Scratch. Burgess, Matthew et. al. eds. 2017
  8. Ippolito, Greg. "Static, Shared Dynamic and Loadable Linux Libraries"
  9. Unix & Linux Stack Exchange Forum: "Why are .so files executable?"
  10. Wikipedia: "systemd: Adoption and reception"
  11. Funtoo Linux FAQ: "Do you support systemd?"
  12. Portage Attic: Diff between v1.9 and v1.8 of /sys-apps/systemd/systemd-208-r2.ebuild
< Previous Home Next >