Custom Initramfs

initramfs is a root filesystem that is embedded into the kernel and loaded at an early stage of the boot process. It is the successor of initrd. It provides early userspace which lets you do things that the kernel can't easily do by itself during the boot process.

Using initramfs is optional. By default, the kernel initializes hardware using built-in drivers, mounts the specified root partition, loads the init system of the installed Linux distribution. The init system then loads additional modules and starts services until it finally allows you to log in. This is a good default behavior and sufficient for many users. initramfs is for users with advanced requirements, for users who need to do things as early as possible, before the root partition is mounted.

Here are some examples of what you can do with initramfs:


 * Mount the root partition (for encrypted, logical, and otherwise special partitions).
 * Provide a minimalistic rescue shell (if something goes wrong).
 * Customize the boot process (e.g. print a welcome message, boot splash, etc.).
 * Load modules (e.g. a third party driver that can not be integrated into the kernel directly).
 * Anything the kernel can't do (as long as you can do it in user space, e.g. by executing commands).

If you don't have advanced requirements, you do not need initramfs.

Prerequisites
There are countless ways to make an initramfs. You can choose not to create an initramfs at all but let other apps, such as Genkernel or Dracut, do the work for you. If you are lucky, one of them does what you want out of the box, and you don't need to bother with how initramfs works and what it does anymore. If you're unlucky, they don't do what you want and you have to extend their functionality, or even build an initramfs all by yourself.

The initramfs usually contains at least one file, /init. This file is executed by the kernel as the main init process (PID 1). It has to do all the work. In addition, there can be any number of additional files and directories that are required by /init. They are usually files you will also find on any other root filesystem, such as for device nodes,  for kernel information,  for binaries, and so on. The structure of the initramfs can be simple, or it can be complicated, depending on what you are planning to do.

When the kernel mounts the initramfs, your target root partition is not yet mounted, so you can't access any of your files. That means there is nothing but the initramfs. So everything you need, everything you want, you have to include it in your initramfs. If you want a shell, you have to include it in your initramfs. If you want to mount something, you need a mount utility. If you need to load a module, your initramfs has to provide both the module, as well as a utility to load it. If the utility depends on libraries in order to work, you have to include the libraries as well. This seems complicated, and it is, because the initramfs has to function independently.

Basics
In this section you will learn the easy and straightforward way to initramfs creation. You will make a functional - albeit minimalistic - initramfs which you then can extend according to your own requirements.

Directory Structure
Create a basic initramfs directory structure that will later become your initramfs root. For consistency. we'll work in, but any location would do.

Device Nodes
Most things you do in initramfs will require a couple of device nodes to be present, especially the device for your root partition. Throughout this document, will be used as example device. Copy basic device nodes.

Which devices you need exactly depends entirely on what you are going to use initramfs for. Please adapt to your own needs.

Applications
Any binary you want to execute at boot needs to be copied into your initramfs layout. You also need to copy any libraries that your binaries require. To see what libraries any particular binary requires, use the ldd tool. An example examining what libraries requires:

Here you see that for to work in your initramfs, you not only need to copy  to your, but also  and  to your. Note that is not needed.

Additionally, some applications may be depend on other files and libraries to work. For example, also needs a terminfo file  from, so that has to be copied to your initramfs as well. To find these dependencies, tools like equery and strace prove to be helpful.

Busybox
Instead of collecting countless utilities and libraries (and never seeing the end of it), you can just use. It's a set of utilities for rescue and embedded systems, it contains a shell, utilities like ls, mkdir, cp, mount, insmod, and many more - all in a single binary called. For busybox to work properly in a initramfs, you'll firstly need to emerge it with the  USE flag enabled, then copy the  binary into your initramfs layout as :

Init
The file structure of your initramfs is almost complete. The only thing that is missing is /init, the executable in the root of the initramfs that is executed by the kernel once it is loaded. Because includes a fully functional shell, this means you can write your /init binary as a simple shell script (instead of making it a complicated application written in Assembler or C that you have to compile).

The following example realizes this executable as a minimalistic shell script, based on the busybox shell:

This example needs some device nodes to work, mainly the root block device. Change the script and copy the the corresponding node to fit your needs.

Lastly, make the /init executable:

Packaging Your Initramfs
Your initramfs now has to be made available to your kernel at boot time. This is done by packaging it as a compressed cpio archive. This archive is then either embedded directly into your kernel image, or stored as a separate file which can be loaded by grub during the boot process. Both methods perform equally well, simply choose the method that you find most convenient.

Kernel Configuration
With either method, you need to enable support for Initial RAM filesystem and RAM disk (initramfs/initrd) support for the initramfs functionality.

Also enable all drivers, filesystems, and other settings that are required for booting and accessing your root partition. If you select such drivers as modules, you will have to collect and integrate the module files into your initramfs and load them in your /init. This generally means a lot of unnecessary extra work, so just use built-in drivers for now.

Embedding into the Kernel
If you want the initramfs to be embedded into the kernel image, edit your kernel config and set Initramfs source file(s) to the root of your initramfs, (e.g ):

Now when you compile your kernel it will automatically put the files into a cpio archive and embed it into the kernel image. You will need to rebuild your kernel any time you make any changes to your initramfs.

Creating a Separate File
You can create a standalone archive file by running the following commands:

This will create a file called in your  directory. You now need to instruct grub to load this file at boot time, you do this with the initrd line.

Finalizing
You can now reboot your machine. On boot, the kernel will extract the files from your initramfs archive automatically and execute your /init script, which in turn should then take care of mounting your root partition and exec the init of your installed Linux distribution.

Functionality
Now that you've covered the initramfs basics, in this section you will learn how to extend your /init script with more advanced functionality.

Rescue Shell
If you want to be dropped to a rescue shell if an error occurs, you can add the following function to your /init and call it when something goes wrong.

In the example below, the rescue_shell will be executed if the root partition fails to mount:

Dynamic Devices
For populating dynamically, you can use either devtmpfs or mdev. Please note that the kernel can take some time detecting devices (such as external USB drives), so you may also have to add a sleep statement to your script.

devtmpfs
Provided by the kernel, devtmpfs is designed to offer device nodes early at bootup.

You can include the following snippet in your /init script to have it mount at boot:

Don't forget to unmount it again in the cleanup phase of the script:

mdev
Although devtmpfs is the preferred solution today, you can alternatively use mdev, the udev replacement of busybox.

For mdev to work, you have to make a symlink to  in your initramfs.

Then add the following snippet to your /init, after mounting and :

Mount by UUID or LABEL
With Dynamic Devices enabled, you might prefer to use UUID or LABEL to mount the root partition instead of using a static device name. For that purpose, Busybox comes with findfs.

Doing it this way is simple, but it means that your UUID or LABEL is hardcoded in /init. Alternatively, you can also use Kernel Parameters.

Kernel Parameters
If you want to use kernel parameters instead of hardcoding device names or UUIDs, you will have to parse. There are many ways to do so, the following method is just an example to give you the general idea. It uses string manipulation of the shell and only supports  parameters. Add the following function to your /init and call it whenever you need a kernel parameter.

The function is called with the name of the kernel parameter you are interested in. In the example below, it uses the root parameter to mount the root partition.

It works for both  and   but fails when the parameter is missing.

LVM
If your root partition is located on a logical volume, you need to include the LVM binary in your initramfs. You can get a static binary by enabling the  USE flag for. Copy it to your initramfs directory.

Now you can enable your LVM root partition in /init. This example assumes that your volume group is called VG, and your root volume is called root. Replace them with the names you chose when creating the volume.

Your root partition may then be called or.

Recent versions of rely on  to create the named LV device nodes, but there is no udev in a simple initramfs. You have the following choices:


 * use vgscan as shown above (simplest solution)
 * Mount by UUID or LABEL instead of using . It works because findfs is happy with just
 * build a LVM binary with the  USE flag (specifically for your initramfs only!)
 * disable udev dependency by including a minimal in your initramfs:

Software RAID
Normally the Linux kernel will automatically scan for any "Linux raid autodetect" partitions and start as many software RAIDs as it can find. But if you use an initramfs, the kernel will not automatically scan for RAIDs until it is told to. In the following example instructs the kernel to scan for software RAIDs and start as many as it can find. This will actually start all autodetected arrays, not just :

mdadm
If you are not using "Linux raid autodetect" partitions, or need to do more advanced RAID setup, then you will need to include mdadm in your initramfs. You can get a static binary by enabling the  USE flag for.

Copy the binary and your  into your initramfs:

Edit the in your initramfs to your liking. An example follows:

This will scan all  devices and assemble the RAID device fitting the UUID 627125a5:abce6b82:6c738e49:50adadae.

Now you can initialize your Software RAID in /init:

With this, you should be able to mount your root partition.

DM-Crypt
If your root partition is LUKS encrypted, you need to include the cryptsetup binary in your initramfs. You can get a static binary by setting the  USE flag for. Copy it to your initramfs directory. Since cryptsetup also often requires the use of the kernel's random device, include them as well.

Now you can unlock your encrypted root partition in /init:

Once you entered your passphrase, your root partition will be available as.

Encrypted Keyfile
If you need encrypted keyfiles, use cryptsetup to encrypt them. It keeps your initramfs simple as that's the encryption tool you already have - no need to add other binaries. Plus, unlike some of the alternatives, it offers a nice password prompt.

The following example creates a random 512 byte key, encrypted with LUKS, and adds it to your LUKS container.

Unlocking the root device using this key in your /init can then be done like this:

As before, your root partition should then be available as.

Networking
If you need networking in your initramfs, all required network related drivers have to be built into your kernel, and you'll have to configure the network interfaces in your /init. How exactly this has to be done, depends on your network situation. The following sections cover only the most common cases.

Static IP
If your network situation allows you to use a static network IP, you can set it up using the ifconfig and route commands, which are included in Busybox. This is by far the easiest solution, so if it's at all possible, go for it.

DHCP
To obtain a dynamic IP address from your network's DHCP server, you need a DHCP client. Busybox comes with a minimalistic DHCP client called udhcpc, which is sufficient for most users. Unfortunately, udhcpc has a dependency: it requires the help of a separate script to actually configure the network interface. An example for such a script is included in the Busybox distribution, but it's not installed by Gentoo. You will have to obtain it directly from the Busybox tarball (it's called ) or download it from the Busybox project page.

Copy the script to your initramfs and make it executable.

Edit the script's first line to read #!/bin/busybox sh or create a symlink for :

Now, you can obtain a dynamic IP address for eth0 using DHCP:

DNS
Your network should be up and running now. However, that's only if you know exactly which IPs to talk to. If all you have is a host or domain name, it's a different story entirely. In that case, you need to be able to resolve hostnames. Unfortunately, this is where our luck leaves us. Until now, everything could be done with just the static binary of Busybox - however, this is not the case with DNS.

This is because itself dynamically includes additional libraries for DNS lookups. As long as you don't need that functionality, you're fine, but if you need it, you have no choice but to include those libraries in your initramfs. The only alternative would be building Busybox against another libc such as, however that would go beyond the scope of this document.

This is a good chance to demonstrate how to use to reveal such hidden dependencies.

As you can see, the command accesses quite a lot of files, some of which are mandatory for it to work.

Copy the necessary libraries to your initramfs.

Create a with at least one nameserver you can use. Note that this may be done automatically for you if you use DHCP.

With this, DNS lookups should now work for you.

Troubleshooting
The following section tries to provide help with common issues and pitfalls you may run into.

Static vs. Dynamic binaries
Any custom binaries you need to use in your initramfs before mounting have to be fully functional, independently from any files you may have installed on your root partition. This is much easier to achieve with static binaries (which usually work as single file) than with dynamic binaries (which need any number of additional libraries to work).

Gentoo provides static binaries for some ebuilds. Check if the ebuild for your binary offers a  or   USE flag. That's by far the easiest method to get a static binary for something, but unfortunately only a select few ebuilds support it.

Many applications also offer static builds with an option in their configure scripts. There is no standard name for the option, it may be --enable-static or similar. When compiling a package manually, check the list of available options with ./configure --help to see if it supports building static binaries for you.

You can check whether or not a binary is static by using the ldd command. The strace command is also very useful to find out about additional dependencies. With equery files you can see which files a certain package brought into your system, some of which may also be candidates for additional dependencies of that package.

Including libraries into your initramfs in order to make a dynamic executable work is a last resort only. It makes the initramfs much larger and more complicated to maintain than necessary. It also makes the initramfs harder to maintain, as the dependencies might change with every update of the program in question.

Kernel panics
When working with initramfs and writing custom init scripts for it, you may experience the following kernel panic on boot:

Kernel panic - not syncing: Attempted to kill init!

This is not an error in the kernel, but an error in your /init script. This script is executed as the init process with PID 1. Unlike other processes, the PID 1 init process is special. It is the only process that is started by the kernel on boot. It's the process that takes care of starting other processes (boot process, init scripts) which in turn start other processes (daemons, login prompts, X), which in turn start other processes (bash, window manager, browser, ...). The init process is the mother of all other processes, and therefore it mustn't be killed. On shutdown, it's again the init process that takes care of cleaning up by shutting down other processes first, then running processes that will unmount the filesystems, until it is safe to actually do a shutdown without corrupting anything.

If you have some error in your /init script, that causes the init process to end, this basically means there are no processes left to run, there is nothing that could take care of cleaning up, and the kernel has no choice but to panic. For this reason there are some things in /init that you can't do like you can do them in a normal shell script, like using return or exit, or letting the script just run a series of commands and then simply end.

If you want /init to end, you have to pass the responsibility of the init process to someone else using exec. See the examples above how exec is used to either run /sbin/init of the mounted root partition or to run a rescue shell in case something went wrong.

Job Control
While working with initramfs, especially the Rescue Shell, you may come across this message:

/bin/sh: can't access tty; job control turned off

The lack of Job Control is usually not a problem, since /init is not supposed to be interactive. However, if you want to work with the Busybox shell on a regular basis, being unable to control programs with or  can easily become a huge issue. In worst case, if Job Control is not available, and a program refuses to quit, you have to reboot.

The busybox FAQ offers some help here. You can either use

or

to start a shell on tty1 with jobcontrol enabled.

Salvaging
If for whatever reason you lost your structure, but you still got either the kernel image with the built-in initramfs, or the separate cpio archive, it's possible to salvage it from there. Although it may be easier to just redo it from scratch - if you've done it once, doing it again should be a piece of cake. So this is just in case.

Dismantling the Kernel
You can skip this step if your initramfs is a separate cpio archive already. Otherwise, you'll have to get the built-in cpio archive out of the kernel image. To do that, you have to dismantle it, which isn't easy, since the kernel image is a combination of boot sector and compressed archive itself. It also depends on the compression you are using for your kernel and for your initramfs. For simplicity, this example assumes bzip2 - however, the principle is the same for other compression methods.

The utility of choice when dismantling kernel images is. In analyzes arbitrary files for known signatures, and prints their offsets. While there are usually a bunch of false matches in the output, it should be easy to pick the correct ones.

A less sophisticated method would be to grep for signatures. For bzip2, this is BZh. For gzip, you can use  $'\x1f'$'\x8b' .

In this case the offset we are looking for is 15949 bytes. Now you can extract the compressed kernel image:

Now, you have the uncompressed kernel image. Somewhere within this image resides the compressed initramfs archive, so just iterate the previous process to find it. Depending on your kernel configuration, you're looking for another bzip2, gzip, or cpio container.

Suppose the offset is 171424 bytes this time. Now you can extract the initramfs cpio archive:

If you want to verify that you actually got a cpio archive from that, use file:

Extracting the cpio archive
If your initramfs cpio archive was a separate file, you have to uncompress it first.

To extract the uncompressed, you can do so with the following command:

With this, you should have successfully recovered your initramfs structure.

Integrated initramfs doesn't always update
If your initramfs is integrated into your kernel (instead of using a separate file), there's a possibility that a make bzImage does not actually update it every time. So you might be making changes to your initramfs but actually keep booting using your old, buggy one. In this case you have to manually delete the integrated image to force the kernel to integrate a fresh initramfs archive:

Alternatively, you could also make clean, but then the entire kernel will need to be recompiled.

Examples
Fully functional examples of finished /init scripts can be found here: Custom_Initramfs/Examples

Resources

 * less (or read on kernel.org)
 * Initramfs/HOWTO
 * Dracut
 * Genkernel