Custom Initramfs

initramfs is a root filesystem that is embedded into the kernel and loaded at an early stage of the boot process. It is the successor of initrd. It provides early userspace which lets the system do things that the kernel cannot easily do by itself during the boot process.

Using initramfs is optional. By default, the kernel initializes hardware using built-in drivers, mounts the specified root partition, loads the init system of the installed Linux distribution. The init system then loads additional modules and starts services until it eventually allows users to log in. This is a good default behavior and sufficient for many users. An initramfs is generally for users with advanced requirements; for users who need to perform certain tasks as early as possible, even before the root partition is mounted.

Here are some examples of what can be done with an initramfs:


 * Mount the root partition (for encrypted, logical, and otherwise special partitions);
 * Provide a minimalistic rescue shell (if something goes wrong during the init process);
 * Customize the boot process (e.g. print a welcome message, boot splash, etc.);
 * Load modules (e.g. third party drivers that cannot be integrated into the kernel directly);
 * Anything the kernel is not able to do (as long as the task can be done in user space, e.g. by executing commands).

Users who do not have advanced requirements do not need to use initramfs.

Prerequisites
There are countless ways to make an initramfs. Users can choose not to create an initramfs at all but let utilities, such as Genkernel or Dracut, do the work for them. In some situations an automatically generated initramfs does what is needed 'out of the box.' If this is the case most users need not be bothered with how the initramfs works and what actions it is accomplishing. For less fortunate users an initramfs does not perform the needed actions, and its functionality must be extended. In special cases an initramfs will have to be (almost) entirely re-written in order to make proper operations possible.

An initramfs contains at least one file called. This file is executed by the kernel as the main init process (PID 1). It has to do all the work. In addition, there can be any number of additional files and directories that are required by. They are usually files that can be found on any root filesystem, such as for device nodes,  for kernel information,  for binaries, and so on. The structure of an initramfs can be simple, or it can be complicated, depending on the actions that are needed to be undertaken.

When the kernel mounts the initramfs, the target root partition is not yet mounted, so it cannot provide access any files contained therein. As far as the system is concerned, this means nothing but the files in the initramfs are visible. Everything that is needed or wanted must be included in the initramfs. If a rescue shell shell is desired, it must be included in the initramfs. If something needs to be mounted, a mount utility must be included. If a module needs loaded, the initramfs has to provide both the module as well as a utility to load it. If the utility depends on certain libraries in order to work, the libraries must be included as well. This seems complicated, and it is, because the initramfs needs to be able function entirely independent from any other devices or filesystems. In order for all parts to work smoothly and seamlessly an initramfs should be a self-contained system.

Basics
This section will detail the easy and straightforward way to initramfs creation. It will provide instructions on how to make a functional - albeit minimalistic - initramfs which then can be extended according to each system's requirements.

Directory structure
It is time to create a basic initramfs directory structure that will become the initramfs' root. For consistency in this article the path will be used, however any location will do. If an alternate directory is chosen remember to consistently update the paths accordingly.

Create a basic directory structure:

Device nodes
Most things the initramfs does will require a couple of device nodes to be present, especially the device for the root partition. Throughout this document, will be used as example device. Copy basic device nodes from the root filesystem to the initramfs example location:

The devices needed depends entirely on what the initramfs is going to be used for. Feel free to adapt the structure to meet individual needs.

Applications
Any binary required to be executed at boot needs to be copied into the initramfs. Remember: any libraries required by the binaries also need to be copied. A way to discover which libraries the particular binary requires is to use the ldd tool. In the following example the ldd tool is used to examine what libraries the binary requires:

From the output of ldd it is possible to see in order for to work in an initramfs two libraries must be present with it. Explained with more verbosity, not only must be copied to the  folder, but  and  must be copied to the  folder. Note that is not needed.

Some applications may be depend on other files and libraries to work. For example, needs a terminfo file  from, so they must be copied to the initramfs as well. To find these types of dependencies tools like equery and strace prove to be most helpful.

Busybox
Instead of collecting countless utilities and libraries (and never seeing the end of it), using multi-tool binary packages such as is sometimes the best approach. Busybox is a set of utilities for rescue and embedded systems. It contains a shell, utilities like ls, mkdir, cp, mount</tt>, insmod</tt>, and many more - all in a single binary. For busybox</tt> to work properly in an initramfs, it will need to be emerged with the  USE flag enabled. Once correctly emerged copy the binary into the initramfs layout as :

Init
The file structure of the initramfs is almost complete. The only thing that is missing is itself. is the executable in the root of the initramfs that is executed by the kernel once it is loaded into memory. Because includes a fully functional shell, this means the  binary can be written as a simple shell script (instead of making it a complicated application written in Assembly or C). When an interpreter is available, writing the in a shell language avoids the need to compiling which will dramatically save time when attempting to debug any issues that may develop.

The following example displays the executable as a minimalistic shell script, based on the busybox</tt> shell (sh):

This example needs some device nodes to work, mainly the root block device. Change the script and copy the corresponding node to suit each need.

Do not forget to make the file executable:

Packaging the initramfs
The initramfs now has to be made available to the kernel at boot time. This is done by packaging it as a compressed cpio archive. This archive is then either embedded directly into the kernel image, or stored as a separate file which can be loaded by GRUB during the boot process. Both methods perform equally well, simply choose the method that is most convenient.

Kernel configuration
With either method, support for the Initial RAM filesystem and RAM disk (initramfs/initrd) must be enabled for correct initramfs functionality. Be sure the kernel includes the following options:

Be sure to enable all drivers, filesystems, and other settings that are required for booting and accessing the root partition. When selecting such drivers as modules remember to collect and integrate the module files into the initramfs and load them into memory via the. Generally this means a lot of unnecessary extra work, so use built-in drivers for now.

Embedding into the Kernel
For the initramfs to be embedded into the kernel image, edit the kernel configuration to set the Initramfs source file(s) to the root directory of the initramfs (e.g. ):

Now when the kernel is compiled it will automatically put the files into a cpio archive and embed it into the kernel image. A disadvantage to using this method is that the kernel must be rebuilt any time changes to the initramfs (adding a file, making a script change, etc.).

Creating a separate file
Creating a standalone archive file can be done by running the following commands:

This will create a file called in the system's  directory. To instruct a bootloader such as GRUB2 to load this file at boot time, do so with the initrd line:

Finalizing
It is now time to reboot the machine. On boot, the kernel should extract the files from the initramfs archive automatically and execute the script. In turn this should take care of mounting the system's root partition and execute the file of the installed Linux distribution.

Functionality
Now that the basics of initramfs have been covered, this section will explain how to extend the script with advanced functionality.

Rescue shell
If an error occurs during the init process, a wise idea is drop into to a rescue shell so that debugging can be preformed as needed. The following function can be added to the program to be referenced when something goes wrong:

In the example below, the rescue_shell will be executed if the root partition fails to mount:

Dynamic devices
For populating dynamically, either devtmpfs or mdev can be used. Note that the Kernel can take some time detecting devices (such as external USB drives), so a sleep statement may need to be added to the script in order to find all present devices.

devtmpfs
Provided by the kernel, devtmpfs is designed to offer device nodes during early boot.

The following snippet can be included in the script to have it mount at boot:

Do not forget to unmount devtmpfs it in the cleanup phase of the script:

mdev
Although devtmpfs is the modern preferred solution, an alternate method is to use mdev, which is busybox</tt>'s udev replacement.

For mdev to work, must be symlinked to  in the initramfs:

The following snippet must be added to the, after mounting the and  filesystems:

Mount by UUID or label
With Dynamic devices enabled, it may be preferable to use UUID or label to mount the root partition instead of using a static device name. For that purpose, busybox</tt> comes with a utility called findfs</tt>.

Doing it this way is simple, but it means that the UUID or label is hardcoded in the. Alternatively, Kernel parameters can also be used.

Kernel parameters
To use kernel parameters instead of hardcoding device names or UUIDs, the kernel's file will need to be parsed. There are many ways to do so, the following method is an example to give the general idea of parsing. It uses string manipulation of the shell and only supports  parameters. Add the following function to the and call it whenever a kernel parameter is needed.

The function is called with the name of the kernel parameter. In the example below it uses the root parameter to mount the root partition.

It works for both  and   but will fail when the parameter is missing.

LVM
If the root partition is located on a logical volume, then the LVM binary must be included in the initramfs. A static binary can be generated by enabling the  USE flag for. Copy it to the initramfs' directory.

After the above step has been performed, the LVM root partition can be enabled in the file. This example assumes the volume group is called VG, and the root volume is called root. Replace them with the appropriate names used to create the volume.

The root partition may then be called or.

Recent versions of rely on  to create the named LV device nodes, but there is no udev in a simple initramfs. The following choices are available:


 * Use vgscan</tt> as shown above (simplest solution);
 * Mount by UUID or label instead of using . It works because findfs</tt> is happy with just ;
 * Build a LVM binary with the  USE flag (specifically for the initramfs only!);
 * Disable udev dependency by including a minimal in the initramfs:

Software RAID
Normally the Linux kernel will automatically scan for any "Linux raid autodetect" partitions and start as many software RAIDs as it can find. However, if an initramfs is used the kernel will not automatically scan for RAIDs until it is told to. The following example instructs the kernel to scan for software RAIDs and start as many as it can find. This method will actually start all autodetected arrays, not just :

mdadm
When not using "Linux raid autodetect" partitions, or when a more advanced RAID setup exists, the mdadm</tt> binary must be included in the initramfs. As stated above, a static binary can be generated by enabling the  USE flag when installing.

Copy the binary and the  file into the initramfs:

Edit the in the initramfs as needed. An example follows:

This will scan all  devices and assemble the RAID device fitting the UUID 627125a5:abce6b82:6c738e49:50adadae.

Now initialize the Software RAID in the :

By implementing these steps it should be possible to mount a root partition located on.

DM-Crypt
If the root partition is LUKS encrypted, the cryptsetup binary needs to be included in the initramfs. A static binary can be generated by setting the  USE flag and emerging the  package. Copy the binary to the initramfs' directory. Since cryptsetup also often requires the use of the kernel's random device, they need to be included as well.

It is now possible to unlock an encrypted root partition in the :

Once the passphrase has been entered the root partition will be available as.

Encrypted keyfile
If encrypted keyfiles are needed, use cryptsetup</tt> to encrypt them. It keeps the initramfs simple since it will be using the encryption tool that is already available - no need to add other binaries. Plus, unlike some of the alternatives, it offers a nice password prompt.

The following example creates a random 512 byte key, encrypted with LUKS, and adds it to a LUKS container.

Unlocking the root device using this key in the can then be done like this:

As before, the root partition should then be available as.

Networking
To add network support to the initramfs, all required network-related drivers must be built into the kernel, and the network interfaces will have to be configured in the. How exactly this has to be done, depends on the network situation. The following sections cover only the most common cases.

Static IP
If the network situation allows for the use of a static network IP, it can be configured using the ifconfig</tt> and route</tt> commands, both of which are included with Busybox. This is by far the easiest solution, so if it is at all possible, go for it.

DHCP
To obtain a dynamic IP address from the network's DHCP server, a DHCP client is needed. Busybox comes with a minimalistic DHCP client called udhcpc</tt>, which is sufficient for most users. Unfortunately, udhcpc</tt> has a dependency: it requires the help of a separate script to actually configure the network interface. An example for such a script is included in the Busybox distribution, but it is not installed by Portage. It will have to be obtained directly from either the Busybox tarball (it is called ) or by downloading it from the Busybox project page. Either method will work. Downloading is probably the easiest route.

Copy the script to the initramfs and remember to make it executable.

Edit the script's first line to read #!/bin/busybox sh or create a symlink for :

It should now be possible to obtain a dynamic IP address for the network interface using DHCP:

DNS
The network should now be up and running. However, that is only if the system knows exactly which IPs to talk to. For situations that only have a host or domain name, it is a different story entirely. In that case, system must be able to resolve hostnames. Unfortunately, this is where simplicity takes a leave of absence. Until now, everything could be done with just the static binary of Busybox - however, this is not the case with DNS.

This is because itself dynamically includes additional libraries for DNS lookups. As long as this type of functionality is not needed, it is fine, but if they are needed, then there is no choice but to include the libraries in the initramfs. The only alternative would be building (recompiling) Busybox against another libc such as, however that would go far beyond the scope of this article.

This is a good chance to demonstrate how to use the strace</tt> tool to reveal hidden dependencies.

As visible above, the command accesses quite a lot of files, some of which are mandatory for it to work.

Copy the necessary libraries to the initramfs:

Create a file with at least one useable nameserver. Note that this step may be done automatically if DHCP is used.

With these steps taken DNS lookups should now be possible.

Troubleshooting
The following section tries to provide help for common issues and pitfalls.

Static vs. dynamic binaries
Any custom binaries required in the initramfs before mounting the root partition have to be fully functional, independent from any files that may be installed on the root partition. This is much easier to achieve with static binaries (which usually work as single file) than with dynamic binaries (which need any number of additional libraries to work).

Gentoo package maintainers have provided static binaries for some ebuilds. Check if the ebuild for the desired binary offers a  or   USE flag. This is by far the easiest method to get a static binary, but unfortunately only a select few ebuilds are supported.

Many applications offer static builds with an option in their configure scripts. There is no standard name for the option, it may be  or something similar. When compiling a package manually, check the list of available options by using <tt>./configure --help</tt> to see if the package supports building static binaries.

As mentioned earlier in this article, it is possible to verify if binary is static by using the <tt>ldd</tt> command. The <tt>strace</tt> command is also very useful to find out about additional dependencies. By using <tt>equery files</tt> it is possible to see which files a certain package has brought into the system, some of which may also be candidates for additional dependencies of that package.

Including libraries into the initramfs in order to make a dynamic executable work should only be taken as a last resort. Not only does including dynamic executable make the initramfs much larger (since the libraries are needed), they also make it more complicated to maintain than necessary; the dependencies (libraries) for each dynamic programs might change with each new version of the program.

Kernel panics
When working with an initramfs and writing custom init scripts for it, it is not uncommon to experience kernel panics on boot:

Kernel panic - not syncing: Attempted to kill init!

This is not an error in the kernel, but an error in the script. This script is executed as the init process with PID 1. Unlike other processes, the PID 1 init process is special. It is the only process that is started by the kernel on boot. It is the process that takes care of starting other processes (boot process, init scripts) which in turn start other processes (daemons, login prompts, X), which in turn start even more processes (bash, window manager, browsers, etc.). The init process is the mother of all other processes, and therefore it must not be killed. On shutdown, it is again the init process that takes care of cleaning up by shutting down other processes first, then running processes that will unmount the filesystems, until it is safe to actually do a shutdown without corrupting anything.

When experiencing some error in the script, that causes the init process to end, it basically means there are no processes left to run, there is nothing that could take care of cleaning up, and the kernel has no choice but to panic. For this reason there are some things in the that cannot be performed as they can in a normal shell script, a return or an exit statement, or letting the script run a series of commands that end the init process after they are finished.

When wanting the /init to end, you have to pass the responsibility of the init process to another process using <tt>exec</tt>. See the examples above for how <tt>exec</tt> is used to either run of the mounted root partition or to run a rescue shell in case something went wrong.

Job control
While working with initramfs, especially the Rescue shell, the following message may be displayed:

/bin/sh: can't access tty; job control turned off

The lack of job control is usually not a problem, since the is not supposed to be interactive. However, when wanting to work with the Busybox shell on a regular basis, being unable to control programs with + or + as one would normally do with jobs can easily become an issue. In worst case, if job control is not available, and a program refuses to quit, the machine will have to be rebooted.

The job control section in the Busybox FAQ offers some help here. According the the FAQ either

or

can be used to start a shell on tty1 with jobcontrol enabled.

Salvaging
If for whatever reason the structure has been lost or destroyed, but a copy of the kernel image with the built-in initramfs or the separate cpio archive is still available, it is possible to re-create it. Although it may be easier to redo it from scratch (if it has been done once, doing it again should be a piece of cake, right?). So this is just-in-case.

Dismantling the Kernel
This step can be skipped if the initramfs is a separate cpio archive already. Otherwise, the built-in cpio archive must be extracted from the kernel image. To do that, it must be dismantled, which is not easy, since the kernel image is a combination of a boot sector and compressed archive itself. It also depends on the compression being use for the kernel and the initramfs. For simplicity, this example assumes bzip2 compresses has been used; with that being said the principle is the same for other compression methods.

The utility of choice when dismantling kernel images is. In analyzes arbitrary files for known signatures, and prints their offsets. While there are usually a bunch of false matches in the output, it should be easy to pick the correct ones.

A less sophisticated method would be to use <tt>grep</tt> to search for signatures. For bzip2 compression, this is BZh. For gzip, use  $'\x1f'$'\x8b' .

In this case the offset to search for is 15949 bytes. Now it is possible to extract the compressed kernel image:

Now the kernel image has been extracted. Somewhere within this image resides the compressed initramfs archive, so just iterate the previous process to find it. Depending on the kernel configuration, another bzip2, gzip, or cpio container should be searched for.

Suppose the offset is 171424 bytes this time, extract the initramfs cpio archive using this command:

To verify that a cpio archive was extracted from that, use the <tt>file</tt> command:

Extracting the cpio archive
If the initramfs cpio archive was a separate file, it needs to be uncompressed first.

To extract the uncompressed, you can do so with the following command:

The initramfs structure should be successfully recovered.

Integrated initramfs does not always update
If the initramfs is integrated into the kernel (instead of using a separate initramfs file), there is a possibility that the <tt>make bzImage</tt> does not actually update it every time. It is more than possible that changes have been made to the initramfs but tests have been performed using the old, buggy version. In this case it is necessary to manually delete the integrated image in order to force the kernel to integrate a fresh initramfs archive:

Alternatively, the same action can completed using the <tt>make clean</tt> command, but then the entire kernel will need to be recompiled, which will take quite a bit longer.

Command not found
In Gentoo, Busybox is configured as standalone shell by default, which allows Busybox to execute its own applets directly. Without this setting, standard Busybox commands (<tt>mkdir</tt>, <tt>mount</tt>, etc.) will not work unless there is explicitly a symlink created for them. This symlink should be created at the top of the script:

An alternative method is to create the symlinks directly in the directory so they will already be included in the initramfs:

Custom init script examples
Fully functional examples of finished scripts can be found in the  custom init script examples sub-article.

External resources

 * Official initramfs documentation locally ( less ) or online at kernel.org