/usr move

The /usr move is a concept of moving various applications installed into the filesystem root to prefix in order to unify the filesystem layout.

Current filesystem layout
Right now, the filesystem layout involves arbitrarily splitting the installed applications and libraries between and  prefixes. The FHS specifies that the prefix should be used by 'essential' executables and libraries.

This definition is quite blurry. In Gentoo, we usually interpret 'essential' as necessary for the system to boot and mount filesystems (especially ). This also involves rescue from common problems like filesystem damage.

The current filesystem layout looks like the following:

/ ├── bin ├── boot ├── dev ├── etc ├── home ├── lib ├── lib32 ├── lib64 ├── media ├── mnt ├── opt ├── proc ├── root ├── run ├── sbin ├── sys ├── tmp ├── usr │   ├── bin │   ├── games │   ├── include │   ├── lib │   ├── lib32 │   ├── lib64 │   ├── libexec │   ├── local │   ├── portage │   ├── sbin │   ├── share │   ├── src │   └── └── var

You can notice two things:
 * 1)  contains both 'global' directories (, ...) and 'prefix' directories (, ...),
 * 2)  does not contain all 'prefix' directories  has (there is no  and no ).

Wanted filesystem layout
The idea of complete move involves not using the  prefix for installed packages and installing everything to  instead. However, this is practically not possible due to heavy reliance on rootfs paths in compiled executables and standards.

With a complete move, the layout would look like the following:

/ ├── boot ├── dev ├── etc ├── home ├── media ├── mnt ├── opt ├── proc ├── root ├── run ├── sys ├── tmp ├── usr │   ├── bin │   ├── games │   ├── include │   ├── lib │   ├── lib32 │   ├── lib64 │   ├── libexec │   ├── local │   ├── portage │   ├── sbin │   ├── share │   ├── src │   └── └── var

Now 'prefix' directories start appearing in prefix and  contains just 'global' paths.

Realistic semi-clear layout
Sadly, as many applications and libraries expect various files to exist in a fixed locations, we would have to keep them available there. For that reason, the old 'prefix' directories will exist with a number of compatibility symlinks including:

/ ├── bin │   ├── sh │   └── (...) ├── lib │   └── ld-linux.so.2 └── lib64 └── ld-linux-x86-64.so.2

A common location for all shared data
Currently, the read-only shared data for various programs are split between subdirectories and. This means, that the rootfs usually contains both read-only and writable data which separating requires using a number of additional, small mountpoints.

The concept is that after the split, all shared, read-only data would be stored in, while other locations in rootfs will be used to perform their respective (mostly read/write) tasks.

Possible uses include:


 * 1) sharing the whole system over NFS. Right now this requires sharing, ,  and . After the move, only  would be necessary;
 * 2) mounting  read-only while keeping  read/write. Right now, this keeps the most important system executables and libraries writable. Of course, a preferable alternative is to mount whole  read-only but it usually involves moving some files from  to other locations (like  when using DHCP).

Avoiding excessive use of rootfs
The requirements for system boot are raising, and along with that, more files are moved onto rootfs to satisfy the dependencies. This slowly introduces the following problems:


 * 1) Increasing space consumption on rootfs. The more dependencies are moved there, the more space is consumed. At some point, users may be required to resize their rootfs in order to fit the new dependencies being introduced;
 * 2) Increasing number of statically linked executables. As an alternative to moving libraries into rootfs, some packages are using static linkage for their executables. Disadvantages of static linkage are not the topic of this article but their include inability to handle library upgrades through the package manager and thus make it harder to fix security issues;
 * 3) Necessity of introducing additional directories on rootfs or moving files elsewhere. Right now, rootfs does not specify any location for shared, read-only data  so those files are either placed in,  or.

Simplification of ebuilds
Right now, an ebuild wishing to install a library to rootfs needs to:


 * 1) Adjust the package install prefix from the default one  to ,
 * 2) If a package uses static libraries, manually move them from  to ,
 * 3) If a package uses static libraries, create shared library wrappers in  to avoid linker preferring the shared ones,
 * 4) If some non-core executables depend on libraries in, manually move them to.

The ebuild maintainer basically has to ensure that the program will work without mounted. This means that he needs to ensure that all the dependencies, possibly with all flag combinations, end up in rootfs.

Moving the application to simply makes the maintenance easier.

initramfs or other pre-mount (for system packages)
In order to perform the move on packages required to boot the system, it is necessary that all users having separate  partition have an enabled and working initramfs. Such an initramfs would need to mount the filesystem in order to execute the system init.

A possible alternative for an initramfs (which many people oppose just because they can) is creating a single, statically linked pre-init executable on rootfs and using it to mount before executing the actual init.

/usr remount in system RC
Considering that initramfs would mount read-only (alike  is mounted now), OpenRC or any other RC used in Gentoo would need to be able to perform fsck on mounted  and re-mount it read/write afterwards. In other words, would need to be handled alike  is handled now.

Testing overlays
The move variant described in this article is being tested in the usr-gentoo overlay.

Modifying package install prefixes
The most important process in the move is updating the packages to not use  prefix. In most cases, this simply involves removing the --prefix argument to configure calls since is the default prefix used by econf.

Some packages install files to rootfs directories explicitly. Those packages may require actually modifying the build system or specifying the prefix some other way.

Adding necessary symlinks
To maintain compatibility with various packages, a symlinks to various binaries or libraries may need to be created on rootfs. However, it is preferred that packages in question are checked for possibility of fixing first.

For example, if a script calls, it is preferred that it is fixed to just call foo and let the shell handle ${PATH} resolution rather than creating an unnecessary symlink.

The /usr move in other distributions
A similar concept is being considered in Fedora yet it assumes that the rootfs directories would become symlinks to counterparts.

External resources

 * https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/