This guide tells you how to make use of prelink support in Portage 2.0.46 and later.
- 1 Introduction
- 2 Setting up Prelink
- 3 Prelinking
- 4 Known Problems and Fixes
- 4.1 "Cannot prelink against non-PIC shared library"
- 4.2 When I prelink my system some static binaries don't work anymore
- 4.3 Prelink aborts with "prelink: dso.c:306: fdopen_dso: Assertion `j == k' failed."
- 4.4 I use grsecurity and it seems that prelinking doesn't work.
- 4.5 Prelink fails with error "prelink: Can't walk directory tree XXXX: Too many levels of symbolic links"
- 5 Conclusion
- 6 Acknowledgements
Most common applications make use of shared libraries. These shared libraries need to be loaded into memory at runtime and the various symbol references need to be resolved. For most small programs this dynamic linking is very quick. But for programs written in C++ and that have many library dependencies, the dynamic linking can take a fair amount of time.
On most systems, libraries are not changed very often and when a program is run, the operations taken to link the program are the same every time. Prelink takes advantage of this by carrying out the linking and storing it in the executable, in effect prelinking it.
Prelinking can cut the startup times of applications. For example, a typical KDE program's loading time can be cut by as much as 50%. The only maintenance required is re-running prelink every time a library is upgraded for a pre-linked executable.
- Prelinking is done via a program called, surprisingly,
prelink. It changes the binary to make it start faster.
- If an application's dependent libraries change after you have prelinked it, you need to re-prelink the application, otherwise you lose the speed advantage. This is to say, everytime you update a package via portage that updates libraries, they need to be re-prelinked.
- The change to the binary is fully reversible.
prelinkhas an undo function.
- Current versions of Portage can handle, via
prelink, the changing MD5sums and mtimes of the binaries.
- You do not need to set
FEATURES="prelink"in your make.conf file; Portage will automatically support prelink if it can find the prelink binary.
Installing the Programs
First you need to install the
prelink tool. The emerge process automatically verifies that your system can prelink safely.
A number of people get errors in emerging prelink because of the failed tests. The tests were put in for safety reasons, prelink's behavior is undefined if you disable them. The emerge errors are usually only dependent on the core packages; binutils, gcc, and glibc. Try re-emerging those packages in that order.
If you have a set of steps that reproduces the emerge error on another system please check Bugzilla and create a bug report if it has not already been reported.
Preparing your System
Also make sure that you do not have -fPIC set in your CFLAGS/CXXFLAGS. If you do, you will need to rebuild your entire system without.
env-update will generate the /etc/prelink.conf file that tells prelink which files to prelink.
Unfortunately you cannot prelink files that were compiled by old versions of binutils. Most of these applications come from pre-compiled, binary only packages which are installed in /opt . Making the following file will tell prelink not to attempt to prelink them.
I use the following command to prelink all the binaries in the directories given by /etc/prelink.conf .
|The options explained:|
|-a||"All": prelinks all the binaries|
|-m||Conserve the virtual memory space. This is needed if you have a lot of libraries that need to be prelinked.|
|-R||Random -- randomize the address ordering, this enhances security against buffer overflows.|
sys-devel/prelink-20060213 and later install a cron job in /etc/cron.daily/prelink . To enable it, edit the configuration file /etc/conf.d/prelink . This will run prelink daily in the background, as needed, saving you running the command manually.
KDE's loading time can be greatly reduced after prelinking. If you inform KDE that it has been prelinked it will disable the loading of
kdeinit (as it isn't required anymore) which speeds up KDE even more.
KDE_IS_PRELINKED=1 in /etc/env.d/*kdepaths* to inform KDE about the prelinking.
If you ever change your mind about prelinking, before you unmerge prelink, you'll first need to remove the prelink cronjob from /etc/cron.daily and /etc/conf.d/prelink . Next, you'll have to remove prelinking from all binaries:
Finally, unmerge the
prelink package itself:
Known Problems and Fixes
The cause of this problem is from badly compiled shared libraries that were compiled without the -fPIC gcc option for all their object files.
Here are the libraries that haven't been fixed or cannot be fixed:
- The libraries in the wine package, including winex. Prelinking wouldn't speed up MS Windows executables anyway.
- The library in media-video/mjpegtools, /usr/lib/liblavfile-1.6.so.0 .
- Nvidia OpenGl libraries, /usr/lib/opengl/nvidia/lib/libGL.so.* . Due to performance reasons they were compiled without PIC support.
If your problem library was not listed please report it with, preferably, a patch to add
-fPIC to the relevant CFLAGS.
Where glibc is concerned, there is no such thing as a 100% static binary. If you statically compile a binary with glibc, it may still depend on other system files. Below is an explanation by Dick Howell,
"I suppose the idea is that everything will be in the downloaded file, so nothing depends on the local libraries on the target system. Unfortunately with Linux, and I think anything else using GLIBC, this still isn't quite true. There's this "libnss" (name service switch, some people seem to call it network security system) which provides functions for accessing various databases for authentication, network information, and other things. It's supposed to make application programs independent of the separately configured actual network environment of the machine. A nice idea, but changes to GLIBC can lead to problems loading it. And you can't statically link "libnss", since it is configured for each machine individually. The problem comes, I think, mainly from statically linking other GLIBC libraries, notably "libpthread", "libm", and "libc", from which come incompatible calls to "libnss" functions."
This a known problem, kindly diagnosed here . Prelink cannot cope with UPX-compressed executables. As of prelink-20021213 there is no fix except to hide the executables while you are prelinking. See the section above for an easy way to do this.
In order to prelink on a system with grsecurity using a randomized mmap() base, it is necessary to turn "randomized mmap() base" OFF for /lib/ld-2.3.*.so . This can be done with the
chpax utility, but it must be done when the file is not in use (f.i. boot from a rescue CD).
Your symlinks are nested too deeply. This happens when a symlink points to itself. For example, /usr/lib/lib -> lib is the most common. To fix this, you can find the symlink manually or use the utility provided by the
Prelinking can drastically speed up the start up times for a number of large applications. Support is built into Portage. Prelinking is also safe as you can always undo the prelinking for any binary if you come across any problems. Just remember that when you update glibc or other libraries that you prelinked with, you need to rerun prelink ! In short good luck!
We would like to thank the following authors and editors for their contributions to this guide:
- Stefan Jones
- John P. Davis
- Jorge Paulo
- Sven Vermeulen