Project:Quality Assurance/As-needed

This guide is meant to explain how the --as-needed LDFLAG works and to provide steps to fix simple cases where --as-needed fails to build a package.

What is --as-needed?
The  flag is passed to the GNU linker (GNU   ). The flag tells the linker to link in the produced binary only the libraries containing symbols actually used by the binary itself. This binary can be either a final executable or another library.

In theory, when linking something, only the needed libraries are passed to the command line used to invoke the linker. But to workaround systems with broken linkers or not using ELF format, many libraries declare some "dependencies" that get pulled in while linking. A simple example can be found by looking at the libraries declared as dependencies by :

libraries needed to link to gtk+ 2.0

Of course, if the application is just using functions from gtk+ 2.0, a simple link line with  should make it build fine, but looking at which libraries are needed and which are not from a package point of view is often an impossible task.

How can --as-needed be useful?
The use of the  flag allows the linker to avoid linking extra libraries in a binary. This not only improves startup times (as the loader does not have to load all the libraries for every step) but might avoid the full initialization of things like KDE's KIO for a binary if it's not using the KIO framework.

More importantly, the use of  avoids adding dependencies to a binary that are prerequisites of one of its direct or indirect dependencies. This is important because when a library changes SONAME after an ABI change, all the binaries directly linking to it have to be rebuilt. By linking only the libraries that are actually needed, the breakage due to an ABI change is reduced. It is particularly useful when the ABI breakage happens in a library used by some other high level library (like, which is used directly by  , and gets linked indirectly in applications using the latter), as it prevents the rebuild of the final binaries and thus of the packages carrying them.

It is also useful to check whether the dependencies stated by the documentation are actually used by a package: it's not impossible that a package checks in a configure script for some library, and then links to it, but without using it at all because the code using it was removed or refactored or has not been written.

How to use --as-needed
If you want to use the  flag, you must make sure that you do not override the profiles in  file or instead manually add the flag back; check   to ensure that it is there, if not, you'll want to use the below code example to add it.

The below code example shows how to enable it by adding a line in case that you start from a LDFLAGS variable different than those from the profiles:

Enabling --as-needed in /etc/portage/make.conf

This flag should conditionally work on every Linux platform supported by binutils. It's known not to work on FreeBSD and probably does not work on other non-Linux targets. As  depends not only on the way the package you're building is linked but also its dependencies, there are quite a few packages that were silently patched and fixed and might require rebuilding. Please make sure to rebuild the dependencies failing to link against before filing a bug.

Forced --as-needed
Using  through LDFLAGS is the suggested method for standard users and users who don't want to have too many problems with packages failing to build. For developers, power users, and people who want to stress test buildsystems, a different strategy is also available, that forces each and every build to use  during the linking phase, as long as the   frontend is used as linker.

The forced approach is useful to test packages that don't respect the LDFLAGS variable or packages that incorrectly filter the flag (see the next section for details about that), and to work around the  reordering bug. A compiler set to force  is easily reversed so that if a package is needed to be built without the flag a single switch is required.

To force the  flag on the compiler, GCC spec files are used, creating a new profile for the compiler that always use it during linking phase. For any version of  it is possible to build the modified spec file using the following series of commands

commands to set up a forced --as-needed compiler

To switch between the  and standard compilers, just use   like they were different compiler versions or hardened and standard compilers.

Properly filtering --as-needed
Sometimes it is needed to filter  as the software is designed in such a way that it is not fixable to make use of it. Unfortunately just filtering the flag is often not the best choice because can be tricky (they can be concatenated through commas, making it impossible to filter them out), and because it is possible to set it as a default through GCC's spec files.

If you really need to disable the  behaviour, because of design choices that conflicts with the way   works (and thus not just because the package is broken, and fixable, when building with   ), what you should be doing is something along these lines.

correct filter for --as-needed behaviour

Identifying the problem
Developers willing to fix failures related to the  flag should be aware that there are many cases of failures that may fall into one of a few different categories. I'll try to explain here the reasoning behind the failures and ways to fix them; some of them are really simple, others are not.

If you find a package failing when using  you may file a bug in Gentoo Bugzilla, blocking bug #129413.

Failure in compile, unrecognized option
This is at the same time the most simple and the most annoying problem that can be found. There might be packages failing with an error like the following after adding  to LDFLAGS:

common error while using --as-needed

This is caused by  being called with the  variable instead of  , thus breaking because it doesn't recognize the   prefix used to tell   to pass the option down to the linker. To fix this, one must pass the output of the  function to the   process.

example of raw-ldflags usage

Failure in final linking, undefined symbols
This is the most common error that happens while using. It happens during the final linking stage of an executable (libraries don't create problems, because they are allowed to have undefined symbols). The executable linking stage dies because of an undefined symbol that is present in one of the libraries fed to the command line. However, the library is not used by the executable itself, thus it gets removed by.

This usually means that a library was not linked to another library, but was using it, and then relying on the final executable to link them together. This behavior is also an extra encumbrance on developers using that library because they have to check for the requirements.

The fix to this kind of problem is usually simple: just find which library provides the symbols and which one is requiring them (the error message from the linker should contain the name of the latter). Then make sure that when the library is linked from the source files it's also linked to the first. While using autotools, the dependent library has to be checked in the configure (this should already be the case to specify the dependencies in the  data file or in the script provided) and then the variable carrying this value should be added to the LIBADD variable for the library to be built.

Failure in execution, undefined symbols
Sometimes the undefined symbol errors don't happen while linking, but rather at the execution of an application built with --as-needed. The cause, however, is just the same as for the undefined symbols in linking: a directly-linked library did not link one of its dependencies. It also has the same solution: find which library carries the undefined symbols and make sure that it gets linked to the library providing them.

Failure in ./configure
Albeit less common than other kind of failures,  execution can fail because of   too. With this kind of failures, though, it's difficult to give a single and simple solution, as there are multiple reasons that might make the script fail.

The most common option between those that we have now is that a library is checked for, but that library wasn't linked to all its prerequisites (and thus required them to be passed afterward). As  makes the linker ignore all the libraries not needed by the current target, the prerequisites will result missing.

To check a or  file for this kind of failures, you can look for the   macro, and see whether the fifth parameter is used ( other-libraries ). When it is, it often means that those libraries need to be linked in the final binary to satisfy the dependencies of the library to check for. At that point, the library need to be fixed.

example of AC_CHECK_LIB call with fifth-parameter

Another possible failure during  execution happens when the code is going to check for functions or other particular items (symbols, behaviour) and mistakenly pass the dependency libraries through the  variable.

example of mistake in library checks

In this case the solution is very quick: just replace LDFLAGS with LIBS in both saving and restoring. This way the libraries will be passed in the correct order (see the following section for more insight about this kind of problem).

Importance of linking order
One thing that might be frustrating when fixing packages for  is that sometimes, although all the libraries are present in the linking line, they are just ignored and not linked at all. This leads to the same issues as above; missing symbols either during final link or execution. This is because of a behaviour of the GNU linker that's enforced by.

Basically, what the linker does is look for the symbols missing in a given file (either an object file, a static archive or a library) only in the files coming after it. When using the normal linking, without , this is not a problem, although there might be some internal drawbacks on the linking stage, the files are linked together without considering ordering. But with the flag, the libraries that aren't used for resolving symbols are discarded and thus not linked.

example of wrong and correct linking order

The fix in this case is to simply fix the linking order so that the libraries given to the linker are all after the object files and the static archives.

While using autotools there are usually small cases where this happens, because usually libs are fed either via the variable in the configure script or are listed in the LDADD/LIBADD variables for the target which is being built. The only case when this happens to be a problem is when the libraries get fed into LDFLAGS variable (which is incorrect).

Initializers and Deconstructors
There exists a class of applications at the moment that break when using. These applications are not at fault, but rather the linker itself. The linker is unable to detect dependencies between the initializers and deconstructors (.init/.fini ELF sections) when working with C++ code. As such, it may discard libraries when none of the symbols are used from it, thus mistakenly changing the initialization and deconstruction code paths.

While this class of applications is small and there are no known applications yet which fall into this category, this is something to keep in mind. The only way to really detect such a thing is by proper source code and runtime analysis.

Acknowledgements
We would like to thank the following authors and editors for their contributions to this guide:


 * Diego E. Pettenò