Project:Quality Assurance/Autotools failures

This guide aim to describe the common situations which makes autotools fail to run in an ebuild, giving advices on how to fix those.

Introduction
With the term autotools we usually refer to the tools developed by the GNU project to create platform and operating-system independent build systems, ,   and. Although not every package uses all of them at the same time, most of the modern ones do so; older packages often does not use  and   instead; KDE packages uses a more complex build system that relies at the end on the three software above.

It's simple to recognize a package which build system is based on autotools: if there's a script, and a  or  file, the build system is based on   ; if there are one or more  files in the various sub-directories, it's also   based; if there's a  script, it's also using.

To build a package that uses an autotools-based build system, the tools themselves aren't strictly needed: the script is a simple Bourne Shell script (usually, but this will be discussed lately) and it transforms the  files into simpler  for   (or, more often,   ). In spite of them being optional for building the software, often the patches needed to solve problems like --as-needed build failure or automagic dependencies requires us to re-run the tools to recreate scripts and makefiles' templates.

This guide won't give directions on how to fix packages' errors with use of autotools, as that's a wide topic that requires lot of stuff to be explained. For a simple introductions of the most common errors while using autotools, it's rather suggested to read the best practices with autotools article. Instead, it will describe the common cases where re-running autotools lead to failures, either on the rebuilding of the script or at build time.

Re-running autotools
The first important thing to know is how to properly rebuild autotools support, as that is a common problem that makes some ebuild fail. The order in which autotools are run is important, as one rely on the other and the final output depends vastly on the order to be respected.

Many packages provide a single script, often called or  that is uses to execute the tools in the order that upstream thinks it's the right one, often setting variables so that the right version of the tools is run (different versions of autotools often does not get along well). These scripts are, in general, preferred over other methods but often they contains mistakes, or assumptions of a given environment that might be unique of another distribution, for this reason, they should be checked carefully, and when they does not provide any advantage to the other methods (as they might be calling the tools one after the other not even passing them special parameters or checking their return value), they should be discarded.

package provides an automated script, called that should automatically detect which autotools are used and call them, but this too often fail to recognize the right versions or breaks because of corner cases. Also, it does run, the script that adds  support to a package, that is almost never required to be run after patching a package. For this reason, is deprecated and avoided as possible (the same goes for custom upstream-provided scripts that uses it).

To overcome these problems, the eclass was added; this eclass provides wrapper functions around the GNU autotools:  ,   ,   (_ is prefixed to avoid collision with   functions from  eclass instead) and the most important   function. This function does not wrap around the broken  script, but rather analyse the autotools support files present and run the tools in their right order. It also run the  function to patch libtool support files if needed, avoiding problems when it's being called before the actual recreation of autotools files.

The functions in eclass have also the advantage of not presenting to the user a lot of raw output (in case of warnings) or nothing (in case there are no problems); instead they provide  /   status messages so users will know what's happening, and also they track the error situations by providing an   -like message in case of failure. For this reason, those functions are preferred over manual call or broken or almost pointless custom scripts. Another point is that eclass also add the build-time dependency over the needed packages.

Special case: KDE packages using kde.eclass
While KDE 3.x uses autotools like many other software packages, it is using a custom setup for them, with lots of custom macros, and further steps to complete the regeneration of all the needed files. For this reason, should not be used to rebuild the autotools for KDE packages that use the  eclass for building.

As a special exception to the common rule of rebuilding autotools during  phase, KDE packages rebuild their autotools during the   phase whenever the toplevel  file is missing. For this reason, if you want to rebuild autotools files for a KDE package, you just have to remove the file from the base source directory.

Most of the issues that are caused by KDE's own build system and appear with newer autotools versions can usually be solved by replacing the directory in the source tarball with a fresh copy, either out of the latest KDE release or from SVN. To do so, you just have to add a tarball containing the new directory as sole content to the   variable,   takes care of replacing it.

Possibly undefined macros
The most common failure with autotools is related to the  message "possibly undefined macro: SOME_MACRO". This message is issued when a macro is called from the or  file but it's not defined in the  file created by.

This happens often because the mentioned macro is not available when  is run; as it, by default, loads macros found in , this means that the package providing that macro is not installed (or the macro is called with the wrong name). As the second case is either trivial or complex to resolve, we'll focus on the first name, a missing macro definition.

Macros written by developers for their software to be detected in the system by use of autotools are usually written in m4 files that are installed in the aforementioned directory. As many packages uses those macros for optional dependencies, there might be need for an m4 file that is not installed in the system where autotools are being run; to solve this problem, it's possible to copy the m4 file in a subdirectory shipped with the package itself.

Unfortunately, to make use of this subdirectory, often called,   has to be told about it. In projects using  is possible to set that inside the main  file by setting the ACLOCAL_AMFLAGS variable:

example of telling aclocal to search for macro files in 'm4' directory

This often is overlooked by upstream developers that simply pass the  parameter to aclocal while building their package. As adding a patch to fix this problem is an overkill, it's simpler, if the package has a directory with the needed m4 files, to set it in AT_M4DIR variable. The same goes if the package is not using  but just.

telling eautoreconf to search for macro files in 'm4' directory

In the rare case the software is using a Cygnus-style sub-configure build system, the above example might fail, as it tries to find a m4 subdirectory from where the configure is; to solve this kind of problems, set it to instead.

Less often, but still happens, there are no directories with m4 files, or the files with the undefined macro isn't there; to solve this issue, you have to search for which package provides the m4, and then add it to that directory, either with a patch or by putting it on mirrors and then adding it to SRC_URI (in which case you have to add ${WORKDIR} to the list of directories to search or move it in the right directory. This kind of issue is one of the most annoying one, so it's usually better to inform as soon as possible upstream so that following releases wouldn't require such hacks.

automake version mismatch when running eautoreconf
Sometimes when running , it fails reporting a version mismatch error. You'd expect not to see this ever, as the  function will take care of re-running all the autotools when the   version used to build the package differs from the one used by the ebuild; additionally, during  , the tools are being ran to force replacing files, so the references to the   used by the original packager should be gone at that point.

The only (or at least most likely) cause of this is a bad knowledge of autotools by the developer of the ebuild. When faced with the problem described above of Possibly undefined macros, the developer might feel compelled to just copy the previous file from the original tarball to a different name, to preserve the macro definitions there. Unfortunately this overrides the  macros, causing this almost incomprehensible failure.

automake fails, requires missing files
Another common error, this time with  is a failure because of missing files, such as  or. This is because all the autotools assume, if nobody say otherwise to them, that they are working on a GNU package, having a series of files because of the code style guide from GNU itself, and fails when those files are missing. In those cases  should be called with the   parameter, that tells it to not fail if the GNU-required files are missing.

Again,  function tries to make simpler autotools rebuilding by checking if some of the GNU needed files are present, and then calling   with the right option if not. The right way to solve this issue (to send upstream) is to add to the AUTOMAKE_OPTIONS variable the option foreign so that it will know to use foreign support out of the box.

When the files are requested by or  instead of , and are usually the two files  and  , the problem is that the package is not properly bootstrapped. To fix this,  should be called with the options. This is what the  function already do, so if you find this problem, you should consider using the functions provided by  eclass instead of running the tools manually or with the eventual script provided with the package itself.

Missing dependencies' version
Since about Summer 2006,  and   wrappers don't depend on all the versions of the respective packages forcefully, which means you cannot rely on the users to have all the versions installed, and the dependencies has to be fixed according to the used packages. To simplify the dependency mangling and the enforcing of the needed versions, the variables WANT_AUTOCONF and WANT_AUTOMAKE are considered inputs to the eclass that will then handle both dependencies and enforcement.

depending on autoconf 2.1 and automake 1.4

In many cases, instead of depending on a given version of automake or autoconf, we want to depend on the latest version available, more likely to be already in users' systems. For this reason, the autotools eclass will accept a special value for the mentioned variables, latest, that will then be expanded to  2.5 and   either 1.9 or 1.10 depending on what is unmasked for that given system. This is suggested when the package does not depend on some features or misbehaviour of an older version of them.

depending on the latest versions of autotools

Errors in build phase about autoconf version
Sometimes you can get errors during the build of a package, related to the  version, even though you didn't rebuild autotools files, or actaully because you didn't rebuild autotools files.

common error in build phase

This message comes from the code added by the so-called "maintainer mode" provided by. This mode is mostly intended to make sure that developers, and users building manually, get the correct version of and  even if they were modified after running   to package the sources.

While maintainer mode is quite important for both developers and users building manually, it comes a bit in the way for ebuilds, as it will automatically run autotools if you patch the or  files, even when autotools are not in the build-time dependencies on the ebuild.

Even worse, if the  version used by the package is not installed (for instance if the package uses the old 1.8 version, while the user only has the last 1.10 version, it will skip the rebuild entirely, making the actual result non-deterministic from the ebuild point of view.

The error above is caused by a package that had one of its files modified, usually by a patch, without rebuilding the autotools. In these cases,  will be invoked to just rebuild the involved , but it will work only if the   version in the system is the same as the one used to create the original  script. This is not the case once a new  version is released.

This is solved by rebuilding autotools properly as described above, through the  function (or other method depending on eventual higher level eclasses), instead of leaving maintainer mode to take care of it.

Failure during configure on some locales (like et_EE)
Some packages, using  2.13, fail to configure on some systems with locales like et_EE. This is caused by a bug in that version of autoconf (fixed in following versions, which are not backward compatible), where sed is trying to use local-dependent syntax before the LANG variable is reset to use C locale (making it locale-independent).

This can be seen for instance in bug #103483.

To fix these cases, it's possible to apply configure-LANG.patch, that moves the LANG reset before first use of locale-dependent syntax.

If possible, it's anyway suggested to try using newer versions of  (2.59 or later) where the issue is fixed already. Unfortunately this is not feasible for all packages, so applying the patch is a nice way to fix the issue when  2.1 is actually needed.

Acknowledgements
We would like to thank the following authors and editors for their contributions to this guide:


 * Diego Pettenò