Distcc/Cross-Compiling

From Gentoo Wiki
Jump to: navigation, search
This page contains changes which are not marked for translation.

Other languages:
English • ‎español • ‎français • ‎日本語 • ‎한국어 • ‎русский • ‎中文(中国大陆)‎

This guide shows you how to set up distcc for cross-compiling across different processor architectures.

Cross-compiling with distcc

Introduction

distcc is a tool that lets you share the burden of software compiling across several networked computers. As long as the networked boxes are all using the same toolchain built for the same processor architecture, no special distcc setup is required. But what do you do if you need to compile for a different architecture using differing computers? This guide will show you how to configure distcc to compile for different architectures.

Emerge the needed utilities

First, you will need to emerge crossdev on all the machines that will be involved in the compiling process. crossdev is a tool that makes building cross-architecture toolchains easy. It was originally written by Joshua Kinard and was re-written from the ground up by Mike Frysinger. Its usage is straightforward: crossdev -t sparc will build a full cross-toolchain targetting the Sparc architecture. This includes binutils, gcc, glibc, and linux-headers. If you need more help, try running crossdev --help. Obviously, you will need to emerge the proper cross-toolchain on all the helper boxes.

If you want to fine tune the cross-toolchain, here is a script that will produce a command line with the exact versions of the cross development packages to be built on the helper boxes (the script is to be run on the target box).

CODE Script to fine-tune cross development tools
#! /bin/bash
A="sys-devel/binutils" ; B=`equery l $A` ; BINUTILS_VER=`echo $B | cut -d- -f3-`
A=`/usr/bin/gcc-config -c` ; B=`echo $A | cut -d- -f5` ; GCC_VER=`equery l sys-devel/gcc | grep $B | cut -d- -f3-`
KERNEL_VER=`uname -r | sed s/-gentoo//`
A="sys-libs/glibc" ; B=`equery l $A` ; LIBC_VER=`echo $B | cut -d- -f3-`
echo "crossdev --b =$BINUTILS_VER --g =$GCC_VER --k =$KERNEL_VER --l =$LIBC_VER -t `uname -m`"

Next, you will need to emerge distcc on all the machines that will be involved in the process. This includes the box that will run emerge and the boxes with the cross-compilers. Please see the Gentoo Distcc Documentation for more information on setting up and using distcc.

Note
Current versions of crossdev have a --s/--stable flag for installing only stable versions of compiler tools. (ie. crossdev -t i686-pc-linux-gnu --stable --ex-gcc --ex-gdb --portage --pretend) Else, crossdev installs the latest experimental compiler tools packages! The above script is no longer really needed, else you have unmasked specific versions of package tools and/or headers.

Arch-specific notes

Note
Obtain architectures names by looking at your compile target's CHOST variable within /etc/make.conf. If you mangle the architecture name for the "crossdev -t" option, crossdev will merrily guess and install tools using your mangled architecture name for folder names within /usr! (ie. /usr/i686-pc-linux-gnu, /usr/i686-linux-gnu, ...) If you do mangle the names, you'll need to specify each mangled architecture/folder name to crossdev --clean for uninstalling, else just manually use rm or your remove command. Obviously, this isn't tidy!

Intel x86 subarchitectures

If you are cross-compiling between different subarchitectures for Intel x86 (e.g. i586 and i686), you must still build a full cross-toolchain for the desired CHOST, or else the compilation will fail. This is because i586 and i686 are actually different CHOSTs, despite the fact that they are both considered "x86." Please keep this in mind when you build your cross-toolchains. For example, if the target box is i586, this means that you must build i586 cross-toolchains on your i686 helper boxes.

SPARC

Using crossdev -t sparc might fail with one of the following errors:

CODE Errors displayed when running crossdev -t sparc
linker with -z relro support required
support for the tls_model attribute is required
this configuration requires -mlong-double-128 support

If this happens, try using the following command instead:

user $crossdev --lenv "CC=sparc-unknown-linux-gnu-gcc" -t sparc-unknown-linux-gnu

Configuring distcc to cross-compile correctly

In the default distcc setup, cross-compiling will not work properly. The problem is that many builds just call gcc instead of the full compiler name (e.g. sparc-unknown-linux-gnu-gcc). When this compile gets distributed to a distcc helper box, the native compiler gets called instead of your shiny new cross-compiler.

Fortunately, there is a workaround for this little problem. All it takes is a wrapper script and a few symlinks on the box that will be running emerge. We'll use a Sparc box as an example. Wherever you see sparc-unknown-linux-gnu below, you will want to insert your own CHOST (x86_64-pc-linux-gnu for an AMD64 box, for example). When you first emerge distcc, the /usr/lib/distcc/bin directory looks like this:

Note
The following instructions are to be performed only on the box running the emerge. Do not perform these steps on the helper boxes.
root #cd /usr/lib/distcc/bin
root #ls -l
total 0
lrwxrwxrwx  1 root root 15 Dec 23 20:13 c++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 cc -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 g++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 gcc -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 sparc-unknown-linux-gnu-c++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 sparc-unknown-linux-gnu-g++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 sparc-unknown-linux-gnu-gcc -> /usr/bin/distcc

Here is what you want to do:

root #rm c++ g++ gcc cc

Next, we'll create the new script on this box. Fire up your favorite editor and create a file with the following text in it, then save it as sparc-unknown-linux-gnu-wrapper. Remember to change the CHOST (in this case, sparc-unknown-linux-gnu) to the actual CHOST of the box that will be running the emerge.

CODE The new wrapper script
#!/bin/bash
exec /usr/lib/distcc/bin/sparc-unknown-linux-gnu-g${0:$[-2]} "$@"

Next, we'll make the script executable and create the proper symlinks:

root #chmod a+x sparc-unknown-linux-gnu-wrapper
root #ln -s sparc-unknown-linux-gnu-wrapper cc
root #ln -s sparc-unknown-linux-gnu-wrapper gcc
root #ln -s sparc-unknown-linux-gnu-wrapper g++
root #ln -s sparc-unknown-linux-gnu-wrapper c++

When you're done, /usr/lib/distcc/bin will look like this:

root #ls -l
total 4
lrwxrwxrwx  1 root root 25 Jan 18 14:20 c++ -> sparc-unknown-linux-gnu-wrapper
lrwxrwxrwx  1 root root 25 Jan 18 14:20 cc -> sparc-unknown-linux-gnu-wrapper
lrwxrwxrwx  1 root root 25 Jan 18 14:20 g++ -> sparc-unknown-linux-gnu-wrapper
lrwxrwxrwx  1 root root 25 Jan 18 14:20 gcc -> sparc-unknown-linux-gnu-wrapper
lrwxrwxrwx  1 root root 15 Nov 21 10:42 sparc-unknown-linux-gnu-c++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Nov 21 10:42 sparc-unknown-linux-gnu-g++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Jul 27 10:52 sparc-unknown-linux-gnu-gcc -> /usr/bin/distcc
-rwxr-xr-x  1 root root 70 Jan 18 14:20 sparc-unknown-linux-gnu-wrapper

Next we want to make sure that these wrappers stay available after upgrading the distcc package as it will overwrite the symbolic links. We can do this through a /etc/portage/bashrc file that looks like so:

FILE /etc/portage/bashrc
case ${CATEGORY}/${PN} in
	sys-devel/distcc)
		# Hey man, how come that CONFIG PROTECT don't work?
		if [ "${EBUILD_PHASE}" == "postinst" ] || [ "${EBUILD_PHASE}" == "postrm" ];
		then
			cd /usr/lib/distcc/bin
			rm cc c++ gcc g++
			ln -s sparc-unknown-linux-gnu-wrapper cc
			ln -s sparc-unknown-linux-gnu-wrapper c++
			ln -s sparc-unknown-linux-gnu-wrapper gcc
			ln -s sparc-unknown-linux-gnu-wrapper g++
		fi
	;;
esac

Congratulations; you (hopefully) now have a working cross-distcc setup.

How this works

When distcc is called, it checks to see what it was called as (e.g. i686-pc-linux-gnu-gcc, sparc-unknown-linux-gnu-g++, etc.) When distcc then distributes the compile to a helper box, it passes along the name it was called as. The distcc daemon on the other helper box then looks for a binary with that same name. If it sees just gcc, it will look for gcc, which is likely to be the native compiler on the helper box, if it is not the same architecture as the box running emerge. When the full name of the compiler is sent (e.g. sparc-unknown-linux-gnu-gcc), there is no confusion.

Troubleshooting

Remote Host distccd COMPILE ERRORS

If you receive COMPILE ERRORS within a remote host's /var/log/distccd.log file, see the above notes concerning specifying the correct architecture name. (ie. crossdev -t $TARGET)

Another solution is to uninstall and re-install crossdev compiler tools, using the "crossdev --clean" option, or ensuring /usr/$TARGET no longer exists, and then completely reinstall the cross compiler.

(Might also be wise to edit the remote host's /usr/$TARGET/etc/portage/make.conf, and ensure the contents of the CFLAG variable are similar on all computers or hosts performing compiler operations?)

Failed to exec $TARGET-uknown-linux-gnu-gcc: No such file or directory

Problem: For some reason the above wrapper scripts fail to execute, even with correct permissions. (ie. distcc[6195] (dcc_execvp) ERROR: failed to exec i686-unknown-linux-gnu-gcc: No such file or directory)

Solution: Make sure you have created the wrapper script with the complete name of your architecture target! (ie. ls -alh /usr/lib/distcc/bin/c++ ->./i686-pc-linux-gnu-wrapper)



This article is based on a document formerly found on our main website gentoo.org.
The following people have contributed to the original document: Andrew Gaffney, Joshua Saddler
They are listed here as the Wiki history does not provide for any attribution. If you edit the Wiki article, please do not add yourself here, your contributions are recorded on the history page.