Clang
sys-devel/clang is a "LLVM native" C/C++/Objective-C compiler using LLVM as a backend and optimizer. It aims to be GCC compatible yet stricter, offers fast compile times with low memory usage, and has useful error and warning messages for easier compile troubleshooting.
Contents
Before getting started with Clang
One of the goals of the Clang project is to be compatible with GCC. Occasionally some packages will fail to build correctly with it and some may build successfully but segfault when executed. Some packages also have GCC specific code and will also fail during compiling. In these events, we need to use GCC as a fallback.
Setting up GCC fallback environments
Let's create an environment variable to override any packages for these case scenarios. The name used below is just an example, so feel free to choose whatever you want when naming the environment you create. Be sure to substitute the name you chose with the examples in the article.
/etc/portage/env/compiler-gccEnvironment named compiler-gccCC="gcc" CXX="g++"
The above is the most basic environmental variable needed. You can change it to suit your needs, such as enabling/disabling link-time optimizations, alternative AR, NM, RANLIB, and so on. Here are two examples below:
/etc/portage/env/compiler-gcc-fltoEnvironment variable named compiler-gcc-fltoCC="gcc" CXX="g++" CFLAGS="-flto -march=native -O2 -pipe" CXXFLAGS="${CFLAGS}" AR="ar" NM="nm" RANLIB="ranlib"
/etc/portage/env/compiler-gcc-fno-ltoEnvironment variable named compiler-gcc-fno-ltoCC="gcc" CXX="g++" CFLAGS="-fno-lto -march=native -O2 -pipe" CXXFLAGS="${CFLAGS}" AR="ar" NM="nm" RANLIB="ranlib"
Basically, copy over your current working GCC config from your make.conf in the event we need to use it as a fallback. If you choose to use LLVM's implementation of AR, NM, and RANLIB as detailed later in the article, be sure to set them back to the GNU versions for your GCC fallback environments as shown in the above example. If you choose not to, you can ignore the AR, NM, and RANLIB variables. If you want to use link-time optimization it's a good idea to have two separate environments like the above examples.
In the event you have to use the GCC fallback environment(s) set the appropriate flags in /etc/portage/package.env.
/etc/portage/package.envFalling back to GCC for app-foo/bar and app-bar/bazapp-foo/bar compiler-gcc-flto #compiled using GCC with link-time optimization since package bar compiles using flto app-bar/baz compiler-gcc-fno-lto #compiled using GGC with no link-time optimization since package baz fails using flto
Alternatively, if you have app-portage/flaggie installed, the /etc/portage/package.env file could be modified using:
root #flaggie app-foo/bar app-bar/baz +compiler-gccTesting against GCC
If you'd like to build your system with clang, the first good thing to do would be enabling tests via FEATURES=test. Note that many developers don't actually care about their own tests, and as a result you are likely to get many false positives. Because of this, you may need to compare the results with a GCC build.
Getting started with Clang
Now that we've set up a safe fallback we can proceed to enable the usage of Clang in Gentoo. There are two ways to do this: System wide using /etc/portage/make.conf or via environmental variables like the one(s) we created for the GCC fallback.
Setting up Clang environments
We'll use the same process as we did earlier in the article for setting up GCC fallbacks.
/etc/portage/env/compiler-clangEnvironment variable named compiler-clangCC="clang" CXX="clang++"
You can now use Clang on a per package basis by invoking the compiler-clang environmental variable you created.
/etc/portage/package.envUsing the Clang compiler for app-foo/bar and app-bar/bazapp-foo/bar compiler-clang app-bar/baz compiler-clang
Of course, that's about as basic as it gets. You can create more complex environments if you wish. It's a good idea to have a variable for link-time optimized Clang and one without if you want to take advantage of LTO as detailed later in the article. Simply create two environments, one with the flag -flto and one with -fno-lto and apply each to packages as needed.
/etc/portage/env/compiler-clang-fltoEnvironment variable named compiler-clang-fltoCC="clang" CXX="clang++" CFLAGS="${CFLAGS} -flto" CXXFLAGS="${CXXFLAGS} -flto" LDFLAGS="-Wl,-O2 -Wl,--as-needed" #please use whichever optimization level you're comfortable with
/etc/portage/env/compiler-clang-fno-ltoEnvironment variable named compiler-clang-fno-ltoCC="clang" CXX="clang++" CFLAGS="${CFLAGS} -fno-lto" CXXFLAGS="${CXXFLAGS} -fno-lto"
You can now use Clang with or without link-time optimization on a per package basis by invoking the proper environmental you created. Please see "Using link-time optimizations with Clang" for a detailed guide.
/etc/portage/package.envUsing the Clang compiler for app-foo/bar and app-bar/bazapp-foo/bar compiler-clang-flto #compiled using Clang with link-time optimization since package bar compiles using flto app-bar/baz compiler-clang-fno-lto #compiled using Clang with no link-time optimization since package baz fails using flto
Using Clang system wide via make.conf
If you choose to use Clang system wide you absolutely must have a GCC fallback! This can't be stressed enough as you will not be able to compile everything using Clang at the moment, such as the GCC compiler. Setting up Gentoo to use Clang system wide is simple. Change the CC and CXX variables in /etc/portage/make.conf to Clang's. You don't need to do anything further if this is all you want to do.
/etc/portage/make.confSetting the system compiler to ClangCC="clang" CXX="clang++"
If you need to compile a package using GCC as a fallback, use one of the environments we created earlier.
Using link-time optimizations with Clang
The link-time optimization feature defers optimizing the resulting executables to linking phase. This can result in better optimization of packages but isn't standard behavior in Gentoo yet. In order to use LTO we must first install LLVM with the Gold linker plugin so Clang can pass the -plugin flag to the linker.
Installing and enabling the Gold linker
Set the gold flag in the LLVM package so portage knows to include the Gold plugin.
/etc/portage/package.usesys-devel/llvm gold
Now emerge LLVM and check the use flags to make sure your changes went into effect.
root #emerge -1vN sys-devel/llvm
These are the packages that would be merged:
Calculating dependencies... done!
[ebuild U ] sys-devel/llvm-3.8.0-r3:0/3.8.0::gentoo [3.8.0-r2:0/3.8.0::gentoo] USE="clang gold libffi python static-analyzer xml -debug -doc -libedit -lldb -multitarget -ncurses -ocaml {-test}" ABI_X86="32 (64) (-x32)" PYTHON_TARGETS="python2_7" VIDEO_CARDS="-radeon" 0 KiB
Total: 1 package (1 upgrade), Size of downloads: 0 KiB
Would you like to merge these packages? [Yes/No]
We must set Gold as the default linker since it allows clang to pass the -plugin flag. By default, Gentoo uses the bfd linker. As of July 2016, the bfd linker bundled with binutils does not support passing the -plugin as seen below.
root #/usr/bin/ld -plugin/usr/bin/ld: unrecognized option '-plugin' /usr/bin/ld: use the --help option for usage information
Using the sys-devel/binutils-config package, we'll set ld to default to ld.gold instead of ld.bfd.
root #binutils-config --linker ld.gold* Setting default linker to ld.gold for x86_64-pc-linux-gnu-2.25.1 ...
Once ld is set to Gold, check the -plugin flag to see if it works.
root #/usr/bin/ld -plugin/usr/bin/ld: -plugin: missing argument /usr/bin/ld: use the --help option for usage information
Great! It works. If you want to switch back to the default linker change the command from ld.gold to ld.bfd.
root #binutils-config --linker ld.bfd* Setting default linker to ld.bfd for x86_64-pc-linux-gnu-2.25.1 ...
Using LTO in Clang environments
Now that we're done setting up the Gold linker, we can create a new environment for LTO enabled clang. We covered this a little bit earlier but we'll go more into depth here. We enable LTO by passing the -flto flag to clang via the CFLAGS variable which in turn passes the -plugin flag to the linker.[1] You need to set optimization flags via LDFLAGS which Clang and Gold use when linking because that's when code optimization occurs. Linker flags should have -Wl, preceding them since they're being invoked by the compiler and not run manually[2].
/etc/portage/env/compiler-clang-fltoEnvironment named compiler-clang-fltoCC="clang" CXX="clang++" CFLAGS="${CFLAGS} -flto" CXXFLAGS="${CXXFLAGS} -flto" LDFLAGS="-Wl,-O2 -Wl,--as-needed" #please use whichever optimization level you're comfortable with
As an alternative, LLVM provides their own ar, nm, and ranlib. You're free to use them and may or may not get more mileage over using the standard ar, nm, and ranlib since they're intended to handle LLVM bitcode which Clang produces when using the -flto flag.
/etc/portage/env/compiler-clang-fltoEnvironment named compiler-clang-fltoCC="clang" CXX="clang++" CFLAGS="${CFLAGS} -flto" CXXFLAGS="${CXXFLAGS} -flto" LDFLAGS="-Wl,-O2 -Wl,--as-needed" #please use whichever optimization level you're comfortable with AR="llvm-ar" NM="llvm-nm" RANLIB="llvm-ranlib"
Now you can set /etc/portage/package.env overrides using Clang with LTO enabled.
/etc/portage/package.envEnabling LTO for app-foo/bar and app-bar/bazapp-foo/bar compiler-clang-flto app-bar/baz compiler-clang-flto
Using LTO system wide
Similar to what we covered earlier in the article, we can do a system wide Clang with LTO enabled setup by changing our /etc/portage/make.conf file.
/etc/portage/make.confSetting the system compiler to ClangCC="clang" CXX="clang++" CFLAGS="${CFLAGS} -flto" CXXFLAGS="${CXXFLAGS} -flto" LDFLAGS="-Wl,-O2 -Wl,--as-needed" #please use whichever optimization level you're comfortable with AR="llvm-ar" NM="llvm-nm" RANLIB="llvm-ranlib"
Again, it's up to you if you want to set the AR, NM, and RANLIB to the LLVM implementations. Since earlier in the article we set up compiler environments using Clang without LTO, GCC without LTO, and GCC with LTO, we can pick and choose which is best on a per package basis. Since the goal is to compile packages system wide with Clang using LTO and not every package will successfully compile using it, we'll have to fall back to Clang with LTO disabled or GCC. Your /etc/portage/package.env may look like this:
/etc/portage/package.envExample package.env setupapp-foo/bar compiler-clang-fno-lto #compiled using Clang with no link-time optimization since package bar fails using flto app-bar/baz compiler-gcc-fno-lto #compiled using GCC with no link-time optimization since package bar fails using flto app-baz/foo compiler-gcc-flto #compiled using GCC with link-time optimization since package foo compiles using flto
Using ThinLTO starting with Clang 3.9
As of Clang 3.9, a new feature called ThinLTO has been introduced. In a nutshell, ThinLTO gives you the performance of full LTO, but with the compile times of no LTO.[3] To invoke the command, simply change -flto (which defaults to full when invoked) to -flto=thin in your configuration files.
The Gold linker is still required for this so make sure you've followed the steps above to set it as the default linker.
/etc/portage/make.confEnabling Clang's ThinLTO featureCC="clang" CXX="clang++" CFLAGS="${CFLAGS} -flto=thin" CXXFLAGS="${CXXFLAGS} -flto=thin" LDFLAGS="-Wl,-O2 -Wl,--as-needed" #please use whichever optimization level you're comfortable with
Using Clang with distcc
In order to use clang on a distcc client, additional symlinks have to be created in /usr/lib*/distcc/bin:
root #ln -s /usr/bin/distcc /usr/lib/distcc/bin/clangroot #ln -s /usr/bin/distcc /usr/lib/distcc/bin/clang++
Using Clang with ccache
Automatic with `>=ccache-3.9-r3` when Clang is emerged.
Troubleshooting
The main place for looking up known failures with clang is bug #408963. If you hit one not reported on our Bugzilla already, please open a new bug report and make it block 408963.
Compile errors when using Clang with -flto
If the packages you're installing are failing, check your logs. Often times packages with errors like the following will need to disable LTO by invoking the compiler-clang-fno-lto environment.
/var/log/portage/sys-apps:less-483-r1:20160712-034715.log/usr/bin/x86_64-pc-linux-gnu-ld: error: version.o:1:3: invalid character /usr/bin/x86_64-pc-linux-gnu-ld: error: version.o:1:3: syntax error, unexpected $end /usr/bin/x86_64-pc-linux-gnu-ld: error: version.o: not an object or archive
You will also most likely see this error in every LTO failure case.
/var/log/portage/sys-apps:less-483-r1:20160712-034715.logx86_64-pc-linux-gnu-clang-3.8: error: linker command failed with exit code 1 (use -v to see invocation)
Simply add the failing package to your /etc/portage/package.env. In this case it's sys-apps/less, so we'll apply the proper override.
/etc/portage/package.envExample package.env setupsys-apps/less compiler-clang-fno-lto #compiled using Clang with no link-time optimization since package less fails using flto
Sometimes a package will fail to compile even when disabling LTO because it requires another package which was compiled using -flto and works incorrectly. You may see an error like this:
/var/log/portage/dev-libs:boehm-gc-7.4.2:20160713-085706.log/usr/lib64/libatomic_ops.a: error adding symbols: Archive has no index; run ranlib to add one
In this case libatomic_ops is causing boehm-gc to fail compiling. Recompile the program causing the failure using your -fno-lto environment and then recompile the new program. In this case, boehm-gc fails when using LTO, so we'll add both of them to our /etc/portage/package.env file and assign the environments using the -fno-lto flags.
/etc/portage/package.envExample package.env setupdev-libs/boehm-gc compiler-clang-fno-lto dev-libs/libatomic_ops compiler-clang-fno-lto
Use of GNU extensions without proper -std=
Some packages tend to use GNU extensions in their code without specifying -std= appropriately. GCC allows that usage, yet Clang disables some of more specific GNU extensions by default.
If a particular package relies on such extensions being available, you will need to append the correct -std= flag to it:
- -std=gnu89 for C89/C90 with GNU extensions,
- -std=gnu99 for C99 with GNU extensions,
- -std=gnu++98 for C++:1998 with GNU extensions.
A common symptom of this problem are multiple definitions of inline functions like this:
/var/log/portage/Example package error in example log/usr/bin/x86_64-pc-linux-gnu-ld: error: ../mpi/.libs/libmpi.a(mpi-bit.o): multiple definition of '_gcry_mpih_add' /usr/bin/x86_64-pc-linux-gnu-ld: ../mpi/.libs/libmpi.a(mpi-add.o): previous definition here /usr/bin/x86_64-pc-linux-gnu-ld: error: ../mpi/.libs/libmpi.a(mpi-bit.o): multiple definition of '_gcry_mpih_add_1' /usr/bin/x86_64-pc-linux-gnu-ld: ../mpi/.libs/libmpi.a(mpi-add.o): previous definition here