Prefix/libc

From Gentoo Wiki
< Prefix
Jump to: navigation, search

This is a project to support libc inside a Prefix, code named RAP (Rap Ain't Prefix). See also Project:Android. A general use case is for Prefix on RHEL 5 (CentOS 5 ans SL 5), where the host glibc-2.5 is too old to support modern features such as fortify (bug #289757). Practically, one can install Gentoo Linux in some location, for example in /scratch/gentoo on SGI Irix. This relocated installation path is stored in the ${EPREFIX} variable. All files are installed in ${EPREFIX}/{bin,etc,lib,tmp,usr,var}, etc.

Classical Prefix is based on rpath but it does not give you the freedom of having a separate sys-libs/glibc from the host. Many packages in Gentoo Linux will not compile against archaic glibc, e.g. =sys-libs/glibc-2.13 and many utilities need at least =>sys-libs/glibc-2.17. Also binary packages like Oracle java-1.8 JRE/SDK need a more recent glibc. Java IcedTea-3.2.0 needs 2.14.

So then there is a Prefix/libc (sysroot method, called Gentoo::RAP).

Finally, at some point there will be another implementation of Prefix/libc approach (native paths method, experimental). It is developed by @redlizard. In the future it will be merged with Prefix/libc (sysroot method, aka Gentoo::RAP).

Gentoo::RAP uses the main Gentoo x86 portage tree so you get more recent packages. However, there is usually a small set of altered ebuilds, patches and eclasses in a RAP overlay. This overlay contains packages that are needed, or modifications that aren't yet finalized or merged with the main gentoo portage tree. Gentoo::RAP tries to override only a few classes from the bare Gentoo code base and simply relies on the latest and greatest portage tree. Of note, the bash package has some overrides, time to time pushed into the main gentoo-x86 repo. The Gentoo::RAP overlay may have some issues in ebuilds and eclasses which have been discovered and fixed in the Gentoo Prefix repo.

The bootstrap scripts will fetch quite a lot of tarball data, the portage tree, and package sources. There is no easy way to pre-fetch all the stuff needed beforehand, and be off-line during compilation. Watch (bug #586152) for improvements.

Bug reports in Gentoo::Prefix have a separate category at https://bugs.gentoo.org while Gentoo::RAP issues can go along all other bug report tagged as "Current packages". Current issues with mostly ebuilds not respecting $EPREFIX variable can be seen here: https://bugs.gentoo.org/buglist.cgi?quicksearch=EPREFIX&list_id=3399810 . Some might be already fixed in the Gentoo::Prefix tree.

bootstrap-rap.sh will nicely copy /etc/{passwd,hosts} from the host system setup. The files maybe get a bit outdated over the time as new users are added to the host's passwd/group files so the section 'Username_becomes_invalid_inside_RAP' below may be helpful for recreating the files using current info. bootstrap-prefix.sh does not need to copy user/group info into Gentoo::Prefix installation because it does not have its own glibc.

Other users on the same host can use the installed Gentoo::RAP or Gentoo::Prefix, but they cannot use eselect or gcc-config, etc. to switch between versions of the many tools. However they can call gcc-4.1.2 or python-3.4 directly. Only the user who has write permissions to the Gentoo::RAP or Gentoo::Prefix installation can adjust the 'system-wide' defaults.

The differences between the three approaches are summarized in this table.

Prefix-rpath (classic) Prefix/libc (sysroot method) Prefix/libc (native paths method, experimental)
Feature name prefix-rpath prefix-standalone
Linux profile prefix/linux prefix/linux-standalone
Linux keyword ~ARCH-linux ARCH ~ARCH
Portage prefix.git gentoo.git overlaid with android.git
sys-devel/gcc EXTRA_ECONF+=(--with-sysroot=$EPREFIX) but do not pass --sysroot to ld, prefix dynamic loader change NATIVE_SYSTEM_HEADER_DIR, STANDARD_STARTFILE_PREFIX, prefix dynamic loader
sys-devel/binutils prefixify NATIVE_LIB_DIRS --with-lib-path=${EPREFIX}/usr/lib:${EPREFIX}/lib
sys-devel/binutils-config inject -rpath and -R
sys-libs/glibc No Yes prefixify hardcoded /etc,/usr,/var,/bin Yes
Bootstrap bootstrap-prefix.sh bootstrap-rap.sh

Installation

Download Gentoo RAP bootstrap-rap.sh. You can use various ways to obtain the script and get it to a place where you can execute it. Once in position, preform the following commands:

user $chmod 755 bootstrap-rap.sh
user $ ./bootstrap-rap.sh
(follow the instructions)

That's all! The script will guide you through the full process, and tell you how to start your freshly bootstrapped Gentoo RAP system assuming it successfully completes. In normal cases, you don't need any more than just this.

Manual Installation

RAP can also be installed manually as:

user $export EPREFIX="$HOME/gentoo" # replace with your EPREFIX
user $export PATH="$EPREFIX/usr/bin:$EPREFIX/bin:$EPREFIX/tmp/usr/bin:$EPREFIX/tmp/bin:/usr/bin:/bin"
user $chmod 755 bootstrap-rap.sh
user $./bootstrap-rap.sh $EPREFIX stage1
user $./bootstrap-rap.sh $EPREFIX stage2
user $./bootstrap-rap.sh $EPREFIX stage3
user $hash -r
user $emerge -e system
user $mkdir -p "${EPREFIX}"/usr/portage/scripts
user $wget https://gitweb.gentoo.org/repo/proj/prefix.git/plain/scripts/startprefix.in -O "${EPREFIX}"/usr/portage/scripts/startprefix.in
user $./bootstrap-rap.sh $EPREFIX startscript

Refer to Prefix/Manual_Bootstrap for more details.

Tested distributions

At the moment, RAP supports Linux distributions only. The following tables tracks the tested distributions with the script. Feel free to add your own.

Distribution Version ARCH linux glibc gcc binutils Tester Works till ARCH of RAP Last tried
CentOS 7.2 amd64 4.3.0 2.17 4.8.5 2.23.52 heroxbd the end amd64 20160608
CentOS 6.8 amd64 2.6.32 2.12 4.4.7 2.20.51 heroxbd the end amd64 20160604
CentOS 6.8 amd64 2.6.32 2.12 4.4.6 2.20.51 xdej the end amd64 20160606
CentOS 5.6 x86 2.6.18 2.5 4.1.2 2.17.50 xdej the end x86 20160606
Debian sid amd64 4.3.0 2.22 5.3.1 2.26 heroxbd the end amd64 20160603
Debian sid amd64 4.7.0 2.24 6.3.0 2.28 heroxbd the end amd64 20170515
Debian 6.0 amd64 2.6.32 2.11.3 4.4.5 2.20.1 R0b0t1 the end amd64 20170910
Debian 7.10 amd64 3.16.0 2.13 4.7.2 2.22 heroxbd the end amd64 20160615
Debian 7.10 amd64 3.16.0 2.13 4.7.2 2.22 heroxbd the end x86 20160608
Debian 7.11 x86 3.16.0 2.13 4.7.2 2.22 heroxbd the end x86 20160608
Debian 8.4 amd64 3.16.0 2.19 4.9.2 2.25 heroxbd the end amd64 20160607
Devuan 1.0 beta amd64 3.16.0 2.19 4.9.2 2.25 lightbringer the end amd64 20160622
Fedora 10 x86 2.6.27 2.9 4.3.2 2.18.50 heroxbd the end x86 20160604
Gentoo ~ amd64 4.3.0 2.20 5.3.0 2.26 heroxbd the end amd64 20160605
Gentoo 13.0 amd64 3.8.13 2.15 4.6.3 2.23.1 heroxbd the end amd64 20160608
RHEL 7.2 amd64 3.10.0 2.17 4.8.5 2.23.52 heroxbd the end amd64 20160720
RHEL 6.4 amd64 2.6.32 2.12 4.4.7 2.20.51 heroxbd the end amd64 20160615
RHEL 5.10 amd64 2.6.18 2.5 4.1.2 2.17.50 heroxbd the end amd64 20160615
Ubuntu 14.04 amd64 3.13.0 2.19 4.8.4 2.24 heroxbd the end amd64 20170823
Ubuntu 16.04 amd64 4.4.0 2.23 5.3.1 2.26 prefixer the end amd64 20160616
Arch 2016.06.01 amd64 4.5.4 2.23 6.1.1 2.26.0 yangsheng6810 the end amd64 20160615
openSUSE 42.1 amd64 3.16.36 2.19 4.8.5 2.26.1 heroxbd the end amd64 20161116
Slackware current x86_64 4.9.31 2.25 7.1.0 2.28 lightbringer the end amd64 20170628

Tips

Compile inside memory

On a cluster node with large memory and shared network filesystem, compiling in a memory tmpfs can be significantly faster. /dev/shm is mounted as tmpfs by many distributions. For example:

FILE ${EPREFIX}/etc/portage/make.conf
PORTAGE_TMPDIR=/dev/shm

Before calling bootstrap-rap.sh, PORTAGE_TMPDIR can be changed with:

user $export PORTAGE_TMPDIR=/dev/shm

On systemd/RedHat systems tmpfs (shm) is mounted by default on /tmp, /dev/shm doesn't exist any more there (at least when last checked with CO7/OL7).

Add an en_US.UTF-8 locale

Gentoo::RAP uses it's own libc so the locales should be generated in the Prefix. Add an entry to EPREFIX/etc/locale.gen

FILE ${EPREFIX}/etc/locale.gen
en_US.UTF-8 UTF-8

Then generate the locale by running:

user $locale-gen
 * Using locale.gen from ROOT /auto/external/buffalo/data02a/gentoo/etc/
 * Generating locale-archive: forcing # of jobs to 1
 * Generating 1 locales (this might take a while) with 1 jobs
 *  (1/1) Generating en_US.UTF-8
 * Generation complete

For more details, refer to the handbook.

Use a nearby mirror

The bootstrap script needs to download quite a bit. It can be accelerated by using a Gentoo mirror nearby. To use a mirror, GENTOO_MIRRORS should be set both in the environment

user $export GENTOO_MIRRORS=http://mirrors.tuna.tsinghua.edu.cn/gentoo
user $./bootstrap-rap.sh

And make.conf:

FILE ${EPREFIX}/etc/portage/make.conf ${EPREFIX}/tmp/etc/portage/make.conf
GENTOO_MIRRORS=http://mirrors.tuna.tsinghua.edu.cn/gentoo

Replace http://mirrors.tuna.tsinghua.edu.cn/gentoo with a favorite mirror.

For more details, refer to the GENTOO_MIRRORS article.

The rsync mirror is set in repos.conf:

FILE ${EPREFIX}/etc/portage/repos.conf
[gentoo]

sync-type = rsync
sync-uri = rsync://mirrors.tuna.tsinghua.edu.cn/gentoo-portage

Replace rsync://mirrors.tuna.tsinghua.edu.cn/gentoo-portage with your favorite mirror. Refer to /etc/portage/repos.conf repos.conf for details. Instead of emerge --syncremember to use emaint sync -r rap.

Troubleshooting

RAP is expected to run on a variety of environments. Documented here are some problems met during the bootstrap and the possible solutions. It is very funny to see how many host systems are badly screwed.

/bin/tar exists but does not work. The working one is in /usr/local/bin

user $/bin/tar
user $strace /bin/tar
bash: /bin/tar: Permission denied
execve("/bin/tar", ["/bin/tar"], [/* 142 vars */]) = -1 EACCES (Permission denied)

Solution: edit the script to prepend /usr/local/bin to PATH.

FILE bootstrap-rap.sh
# the standard path we want to start with, override anything from
        # the user on purpose
-       PATH="/usr/bin:/bin"
+       PATH="/usr/local/bin:/usr/bin:bin"

Username becomes invalid inside RAP

Gentoo::RAP uses its own glibc, which performs independent name service from the host. The uid to username mapping is managed by nss within glibc. A new install of glibc requires a new set of mappings. That is not needed to be done by the Gentoo::Prefix approach (since it does not provide a separate glibc).

The bootstrap script first tries to generate passwd and group from getent and links host /etc/{passwd,group} as fallback:

user $getent passwd > ${EPREFIX}/etc/passwd || ln -s /etc/passwd ${EPREFIX}/etc/passwd
user $getent group > ${EPREFIX}/etc/group || ln -s /etc/group ${EPREFIX}/etc/group

If usernames are provided by LDAP, sometimes the system is configured such that a normal user cannot query the entire passwd or group by getent; in such a case portage breaks, portage will not be able to resolve the username and group. A quick fix is to show yourself to Prefix by specifying who you are:

user $alias id=$(which -a id | tail -n1) # use host id
user $alias getent=$(which -a getent | tail -n1) # use host getent
user $getent passwd $(id -u) > "${EPREFIX}/etc/passwd"
user $getent group $(id -g) > "${EPREFIX}/etc/group"

But other uses on the host cannot be resolved. To get a fair sample of all users on the system, enumerate the /home directory:

user $alias getent=$(which -a getent | tail -n1) # use host getent
user $alias stat=$(which -a stat | tail -n1) # use host stat
user $for d in /home/*; do stat -c '%u' $d; done | sort -u | xargs getent passwd > "${EPREFIX}/etc/passwd"
user $for d in /home/*; do stat -c '%g' $d; done | sort -u | xargs getent group > "${EPREFIX}/etc/group"

Another solution is to use host nss_ldap library. (nss is part of glibc, which resolves uid/gid.)

user $LIBDIR=$(portageq envvar LIBDIR_$(portageq envvar ABI))
user $ln -s /${LIBDIR}/libnss_ldap.so.2 ${EPREFIX}/${LIBDIR}/

Then set nsswitch to query ldap.

FILE EPREFIX/etc/nsswitch.conf
passwd: ldap compat
group: ldap compat

FAQ

What is Gentoo::Prefix / Gentoo::RAP not good for?

Java binaries, even the Oracle jre/jdk packages contain some binary code which needs glib-2.17. If the host glibc is older, getting some key parts of java environment installed on Gentoo::Prefix will not work. Gentoo::Prefix has no multilib support so forget about running a 32bit Adobe acroread through it (although Gentoo::RAP also lacks 32bit support).

Can Gentoo::RAP execute 32bit and 64bit binaries (aka having multilib support)?

No, the profile used in it is amd64-oly.

user $eselect profile list
Available profile symlink targets:
  [1]   prefix/linux/amd64
  [2]   prefix/linux-standalone/amd64 *
  [3]   prefix/linux-standalone/amd64/legacy

The profile we are using is not multilib. Sometime before we thought bootstrapping another Prefix makes more sense if x86 is needed. Seems that we should revisit this, to test out multilib.

Any pressing issue with Gentoo::RAP?

Java support. You have to patch manually $EPREFIX/usr/portage/eclass/java-utils-2.eclass from bug #595002. Because the file does not belong to any package it will be overwritten by the not yet fixed file from portage tree, so after every 'emerge --sync' you have to reapply the patch. Although in general portage tree is updated using 'emerge --sync', on Gentoo::RAP you should use 'emaint sync -r rap' instead. Benda Xu said both approaches do the same but but act on different directory trees. You will realize once you want to apply the above mentioned java patch:

user $find /apps/gentoo -name java-utils-2.eclass
 /apps/gentoo/usr/portage/eclass/java-utils-2.eclass
 /apps/gentoo/usr/portage-stage/eclass/java-utils-2.eclass

The first file still needs to be patched while the second is already patched by Benda Xu. So, 'emerge --sync' updates a portage tree copy in /apps/gentoo/usr/portage/ whereras 'emaint sync -r rap' updates /apps/gentoo/usr/portage-stage/ instead. Now figure out which one is actually used in your installation. ;-)


Further, most apps using java-config and Apache ant still have issues. But it would be only worse if you tried Gentoo::Prefix instead (you would not get even that far).

Perl support. There is likely a general Prefix-related issue in some perl-related eclass or my own programming style issue. Many perl-based packages do not install because of 'Aborting due to QA concerns: double prefix files installed' (sci-biology/TransDecoder-2.1.0::science and many other). See bug #587702.


Can I emerge myself glibc on Prefix?

If I remember no but Fabian claims it should be possible. However, it will be ignored anyways by the binaries and your binaries will anyway use host's glibc (supposedly an old one).

Are there any other special=difficult libs, like iconv, icu, readline, or is only glibc being the main problem?

Seems you can install all these on Gentoo::Prefix except the glibc.

Can I switch from Gentoo:Prefix to RAP?

No idea, probably not.

Can I switch from RAP to the third thingie ("Prefix/libc (native paths method, experimental)"), whatever that does?

Will be merged into RAP, once fully working.

Can I move binaries to another host?

It sounds one can understand that the binaries with -rpath compiled in may not work on another system unless LD_LIBRARY_PATH redirects the paths, but what about Gentoo:RAP binaries. Can they be moved? Could I check something in the binaries using "ldd or some elfutils program" to learn more? They use a modified loader with a hardcoded path, so only if you put them into very same place?

Install elfutils on the hosting operating system. See for example:

user $ldd ${EPREFIX}/bin/bash
user $eu-readelf --relocs ${EPREFIX}/bin/bash
user $eu-readelf --all ${EPREFIX}/bin/bash

External resources