User:Goverp/ClientServerGentoo

Gentoo is a meta-distribution - you configure your own distribution through a combination of, USE flag settings, settings such as CHOST and packages. Many Gentoo users run the result on just one computer, but it can be easy to use it on several computers using binary packages. This can be done in several ways; this document describes one of them. It may or may not be the most appropriate for any particular situation.

Concepts and Facilities
This document describes one way to use a binary package host. The Binary package guide contains reference details of the Gentoo facilities being used.

One computer, the does the majority of the "heavy lifting" of compiling the packages. In most cases the others, the, just install the compiled code. will compile and install different versions of the code if the server version is unsuitable for a particular client.

In the approach described, clients use to access the binary packages on the server. Other access methods such as are possible: refer to the binary package guide for details.

Matching processor architectures
The server and clients must have the same CHOST. For best code sharing, their CFLAGS should be similar. For example, the server could be an old but powerful desktop computer with a 4-way AMD Phenom processor, lots of memory and a large RAID disk array, while a client might be a laptop with a later but smaller AMD a9-9420 processor. It turns out that in most cases code for the Phenom runs well on the a9-9420, and magically, Gentoo can identify the cases where it doesn't and this approach will lead to the client compiling its own custom code.

It might not work if the server has a later version of the similar architecture; for example, an a9-9420 server and a Phenom client, unless you compromise by restricting code to the older architecture. This combination is untried; it might work just as well as the other way round, or not. For more details, see for a discussion.

Matching implementation profiles and USE flags
Portage can handle differing and  settings for server and clients. A client to use server code for packages with USE settings the same on both machines, while compiling different code where the settings differ.

Different profiles mean many different USE settings between the machines. This may mean the majority of packages have to be recompiled for the client, negating the benefits of the server/client approach.

Share the portage tree
The server and client machines may share the same portage tree over. This saves the client from carrying the portage tree, and saves having to process. However, it's impossible to run any commands on the client when not connected to the server. (This configuration works well when both DISTDIR and PKGDIR have their default values, so the one directory contains everything needed to build code.)

Alternatively, they may share just (or whatever PKGDIR is set to) via NFS, and the each have their own portage tree. In this case, the client may use against the server's  - see Local Mirror for details. This approach is a little more cumbersome, but allows the client to process portage commands whether or not connected to the server.

The following works using the first arrangement under NFSv4, with being a bind mount for :

(See NFSv4 for more information. NFSv3 needs a slightly different configuration (see NFS).

Share the configuration
contains the configuration. Differing configurations prohibit simply sharing this through NFS. Instead, use, and exploit portage's ability to handle configuration directories instead of files for and so on.

Portage processes files in configuration directories in collating sequence order. Give the server files beginning 01... and the clients files beginning 02... . portage will use the server files as a basis, and override them with the entries in the client files. Keep the client's 01 files in step with the server using :

The following shell script copies all 01-prefixed files of interest. The contents of get synchronised 'in toto', as they're not themselves configuration; the relevant part of that is in. (Substitute server with your server system's network name or address.)

Create the binary package tree
By default, as described in Binary_package_guide.

Server configuration
As described, put all the generic configuration information in files with names prefixed by 01. Here's a couple of examples:

The server configuration specifies hardware optimizations using CPU_FLAGS_X86 and legacy USE flags. These will be overridden in the client's configuration.

Client configuration
Files with 02- prefixed names contain the differences between the client and server configurations.

The majority of the magic lies in. Portage executes the client's 02... configuration after the server's 01..., so its USE above starts with ${USE} to inherit values from server before overriding them. Other settings are straight overrides.

Override DISTDIR to avoid writing to the read-only NFS shared portage tree.

Alter CPU_FLAGS_X86 and USE overriding the legacy settings to allow for differences in server and client CPUs. For example, Phenom processors support 3dNow!, whereas a9-9420 uses newer speed-ups.

Example:

Other configuration files such as package.use can override and add values as necessary. For example:

Operation
Run as normal on the server.

Before running it on the client, start on the server, and run  on the client. Then start on the server. and on the client, run:

Note the purple output, which is portage's way of saying "Here's one I prepared earlier". Notice also the green output where portage calculated that the server package's code is unsuitable for this client.

Compiler --march settings
The configurations above use --march=native on both server and client. This works for me. Gcc is at liberty to generate code exploiting hardware features on the server that might not exist on the client. As it works for me, it appears that it only generates feature-sensitive code if the appropriate value is present in CPU_FLAGS_X86, and portage detects packages sensitive to these flags and compiles separately for the client. Truly magic. I expected this to end in tears.

If your experience differs, the solution is to choose an explicit, less-specific, value of --march such as k8 for AMD processors.

The code generated, in either case, will get --mtune defaulting to the same architecture, unless overridden. Thus the client will be running code tuned for the server, and there may be a performance impact, though probably less than for running generic IA-64 or X86 code. You could of course set --mtune to something more generic.