S6/File descriptor holder

The Linux kernel allows one process to send a copy if its open file descriptors to a different process. This is done by transmitting  control messages that contain an array of file descriptors from one process to the other, over a UNIX domain socket, as ancillary data (i.e. over a socket for address family   as the object pointed to by the   field of a   object) using the POSIX   and   calls. This works like POSIX  does for a single process, and in s6's documentation, this is called fd-passing. A file descriptor holder (of fd-holder) is a process that receives file descriptors from others via fd-passing and holds them, for later retrieval either from the same process or from a different one. The fd-holder doesn't do anything with the file descriptors, it only keeps them open.

This article Article description::describes [[s6's file descriptor holder and related tools.]]

Environment variables

 * S6_FD# — Number of file descriptors transferred to an process by.
 * S6_FD_0, S6_FD_1 , ... — File descriptors transferred to an process by.
 * S6_FDID_0, S6_FDID_1 , ... — Identifiers of the file descriptors transferred to an process by.
 * S6_FDLIMIT_0, S6_FDLIMIT_1 , ... — Expiration times of the file descriptors transferred to an process by , as a timestamp in external TAI64N format.

s6-fdholderd
s6 provides a daemon that implements fd-holder functionality, named. It must have its standard input redirected to a bound and listening stream mode UNIX domain socket (i.e. a  socket), and accepts a set of options that control its behaviour. It has builtin access control features, and all operations requested to it must be explicitly granted to the requesting client. For this purpose, it accepts either an  option specifying the pathname of a rules directory, or an   option specifying the pathname of a rules file, just like. will exit with an error status if neither of these options is supplied. The environment specified via appropriate subdirectories of the rules directory controls which operations supported by  are allowed to clients. ignores the files of the rules directory.

runs until told to exit with a  signal; after that, it can keep running for limited time to allow connected clients to finish their ongoing operations. This is called the lame duck timeout, which can be specified with a  option followed by a time value in milliseconds. If is invoked with a   option, or no   option, the lame duck timeout is infinite: after receiving the signal, it will wait until all clients disconnect before exiting. supports the s6 readiness protocol, and if it is invoked with a  option, it will turn readiness notification on, with file descriptor 1 (i.e. its standard output) as the notification channel's file descriptor. If was invoked with a rules file  and receives a   signal, it will re-read it. If was invoked with a rules directory, changes are picked up automatically so   isn't needed.

is a helper program that accepts options and a UNIX domain socket pathname, and invokes chained to, or  chained to , chained to , depending on the options. The socket pathname is passed to. options specify corresponding, and  options.

For further information about or  please consult the HTML documentation in the s6 package's  subdirectory.

fd-holder operations
The operations supported by are store, retrieve, list, delete, get dump and set dump. To store, retrieve or delete a file descriptor, uses identifiers to refer to them. An identifier is a character string containing 1 to 255 characters; any non-null character can be used in an identifier, but it is recommended to only use reasonable ones. For file descriptor identifiers associated with UNIX domain sockets, it is conventional to adhere to the  format, where $path is a placeholder for the socket's absoulte pathname. For file descriptor identifiers associated with TCP or UDP sockets, it is conventional to adhere to the  or   format, where $protocol is 'tcp' or 'udp', $address is is a placeholder for an IPv4 or IPv6 address, $host is is a placeholder for a domain name, and $port is is a placeholder for the TCP or UDP port number. If an identifier is currently in use by, it cannot be reused to store a new file descriptor until the one currently using the identifier is deleted or has expired.


 * A store operation transfers a file descriptor from a client to the fd-holder, specifying an identifier for it, and optionally an expiration time. When a file descriptor has been held for a period of time equal to the expiration time, the fd-holder closes it and frees its identifier. To allow store operations, a file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory, containing a POSIX extended regular expression (i.e those accepted by the  command). A store operation must specify an identifier that matches the regular expression.
 * A retrieve operation transfers a file descriptor from the fd-holder to a client, specifying its corresponding identifier. To allow retrieve operations, a file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory, containing a POSIX extended regular expression. A retrieve operation must specify an identifier that matches the regular expression.
 * A delete operation requests the fd-holder to close a currently held file descriptor, specifying its corresponding identifier, which is then freed. A delete operation is allowed if a store operation specifying the same identifier would be allowed.
 * A list operation requests the fd-holder a list of the identifiers of all currently held file descriptors. To allow list operations, a nonempty file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory.
 * A get dump operation requests the fd-holder to transfer to a client all currently held file descriptors. The transfered file descriptors are not deleted. To allow get dump operations, a nonempty file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory.
 * A set dump operation transfers a subset of a client's file descriptors to the fd-holder, under the control of environment variables. To allow set dump operations, a nonempty file named must exist in the appropriate subdirectory of 's rules directory, or rules file created from a rules directory.

fd-holder clients
s6 provides a client program for each supported fd-holder operation. They all accept options, a UNIX domain socket pathname, and client-specific arguments. A  option followed by a time value in millisecond makes the client program fail with an error message if the requested operation is not processed after the specified time elapses.

Store, retrieve, list and delete operations
The client program that performs store operations is. It accepts options, a UNIX domain socket pathname and a file descriptor identifier, and makes the store request to the fd-holder listening on the specified socket, transferring the specified identifier and file descriptor to it via fd-passing. A  option followed by an unsigned integer value specifies the file descriptor to store; no   option is equivalent to , i.e.  will store its standard input. If is invoked with a   option followed by a time value in milliseconds, the file descriptor's expiration time is set to that value in the fd-holder. If it is invoked with a  option or no   option, the held file descriptor does not expire.

The client program that performs retrieve operations is. It is a a chain loading program that accepts options, a UNIX domain socket pathname and a file descriptor identifier, and makes the retrieve request to the fd-holder listening on the specified socket, transferring the specified identifier to it, receiving the corresponding file descriptor via fd-passing, and executing the next program in the chain. This program's standard input will be a copy of the received descriptor. If is invoked with a   option, it will also request a delete operation for the file descriptor after retrieving it.

The client program that performs delete operations is. It accepts options, a UNIX domain socket pathname and a file descriptor identifier, and makes the delete request to the fd-holder listening on the specified socket, transferring the specified identifier to it.

The client program that performs list operations is. It accepts options and a UNIX domain socket pathname, and makes the list request to the fd-holder listening on the specified socket, printing the received identifier list to its standard output.

For further information about all described fd-holder clients please consult the HTML documentation in the s6 package's subdirectory.

Dump operations
Dump operations are controlled by environment variables. For each file descriptor transfered to or from an process, there are three corresponding variables: S6_FD_${n}, S6_FDID_${n} and S6_FDLIMIT_${n} , where ${n} is a placeholder for an unsigned integer value between 0 and N - 1, and N represents the value of environment variable S6_FD#. The value of S6_FD_${n} specifies the file descriptor as an unsigned integer value, the value of S6_FDID_${n} specifies the file descriptor's identifier, and the value of S6_FDLIMIT_${n} specifies the file descriptor's expiration time for set dump operations, or the file descriptor's remaining time until expiration for get dump operations.

The client program that performs get dump operations is. It is a chain loading program that accepts options and a UNIX domain socket pathname, and makes the get dump request to the fd-holder listening on the specified socket, receiving all file descriptors held by it via fd-passing, and executing the next program in the chain. This program will have a copy of all the received descriptors, and its environment will have the S6_FD* variables set to appropriate values.

The client program that performs set dump operations is. It accepts options and a UNIX domain socket pathname, and makes the set dump request to the fd-holder listening on the specified socket, transferring the subset of its file descriptors specified by the S6_FD* environment variables —which are expected to be set to appropriate values— to the fd-holder, via fd-passing.

For further information about all described fd-holder clients please consult the HTML documentation in the s6 package's subdirectory.

File descriptor transfers
s6 also provides an program, that allows transferring all currently held file descriptors in one  process to another one. The transferred set of file descriptors are added to the destination process' currently held ones. This is implemented by performing a get dump operation with the source process, followed by a set dump operation with the destination process, so these operations must be allowed by the corresponding fd-holder's access control policy. A typical use case of is when an  process needs to upgrade or restart for some reason. Transferring the file descriptors to another, temporary fd-holder allows it to restart without losing all descriptors.

accepts options and the UNIX domain socket pathnames of the source and destinations fd-holders, and performs the transfer between them. If a  option is specified followed by two time values in milliseconds separated by a colon (':'), they will be used as timeouts for the get dump and set dump operations, working just like the   option does for  and.

For further information about please consult the HTML documentation in the s6 package's  subdirectory.

Historical note
The s6 package used to have programs named, , , , , and , which where applications written to be run by an IPC UCSPI client.

,, , , , and  used to simply call one of these programs, using chain loading with  and programs  and  from.

These programs where removed in s6 version 2.9.0.0.

Examples
Example s6-rc service definitions in source format for holding the reading end of a FIFO:

is assumed to be a program that prints to its standard output a message of the form "Message #n", with an incrementing number n between 0 and 9, each time it receives a  signal. Service, a longrun, is a process with its standard output redirected to the  FIFO.

Service, a longrun, is an process that reads from the  FIFO and logs to the  logging directory. Creating the FIFO:

Starting both services, assuming that has been called on the service definitions set to create a compiled services database, that  has been called after, and that the s6 scan directory and the s6-rc live state directory (named ) are both in the same directory:

's  argument increments its verbosity level. Sending three  signals to :

This shows that 's messages were sent over the FIFO to and logged. Stopping momentarily the service:

Sending three more  signals to :

Since the FIFO no longer has any reader, writing to it makes the kernel send a  signal to the  process. Restarting the service and sending three final   signals to :

This shows that the messages sent by when  was not running are effectively lost. This can be avoided by modifying the services so that the reading end of the FIFO is held by an fd-holder process.

Service, a longrun, is an process bound to socket , with readiness notification enabled , and using rules file  in the  subdirectory of its compiled s6 service directory. s6-fdholder-daemon -1 -x data/rules /home/user/fdholder-socket is equivalent to s6-ipcserver-socketbinder /home/user/fdholder-socket s6-fdholderd -1 -x data/rules, but shorter. The rules file is created from the following rules directory using a s6-accessrules-cdb-from-fs test-fdholder/data/rules ../rules.d command:

This rules directory allows user (assumed to have user ID 1000) to perform store (and therefore delete), retrieve and list operations. The file contains a single empty line.

Service, a oneshot, opens FIFO for reading (in non-blocking mode that is changed to blocking afterwards, using execline's  program with options   and  ) and stores the corresponding file descriptor in the fd-holder using identifier fifo:/home/user/test-fifo. s6-fdholder-store /home/user/fdholder-socket fifo:/home/user/test-fifo and s6-fdholder-delete /home/user/fdholder-socket fifo:/home/user/test-fifo are equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-storec fifo:/home/user/test-fifo and s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-deletec fifo:/home/user/test-fifo, respectively, but shorter. The dependency on service ensures that  will start  before trying to store to it. When transitions to the down state, it will delete the held file descriptor.

Service, a longrun, is a modified version of that retrieves the FIFO's reading end from the fd-holder. s6-fdholder-retrieve /home/user/fdholder-socket fifo:/home/user/test-fifo is equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-retrievec fifo:/home/user/test-fifo fdclose 6 fdclose 7, but shorter. The dependency on service ensures that  will have stored the FIFO's reading end first in the fd-holder with the appropriate identifier. Starting the and  services:

Sending three  signals to, stopping momentarily the  service, sending three more   signals to :

This shows that no more messages are being logged as a consequence, but because there is still an open file descriptor held by ), there is no  signal this time. s6-fdholder-list /home/user/fdholder-socket is equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-listc, but shorter. Restarting the  service:

This shows that once the file descriptor is retrieved for, all pending messages are delivered and logged. Sending three final  signals to :

This shows that no messages are lost even if the FIFO reader momentarily stops.

Pre-opening sockets before their servers are running
systemd supports a mechanism it calls socket activation, that makes a process pre-open UNIX domain, TCP/IP or Netlink sockets, FIFOs and other special files, and pass them to a child process when a connection is made, a writer opens the FIFO, new data is available for reading, etc. Socket activation is performed when a socket unit file is provided with an accompanying service unit file. This mechanism combines superserver and fd-holder functionality implemented in process 1, file descriptor inheritance from parent process to child process, and programs written to communicate using pre-opened file descriptors. Similar behaviour can be achieved, for services that want it, by a combination of individual s6 programs like s6/UNIX domain super-server, and the fd-holder client programs, without involving process 1. The software package's author notes, however, that, since read operations, and write operations when buffers are full, on a file descriptor held by an fd-holder will block, speed gains might no be that significant, and that, dependending on the scenario (e.g. logging infrastructure), communicating with a service assuming it is ready when it actually isn't might hurt reliability.

Example s6-rc service definitions in source format for pre-opening a UNIX domain socket and pass it to a program written to be executed by a IPC UCSPI server:

Service is the same as in the previous FIFO example.

Service, a oneshot, pre-opens a listening UNIX domain socket bound to using , and stores the corresponding file descriptor in the fd-holder using identifier unix:/home/user/test-socket. The dependency on service ensures that  will start  before trying to store to it. When transitions to the down state, it will delete the held file descriptor.

Service, a longrun, retrieves the listening socket's file descriptor from the fd-holder, and invokes super-server to handle incoming connections, spawning a  process for each one. is the same program used in the example contained here. The dependency on service ensures that  will have stored the listening socket's file descriptor first in the fd-holder with the appropriate identifier. Starting the service:

's  argument increments its verbosity level. Starting a UCSPI client in the background to connect to the server:

is the same program used in the example contained here. This shows that has been launched with a connection to the server's listening socket, but because its file descriptor is held by  and no server program is currently running, it blocks on an I/O operation.

This shows that is holding the listening socket's file descriptor. Starting the service:

This shows that once has started and retrieved the listening socket's file descriptor from, it accepts the connection and spawns  to handle it.