S6/File descriptor holder

The Linux kernel allows one process to send a copy if its open file descriptors to a different process. This is done by transmitting  control messages that contain an array of file descriptors from one process to the other, over a UNIX domain socket, as ancillary data (i.e. over a socket for address family   as the object pointed to by the   field of a   object) using the POSIX   and   calls. This works like POSIX  does for a single process, and in s6's documentation, this is called fd-passing. A file descriptor holder (of fd-holder) is a process that receives file descriptors from others via fd-passing and holds them, for later retrieval either from the same process or from a different one. The fd-holder doesn't do anything with the file descriptors, it only keeps them open.

This article Article description::describes [[s6's file descriptor holder and related tools.]]

Environment variables

 * S6_FD# — Number of file descriptors transferred to or from an process by  or.
 * S6_FD_0, S6_FD_1 , ... — File descriptors transferred to or from an process by  or.
 * S6_FDID_0, S6_FDID_1 , ... — Identifiers of the file descriptors transferred to or from an process by  or.
 * S6_FDLIMIT_0, S6_FDLIMIT_1 , ... — Expiration times, or remaining time until expiration, of the file descriptors transferred to or from an process by  or , respectively, as a timestamp in external TAI64N format.

s6-fdholderd
s6 provides a daemon that implements fd-holder functionality, named. It must have its standard input redirected to a bound and listening stream mode UNIX domain socket (i.e. a  socket), and accepts a set of options that control its behaviour. It has builtin access control features, and all operations requested to it must be explicitly granted to the requesting client. For this purpose, it accepts either an  option specifying the pathname of a rules directory, or an   option specifying the pathname of a rules file, just like. will exit with an error status if neither of these options is supplied. The environment specified via appropriate subdirectories in a rules directory controls which operations supported by  are allowed to clients. ignores files in a rules directory.

runs until told to exit with a  signal; after that, it can keep running for limited time to allow connected clients to finish their ongoing operations. This is called the lame duck timeout, which can be specified with a  option followed by a time value in milliseconds. If is invoked with a   option, or no   option, the lame duck timeout is infinite: after receiving the signal, it will wait until all clients disconnect before exiting. supports the s6 readiness protocol, and if it is invoked with a  option, it will turn readiness notification on, with file descriptor 1 (i.e. its standard output) as the notification channel's file descriptor. If was invoked with a rules file  and receives a   signal, it will re-read it. If was invoked with a rules directory, changes are picked up automatically so   isn't needed.

is a helper program that accepts options and a UNIX domain socket pathname, and invokes chained to, or  chained to , chained to , depending on the options. The socket pathname is passed to. options specify corresponding, and  options.

For further information about or  please consult the HTML documentation in the s6 package's  subdirectory.

fd-holder operations
The operations supported by are store, retrieve, list, delete, get dump and set dump. To store, retrieve or delete a file descriptor, uses identifiers to refer to them. An identifier is a character string containing 1 to 255 characters; any non-null character can be used in an identifier, however it is recommended to only use reasonable characters. For file descriptor identifiers associated with UNIX domain sockets, it is conventional to adhere to the  format, where $path is a placeholder for the socket's absoulte pathname, and for file descriptor identifiers associated with TCP or UDP sockets, it is conventional to adhere to the   or   format, where $protocol is 'tcp' or 'udp', $address is is a placeholder for an IPv4 or IPv6 address, $host is is a placeholder for a domain name, and $port is is a placeholder for the TCP or UDP port number. If an identifier is currently in use by, it cannot be reused to store a new file descriptor until the one currently using the identifier is deleted or has expired.


 * A store operation transfers a file descriptor from a client to the fd-holder, specifying an identifier for it, and optionally an expiration time. When a file descriptor has been held for a period of time equal to the expiration time, the fd-holder closes it and frees its identifier. To allow store operations, a file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory, containing a POSIX extended regular expression (i.e those accepted by the  command). A store operation must specify an identifier that matches the regular expression.
 * A retrieve operation transfers a file descriptor from the fd-holder to a client, specifying its corresponding identifier. To allow retrieve operations, a file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory, containing a POSIX extended regular expression. A retrieve operation must specify an identifier that matches the regular expression.
 * A delete operation requests the fd-holder to close a currently held file descriptor, specifying its corresponding identifier, which is then freed. A delete operation is allowed if a store operation specifying the same identifier would be allowed.
 * A list operation requests the fd-holder a list of the identifiers of all currently held file descriptors. To allow list operations, a nonempty file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory.
 * A get dump operation requests the fd-holder to transfer to a client all currently held file descriptors. The transfered file descriptors are not deleted. To allow get dump operations, a nonempty file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory.
 * A set dump operation transfers a subset of a client's file descriptors to the fd-holder, under the control of environment variables. To allow set dump operations, a nonempty file named must exist in the appropriate subdirectory of 's rules directory, or rules file created from a rules directory.

fd-holder clients
s6 provides a client program for each supported fd-holder operation, that invokes, using a POSIX  call,  chained to an operation-specific program that handles communications with, using the connection set up by. They all accept options, a UNIX domain socket pathname, and client-specific arguments. The socket pathname is passed to, and everything else, to the operation-specific program. All operation-specific programs accept a  option followed by a time value in millisecond; if the requested operation is not processed after the specified time elapses, they fail with an error message.

Store, retrieve, list and delete operations
The client program that performs store operations is. It invokes chained to the  program. accepts a  option followed by an unsigned integer value that specifies the file descriptor to store. If it is invoked without a  option, it will store its standard input (file descriptor 0). copies the file descriptor specified by the  option to its standard input before replacing itself with. accepts options and a file descriptor identifier; it makes the store request, transferring the specified identifier over the connection to the server, and file descriptor 0 via fd-passing. If it is invoked with a  option followed by a time value in milliseconds, the file descriptor's expiration time is set to that value in the fd-holder. If it is invoked with a  option or no   option, the held file descriptor does not expire.

The client program that performs retrieve operations is. It invokes chained to the  program, chained to the execline package's  program. is a a chain loading program that accepts options and a file descriptor identifier; it makes the retrieve request, transferring the specified identifier over the connection to the server, and receiving the corresponding file descriptor via fd-passing. is used to close UCSPI socket file descriptors 6 and 7 inherited from ; its standard input will be a copy of the descriptor received from, which will be passed to the next program in the chain. If is invoked with a   option, it will also request a delete operation for the file descriptor after retrieving it.

The client program that performs delete operations is. It invokes chained to the  program. accepts a file descriptor identifier; it makes the delete request, transferring the specified identifier over the connection to the server.

The client program that performs list operations is. It invokes chained to the  program. makes the list request over the connection to the server, and prints the identifier list received from to its standard output.

For further information about all described fd-holder clients please consult the HTML documentation in the s6 package's subdirectory.

Dump operations
Dump operations are controlled by environment variables. For each file descriptor transfered to or from an process, there are three corresponding variables: S6_FD_${n}, S6_FDID_${n} and S6_FDLIMIT_${n} , where ${n} is a placeholder for an unsigned integer value between 0 and N - 1, and N represents the value of environment variable S6_FD#. The value of S6_FD_${n} specifies the file descriptor as an unsigned integer value, the value of S6_FDID_${n} specifies the file descriptor's identifier, and the value of S6_FDLIMIT_${n} specifies the file descriptor's expiration time for set dump operations, or the file descriptor's remaining time until expiration for get dump operations.

The client program that performs get dump operations is. It invokes chained to the  program, chained to the execline package's  program. is a chain loading program; it makes the get dump request over the connection to the server, receiving all file descriptors held by via fd-passing, and sets all dump environment variables to appropriate values. is used to close UCSPI socket file descriptors 6 and 7 inherited from, and passes the received file descriptors and dump environment variables to the next program in the chain.

The client program that performs set dump operations is. It invokes chained to the  program. makes the set dump request over the connection to the server, transferring a subset of its file descriptors via fd-passing as specified by the dump environment variables, which are expected to be set to appropriate values.

For further information about all described fd-holder clients please consult the HTML documentation in the s6 package's subdirectory.

File descriptor transfers
s6 also provides an program, and a helper program,, that allow transferring all currently held file descriptors in one  process to a another one. The transferred set of file descriptors are added in the destination process to the currently held ones. This is implemented by performing a get dump operation with the source process, followed by a set dump operation with the destination process, so these operations must be allowed by the corresponding fd-holder's access control policy.

handles communications with the fd-holders to make the get dump and set dump requests; its standard input must be redirected to the source process' socket, and its standard output, to the destination  process' socket. It accepts  and   options followed by a time value in milliseconds, for the source and destination fd-holder processes, respectively; if if the corresponding dump operations are not processed after the specified time elapses,  fails with an error message.

accepts the socket pathnames of the source and destinations fd-holders, and invokes, using a POSIX  call, a chain of programs that include, the execline package's  and  programs, and. The socket pathnames are passed to. If a  option is specified, it is used to construct   and   options for. and are used to move the fd-holder socket file descriptors inherited from  to 's standard input and output, and close the unused ones.

For further information about and  please consult the HTML documentation in the s6 package's  subdirectory.

Examples
Example s6-rc service definitions in source format for holding the reading end of a FIFO:

is assumed to be a program that prints to its standard output a message of the form "Message #n", with an incrementing number n between 0 and 9, each time it receives a  signal. Service, a longrun, is a process with its standard output redirected to the  FIFO.

Service, a longrun, is an process that reads from the  FIFO and logs to the  logging directory. Creating the FIFO:

Starting both services, assuming that has been called on the service definitions set to create a compiled services database, that  has been called after, and that the s6 scan directory and the s6-rc live state directory (named ) are both in the same directory:

's  argument increments its verbosity level. Sending three  signals to :

This shows that 's messages were sent over the FIFO to and logged. Stopping momentarily the service:

Sending three more  signals to :

Since the FIFO no longer has any reader, writing to it makes the kernel send a  signal to the  process. Restarting the service and sending three final   signals to :

This shows that the messages sent by when  was not running are effectively lost. This can be avoided by modifying the services so that the reading end of the FIFO is held by an fd-holder process.

Service, a longrun, is an process bound to socket , with readiness notification enabled , and using rules file  in the  subdirectory of its compiled s6 service directory. s6-fdholder-daemon -1 -x data/rules /home/user/fdholder-socket is equivalent to s6-ipcserver-socketbinder /home/user/fdholder-socket s6-fdholderd -1 -x data/rules, but shorter. The rules file is created from the following rules directory using a s6-accessrules-cdb-from-fs test-fdholder/data/rules ../rules.d command:

This rules directory allows user (assumed to have user ID 1000) to perform store (and therefore delete), retrieve and list operations. The file contains a single empty line.

Service, a oneshot, opens FIFO for reading (in non-blocking mode that is changed to blocking afterwards, using execline's  program with options   and  ) and stores the corresponding file descriptor in the fd-holder using identifier fifo:/home/user/test-fifo. s6-fdholder-store /home/user/fdholder-socket fifo:/home/user/test-fifo and s6-fdholder-delete /home/user/fdholder-socket fifo:/home/user/test-fifo are equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-storec fifo:/home/user/test-fifo and s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-deletec fifo:/home/user/test-fifo, respectively, but shorter. The dependency on service ensures that  will start  before trying to store to it. When transitions to the down state, it will delete the held file descriptor.

Service, a longrun, is a modified version of that retrieves the FIFO's reading end from the fd-holder. s6-fdholder-retrieve /home/user/fdholder-socket fifo:/home/user/test-fifo is equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-retrievec fifo:/home/user/test-fifo fdclose 6 fdclose 7, but shorter. The dependency on service ensures that  will have stored the FIFO's reading end first in the fd-holder with the appropriate identifier. Starting the and  services:

Sending three  signals to, stopping momentarily the  service, sending three more   signals to :

This shows that no more messages are being logged as a consequence, but because there is still an open file descriptor held by ), there is no  signal this time. s6-fdholder-list /home/user/fdholder-socket is equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-listc, but shorter. Restarting the  service:

This shows that once the file descriptor is retrieved for, all pending messages are delivered and logged. Sending three final  signals to :

This shows that no messages are lost even if the FIFO reader momentarily stops.

Pre-opening sockets before their servers are running
systemd supports a mechanism it calls socket activation, that makes a process pre-open UNIX domain, TCP/IP or Netlink sockets, FIFOs and other special files, and pass them to a child process when a connection is made, a writer opens the FIFO, new data is available for reading, etc. Socket activation is performed when a socket unit file is provided with an accompanying service unit file. This mechanism combines superserver and fd-holder functionality implemented in process 1, file descriptor inheritance from parent process to child process, and programs written to communicate using pre-opened file descriptors. Similar behaviour can be achieved, for services that want it, by a combination of individual s6 programs like s6/UNIX domain super-server, and the fd-holder client programs, without involving process 1. The software package's author notes, however, that, since read operations, and write operations when buffers are full, on a file descriptor held by an fd-holder will block, speed gains might no be that significant, and that, dependending on the scenario (e.g. logging infrastructure), communicating with a service assuming it is ready when it actually isn't might hurt reliability.

Example s6-rc service definitions in source format for pre-opening a UNIX domain socket and pass it to a program written to be executed by a IPC UCSPI server:

Service is the same as in the previous FIFO example.

Service, a oneshot, pre-opens a listening UNIX domain socket bound to using , and stores the corresponding file descriptor in the fd-holder using identifier unix:/home/user/test-socket. The dependency on service ensures that  will start  before trying to store to it. When transitions to the down state, it will delete the held file descriptor.

Service, a longrun, retrieves the listening socket's file descriptor from the fd-holder, and invokes super-server to handle incoming connections, spawning a  process for each one. is the same program used in the example contained here. The dependency on service ensures that  will have stored the listening socket's file descriptor first in the fd-holder with the appropriate identifier. Starting the service:

's  argument increments its verbosity level. Starting a UCSPI client in the background to connect to the server:

is the same program used in the example contained here. This shows that has been launched with a connection to the server's listening socket, but because its file descriptor is held by  and no server program is currently running, it blocks on an I/O operation.

This shows that is holding the listening socket's file descriptor. Starting the service:

This shows that once has started and retrieved the listening socket's file descriptor from, it accepts the connection and spawns  to handle it.