S6

s6 is daemontools-inspired process supervision suite, a notification framework, a UNIX domain super-server, and tools for file descriptor holding and suidless privilege gain]]. It can be used as an init system component, and also as a helper for supervising OpenRC services. A high level overview of s6 is available here. The package's documentation is provided in HTML format, and can be read on a text user interface using for example.

Environment variables

 * UID - The process' user ID set by when invoked with the   option.
 * GID - The process' group ID set by when invoked with the   option.
 * GIDLIST - The process' supplementary group list set by when invoked with the   option. Must be a comma-separated list of numeric group IDs, without spaces.
 * PROTO - Set by and  to the value 'IPC', as per the IPC UCSPI specification, and used by  and  to construct the names of other environment variables.
 * IPCLOCALPATH - Set by to the pathname associated with the local UNIX domain socket it is using for the connection, as per the IPC UCSPI specification. Also set by  (if the value of PROTO is 'IPC') to the pathname associated with the local UNIX domain socket (the one its standard input and output read from and write to, respectively), as reported by the POSIX   call.
 * IPCREMOTEPATH - Set by to the pathname associated with the remote UNIX domain socket (on Gentoo, as contained in the   field of the   object filled by the Linux   call), if any, as per the IPC UCSPI specification. Be aware that it may contain arbitrary characters.
 * IPCREMOTEEUID - Set by to the effective user ID of the client, as per the IPC UCSPI specification, unless credentials lookups have been disabled. Read by  (if the value of PROTO is 'IPC') to decide whether to allow or refuse access to the server.
 * IPCREMOTEEGID - Set by to the effective group ID of the client, as per the IPC UCSPI specification, unless credentials lookups have been disabled. Read by  (if the value of PROTO is 'IPC') to decide whether to allow or refuse access to the server.
 * IPCCONNNUM - Set by to the number of connections originating from the same user (i.e. same user ID), and read by  (if the value of PROTO is 'IPC') to decide if the maximum number of connections originating from the same user will exceed the maximum allowed.
 * IPCCONNMAX - Maximum number of connections originating from the same client allowed by (if the value of PROTO is 'IPC').
 * S6_FD# - Number of file descriptors transferred to or from an process by  or.
 * S6_FD_0, S6_FD_1 , ... - File descriptors transferred to or from an process by  or.
 * S6_FDID_0, S6_FDID_1 , ... - Identifiers of the file descriptors transferred to or from an process by  or.
 * S6_FDLIMIT_0, S6_FDLIMIT_1 , ... - Expiration dates, or remaining time until expiration, of the file descriptors transferred to or from an process by  or , respectively, as a timestamp in external TAI64N format.

Files

 * - 's scan directory when using OpenRC's s6 integration feature.
 * - Service directory repository searched by OpenRC when using the s6 integration feature.

OpenRC
See here.

Process supervision
For more in-depth information about the process supervision aspects of s6, see daemontools-encore. A summary follows.

Other s6 programs that have a functionality similar to a daemontools program have the daemontools name prefixed with s6-.

The program that implements the supervisor features in s6 is, and just like daemontools' , it takes the (absolute or relative to the working directory) pathname of a service directory (or servicedir) as an argument. An s6 service directory must contain at least an executable file named, and can contain an optional, regular file named , and an optional subdirectory or symbolic link to directory named , all of which work like their daemontools counterparts. Like runit service directories, it can also contain an optional, executable file named, that can be used to perfom cleanup actions each time the supervised process terminates, possibly depending on its exit status information. calls with two arguments: the first one is the supervised process' exit code, or 256 if it was killed by a signal, and the second one is the signal number if the supervised process was killed by a signal, or an undefined number otherwise. Unlike runit's, sends the  process a   signal if it runs for too long. If using s6 version 2.2.0.0 or later, there can be an optional, regular file in the service directory, named, and containing an unsigned integer value that specifies how much time (in milliseconds) the process is allowed to run until being killed. If that file is absent, a default value of 5 seconds is used, which is the fixed value used by earlier versions. Like daemontools-encore, makes its child process the leader of a new session using the POSIX   call, unless the servicedir contains a regular file named  (daemontools-encore's counterpart file is named, though). In that case, the child process will run in 's session instead. waits for a minimum of 1 second between two spawns, so that it does not loop too quickly if the supervised process exits immediately. If receives a   signal, it behaves as if an  command naming the corresponding service directory had been used (see later), and if it receives a   signal, it behaves as if an  command naming the corresponding service directory had been used.

Just like daemontools', keeps control files in a subdirectory of the servicedir, named , and if it finds a symbolic link to directory with that name,  will follow it and use the linked-to directory for its control files. Unlike other process supervision suites, also uses a subdirectory in the servicedir, named, for notifications about the supervised process' state changes (see the notification framework). If doesn't exist,  will create it as a FIFO directory restricted to members of its effective group. If exists,  will use it as-is, and if it is a symbolic link to directory,  will follow it. Complete information about the service directory structure is available here, and for further information about, please consult the HTML documentation in the package's subdirectory.

The author of s6 is also the author of the execline package, that implements the execline language, a scripting language built around chain loading. Execline aims to help producing lightweight and efficient scripts, among other things, by reducing the time involved in spawning and initializing a big command interpreter (like e.g. the program for shell scripts), and by simplifying parsing and doing it only once, when the script is read by the interpreter. The s6 package depends on execline because some of its programs call execline programs or use the execline library,. However, it is not required that or  files be execline scripts. Just like with daemontools, any file format that the kernel knows how to execute is acceptable, and, in particular, they can be shell scripts if so desired.

The program allows supervising a set of processes running in parallel using a scan directory (or scandir), just like daemontools', so it will be the supervision tree's root. from package version 2.3.0.0 or later does not perform periodic scans by default, like other process supervision suites do, unless it it passed a  option with a scan period (as an unsigned integer value in milliseconds). Earlier versions had a default scan period of 5 seconds (equivalent to ) for compatibiliy with daemontools, that could be turned off by explicitly specifying a scan period of 0. can be forced to perform a scan by sending it a  signal, or by using  (see later). When performs a scan, it checks the scan directory and launches an  child process for each new servicedir it finds, or old servicedir for which it finds its  process has exited. All services with a corresponding servicedir are considered active. children for which finds that their corresponding servicedir is no longer present are not stopped, but the service is considered inactive.

keeps control files in a subdirectory of the scandir, named. If this subdirectory or any of its files doesn't exist when is invoked, they will be created. can be controlled by sending it signals, or by using the program. communicates with using a FIFO in the  subdirectory, and accepts a scan directory pathname, and options that specify what to do. Some of 's options are:


 * (alarm): make perform a scan. Equivalent to sending  a   signal.
 * (nuke): make stop  child processes corresponding to inactive services, by sending each of them a   signal, or a   signal if they are running on the  subdirectory of a service directory. This way, if the  subdirectory's  file executes a logger, its corresponding  process will wait for it to exit instead of killing it. Daemontools-style loggers like  would normally exit on their own when the 'main' process terminates, because they would detect an end-of-file condition when trying to read from the pipe that connects the processes.
 * (really nuke): make stop  child processes corresponding to inactive services, by sending each of them a   signal, even if they are running on the  subdirectory of a service directory.
 * (terminate): make stop all  child processes by sending each of them a   signal, or a   signal if they are running on the  subdirectory of a service directory, and then make  start its finish procedure. Therefore,  provides a way to tear down an s6 supervision tree. Equivalent to sending  a   signal, unless signal diversion is turned on.
 * (quit): make stop all  child processes by sending each of them a   signal, even if they are running on the  subdirectory of a service directory, and then make  start its finish procedure. Equivalent to sending  a   signal, unless signal diversion is turned on.
 * (hangup): make stop all  child processes by sending each of them a   signal, and then make  start its finish procedure. Equivalent to sending  a   signal, unless signal diversion is turned on.
 * (abort): make start its finish procedure without stopping its  child processes. Equivalent to sending  a   signal.

Other options are used to control 's finish procedure. For further information about, and the full description of 's functionality, please consult the HTML documentation in the package's subdirectory.

is the logger program provided by the s6 package. Just like daemontools' program, it treats its arguments as a logging script, composed by a sequence of directives that specify what to do. Directives that start with . or / (daemontools-style automatically rotated logging directories or logdirs) behave like their daemontools' counterparts, and so do directives that start with s, n, !, t, + and -, except that patterns in + and - directives are full POSIX extended regular expressions (i.e. those accepted by the  command), and the processor specified in an ! directive invokes, the script parser and launcher from the execline package, with   and   options, instead of. Therefore, the processor's arguments can use execline syntax and, for example, an s6-log '!processor-script arg1 arg2' ./dirname command makes launch a processor with the equivalent of an execlineb -Pc 'processor-script arg1 arg2' command during a rotation of logdir  in 's working directory. If an executable file named is found via PATH search, this will invoke it with arguments   and feed it the contents of  on its standard input. A T directive prepends each logged line with a timestamp in ISO 8601 format for combined date and time representing local time according to the system's timezone, with a space (not a 'T') between the date and the time and two spaces after the time. For, t (timestamp in external TAI64N format) and T directives can appear in any place of the logging script; directives appearing before them apply to lines without the prepended timestamp, and directives appearing after them apply to lines with the prepended timesptamp. can be forced to perform a rotation on a logdir by sending it a  signal. For the full description of 's functionality please consult the HTML documentation in the package's subdirectory.

s6 also provides chain loading programs that can be used to modify a supervised process' execution state. ,, , , and  are similar to daemontools' , , , ,  and , respectively. Besides UID and GID, also sets environment variable GIDLIST to the supplementary group list (as a comma separated list of group IDs) of its effective user, obtained using the POSIX   call. can also accept an argument of the form uid:gid with a numeric user and group ID as an alternative to an account database username. can also take a shared lock on a file (calling Linux  with a   operation on Gentoo) instead of an exclusive lock by invoking it with an   option, and can take a timed lock (using a helper program, ) by invoking it with a   option followed by a time value in milliseconds specifying the timeout. can also make the process the leader of a new (background) process group without creating a new session (using POSIX ) by invoking it with a   option, and can make the process the leader of a new process group in the same session and then attach the session's controlling terminal to the process group to make it the foreground group (using POSIX  ) by invoking it with an   or   option. In the latter case, the process will ignore the resulting  signal, so that it doesn't get stopped. There is also a generalized version of, named : s6-applyuidgid -u uid sets the user ID of the process to uid, s6-applyuidgid -g gid sets the group ID of the process to gid, s6-applyuidgid -G gidlist sets the supplementary group list of the process to gidlist (using Linux  on Gentoo), which must be a comma-separated list of numeric group IDs, without spaces, and s6-applyuidgid -U sets the user ID, group ID and supplementary group list of the process to the values of environment variables UID , GID and GIDLIST , respectively. For the full description of all these programs' functionality please consult the HTML documentation in the package's subdirectory.

is s6's program for controlling supervised processes, and, the program for querying status information about them. accepts a service directory pathname, and options that specify what to do. Unlike daemontools', any pathname after the first one will be ignored. The,  ,   and   options behave just like daemontools 's; in particular,  is actually defined as the equivalent of. The  (capital 'o') option behaves like the   (small 'o') option, except that if the supervised process is not running, it won't be started. Other options allow reliably sending signals to a supervised process, and interacting with 's notification features. In particular, can be used to send a   signal to a supervised  process to force it to perform a rotation. As of version 2.5.1.0, can send a   signal to kill a supervised process that has been stopped using the   option, but still hasn't died after a specified timeout. This will happen if there is a regular file named in the corresponding service directory, containing a nonzero time value in milliseconds (as an unsigned integer) that specifies the timeout. A file containing 0 or an invalid value is equivalent to an absent. And as of version 2.7.2.0, 's  option can make  send a custom signal to the supervised process to stop it (following it with a   signal), and  also accepts an   option (restart) to send the same custom signal but without stopping the process: the corresponding  file, if any, and then the  file, will be executed after process termination, as usual, unless an  or  command has been used before. A regular file named can be placed in the corresponding service directory, containing a signal name or number, to specify the custom signal. An absent file, or one with invalid content, is equivalent to one containing SIGTERM (making  equivalent to  in this case). The mechanism is intended for daemons that use a signal other than   as their default 'stop' command.

accepts a service directory pathname and options. Unlike daemontools', any pathname after the first one will be ignored. Without options, or with only the  option,  prints a human-readable summary of all the available information on the service. In this case, it displays whether the supervised process is running ("run") or not ("down"), whether it is transitioning to the desired state or already there ("want up" or "want down"), how long it has been in the current state, and whether its current up or down status matches the presence or absence of a file in the servicedir ("normally up" or "normally down"). It also shows if the supervised process is paused (because of a  signal). If the process is up, prints its process ID (PID), and, if it suports readiness notification, whether  has been already notified ("ready") or not, and how much time has passed since the notification has been received. If the process is down, prints its exit status (signal name, or signal number if the   option was given, if the supervised process was killed by a signal, or exit code otherwise), whether the really down event has happened or not, and how much time has passed since it did. When is invoked with options other than , it outputs programmatically parsable information instead, as a series of space-separated values, one value per requested field. For example,, or equivalently , will only print the supervised process' PID if is being executed (or -1 if it isn't), and , or equivalently  or  , will only print whether service is up or not ("true" or "false"), and the supervised process' exit code, or -1 if it is running or was killed by a signal.

s6 also provides an program similar to daemontools', that checks whether a  process is currently running on a service directory specified as an argument. Its exit status is 0 if there is one, and 1 if there isn't. For the full description of 's and 's functionality please consult the HTML documentation in the package's subdirectory.

As of version 2.5.0.0, if a service directory contains a file that exits with code 125 (indicating permanent failure), the supervised process won't be started, as if an  command naming the corresponding service directory had been used while the process was running. And as of version 2.7.1.0, keeps a record of supervised process terminations, the death tally, and new programs are available to use the recorded information. The program accepts a servicedir pathname, and prints the corresponding service's death tally. For each recorded process termination, on line is printed. Each line starts with a timestamp in external TAI64N format, and either contains the word "exitcode" followed by the supervised process' exit code if it exited, or the word "signal" followed by a signal name if it was killed by a signal. If an  option is passed to, the numerical value of the signal will be printed instead of the signal name, and if an   option followed by an unsigned integer value is passed to , only the last specified number of termination events are printed. The program accepts a servicedir pathname, and clears the corresponding service's death tally. The maximum number of termination events recorded by can be customized by placing a regular file named  in the corresponding service directory, containing an unsigned integer value that specifies this maximum number. This value cannot be greater than 4096. Every new termination event after the maximum number of recorded events is reached will discard the oldest one in the death tally. If is absent, a default maximum of 100 events is recorded.

is a chain loading program that assumes that its working directory is a servicedir, and checks termination events in the corresponding death tally. If they match specified conditions, the program exits with code 125, otherwise, it executes the next program in the chain. This makes it suitable for use in files, to signal permanent failure to  when certain conditions are met. It takes two unsigned integers as arguments, specifying a time value in seconds and an event count, followed by a comma-separated list of conditions. If the number of termination events with recorded times not earlier that the specified number of seconds in the past, that match any of the conditions, is greater or equal to the specified count, exits with code 125. A condition can be specified as an integer between 0 and 255, representing an exit code, as two such integers separated by a hyphen ('-'), representing a range of exit codes, as a (case insensitive) signal name, such as "SIGTERM", or as the (case insensitive) word "SIG" immediately followed by a signal number, such as "SIG6" for. If the supervised process exits with the specified code, or with a code in the specified range, the corresponding termination event matches the first or second kind of condition, respectively, and if the supervised process is killed by the specified signal, the corresponding termination event matches the last two kinds of conditions. For example, an s6-permafailon 30 5 100,sigabrt prog1 command exits with code 125 if the supervised process has exited with code 100, or has been killed by a  signal, 5 or more times in the last 30 seconds, and executes program prog1 otherwise. And an s6-permafailon 60 13 1-64,sig3 prog2 arg1 arg2 command exits with code 125 if the supervised process has exited with a code greater than or equal to 1 and less than or equal to 64, or has been killed by a  signal, 13 or more times in the last minute, and executes program prog2 with arguments   otherwise.

For further information about, and  please consult the HTML documentation in the package's  subdirectory.

Example s6 scan directory with, , and files, as well as a symbolic link to a  directory elsewhere, and execline scripts:

This file is used for 's finish procedure.

This file allows executing a hipothetical program as a supervised process.

This makes fail permanently if  is killed by a   signal 2 or more times in 10 seconds or less.

Since the script runs for more than 5 seconds, a  file is needed to prevent the process from being killed by  before it completes its execution.

This file allows executing a hipothetical program as a supervised process, that is assumed to use signal   as its 'stop' command, instead of.

This makes send  a   signal if it is still alive after 10 seconds have elapsed since an  command has been used to try stop the daemon.

Resulting supervision tree when is run on this scandir as a background process in an interactive shell, assuming it is the working directory (i.e. launched with s6-svscan & ):

Status of all services reported by in human-readable format:

Output when only the service state, PID, exit code and killing signal information is requested:

This invocation is equivalent to s6-svstat -o up,pid,exitcode,signal $i. The PID is displayed as "-1" for because it is in down state.

subdirectory contents:

Messages sent by to 's standard output when manually started:

Current death tally for :

The timestamps are in external TAI64N format; they can be displaed in human-readable format and local time using :

After enough seconds have elapsed:

The output of, and  shows that  exits each time with an exit code of 0. Reliably sending a  signal, and later a   signal, to :

The output of shows that  is stopped indeed ("paused"), so   doesn't have any efect yet. To resume the process a  signal is needed:

The output of shows that after resuming execution,  was killed by the   signal that was awaiting delivery (signal 15), and since the process is supervised,  restarts  after  exits.

Messages sent by to 's standard output when manually stopped:

As shown by, stopped  by killing it with a   signal (signal 15).

Sending two consecutive and sufficiently close  signals to :

This shows that 's condition was triggered, so it exited with code 125. Because it was executed from, this signals permanent failure to , so is not restarted:

This shows 's two recorded termination events ("involving signal 2", i.e., as reported by 's message).

Manually stopping :

The output of shows that  could not be stopped ("up" but also "want down") because it ignores. The service directory contains a file, so after waiting the specified 10 seconds,  killed  with a   signal (signal 9), as shown by 's message.

The output of confirms that  was killed by a   signal. Output of when only the service state, PID, exit code and killing signal information is requested:

This shows that no services are currently in up state ("false"), so their PIDs are displayed as "-1", and that the processes have been killed by signals, so their exit codes are displayed as "-1".

Creating a file in service directory, restarting  and then using an  command:

The output of and  shows that  exited normally with code 0, because  sent it the required 'stop' signal (, as shown by 's message), and was then restarted it as usual. Stopping with an  command:

Again, as shown by the output of and,  exited normally with code 0. Displaying its death tally:

s6-svscan's finish procedure
When is asked to exit using, it tries to execute a file named , expected to be in the  control subdirectory of the scan directory. The program does this using the POSIX  call, so no new process will be created, and  will have the same process ID as.

is invoked with a single argument that depends on how is invoked:


 * If is invoked with the   option,  will be invoked with a halt argument.
 * If is invoked with the   option,  will be invoked with a poweroff argument.
 * If is invoked with the   option,  will be invoked with a reboot argument.

This behaviour supports running as process 1. Just as or  files in a service directory,  can have any file format that the kernel knows how to execute, but is usually an execline script. If is not running as process 1, the argument supplied to  is usually meaningless and can be ignored. The file can be used just for cleanup in that case, and if no special cleanup is needed, it can be this minimal do-nothing execline script:

If no,   or   option is passed to , or if  receives a  , or if  receives a  ,   or   signal and signal diversion is turned off,  will be invoked with a 'reboot' argument.

If encounters a error situation it cannot handle, or if it is asked to exit and there is no  file, it will try to execute a file named, also expected to be in the  control subdirectory. This is also done using, so no new process will be created, and  will have the same process ID as. If there is no file,  will give up and exit with an exit code of 111.

can also be invoked in this abbreviated forms:


 * (halt) is equivalent to.
 * (reboot) is equivalent to.
 * (poweroff) is equivalent to.
 * (other) is equivalent to, but will be invoked with an 'other' argument instead of a 'halt' argument.
 * (interrupt) is equivalent to, and equivalent to sending a   signal, unless signal diversion is turned on.

Contents of the subdirectory with example  and  files, once  is running:

Messages sent by to 's standard output as a result of different  invocations:

Messages printed by on its standard error, and sent by  to 's standard output, as a result of invoking  after deleting :

s6-svscan's signal diversion feature
When is invoked with an   option, or with neither an   nor an   option, and it receives a ,  ,   or   signal, it behaves as if  had been invoked with its scan directory pathname and an option that depends on the signal.

When is invoked with an   option, signal diversion is turned on: if it receives any of the aforementioned signals, a   signal, or a   signal,  tries to execute a file with the same name as the received signal, expected to be in the  control subdirectory of the scan directory (e.g., , etc.). These files will be called diverted signal handlers, and are executed as a child process of. Just as or  files in a service directory, they can have any file format that the kernel knows how to execute, but are usually execline scripts. If the diverted signal handler corresponding to a received signal does not exist, the signal will have no effect. When signal diversion is turned on, can still be controlled using.

The best known use of this feature is to support the s6-rc service manager as an init system component when is running as process 1; see s6 and s6-rc-based init system.

Example subdirectory with diverted signal handlers for ,   and  :

Output of showing 's process ID and arguments:

Messages printed to 's standard output as a result of sending signals with the utility:

From OpenRC
As of version 0.16, OpenRC provides a service script that can launch, also named. On Gentoo, the scan directory will be. This script exists to support the OpenRC-s6 integration feature, but can be used to just launch an s6 supervision tree when the machine boots by adding it to an OpenRC runlevel:

Or it can also be started manually:

Because is a tmpfs, and therefore volatile, servicedir symlinks must be created in the scan directory each time the machine boots, before  starts. The tmpfiles.d interface, which is supported by OpenRC using package opentmpfiles, can be used for this:

As an alternative, OpenRC's service could be used to start the supervision tree when entering OpenRC's   runlevel, by placing '.start' and '.stop' files in  (please read  for more details) that perform actions similar to those of the  service script:

The  option will explicitly disable signal diversion so that the   signal that  sends to  will make it act as if an  command had been used.

And as another alternative, OpenRC's service could be used to start the supervision tree when entering OpenRC's   runlevel, with  as the scan directory, using a '.start' file that calls the  script provided as an example (see starting the supervision tree from sysvinit), instead of  directly. This allows setting up a logger program to log messages sent by supervision tree processes to 's standard output and error, provided a service directory for the logger exists in :

From sysvinit
The s6 package provides a script called, that can be launched and supervised by sysvinit by adding a  line for it in. It is an execline script that launches an process, with its standard output and error redirected to. This allows setting up a FIFO and a logger program to log messages sent by supervision tree processes to 's standard output and error, with the the same technique used by s6 and s6-rc-based init systems. 's standard input will be redirected to. The enviroment will be emptied and then set according to the contents of environment directory, if it exists, with an invocation. The scan directory will be.

is provided as an example; it is the file in the package's  subdirectory. Users that want this setup will need to copy (and possibly uncompress) the script to, manually edit , and then call :

This will make sysvinit launch and supervise when entering runlevels 1 to 5. Because s6 and execline programs used in the script and invoked using absolute pathnames are asumed to be in directory, a symlink to the correct path for Gentoo must be created:

An s6 service directory for the logger can be created with the  program from package s6-linux-init:

The logger will be an process that logs to directory, prepending messages with a timestamp in external TAI64N format. Username user should be replaced by a valid account's username, to allow to run as an unprivileged process, and  will be a temporary directory created by  on the working directory, that can be removed once the necessary subdirectories are copied to.

The logging chain
A supervision tree where all leaf processes have a logger can be arranged into what the software package's author calls the logging chain, which he considers to be technically superior to the traditional syslog-based centralized approach.

Since processes in a supervision tree are created using the POSIX  call, each of them will inherit 's standard input, output and error. A logging chain arrangement is as follows:


 * Leaf processes should normally have a logger, so their standard output and error connect to their logger's standard input. Therefore, all their messages are collected and stored in dedicated, per-service logs by their logger. Some programs might need to be invoked with special options to make them send messages to their standard error, and redirection of standard error to standard output (i.e. 2>&1 in a shell script or fdmove -c 2 1 in an execline script) must be performed in the servicedir's file.
 * Leaf processes with a controlling terminal are an exception: their standard input, output and error connect to the terminal.
 * , the loggers, and leaf processes that exceptionally don't have logger for some reason, inherit their standard input, output and error from, so their messages are sent wherever the ones from are.
 * Leaf processes that still unavoidably report their messages using  have them collected and logged by a (possibly supervised) syslog server.

s6 and s6-rc-based init systems are arranged in such a way that 's messages are collected by a catch-all logger, and that logger's standard error is redirected to.

The notification framework
Notification is a mechanism by which a process can become instantly aware that a certain event has happened, as opposed to the process actively and periodically checking whether it happened (which is called polling). The s6 package provides a general notification framework that doesn't rely on a long-lived process (e.g. a bus daemon), so that it can be integrated with its supervision suite. The notification framework is based instead on FIFO directories.

FIFO directories and related tools
A FIFO directory (or fifodir) is a directory in the filesystem asociated with a notifier, a process in charge of notifying other processes about some set of events. As the name implies, the directory contains FIFOs, each of them associated with a listener, a process that wants to be notified about one or more events. A listener creates a FIFO in the fifodir and opens it for reading, this is called subscribing to the fifodir. When a certain event happens, the notifier writes to each FIFO in the fifodir. Written data is conventionally a single character encoding the identity of the event. Listeners wait for notifications using some blocking I/O call on the FIFO; unblocking and successfully reading data from it is their notification. A listener that no longer wants to receive notifications removes its FIFO from the fifodir, this is called unsubscribing.

FIFOs and FIFO directories need a special ownership and permission setup to work. The owner of a fifodir must be the notifier's effective user. A publically accesible fifodir can be subscribed to by any user, and its permissions must be 1733 (i.e. the output of would display  ). A restricted fifodir can be subscribed to only by members of the fifodir's group, and its permissions must be 3730 (i.e. the output of would display  ). The owner of a FIFO in the fifodir must be the corresponding listener's effective user, and its permissions must be 0622 (i.e. the output of would display  ). Complete information about the FIFO directory internals is available here.

s6 provides an program that creates a FIFO directory with correct ownership and permissions. It accepts the pathname of the fifodir. A restricted fifodir is created by specifying the  option followed by a numeric group ID, which 's effective user must be a member of. without a  option creates a publically accesible fifodir. A fifodir can be removed with an command. There is also a program that accepts the pathname of a fifodir and removes all FIFOs in it that don't have an active listener. Its effective user must be a member of the fifodir's group. In the normal case FIFOs are removed when the corresponding listener unsubscribes, so is a cleanup tool for cases when this fails (e.g. the listener was killed by a signal). For further information about and  please consult the HTML documentation in the package's  subdirectory.

The program allows notifying all subscribers of a fifodir, so it can be used to create a notifier program. It accepts the pathname of a fifodir and a message that is written as-is to all FIFOs in the fifodir. Each character in the message is assumed to encode an event, and the character sequence should reflect the events sequence. The program allows subscription to a fifodir and waiting for a notification, so it can be used to create a listener program. It accepts the pathname of a fifodir and a POSIX extended regular expression (i.e those accepted by the command), creates a FIFO in the fifodir with correct ownership and permissions, and waits until it reads a sequence of characters that match the regular expression. Then it unsubcribes from the fifodir by removing the FIFO, prints the last character read from it to its standard output, and exits. For further information about and  please consult the HTML documentation in the package's  subdirectory.

Because performing an action that might trigger an event recognized by a notifier, and subscribing to its fifodir to be notified of the event is susceptible to races that might lead to missing the notification, s6 provides two additional programs, and. is a program that accepts options, a set of fifodir pathname and extended regular expression pairs, a program name and its arguments. It subscribes to each specified fifodir, runs the program as a child process with the supplied arguments, and waits for notifications. It makes sure that the program is executed after there are listeners reading from their FIFOs.

expects its arguments to be in the format execline's program generates when parsing the block syntax, so the forward compatible way to use it is in an execline script or  command: the invocation can be written using a the syntax s6-ftrig-listen { f1 re1 f2 re2 ... } prog args, where f1, f2, ... are the fifodir pathnames, re1, re2, ... are the regular expressions corresponding to f1, f2, ..., respectively, prog is the program name and args, the program's arguments. If is invoked with an   option (or), it will unsubscribe from all fifodirs and exit when it reads a matching sequence of characters from any of the created FIFOs. If is invoked without an   option, or with an explicit   option (and), it will wait until it reads a matching sequence from every FIFO. The program is a single fifodir and regular expression version of  that doesn't need -encoded arguments, and that prints the last character read from the created FIFO to its standard output. For further information about and  please consult the HTML documentation in the package's  subdirectory.

A timeout can be set for, and  by specifying a   option followed by a time value in milliseconds. The programs exit with an error status it they haven't been notified about the desired events after the specified time.

The fifodir and notification management code are implemented in the s6 package's library, libs6, and an internal helper program,. The library exposes a public C language API than can be used by programs; for details about the API for notifiers see here, and for details about the API for listeners see here. is launched by the library code.

Creating a publically accesible fifodir named and a fifodir restricted to members of group user (assumed to have group ID 1000) named :

Creating listeners that subscribe to and wait for event sequences 'message1' and 'message2', respectively, as background processes:

This shows that a FIFO has been created in the fifodir for each process, with names starting with 'ftrig1:'.

The output of shows that each  process has spawned a child  helper, and because the one waiting for event sequence 'message2' has a timeout of 20 seconds ("-t 20000"), after that time has elapsed whithout getting the expected notifications it unsubscribes, and exits with an error status that is printed on the shell's terminal ("Connection timed out").

This shows that the process without a timeout is still running, and its FIFO is still there. Notifying all listeners about event sequence 'message1':

The '1' printed on the shell's terminal after the invocation is the last event the  process was notified about (i.e. the last character in string 'message1'), which then exits because the notifications have matched its regular expression.

This shows that since all listeners have unsubscribed, the fifodir is empty.

Executing the example script:

The output of shows that two listeners were created, one subscribed to  and the other to, and the output of  shows that both are implemented by a single  process that is a child of. It also shows that has another child process, executing (at that time) the execline  program, which in turn has spawned the  process. After that, replaces itself with, which notifies all  listeners about event sequence 'message'. Because was invoked with an   option, and the  listener got notifications that match its regular expression,  exits at that point ("s6-ftrig-listen exited").

This shows that the listener subscribed to has unsubscribed and exited, even if it didn't get the expected notifications.

Modifying the test script to invoke with the   option instead (i.e. as s6-ftrig-listen -a { fifodir1 message fifodir2 message } ) and reexecuting it in the background:

The output of the script does not have a "s6-ftrig-listen exited" message, so it is still running:

This confirms that the listener subscribed to is still running, waiting for events. Notifying all listeners about event sequence 'message':

This shows that once the remaining listener has gotten notifications that match its regular expression, exits.

The process supervision suite's use of notification
The subdirectory of an s6 service directory is a fifodir used by  to notify interested listeners about its supervised process' state changes. That is, acts as the notifier associated with the  fifodir, and writes a single character to each FIFO in it when there is a state change:


 * At program startup, after creating if it doesn't exist,  writes an s character (start event).
 * Each time spawns a child process executing the  file, it writes a u character (up event).
 * If the supervised process supports readiness notification, writes a U character (up and ready event) when the child process notifies its readiness.
 * If the service directory contains a file, and, when executed, exits with exit code 125 (permanent failure),  writes an O character (once event, the character is a capital 'o').
 * Each time the supervised process stops running, writes a d character (down event).
 * If the service directory contains a file,  writes a D character (really down) each time  exits or is killed. Otherwise,  writes the character right after the down event notification.
 * When is about to exit normally, it writes an x character (exit event) after the supervised process stops and it has notified listeners about the really down event.

s6 provides an program, that is a process supervision-specific notification tool. It accepts service directory pathnames and options that specify an event to wait for. At program startup, for each specified servicedir it checks the file in its  control subdirectory to see if the corresponding supervised process is already in the state implied by the specified event, and if not, it subscribes to the  fifodir and waits for notifications from the corresponding  process. A  option specifies an up event, a   option, an up and ready event, a   option, a down event, and a   option, a really down event. Options  and   work as for.

There is also an program, that is a process supervision-specific version of. It accepts servicedir pathnames in the format execline's program generates when parsing the block syntax, a program name and its arguments, and options that specify an event to wait for. Therefore, the forward compatible way to use it is in an execline script or command: the invocation can be written using the syntax s6-svlisten { s1 s2 ... } prog args, where s1, s2, ... are the servicedir pathnames, prog is the program name and args are the program's arguments. Options,  ,   and   work as for. Options  and   work as for. also accepts an  option (restart event) that makes it wait for a down event followed by an up event, and a   option (restart and ready event) that makes it wait for a down event followed by an up and ready event. The program is a single servicedir version of  that doesn't need -encoded arguments.

, and  accept a   option to specify a timeout in the same way as. For further information about these programs please consult the HTML documentation in the package's subdirectory.

Finally, the program accepts a   option that makes it wait for notifications from the  process corresponding to the service directory specified as argument, after asking it to perform an action on its child process. An, , , , or  command is equivalent to an , , , ,  or  command, respectively, specifying the same servicedir, and  with the same arguments except for the   option, as the spawned program. also accepts a timeout specified with a  option, that is translated to the    option.

See the service readiness notification section for usage examples.

Service readiness notification
When a process is supervised, it transitions to the 'up' state when its supervisor has successfully spawned a child process executing the file. considers this an up event, and notifies all listeners subscribed to the corresponding fifodir about it. But when the supervised process is executing a server program for example, it might not be ready to provide its service immediately after startup. Programs might do initialization work that could take some noticeable time before they are actually ready to serve, but it is impossible for the supervisor to know exactly how much. Because of this, and because the kind of initialization to do is program-specific, some sort of collaboration from the supervised process is needed to help the supervisor know when it is ready. This is called readiness notification.

systemd has the concept of readiness notification, called start-up completion notification in its documentation. To support readiness notification under systemd, a program implements the $NOTIFY_SOCKET protocol, based on message passing over a datagram mode UNIX domain socket, bound to a pathname specified as the value of the NOTIFY_SOCKET environment variable. The protocol is implemented by libsystemd's sd_..._notify... family of functions, although it is covered by systemd's interface stability promise, so it is possible to have alternative implementations of it. The program can perform start-up completion notification by linking to libsystemd and calling one of those functions. systemd uses start-up completion notification when a service unit file contains a 'Type=notify' directive.

To support readiness notification under s6, a program implements the s6 readiness notification protocol, which works like this:


 * 1) At program startup, the program expects to have a file descriptor open for writing, associated with a notification channel. The program chooses the file descriptor. For example, it can be specified as a program argument, or be a fixed, program-specific well-know number specified in the program's documentation.
 * 2) When all initialization work necessary to reach the program's definition of 'service ready state' has been completed, it writes a newline character to the notification channel.
 * 3) The program closes the notification channel after writing to it.

Therefore, a typical code snippet in the C language that implements the last two steps would be as follows:

The code only relies on POSIX calls, so the program doesn't need to link to any specific library other than the libc to implement the readiness protocol. s6 uses readiness notification when a regular file named is present in a service directory, containing an integer that specifies the program's chosen notification channel file descriptor. implements the notification channel as a pipe between the supervised process and itself; when it receives a newline character signalling the service's readiness, it considers that an up and ready event and notifies all listeners subscribed to the fifodir about it. After that, no longer reads from the notification pipe, so it can be safely closed by the child process.

As of s6 version 2.7.2.0, itself supports the readiness nofication protocol: a   option followed by an unsigned integer value can be passed to it, specifying the notification channel's file descriptor. will use the notification channel to signal readiness when it has initialized all its necessary resources, and is ready to perform the first scan of the supplied scan directory, and ready to receive commands from and to react to signals. This can be useful when is launched as a supervised process that is part of a supervision tree. Typically, to create an s6 subtree, and often to run with an unprivileged user (so that all subtree processes do so as well). When the  option is not used,  does not signal readiness.

Example s6 scan directory containing services that support readiness notification:

It is assumed that is a program that supports an   option to turn readiness notification on, specifying the notification channel's file descriptor (5), which is also stored in a  file. exits with an exit code of 125, so that if the corresponding process stops, it won't be restarted. The invocation creates  as a publically accesible fifodir. Using on it to start the supervision tree and verify that  notifies listeners about the start event:

This shows that has created all missing  directories as restricted fifodirs, but uses the publicly accessible one created by.

Executing the example script:

This shows that the process has spawned a child  helper, and created a FIFO in  so that it can be notified about the up event. Manually starting :

The message printed by the test script to its standard output shows that the process got the expected notification, so it exited.

The script calls to subscribe to fifodirs  and  and wait for up and ready events. Then it uses a s6-svc -wu -u command to manually start and, and wait for up events. Both scripts invoke  with readiness notification on. A message timestamped using is printed to the standard output when the listeners get their expected notifications. Executing the example script:

This shows that the processes waiting for up events are notified first, so they exit, and that the  process waiting for up and ready events is notified 10 seconds later. The output of shows that when the  processes exited, the  process and its  child were still running.

This confirms that both processes have notified readiness to their  parent ("ready 19 seconds") 10 seconds after being started. Using on fifodir  to verify that  notifies listeners about a once event when  is killed with a , because of 's exit code:

The script calls to subscribe to fifodirs  and  and wait for down events. Then it uses a s6-svc -wD -d command to manually stop the processes corresponding to  and, and wait for really down events. has a script that sleeps for 10 seconds, so  listeners should be notified earlier than  listeners. A message timestamped using is printed to the standard output when the listeners get their expected notifications. Executing the example script:

This shows that the process waiting for down events and the  process subscribed to  and waiting for a really down event are notified first with almost no delay between them, so they exit, and that the  process subscribed to  and waiting for a really down event is notified 10 seconds later. The output of shows that when the  process exited, an  process that had replaced itself with  (because of the   option) and its  child were still running.

This confirms that the process corresponding to  hasn't been restarted after  exited (83 seconds in down state and no 'wanted up'), and that the down and ready events for the  processes corresponding to  and  have a 10 seconds delay between them ("ready 21 seconds" compared to "ready 31 seconds"). Using on fifodir  to stop the supervision tree and verify that  notifies listeners about the exit event:

s6-notifyoncheck
As of version 2.6.1.0, s6 provides the program, which can be used in combination with programs that don't support readiness notificaton, but can be polled for readiness somehow. In that case, can be invoked from a  file, use the available polling mechanisms, and signal readiness itself to  using the s6 readiness notification protocol.

is a chain loading program that assumes its working directory is a servicedir, spawns a child process that polls for readiness, and then executes the next program in the chain. By default, the child process will try to execute a file named in a subdirectory named  as a child process (i.e. the pathname of the file, relative to 's working directory, is ). Just like or,  can have any file format that the kernel knows how to execute, but is usually an execline or shell script. When executed, is expected to poll the supervised process for readiness, and then exit with code 0 if the process was verified to be ready, or exit with a nonzero code otherwise. This is similar to runit's file mechanism. If a  option is passed to, instead of looking for a  file, the child process will invoke , the script parser and launcher from the execline package, and pass it the   option and the argument that follows it. For example, s6-notifyoncheck -c eargs prog arg1 arg2 will spawn a child process that uses an execlineb -c eargs command to poll for readiness (so eargs can have execline syntax), while it executes program prog with arguments  (without creating a new process). This option is mainly useful if the program used to poll the supervised process is very simple and can be inlined as a simple command line, to avoid having to manage a whole script and a file.

By default, expects to be able to read the file descriptor it should use for the notification channel from a  file in its working directory (i.e. the file used by ), and uses a single POSIX   call to create the poller child process, so the next program in the chain must be able to reap it when it terminates (e.g. with a POSIX   call). If a  option followed by an unsigned integer value is passed to, it will use the specified value as the notification channel's file descriptor, ignoring. And if a  option is passed to, it will use two   calls instead of one (i.e. it will double fork), so the poller process will be reparented to process 1 (or to a local reaper), and the next program in the chain won't have to reap it. This is useful to avoid having a lingering zombie process if the next program in the chain does not reap child processes it doesn't know of.

If the poll for readiness is successful, 's poller process signals readiness using the notification channel. Unlike runit's command, which executes the  file only once, 's poller process periodically retries execution of  or invocation of  until a poll is successful, until a timeout period expires, or until a certain number of unsuccessful polls has been reached, depending on the options supplied to, and then exits. A retry is performed once every second by default, i.e. the default polling period is 1 second, but a different one can be specified by passing a  option to, followed by a time value in milliseconds. Execution of the or  process can be limited by passing a   option to, followed by a time value in milliseconds. If the process runs for longer than the specified time, 's poller process sends it a  signal to kill it, and then exits without signalling readiness. If the  option is not used, 's poller process will wait indefinitely for the  or  process to exit.

For the full description of please consult the HTML documentation in the package's  subdirectory.

Example scan directory with a servicedir for a program that can be polled for readiness:

This file allows executing a hipothetical program as a supervised process. is used to poll the process for readiness. A  option is used to make  double fork, because  is assumed to not reap child processes, and the   option is used to make 's poller process retry execution of  after each unsuccessful poll, until the number of retries equals the value of the ATTEMPTS environment variable. The environment of the process is modified by the contents of the  environment directory using an  invocation.

This file allows executing a hipothetical program that is assumed to be able to poll  for readiness. Messages are printed to the process' standard output to report the outcome.

This file informs that  supports the s6 readiness notification protocol using file descriptor 3 for the notification channel, and will also be used by.

Environment directory sets the value of environment variable ATTEMPTS to 5. This means that after 5 executions of that result in a nonzero exit code (i.e. if  is determined to not be ready when polled), 's poller process will exit without signalling readiness to 's  parent.

Starting and waiting for the supervised process to be ready, using 's   option:

This shows that was unsuccessfully polled 5 times, and because of 's   option, it exited without signalling readiness to. Therefore, after the 12 seconds specified with the  option,  timed out, as shown by 's error message and the output of the  utility.

This shows that is actually running ("up"), but  does not consider the process to be ready (there is no "ready" in 's output). Modifying the directory so that the value of environment variable ATTEMPTS is 0, restarting  and waiting for the supervised process to be ready:

Because of the new value of ATTEMPTS, was invoked with an   option, which tells it to keep executing  until there is a successful poll. This shows that was polled for readiness once per second until the 11th attempt, which was successful. The output of of the utility shows that this happend after approximately 10 seconds, i.e. before 's timeout of 12 seconds, which must mean it received an up and ready notification from 's supervisor.

This confirms that up and ready. It became ready approximately 10 seconds after it was started.

The UNIX domain super-server and related tools
s6 provides two programs, and, that together implement a UNIX domain super-server. A UNIX domain super-server creates a listening sream mode UNIX domain socket (i.e. a  socket for address family  ) and spawns a server program to handle each incoming connection after accepting it, so that from there on, the client communicates over the connection with the spawned server. This a UNIX domain equivalent of a TCP/IP super-server, such as xinetd or ipsvd.

More specifically, what and  implement together is an IPC UCSPI super-server, i.e. a super-server that adheres to the server side of Daniel J. Bernstein's UNIX Client-Server Program Interface (UCSPI) and supports the IPC UCSPI protocol. The UCSPI defines an interface for client-server communications tools; UCSPI tools are executable programs that accept options, a protocol-specific address, an application name and its arguments. Tools can be either clients or servers, clients communicate with servers using a connection. If the UCSPI tool is a server, the application is invoked with the supplied arguments each time there is an incoming connection to the specified address, with file descriptor 0 opened for reading from the connection, file descriptor 1 opened for writing to the connection, and environment variables set to defined values. If the UCSPI tool is a client, a connection is made to the specified address and, if successful, the application is invoked with the supplied arguments, with file descriptor 6 opened for reading from the connection, file descriptor 7 opened for writing to the connection, and environment variables set to defined values. One of the application environment variables set by both UCSPI clients and servers is PROTO, with the name of the supported protocol as its value. The protocol implemented by the s6 programs is the IPC UCSPI protocol, for which the address specified to UCSPI tools is defined to be a UNIX domain socket pathname, and the value of PROTO is defined to be 'IPC'.

is a chain loading program that accepts options and a pathname. It creates a UNIX domain socket, binds it to the specified pathname and prepares it to accept connections with the POSIX  call. The next program in the chain will have its standard input connected to the listening socket, which will be non-blocking. The number of backlog connections, i.e. the number of outstanding connections in the socket's listen queue, can be set with the  option; additional connection attempts will be rejected by the kernel. If is invoked with a   argument, the socket will be created but won't be listening, i.e.   won't be called. If it is invoked with the  option, the created socket will be a datagram mode socket, which also requires specifying a   argument. If it is invoked without the  option, or with the   option, the created socket will be a stream mode socket.

is a program that must have its standard input redirected to a bound and listening stream mode UNIX domain socket, and accepts a program name and its arguments. For each connection made to the socket, executes the program with the supplied arguments as a child process, that has its file descriptors 0 and 1 redirected, on Gentoo, to the socket returned by a Linux   with the listening socket's file descriptor as an argument, and the following environment variables set to the appropriate values (see environment variables): PROTO, IPCREMOTEPATH , IPCREMOTEEUID , IPCREMOTEEGID and IPCCONNNUM. On Gentoo, the variables IPCREMOTEEUID, IPCREMOTEEGID are set with information obtained from a POSIX  call with the Linux   option; this is called credentials lookup. If is invoked with a   option, credentials lookup will be disabled, and IPCREMOTEEUID and IPCREMOTEEGID will be unset. If it is invoked without a  option, or with a   option, credentials lookup will be enabled. supports the s6 readiness protocol, and if it is invoked with a  option, it will turn readiness notification on, with file descriptor 1 (i.e. its standard output) as the notification channel's file descriptor. If it receives a  signal it will exit, but its children will continue running. If it receives a  signal, it will send its children a   signal followed by a   signal and then exit, and if it receives a   signal, it will send its children a   signal and then exit. It is possible to make kill its children without exiting (with a   signal followed by a   signal), by sending it a   signal.

is a helper program that accepts options, a UNIX domain socket pathname, a program name and its arguments, and invokes chained to, or  chained to , chained to , depending on the options. The socket pathname is passed to, and the program name and its arguments, to. options specify corresponding, and  options. The created socket is a stream mode socket. For further information about, or  please consult the HTML documentation in the package's  subdirectory.

s6 also provides an IPC UCSPI client,, and an access control tool for UNIX domain sockets,. is a chain loading program that accepts options and a UNIX domain socket pathname. It creates a stream mode socket, makes a connection to the socket specified by the supplied pathname, and executes the next program in the chain with file descriptors 6 and 7 redirected to the local connected socket, and the environment variables PROTO and IPCLOCALPATH set to the appropriate values (see environment variables). If is invoked with a   argument, it will bind the created socket to pathname (using the POSIX   call) before initiating the connection to the remote socket. If it is invoked with a  argument, it will set IPCLOCALPATH to value instead of setting it with information obtained from a POSIX   call. is a chain loading program that must be spawned by a UCSPI server (like ) that appropriately sets the PROTO, ${PROTO}REMOTEEUID and ${PROTO}REMOTEEGID environment variables, where ${PROTO} is the value of the PROTO variable. It decides whether or not to execute the next program in the chain, or to execute a completely different program instead, based on either a rules directory or a rules file. The  option specifies the pathname of a rules directory and the   option specifies the pathname of a rules file. A rules file is a constant database (CDB) file created from a rules directory using the program.

When a rules directory R is specified to :


 * On program invocation, searches the  directory for a subdirectory with a name that matches the value of the ${PROTO}REMOTEEUID variable, which must be a numeric user ID. If it finds a matching subdirectory, and it contains a file named,  executes the next program in the chain. If the subdirectory does not contain a file named , but contains a file named , or does not contain any of those files, it exits with exit code 1, and the next program in the chain is not executed.
 * If does not contain a subdirectory with a matching name,  will then search the  directory for a subdirectory with a name that matches the value of the ${PROTO}REMOTEEGID variable, which must be a numeric group ID. If it finds finds a matching subdirectory, it follows the procedure described for the  directory.
 * If does not contain a subdirectory with a matching name,  will finally search for an  directory. If the directory exists, it follows the procedure described for the  directory. If it doesn't, it exits with exit code 1, and the next program in the chain is not executed.
 * If the next program in the chain is executed, and if the matching directory M contains a subdirectory named, will modify its environment as if an s6-envdir M/env command had been used.
 * If the next program in the chain is executed, will also set the IPCLOCALPATH environment variable (see environment variables), unless it was invoked with an   option. If  is invoked without a   option, or with a   option, IPCLOCALPATH will be set.
 * If the next program in the chain is executed, and was invoked with an   option, it will also unset environment variables PROTO, ${PROTO}REMOTEPATH , ${PROTO}REMOTEEUID , ${PROTO}REMOTEEGID and ${PROTO}CONNNUM.
 * If the next program in the chain would otherwise be executed, but the matching directory M contains a regular file named, will execute a different program instead, as if an execlineb -c contents command had been used, where contents is the contents of the  file (e.g. the name of a program that can be found via PATH search plus its arguments).

If is invoked with neither the   option nor the   option, it will execute the next program in the chain, i.e. it will unconditionally grant access.

A rules directory can be re-created from a rules CDB file using the program. For the full description of 's, 's, 's and 's functionality please consult the HTML documentation in the package's subdirectory.

Finally, s6 also provides the program, a chain loading program that limits connections from the same client to an UCSPI server based on the PROTO, ${PROTO}CONNNUM , ${PROTO}CONNMAX environment variables (see environment variables), where ${PROTO} is the value of the PROTO variable, and , a program that performs data transmission from file descriptor 0 to file descriptor 7, and from file descriptor 6 to file descriptor 1, all of them assumed to be open at program startup. That is, performs full-duplex data transmission. For the full description of 's and 's functionality please consult the HTML documentation in the package's subdirectory.

The application reads from the open file descriptor supplied by the UCSPI server, expecting to receive a 'Hello!' message from the client, and if it does, it sends a response that contains the application's process ID and the account database username corresponding to the client's user ID, supplied by the UCSPI server via the IPCREMOTEEGID environment variable. The application then waits for 10 seconds, and finally exits. Because a  call on a socket file descriptor can fail with a 'broken pipe' error, and because the kernel will send a   signal to the process if it does, an external   function assumed to be available to the program is called, that makes the process ignore the signal using the POSIX   call.

The application prints 'Connecting to server...' to its standard output, sends a 'Hello!' message to the server using the open file descriptor supplied by the UCSPI client, and waits for a reply, which is printed to its standard output. The application then exits.

Starting the UCSPI super-server:

This shows that a socket named has been created in the current working directory. Starting a UCSPI client to connect to the socket three times:

This shows that the super-server spawned three processes to handle each connection, and set the IPCREMOTEEUID environment variable to user1's user ID. s6-ipcserver test-socket ./test-server is equivalent to s6-ipcserver-socketbinder test-socket s6-ipcserverd ./test-server, but shorter. Starting a UCSPI client with effective user user2:

This shows that the super-server set the IPCREMOTEEUID environment variable to user2's user ID. Starting the super-server with the  option, and a client to connect to :

s6-ipcserver -P test-socket ./test-server is equivalent to s6-ipcserver-socketbinder test-socket s6-ipcserverd -P ./test-server, but shorter. This shows that since credentials lookup was disabled, environment variable IPCREMOTEEGID is unset, and displays '&lt;unavailable&gt;' in place of a username.

See the suidless privilege gain section for usage examples.

Suidless privilege gain tools
s6 provides two programs, and, that can be used to implement controlled privilege gains without setuid programs. This is achieved by having run as a long-lived process with an effective user that has the required privileges, and bound to a stream mode UNIX domain socket, and having, which can run with an unprivileged effective user, ask the  process over a connection to its socket to perform an action on its behalf.

is a program that must be spawned by a UCSPI server (like ) and accepts options and an argument sequence s1, s2, ... that can be empty. is a program that must be spawned by a UCSPI client and accepts options and an argument sequence c1, c2, ... that can also be empty. transmits the argument sequence over the connection to the server, that must be an process, and its environment variables, unless it is invoked with an   option. concatenates its argument sequence with the one received from the client, and passes it to a POSIX  call, which results in a program invocation. also transmits its standard input, output and error file descriptors to using   control messages (i.e. fd-passing, see the file descriptor holder and related tools), so that the invoked program will run as a child process of, with 's effective user, but its standard input, output and error descriptors will be a copy of 's. The program's environment will be 's environment, except that every variable that is defined but has an empty value will set to the value it has in  's enviroment, if it is also set. waits until 's child process exits. If it is invoked with a  option followed by a time value in milliseconds, it will close the conection and exit after the specified time has passed if 's child is still running.

is a helper program that accepts options, a UNIX domain socket pathname and an argument sequence, and invokes  chained to. The socket pathname is passed to, and the argument sequence, to. options specify corresponding and  options. For the full description of 's, 's and 's functionality please consult the HTML documentation in the package's subdirectory.

Standard permissions settings on 's listening socket can be used to implement some access control, and credentials passing over a UNIX domain socket also allows finer-grained control. The program can be used to take advantage of credentials passing.

Testing the script by executing it directly:

The script is executed with effective user user1 (UID 1000), IPCREMOTEEUID and VAR3 are unset, and VAR1 and VAR2 are set to the specified values.

's  argument increments its verbosity level. Contents of rules directory :

File contains an empty line, so the corresponding environment variable will be set, but empty. Launching the process:

This shows that a UNIX domain socket named was created in the working directory. Running with effective user user2 (UID 1001):

s6-sudo run-test-script arg3 arg4 is equivalent to s6-ipcclient run-test-script s6-sudoc arg3 arg4, but shorter. This shows that the rules directory setup denied execution of to user2 (UID 1001); it only allows it to the user with UID 1002. Modifying :

Retrying :

Comparing to the output of the script when run directly by user1, this shows that 's arguments are the concatenation of the ones supplied to in script, arg1 and arg2, and the ones specified in the  invocation, arg3 and arg4. Also, 's environment has 's variables: IPCREMOTEEUID, inherited from , and VAR3 , inherited from , which in turn sets it based on environment directory. Because variable VAR1 is set by but empty,  sets it to the value it has in 's environment. And because variable VAR2 is set in 's environment but not in 's, it is also unset in 's environment.

The file descriptor holder and related tools
The Linux kernel allows one process to send a copy if its open file descriptors to a different process. This is done by transmitting  control messages with an array of file descriptors from one process to the other over a UNIX domain socket as ancillary data (i.e. over a socket for address family   as the object pointed to by the   field of a   object) using the POSIX   and   calls. This works like POSIX  does for a single process, and in s6's documentation, this is called fd-passing. A file descriptor holder (of fd-holder) is a process that receives file descriptors from others via fd-passing and holds them, for later retrieval either from the same process or from a different one. The fd-holder doesn't do anything with the file descriptors, it only keeps them open. For some possible uses of this feature, see later.

s6 provides a program implementing fd-holder functionality, named. It must have its standard input redirected to a bound and listening stream mode UNIX domain socket (i.e. a  socket), and accepts a set of options that control its behaviour. It has builtin access control features, and all operations requested to it must be explicitly granted to the requesting client. For this purpose, it accepts either an  option specifying the pathname of a rules directory, or an   option specifying the pathname of a rules file, just like. will exit with an error status if neither of these options is supplied. The environment specified via appropriate subdirectories in a rules directory controls which operations supported by  are allowed to which clients. ignores files in a rules directory.

runs until told to exit with a  signal; after that it can keep running for limited time to allow connected clients to finish their ongoing operations. This is called the lame duck timeout, which can be specified with a  option followed by a time value in milliseconds. If is invoked with a   argument or no   option, the lame duck timeout is infinite: after receiving the signal, it will wait until all clients disconnect before exiting. supports the s6 readiness protocol, and if it is invoked with a  option, it will turn readiness notification on, with file descriptor 1 (i.e. its standard output) as the notification channel's file descriptor. If was invoked with a rules file  and receives a   signal, it will re-read it. If was invoked with a rules directory, changes are picked up automatically so   isn't needed.

is a helper program that accepts options and a UNIX domain socket pathname, and invokes chained to, or  chained to , chained to , depending on the options. The socket pathname is passed to. options specify corresponding, and  options. For further information about or  please consult the HTML documentation in the package's  subdirectory.

Operations supported by are store, retrieve, list, delete, get dump and set dump. s6 provides a client program for each fd-holder operation, that invokes, using a POSIX  call,  chained to an operation-specific program that handles communications with, using the connection set up by. To store, retrieve or delete a file descriptor, uses identifiers to refer to them. An identifier is a character string containing 1 to 255 characters; any non-null character can be used in an identifier, however it is recommended to only use reasonable characters. For file descriptor identifiers associated with UNIX domain sockets, it is conventional to adhere to the 'unix:$path' format, where $path is the socket's absoulte pathname, and for file descriptor identifiers associated with TCP or UDP sockets, it is conventional to adhere to the '$protocol:$address:$port' or '$protocol:$host:$port' format, where $protocol is 'tcp' or 'udp', $address is an IPv4 or IPv6 address, $host is a domain name, and $port is the TCP or UDP port number. If an identifier is currently in use by, it cannot be reused to store a new file descriptor until the one currently using the identifier is deleted or has expired.


 * A store operation transfers a file descriptor from a client to the fd-holder, specifying an identifier for it, and optionally an expiration time. When a file descriptor has been held for a period of time equal to the expiration time, the fd-holder closes it and frees its identifier. To allow store operations, a file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory, containing a POSIX extended regular expression (i.e those accepted by the  command). A store operation must specify an identifier that matches the regular expression.
 * The client program that performs store operations is . It accepts options, a UNIX domain socket pathname and a file descriptor identifier, and invokes chained to the  program. The socket pathname is passed to, and the identifier, to .  accepts a   option followed by an unsigned integer value, that specifies the file descriptor to store. If it is invoked without a   option, it will store its standard input (file descriptor 0). All other supplied options are passed to .  makes the store request, transferring the specified identifier to the server over the connection, and file descriptor 0 via fd-passing. If it is invoked with a   option followed by a time value in milliseconds, the file descriptor's expiration time is set to that value in the fd-holder. If it is invoked with a   argument or no   option, the held file descriptor does not expire.  copies the file descriptor specified by the   option to its standard input before replacing itself with.
 * A retrieve operation transfers a file descriptor from the fd-holder to a client, specifying its corresponding identifier. To allow retrieve operations, a file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory, containing a POSIX extended regular expression. A retrieve operation must specify an identifier that matches the regular expression.
 * The client program that performs retrieve operations is . It is a chain loading program that accepts options, a UNIX domain socket pathname and a file descriptor identifier, and invokes chained to the  program, chained to execline's  program. The socket pathname is passed to, and the options and identifier, to .  makes the retrieve request, transferring the specified identifier over the connection to the server, and receiving the corresponding file descriptor via fd-passing.  closes UCSPI socket file descriptors 6 and 7 inherited from ; its standard input will be a copy of the descriptor received from , which will be passed to the next program in the chain. If  is invoked with a   option, it will also request a delete operation for the file descriptor after retrieving it.
 * A delete operation requests the fd-holder to close a currently held file descriptor, specifying its corresponding identifier, which is then freed. A delete operation is allowed if a store operation specifying the same identifier would be allowed.
 * The s6 client that performs delete operations is . It accepts options, a UNIX domain socket pathname and a file descriptor identifier, and invokes chained to the  program. The socket pathname is passed to, and the options and identifier, to  .  makes the delete request, transferring the specified identifier over the connection to the server.
 * A list operation requests the fd-holder a list of the identifiers of all currently held file descriptors. To allow list operations, a nonempty file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory.
 * The client program that performs list operations is . It accepts options and a UNIX domain socket pathname, and invokes chained to the  program. The socket pathname is passed to, and the options, to .  makes the list request over the connection to the server, and prints the identifier list received from  to its standard output.
 * A get dump operation requests the fd-holder to transfer to a client all currently held file descriptors. The file descriptors are not deleted. To allow get dump operations, a nonempty file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory.
 * The client program that performs get dump operations is . It is a chain loading program that accepts options and a UNIX domain socket pathname, and invokes chained to the  program, chained to execline's  program. The socket pathname is passed to, and the options, to .  makes the get dump request over the connection to the server, receiving all file descriptors held by  via fd-passing, and sets environment variables.  closes UCSPI socket file descriptors 6 and 7 inherited from , and passes the received file descriptors and environment variables set by  to the next program in the chain.
 * A set dump operation transfers a subset of a client's file descriptors to the fd-holder, under the control of environment variables. To allow set dump operations, a nonempty file named must exist in the appropriate  subdirectory of 's rules directory, or rules file created from a rules directory.
 * The client program that performs get dump operations is . It accepts options and a UNIX domain socket pathname, and invokes chained to the  program. The socket pathname is passed to, and the options, to .  makes the set dump request over the connection to the server, transferring a subset of its file descriptors via fd-passing under the control of environment variables.

The value of the S6_FD# environment variable is set by to the number of file descriptors received via fd-passing, and used by  to construct the names of other environment variables. The values of environment variables S6_FD_0, S6_FD_1 , ..., S6_FD_${N} , where ${N} is the value of S6_FD# , are set by to the file descriptors received via fd-passing, and used by  to select the file descriptors that will be transferred via fd-passing. The values of environment variables S6_FDID_0, S6_FDID_1 , ..., S6_FDID_${N} are set by to the corresponding file descriptor identifiers, and used by  to specify to the fd-holder the corresponding file descriptor identifiers. The values of environment variables S6_FDLIMIT_0, S6_FDLIMIT_1 , ..., S6_FDLIMIT_${N} are set by to the time remaining for each file descriptor before expiration, for those that have an expiration time set. The values of environment variables S6_FDLIMIT_0, S6_FDLIMIT_1 , ..., S6_FDLIMIT_${N} , if set, are used by to specify to the fd-holder an expiration time for the corresponding file descriptors. For the full description of all s6 fd-holder client programs' functionality please consult the HTML documentation in the package's subdirectory.

Finally, s6 provides an program, and a helper program,, that allow transferring all currently held file descriptors in one fd-holder process to a another one. The transferred set of file descriptors are added in the destination process to the currently held ones. This is implemented by performing a get dump operation on the source process and a set dump operation on the destination process, so they must be allowed by each 's access control policies. handles communications with the fd-holders to make the get dump and set dump requests; its standard input must be redirected to the source process' socket, and its standard output, to the destination  process' socket. accepts options and the socket pathnames of the source and destinations fd-holders, and invokes, using a POSIX  call,  chained to execline's  and  programs, chained to. The socket pathnames are passed to, and the options are used to construct options. and perform all file descriptor manipulations neccesary to move UCSPI socket file descriptors inherited from  to 's standard input and output. For the full description of 's and 's functionality please consult the HTML documentation in the package's subdirectory.

Example s6-rc service definitions in source format for holding the reading end of a FIFO:

is assumed to be a program that prints to its standard output a message of the form 'Message #n', with an incrementing number n between 0 and 9, each time it receives a  signal. Service, a longrun, is a process with its standard output redirected to the  FIFO.

Service, a longrun, is an process that reads from the  FIFO and logs to the  logging directory. Creating the FIFO:

Starting both services, assuming that has been called on the service definitions set to create a compiled services database, that  has been called after, and that the s6 scan directory and the s6-rc live state directory (named ) are both in the same directory:

's  argument increments its verbosity level. Sending three  signals to :

This shows that 's messages were sent over the FIFO to and logged. Stopping momentarily the service:

Sending three more  signals to :

Since the FIFO no longer has any reader, writing to it makes the kernel send a  signal to the  process. Restarting the service and sending three final   signals to :

This shows that the messages sent by when  was not running are effectively lost. This can be avoided by modifying the services so that the reading end of the FIFO is held by an fd-holder process.

Service, a longrun, is an process bound to socket , with readiness notification enabled , and using rules file  in the  subdirectory of its compiled s6 service directory. s6-fdholder-daemon -1 -x data/rules /home/user/fdholder-socket is equivalent to s6-ipcserver-socketbinder /home/user/fdholder-socket s6-fdholderd -1 -x data/rules, but shorter. The rules file is created from the following rules directory using s6-accessrules-cdb-from-fs test-fdholder/data/rules ../rules.d :

This rules directory allows user (assumed to have user ID 1000) to perform store (and therefore delete), retrieve and list operations. The file contains a single empty line.

Service, a oneshot, opens FIFO for reading (in non-blocking mode that is changed to blocking afterwards, using execline's  program with options   and  ) and stores the corresponding file descriptor in the fd-holder using identifier fifo:/home/user/test-fifo. s6-fdholder-store /home/user/fdholder-socket fifo:/home/user/test-fifo and s6-fdholder-delete /home/user/fdholder-socket fifo:/home/user/test-fifo are equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-storec fifo:/home/user/test-fifo and s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-deletec fifo:/home/user/test-fifo, respectively, but shorter. The dependency on service ensures that s6-rc -u change will start  before trying to store to it. When transitions to the down state, it will delete the held file descriptor.

Service, a longrun, is a modified version of that retrieves the FIFO's reading end from the fd-holder. s6-fdholder-retrieve /home/user/fdholder-socket fifo:/home/user/test-fifo is equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-retrievec fifo:/home/user/test-fifo fdclose 6 fdclose 7, but shorter. The dependency on service ensures that s6-rc -u change will have stored the FIFO's reading end first in the fd-holder with the appropriate identifier. Starting the and  services:

Sending three  signals to, stopping momentarily the  service, sending three more   signals to :

This shows that no more messages are being logged as a consequence, but because there is still an open file descriptor held by ), there is no  signal this time. s6-fdholder-list /home/user/fdholder-socket is equivalent to s6-ipcclient -l 0 /home/user/fdholder-socket s6-fdholder-listc, but shorter. Restarting the  service:

This shows that once the file descriptor is retrieved for, all pending messages are delivered and logged. Sending three final  signals to :

This shows that no messages are lost even if the FIFO reader momentarily stops.

Pre-opening sockets before their servers are running
systemd supports a mechanism it calls socket activation, that makes a process pre-open UNIX domain, TCP/IP or Netlink sockets, FIFOs and other special files, and pass them to a child process when a connection is made, a writer opens the FIFO, new data is available for reading, etc. Socket activation is performed when a socket unit file is provided with an accompanying service unit file. This mechanism combines superserver, fd-passing and fd-holding funcionality implemented in process 1, with programs written to communicate using pre-opened file descriptors. Similar behaviour can be achieved, for services that want it, by a combination of individual s6 programs like, , and the fd-holder client programs, without involving process 1. The software package's author notes however that since read operations, and write operations when buffers are full, on a file descriptor held by an fd-holder will block, speed gains might no be that significant, and that, dependending on the scenario (e.g. logging infrastructure), communicating with a service assuming it is ready when it actually isn't might hurt reliability.

Example s6-rc service definitions in source format for pre-opening a UNIX domain socket and pass it to a program written to be executed by a IPC UCSPI server:

Service is the same as in the previous FIFO example.

Service, a oneshot, pre-opens a listening UNIX domain socket bound to using , and stores the corresponding file descriptor in the fd-holder using identifier unix:/home/user/test-socket. The dependency on service ensures that s6-rc -u change will start  before trying to store to it. When transitions to the down state, it will delete the held file descriptor.

Service, a longrun, retrieves the listening socket's file descriptor from the fd-holder, and invokes super-server to handle incoming connections, spawning a  process for each one. is the same program used in the example contained in section "The UNIX domain super-server and related tools". The dependency on service ensures that s6-rc -u change will have stored the listening socket's file descriptor first in the fd-holder with the appropriate identifier. Starting the service:

's  argument increments its verbosity level. Starting a UCSPI client in the background to connect to the server:

is the same program used in the example contained in section "The UNIX domain super-server and related tools". This shows that has been launched with a connection to the server's listening socket, but because its file descriptor is held by  and no server program is currently running, it blocks on an I/O operation.

This shows that is holding the listening socket's file descriptor. Starting the service:

This shows that once has started and retrieved the listening socket's file descriptor from, it accepts the connection and spawns  to handle it.

as process 1
The program was also written to be robust enough and go out of its way to stay alive, even in dire situations, so that it is suitable for running as process 1 during most of a machine's uptime. However, the duties of process 1 vary widely during the machine's boot sequence, its normal, stable 'up and running' state, and its shutdown sequence, and in the first and third cases, they are heavily system-dependent, so it is not possible to use a program designed to be as portable as possible. Because of that, auxiliary and system-dependent programs, named the stage1 init and the stage3 init, are used during the boot sequence and the shutdown sequence, respectively, to run as process 1, and is used the rest of the time. For details, see s6 and s6-rc-based init system.

To support its role as process 1, performs a reaper routine each time it receives a   signal, i.e. it uses a POSIX   call for each child process that becomes a zombie, both the ones it has spawned itself, and the ones that were reparented to process 1 by the kernel because its parent process died. An command naming its scan directory can be used to force  to perform its reaper routine.

OpenRC's s6 integration feature
Starting with version 0.16, OpenRC can launch supervised long-lived processes using the s6 package as a helper. This is an alternative to 'classic' unsupervised long-lived processes launched using the program. It should be noted that service scripts that don't contain  and   functions implicitly use.

OpenRC services that want to use s6 supervision need both a service script in and an s6 service directory. The service script must contain a  variable assignment to turn the feature on, and must have a 'need' dependency on the  service in its   function, to make sure the  program is launched. It can contain neither a  function, nor a   function (but their   and   variants are OK), nor a   function:


 * OpenRC internally invokes with a   option when the service script is called with a 'start' argument, and can also call  after  to wait for an event, by assigning  options to the s6_svwait_options_start variable (e.g. in the service script or the service-specific configuration file in ). For example, if the service supports readiness notification,   could be used to make OpenRC wait for the up and ready event with a 5 seconds timeout.
 * OpenRC internally invokes with ,   and   options when the service script is called with a 'stop' argument, so it will wait for a really down event with a default timeout of 10 seconds. The timeout can be changed by assigning a time value in milliseconds to s6_service_timeout_stop variable (e.g. in the service script or the service-specific configuration file in ).
 * OpenRC internally invokes when the service script is called with a 'status' argument.

The s6 service directory can be placed anywhere in the filesystem, and have any name, as long as the service script (or the service-specific configuration file in ) assigns the servicedir's absolute path to the s6_service_path variable. If s6_service_path is not assigned to, the s6 servicedir must have the same name as the OpenRC service script, and will be searched in. The scan directory when using this feature is, and OpenRC will create a symlink to the service directory when the service is started.

Example setup for a hypothetical supervised test-daemon process with a dedicated logger:

The service directory:

This launches with effective user daemon and the maximum number of open file descriptors set to 5. This is the same as if performed a   call itself with   set to 5, provided that value does not exceed the corresponding hard limit. The program supports an  option to turn readiness notification on, specifying the notification file descriptor (5), and also periodically prints to its standard error a message of the form 'Logged message #n', with an incrementing number n between 0 and 9. The redirection of 's standard error to standard output, using execline's program with the   (copy) option, allows logging its messages using :

An automatically rotated logging directory named logdir will be used, and messages will have a timestamp in external TAI64N format prepended to them.

Manually starting :

This shows that took about 10 seconds to notify readiness to, and that the  command waited until the up and ready event, because of the    option passed via s6_svwait_options_start in.

The scan directory:

The supervision tree:

Messages from the process go to the logging directory:

Unmerge
All scan directories, service directories, the symlink, etc. must be manually deleted if no longer wanted after removing the package. Also, all modifications to sysvinit's must be manually reverted: lines for  must be deleted, and a  command must be used afterwards. And obviously, if is running as process 1, an alternative init system must be installed in parallel, and the machine rebooted to use it (possibly by reconfiguring the bootloader), before the package is removed, or otherwise the machine will become unbootable.

External resources

 * A thread containing posts about s6 on the Gentoo Forums.
 * Tokiclover's supervision framework (OpenRC friendly)
 * Avery Payne's supervision-scripts project, compatible with s6.