Project:Sound/How to Enable Realtime for Multimedia Applications
Introduction
What is realtime support?
Some multimedia applications, like digital audio recording, processing and playback, as well as video playback, require some degree of realtime processing. Think for instance of a musician playing with a MIDI instrument connected to a software synthesizer: if the timing is off, or the latency is too high, it will be difficult to get the proper result even when the instrument is played correctly.
To allow proper multimedia application usage on Linux, there are a few methods that enable realtime support for users' processes and threads, so that important processes like JACK can run with the highest priority available, without having other processes getting in their way.
Please note that allowing processes to run at realtime priority will let them make your system unusable, either because of bugs in the code, or malicious code that locks memory or executes instructions that end up using all the CPU power. For this reason, enabling realtime processing will require reducing security a notch on the system.
Do I need realtime support?
If your idea of multimedia is playing songs, and watching movies while doing a few other tasks such as browsing the web, reading and writing emails, you probably don't need realtime support.
Realtime processing support is important for people who actually need the maximum priority for multimedia process; this includes people who record audio or video (without losing data) from analog input, people who play with MIDI musical instruments and set-top boxes wherein video and audio playback is the principal task.
In general, if you're satisfied with your multimedia applications as they are, you likely don't need realtime support, and you might mess up your system a whole lot if you try to force it.
A different class of users is embedded users who need realtime processing of signals or input from the users for low latency appliances. These users need rather different setups than most of the multimedia uses of realtime.
The different options for realtime
Since the realtime scheduler is already built into the recent 2.6 kernels, the only thing that needs to be done is to give users the ability to effectively execute processes or threads with realtime priority. This is because, as said earlier, enabling realtime priority is a security issue, and shouldn't be done lightly.
The aim of this guide is to set up a realtime group whose members can use realtime priority. Some software, like PulseAudio, expects to find such a setup to allow users to make use of realtime support. Also, by creating its own group, the admin does not have to allow realtime scheduling for more users than needed.
Variations on this guide can tighten security a bit, but might require further fiddling with the configuration of other software that would be able to use these features.
There are two ways to setup and use realtime:
realtime-lsmpam
One of the most common options described in guides for various multimedia software is to use realtime-lsm, a kernel module that can enable either the whole system or a given group to use realtime priority, and at the same time disable memory locking to reduce the chances of a DoS attack. Although this option is very easy to use, it's currently deprecated, as the kernel developers suggest to use the Linux rlimits instead.
The other option is to use the pam_limits PAM module that ships with Linux-PAM (sys-libs/pam), that allows realtime priority for particular users or groups. The disadvantage of this is that there is no default configuration available, it has to be configured from scratch.
Configuration
The need for capabilities
Some of the software that supports realtime priority can make use of the Linux system capabilities to reduce the chances that bugs or malicious code (in plugins) might misuse the improved privileges it was given. Enabling these features is quite important, so it's recommended to enable support for them by enabling the caps USE flag in either /etc/make.conf or for the software that will run with realtime priority through /etc/portage/package.use.
On a kernel level, capabilities are enabled by default, but they can be disabled, so before starting, make sure that the support is at least enabled as a module (as you'll see later, for realtime-lsm to work they cannot be built-in, they have to be built as a module).
Security options --->
[ ] Enable different security models
The default configuration is good enough for most cases; if you really need to enable support for extra security modules, you should then also enable the capabilities, or they will not work:
Security options --->
[*] Enable different security models
<*> Default Linux Capabilities
As stated earlier, for realtime-lsm to work you must make capabilities a module, or realtime-lsm will not work at all. In that case you not only need to configure the kernel properly:
Security options --->
[*] Enable different security models
<M> Default Linux Capabilities
However, you also need to insert the module in the kernel automatically, otherwise avahi or any other software that expects capabilities to be supported will simply fail. To do so, just add capability in a non-commented line in /etc/modules.autoload.d/kernel-2.6.
If you build capabilities as a module, it's really important to put the capability module in autoload, otherwise services might fail to start at boot, or might start with reduced security options.
Since Linux kernel version 2.6.24, capabilities can't be compiled as a module anymore. For this reason, as it will be repeated soon, you won't be able to use
realtime-lsm anymore.Creating a realtime group
This step is not mandatory; specifically if you're going to use the last version of realtime-lsm, the group will be created for you; the group will also be created by installing newer versions of PulseAudio ebuild (0.9.6-r1; see the notes at the end of the document). Creating the user here won't hurt anyway, as the ebuilds will gladly accept the manually-created group.
It's more important to add your own user to the realtime group, so that it can use the feature you're enabling. Depending on your setup, it might be your normal user account, or a specifically-created account that you only use to use the multimedia applications requiring realtime support. In the example we'll use the musician user, which you should replace with your own user.
# groupadd realtime
# usermod -a -G realtime musician
Option 1: Using Linux rlimits with PAM
As mentioned earlier, there are two options for getting realtime to work,
pam and realtime-lsm. If you don't want to use pam, skip down to Option 2: Using realtime-lsm.The recommended way to enable realtime priority support for Linux on recent kernels (2.6 series) is to use rlimits to enable a user or a group to allow them the use of the realtime priority.
Linux rlimits contain two values for limit:, one is the hard limit, the other is the soft limit. The hard limit describes the maximum value the user can request (through ulimit), while the soft limit (that always has to be lower than the other one) defines the default value given to the user's (or group's) processes.
To set the rlimit for realtime priority, at the moment, the only choice is to use the pam_limits module, shipped with Linux-PAM (sys-libs/pam): when this module is executed, the limits are read from /etc/security/limits.conf and set accordingly.
The default configuration for PAM-based logins already execute the limit, both with console login and with graphical logins like XDM and similar. On the other hand, to make sure the limits are respected by the services started through init scripts and by processes run via cron, the pam USE flag should be enabled on the two respective packages (sys-apps/baselayout and at least for sys-process/vixie-cron). It is also required for the sys-apps/shadow package in order to enable pam in /bin/login.
The pam USE flag in baselayout only affects the behaviour of
start-stop-daemon, enabling the use of pam_limits. Unfortunately, not every service can be started through that command, which means that for those services, the limits configuration will not apply and instead the default values will be used.The name of the rlimit is rtprio and can be seen by using the ulimit -r command. By default every user has a hard (and thus also soft) limit of 0. The value of the rtprio limit is the maximum realtime priority a process can get: the higher the value, the more priority the process is given; for most multimedia uses, a value of 10 is usually suitable, and is also the default requested by JACK.
To change the default values (that don't allow to use realtime priority to users), you have to edit the file /etc/security/limits.conf, adding the following lines:
* hard rtprio 0
* soft rtprio 0
@realtime hard rtprio 20
@realtime soft rtprio 10
The @ sign in the front of realtime tells pam_limits that the limit has to be applied to a group, rather than an user. The limits in the above example are set so that the user gets a default realtime priority limit of 10 (which means that software like JACK will just work out of the box), while allowing the users to temporarily change their limit to 20 for special cases and testing.
The first two lines are really needed, otherwise when changing user from a realtime-enabled user to a limited one through
su or sudo, the limit will not be applied. This is a design constrain of pam_limits.Depending on the needs and the setup you're using, you might need to tweak the limit values to set an higher realtime priority default.
Using realtime-lsm
Since Linux kernel version 2.6.24 this is no longer an option, as capabilities can't be built into a module, and external security modules can't be compiled anymore. This option is still documented for older versions of the kernel, for people not using PAM, but it cannot be supported anymore.
Now, we'll see how to configure realtime-lsm to allow realtime priority. This is the most commonly-documented way to get this feature, although it's not the preferred way to do it. The advantage of realtime-lsm is that configuration is almost automatic, and works even when not using PAM and on any login method (direct, SSH, services running since boot in the system, ...).
The first step is to enable capabilities as a module, which is a requirement for this method to work, as stated earlier, and make sure to add capability to the list of automatically loaded modules in /etc/modules.autoload.d/kernel-2.6.
Security options --->
[*] Enable different security models
<M> Default Linux Capabilities
The next step is to install sys-apps/realtime-lsm. The immediate next step depends vastly on the version of realtime-lsm you'll be using. The last version, at the time of writing this guide, is 0.8.5-r2, which already provides usable defaults, identical to the ones that this guide will provide. For older versions, you'll have to create the realtime group manually, following the Creating a realtime group section, and then configure the module by hand.
options realtime gid=1017
The gid parameter should point to the numerical group id of the realtime group. To find out which one it is, just use the getent command, and look at the third entry in the list:
user $getent group realtimerealtime:x:1017:musician
The numerical group id will be quite different if you added the group by hand: 1017 is a possible value for a group when added by an ebuild.
Applications notes
PulseAudio
PulseAudio (media-sound/pulseaudio) is a non-professional audio daemon; the design of this software still allows for meaningful use of realtime priority.
As PulseAudio can run either with a system-wide instance, that runs under its own user, or as multiple instances launched by the user as needed, the code detects the ability to use realtime priority by checking the availability of a realtime group. As up to now there was no official way to handle this, up to version 0.9.6, the ebuild created a pulse-rt group, and assumed that being the realtime group.
As of version 0.9.6-r1, the pulse-rt group has been replaced with the realtime group discussed in this guide, so following this guide and installing a new version of PulseAudio should be enough to make it work, with a couple tweaks that will be now described.
The first important thing is to make sure that the caps USE flag is enabled for PulseAudio: this will make sure that when run with enhanced privileges, the ones that are not needed anymore can be dropped right away. To verify that PulseAudio has capabilities enabled:
user $emerge -pv pulseaudioThese are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] media-sound/pulseaudio-0.9.6-r1 USE="-X alsa avahi caps hal -jack -lirc -oss -tcpd" 0 kB
Then, if you run a system-wide instance, you have to add the pulse user (which PulseAudio runs under) to the realtime group you created, so that it can make proper use of realtime priority. To add the pulse user to the realtime group:
user $usermod -a -G realtime pulseAt the next restart of the pulseaudio service, it will be executed with realtime priority to reduce playback latency.