Project:Sound/How to Enable Realtime for Multimedia Applications

What is realtime support?
Some multimedia applications, like digital audio recording, processing and playback, as well as video playback, require some degree of realtime processing. Think for instance of a musician playing with a MIDI instrument connected to a software synthesizer: if the timing is off, or the latency is too high, it will be difficult to get the proper result even when the instrument is played correctly.

To allow proper multimedia application usage on Linux, there are a few methods that enable realtime support for users' processes and threads, so that important processes like JACK can run with the highest priority available, without having other processes getting in their way.

Please note that allowing processes to run at realtime priority will let them make your system unusable, either because of bugs in the code, or malicious code that locks memory or executes instructions that end up using all the CPU power. For this reason, enabling realtime processing will require reducing security a notch on the system.

Do I need realtime support?
If your idea of multimedia is playing songs, and watching movies while doing a few other tasks such as browsing the web, reading and writing emails, you probably don't need realtime support.

Realtime processing support is important for people who actually need the maximum priority for multimedia process; this includes people who record audio or video (without losing data) from analog input, people who play with MIDI musical instruments and set-top boxes wherein video and audio playback is the principal task.

In general, if you're satisfied with your multimedia applications as they are, you likely don't need realtime support, and you might mess up your system a whole lot if you try to force it.

A different class of users is embedded users who need realtime processing of signals or input from the users for low latency appliances. These users need rather different setups than most of the multimedia uses of realtime.

The different options for realtime
Since the realtime scheduler is already built into the recent 2.6 kernels, the only thing that needs to be done is to give users the ability to effectively execute processes or threads with realtime priority. This is because, as said earlier, enabling realtime priority is a security issue, and shouldn't be done lightly.

The aim of this guide is to set up a realtime group whose members can use realtime priority. Some software, like PulseAudio, expects to find such a setup to allow users to make use of realtime support. Also, by creating its own group, the admin does not have to allow realtime scheduling for more users than needed.

There are two ways to setup and use realtime:

One of the most common options described in guides for various multimedia software is to use, a kernel module that can enable either the whole system or a given group to use realtime priority, and at the same time disable memory locking to reduce the chances of a DoS attack. Although this option is very easy to use, it's currently deprecated, as the kernel developers suggest to use the Linux rlimits instead.

The other option is to use the  PAM module that ships with Linux-PAM, that allows realtime priority for particular users or groups. The disadvantage of this is that there is no default configuration available, it has to be configured from scratch.

The need for capabilities
Some of the software that supports realtime priority can make use of the Linux system capabilities to reduce the chances that bugs or malicious code (in plugins) might misuse the improved privileges it was given. Enabling these features is quite important, so it's recommended to enable support for them by enabling the caps USE flag in either  or for the software that will run with realtime priority through.

On a kernel level, capabilities are enabled by default, but they can be disabled, so before starting, make sure that the support is at least enabled as a module (as you'll see later, for  to work they cannot be built-in, they have to be built as a module).

Option 1: default configuration

The default configuration is good enough for most cases; if you really need to enable support for extra security modules, you should then also enable the capabilities, or they will not work:

Option 2: explicitly enable capabilities As stated earlier, for  to work you must make capabilities a module, or   will not work at all. In that case you not only need to configure the kernel properly:

Option 3: make capabilities a module

However, you also need to insert the module in the kernel automatically, otherwise  or any other software that expects capabilities to be supported will simply fail. To do so, just add capability in a non-commented line in /etc/modules.autoload.d/kernel-2.6.

Creating a realtime group
This step is not mandatory; specifically if you're going to use the last version of, the group will be created for you; the group will also be created by installing newer versions of PulseAudio ebuild (0.9.6-r1; see the notes at the end of the document). Creating the user here won't hurt anyway, as the ebuilds will gladly accept the manually-created group.

It's more important to add your own user to the realtime group, so that it can use the feature you're enabling. Depending on your setup, it might be your normal user account, or a specifically-created account that you only use to use the multimedia applications requiring realtime support. In the example we'll use the musician user, which you should replace with your own user.

Create the realtime group, and add a user to it

Option 1: Using Linux rlimits with PAM
The recommended way to enable realtime priority support for Linux on recent kernels (2.6 series) is to use rlimits to enable a user or a group to allow them the use of the realtime priority.

Linux rlimits contain two values for limit:, one is the hard limit, the other is the soft limit. The hard limit describes the maximum value the user can request (through ), while the soft limit (that always has to be lower than the other one) defines the default value given to the user's (or group's) processes.

To set the rlimit for realtime priority, at the moment, the only choice is to use the pam_limits module, shipped with Linux-PAM : when this module is executed, the limits are read from  and set accordingly.

The default configuration for PAM-based logins already execute the limit, both with console login and with graphical logins like XDM and similar. On the other hand, to make sure the limits are respected by the services started through init scripts and by processes run via cron, the pam USE flag should be enabled on the two respective packages ( and at least for  ).

The name of the rlimit is rtprio and can be seen by using the  command. By default every user has a hard (and thus also soft) limit of 0. The value of the rtprio limit is the maximum realtime priority a process can get: the higher the value, the more priority the process is given; for most multimedia uses, a value of 10 is usually suitable, and is also the default requested by JACK.

To change the default values (that don't allow to use realtime priority to users), you have to edit the file, adding the following lines:

Allowing the realtime group to use realtime priority

The @ sign in the front of realtime tells  that the limit has to be applied to a group, rather than an user. The limits in the above example are set so that the user gets a default realtime priority limit of 10 (which means that software like JACK will just work out of the box), while allowing the users to temporarily change their limit to 20 for special cases and testing.

Depending on the needs and the setup you're using, you might need to tweak the limit values to set an higher realtime priority default.

Using realtime-lsm
Now, we'll see how to configure  to allow realtime priority. This is the most commonly-documented way to get this feature, although it's not the preferred way to do it. The advantage of  is that configuration is almost automatic, and works even when not using PAM and on any login method (direct, SSH, services running since boot in the system, ...).

The first step is to enable capabilities as a module, which is a requirement for this method to work, as stated earlier, and make sure to add capability to the list of automatically loaded modules in /etc/modules.autoload.d/kernel-2.6.

Kernel configuration to make capabilities a module

The next step is to install. The immediate next step depends vastly on the version of  you'll be using. The last version, at the time of writing this guide, is 0.8.5-r2, which already provides usable defaults, identical to the ones that this guide will provide. For older versions, you'll have to create the  group manually, following the Creating a realtime group section, and then configure the module by hand.

Sample configuration for the realtime-lsm module

The gid parameter should point to the numerical group id of the  group. To find out which one it is, just use the  command, and look at the third entry in the list:

PulseAudio
PulseAudio is a non-professional audio daemon; the design of this software still allows for meaningful use of realtime priority.

As PulseAudio can run either with a system-wide instance, that runs under its own user, or as multiple instances launched by the user as needed, the code detects the ability to use realtime priority by checking the availability of a realtime group. As up to now there was no official way to handle this, up to version 0.9.6, the ebuild created a  group, and assumed that being the realtime group.

As of version 0.9.6-r1, the  group has been replaced with the   group discussed in this guide, so following this guide and installing a new version of PulseAudio should be enough to make it work, with a couple tweaks that will be now described.

The first important thing is to make sure that the caps USE flag is enabled for PulseAudio: this will make sure that when run with enhanced privileges, the ones that are not needed anymore can be dropped right away. To verify that PulseAudio has capabilities enabled:

Then, if you run a system-wide instance, you have to add the  user (which PulseAudio runs under) to the   group you created, so that it can make proper use of realtime priority. To add the pulse user to the realtime group:

At the next restart of the  service, it will be executed with realtime priority to reduce playback latency.