zram

This page contains changes which are not marked for translation.

Resources

zram (previously called compcache) is a Linux kernel feature and userspace tools for creating compressible RAM-based block devices. It has been included as a module of the mainline Linux kernel since version 3.14. Starting with kernel version 3.15, zram supports multiple compression streams and the ability to change the compression algorithms without a system restart.

The zram kernel module enables support for creating compressed block devices in RAM. These block devices can then be used for swap or general purpose RAM disks. Popular uses for it on Gentoo are to extend the available amount of RAM to processes (swap space), virtualize /tmp, and /var/tmp/portage - which is Portage's temporary directory used for software compilation.

The RAM used for the block device is dynamically obtained and released up to its predefined uncompressed maximum size. The way it extends the amount of available RAM to a system is by using a portion of the RAM as compressed swap. It can therefore hold more pages of memory in the compressed swap than the amount of actual memory used.

Typically it compresses to a 3:1 ratio. So, 1 GiB of swap uses only 333 MiB of RAM on average. The compression ratio including memory used for disk overhead varies depending on the percent of maximum space utilized. This may typically vary from 1.5:1 for a 1.5 GiB disk with only 5% space used, to over 3:1 when nearly full. It also is much faster at swapping pages than typical speeds for hard disk swap.

To be more memory efficient, zram offers the possibility to configure a backing store. The backing store would store blocks that are not compressible and blocks that are not accessed in a while.

Combining zram with a correctly tuned Portage configuration should keep a desktop system running in a responsive manner, even during intensive software compilation.

Caveats

Prior to kernel 3.15, each zram device contains its own compression buffer, memory pools, and other metadata as well as per-device locks. This can become a serious bottleneck for multi-core machines. To work around this problem, zram is capable of initializing multiple devices. For this reason, the recommended amount of devices for swap is 1 per CPU core for kernels prior to 3.15.

Another caveat for systems with limited memory, non swap use can reduce the amount of available memory to run applications.

When using this with an SSD, avoid setting rc_parallel=YES in /etc/rc.conf (Which starts services in parallel). Depending on the size of the zram partitions and the speed of the RAM, some swap partitions and filesystems might not be ready when the swap and localmount services are started.

In such case, if parallel must be used, consider removing these services from runlevel boot and adding them to default instead.

Enabling zram support in kernel

Enable the following options in the kernel's configuration file:

KERNEL

Device Drivers  --->
    [*] Block devices Search for <code>CONFIG_BLK_DEV</code> to find this item. --->
        <M> Compressed RAM block device support Search for <code>CONFIG_ZRAM</code> to find this item.
        [*] Write back incompressible or idle page to backing device Search for <code>CONFIG_ZRAM_WRITEBACK</code> to find this item.
        [*] Track zRam block status Search for <code>CONFIG_ZRAM_MEMORY_TRACKING</code> to find this item.

It is recommended that zram be built as a loadable module. This allows changing number of zram devices without rebooting, by deactivating zram devices and re-loading module with new parameters. If zram is built-in, then the number of devices can only be changed at boot time by using the kernel boot parameter zram.num_devices=#.

In order to use the LZ4 compression algorithm, enable it in kernel config:

KERNEL

Cryptographic API --->
    Compression --->
        <*> LZ4 Search for <code>CONFIG_CRYPTO_LZ4</code> to find this item.

Snippet

FILE /etc/kernel/config.d/zram-linux6-1-111.config

CONFIG_ZSMALLOC=m
CONFIG_BLK_DEV=y
CONFIG_ZRAM=m
CONFIG_ZRAM_DEF_COMP_LZ4=y
CONFIG_ZRAM_DEF_COMP="lz4"
CONFIG_ZRAM_WRITEBACK=y
CONFIG_ZRAM_MEMORY_TRACKING=y
CONFIG_CRYPTO_LZO=y
CONFIG_CRYPTO_LZ4=y
CONFIG_CRYPTO_ZSTD=y
CONFIG_ZLIB_DEFLATE=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_LZ4_COMPRESS=y
CONFIG_LZ4_DECOMPRESS=y
CONFIG_ZSTD_COMMON=y
CONFIG_ZSTD_COMPRESS=y

Initialization

Using zram-init service

By far the easiest method of utilizing zram disk(s) is by using Martin Väth's zram-init script.

root #emerge --ask sys-block/zram-init

OpenRC

Edit the /etc/conf.d/zram-init file and create/configure the desired zram devices. There are lots of comments/instructions in the file. So proceed with editing and be sure to save it when the appropriate modifications have been made.

Note
Set priority of hard drive swap to low, e.g. via /etc/fstab. For multicore systems, set maxs equal to the number of cores.

An example configuration:

FILE /etc/conf.d/zram-init

load_on_start="yes"
unload_on_stop="yes"
num_devices="3"

type0="swap"
flag0=
size0="4096"
maxs0=2
algo0=lz4

type1="/tmp"
flag1="ext4"
opts1="noatime"
mode1=777
size1="1024"
labl1="zram-tmp"

type2=/var/tmp/portage
flag2=ext4
opts2="noatime"
size2=8192
algo2=zstd
owgr2="portage:portage"
mode2=775
labl2=zram-var-tmp-portage
back2=/dev/mapper/zram-nvme0n1p4
icmp2=y
idle2=600

This example creates three zram devices:

A 4GB swap device, using the lz4 compression algorithm. Do not forget to enable the new swap device with low priority in /etc/fstab.
A 1GB device mounted at /tmp. It is formatted with the ext4 filesystem with mount options noatime and data=writeback for optimal performance.
An 8GB device mounted at /var/tmp/portage with a backing store located at the /dev/mapper/zram-nvme0n1p4 partition for incompressible blocks and for blocks that have not been accessed for 10 minutes. It changes the ownership to user and group portage.

Then, add the init script to the desired runlevel, usually boot, and start the service:

root #

rc-config add zram-init boot

root #

/etc/init.d/zram-init start

In this case the boot runlevel is preferable to the default runlevel because zram is providing temporary storage filesystems at /tmp or /var/tmp which are prerequisites for other services which will start during the default runlevel.

Warning
Be sure to monitor closely for any issues on the computer(s) for a few days when starting in the boot runlevel whilst using /tmp in zram. Additional configuration of services (utilizing rc_need or rc_want in their respective conf.d files) using a path of zram may be required.

For example, service "systemd-tmpfiles-setup" may depend on zram, if zram is providing temporary filesystem. In this case "zram-init" should be started before "systemd-tmpfiles-setup" :

FILE /etc/conf.d/zram-init

rc_before="systemd-tmpfiles-setup"

If the config was changed, restart zram-init:

root #

/etc/init.d/zram-init restart

systemd

The sys-block/zram-init package provides systemd units with self explained names:

zram_btrfs.service
zram_swap.service
zram_tmp.service
zram_var_tmp.service

These units feature default zram device settings which need to be adjusted first.

Service drop in override example

For example, to enable the /var/tmp directory (which includes Portage's temporary directory (/var/tmp/portage) used for compiling packages) in zram it is first required to override the defaults. This can be done using the systemctl edit utility:

root #systemctl edit zram_var_tmp.service

Note
The empty ExecStart= clears the list of ExecStart= commands obtained from the unit being overridden.

FILE /etc/systemd/system/zram_var_tmp.service.d/override.confEdited service drop-in

### Editing /etc/systemd/system/zram_var_tmp.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file

[Service]
ExecStart=
ExecStart=/sbin/zram-init -d2 -s2 -alz4 -text4 -orelatime -m1777 -Lvar_tmp_dir 16384 /var/tmp

Important
The file size should be adjusted to a size large enough to hold the working directories for the packages that will be compiled. Consider adjusting the default value from 2048 to at least 16384 (16GiB). See the dedicated article for estimates of uncompressed space necessary for successful compilation.

After this configuration is finalized, the service can be enabled and activated via:

root #systemctl enable --now zram_var_tmp.service

Created symlink /etc/systemd/system/local-fs-pre.target.wants/zram_var_tmp.service → /etc/systemd/system/zram_var_tmp.service.

zram_var_tmp_portage.service example

It appears the .service file examples are no longer included with the zram-init package, therefore this example should aid in systems that would benefit from exclusively compressing the /var/tmp/portage directory.

FILE /etc/systemd/system/zram_var_tmp_portage.service64 GiB of zstd compressed space example

# SPDX-License-Identifier: GPL-2.0-only
[Unit]
Description=Mount /var/tmp/portage as zram
DefaultDependencies=no
Conflicts=umount.target
Before=local-fs.target umount.target
After=local-fs-pre.target
RequiresMountsFor=/var/tmp

[Service]
Type=oneshot
RemainAfterExit=true

# Adapt the maximal size (in MB) and other options like maximal number of
# parallel streams (-s) as necessary.
# Make sure to avoid collision of devices (-d...) with
# zram_swap.service and zram_tmp.service and
# that num_devices in modprobe.d/zram.conf contains the maximal used number + 1

# 64GiB of zstd compressed space
ExecStart=/sbin/zram-init -d2 -s3 -azstd -text4 -orelatime -m1777 -Lvar_tmp_portage 65536 /var/tmp/portage
ExecStop=/sbin/zram-init -d2 0 /var/tmp/portage

[Install]
WantedBy=local-fs-pre.target

After this configuration is finalized, the service can be enabled and activated via:

root #systemctl enable --now zram_var_tmp_portage.service

Created symlink /etc/systemd/system/local-fs-pre.target.wants/zram_var_tmp_portage.service → /etc/systemd/system/zram_var_tmp_portage.service.

Using systemd zramctl

Zramctl is a part of the systemd suit and can be used for creating zram. Below is an example of creating a 4G zram device using zramctl with lz4 compression. For making a directory the same methods apply.

root #modprobe zram

root #zramctl /dev/zram0 --size 4G --algorithm lz4

root #mkswap /dev/zram0

root #swapon /dev/zram0

Using OpenRC

For manual creation create two /etc/local.d files: zram.start and zram.stop. OpenRC will run these during the appropriate as part of service run process when booting or changing run levels.

An example:

Specs: 4 cpu cores, 4G RAM total
Configure 6G zram swap and activate.
Estimated maximum ram used 2G @ 3:1 compression ratio.

FILE /etc/local.d/zram.start

#!/bin/bash

modprobe zram

SIZE=6144
echo $(($SIZE*1024*1024)) > /sys/block/zram0/disksize

mkswap /dev/zram0

swapon /dev/zram0 -p 10

FILE /etc/local.d/zram.stop

#!/bin/bash

swapoff /dev/zram0

echo 1 > /sys/block/zram0/reset

modprobe -r zram

Note
Disksize may also be specified using mem suffixes (K, M, G): echo 6144M > /sys/block/zram0/disksize

Now make both of these files executable:

root #chmod +x /etc/local.d/zram.start /etc/local.d/zram.stop

For manual installation of zram mounted in /var/tmp/portage, the following script can be used to create 4G of zram with lzo compression:

CODE

#!/bin/bash

modprobe zram

echo 4G > /sys/block/zram0/disksize
echo lzo > /sys/block/zram0/comp_algorithm
mkfs.ext4 /dev/zram0
mount /dev/zram0 /var/tmp/portage
chown -R portage:portage /var/tmp/portage

Using systemd zram-generator

systemd provides an external package for configuring zram devices. To install it, run:

root #emerge --ask sys-apps/zram-generator

For the simplest zram swap configuration, simply define an empty zram device section in the config:

root #echo '[zram0]' > /etc/systemd/zram-generator.conf

This will configure a zram swap device 4GB in size or half the RAM, whichever is smaller, to start on early boot. To start it immediately, run:

root #

systemctl daemon-reload

root #

systemctl start dev-zram0.swap

The configuration directory can also be .d'ed:

root #

mkdir /etc/systemd/zram-generator.conf.d

root #

mv /etc/systemd/zram-generator.conf /etc/systemd/zram-generator.conf.d/zram0-swap.conf

For all available settings and the defaults, refer to the man page:

user $

man zram-generator.conf

A more complete example for zram-swap would be:

FILE /etc/systemd/zram-generator.conf.d/zram0-swap.conf

[zram0]
zram-size = ram / 2
compression-algorithm = zstd
swap-priority = 100
fs-type = swap

Note that on systems with up to 8GB RAM, those settings are basically the same as the very simple configuration with just the line [zram0], because in the default setting half of the systems RAM is used, but not more than 4GB. Depending on the kernel configuration, the compression algorithm zstd could also be what the system defaults to. For all fs-types, including swap, options default to discard.

On a modern and fast system, zram is also a viable replacement for tmpfs. Especially text files, like logs or source code, are very compressible. Unlike tmpfs, which is its own filesystem, on a zram device it has to be created manually using standard tools, i.e. any file system available on the user's current Gentoo Linux can be used. Bear in mind that complex file systems waste additional space, like for journals, with little to no gain for the in-ram use-case. ext2 doesn't have a journal, but it's much older than e.g. ext4 and has some limitations, e.g. it's affected by the year 2038 problem. ext4 on the other hand defaults to being created with a journal. However, for all extended filesystems a new fs-type can be added to /etc/mke2fs.conf. It suffices to just copy the ext4 lines and only remove the has_journal option, and to give the new fs-type a unique name, like ext4withoutjournal:

FILE /etc/mke2fs.conf

[fs_types]
        ext4 = {
                features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
        }
        ext4withoutjournal = {
                features = extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
        }

The zram-generator will create a filesystem by running mkfs.*, where * stands for the fs-type as specified in the zram configuration file. In order for this new fs-type ext4withoutjournal to be available in this way, make a new mkfs.fs-type link to mke2fs:

root #

ln -s mke2fs /sbin/mkfs.ext4withoutjournal

The mount options are specific to each filesystem and wrong options will cause systemd-zram-generator to fail. Since options to set a default user (uid/gid) or mode, like for tmpfs, are not available for ext4, systemd's X-mount.* statements have to be used.

To setup /tmp as a zram device of 4GB:

FILE /etc/systemd/zram-generator.conf.d/zram1-tmp.conf

[zram1]
# use compressed RAM (zram) as system /tmp

## SIZE
# use 1/16 of physical RAM (e.g. on a system with 64 GB RAM: ~4 GB)
#zram-size = ram / 16
# fixed size:
zram-size = 4096m

## COMPRESSION
# slowest but best compression: zstd
# fast: lzo
# faster: lz4
compression-algorithm = zstd

## MOUNT-POINT
mount-point = /tmp

## MOUNT OPTIONS
#fs-type = ext2
fs-type = ext4withoutjournal
options = rw,nosuid,nodev,discard,X-mount.mode=1777

In a very similar manner and as an alternative to Portage TMPDIR on tmpfs, /var/tmp/portage can be setup as a zram device:

FILE /etc/systemd/zram-generator.conf.d/zram2-portage-tmpdir.conf

[zram2]
# use compressed RAM (zram) as portage TMPDIR /var/tmp/portage
# (make sure TMPDIR in /etc/make.conf matches the mount-point below)

## SIZE
# use 1/2 of physical RAM (e.g. on a system with 64 GB RAM: ~32 GB)
zram-size = ram / 2
# use 3/4 of physical RAM (e.g. 64 GB: ~48 GB)
#zram-size = ram / 4 * 3
# use 7/8 of physical RAM (e.g. 64 GB: ~56 GB)
#zram-size = ram / 8 * 7
# fixed size:
#zram-size = 32G

## COMPRESSION
# slowest but best compression: zstd
# fast: lzo
# faster: lz4
compression-algorithm = zstd

## MOUNT-POINT TMPDIR
mount-point = /var/tmp/portage

## MOUNT OPTIONS
#fs-type = ext2
fs-type = ext4withoutjournal
options = rw,nosuid,nodev,discard,X-mount.owner=portage,X-mount.group=portage,X-mount.mode=775

For more non-swap configurations, see zram-generator's README.

Using udev

Other possibility is to use existing configuration files - this option works on vanilla Gentoo without need to install additional software, also useful if using systemd instead of OpenRC. The first example can be implemented using:

FILE /etc/udev/rules.d/10-zram.rules

KERNEL=="zram0", SUBSYSTEM=="block", DRIVER=="", ACTION=="add", ATTR{disksize}=="0", ATTR{disksize}="512M", RUN+="/sbin/mkswap $env{DEVNAME}"
KERNEL=="zram1", SUBSYSTEM=="block", DRIVER=="", ACTION=="add", ATTR{disksize}=="0", ATTR{disksize}="512M", RUN+="/sbin/mkswap $env{DEVNAME}"
KERNEL=="zram2", SUBSYSTEM=="block", DRIVER=="", ACTION=="add", ATTR{disksize}=="0", ATTR{disksize}="512M", RUN+="/sbin/mkfs.ext4 $env{DEVNAME}"
# if you want lz4 support (since kernel 3.15) and without ext4 journaling 
# KERNEL=="zram2", SUBSYSTEM=="block", DRIVER=="", ACTION=="add", ATTR{initstate}=="0", ATTR{comp_algorithm}="lz4", ATTR{disksize}="512M", RUN+="/sbin/mkfs.ext4 -O ^has_journal -L $name $env{DEVNAME}"

FILE /etc/fstab

/dev/zram0              swap                    swap            pri=16383                                                       0 0
/dev/zram1              swap                    swap            pri=16383                                                       0 0
/dev/zram2              /tmp                    ext4            defaults                                                        0 0

FILE /etc/modprobe.d/zram.conf

options zram num_devices=3

Additionally, the zramctl utility is part of sys-apps/util-linux and can be used to configure zram devices. See man zramctl for examples of usage.

systemd

If using systemd with this method, ensure that the zram module is loaded by systemd. The simplest way to achieve that is to include a file in /etc/modules-load.d/ like this one:

FILE /etc/modules-load.d/zram.conf

zram

Checking that zram is used

Check if zram is mounted as swap:

user $grep zram /proc/swaps

/dev/zram0                              partition       2097148 2816    16383

Check if zram is mounted as directories:

user $grep zram /proc/mounts

/dev/zram1 /var/tmp/portage ext4 rw,nosuid,nodev,block_validity,discard,delalloc,barrier,user_xattr,acl 0 0

Troubleshooting

OpenRC: zram already mounted or mount point busy

Problem: "* Mounting local filesystems ... " doesn't wait for mkfs.ext4 to finish, thus zram will fail to mount at boot time.

Solution: Set rc_need="udev-settle" in /etc/conf.d/localmount.

This situation occurs when using mkfs.ext4 in /etc/udev/rules.d/10-zram.rules. It is caused by mkfs.ext4 being not finished running when /etc/init.d/localmount runs (even with rc_parallel="NO" in /etc/rc.conf) and thus causes mount with fail with mount: /tmp: /dev/zram1 already mounted or mount point busy.

External resources

zram in official kernel documentation.

zram

Contents

Caveats

Enabling zram support in kernel

Snippet

Initialization

Using zram-init service

OpenRC

systemd

Service drop in override example

zram_var_tmp_portage.service example

Using systemd zramctl

Using OpenRC

Using systemd zram-generator

Using udev

systemd

Checking that zram is used

Troubleshooting

OpenRC: zram already mounted or mount point busy

See also

External resources