LVM

LVM (Logical Volume Manager) Article description::allows administrators to create meta devices that provide an abstraction layer between a file system and the physical storage that is used underneath. The meta devices (on which file systems are placed) are logical volumes, which use storage from storage pools called volume groups. A volume group is provisioned with one or more physical volumes which are the true devices on which the data is stored.

Physical volumes can be partitions, whole hard drives grouped as JBOD (Just a Bunch Of Disks), RAID systems, iSCSI, Fibre Channel, eSATA etc.

Installation
LVM is handled by both kernel-level drivers and user space applications to manage the LVM configuration.

Kernel
Activate the following kernel options:

Emerge
After reviewing the USE flags, ask Portage to install the package:

Configuration
Configuring LVM is done on several levels:
 * LV, PV and VG management through the management utilities;
 * 1) LVM subsystem fine-tuning through the configuration file;
 * 2) Service management at the distribution level;
 * 3) Setup through an initial ram file system (initramfs).

Management of the logical and physical volumes as well as the volume groups is handled through the Usage chapter.

LVM configuration file
LVM has an extensive configuration file at. Most users will not need to modify settings in this file in order to start using LVM.

Service management
Gentoo provides the LVM service to automatically detect and activate the volume groups and logical volumes.

The service can be managed through the init system.

openrc
To start LVM manually:

To start LVM at boot time:

systemd
To start lvm manually:

To start LVM at boot time:

Using LVM in an initramfs
Most bootloaders cannot boot from LVM directly - neither GRUB legacy nor LILO can. GRUB2 can boot from an LVM linear logical volume, mirrored logical volume and possibly some kinds of RAID logical volumes. No bootloader currently supports thin logical volumes.

For that reason, it is recommended to use a non-LVM /boot partition and mount the LVM root from an initramfs. An initramfs can be generated automatically through genkernel, dracut or manually as Custom Initramfs:


 * can boot from all types except thin volumes (as it neither builds nor copies the binaries from the build host) and maybe RAID10 (RAID10 support requires LVM2 2.02.98, but genkernel builds 2.02.89, however if static binaries are available it can copy those);
 * should boot all types, but only includes thin support in the initramfs if the host being run on has a thin root.

Genkernel
Emerge either. The static USE flag may also be enabled on the package so that genkernel will use the system binaries (otherwise it will build its own private copy). The following example will build only an initramfs (not an entire kernel) and enable support for LVM.

The genkernel manpage outlines other options depending on system requirements.

The initrd will require parameters to tell it how to start LVM, and they are supplied the same way as other kernel parameters. For example:

Dracut
The package was ported from the RedHat project and serves a similar tool for generating an initramfs. For details, please refer to Dracut. Generally, the following command will generate a usable default initramfs.

The initrd will require parameters to tell it how to start LVM, and they are supplied the same way as other kernel parameters. For example:

For a comprehensive list of LVM options within please see the section in the Dracut Manual.

Usage
LVM organizes storage in three different levels as follows:
 * hard drives, partitions, RAID systems or other means of storage are initialized as physical volumes (PVs)
 * Physical Volumes (PV) are grouped together in Volume Groups (VG)
 * Logical Volumes (LV) are managed in Volume Groups (VG)

PV (Physical Volume)
Physical Volumes are the actual hardware or storage system LVM builds up upon.

Partitioning
The partition type for LVM is 8e (Linux LVM).

For instance, to set the type through for a partition on :

In, create partitions using the key and then change the partition type with the  key to 8e.

Create PV
Physical volumes can be created / initialized with the command.

For instance, the following command creates a physical volume on the first primary partition of and :

List PV
With the command, an overview of all active physical volumes on the system can be obtained.

If more physical volumes should be displayed, then can detect inactive physical volumes and activate those.

Remove PV
LVM automatically distributes the data onto all available physical volumes (unless told otherwise) but in a linear approach. If a requested logical volume (within a volume group) is smaller than the amount of free space on a single physical volume, then all space for the logical volume is claimed on that (single) physical volume in a contiguous manner. This is done for performance reasons.

If a physical volume needs to be removed from a volume group, the data first needs to be moved away from the physical volume. With the command, all data on a physical volume is moved to other physical volumes within the same volume group.

Such an operation can take a while depending on the amount of data that needs to be moved. Once finished, there should be no data left on the device. Verify with pvdisplay that the physical volume is no longer used by any logical volume.

The next step is to remove the physical volume from the volume group using after which the device can be "deselected" as a physical volume using pvremove:

VG (Volume Group)
A volume group (VG) groups a number of physical volumes and show up as in the device file system. The name of a volume group is chosen by the administrator.

Create VG
The following command creates a volume group called vg0 with two physical volumes assigned to it: and.

List VG
To list all active volume groups, use the command:

If volume groups are missing, use the command to locate volume groups:

Extend VG
Volume groups group physical volumes, allowing administrators to use a pool of storage resources to allocate to file systems. When a volume group does not hold enough storage resources, it is necessary to extend the volume group with additional physical volumes.

The next example extends the volume group vg0 with a physical volume at :

Remember that the physical volume first needs to be initialized as such!

Reduce VG
If physical volumes need to be removed from the volume group, all data still in use on the physical volume needs to be moved to other physical volumes in the volume group. As seen before, this is handled through the command, after which the physical volume can be removed from the volume group using vgreduce:

Remove VG
If a volume group is no longer necessary (or, in other words, the storage pool that it represents is no longer used and the physical volumes in it need to be freed for other purposes) then the volume group can be removed with vgremove. This only works if no logical volume is defined for the volume group, and all but one physical volume have already been removed from the pool.

LV (Logical Volume)
Logical volumes are the final meta devices which are made available to the system, usually to create file systems on. They are created and managed in volume groups and show up as. Like with volume groups, the name used for a logical volume is decided by the administrator.

Create LV
To create a logical volume, the command is used. The parameters to the command consist out of the requested size for the logical volume (which cannot be larger than the amount of free space in the volume group), the volume group from which the space is to be claimed and the name of the logical volume to be created.

In the next example, a logical volume named lvol1 is created from the volume group named vg0 and with a size of 150MB:

It is possible to tell to use all free space inside a volume group. This is done through the  option which selects the amount of extents rather than a (human readable) size. Logical volumes are split into logical extents which are data chunks inside a volume group. All extents in a volume group have the same size. With the  option  can be asked to allocate all free extents:

Next to FREE the VG key can be used to denote the entire size of a volume group.

List LV
To list all logical volumes, use the command:

If logical volumes are missing, then the command can be used to scan for logical volumes on all available volume groups.

Extend LV
When a logical volume needs to be expanded, then the command can be used to grow the allocated space for the logical volume.

For instance, to extend the logical volume lvol1 to a total of 500 MB:

It is also possible to use the size to be added rather than the total size:

An extended volume group does not immediately provide the additional storage to the end users. For that, the file system on top of the volume group needs to be increased in size as well. Not all file systems allow online resizing, so check the documentation for the file system in question for more information.

For instance, to resize an ext4 file system to become 500MB in size:

For certain file systems, lvreseize extends the logical volume and the file system in one step. For instance, to extend the logical volume lvol1 and resize the ext4 file system:

Reduce LV
If a logical volume needs to be reduced in size, first shrink the file system itself. Not all file systems support online shrinking.

For instance, ext4 does not support online shrinking so the file system needs to be unmounted first. It is also recommended to do a file system check to make sure there are no inconsistencies:

With a reduced file system, it is now possible to reduce the logical volume as well:

LV Permissions
LVM supports permission states on the logical volumes.

For instance, a logical volume can be set to read only using the command:

The remount is needed as the change is not enforced immediately.

To mark the logical volume as writable again, use the rw permission bit:

Remove LV
Before removing a logical volume, make sure it is no longer mounted:

Deactivate the logical volume so that no further write activity can take place:

With the volume unmounted and deactivated, it can now be removed, freeing the extents allocated to it for use by other logical volumes in the volume group:

Features
LVM provides quite a few interesting features for storage administrators, including (but not limited to)
 * thin provisioning (over-committing storage)
 * snapshot support
 * volume types with different storage allocation methods

Thin provisioning
Recent versions of LVM2 (2.02.89) support "thin" volumes. Thin volumes are to block devices what sparse files are to file systems. Thus, a thin logical volume within a pool can be "over-committed": its presented size can be larger than the allocated size - it can even be larger than the pool itself. Just like a sparse file, the extents are allocated as the block device gets populated. If the file system has discard support extents are freed again as files are removed, reducing space utilization of the pool.

Within LVM, such a thin pool is a special type of logical volume, which itself can host logical volumes.

Creating a thin pool
Each thin pool has metadata associated with it, which is added to the thin pool size. LVM will compute the size of the metadata based on the size of the thin pool as the minimum of pool_chunks * 64 bytes or 2MiB, whichever is larger. The administrator can select a different metadata size as well.

To create a thin pool, add the  options to :

The above example creates a thin pool called thin_pool with a total size of 150 MB. This is the real allocated size for the thin pool (and thus the total amount of actual storage that can be used).

To explicitly ask for a certain metadata size, use the  option:

Due to the metadata that is added to the thin pool, the intuitive way of using all available size in a volume group for a logical volume does not work (see LVM bug |812726):

Note the thin pool does not have an associated device node like other LV's.

Creating a thin logical volume
A thin logical volume is a logical volume inside the thin pool (which itself is a logical volume). As thin logical volumes are sparse, a virtual size instead of a physical size is specified using the  option:

In this example, the (thin) logical volume lvol1 is exposed as a 300MB-sized device, even though the underlying pool only holds 150MB of real allocated storage.

It is also possible to create both the thin pool as well as the logical volume inside the thin pool in one command:

Listing thin pools and thin logical volumes
Thin pools and thin logical volumes are special types of logical volumes, and as such as displayed through the command. The command will also detect these logical volumes.

Extending a thin pool
The thin pool is expanded like a non-thin logical volume using. For instance:

Extending a thin logical volume
A thin logical volume is expanded just like a regular one:

Note that the command uses the   option (or   if extent counts are used) and not a "virtual size" option as was used during the creation.

Reducing a thin pool
Currently, LVM cannot reduce the size of the thin pool. See LVM bug |812731.

Reducing a thin logical volume
Thin logical volumes are reduced just like regular logical volumes.

For instance:

Note that the command uses the   option (or   if extent counts are used) and not a "virtual size" option as was used during the creation.

Removing thin pools
Thin pools cannot be removed until all the thin logical volumes inside it are removed.

When a thin pool no longer services any thin logical volume, it can be removed through the command:

LVM2 snapshots and thin snapshots
A snapshot is a logical volume that acts as copy of another logical volume. It displays the state of the original logical volume at the time of snapshot creation.

Creating a snapshot logical volume
A snapshot logical volume is created using the  option to. Snapshot logical volumes are still given allocated storage as LVM "registers" all changes made to the original logical volume and stores these changes in the allocated storage for the snapshot. When querying the snapshot state, LVM will start from the original logical volume and then check all changes registered, "undoing" the changes before showing the result to the user.

A snapshot logical volume henceforth "growths" at the rate that changes are made on the original logical volume. When the allocated storage for the snapshot is completely used, then the snapshot will be removed automatically from the system.

The above example creates a snapshot logical volume called 20140412_lvol1, based on the logical volume lvol1 in volume group vg0. It uses 10% of the space (extents actually) allocated to the volume group.

Accessing a snapshot logical volume
Snapshot logical volumes can be mounted like regular logical volumes. They are even not restricted to read-only operations - it is possible to modify snapshots and thus use it for things such as testing changes before doing these on a "production" file system.

As long as snapshot logical volumes exist, the regular/original logical volume cannot be reduced in size or removed.

LVM thin snapshots
To create a thin snapshot, the command is used with the   option. No size declaration needs to be passed on:

Thin logical volume snapshots have the same size as their original thin logical volume, and use a physical allocation of 0 just like all other thin logical volumes.

It is also possible to take snapshots of snapshots:

Thin snapshots have several advantages over regular snapshots. First, thin snapshots are independent of their original logical volume once created. The original logical volume can be shrunk or deleted without affecting the snapshot. Second, thin snapshots can be efficiently created recursively (snapshots of snapshots) without the "chaining" overhead of regular recursive LVM snapshots.

Rolling back to snapshot state
To rollback the logical volume to the version of the snapshot, use the following command:

This might take a couple of minutes, depending on the size of the volume. Please note that the rollback will only happen once the parent logical volume is offline. Hence a reboot might be required.

Rolling back thin snapshots
For thin volumes, does not work. Instead, delete the original logical volume and rename the snapshot:

Different storage allocation methods
LVM supports different allocation methods for storage:
 * Linear volumes (which is the default);
 * Mirrored volumes (in a more-or-less active/standby setup);
 * Striping (RAID0);
 * Mirrored volumes (RAID1 - which is more an active/active setup);
 * Striping with parity (RAID4 and RAID5);
 * Striping with double parity (RAID6);
 * Striping and mirroring (RAID10).

Linear volumes
Linear volumes are the most common kind of LVM volumes. LVM will attempt to allocate the logical volume to be as physically contiguous as possible. If there is a physical volume large enough to hold the entire logical volume, then LVM will allocate it there, otherwise it will split it up into as few pieces as possible.

The commands introduced earlier on to create volume groups and logical volumes create linear volumes.

Because linear volumes have no special requirements, they are the easiest to manipulate and can be resized and relocated at will. If a logical volume is allocated across multiple physical volumes, and any of the physical volumes become unavailable, then that logical volume cannot be started anymore and will be unusable.

Mirrored volumes
LVM supports mirrored volumes, which provide fault tolerance in the event of drive failure. Unlike RAID1, there is no performance benefit - all reads and writes are delivered to a single side of the mirror.

To keep track of the mirror state, LVM requires a log to be kept. It is recommended (and often even mandatory) to position this log on a physical volume that does not contain any of the mirrored logical volumes. There are three kind of logs that can be used for mirrors:


 * 1) Disk is the default log type. All changes made are logged into extra metadata extents, which LVM manages. If a device fails, then the changes are kept in the log until the mirror can be restored again.
 * 2) Mirror logs are disk logs that are themselves mirrored.
 * 3) Core mirror logs record the state of the mirror in memory only. LVM will have to rebuild the mirror every time it is activated. This type is useful for temporary mirrors.

To create a logical volume with a single mirror, pass the -m 1 argument (to select standard mirroring) with optionally  to select a particular log type:

The -m 1 tells LVM to create one (additional) mirror, so requiring 2 physical volumes. The  option is an optimization - without it LVM will try synchronize the mirror by copying empty sectors from one logical volume to another.

It is possible to create a mirror of an existing logical volume:

The  option does the conversion in the background as this can take quite a while.

To remove a mirror, set the number of mirrors (back) to 0:

If part of the mirror is unavailable (usually because the disk containing the physical volume has failed), the volume group will need to be brought up in degraded mode:

On the first write, LVM will notice the mirror is broken. The default policy ("remove") is to automatically reduce/break the mirror according to the number of pieces available. A 3-way mirror with a missing physical volume will be reduced to 2-way mirror; a 2-way mirror will be reduced to a regular linear volume. If the failure is only transient, and the missing physical volume returns after LVM has broken the mirror, the mirrored logical volume will need to be recreated on it.

To recover the mirror, the failed physical volume needs to be removed from the volume group, and a replacement physical volume needs to be added (or if the volume group has a free physical volume, it can be created on that one). Then the mirror can be recreated with at which point the old physical volume can be removed from the volume group:

It is possible to have LVM recreate the mirror with free extents on a different physical volume if one side fails. To accomplish that, set  to allocate in.

Thin mirrors
It is not (yet) possible to create a mirrored thin pool or thin volume. It is possible to create a mirrored thin pool by creating a normal mirrored logical volume and then converting the logical volume to a thin pool with. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will merge them into a single logical volume.

Striping (RAID0)
Instead of a linear volume, where multiple contiguous physical volumes are appended, it possible to create a striped or RAID0 volume for better performance. This will alternate storage allocations across the available physical volumes.

To create a striped volume over three physical volumes:

The  option indicates over how many physical volumes the striping should be done.

It is possible to mirror a stripe set. The  and   options can be combined to create a striped mirror:

This creates a 2 physical volume stripe set and mirrors it on 2 different physical volumes, for a total of 4 physical volumes. An existing stripe set can be mirrored with.

A thin pool can be striped like any other logical volume. All the thin volumes created from the pool inherit that settings - do not specify it manually when creating a thin volume.

It is not possible to stripe an existing volume, nor reshape the stripes across more/less physical volumes, nor to convert to a different RAID level/linear volume. A stripe set can be mirrored. It is possible to extend a stripe set across additional physical volumes, but they must be added in multiples of the original stripe set (which will effectively linearly append a new stripe set).

Mirroring (RAID1)
Unlike RAID0, which is striping, RAID1 is mirroring, but implemented differently than the original LVM mirror. Under RAID1, reads are spread out across physical volumes, improving performance. RAID1 mirror failures do not cause I/O to block because LVM does not need to break it on write.

Any place where an LVM mirror could be used, a RAID1 mirror can be used in its place. It is possible to have LVM create RAID1 mirrors instead of regular mirrors implicitly by setting mirror_segtype_default to raid1 in.

To create a logical volume with a single mirror:

Note the difference for creating a mirror: There is no mirrorlog specified, because RAID1 logical volumes do not have an explicit mirror log - it built-in to the logical volume.

It is possible to convert an existing logical volume to RAID1:

To remove a RAID1 mirror, set the number of mirrors to 0:

If part of the RAID1 is unavailable (usually because the disk containing the physical volume has failed), the volume group will need to be brought up in degraded mode:

Unlike an LVM mirror, writing does NOT break the mirroring. If the failure is only transient, and the missing physical volume returns, LVM will resync the mirror by copying cover the out-of-date segments instead of the entire logical volume. If the failure is permanent, then the failed physical volume needs to be removed from the volume group, and a replacement physical volume needs to be added (or if the volume group has a free physical volume, it can be created on a different PV). The mirror can then be repaired with, and the old physical volume can be removed from the volume group:

Thin RAID1
It is not (yet) possible to create a RAID1 thin pool or thin volume. It is possible to create a RAID1 thin pool by creating a normal mirrored logical volume and then converting the logical volume to a thin pool with. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will then merge them into a single logical volume.

Striping with parity (RAID4 and RAID5)
RAID0 is not fault-tolerant - if any of the physical volumes fail then the logical volume is unusable. By adding a parity stripe to RAID0 the logical volume can still function if a physical volume is missing. A new physical volume can then be added to restore fault tolerance.

Stripsets with parity come in 2 flavors: RAID4 and RAID5. Under RAID4, all the parity stripes are stored on the same physical volume. This can become a bottleneck because all writes hit that physical volume, and it gets worse the more physical volumes are in the array. With RAID5, the parity data is distributed evenly across the physical volumes so none of them become a bottleneck. For that reason, RAID4 is rare and is considered obsolete/historical. In practice, all stripesets with parity are RAID5.

Only the data physical volumes are specified with -i, LVM adds one to it automatically for the parity. So for a 3 physical volume RAID5, -i 2 is passed on and not -i 3.

When a physical volume fails, then the volume group will need to be brought up in degraded mode:

The volume will work normally at this point, however this degrades the array to RAID0 until a replacement physical volume is added. Performance is unlikely to be affected while the array is degraded - although it does need to recompute its missing data via parity, it only requires simple XOR for the parity block with the remaining data. The overhead is negligible compared to the disk I/O.

To repair the RAID5:

It is possible to replace a still working physical volume in RAID5 as well:

The same restrictions of stripe sets apply to stripe sets with parity as well: it is not possible to enable striping with parity on an existing volume, nor reshape the stripes with parity across more/less physical volumes, nor to convert to a different RAID level/linear volume. A stripe set with parity can be mirrored. It is possible to extend a stripe set with parity across additional physical volumes, but they must be added in multiples of the original stripe set with parity (which will effectively linearly append a new stripe set with parity).

Thin RAID5 logical volumes
It is not (yet) possible to create stripe set with parity (RAID5) thin pools or thin logical volumes. It is possible to create a RAID5 thin pool by creating a normal RAID5 logical volume and then converting the logical volume into a thin pool with. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will merge them into a single logical volume.

Striping with double parity (RAID6)
RAID6 is similar to RAID5, however RAID6 can survive up to two physical volume failures, thus offering more fault tolerance than RAID5 at the expense of extra physical volumes.

Like RAID5, the  option is used to specify the number of physical volumes to stripe, excluding the 2 physical volumes for parity. So for a 5 physical volume RAID6, pass on  and not.

Recovery for RAID6 is the same as RAID5.

Thin RAID6 logical volumes
It is not (yet) possible to create a RAID6 thin pool or thin volumes. It is possible to create a RAID6 thin pool by creating a normal RAID6 logical volume and then converting the logical volume into a thin pool with. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will merge them into a single logical volume.

LVM RAID10
RAID10 is a combination of RAID0 and RAID1. It is more powerful than RAID0+RAID1 as the mirroring is done at the stripe level instead of the logical volume level, and therefore the layout doesn't need to be symmetric. A RAID10 volume can tolerate at least a single missing physical volume, and possibly more.

Both the  and   options are specified:   is the number of stripes and   is the number of mirrors. Two stripes and 1 mirror requires 4 physical volumes.

Thin RAID10
It is not (yet) possible to create a RAID10 thin pool or thin volumes. It is possible to create a RAID10 thin pool by creating a normal RAID10 logical volume and then converting the logical volume into a thin pool with. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will merge them into a single logical volume.

Experimenting with LVM
It is possible to experiment with LVM without using real storage devices. To accomplish this, loopback devices are created.

First make sure to have the loopback module loaded.

Next configure LVM to not use udev to scan for devices:

Create some image files which will become the storage devices. The next example uses five files for a total of about ~10GB of real hard drive space:

Check which loopback devices are available:

Assuming all loopback devices are available, next create the devices:

The devices are now available to use as any other hard drive in the system (and thus be perfect for physical volumes).

Troubleshooting
LVM has a few features that already provide some level of redundancy. However, there are situations where it is possible to restore lost physical volumes or logical volumes.

vgcfgrestore utility
By default, on any change to a LVM physical volume, volume group, or logical volume, LVM2 create a backup file of the metadata in. These files can be used to recover from an accidental change (like deleting the wrong logical volume). LVM also keeps a backup copy of the most recent metadata in. These can be used to restore metadata to a replacement disk, or repair corrupted metadata.

To see what states of the volume group are available to be restored (partial output to improve readability):

Recovering an accidentally deleted logical volume
Assuming the logical volume lvm_raid1 was accidentally removed from volume group vg0, it is possible to recover it as follows:

Replacing a failed physical volume
It possible to do a true "replace" and recreate the metadata on the new physical volume to be the same as the old physical volume:

The important line here is the UUID "unknown device".

This recreates the physical volume metadata, but not the missing logical volume or volume group data on the physical volume.

This now reconstructs all the missing metadata on the physical volume, including the logical volume and volume group data. However it doesn't restore the data, so the mirror is out of sync.

This will resync the mirror. This works with RAID 4,5 and 6 as well.

Deactivating a logical volume
It is possible to deactivate a logical volume with the following command:

It is not possible to mount the logical volume anywhere before it gets reactivated:

External resources

 * LVM2 sourceware.org
 * LVM tldp.org
 * LVM2 Wiki redhat.com