LVM (Logical Volume Manager) allows administrators to create meta devices that provide an abstraction layer between a file system and the physical storage that is used underneath. The meta devices (on which file systems are placed) are logical volumes, which use storage from storage pools called volume groups. A volume group is provisioned with one or more physical volumes which are the true devices on which the data is stored.
Physical volumes can be partitions, whole SATA hard drives grouped as JBOD (Just a Bunch Of Disks), RAID systems, iSCSI, Fibre Channel, eSATA etc.
- 1 Installation
- 2 Configuration
- 3 Usage
- 3.1 PV (Physical Volume)
- 3.2 VG (Volume Group)
- 3.3 LV (Logical Volume)
- 4 Features
- 4.1 Thin provisioning
- 4.2 LVM2 snapshots and thin snapshots
- 4.3 Different storage allocation methods
- 4.3.1 Linear volumes
- 4.3.2 Mirrored volumes
- 4.3.3 Thin mirrors
- 4.3.4 Striping (RAID0)
- 4.3.5 Mirroring (RAID1)
- 4.3.6 Thin RAID1
- 4.3.7 Striping with parity (RAID4 and RAID5)
- 4.3.8 Thin RAID5 logical volumes
- 4.3.9 Striping with double parity (RAID6)
- 4.3.10 Thin RAID6 logical volumes
- 4.3.11 LVM RAID10
- 4.3.12 Thin RAID10
- 5 Experimenting with LVM
- 6 Troubleshooting
- 7 External resources
LVM is handled by both kernel-level drivers and user space applications to manage the LVM configuration.
Activate the following kernel options:
|clvm||No||Allow users to build clustered lvm2|
|cman||No||Cman support for clustered lvm|
|lvm1||Yes||Allow users to build lvm2 with lvm1 support|
|readline||Yes||Enables support for libreadline, a GNU line-editing library that almost everyone wants|
|selinux||No||No||!!internal use only!! Security Enhanced Linux support, this must be set by the selinux profile or breakage will occur|
|static||No||!!do not set this during bootstrap!! Causes binaries to be statically linked instead of dynamically|
|static-libs||No||Build static libraries|
|thin||Yes||Support for thin volumes|
|udev||Yes||Enable sys-fs/udev integration (device discovery, power and storage device support, etc)|
Configuring LVM is done on several levels:
- LV, PV and VG management through the management utilities
- LVM subsystem fine-tuning through the configuration file
- service management at the distribution level
- setup through an initial ram file system
Management of the logical and physical volumes as well as the volume groups is handled through the Usage chapter.
LVM configuration file
LVM has an extensive configuration file at /etc/lvm/lvm.conf. Most users will not need to modify settings in this file in order to start using LVM.
Gentoo provides the LVM service to automatically detect and activate the volume groups and logical volumes.
The service can be managed through the init system.
To start LVM manually:
To start LVM at boot time:
To start lvm manually:
To start LVM at boot time:
Using LVM in an initramfs
Most bootloaders cannot boot from LVM directly - neither GRUB legacy nor LILO can. Grub 2 CAN boot from an LVM linear logical volume, mirrored logical volume and possibly some kinds of RAID logical volumes. No bootloader currently support thin logical volumes.
For that reason, it is recommended to use a non-LVM /boot partition and mount the LVM root from an initramfs. Such an initramfs can be generated automatically through genkernel, genkernel-next and dracut:
- genkernel can boot from all types except thin volumes (as it neither builds nor copies the thin-provisioning-tools binaries from the build host) and maybe RAID10 (RAID10 support requires LVM2 2.02.98, but genkernel builds 2.02.89, however if static binaries are available it can copy those)
- genkernel-next can boot from all types volumes. but needs a new enough app-misc/pax-utils or the resulting thin binaries will be broken (See bug #482504)
- dracut should boot all types, but only includes thin support in the initramfs if the host being run on has a thin root.
LVM organizes storage in three different levels as follows:
- hard drives, partitions, RAID systems or other means of storage are initialized as physical volumes (PVs)
- Physical Volumes (PV) are grouped together in Volume Groups (VG)
- Logical Volumes (LV) are managed in Volume Groups (VG)
PV (Physical Volume)
Physical Volumes are the actual hardware or storage system LVM builds up upon.
The partition type for LVM is 8e (Linux LVM).
For instance, to set the type through
fdisk for a partition on /dev/sda:
fdisk, create partitions using the n key and then change the partition type with the t key to 8e.
Physical volumes can be created / initialized with the
For instance, the following command creates a physical volume on the first primary partition of /dev/sda and /dev/sdb:
pvdisplay command, an overview of all active physical volumes on the system can be obtained.
If more physical volumes should be displayed, then
pvscan can detect inactive physical volumes and activate those.
LVM automatically distributes the data onto all available physical volumes (unless told otherwise) but in a linear approach. If a requested logical volume (within a volume group) is smaller than the amount of free space on a single physical volume, then all space for the logical volume is claimed on that (single) physical volume in a contiguous manner. This is done for performance reasons.
If a physical volume needs to be removed from a volume group, the data first needs to be moved away from the physical volume. With the
pvmove command, all data on a physical volume is moved to other physical volumes within the same volume group.
Such an operation can take a while depending on the amount of data that needs to be moved. Once finished, there should be no data left on the device. Verify with
pvdisplay that the physical volume is no longer used by any logical volume.
The next step is to remove the physical volume from the volume group using
vgreduce after which the device can be "deselected" as a physical volume using
VG (Volume Group)
A volume group (VG) groups a number of physical volumes and show up as /dev/VG_NAME in the device file system. The name of a volume group is chosen by the administrator.
The following command creates a volume group called vg0 with two physical volumes assigned to it: /dev/sda1 and /dev/sdb1.
To list all active volume groups, use the
If volume groups are missing, use the
vgscan command to locate volume groups:
Volume groups group physical volumes, allowing administrators to use a pool of storage resources to allocate to file systems. When a volume group does not hold enough storage resources, it is necessary to extend the volume group with additional physical volumes.
The next example extends the volume group vg0 with a physical volume at /dev/sdc1:
Remember that the physical volume first needs to be initialized as such!
If physical volumes need to be removed from the volume group, all data still in use on the physical volume needs to be moved to other physical volumes in the volume group. As seen before, this is handled through the
pvmove command, after which the physical volume can be removed from the volume group using
If a volume group is no longer necessary (or, in other words, the storage pool that it represents is no longer used and the physical volumes in it need to be freed for other purposes) then the volume group can be removed with
vgremove. This only works if no logical volume is defined for the volume group, and all but one physical volume have already been removed from the pool.
LV (Logical Volume)
Logical volumes are the final meta devices which are made available to the system, usually to create file systems on. They are created and managed in volume groups and show up as /dev/VG_NAME/LV_NAME. Like with volume groups, the name used for a logical volume is decided by the administrator.
To create a logical volume, the
lvcreate command is used. The parameters to the command consist out of the requested size for the logical volume (which cannot be larger than the amount of free space in the volume group), the volume group from which the space is to be claimed and the name of the logical volume to be created.
In the next example, a logical volume named lvol1 is created from the volume group named vg0 and with a size of 150MB:
It is possible to tell
lvcreate to use all free space inside a volume group. This is done through the -l parameter which selects the amount of extents rather than a (human readable) size. Logical volumes are split into logical extents which are data chunks inside a volume group. All extents in a volume group have the same size. With the -l parameter
lvcreate can be asked to allocate all free extents:
Next to FREE the VG key can be used to denote the entire size of a volume group.
To list all logical volumes, use the
If logical volumes are missing, then the
lvscan command can be used to scan for logical volumes on all available volume groups.
When a logical volume needs to be expanded, then the
lvextend command can be used to grow the allocated space for the logical volume.
For instance, to extend the logical volume lvol1 to a total of 500 MB:
It is also possible to use the size to be added rather than the total size:
An extended volume group does not immediately provide the additional storage to the end users. For that, the file system on top of the volume group needs to be increased in size as well. Not all file systems allow online resizing, so check the documentation for the file system in question for more information.
For instance, to resize an ext4 file system to become 500MB in size:
If a logical volume needs to be reduced in size, first shrink the file system itself. Not all file systems support online shrinking.
For instance, ext4 does not support online shrinking so the file system needs to be unmounted first. It is also recommended to do a file system check to make sure there are no inconsistencies:
With a reduced file system, it is now possible to reduce the logical volume as well:
LVM supports permission states on the logical volumes.
For instance, a logical volume can be set to read only using the
The remount is needed as the change is not enforced immediately.
To mark the logical volume as writable again, use the rw permission bit:
Before removing a logical volume, make sure it is no longer mounted:
Deactivate the logical volume so that no further write activity can take place:
With the volume unmounted and deactivated, it can now be removed, freeing the extents allocated to it for use by other logical volumes in the volume group:
LVM provides quite a few interesting features for storage administrators, including (but not limited to)
- thin provisioning (over-committing storage)
- snapshot support
- volume types with different storage allocation methods
Recent versions of LVM2 (2.02.89) support "thin" volumes. Thin volumes are to block devices what sparse files are to file systems. Thus, a thin logical volume within a pool can be "over-committed": its presented size can be larger than the allocated size - it can even be larger than the pool itself. Just like a sparse file, the extents are allocated as the block device gets populated. If the file system has discard support extents are freed again as files are removed, reducing space utilization of the pool.
Within LVM, such a thin pool is a special type of logical volume, which itself can host logical volumes.
Creating a thin pool
Each thin pool has metadata associated with it, which is added to the thin pool size. LVM will compute the size of the metadata based on the size of the thin pool as the minimum of pool_chunks * 64 bytes or 2MiB, whichever is larger. The administrator can select a different metadata size as well.
To create a thin pool, add the --type thin-pool --thinpool thin_pool parameters to
The above example creates a thin pool called thin_pool with a total size of 150 MB. This is the real allocated size for the thin pool (and thus the total amount of actual storage that can be used).
To explicitly ask for a certain metadata size, use the --metadatasize parameter:
Due to the metadata that is added to the thin pool, the intuitive way of using all available size in a volume group for a logical volume does not work (see LVM bug ):
Note the thin pool does not have an associated device node like other LV's.
Creating a thin logical volume
A thin logical volume is a logical volume inside the thin pool (which itself is a logical volume). As thin logical volumes are sparse, a virtual size instead of a physical size is specified using the -V parameter:
In this example, the (thin) logical volume lvol1 is exposed as a 300MB-sized device, even though the underlying pool only holds 150MB of real allocated storage.
It is also possible to create both the thin pool as well as the logical volume inside the thin pool in one command:
Listing thin pools and thin logical volumes
Thin pools and thin logical volumes are special types of logical volumes, and as such as displayed through the
lvdisplay command. The
lvscan command will also detect these logical volumes.
Extending a thin pool
The thin pool is expanded like a non-thin logical volume using
lvextend. For instance:
Extending a thin logical volume
A thin logical volume is expanded just like a regular one:
Note that the
lvextend command uses the -L option (or -l if extent counts are used) and not a "virtual size" option as was used during the creation.
Reducing a thin pool
Currently, LVM cannot reduce the size of the thin pool. See LVM bug .
Reducing a thin logical volume
Thin logical volumes are reduced just like regular logical volumes.
Note that the
lvreduce command uses the -L option (or -l if extent counts are used) and not a "virtual size" option as was used during the creation.
Removing thin pools
Thin pools cannot be removed until all the thin logical volumes inside it are removed.
When a thin pool no longer services any thin logical volume, it can be removed through the
LVM2 snapshots and thin snapshots
A snapshot is a logical volume that acts as copy of another logical volume. It displays the state of the original logical volume at the time of snapshot creation.
Creating a snapshot logical volume
A snapshot logical volume is created using the -s option to
lvcreate. Snapshot logical volumes are still given allocated storage as LVM "registers" all changes made to the original logical volume and stores these changes in the allocated storage for the snapshot. When querying the snapshot state, LVM will start from the original logical volume and then check all changes registered, "undoing" the changes before showing the result to the user.
A snapshot logical volume henceforth "growths" at the rate that changes are made on the original logical volume. When the allocated storage for the snapshot is completely used, then the snapshot will be removed automatically from the system.
The above example creates a snapshot logical volume called 20140412_lvol1, based on the logical volume lvol1 in volume group vg0. It uses 10% of the space (extents actually) allocated to the volume group.
Accessing a snapshot logical volume
Snapshot logical volumes can be mounted like regular logical volumes. They are even not restricted to read-only operations - it is possible to modify snapshots and thus use it for things such as testing changes before doing these on a "production" file system.
As long as snapshot logical volumes exist, the regular/original logical volume cannot be reduced in size or removed.
LVM thin snapshots
To create a thin snapshot, the
lvcreate command is used with the
-s option. No size declaration needs to be passed on:
Thin logical volume snapshots have the same size as their original thin logical volume, and use a physical allocation of 0 just like all other thin logical volumes.
It is also possible to take snapshots of snapshots:
Thin snapshots have several advantages over regular snapshots. First, thin snapshots are independent of their original logical volume once created. The original logical volume can be shrunk or deleted without affecting the snapshot. Second, thin snapshots can be efficiently created recursively (snapshots of snapshots) without the "chaining" overhead of regular recursive LVM snapshots.
Rolling back to snapshot state
To rollback the logical volume to the version of the snapshot, use the following command:
This might take a couple of minutes, depending on the size of the volume.
Rolling back thin snapshots
For thin volumes,
lvconvert --merge does not work. Instead, delete the original logical volume and rename the snapshot:
Different storage allocation methods
LVM supports different allocation methods for storage:
- linear volumes (which is the default)
- mirrored volumes (in a more-or-less active/standby setup)
- striping (RAID0)
- mirrored volumes (RAID1 - which is more an active/active setup)
- striping with parity (RAID4 and RAID5)
- striping with double parity (RAID6)
- striping and mirroring (RAID10)
Linear volumes are the most common kind of LVM volumes. LVM will attempt to allocate the logical volume to be as physically contiguous as possible. If there is a physical volume large enough to hold the entire logical volume, then LVM will allocate it there, otherwise it will split it up into as few pieces as possible.
The commands introduced earlier on to create volume groups and logical volumes create linear volumes.
Because linear volumes have no special requirements, they are the easiest to manipulate and can be resized and relocated at will. If a logical volume is allocated across multiple physical volumes, and any of the physical volumes become unavailable, then that logical volume cannot be started anymore and will be unusable.
LVM supports mirrored volumes, which provide fault tolerance in the event of drive failure. Unlike RAID1, there is no performance benefit - all reads and writes are delivered to a single side of the mirror.
To keep track of the mirror state, LVM requires a log to be kept. It is recommended (and often even mandatory) to position this log on a physical volume that does not contain any of the mirrored logical volumes. There are three kind of logs that can be used for mirrors:
- Disk is the default log type. All changes made are logged into extra metadata extents, which LVM manages. If a device fails, then the changes are kept in the log until the mirror can be restored again.
- Mirror logs are disk logs that are themselves mirrored.
- Core mirror logs record the state of the mirror in memory only. LVM will have to rebuild the mirror every time it is activated. This type is useful for temporary mirrors.
To create a logical volume with a single mirror, pass the -m 1 argument (to select standard mirroring) with optionally --mirrorlog to select a particular log type:
The -m 1 tells LVM to create one (additional) mirror, so requiring 2 physical volumes. The --nosync option is an optimization - without it LVM will try synchronize the mirror by copying empty sectors from one logical volume to another.
It is possible to create a mirror of an existing logical volume:
The -b option does the conversion in the background as this can take quite a while.
To remove a mirror, set the number of mirrors (back) to 0:
If part of the mirror is unavailable (usually because the disk containing the physical volume has failed), the volume group will need to be brought up in degraded mode:
On the first write, LVM will notice the mirror is broken. The default policy ("remove") is to automatically reduce/break the mirror according to the number of pieces available. A 3-way mirror with a missing physical volume will be reduced to 2-way mirror; a 2-way mirror will be reduced to a regular linear volume. If the failure is only transient, and the missing physical volume returns after LVM has broken the mirror, the mirrored logical volume will need to be recreated on it.
To recover the mirror, the failed physical volume needs to be removed from the volume group, and a replacement physical volume needs to be added (or if the volume group has a free physical volume, it can be created on that one). Then the mirror can be recreated with lvconvert at which point the old physical volume can be removed from the volume group:
It is possible to have LVM recreate the mirror with free extents on a different physical volume if one side fails. To accomplish that, set
mirror_image_fault_policy to allocate in lvm.conf.
It is not (yet) possible to create a mirrored thin pool or thin volume. It is possible to create a mirrored thin pool my creating a normal mirrored logical volume and then converting the logical volume to a thin pool with lvconvert. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will merge them into a single logical volume.
Instead of a linear volume, where multiple contiguous physical volumes are appended, it possible to create a striped or RAID 0 volume for better performance. This will alternate storage allocations across the available physical volumes.
To create a striped volume over three physical volumes:
The -i option indicates over how many physical volumes the striping should be done.
It is possible to mirror a stripe set. The -i and -m options can be combined to create a striped mirror:
This creates a 2 physical volume stripe set and mirrors it on 2 different physical volumes, for a total of 4 physical volumes. An existing stripe set can be mirrored with lvconvert.
A thin pool can be striped like any other logical volume. All the thin volumes created from the pool inherit that settings - do not specify it manually when creating a thin volume.
It is not possible to stripe an existing volume, nor reshape the stripes across more/less physical volumes, nor to convert to a different RAID level/linear volume. A stripe set can be mirrored. It is possible to extend a stripe set across additional physical volumes, but they must be added in multiples of the original stripe set (which will effectively linearly append a new stripe set).
Unlike RAID 0, which is striping, RAID 1 is mirroring, but implemented differently than the original LVM mirror. Under RAID1, reads are spread out across physical volumes, improving performance. RAID1 mirror failures do not cause I/O to block because LVM does not need to break it on write.
Any place where an LVM mirror could be used, a RAID 1 mirror can be used in its place. It is possible to have LVM create RAID1 mirrors instead of regular mirrors implicitly by setting mirror_segtype_default to raid1 in lvm.conf.
To create a logical volume with a single mirror:
Note the difference for creating a mirror: There is no mirrorlog specified, because RAID1 logical volumes do not have an explicit mirror log - it built-in to the logical volume.
It is possible to convert an existing logical volume to RAID 1:
To remove a RAID 1 mirror, set the number of mirrors to 0:
If part of the RAID1 is unavailable (usually because the disk containing the physical volume has failed), the volume group will need to be brought up in degraded mode:
Unlike an LVM mirror, writing does NOT breaking the mirroring. If the failure is only transient, and the missing physical volume returns, LVM will resync the mirror by copying cover the out-of-date segments instead of the entire logical volume. If the failure is permanent, then the failed physical volume needs to be removed from the volume group, and a replacement physical volume needs to be added (or if the volume group has a free physical volume, it can be created on a different PV). The mirror can then be repaired with lvconvert, and the old physical volume can be removed from the volume group:
It is not (yet) possible to create a RAID 1 thin pool or thin volume. It is possible to create a RAID 1 thin pool by creating a normal mirrored logical volume and then converting the logical volume to a thin pool with lvconvert. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will then merge them into a single logical volume.
Striping with parity (RAID4 and RAID5)
RAID 0 is not fault-tolerant - if any of the physical volumes fail then the logical volume is unusable. By adding a parity stripe to RAID 0 the logical volume can still function if a physical volume is missing. A new physical volume can then be added to restore fault tolerance.
Stripsets with parity come in 2 flavors: RAID 4 and RAID 5. Under RAID 4, all the parity stripes are stored on the same physical volume. This can become a bottleneck because all writes hit that physical volume, and it gets worse the more physical volumes are in the array. With RAID 5, the parity data is distributed evenly across the physical volumes so none of them become a bottleneck. For that reason, RAID 4 is rare and is considered obsolete/historical. In practice, all stripesets with parity are RAID 5.
Only the data physical volumes are specified with -i, LVM adds one to it automatically for the parity. So for a 3 physical volume RAID5, -i 2 is passed on and not -i 3.
When a physical volume fails, then the volume group will need to be brought up in degraded mode:
The volume will work normally at this point, however this degrades the array to RAID 0 until a replacement physical volume is added. Performance is unlikely to be affected while the array is degraded - although it does need to recompute its missing data via parity, it only requires simple XOR for the parity block with the remaining data. The overhead is negligible compared to the disk I/O.
To repair the RAID5:
It is possible to replace a still working physical volume in RAID5 as well:
The same restrictions of stripe sets apply to stripe sets with parity as well: it is not possible to enable striping with parity on an existing volume, nor reshape the stripes with parity across more/less physical volumes, nor to convert to a different RAID level/linear volume. A stripe set with parity can be mirrored. It is possible to extend a stripe set with parity across additional physical volumes, but they must be added in multiples of the original stripe set with parity (which will effectively linearly append a new stripe set with parity).
Thin RAID5 logical volumes
It is not (yet) possible to create stripe set with parity (RAID5) thin pools or thin logical volumes. It is possible to create a RAID5 thin pool by creating a normal RAID5 logical volume and then converting the logical volume into a thin pool with lvconvert. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will merge them into a single logical volume.
Striping with double parity (RAID6)
RAID 6 is similar to RAID 5, however RAID 6 can survive up to two physical volume failures, thus offering more fault tolerance than RAID5 at the expense of extra physical volumes.
Like raid5, the -i option is used to specify the number of physical volumes to stripe, excluding the 2 physical volumes for parity. So for a 5 physical volume RAID6, pass on -i 3 and not -i 5.
Recovery for RAID6 is the same as RAID5.
Thin RAID6 logical volumes
It is not (yet) possible to create a RAID6 thin pool or thin volumes. It is possible to create a RAID6 thin pool by creating a normal RAID6 logical volume and then converting the logical volume into a thin pool with lvconvert. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will merge them into a single logical volume.
RAID10 is a combination of RAID0 and RAID1. It is more powerful than RAID0+RAID1 as the mirroring is done at the stripe level instead of the logical volume level, and therefore the layout doesn't need to be symmetric. A RAID10 volume can tolerate at least a single missing physical volume, and possibly more.
Both the -i and -m options are specified: -i is the number of stripes and -m is the number of mirrors. Two stripes and 1 mirror requires 4 physical volumes.
It is not (yet) possible to create a RAID10 thin pool or thin volumes. It is possible to create a RAID10 thin pool by creating a normal RAID10 logical volume and then converting the logical volume into a thin pool with
lvconvert. 2 logical volumes are required: one for the thin pool and one for the thin metadata; the conversion process will merge them into a single logical volume.
Experimenting with LVM
It is possible to experiment with LVM without using real storage devices. To accomplish this, loopback devices are created.
First make sure to have the loopback module loaded.
Next configure LVM to not use udev to scan for devices:
Create some image files which will become the storage devices. The next example uses five files for a total of about ~10GB of real hard drive space:
Check which loopback devices are available:
Assuming all loopback devices are available, next create the devices:
The /dev/loop[0-4] devices are now available to use as any other hard drive in the system (and thus be perfect for physical volumes).
LVM has a few features that already provide some level of redundancy. However, there are situations where it is possible to restore lost physical volumes or logical volumes.
By default, on any change to a LVM physical volume, volume group, or logical volume, LVM2 create a backup file of the metadata in /etc/lvm/archive. These files can be used to recover from an accidental change (like deleting the wrong logical volume). LVM also keeps a backup copy of the most recent metadata in /etc/lvm/backup. These can be used to restore metadata to a replacement disk, or repair corrupted metadata.
To see what states of the volume group are available to be restored (partial output to improve readability):
Recovering an accidentally deleted logical volume
Assuming the logical volume lvm_raid1 was accidentally removed from volume group vg0, it is possible to recover it as follows:
Replacing a failed physical volume
It possible to do a true "replace" and recreate the metadata on the new physical volume to be the same as the old physical volume:
The important line here is the UUID "unknown device".
This recreates the physical volume metadata, but not the missing logical volume or volume group data on the physical volume.
This now reconstructs all the missing metadata on the physical volume, including the logical volume and volume group data. However it doesn't restore the data, so the mirror is out of sync.
This will resync the mirror. This works with RAID 4,5 and 6 as well.
Deactivating a logical volume
It is possible to deactivate a logical volume with the following command:
It is not possible to mount the logical volume anywhere before it gets reactivated: