User:Maffblaster/Infrastructure/Disk images

When working with small systems (such as SoCs, older slower systems, or embedded systems), it is useful to know how to manipulate disk images. This article will attempt to flush out different capture methods, mounting/attaching, conversion to run via virtual machines, routine maintenance/package updates, etc.

Capture
There are many tools available in Gentoo to capture disk images. Capturing an image is nothing more than making backup of the system. Generally this involves physically connecting the disk to a system and creating a file from the disk image. Starting from easier and faster methods, and moving to harder and slower methods:


 * 1) Connect the disk by inserting it into an external USB device, then connecting the USB device to a system.
 * 2) Connect the disk directly to the system's motherboard.
 * 3) Connect the disk to another system, and then send the data to a new system

See the Backup article for more information on creating backups in general.

Once the disk is connected choose a tool to use to capture the disk image. For this example, a variant of dd will be used to ensure the integrity of the backup.

ddrescue
In the following example, is used to capture one disk as a disk image.

If all goes well, the captured disk image will be available at.

Mount
After the disk image has been captured, it is typical to mount the image as loop devices.

Discovery
It is wise to first check to see what loop devices are presently in use:

The following command can be used to attach to either a single or multi-partition disk image to a loop device:

Listing the loop devices again should show something like the following:

With the image attached to a loop device, is it now possible to run commands on the image, or mount the image at a mount point.

View available partitions in the image by querying the loop device:

Filesystem types for each partition can be checked or resized using either a disk partitioning utility such as, , or on the loop device as a whole.

Opening gparted for an easy online resize:

btrfs images
Presuming the file is a btrfs filesystem, the image can be checked or repaired just like any other btrfs device. See the Troubleshooting section below on repairing dirty filesystems.

Mount the image:

Unmount
Unmounting the image is typically no different any other unmount. In a rare instance, it may be necessary to lazily unmount an image when there's heavy disk I/O.

Detach
If the image was attached to a loop device, it is proper to detach when finished to free up the loop device for other uses:

Converting raw disk image to VM disk image
Converting the raw image file to qcow2 format (requires ):

Trim
Resizing a partition inside a disk image file does not decrease the allocated, on disk size of the disk image file. However, after shrinking the size of the partition(s) inside the disk image, the end of the (uncompressed) file can be trimmed off to save space. Compression can be used to save an additional amount of space. This can be performed by determining the last sector number for the last partition in the disk image, then cutting off anything preceding the last used sector.

The command to do so will look something like the following:

Disk sector size can be determined using or :

Compress
To get the best backup-to-space ratio, compressing raw images after the capture will be highly beneficial.

Disk images can be compressed just like any other file. For example, to compress with an "ultra" high compression using 16 threads  with the zstd compression algorithm (requires ):

The example above displays a 59.6 GiB disk image compressing down to less than 8 GiBs. Successful compression levels may vary depending on the how much free space is in the disk image, among other variables.

Restore
As long as the block device which will be used for the restoration is large enough to hold the disk image, restoration is possible by writing the disk image directly to the attached block device. This can be performed a number of ways, such as using a implementation, like :

Compressed disk images can be restored to a attached block device by decompressing the file and writing the output directly to the block device.

For example, the file disk.img.xz was compressed using the utility (from ). can be used to decompress the file directly onto a new block device:

xz decompression (with progress):

zstd decompression (with progress):

This has the advantage of not needing to decompress the compressed disk image, which may require more space than the host file system has available.

Filesystem image will not mount
Filesystems occasionally have problems. For example, presume in the example below that is a btrfs formatted filesystem image and is already attached to.

Oh, no! Looks like there's a problem with the filesystem image. No to worry, this may be correctable. Filesystem images can be checked just like actual devices.

btrfs
If the image is just one large btrfs filesystem, the following repair command should work.

Now that the filesystem has been repaired, proceed to mount as normal:

Yay! The issue has been fixed!