QEMU
QEMU (Quick EMUlator) is a generic, open source hardware emulator and virtualization suite.
Introduction
QEMU is a Type-2 hypervisor that runs within user space and performs virtual hardware emulation.
- Firstly, QEMU is a type 2 hypervisor.
- QEMU can be paired with KVM to run VMs at near native speed. This is accomplished by using hardware extensions such as: Intel VT-x or AMD-V.
- It can then emulate for user-level processes that allow applications compiled for one architecture to run on a different one.
- Multiple operating modes: User-mode emu, System emu, KVM hosting, and Xen Hosting,
- QEMU can save and restore the state of VMs of all its running programs.
- QEMU VMs can interface with many types of physical host hardware such as CD-ROM Drives, USB Devices, Audio Interfaces, Hard Disks, Network Cards
- Format of Virtual Disk image defaults to the `qcow2` format. The `qcow2` only uses as much host disk space as the Guest OS grows to use. Using snapshot method, guest OS can revert back to its desire state in time.
- It does not depend on graphical output methods on the host system, instead making use of an integrated VNC server to access the screen of the guest OS.
- QEMU on a host CPU can execute multiple virtual CPUs in parallel.
QEMU has support for acceleration plug-ins.
Available QEMU plugins are:
Virtualizer | Accelerator | Virtualization type | Description | Gentoo package name |
---|---|---|---|---|
qemu | tcg | full[1]/software-emulation | QEMU's own Tiny Code Generator. This is the default. More frequently denoted as qemu and not qemu/tcg so often. | app-emulation/qemu |
qemu | hvf[2] | paravirtualization[3] | Apple's Hypervisor.framework based on Intel VT. | |
qemu | whpx[4] | hybrid | Microsoft's Windows Hypervisor Platform based on Intel VT or AMD-V. | |
qemu | kvm | paravirtualization[5] | Linux Type-2 Hypervisor. This is the common choice for host using amd64, arm64, or mips[6]. Supports Microsoft Windows. | app-emulation/qemu |
qemu | haxm[7] | paravirtualization[8] | Intel VT, by Intel Corporation. |
QEMU when used in conjunction with an accelerator becomes a Type-1 hypervisor that runs in kernel space, that allows a user space program access to the hardware virtualization features of various processors. Such accelerator can be KVM (Kernel-based Virtual Machine) or Xen.
If no accelerator is used, QEMU will run entirely in user-space using its built in binary translator TCG (Tiny Code Generator). Using QEMU without an accelerator is relatively inefficient and slow.
This article typically uses KVM as the accelerator of choice due to its GPL licensing and availability. Without KVM nearly all commands described here will still work (unless KVM specific).
The following sub-articles provide detailed instructions on QEMU configurations and options:
- QEMU/Bridge with Wifi Routing
- QEMU/KVM IPv6 Support — describes IPv6 support in QEMU/KVM.
- QEMU/Linux guest — describes the setup of a Gentoo Linux guest in QEMU using Gentoo bootable media.
- Virtiofs - Describes using virtiofsd to share a directory between the host and a Linux guest.
- Usage options - Contains common configuration options used with QEMU (graphics/display, networking, RAM, storage, processor, etc).
- OS2WarpV3 guest - Describes the configuration steps needed to setup a virtualized OS2WarpVs=3 guest with QEMU.
- QEMU/Windows guest — setup of a Windows guest using QEMU
Installation
BIOS and UEFI firmware
In order to utilize KVM either Vt-x (vmx) or AMD-V (svm) must be supported by the processor. Vt-x or AMD-V are Intel and AMD's respective technologies for permitting multiple operating systems to concurrently execute operations on the processors.
To inspect hardware for virtualization support issue the following command:
user $
grep --color -E "vmx|svm" /proc/cpuinfo
For a period manufacturers were shipping with virtualization turned off by default in the system's firmware. Note that toggling this feature in the firmware may actually require full removal of power from the system to take effect. If restarting the system does not work try shutting down, unplugging the system, and pressing the power button in an unplugged state to discharge any residual energy from the power supply unit (PSU). Reapply power to the system to verify success.
If KVM support is available there should be a "kvm" device listed at /dev/kvm. This will take effect after the system has booted to a KVM enabled kernel.
Kernel
Described below are the basic requirements for KVM kernel configuration for the host OS. A more complete and up-to-date list can be found at the KVM Tuning Kernel page.
Different guest (virtualized) OS may require additional kernel options. These are covered in the corresponding QEMU section pages.
General setup --->
Timers subsystem --->
<*> High Resolution Timer Support
This includes support for ARM64 processors.
Physical CPU processor support - Host
If KVM support is not available, insert CONFIG_KVM=y into the /usr/src/linux/.config and rebuild/reinstall the kernel (and its initramfs image). Come back here after the host gets rebooted.
[*] Virtualization --->
<*> Kernel-based Virtual Machine (KVM) support
This includes support for ARM64 processors.
Processor Support
[*] Virtualization --->
<M> KVM for Intel processors support
[*] Virtualization --->
<M> KVM for AMD processors support
If both "KVM for Intel processors support" and "KVM for AMD processors support" are set as built into the kernel (
*
) an error message will appear from kprint from early boot. Since the system has only one type processor (Intel or AMD) enabling one or both options as modules (M
) will make the error message disappear.Handling kernel config at CLI
To set the various kernel configuration settings from the command lines, the linux/scripts/kconfig/merge_config.sh shall be used here:
Mandatory kernel configuration options to set:
/usr/src/kernel-kconfig-qemu-host.config
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
CONFIG_KVM_INTEL=y
CONFIG_KVM_AMD=y
root #
cd /usr/src/linux
root #
scripts/kconfig/merge_config.sh .config /usr/src/kernel-kconfig-qemu-host.config
Useful kernel configuration options to use:
/usr/src/kernel-kconfig-qemu-host-optional.config
CONFIG_VHOST_NET=y
CONFIG_HIGH_RES_TIMER=y
CONFIG_HPET=y
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
CONFIG_KSM=y
CONFIG_SYSFS=y
CONFIG_PROC_FS=y
CONFIG_TRANSPARENT_HUGEPAGE=y
CONFIG_CGROUPS=y
root #
scripts/kconfig/merge_config.sh .config /usr/src/kernel-kconfig-qemu-host-optional.config
Networking
Accelerated networking, required for vhost-net
USE flag (recommend):
Device Drivers --->
[*] VHOST drivers --->
<*> Host kernel accelerator for virtio net
[*] Virtualization --->
<*> Host kernel accelerator for virtio net
Device Drivers --->
[*] Network device support --->
[*] Network core driver support
<*> Universal TUN/TAP device driver support
Needed for 802.1d Ethernet bridging:
[*] Networking support --->
Networking options --->
<*> The IPv6 protocol
<*> 802.1d Ethernet Bridging
Intel VT-g (integrated graphics adapter virtualization)
Mediated device passthrough for Intel GPUs (Broadwell to Comet Lake) [1].
Device Drivers --->
<*> VFIO Non-Privileged userspace driver framework
<*> Mediated device driver framework
Graphics Support --->
<*> Intel 8xx/9xx/G3x/G4x/HD Graphics
[*] Enable Intel GVT-g graphics virtualization host support
<*> Enable KVM host support Intel GVT-g graphics virtualization
USE flags
Some packages are aware of the qemu USE flag.
Review the possible USE flags for QEMU:
USE flags for app-emulation/qemu QEMU + Kernel-based Virtual Machine userland tools
+aio
|
Enables support for Linux's Async IO |
+curl
|
Support ISOs / -cdrom directives via HTTP or HTTPS. |
+doc
|
Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally |
+fdt
|
Enables firmware device tree support |
+filecaps
|
Use Linux file capabilities to control privilege rather than set*id (this is orthogonal to USE=caps which uses capabilities at runtime e.g. libcap) |
+gnutls
|
Enable TLS support for the VNC console server. For 1.4 and newer this also enables WebSocket support. For 2.0 through 2.3 also enables disk quorum support. |
+jpeg
|
Enable jpeg image support for the VNC console server |
+oss
|
Add support for OSS (Open Sound System) |
+pin-upstream-blobs
|
Pin the versions of BIOS firmware to the version included in the upstream release. This is needed to sanely support migration/suspend/resume/snapshotting/etc... of instances. When the blobs are different, random corruption/bugs/crashes/etc... may be observed. |
+png
|
Enable png image support for the VNC console server |
+seccomp
|
Enable seccomp (secure computing mode) to perform system call filtering at runtime to increase security of programs |
+slirp
|
Enable TCP/IP in hypervisor via net-libs/libslirp |
+vhost-net
|
Enable accelerated networking using vhost-net, see https://www.linux-kvm.org/page/VhostNet |
+vnc
|
Enable VNC (remote desktop viewer) support |
accessibility
|
Adds support for braille displays using brltty |
alsa
|
Enable alsa output for sound emulation |
bpf
|
Enable eBPF support for RSS implementation. |
bzip2
|
Enable bzip2 compression support |
capstone
|
Enable disassembly support with dev-libs/capstone |
debug
|
Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces |
fuse
|
Enables FUSE block device export |
glusterfs
|
Enables GlusterFS cluster fileystem via sys-cluster/glusterfs |
gtk
|
Add support for x11-libs/gtk+ (The GIMP Toolkit) |
infiniband
|
Enable Infiniband RDMA transport support |
io-uring
|
Enable efficient I/O via sys-libs/liburing. |
iscsi
|
Enable direct iSCSI support via net-libs/libiscsi instead of indirectly via the Linux block layer that sys-block/open-iscsi does. |
jack
|
Add support for the JACK Audio Connection Kit |
jemalloc
|
Enable jemalloc allocator support |
keyutils
|
Support Linux keyrings via sys-apps/keyutils |
lzo
|
Enable support for lzo compression |
multipath
|
Enable multipath persistent reservation passthrough via sys-fs/multipath-tools. |
ncurses
|
Enable the ncurses-based console |
nfs
|
Enable NFS support |
nls
|
Add Native Language Support (using gettext - GNU locale utilities) |
numa
|
Enable NUMA support |
opengl
|
Add support for OpenGL (3D graphics) |
pam
|
Add support for PAM (Pluggable Authentication Modules) - DANGEROUS to arbitrarily flip |
pipewire
|
Enable pipewire output for sound emulation |
plugins
|
Enable qemu plugin API via shared library loading. |
pulseaudio
|
Enable pulseaudio output for sound emulation |
python
|
Add optional support/bindings for the Python language |
rbd
|
Enable rados block device backend support, see https://docs.ceph.com/en/mimic/rbd/qemu-rbd/ |
sasl
|
Add support for the Simple Authentication and Security Layer |
sdl
|
Enable the SDL-based console |
sdl-image
|
SDL Image support for icons |
selinux
|
!!internal use only!! Security Enhanced Linux support, this must be set by the selinux profile or breakage will occur |
smartcard
|
Enable smartcard support |
snappy
|
Enable support for Snappy compression (as implemented in app-arch/snappy) |
spice
|
Enable Spice protocol support via app-emulation/spice |
ssh
|
Enable SSH based block device support via net-libs/libssh2 |
static
|
Build the User and Software MMU (system) targets as well as tools as static binaries |
static-user
|
Build the User targets as static binaries |
systemtap
|
Enable SystemTap/DTrace tracing |
test
|
Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled independently) |
udev
|
Enable virtual/udev integration (device discovery, power and storage device support, etc) |
usb
|
Enable USB passthrough via dev-libs/libusb |
usbredir
|
Use sys-apps/usbredir to redirect USB devices to another machine over TCP |
vde
|
Enable VDE-based networking |
virgl
|
Enable experimental Virgil 3d (virtual software GPU) |
virtfs
|
Enable VirtFS via virtio-9p-pci / fsdev. See https://wiki.qemu.org/Documentation/9psetup |
vte
|
Enable terminal support (x11-libs/vte) in the GTK+ interface |
xattr
|
Add support for getting and setting POSIX extended attributes, through sys-apps/attr. Requisite for the virtfs backend. |
xen
|
Enables support for Xen backends |
zstd
|
Enable support for ZSTD compression |
More than one USE flag (
gtk
, ncurses
, sdl
, or spice
) can be enabled for graphical output. If graphics are desired it is generally recommended to enable more than one graphical USE flag.If virt-manager is going to be used, be sure to enable the
usbredir
and spice
USE flags on the qemu package for correct operation.USE_EXPAND
Additional ebuild configuration frobs are provided as the USE_EXPAND variables QEMU_USER_TARGETS and QEMU_SOFTMMU_TARGETS. See app-emulation/qemu for a list of all the available targets (there are many; most are very obscure and may be ignored; leaving these variables at their default values will disable almost everything which is probably just fine for most users).
For each target specified, a qemu executable will be built. A softmmu
target is the standard qemu use-case of emulating an entire system (like VirtualBox or VMWare, but with optional support for emulating CPU hardware along with peripherals). user
targets execute user-mode code only; the (somewhat shockingly ambitious) purpose of these targets is to "magically" allow importing user-space linux ELF binaries from a different architecture into the native system (like multilib, without the awkward need for a software stack or CPU capable of running it).
In order to enable QEMU_USER_TARGETS and QEMU_SOFTMMU_TARGETS we can edit the variables globally in /etc/portage/make.conf, i.e.:
/etc/portage/make.conf
QEMU_SOFTMMU_TARGETS="arm x86_64 sparc"
QEMU_USER_TARGETS="x86_64"
Or, the /etc/portage/package.use file(s) can be modified. Two equivalent syntaxes are available: traditional USE flag syntax, i.e.:
/etc/portage/package.use
app-emulation/qemu qemu_softmmu_targets_arm qemu_softmmu_targets_x86_64 qemu_softmmu_targets_sparc
app-emulation/qemu qemu_user_targets_x86_64
Another alternative is to use the newer USE_EXPAND-specific syntax:
/etc/portage/package.use
app-emulation/qemu QEMU_SOFTMMU_TARGETS: arm x86_64 sparc QEMU_USER_TARGETS: x86_64
Emerge
After reviewing and adding any desired USE flags, emerge app-emulation/qemu:
root #
emerge --ask app-emulation/qemu
Configuration
The following sub-articles provide detailed instructions on QEMU configurations and options:
- Usage options - Contains common configuration options used with QEMU (graphics/display, networking, RAM, storage, processor, etc).
- QEMU/Linux guest — describes the setup of a Gentoo Linux guest in QEMU using Gentoo bootable media.
- QEMU/Windows guest — setup of a Windows guest using QEMU
- OS2WarpV3 guest - Describes the configuration steps needed to setup a virtualized OS2WarpVs=3 guest with QEMU.
Environment variables
- G_MESSAGES_DEBUG
- LISTEN_FDS
- LISTEN_PID
- QEMU_AUDIO_DRV
- QEMU_MODULE_DIR
- XDG_RUNTIME_DIR
Files
Files that QEMU uses.
Single File
- /etc/libvirt/qemu.conf - QEMU configuration file.
- /etc/libvirt/qemu-lockd.conf - QEMU lock files
- /etc/libvirt/qemu-sanlock.conf - QEMU SAN lock
- /etc/libvirt/qemu/<domain-name>.xml - Domain XML setting for a virtual machine or container.
- /etc/libvirt/qemu/autostart/<domain-name>.xml - Autostart this domain (virtual machine or container).
- /etc/libvirt/qemu/networks/<network-name>.xml - Network XML setting file for a network connection
- /etc/libvirt/qemu/networks/autostart/<network-name>.xml - Autostart this network connection.
- /var/lib/libvirt/qemu/channel/target/<domain-name>/<socket-file> - UNIX socket file for Libvertd daemon API
- /var/cache/libvirt/qemu/capabilities/<hash-value>.xml - Host OS capabilities in XML-format
- /var/lib/libvirt/qemu/checkpoint/
- /var/lib/libvirt/qemu/<domain-9-XXXX>/ - holds UNIX sockets and AES keys for this domain.
- /var/lib/libvirt/qemu/dump/
- /var/lib/libvirt/qemu/nvram/
- /var/lib/libvirt/qemu/ram/
- /var/lib/libvirt/qemu/save/ - holding directory of hibernation images
- /var/lib/libvirt/qemu/snapshot/ - holding directory of snapshots
- /var/run/libvirt/qemu - various UNIX socket and PID files for libvirtd daemon.
Image File
QEMU supports the following disk image formats:
- QEMU copy-on-write (.qcow2, .qed, .qcow, .cow)
- VirtualBox Virtual Disk Image (.vdi)
- CD/DVD (ISO-9660) images (.iso)
- Raw images (.img), that guest OS can control
- VFAT-16
- VMware Virtual Machine Disk (.vmdk)
- Virtual PC Virtual Hard Disk (.vhd)
- Parallels disk image (.hdd, .hds) – Read-only
- Apple macos Universal Disk Image Format (.dmg) – Read-only
- Bochs – Read-only
- Linux cloop – Read-only
Additional software
User name qemu is required; Defined by acct-user/qemu; evoked by sys-emulator/qemu package.
Group name qemu is required. Defined by acct-group/qemu; evoked by sys-emulator/qemu package.
Usage
Qemu can be used in two ways, with GUI front ends and through the command line. The configuration of QEMU depends on which method is employed:
- GUI (Front-End) - To make life easier, there are multiple user-friendly front ends to QEMU: See QEMU front-ends
- CLI
Invocation
QEMU supports around 34 different CPU architectures. To find the desired architecture, list what is installed.
user $
ls /usr/bin/qemu-system-*
/usr/bin/qemu-system-aarch64 /usr/bin/qemu-system-mips /usr/bin/qemu-system-rx /usr/bin/qemu-system-alpha /usr/bin/qemu-system-mips64 /usr/bin/qemu-system-s390x /usr/bin/qemu-system-arm /usr/bin/qemu-system-mips64el /usr/bin/qemu-system-sh4 /usr/bin/qemu-system-avr /usr/bin/qemu-system-mipsel /usr/bin/qemu-system-sh4eb /usr/bin/qemu-system-cris /usr/bin/qemu-system-nios2 /usr/bin/qemu-system-sparc /usr/bin/qemu-system-hppa /usr/bin/qemu-system-or1k /usr/bin/qemu-system-sparc64 /usr/bin/qemu-system-i386 /usr/bin/qemu-system-ppc /usr/bin/qemu-system-tricore /usr/bin/qemu-system-loongarch64 /usr/bin/qemu-system-ppc64 /usr/bin/qemu-system-x86_64 /usr/bin/qemu-system-m68k /usr/bin/qemu-system-ppc64le /usr/bin/qemu-system-x86_64-microvm /usr/bin/qemu-system-microblaze /usr/bin/qemu-system-riscv32 /usr/bin/qemu-system-xtensa /usr/bin/qemu-system-microblazeel /usr/bin/qemu-system-riscv64 /usr/bin/qemu-system-xtensaeb
user $
qemu-system-x86_64 -help
QEMU emulator version 7.2.9 (Debian 1:7.2+dfsg-7+deb12u5) Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers usage: qemu-system-x86_64 [options] [disk_image] 'disk_image' is a raw hard disk image for IDE hard disk 0 Standard options: -h or -help display this help and exit -version display version information and exit -machine [type=]name[,prop[=value][,...]] selects emulated machine ('-machine help' for list) property accel=accel1[:accel2[:...]] selects accelerator supported accelerators are kvm, xen, hax, hvf, nvmm, whpx or tcg (default: tcg) vmport=on|off|auto controls emulation of vmport (default: auto) dump-guest-core=on|off include guest memory in a core dump (default=on) mem-merge=on|off controls memory merge support (default: on) aes-key-wrap=on|off controls support for AES key wrapping (default=on) dea-key-wrap=on|off controls support for DEA key wrapping (default=on) suppress-vmdesc=on|off disables self-describing migration (default=off) nvdimm=on|off controls NVDIMM support (default=off) memory-encryption=@var{} memory encryption object to use (default=none) hmat=on|off controls ACPI HMAT support (default=off) memory-backend='backend-id' specifies explicitly provided backend for main RAM (default=none) cxl-fmw.0.targets.0=firsttarget,cxl-fmw.0.targets.1=secondtarget,cxl-fmw.0.size=size[,cxl-fmw.0.interleave-granularity=granularity] sgx-epc.0.memdev=memid,sgx-epc.0.node=numaid -cpu cpu select CPU ('-cpu help' for list) -accel [accel=]accelerator[,prop[=value][,...]] select accelerator (kvm, xen, hax, hvf, nvmm, whpx or tcg; use 'help' for a list) igd-passthru=on|off (enable Xen integrated Intel graphics passthrough, default=off) kernel-irqchip=on|off|split controls accelerated irqchip support (default=on) kvm-shadow-mem=size of KVM shadow MMU in bytes split-wx=on|off (enable TCG split w^x mapping) tb-size=n (TCG translation block cache size) dirty-ring-size=n (KVM dirty ring GFN count, default 0) notify-vmexit=run|internal-error|disable,notify-window=n (enable notify VM exit and set notify window, x86 only) thread=single|multi (enable multi-threaded TCG) -smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads] set the number of initial CPUs to 'n' [default=1] maxcpus= maximum number of total CPUs, including offline CPUs for hotplug, etc sockets= number of sockets on the machine board dies= number of dies in one socket clusters= number of clusters in one die cores= number of cores in one cluster threads= number of threads in one core Note: Different machines may have different subsets of the CPU topology parameters supported, so the actual meaning of the supported parameters will vary accordingly. For example, for a machine type that supports a three-level CPU hierarchy of sockets/cores/threads, the parameters will sequentially mean as below: sockets means the number of sockets on the machine board cores means the number of cores in one socket threads means the number of threads in one core For a particular machine type board, an expected CPU topology hierarchy can be defined through the supported sub-option. Unsupported parameters can also be provided in addition to the sub-option, but their values must be set as 1 in the purpose of correct parsing. -numa node[,mem=size][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=node] -numa node[,memdev=id][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=node] -numa dist,src=source,dst=destination,val=distance -numa cpu,node-id=node[,socket-id=x][,core-id=y][,thread-id=z] -numa hmat-lb,initiator=node,target=node,hierarchy=memory|first-level|second-level|third-level,data-type=access-latency|read-latency|write-latency[,latency=lat][,bandwidth=bw] -numa hmat-cache,node-id=node,size=size,level=level[,associativity=none|direct|complex][,policy=none|write-back|write-through][,line=size] -add-fd fd=fd,set=set[,opaque=opaque] Add 'fd' to fd 'set' -set group.id.arg=value set <arg> parameter for item <id> of type <group> i.e. -set drive.$id.file=/path/to/image -global driver.property=value -global driver=driver,property=property,value=value set a global default for a driver property -boot [order=drives][,once=drives][,menu=on|off] [,splash=sp_name][,splash-time=sp_time][,reboot-timeout=rb_time][,strict=on|off] 'drives': floppy (a), hard disk (c), CD-ROM (d), network (n) 'sp_name': the file's name that would be passed to bios as logo picture, if menu=on 'sp_time': the period that splash picture last if menu=on, unit is ms 'rb_timeout': the timeout before guest reboot when boot failed, unit is ms -m [size=]megs[,slots=n,maxmem=size] configure guest RAM size: initial amount of guest memory slots: number of hotplug slots (default: none) maxmem: maximum amount of guest memory (default: none) NOTE: Some architectures might enforce a specific granularity -mem-path FILE provide backing storage for guest RAM -mem-prealloc preallocate guest memory (use with -mem-path) -k language use keyboard layout (for example 'fr' for French) -audio-help show -audiodev equivalent of the currently specified audio settings -audio [driver=]driver,model=value[,prop[=value][,...]] specifies the audio backend and device to use; apart from 'model', options are the same as for -audiodev. use '-audio model=help' to show possible devices. -audiodev [driver=]driver,id=id[,prop[=value][,...]] specifies the audio backend to use Use ``-audiodev help`` to list the available drivers id= identifier of the backend timer-period= timer period in microseconds in|out.mixing-engine= use mixing engine to mix streams inside QEMU in|out.fixed-settings= use fixed settings for host audio in|out.frequency= frequency to use with fixed settings in|out.channels= number of channels to use with fixed settings in|out.format= sample format to use with fixed settings valid values: s8, s16, s32, u8, u16, u32, f32 in|out.voices= number of voices to use in|out.buffer-length= length of buffer in microseconds -audiodev none,id=id,[,prop[=value][,...]] dummy driver that discards all output -audiodev alsa,id=id[,prop[=value][,...]] in|out.dev= name of the audio device to use in|out.period-length= length of period in microseconds in|out.try-poll= attempt to use poll mode threshold= threshold (in microseconds) when playback starts -audiodev oss,id=id[,prop[=value][,...]] in|out.dev= path of the audio device to use in|out.buffer-count= number of buffers in|out.try-poll= attempt to use poll mode try-mmap= try using memory mapped access exclusive= open device in exclusive mode dsp-policy= set timing policy (0..10), -1 to use fragment mode -audiodev pa,id=id[,prop[=value][,...]] server= PulseAudio server address in|out.name= source/sink device name in|out.latency= desired latency in microseconds -audiodev sdl,id=id[,prop[=value][,...]] in|out.buffer-count= number of buffers -audiodev sndio,id=id[,prop[=value][,...]] -audiodev spice,id=id[,prop[=value][,...]] -audiodev dbus,id=id[,prop[=value][,...]] -audiodev wav,id=id[,prop[=value][,...]] path= path of wav file to record -device driver[,prop[=value][,...]] add device (based on driver) prop=value,... sets driver properties use '-device help' to print all possible drivers use '-device driver,help' to print all possible properties -name string1[,process=string2][,debug-threads=on|off] set the name of the guest string1 sets the window title and string2 the process name When debug-threads is enabled, individual threads are given a separate name NOTE: The thread names are for debugging and not a stable API. -uuid %08x-%04x-%04x-%04x-%012x specify machine UUID Block device options: -fda/-fdb file use 'file' as floppy disk 0/1 image -hda/-hdb file use 'file' as hard disk 0/1 image -hdc/-hdd file use 'file' as hard disk 2/3 image -cdrom file use 'file' as CD-ROM image -blockdev [driver=]driver[,node-name=N][,discard=ignore|unmap] [,cache.direct=on|off][,cache.no-flush=on|off] [,read-only=on|off][,auto-read-only=on|off] [,force-share=on|off][,detect-zeroes=on|off|unmap] [,driver specific parameters...] configure a block backend -drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i] [,cache=writethrough|writeback|none|directsync|unsafe][,format=f] [,snapshot=on|off][,rerror=ignore|stop|report] [,werror=ignore|stop|report|enospc][,id=name] [,aio=threads|native|io_uring] [,readonly=on|off][,copy-on-read=on|off] [,discard=ignore|unmap][,detect-zeroes=on|off|unmap] [[,bps=b]|[[,bps_rd=r][,bps_wr=w]]] [[,iops=i]|[[,iops_rd=r][,iops_wr=w]]] [[,bps_max=bm]|[[,bps_rd_max=rm][,bps_wr_max=wm]]] [[,iops_max=im]|[[,iops_rd_max=irm][,iops_wr_max=iwm]]] [[,iops_size=is]] [[,group=g]] use 'file' as a drive image -mtdblock file use 'file' as on-board Flash memory image -sd file use 'file' as SecureDigital card image -snapshot write to temporary files instead of disk image files -fsdev local,id=id,path=path,security_model=mapped-xattr|mapped-file|passthrough|none [,writeout=immediate][,readonly=on][,fmode=fmode][,dmode=dmode] [[,throttling.bps-total=b]|[[,throttling.bps-read=r][,throttling.bps-write=w]]] [[,throttling.iops-total=i]|[[,throttling.iops-read=r][,throttling.iops-write=w]]] [[,throttling.bps-total-max=bm]|[[,throttling.bps-read-max=rm][,throttling.bps-write-max=wm]]] [[,throttling.iops-total-max=im]|[[,throttling.iops-read-max=irm][,throttling.iops-write-max=iwm]]] [[,throttling.iops-size=is]] -fsdev proxy,id=id,socket=socket[,writeout=immediate][,readonly=on] -fsdev proxy,id=id,sock_fd=sock_fd[,writeout=immediate][,readonly=on] -fsdev synth,id=id -virtfs local,path=path,mount_tag=tag,security_model=mapped-xattr|mapped-file|passthrough|none [,id=id][,writeout=immediate][,readonly=on][,fmode=fmode][,dmode=dmode][,multidevs=remap|forbid|warn] -virtfs proxy,mount_tag=tag,socket=socket[,id=id][,writeout=immediate][,readonly=on] -virtfs proxy,mount_tag=tag,sock_fd=sock_fd[,id=id][,writeout=immediate][,readonly=on] -virtfs synth,mount_tag=tag[,id=id][,readonly=on] -iscsi [user=user][,password=password] [,header-digest=CRC32C|CR32C-NONE|NONE-CRC32C|NONE [,initiator-name=initiator-iqn][,id=target-iqn] [,timeout=timeout] iSCSI session parameters USB convenience options: -usb enable on-board USB host controller (if not enabled by default) -usbdevice name add the host or guest USB device 'name' Display options: -display spice-app[,gl=on|off] -display sdl[,gl=on|core|es|off][,grab-mod=<mod>][,show-cursor=on|off] [,window-close=on|off] -display gtk[,full-screen=on|off][,gl=on|off][,grab-on-hover=on|off] [,show-tabs=on|off][,show-cursor=on|off][,window-close=on|off] [,show-menubar=on|off] -display vnc=<display>[,<optargs>] -display curses[,charset=<encoding>] -display egl-headless[,rendernode=<file>] -display dbus[,addr=<dbusaddr>] [,gl=on|core|es|off][,rendernode=<file>] -display none select display backend type The default display is equivalent to "-display gtk" -nographic disable graphical output and redirect serial I/Os to console -spice [port=port][,tls-port=secured-port][,x509-dir=<dir>] [,x509-key-file=<file>][,x509-key-password=<file>] [,x509-cert-file=<file>][,x509-cacert-file=<file>] [,x509-dh-key-file=<file>][,addr=addr] [,ipv4=on|off][,ipv6=on|off][,unix=on|off] [,tls-ciphers=<list>] [,tls-channel=[main|display|cursor|inputs|record|playback]] [,plaintext-channel=[main|display|cursor|inputs|record|playback]] [,sasl=on|off][,disable-ticketing=on|off] [,password=<string>][,password-secret=<secret-id>] [,image-compression=[auto_glz|auto_lz|quic|glz|lz|off]] [,jpeg-wan-compression=[auto|never|always]] [,zlib-glz-wan-compression=[auto|never|always]] [,streaming-video=[off|all|filter]][,disable-copy-paste=on|off] [,disable-agent-file-xfer=on|off][,agent-mouse=[on|off]] [,playback-compression=[on|off]][,seamless-migration=[on|off]] [,gl=[on|off]][,rendernode=<file>] enable spice at least one of {port, tls-port} is mandatory -portrait rotate graphical output 90 deg left (only PXA LCD) -rotate <deg> rotate graphical output some deg left (only PXA LCD) -vga [std|cirrus|vmware|qxl|xenfb|tcx|cg3|virtio|none] select video card type -full-screen start in full screen -vnc <display> shorthand for -display vnc=<display> i386 target only: -win2k-hack use it when installing Windows 2000 to avoid a disk full bug -no-fd-bootchk disable boot signature checking for floppy disks -no-acpi disable ACPI -no-hpet disable HPET -acpitable [sig=str][,rev=n][,oem_id=str][,oem_table_id=str][,oem_rev=n][,asl_compiler_id=str][,asl_compiler_rev=n][,{data|file}=file1[:file2]...] ACPI table description -smbios file=binary load SMBIOS entry from binary file -smbios type=0[,vendor=str][,version=str][,date=str][,release=%d.%d] [,uefi=on|off] specify SMBIOS type 0 fields -smbios type=1[,manufacturer=str][,product=str][,version=str][,serial=str] [,uuid=uuid][,sku=str][,family=str] specify SMBIOS type 1 fields -smbios type=2[,manufacturer=str][,product=str][,version=str][,serial=str] [,asset=str][,location=str] specify SMBIOS type 2 fields -smbios type=3[,manufacturer=str][,version=str][,serial=str][,asset=str] [,sku=str] specify SMBIOS type 3 fields -smbios type=4[,sock_pfx=str][,manufacturer=str][,version=str][,serial=str] [,asset=str][,part=str][,max-speed=%d][,current-speed=%d] [,processor-id=%d] specify SMBIOS type 4 fields -smbios type=8[,external_reference=str][,internal_reference=str][,connector_type=%d][,port_type=%d] specify SMBIOS type 8 fields -smbios type=11[,value=str][,path=filename] specify SMBIOS type 11 fields -smbios type=17[,loc_pfx=str][,bank=str][,manufacturer=str][,serial=str] [,asset=str][,part=str][,speed=%d] specify SMBIOS type 17 fields -smbios type=41[,designation=str][,kind=str][,instance=%d][,pcidev=str] specify SMBIOS type 41 fields Network options: -netdev user,id=str[,ipv4=on|off][,net=addr[/mask]][,host=addr] [,ipv6=on|off][,ipv6-net=addr[/int]][,ipv6-host=addr] [,restrict=on|off][,hostname=host][,dhcpstart=addr] [,dns=addr][,ipv6-dns=addr][,dnssearch=domain][,domainname=domain] [,tftp=dir][,tftp-server-name=name][,bootfile=f][,hostfwd=rule][,guestfwd=rule][,smb=dir[,smbserver=addr]] configure a user mode network backend with ID 'str', its DHCP server and optional services -netdev tap,id=str[,fd=h][,fds=x:y:...:z][,ifname=name][,script=file][,downscript=dfile] [,br=bridge][,helper=helper][,sndbuf=nbytes][,vnet_hdr=on|off][,vhost=on|off] [,vhostfd=h][,vhostfds=x:y:...:z][,vhostforce=on|off][,queues=n] [,poll-us=n] configure a host TAP network backend with ID 'str' connected to a bridge (default=br0) use network scripts 'file' (default=/etc/qemu-ifup) to configure it and 'dfile' (default=/etc/qemu-ifdown) to deconfigure it use '[down]script=no' to disable script execution use network helper 'helper' (default=/usr/lib/qemu/qemu-bridge-helper) to configure it use 'fd=h' to connect to an already opened TAP interface use 'fds=x:y:...:z' to connect to already opened multiqueue capable TAP interfaces use 'sndbuf=nbytes' to limit the size of the send buffer (the default is disabled 'sndbuf=0' to enable flow control set 'sndbuf=1048576') use vnet_hdr=off to avoid enabling the IFF_VNET_HDR tap flag use vnet_hdr=on to make the lack of IFF_VNET_HDR support an error condition use vhost=on to enable experimental in kernel accelerator (only has effect for virtio guests which use MSIX) use vhostforce=on to force vhost on for non-MSIX virtio guests use 'vhostfd=h' to connect to an already opened vhost net device use 'vhostfds=x:y:...:z to connect to multiple already opened vhost net devices use 'queues=n' to specify the number of queues to be created for multiqueue TAP use 'poll-us=n' to specify the maximum number of microseconds that could be spent on busy polling for vhost net -netdev bridge,id=str[,br=bridge][,helper=helper] configure a host TAP network backend with ID 'str' that is connected to a bridge (default=br0) using the program 'helper (default=/usr/lib/qemu/qemu-bridge-helper) -netdev l2tpv3,id=str,src=srcaddr,dst=dstaddr[,srcport=srcport][,dstport=dstport] [,rxsession=rxsession],txsession=txsession[,ipv6=on|off][,udp=on|off] [,cookie64=on|off][,counter][,pincounter][,txcookie=txcookie] [,rxcookie=rxcookie][,offset=offset] configure a network backend with ID 'str' connected to an Ethernet over L2TPv3 pseudowire. Linux kernel 3.3+ as well as most routers can talk L2TPv3. This transport allows connecting a VM to a VM, VM to a router and even VM to Host. It is a nearly-universal standard (RFC3931). Note - this implementation uses static pre-configured tunnels (same as the Linux kernel). use 'src=' to specify source address use 'dst=' to specify destination address use 'udp=on' to specify udp encapsulation use 'srcport=' to specify source udp port use 'dstport=' to specify destination udp port use 'ipv6=on' to force v6 L2TPv3 uses cookies to prevent misconfiguration as well as a weak security measure use 'rxcookie=0x012345678' to specify a rxcookie use 'txcookie=0x012345678' to specify a txcookie use 'cookie64=on' to set cookie size to 64 bit, otherwise 32 use 'counter=off' to force a 'cut-down' L2TPv3 with no counter use 'pincounter=on' to work around broken counter handling in peer use 'offset=X' to add an extra offset between header and data -netdev socket,id=str[,fd=h][,listen=[host]:port][,connect=host:port] configure a network backend to connect to another network using a socket connection -netdev socket,id=str[,fd=h][,mcast=maddr:port[,localaddr=addr]] configure a network backend to connect to a multicast maddr and port use 'localaddr=addr' to specify the host address to send packets from -netdev socket,id=str[,fd=h][,udp=host:port][,localaddr=host:port] configure a network backend to connect to another network using an UDP tunnel -netdev stream,id=str[,server=on|off],addr.type=inet,addr.host=host,addr.port=port[,to=maxport][,numeric=on|off][,keep-alive=on|off][,mptcp=on|off][,addr.ipv4=on|off][,addr.ipv6=on|off] -netdev stream,id=str[,server=on|off],addr.type=unix,addr.path=path[,abstract=on|off][,tight=on|off] -netdev stream,id=str[,server=on|off],addr.type=fd,addr.str=file-descriptor configure a network backend to connect to another network using a socket connection in stream mode. -netdev dgram,id=str,remote.type=inet,remote.host=maddr,remote.port=port[,local.type=inet,local.host=addr] -netdev dgram,id=str,remote.type=inet,remote.host=maddr,remote.port=port[,local.type=fd,local.str=file-descriptor] configure a network backend to connect to a multicast maddr and port use ``local.host=addr`` to specify the host address to send packets from -netdev dgram,id=str,local.type=inet,local.host=addr,local.port=port[,remote.type=inet,remote.host=addr,remote.port=port] -netdev dgram,id=str,local.type=unix,local.path=path[,remote.type=unix,remote.path=path] -netdev dgram,id=str,local.type=fd,local.str=file-descriptor configure a network backend to connect to another network using an UDP tunnel -netdev vde,id=str[,sock=socketpath][,port=n][,group=groupname][,mode=octalmode] configure a network backend to connect to port 'n' of a vde switch running on host and listening for incoming connections on 'socketpath'. Use group 'groupname' and mode 'octalmode' to change default ownership and permissions for communication port. -netdev vhost-user,id=str,chardev=dev[,vhostforce=on|off] configure a vhost-user network, backed by a chardev 'dev' -netdev vhost-vdpa,id=str[,vhostdev=/path/to/dev][,vhostfd=h] configure a vhost-vdpa network,Establish a vhost-vdpa netdev use 'vhostdev=/path/to/dev' to open a vhost vdpa device use 'vhostfd=h' to connect to an already opened vhost vdpa device -netdev hubport,id=str,hubid=n[,netdev=nd] configure a hub port on the hub with ID 'n' -nic [tap|bridge|user|l2tpv3|vde|vhost-user|socket][,option][,...][mac=macaddr] initialize an on-board / default host NIC (using MAC address macaddr) and connect it to the given host network backend -nic none use it alone to have zero network devices (the default is to provided a 'user' network connection) -net nic[,macaddr=mac][,model=type][,name=str][,addr=str][,vectors=v] configure or create an on-board (or machine default) NIC and connect it to hub 0 (please use -nic unless you need a hub) -net [user|tap|bridge|vde|socket][,option][,option][,...] old way to initialize a host network interface (use the -netdev option if possible instead) Character device options: -chardev help -chardev null,id=id[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev socket,id=id[,host=host],port=port[,to=to][,ipv4=on|off][,ipv6=on|off][,nodelay=on|off] [,server=on|off][,wait=on|off][,telnet=on|off][,websocket=on|off][,reconnect=seconds][,mux=on|off] [,logfile=PATH][,logappend=on|off][,tls-creds=ID][,tls-authz=ID] (tcp) -chardev socket,id=id,path=path[,server=on|off][,wait=on|off][,telnet=on|off][,websocket=on|off][,reconnect=seconds] [,mux=on|off][,logfile=PATH][,logappend=on|off][,abstract=on|off][,tight=on|off] (unix) -chardev udp,id=id[,host=host],port=port[,localaddr=localaddr] [,localport=localport][,ipv4=on|off][,ipv6=on|off][,mux=on|off] [,logfile=PATH][,logappend=on|off] -chardev msmouse,id=id[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev vc,id=id[[,width=width][,height=height]][[,cols=cols][,rows=rows]] [,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev ringbuf,id=id[,size=size][,logfile=PATH][,logappend=on|off] -chardev file,id=id,path=path[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev pipe,id=id,path=path[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev pty,id=id[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev stdio,id=id[,mux=on|off][,signal=on|off][,logfile=PATH][,logappend=on|off] -chardev braille,id=id[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev serial,id=id,path=path[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev tty,id=id,path=path[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev parallel,id=id,path=path[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev parport,id=id,path=path[,mux=on|off][,logfile=PATH][,logappend=on|off] -chardev spicevmc,id=id,name=name[,debug=debug][,logfile=PATH][,logappend=on|off] -chardev spiceport,id=id,name=name[,debug=debug][,logfile=PATH][,logappend=on|off] TPM device options: -tpmdev passthrough,id=id[,path=path][,cancel-path=path] use path to provide path to a character device; default is /dev/tpm0 use cancel-path to provide path to TPM's cancel sysfs entry; if not provided it will be searched for in /sys/class/misc/tpm?/device -tpmdev emulator,id=id,chardev=dev configure the TPM device using chardev backend Boot Image or Kernel specific: -bios file set the filename for the BIOS -pflash file use 'file' as a parallel flash image -kernel bzImage use 'bzImage' as kernel image -append cmdline use 'cmdline' as kernel command line -initrd file use 'file' as initial ram disk -dtb file use 'file' as device tree image Debug/Expert options: -compat [deprecated-input=accept|reject|crash][,deprecated-output=accept|hide] Policy for handling deprecated management interfaces -compat [unstable-input=accept|reject|crash][,unstable-output=accept|hide] Policy for handling unstable management interfaces -fw_cfg [name=]<name>,file=<file> add named fw_cfg entry with contents from file -fw_cfg [name=]<name>,string=<str> add named fw_cfg entry with contents from string -serial dev redirect the serial port to char device 'dev' -parallel dev redirect the parallel port to char device 'dev' -monitor dev redirect the monitor to char device 'dev' -qmp dev like -monitor but opens in 'control' mode -qmp-pretty dev like -qmp but uses pretty JSON formatting -mon [chardev=]name[,mode=readline|control][,pretty[=on|off]] -debugcon dev redirect the debug console to char device 'dev' -pidfile file write PID to 'file' -singlestep always run in singlestep mode --preconfig pause QEMU before machine is initialized (experimental) -S freeze CPU at startup (use 'c' to start execution) -overcommit [mem-lock=on|off][cpu-pm=on|off] run qemu with overcommit hints mem-lock=on|off controls memory lock support (default: off) cpu-pm=on|off controls cpu power management (default: off) -gdb dev accept gdb connection on 'dev'. (QEMU defaults to starting the guest without waiting for gdb to connect; use -S too if you want it to not start execution.) -s shorthand for -gdb tcp::1234 -d item1,... enable logging of specified items (use '-d help' for a list of log items) -D logfile output log to logfile (default stderr) -dfilter range,.. filter debug output to range of addresses (useful for -d cpu,exec,etc..) -seed number seed the pseudo-random number generator -L path set the directory for the BIOS, VGA BIOS and keymaps -enable-kvm enable KVM full virtualization support -xen-domid id specify xen guest domain id -xen-attach attach to existing xen domain libxl will use this when starting QEMU -xen-domid-restrict restrict set of available xen operations to specified domain id. (Does not affect xenpv machine type). -no-reboot exit instead of rebooting -no-shutdown stop before shutdown -action reboot=reset|shutdown action when guest reboots [default=reset] -action shutdown=poweroff|pause action when guest shuts down [default=poweroff] -action panic=pause|shutdown|exit-failure|none action when guest panics [default=shutdown] -action watchdog=reset|shutdown|poweroff|inject-nmi|pause|debug|none action when watchdog fires [default=reset] -loadvm [tag|id] start right away with a saved state (loadvm in monitor) -daemonize daemonize QEMU after initializing -option-rom rom load a file, rom, into the option ROM space -rtc [base=utc|localtime|<datetime>][,clock=host|rt|vm][,driftfix=none|slew] set the RTC base and clock, enable drift fix for clock ticks (x86 only) -icount [shift=N|auto][,align=on|off][,sleep=on|off][,rr=record|replay,rrfile=<filename>[,rrsnapshot=<snapshot>]] enable virtual instruction counter with 2^N clock ticks per instruction, enable aligning the host and virtual clocks or disable real time cpu sleeping, and optionally enable record-and-replay mode -watchdog-action reset|shutdown|poweroff|inject-nmi|pause|debug|none action when watchdog fires [default=reset] -echr chr set terminal escape character instead of ctrl-a -incoming tcp:[host]:port[,to=maxport][,ipv4=on|off][,ipv6=on|off] -incoming rdma:host:port[,ipv4=on|off][,ipv6=on|off] -incoming unix:socketpath prepare for incoming migration, listen on specified protocol and socket address -incoming fd:fd -incoming exec:cmdline accept incoming migration on given file descriptor or from given external command -incoming defer wait for the URI to be specified via migrate_incoming -only-migratable allow only migratable devices -nodefaults don't create default devices -chroot dir chroot to dir just before starting the VM -runas user change to user id user just before starting the VM user can be numeric uid:gid instead -sandbox on[,obsolete=allow|deny][,elevateprivileges=allow|deny|children] [,spawn=allow|deny][,resourcecontrol=allow|deny] Enable seccomp mode 2 system call filter (default 'off'). use 'obsolete' to allow obsolete system calls that are provided by the kernel, but typically no longer used by modern C library implementations. use 'elevateprivileges' to allow or deny the QEMU process ability to elevate privileges using set*uid|gid system calls. The value 'children' will deny set*uid|gid system calls for main QEMU process but will allow forks and execves to run unprivileged use 'spawn' to avoid QEMU to spawn new threads or processes by blocking *fork and execve use 'resourcecontrol' to disable process affinity and schedular priority -readconfig <file> read config file -no-user-config do not load default user-provided config files at startup -trace [[enable=]<pattern>][,events=<file>][,file=<file>] specify tracing options -plugin [file=]<file>[,<argname>=<argvalue>] load a plugin -async-teardown enable asynchronous teardown -msg [timestamp[=on|off]][,guest-name=[on|off]] control error message format timestamp=on enables timestamps (default: off) guest-name=on enables guest name prefix but only if -name guest option is set (default: off) -dump-vmstate <file> Output vmstate information in JSON format to file. Use the scripts/vmstate-static-checker.py file to check for possible regressions in migration code by comparing two such vmstate dumps. -enable-sync-profile enable synchronization profiling Generic object creation: -object TYPENAME[,PROP1=VALUE1,...] create a new object of type TYPENAME setting properties in the order they are specified. Note that the 'id' property must be set. These objects are placed in the '/objects' path. During emulation, the following keys are useful: ctrl-alt-f toggle full screen ctrl-alt-n switch to virtual console 'n' ctrl-alt toggle mouse and keyboard grab When using -nographic, press 'ctrl-a h' to get some help. See <https://qemu.org/contribute/report-a-bug> for how to report bugs. More information on the QEMU project at <https://qemu.org>.
Permissions
In order to run a KVM accelerated virtual machine without logging as root, add normal users to the kvm group. Replace <username>
in the example command below with the appropriate user(s):
root #
gpasswd -a <username> kvm
Creation of a disk image
To create a raw disk image with with 40G size:
user $
qemu-img create -f raw my-systems-disk-image.img 40G
TO create a raw image with copy-on-write disabled (nocow): "nocow" is a file attribute. (check with lsattr)
user $
qemu-img create -f raw my-systems-disk-image.img -o nocow=on 40G
This would create a qcow2 image (useful if your filesystem doesn't support sparse files):
user $
qemu-img create -f qcow2 my-systems-disk-image.qcow2 40G
Preparation of a bootable disk image from scratch
A system can be copied on to a disk image when not using a cdrom installation medium. By default qemu uses a "bios-firmware" to boot the system. The disk can be prepared with a msdos disklabel and a gap between the end of the 512 byte MBR (Master Boot Record) and the start of the first partition. The gap is needed for boot loaders like grub that place boot-code in the gap.
A raw disk image can be prepared by attaching it as a loop device:
user $
losetup -fP /path/to/my-systems-disk-image.img
- -f find the first unused loop device
- -P scans for the partitions
List the loop devices with this command:
user $
losetup -l
Then the loop device can be formatted like a normal disk.
Print the partition table:
user $
parted /dev/loop000 print
Create a msdos disklabel with:
user $
parted /dev/loop000-number-of-the-device-whose-data-will-be-lost mklabel msdos
Create an ext4 partition:
user $
parted /dev/loop000 mkpart primary ext4 1Mib 40GiB
Set the boot flag:
user $
parted /dev/loop000 set 1 boot on
Create a filesystem:
user $
mkfs.ext4 /dev/loop000
Mount it somewhere
user $
mount /dev/loop000 /mnt/my-new-fs
Create a boot/grub folder for grub.
user $
mkdir -p /mnt/my-new-fs/boot/grub
Install grub on the loop device and advice grub to install its files in boot/grub
user $
grub-install --boot-directory=/mnt/my-new-fs/boot/grub /dev/loop000
Unmount the filesystem and detach the loop device
user $
umount /mnt/my-new-fs
user $
losetup -d /dev/loop000
If the loop device is busy it will not return an error. This is verified with
user $
losetup -l
This is enough to boot into a grub2 boot prompt.
This is can be used as the basis for a bootable system.
CPU selection
QEMU has "accelerators" like kvm(Kernel Virtual Machine) or tcg (Tiny Code Generator) or Xen (wikip[2]).
The accelerator can usually only "accelerate" the features that are available on the host cpu. So the selection of the cpu affects the performance.
To get a list of cpus:
user $
qemu-system-x86_64 -cpu help
Available CPUs: x86 486 (alias configured by machine type) x86 486-v1 x86 Broadwell (alias configured by machine type) x86 Broadwell-IBRS (alias of Broadwell-v3) x86 Broadwell-noTSX (alias of Broadwell-v2) x86 Broadwell-noTSX-IBRS (alias of Broadwell-v4) x86 Broadwell-v1 Intel Core Processor (Broadwell) x86 Broadwell-v2 Intel Core Processor (Broadwell, no TSX) x86 Broadwell-v3 Intel Core Processor (Broadwell, IBRS) x86 Broadwell-v4 Intel Core Processor (Broadwell, no TSX, IBRS) x86 Cascadelake-Server (alias configured by machine type) x86 Cascadelake-Server-noTSX (alias of Cascadelake-Server-v3) x86 Cascadelake-Server-v1 Intel Xeon Processor (Cascadelake) x86 Cascadelake-Server-v2 Intel Xeon Processor (Cascadelake) [ARCH_CAPABILITIES] x86 Cascadelake-Server-v3 Intel Xeon Processor (Cascadelake) [ARCH_CAPABILITIES, no TSX] x86 Cascadelake-Server-v4 Intel Xeon Processor (Cascadelake) [ARCH_CAPABILITIES, no TSX] x86 Cascadelake-Server-v5 Intel Xeon Processor (Cascadelake) [ARCH_CAPABILITIES, EPT switching, XSAVES, no TSX] x86 Conroe (alias configured by machine type) x86 Conroe-v1 Intel Celeron_4x0 (Conroe/Merom Class Core 2) x86 Cooperlake (alias configured by machine type) x86 Cooperlake-v1 Intel Xeon Processor (Cooperlake) x86 Cooperlake-v2 Intel Xeon Processor (Cooperlake) [XSAVES] x86 Denverton (alias configured by machine type) x86 Denverton-v1 Intel Atom Processor (Denverton) x86 Denverton-v2 Intel Atom Processor (Denverton) [no MPX, no MONITOR] x86 Denverton-v3 Intel Atom Processor (Denverton) [XSAVES, no MPX, no MONITOR] x86 Dhyana (alias configured by machine type) x86 Dhyana-v1 Hygon Dhyana Processor x86 Dhyana-v2 Hygon Dhyana Processor [XSAVES] x86 EPYC (alias configured by machine type) x86 EPYC-IBPB (alias of EPYC-v2) x86 EPYC-Milan (alias configured by machine type) x86 EPYC-Milan-v1 AMD EPYC-Milan Processor x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor x86 EPYC-Rome-v2 AMD EPYC-Rome Processor x86 EPYC-v1 AMD EPYC Processor x86 EPYC-v2 AMD EPYC Processor (with IBPB) x86 EPYC-v3 AMD EPYC Processor x86 Haswell (alias configured by machine type) x86 Haswell-IBRS (alias of Haswell-v3) x86 Haswell-noTSX (alias of Haswell-v2) x86 Haswell-noTSX-IBRS (alias of Haswell-v4) x86 Haswell-v1 Intel Core Processor (Haswell) x86 Haswell-v2 Intel Core Processor (Haswell, no TSX) x86 Haswell-v3 Intel Core Processor (Haswell, IBRS) x86 Haswell-v4 Intel Core Processor (Haswell, no TSX, IBRS) x86 Icelake-Server (alias configured by machine type) x86 Icelake-Server-noTSX (alias of Icelake-Server-v2) x86 Icelake-Server-v1 Intel Xeon Processor (Icelake) x86 Icelake-Server-v2 Intel Xeon Processor (Icelake) [no TSX] x86 Icelake-Server-v3 Intel Xeon Processor (Icelake) x86 Icelake-Server-v4 Intel Xeon Processor (Icelake) x86 Icelake-Server-v5 Intel Xeon Processor (Icelake) [XSAVES] x86 Icelake-Server-v6 Intel Xeon Processor (Icelake) [5-level EPT] x86 IvyBridge (alias configured by machine type) x86 IvyBridge-IBRS (alias of IvyBridge-v2) x86 IvyBridge-v1 Intel Xeon E3-12xx v2 (Ivy Bridge) x86 IvyBridge-v2 Intel Xeon E3-12xx v2 (Ivy Bridge, IBRS) x86 KnightsMill (alias configured by machine type) x86 KnightsMill-v1 Intel Xeon Phi Processor (Knights Mill) x86 Nehalem (alias configured by machine type) x86 Nehalem-IBRS (alias of Nehalem-v2) x86 Nehalem-v1 Intel Core i7 9xx (Nehalem Class Core i7) x86 Nehalem-v2 Intel Core i7 9xx (Nehalem Core i7, IBRS update) x86 Opteron_G1 (alias configured by machine type) x86 Opteron_G1-v1 AMD Opteron 240 (Gen 1 Class Opteron) x86 Opteron_G2 (alias configured by machine type) x86 Opteron_G2-v1 AMD Opteron 22xx (Gen 2 Class Opteron) x86 Opteron_G3 (alias configured by machine type) x86 Opteron_G3-v1 AMD Opteron 23xx (Gen 3 Class Opteron) x86 Opteron_G4 (alias configured by machine type) x86 Opteron_G4-v1 AMD Opteron 62xx class CPU x86 Opteron_G5 (alias configured by machine type) x86 Opteron_G5-v1 AMD Opteron 63xx class CPU x86 Penryn (alias configured by machine type) x86 Penryn-v1 Intel Core 2 Duo P9xxx (Penryn Class Core 2) x86 SandyBridge (alias configured by machine type) x86 SandyBridge-IBRS (alias of SandyBridge-v2) x86 SandyBridge-v1 Intel Xeon E312xx (Sandy Bridge) x86 SandyBridge-v2 Intel Xeon E312xx (Sandy Bridge, IBRS update) x86 Skylake-Client (alias configured by machine type) x86 Skylake-Client-IBRS (alias of Skylake-Client-v2) x86 Skylake-Client-noTSX-IBRS (alias of Skylake-Client-v3) x86 Skylake-Client-v1 Intel Core Processor (Skylake) x86 Skylake-Client-v2 Intel Core Processor (Skylake, IBRS) x86 Skylake-Client-v3 Intel Core Processor (Skylake, IBRS, no TSX) x86 Skylake-Client-v4 Intel Core Processor (Skylake, IBRS, no TSX) [IBRS, XSAVES, no TSX] x86 Skylake-Server (alias configured by machine type) x86 Skylake-Server-IBRS (alias of Skylake-Server-v2) x86 Skylake-Server-noTSX-IBRS (alias of Skylake-Server-v3) x86 Skylake-Server-v1 Intel Xeon Processor (Skylake) x86 Skylake-Server-v2 Intel Xeon Processor (Skylake, IBRS) x86 Skylake-Server-v3 Intel Xeon Processor (Skylake, IBRS, no TSX) x86 Skylake-Server-v4 Intel Xeon Processor (Skylake, IBRS, no TSX) x86 Skylake-Server-v5 Intel Xeon Processor (Skylake, IBRS, no TSX) [IBRS, XSAVES, EPT switching, no TSX] x86 Snowridge (alias configured by machine type) x86 Snowridge-v1 Intel Atom Processor (SnowRidge) x86 Snowridge-v2 Intel Atom Processor (Snowridge, no MPX) x86 Snowridge-v3 Intel Atom Processor (Snowridge, no MPX) [XSAVES, no MPX] x86 Snowridge-v4 Intel Atom Processor (Snowridge, no MPX) [no split lock detect, no core-capability] x86 Westmere (alias configured by machine type) x86 Westmere-IBRS (alias of Westmere-v2) x86 Westmere-v1 Westmere E56xx/L56xx/X56xx (Nehalem-C) x86 Westmere-v2 Westmere E56xx/L56xx/X56xx (IBRS update) x86 athlon (alias configured by machine type) x86 athlon-v1 QEMU Virtual CPU version 2.5+ x86 core2duo (alias configured by machine type) x86 core2duo-v1 Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz x86 coreduo (alias configured by machine type) x86 coreduo-v1 Genuine Intel(R) CPU T2600 @ 2.16GHz x86 kvm32 (alias configured by machine type) x86 kvm32-v1 Common 32-bit KVM processor x86 kvm64 (alias configured by machine type) x86 kvm64-v1 Common KVM processor x86 n270 (alias configured by machine type) x86 n270-v1 Intel(R) Atom(TM) CPU N270 @ 1.60GHz x86 pentium (alias configured by machine type) x86 pentium-v1 x86 pentium2 (alias configured by machine type) x86 pentium2-v1 x86 pentium3 (alias configured by machine type) x86 pentium3-v1 x86 phenom (alias configured by machine type) x86 phenom-v1 AMD Phenom(tm) 9550 Quad-Core Processor x86 qemu32 (alias configured by machine type) x86 qemu32-v1 QEMU Virtual CPU version 2.5+ x86 qemu64 (alias configured by machine type) x86 qemu64-v1 QEMU Virtual CPU version 2.5+ x86 base base CPU model type with no features enabled x86 host processor with all supported host features x86 max Enables all features supported by the accelerator in the current host Recognized CPUID flags: 3dnow 3dnowext 3dnowprefetch abm ace2 ace2-en acpi adx aes amd-no-ssb amd-ssbd amd-stibp amx-bf16 amx-int8 amx-tile apic arat arch-capabilities arch-lbr avic avx avx-vnni avx2 avx512-4fmaps avx512-4vnniw avx512-bf16 avx512-fp16 avx512-vp2intersect avx512-vpopcntdq avx512bitalg avx512bw avx512cd avx512dq avx512er avx512f avx512ifma avx512pf avx512vbmi avx512vbmi2 avx512vl avx512vnni bmi1 bmi2 bus-lock-detect cid cldemote clflush clflushopt clwb clzero cmov cmp-legacy core-capability cr8legacy cx16 cx8 dca de decodeassists ds ds-cpl dtes64 erms est extapic f16c flushbyasid fma fma4 fpu fsgsbase fsrm full-width-write fxsr fxsr-opt gfni hle ht hypervisor ia64 ibpb ibrs ibrs-all ibs intel-pt intel-pt-lip invpcid invtsc kvm-asyncpf kvm-asyncpf-int kvm-hint-dedicated kvm-mmu kvm-msi-ext-dest-id kvm-nopiodelay kvm-poll-control kvm-pv-eoi kvm-pv-ipi kvm-pv-sched-yield kvm-pv-tlb-flush kvm-pv-unhalt kvm-steal-time kvmclock kvmclock kvmclock-stable-bit la57 lahf-lm lbrv lm lwp mca mce md-clear mds-no misalignsse mmx mmxext monitor movbe movdir64b movdiri mpx msr mtrr nodeid-msr npt nrip-save nx osvw pae pat pause-filter pbe pcid pclmulqdq pcommit pdcm pdpe1gb perfctr-core perfctr-nb pfthreshold pge phe phe-en pks pku pmm pmm-en pn pni popcnt pschange-mc-no pse pse36 rdctl-no rdpid rdrand rdseed rdtscp rsba rtm sep serialize sgx sgx-debug sgx-exinfo sgx-kss sgx-mode64 sgx-provisionkey sgx-tokenkey sgx1 sgx2 sgxlc sha-ni skinit skip-l1dfl-vmentry smap smep smx spec-ctrl split-lock-detect ss ssb-no ssbd sse sse2 sse4.1 sse4.2 sse4a ssse3 stibp svm svm-lock svme-addr-chk syscall taa-no tbm tce tm tm2 topoext tsc tsc-adjust tsc-deadline tsc-scale tsx-ctrl tsx-ldtrk umip v-vmsave-vmload vaes vgif virt-ssbd vmcb-clean vme vmx vmx-activity-hlt vmx-activity-shutdown vmx-activity-wait-sipi vmx-apicv-register vmx-apicv-vid vmx-apicv-x2apic vmx-apicv-xapic vmx-cr3-load-noexit vmx-cr3-store-noexit vmx-cr8-load-exit vmx-cr8-store-exit vmx-desc-exit vmx-encls-exit vmx-entry-ia32e-mode vmx-entry-load-bndcfgs vmx-entry-load-efer vmx-entry-load-pat vmx-entry-load-perf-global-ctrl vmx-entry-load-pkrs vmx-entry-load-rtit-ctl vmx-entry-noload-debugctl vmx-ept vmx-ept-1gb vmx-ept-2mb vmx-ept-advanced-exitinfo vmx-ept-execonly vmx-eptad vmx-eptp-switching vmx-exit-ack-intr vmx-exit-clear-bndcfgs vmx-exit-clear-rtit-ctl vmx-exit-load-efer vmx-exit-load-pat vmx-exit-load-perf-global-ctrl vmx-exit-load-pkrs vmx-exit-nosave-debugctl vmx-exit-save-efer vmx-exit-save-pat vmx-exit-save-preemption-timer vmx-flexpriority vmx-hlt-exit vmx-ins-outs vmx-intr-exit vmx-invept vmx-invept-all-context vmx-invept-single-context vmx-invept-single-context vmx-invept-single-context-noglobals vmx-invlpg-exit vmx-invpcid-exit vmx-invvpid vmx-invvpid-all-context vmx-invvpid-single-addr vmx-io-bitmap vmx-io-exit vmx-monitor-exit vmx-movdr-exit vmx-msr-bitmap vmx-mtf vmx-mwait-exit vmx-nmi-exit vmx-page-walk-4 vmx-page-walk-5 vmx-pause-exit vmx-ple vmx-pml vmx-posted-intr vmx-preemption-timer vmx-rdpmc-exit vmx-rdrand-exit vmx-rdseed-exit vmx-rdtsc-exit vmx-rdtscp-exit vmx-secondary-ctls vmx-shadow-vmcs vmx-store-lma vmx-true-ctls vmx-tsc-offset vmx-tsc-scaling vmx-unrestricted-guest vmx-vintr-pending vmx-vmfunc vmx-vmwrite-vmexit-fields vmx-vnmi vmx-vnmi-pending vmx-vpid vmx-wbinvd-exit vmx-xsaves vmx-zero-len-inject vpclmulqdq waitpkg wbnoinvd wdt x2apic xcrypt xcrypt-en xfd xgetbv1 xop xsave xsavec xsaveerptr xsaveopt xsaves xstore xstore-en xtpr
Show the available accelerators:
user $
qemu-system-x86_64 -accel help
Accelerators supported in QEMU binary: tcg kvm hvf whpx haxm
Starting QEMU
This is how to start a virtual machine with the same feature set as the host cpu, a raw disk image and 2G of ram. By default a vnc server is started that runs with no password protection and listens on the loop interface. QEMU is advised to listen on a local UNIX socket with the following command. Set the file permissions appropriately to protect the VNC server from unauthorized access. A cdrom installation and boot medium is added with "-cdrom filename.img"
user $
qemu-system-x86_64 -vnc unix:/home/user/.qemu-vnc-socket -cpu host -drive file=/var/virt/rootfs-build-tc,format=raw -m 2G
Starting the server with -vnc :0 it listens on port 5900 (first display) on all interfaces with no password protection.
Troubleshooting
"kvm: already loaded the other module"
Sometimes during the early boot splash the error message "kvm: already loaded the other module" can be seen. This message indicates both the Intel and the AMD kernel virtual machine settings have been enabled in the kernel. To fix this, enable as a module or disable either the Intel or AMD KVM option specific to the system's processor in the kernel configuration. For example, if the system has an Intel processor enable the Intel KVM, then make sure the AMD KVM is set as a module (M) or is disabled (N). The relevant options to enable or disable can be found in the kernel's .config file via the CONFIG_KVM_INTEL and CONFIG_KVM_AMD variables or in the configuration section above.
Creating TUN/TAP device - No such file or directory
Sometimes this error can occur if TUN/TAP support cannot be found in the kernel. To solve this, try loading the driver:
root #
modprobe tun
If that works, add this to a file in /etc/modules-load.d/ to load on startup:
/etc/modules-load.d/qemu-modules.conf
tun
Configuration does not support video model 'qxl'
This is usually the case if QEMU is not built with the spice
USE flag. To resolve this issue, try to build QEMU with the correct USE flag.
First add spice
to via a package.use file:
/etc/portage/package.use/qemu
app-emulation/qemu spice
Then rebuild the package:
root #
emerge --ask app-emulation/qemu
My qemu has kvm support on some guest architectures
KVM works only for the same architecture. An ARM64 host cannot handle x86_64 instructions.
Invalid context errors on SELinux systems
By default, Libvirt generates a random SELinux MCS label for the QEMU process when it is started. If the loaded SELinux policy does not support MCS categories, the resulting security context will be invalid:
Error starting domain: unable to set socket security context 'system_u:system_r:svirt_t:s0:c123,c456': Invalid argument
kernel: SELinux: Context system_u:object_r:svirt_image_t:s0:c123,c456 is not valid (left unmapped).
The solution is either to switch to one of the policy types which supports MCS categories or manually set the virtual machine's security labels, without MCS categories:
<domain type="kvm">
<name>fedora</name>
...
<devices>
<disk type="file" device="disk">
<driver name="qemu" type="qcow2"/>
<source file="/var/lib/libvirt/images/fedora.qcow2">
<seclabel model='selinux' relabel='yes'>
<label>system_u:object_r:svirt_image_t</label>
</seclabel>
</source>
<target dev="vda" bus="virtio"/>
<address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
</disk>
...
<seclabel type='static' model='selinux' relabel='yes'>
<label>system_u:system_r:svirt_t</label>
</seclabel>
</domain>
Static-user and LTO
GCC will use huge amount of RAM when LTO is enabled on the system while using the static-user
flag, because of this is recommended to disable LTO while compiling in this configuration or use clang if LTO is required. See bug #883419
lto1: internal compiler error: original not compressed with zstd
This is caused by a mismatch of GCC used to compile zlib and glib to the one being used to compile qemu, this can be fixed by rebuilding both before compiling qemu again.
root #
emerge --ask sys-libs/zlib dev-libs/glib
BSOD when booting Windows 10
Create this file:
/etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1
and restart the system.
Removal
There may be image files left behind after removal of QEMU package.
Unmerge
root #
emerge --ask --depclean --verbose app-emulation/qemu
See also
- Comparison of virtual machines — compares the features of several platform virtual machines.
- Fast Virtio VM — explains a way to build a blazing fast Gentoo VM under KVM using Virtio and mdev.
- GPU passthrough with libvirt qemu kvm — directly present an internal PCI GPU to a virtual machine
- QEMU with Open vSwitch network
- Virtualization — the concept and technique that permits running software in an environment separate from a computer operating system.
- QEMU/QEMU front-ends — user interface application to the QEMU/libvirt API/library.
- Libvirt — a virtualization management toolkit.
- Libvirt/QEMU_networking — details the setup of Gentoo networking by Libvirt for use by guest containers and QEMU-based virtual machines.
- Libvirt/QEMU_guest — covers libvirt and its creation of a virtual machine (VM) for use under the soft-emulation mode QEMU hypervisor Type-2, notably using virsh command.
- Virt-manager — desktop user interface for management of virtual machines and containers through the libvirt library
- Virt-manager/QEMU_guest — QEMU creation of a guest (VM or container)
- QEMU/Linux guest — describes the setup of a Gentoo Linux guest in QEMU using Gentoo bootable media.
External resources
- https://www.linux-kvm.org/page/KvmOnGentoo - The Gentoo article on the KVM wiki
- https://wiki.qemu.org/Main_Page - The Official QEMU wiki
References
- ↑ https://en.wikipedia.org/wiki/Full_virtualization
- ↑ https://developer.apple.com/documentation/hypervisor
- ↑ https://en.wikipedia.org/wiki/Paravirtualization
- ↑ https://github.com/RceNinja/notes/blob/master/notes/build_qemu_with_enabled_hyper-v_acceleration_(whpx)_on_windows.md
- ↑ https://en.wikipedia.org/wiki/Paravirtualization
- ↑ QEMU / KVM CPU model configuration
- ↑ https://github.com/intel/haxm
- ↑ https://en.wikipedia.org/wiki/Paravirtualization