User:Egberts/Drafts/QEMU
QEMU (Quick EMUlator) is a generic, open source hardware emulator and virtualization suite.
QEMU is a Type-2 hypervisor that runs within user space and performs virtual hardware emulation.
QEMU has support for acceleration plug-ins. Available QEMU plugins are:
Virtualizer | Accelerator | Virtualization type | Description | Gentoo package name |
---|---|---|---|---|
qemu | tcg | full[1]/software-emulation | QEMU's own Tiny Code Generator. This is the default. More frequently denoted as qemu and not qemu/tcg so often. | app-emulation/qemu |
qemu | hvf[2] | paravirtualization[3] | Apple's Hypervisor.framework based on Intel VT. | |
qemu | whpx[4] | hybrid | Microsoft's Windows Hypervisor Platform based on Intel VT or AMD-V. | |
qemu | kvm | paravirtualization[5] | Linux Type-2 Hypervisor. This is the common choice for host using amd64, arm64, or mips[6]. Supports Microsoft Windows. | app-emulation/qemu |
qemu | haxm[7] | paravirtualization[8] | Intel VT, by Intel Corporation. |
QEMU when used in conjunction with an accelerator becomes a Type-1 hypervisor that runs in kernel space, that allows a user space program access to the hardware virtualization features of various processors. Such accelerator can be KVM (Kernel-based Virtual Machine) or Xen.
If no accelerator is used, QEMU will run entirely in user-space using its built in binary translator TCG (Tiny Code Generator). Using QEMU without an accelerator is relatively inefficient and slow.
This article typically uses KVM as the accelerator of choice due to its GPL licensing and availability. Without KVM nearly all commands described here will still work (unless KVM specific).
Overview
- Firstly, QEMU is a type 2 hypervisor.
- QEMU can be paired with KVM to run VMs at near native speed. This is accomplished by using hardware extensions such as: Intel VT-x or AMD-V.
- It can then emulate for user-level processes that allow applications compiled for one architecture to run on a different one.
- Multiple operating modes: User-mode emu, System emu, KVM hosting, and Xen Hosting,
- QEMU can save and restore the state of VMs with all programs running.
- QEMU VMs can interface with many types of physical host hardware such as CD-ROM Drives, USB Devices, Audio Interfaces, Hard Disks, Network Cards
- Virtual Disk Images can be stored as qcow or qcow2 format which only uses as much disk space as the Guest OS is actually using. Therefore, allowing for a large emulated disk to only use a small portion of the host’s disk space. In turn, it also allows for the ability to revert the emulated disk’s contents to a previous state. (Snapshots)
- It does not depend on graphical output methods on the host system, instead making use of an integrated VNC server to access the screen of the guest OS.
- QEMU is able to execute multiple virtual CPUs in parallel.
Installation
This section details the setup of a paravirtualization (QEMU/KVM) for a host that has been installed with Gentoo OS on either x86_64}, amd64, or mips CPU hardware platform.
Consult the distro-specific installation documentation for the proper setup of a QEMU host, if the host machine is not installed with Gentoo Linux.
BIOS and UEFI firmware
In order to utilize KVM either Vt-x (vmx) or AMD-V (svm) must be supported by the host processor. Vt-x or AMD-V are Intel and AMD's respective technologies for permitting multiple operating systems to concurrently execute operations on the host processors.
To inspect host hardware for virtualization support, issue the following command:
user $
grep --color -E "vmx|svm" /proc/cpuinfo
For a period manufacturers were shipping with virtualization turned off by default in the system's firmware. Note that toggling this feature in the firmware may actually require full removal of power from the system to take effect. If restarting the system does not work try shutting down, unplugging the system and pressing the power button in an unplugged state to discharge any residual energy from the power supply unit (PSU). Reapply power to the system to verify success.
If vmx (x86_64 arch) nor svm (amd64 arch) CPU flag is not found, then QEMU can still be used but only in the much slower software emulator mode. QEMU/KVM cannot be used due to lack of hardware-assist virtualization in the host CPU(s) (as determined by vmx or svm CPU flag.)
If KVM support is available, there should be a kvm device listed under /dev/. This will take effect after the system has booted to a KVM-enabled kernel.
Kernel
Described below are the basic requirements for KVM kernel configuration for the host OS. A more complete and up-to-date list can be found at the KVM Tuning Kernel page.
Kernel options required for QEMU
Different guest (virtualized) OS may require additional kernel options. These are covered in the corresponding #Usage section pages.
General setup --->
Timers subsystem --->
<*> High Resolution Timer Support
This includes support for ARM64 processors.
Physical CPU processor support - Host
If KVM support is not available, insert <val>CONFIG_KVM=y</val> into the /usr/src/linux/.config and rebuild/reinstall the kernel (and its initramfs image). Come back here after the host gets rebooted.
[*] Virtualization --->
<*> Kernel-based Virtual Machine (KVM) support --->
< > KVM for Intel (and compatible) processors support
< > KVM for AMD processors support (NEW)
For the QEMU host, choose one of the two CPU architectures found on its hardware platform:
[*] Virtualization --->
<*> KVM for Intel processors support
or
[*] Virtualization --->
<*> KVM for AMD processors support
If both "KVM for Intel processors support" and "KVM for AMD processors support" are set as built into the kernel (
*
) an error message will appear from kprint from early boot. Since the system has only one type processor (Intel or AMD) enabling one or both options as modules (M
) will make the error message disappear.Since we are building the KVM into the kernel (and not as a module), only choose one CPU architecture: Intel or AMD.
Virtual CPU processor support - Guest(s)
For a list of supporting CPUs that the guest platform(s) on this host should support, consult the [QEMU use flags] and insert desire target(s) into the USE= string in the portage configuration /etc/portage/make.conf file.
[*] Networking support
Networking Options --->
[*] Network packet filtering framework (Netfilter) --->
[*] Advanced netfilter configuration
Core Netfilter Configuration --->
<*> "conntrack" connection tracking match support
<*> CHECKSUM target support
IPv6: Netfilter Configuration --->
<*> ip6tables NAT support
<*> Ethernet Bridge tables (ebtables) support --->
<*> ebt: nat table support
<*> ebt: mark filter support
[*] QoS and/or fair queueing --->
<*> Hierarchical Token Bucket (HTB)
<*> Stochastic Fairness Queueing (SFQ)
<*> Ingress/classifier-action Qdisc
<*> Netfilter mark (FW)
<*> Universal 32bit comparisons w/ hashing (U32)
[*] Actions
<*> Traffic Policing
Handling kernel config at CLI
To set the various kernel configuration settings from the command lines, the linux/scripts/kconfig/merge_config.sh shall be used here:
Mandatory kernel configuration options to set:
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
CONFIG_KVM_INTEL=y
CONFIG_KVM_AMD=y
root #
cd /usr/src/linux
root #
scripts/kconfig/merge_config.sh /usr/src/kernel-kconfig-qemu-host.config
Useful kernel configuration options to use:
CONFIG_VHOST_NET=y
CONFIG_HIGH_RES_TIMER=y
CONFIG_HPET=y
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
CONFIG_KSM=y
CONFIG_SYSFS=y
CONFIG_PROCFS=y
CONFIG_HUGEPAGE=y
CONFIG_CGROUPS=y
root #
scripts/kconfig/merge_config.sh /usr/src/kernel-kconfig-qemu-host-optional.config
Accelerated networking, required for vhost-net
USE flag (recommend):
Device Drivers --->
[*] VHOST drivers --->
<*> Host kernel accelerator for virtio net
[*] Virtualization --->
<*> Host kernel accelerator for virtio net
Device Drivers --->
[*] Network device support --->
[*] Network core driver support
<*> Universal TUN/TAP device driver support
Needed for 802.1d Ethernet bridging:
[*] Networking support --->
Networking options --->
<*> The IPv6 protocol
<*> 802.1d Ethernet Bridging
Intel VT-g (integrated graphics adapter virtualization)
Mediated device passthrough for Intel GPUs (Broadwell and newer) [1].
Device Drivers --->
<*> VFIO Non-Privileged userspace driver framework
<*> Mediated device driver framework
Graphics Support --->
<*> Intel 8xx/9xx/G3x/G4x/HD Graphics
[*] Enable Intel GVT-g graphics virtualization host support
<*> Enable KVM/VFIO support for Intel GVT-g
USE flags
USE flags for app-emulation/qemu QEMU + Kernel-based Virtual Machine userland tools
+aio
|
Enables support for Linux's Async IO |
+curl
|
Support ISOs / -cdrom directives via HTTP or HTTPS. |
+doc
|
Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally |
+fdt
|
Enables firmware device tree support |
+filecaps
|
Use Linux file capabilities to control privilege rather than set*id (this is orthogonal to USE=caps which uses capabilities at runtime e.g. libcap) |
+gnutls
|
Enable TLS support for the VNC console server. For 1.4 and newer this also enables WebSocket support. For 2.0 through 2.3 also enables disk quorum support. |
+jpeg
|
Enable jpeg image support for the VNC console server |
+oss
|
Add support for OSS (Open Sound System) |
+pin-upstream-blobs
|
Pin the versions of BIOS firmware to the version included in the upstream release. This is needed to sanely support migration/suspend/resume/snapshotting/etc... of instances. When the blobs are different, random corruption/bugs/crashes/etc... may be observed. |
+png
|
Enable png image support for the VNC console server |
+seccomp
|
Enable seccomp (secure computing mode) to perform system call filtering at runtime to increase security of programs |
+slirp
|
Enable TCP/IP in hypervisor via net-libs/libslirp |
+vhost-net
|
Enable accelerated networking using vhost-net, see https://www.linux-kvm.org/page/VhostNet |
+vnc
|
Enable VNC (remote desktop viewer) support |
accessibility
|
Adds support for braille displays using brltty |
alsa
|
Enable alsa output for sound emulation |
bpf
|
Enable eBPF support for RSS implementation. |
bzip2
|
Enable bzip2 compression support |
capstone
|
Enable disassembly support with dev-libs/capstone |
debug
|
Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces |
fuse
|
Enables FUSE block device export |
glusterfs
|
Enables GlusterFS cluster fileystem via sys-cluster/glusterfs |
gtk
|
Add support for x11-libs/gtk+ (The GIMP Toolkit) |
infiniband
|
Enable Infiniband RDMA transport support |
io-uring
|
Enable efficient I/O via sys-libs/liburing. |
iscsi
|
Enable direct iSCSI support via net-libs/libiscsi instead of indirectly via the Linux block layer that sys-block/open-iscsi does. |
jack
|
Add support for the JACK Audio Connection Kit |
jemalloc
|
Use dev-libs/jemalloc for memory management |
keyutils
|
Support Linux keyrings via sys-apps/keyutils |
lzo
|
Enable support for lzo compression |
multipath
|
Enable multipath persistent reservation passthrough via sys-fs/multipath-tools. |
ncurses
|
Enable the ncurses-based console |
nfs
|
Enable NFS support |
nls
|
Add Native Language Support (using gettext - GNU locale utilities) |
numa
|
Enable NUMA support |
opengl
|
Add support for OpenGL (3D graphics) |
pam
|
Add support for PAM (Pluggable Authentication Modules) - DANGEROUS to arbitrarily flip |
pipewire
|
Enable pipewire output for sound emulation |
plugins
|
Enable qemu plugin API via shared library loading. |
pulseaudio
|
Enable pulseaudio output for sound emulation |
python
|
Add optional support/bindings for the Python language |
rbd
|
Enable rados block device backend support, see https://docs.ceph.com/en/mimic/rbd/qemu-rbd/ |
sasl
|
Add support for the Simple Authentication and Security Layer |
sdl
|
Enable the SDL-based console |
sdl-image
|
SDL Image support for icons |
selinux
|
!!internal use only!! Security Enhanced Linux support, this must be set by the selinux profile or breakage will occur |
smartcard
|
Enable smartcard support |
snappy
|
Enable support for Snappy compression (as implemented in app-arch/snappy) |
spice
|
Enable Spice protocol support via app-emulation/spice |
ssh
|
Enable SSH based block device support via net-libs/libssh2 |
static
|
Build the User and Software MMU (system) targets as well as tools as static binaries |
static-user
|
Build the User targets as static binaries |
systemtap
|
Enable SystemTap/DTrace tracing |
test
|
Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled independently) |
udev
|
Enable virtual/udev integration (device discovery, power and storage device support, etc) |
usb
|
Enable USB passthrough via dev-libs/libusb |
usbredir
|
Use sys-apps/usbredir to redirect USB devices to another machine over TCP |
vde
|
Enable VDE-based networking |
virgl
|
Enable experimental Virgil 3d (virtual software GPU) |
virtfs
|
Enable VirtFS via virtio-9p-pci / fsdev. See https://wiki.qemu.org/Documentation/9psetup |
vte
|
Enable terminal support (x11-libs/vte) in the GTK+ interface |
xattr
|
Add support for getting and setting POSIX extended attributes, through sys-apps/attr. Requisite for the virtfs backend. |
xdp
|
Enable support for XDP through net-libs/xdp-tools |
xen
|
Enables support for Xen backends |
zstd
|
Enable support for ZSTD compression |
Emerge
root #
emerge --ask app-emulation/qemu
Additional software
User name qemu is required; Defined by sys-user/qemu; evoked by sys-emulator/qemu package.
Group name qemu is required. Defined by sys-group/qemu; evoked by sys-emulator/qemu package.
Configuration
The following sub-articles provide detailed instructions on QEMU configurations and options:
- Usage options - Contains common configuration options used with QEMU (graphics/display, networking, RAM, storage, processor, etc).
- Linux guest - Describes the configuration steps needed to setup a vitrualized Linux guest with QEMU.
- Windows guest - Describes the configuration steps needed to setup a virtualized Windows guest with QEMU.
- OS2WarpV3 guest - Describes the configuration steps needed to setup a virtualized OS2WarpVs=3 guest with QEMU.
Environment variables
- G_MESSAGES_DEBUG
- LISTEN_FDS
- LISTEN_PID
- QEMU_AUDIO_DRV
- QEMU_MODULE_DIR
- XDG_RUNTIME_DIR
Files
Single File
- /etc/libvirt/qemu.conf - QEMU configuration file.
- /etc/libvirt/qemu-lockd.conf - QEMU lock files
- /etc/libvirt/qemu-sanlock.conf - QEMU SAN lock
- /etc/libvirt/qemu/<domain-name>.xml - Domain XML setting for a virtual machine or container.
- /etc/libvirt/qemu/autostart/<domain-name>.xml - Autostart this domain (virtual machine or container).
- /etc/libvirt/qemu/networks/<network-name>.xml - Network XML setting file for a network connection
- /etc/libvirt/qemu/networks/autostart/<network-name>.xml - Autostart this network connection.
- /var/lib/libvirt/qemu/channel/target/<domain-name>/<socket-file> - UNIX socket file for Libvertd daemon API
- /var/cache/libvirt/qemu/capabilities/<hash-value>.xml - Host OS capabilities in XML-format
- /var/lib/libvirt/qemu/checkpoint/
- /var/lib/libvirt/qemu/<domain-9-XXXX>/ - holds UNIX sockets and AES keys for this domain.
- /var/lib/libvirt/qemu/dump/
- /var/lib/libvirt/qemu/nvram/
- /var/lib/libvirt/qemu/ram/
- /var/lib/libvirt/qemu/save/ - holding directory of hibernation images
- /var/lib/libvirt/qemu/snapshot/ - holding directory of snapshots
- /var/run/libvirt/qemu - various UNIX socket and PID files for libvirtd daemon.
Image File
QEMU supports the following disk image formats:
- QEMU copy-on-write (.qcow2, .qed, .qcow, .cow)
- VirtualBox Virtual Disk Image (.vdi)
- CD/DVD (ISO-9660) images (.iso)
- Raw images (.img), that guest OS can control
- VFAT-16
- VMware Virtual Machine Disk (.vmdk)
- Virtual PC Virtual Hard Disk (.vhd)
- Parallels disk image (.hdd, .hds) – Read-only
- Apple macos Universal Disk Image Format (.dmg) – Read-only
- Bochs – Read-only
- Linux cloop – Read-only
Usage
Invocation
Troubleshooting
TBS
Removal
Unmerge
root #
emerge --ask --depclean --verbose app-emulation/qemu
See also
- User:Egberts/Drafts/Virtualization — the concept and technique that permits running software in an environment separate from a computer operating system.
- User:Egberts/Drafts/QEMU — a generic, open source hardware emulator and virtualization suite.
- QEMU/Front-ends — facilitate VM management and use
- Libvirt — a virtualization management toolkit.
- Libvirt/QEMU_networking — details the setup of Gentoo networking by Libvirt for use by guest containers and QEMU-based virtual machines.
- Libvirt/QEMU_guest — covers libvirt and its creation of a virtual machine (VM) for use under the soft-emulation mode QEMU hypervisor Type-2, notably using virsh command.
- Virt-manager — desktop user interface for management of virtual machines and containers through the libvirt library
- Virt-manager/QEMU_guest — QEMU creation of a guest (VM or container)
- QEMU/Linux guest — describes the setup of a Gentoo Linux guest in QEMU using Gentoo bootable media.
Following "See also" should be relocated into their respective GUI/virt-engine page:
- QEMU/Bridge with Wifi Routing
- QEMU/KVM_IPv6_Support - IPv6 support in QEMU/KVM.
- Linux guest - Describes the configuration steps needed to setup a vitrualized Linux guest with QEMU.
- OS2WarpV3 guest - Describes the configuration steps needed to setup a virtualized OS2WarpVs=3 guest with QEMU.
- Windows guest - Describes the configuration steps needed to setup a virtualized Windows guest with QEMU.
- QEMU with Open vSwitch network
External resources
- ↑ https://en.wikipedia.org/wiki/Full_virtualization
- ↑ https://developer.apple.com/documentation/hypervisor
- ↑ https://en.wikipedia.org/wiki/Paravirtualization
- ↑ https://github.com/RceNinja/notes/blob/master/notes/build_qemu_with_enabled_hyper-v_acceleration_(whpx)_on_windows.md
- ↑ https://en.wikipedia.org/wiki/Paravirtualization
- ↑ QEMU / KVM CPU model configuration
- ↑ https://github.com/intel/haxm
- ↑ https://en.wikipedia.org/wiki/Paravirtualization