GPU passthrough with libvirt qemu kvm

GPU passthrough is a technology that allows you to Article description::directly present an internal PCI GPU to a virtual machine. The device acts as if it were directly driven by the VM, and the VM detects the PCI device as if it were physically connected. GPU passthrough is also often known as IOMMU, although this is a bit of a misnomer, since the IOMMU is the hardware technology that provides this feature but also provides other features such as some protection from DMA attacks or ability to address 64-bit memory spaces with 32-bit addresses.

As you can imagine, the most common application for GPU passthrough at least gaming, since GPU passthrough allows a VM direct access to your graphics card with the end result of being able to play games with nearly the same performance as if you were running your game directly on your computer.

QEMU (Quick EMUlator) is a generic, open source hardware emulator and virtualization suite.

BIOS and UEFI firmware
In order to utilize KVM either Vt-x or AMD-V must be supported by the processor. Vt-x or AMD-V are Intel and AMD's respective technologies for permitting multiple operating systems to concurrently execute operations on the processors.

To inspect hardware for visualization support issue the following command:

For a period manufacturers were shipping with virtualization turned off by default in the system BIOS

Hardware

 * A CPU that supports Intel VT-d or AMD-Vi. Check List of compatible Intel CPUs (Intel VT-x and Intel VT-d)
 * A motherboard that supports the aforementioned technologies. To find this out, check in your motherboard's BIOS configuration for an option to enable IOMMU or something similar. Chances are that your motherboard will support it if it's from 2013 or newer, but make sure to check since this is a niche technology and some manufacturers may save costs by axing it from their motherboards or delivering a defective implementation (such as Gigabyte's 2015-2016 series) simply because NORPs never use it.
 * At least two GPUs: one for your physical OS, another for your VM. (You can in theory run your computer headless through SSH or a serial console, but it might not work and you risk locking yourself away from your computer if you do so).
 * Optional but recommended: Additional monitor, keyboard and mouse

EFI configuration
Go into BIOS (EFI) settings and turn on VT-d and IOMMU support.

IOMMU
IOMMU – or input–output memory management unit – is a memory management unit (MMU) that connects a direct-memory-access–capable (DMA-capable) I/O bus to the main memory. The IOMMU maps a device-visible virtual address ( I/O virtual address or IOVA) to a physical memory address. In other words, it translates the IOVA into a real physical address.

In an ideal world, every device has its own IOVA address space and no two devices share the same IOVA. But in practice this is often not the case. Moreover, the PCI-Express (PCIe) specifications allow PCIe devices to communicate with each other directly, called peer-to-peer transactions, thereby escaping the IOMMU.

That is where PCI Access Control Services (ACS) are called to the rescue. ACS is able to tell whether or not these peer-to-peer transactions are possible between any two or more devices, and can disable them. ACS features are implemented within the CPU and the chipset.

Unfortunately the implementation of ACS varies greatly between different CPU or chip-set models.

Determine IOMMU capabilities
If you already own a PC that you want to use for VGA passthrough, and IOMMU is supported and enabled, you can check the ACS capabilities as follows:

lspci -vv | grep -i 'Access Control Services'

dmesg | grep 'IOMMU enabled'

[   0.000000] DMAR: IOMMU enabled

ACS override patch
ACS patch

* Applying 4400_alpha-sysctl-uac.patch (-p1) ... [ ok ] * Applying 4567_distro-Gentoo-Kconfig.patch (-p1) ... [ ok ] >>> Source unpacked in /var/tmp/portage/sys-kernel/gentoo-sources-4.14.52/work >>> Preparing source in /var/tmp/portage/sys-kernel/gentoo-sources-4.14.52/work/linux-4.14.52-gentoo ... * Applying override_for_missing_acs_capabilities.patch ... [ ok ] * User patches applied.

IOMMU

Software installation

Emerge Grub configuration

Libvirt configuration

Windows guest configuration
 * Workaround for AMD graphic card (turn on/off amd vcard)

Sound
create dir /home/qemu copy /home/USER/.config/pulse to /home/qemu chown qemu:qemu -R /home/qemu

change home dir for qemu: usermod -d /home/qemu qemu

Set home dir:

External resources

 * https://heiko-sieger.info/iommu-groups-what-you-need-to-consider/#What_is_IOMMU_and_why_do_I_need_it
 * https://forum.level1techs.com/t/linux-host-windows-guest-gpu-passthrough-reinitialization-fix/121097?source_topic_id=121737 - AMD GPU on windows guest
 * https://wiki.installgentoo.com/index.php/PCI_passthrough - PCI passthrough on gentoo