User:R7l/Nvidia GPU passthrough with QEMU on Lenovo ThinkPad P53

GPU passthrough is a technology that allows the Linux kernel to directly present an internal PCI GPU to a virtual machine. While both names are not exactly fitting it is mostly known as IOMMU or VFIO and the best way to get a virtual machine running with GPU passthrough is QEMU. It is allows you to run a selected number of systems with nearl native performance. In most cases this will be used for gaming or anythin that requires GPU performance.

Passthrough of dedicated Nvidia GPUs to the virtual machines might turn out to be a pain when using laptops. In case of the Thinkpad P53 and other high end Thinkpads it does work well due to the muxed GPU hardware and the decent IOMMU layout. The external HDMI port is directly wired to the Nvidia GPU and will not be touched by the internal Intel GPU.

This guide will cover a basic setup for Thinkpad P53 series running Gentoo as host on the laptop display and Windows 10 as guest on a monitor connected to the HDMI port. For this reason it will not include instructions for dynamic switching of the Nvidia GPU between host and guest. It is known to work but there are several ways to accomplish this with most of them having issues here and there.

Obstacles
There are a number of obstacles in order to successfully pass the GPU into a Windows guest system and make it work with the Nvidia drivers. Most of them are driver related decisions by Nvidia. While we only expect the powerful Nvidia GPU to work inside the VM, it needs both GPUs to be passed into the VM. But due to Intel GVT-g technology, we will pass a virtualized GPU to it. This GPU will not do anything other then allow the driver to be installed.

Beside that, the vBIOS needs to be extraced and patched into OVMF BIOS and a fake battery needs to exist inside the VM.

Hardware requirements
To proceed with the installation you will need a dedicated monitor connected to the HDMI port and also a mouse & keyboard connected to USB. Usage of docking stations hasn't been tested.

BIOS
It is recommended to update to the latest BIOS version, check the official page.

The following settings are needed.

Grub
The following parameters need to be added to the Grub command line:

QEMU
We need to set the Qemu targets accordingly to the system:

The Qemu package comes with a larger list of possible USE flags. You may want to check those USE flags and set them according to your planned setup. This is a recommended set of flags:

After reviewing and adding any desired USE flags, emerge :

libvirt
As well as Qemu, libvirt comes with a number of Use flags. Please check those flags and set them according to your setup. These are recommended Use flags for libvirt:

After reviewing and adding any desired USE flags, emerge :

User permissions
After emerging, to run virt-manager as a normal user, ensure each user has been added to the group:

Starting the service
The service needs to be started. It's also a good idea to enabled in order to be around once we restart the system. This can be done with:

With OpenRC:

With Systemd:

Virt-Manager
Virt-Manager allows you to manage virtual machines using a destop UI.

Install :

Check the IOMMU layout
Passing through any PCI device can only be done with all devices within a group except the PCI root device. The group layout on the P53 is well organized and suitable for this task. You can check the layout like this:

The Nvidia GPU with its audio device should look similar on your system. In order to pass through the Nvidia GPU you'll also need to pass through the audio device. In theory, other devices could be passed through as well.

Blacklist Nvidia Modules
To prevent the host from holding on to the Nvidia GPU we need to blacklist the drivers.

Selecting Nvidia GPU / Audio
You need to pass the device IDs of the PCI devices for Nvidia GPU and Audio. Check the PCI list with  and copy the IDs into this file accordingly:

Adding device IDs like this, will block the from the host system.

Grub
The following parameters need to be added to the Grub command line in order to enable VFIO and IOMMU: