T20A02 at 03:12, 31 August 2024

2024-08-31T03:12:59Z

← Older revision		Revision as of 03:12, 31 August 2024
Line 6:		Line 6:
	Note: On newer systems, your install may be using "proxmox-boot-tool". If your is then the following must be appended to /etc/kernel/cmdline		Note: On newer systems, your install may be using "proxmox-boot-tool". If your is then the following must be appended to /etc/kernel/cmdline
	<pre>		<pre>
	intel_iommu=on or for AMD amd_iommu=on		intel_iommu=on
			</pre>
			or for AMD
			<pre>
			amd_iommu=on
	</pre>		</pre>
	Then run update-grub if using grub or proxmox-boot-tool refresh to push this file change to the grub boot loader		Then run update-grub if using grub or proxmox-boot-tool refresh to push this file change to the grub boot loader

T20A02: Created page with "== PCIe Pass-Through == This is a quick outline of the configuration changes that must be made to allow PCIe pass through for VM's for proxmox out-of-box. Enable IOMMU features by editing the default grub file, by adding the following to the "GRUB_CMDLINE_LINUX_DEFAULT" options in /etc/default/grub Note: On newer systems, your install may be using "proxmox-boot-tool". If your is then the following must be appended to /etc/kernel/cmdline

 intel_iommu=on or for AMD..."

2024-08-31T03:12:28Z

Created page with "== PCIe Pass-Through == This is a quick outline of the configuration changes that must be made to allow PCIe pass through for VM's for proxmox out-of-box. Enable IOMMU features by editing the default grub file, by adding the following to the "GRUB_CMDLINE_LINUX_DEFAULT" options in /etc/default/grub Note: On newer systems, your install may be using "proxmox-boot-tool". If your is then the following must be appended to /etc/kernel/cmdline <pre> intel_iommu=on or for AMD..."

New page

== PCIe Pass-Through ==

This is a quick outline of the configuration changes that must be made to allow PCIe pass through for VM's for proxmox out-of-box.

Enable IOMMU features by editing the default grub file, by adding the following to the "GRUB_CMDLINE_LINUX_DEFAULT" options in /etc/default/grub
Note: On newer systems, your install may be using "proxmox-boot-tool". If your is then the following must be appended to /etc/kernel/cmdline
<pre>
intel_iommu=on or for AMD amd_iommu=on
</pre>
Then run update-grub if using grub or proxmox-boot-tool refresh to push this file change to the grub boot loader

Note: The config file on this install, of PVE 8.0 had a ' at the beginning that was causing the update-grub command to fail.

Now reboot the system and check that the system supports IOMMU physically, by checking for different IOMMU groups in the output. It should look something like the following:
<pre>
pvesh get /nodes/proxymoxy2/hardware/pci --pci-class-blacklist ""
</pre>

Note the values are different between devices, if there are some with the same it means they are on the same device and share a "pcie hub"

[[File:269135969-33454b73-5d47-45e7-a7bf-5096dc2cf422.png|800px]]

If you see -1 as the IOMMU value on all the devices it means IOMMU is not initialized.

If you have some devices sharing IOMMU groups, that should not try adding pcie_acs_override=downstream to the grub file aswell. This will breakup the IOMMU group's even more on some hardware.

If this is for GPU devices, make sure to blacklist the drivers for the device. Note this may cause issues if your integrated graphics use the same driver:
<pre>
AMD:

echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
Nvidia:

echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia*" >> /etc/modprobe.d/blacklist.conf
Intel:

echo "blacklist i915" >> /etc/modprobe.d/blacklist.conf
</pre>

Now that these drivers are blacklisted on the host, reboot one more time, which universalizes the drivers.
Now once back up, connect to the web GUI of the server, and create a new VM. From there go to hardware settings > top left to Add > PCI device > and select a device from the drop down list:

Here we can see the 1080ti installed in this IBM server

[[File:269137439-62d49d8b-f2f5-458c-af74-115a46bd8b7a.png|800px]]

Now we have access to the PCI devices in use on the system. In this configuration with IOMMU, the device is elusive to the system. There are ways to bypass IOMMU and attempt the device anyways, but if the host system tries to access the device, it will cause a hard system crash, and is highly unstable.

Proxmox VM Passthrough - Revision history

T20A02 at 03:12, 31 August 2024