Trying to passthru my RTX 2080 ti from FreeBSD (14) to a Windows 11 vm (it generates a resource conflict and requires additional installation)

Hello to everyone.

What I would like to understand if why,when I try to passthru my RTX 2080 ti from FreeBSD to Windows 11,it won't do it,causing the error 12. It says that it generates a resource conflict and requires additional installation).

I'm trying to debug the error. Below you see the IOMMU group of my Nvidia Geforce RTX 2080 ti passed in a Windows 11 / bhyve VM and of Windows 11 installed physically. I'm not able to understand if there are inconsistencies.


VM :



Physical installation :



As an experiment I've passed only this device from FreeBSD to WIndows 11 :


I have excluded "2/0/1 ; 2/0/2 ; 2/0/3". I want to show you what means these addresses :


What happened when I launched the vm using these parameters ?


this error :

Assertion failed: (!err), function hda_init, file /usr/corvin-src-releng-140/usr.sbin/bhyve/pci_hda.c, line 353.

Click to expand...

The error is caused by this parameter :


I have excluded it and Windows 11 booted. Now,inside it I still see the error 12 :

The device PCI\VEN_10DE&DEV_1E04&SUBSYS_250319DA&REV_A1\3&61aaa01&0&48 generates a resource conflict and requires additional installation.

If I pass to the vm only -s 9:0,passthru,2/0/0 \ ; it does not work anyway,because AFAIK,MAYBE there is one and only IOMMU group,so I should pass everything ? But if I pass everything,the error 12 is there anyway.

I would like to understand if there is a problem with the IOMMU thing,if there is maybe there is the needing to develop a patch like this one : GitHub - benbaker76/linux-acs-override for FreeBSD.

For an experienced developer should not be so hard to understand which devices conflict. I want to ask some help to understand how to debug the error. If we will be able to understand which devices conflict,maybe I can explain this to the bhyve developer with which I'm in contact and he can develop a patch. So,we will be able to use every modern nvidia gpus inside a Windows VM with bhyve under FreeBSD ! That's a nice goal,isn't it ? So,please ask me more informations that you need to understand the origin of the error.

If you run msinfo32 and then check the resource sharing and conflicts tab then it should show which devices are conflicting with each other.

You can't pass through the GPU if it's used by the host. Or you can but as the setup told you, there's a conflict and requires additional software. Support must exist for both host and client. Another way is to dedicated the GPU entirely to the VM, but then you need something that can drive the host screen. Intel iGPU could be used for that...

SlicEnDicE said:

You can't pass through the GPU if it's used by the host. Or you can but as the setup told you, there's a conflict and requires additional software. Support must exist for both host and client. Another way is to dedicated the GPU entirely to the VM, but then you need something that can drive the host screen. Intel iGPU could be used for that...

Click to expand...

Excellent !!! @SlicEnDicE

You can only pass thru devices from a Linux Host ( Virtualisation using KVM/QEMU) if they are not being used by the host or other VM's currently powered on. You if passing through a GPU need a SEPARATE video card (or port) on it. You might / might not need IOMMU settings any more -- that depends on the "Bios" in the Mobo / video card.

If using CITRIX / ZEN for virtualisation --then I've no idea but IMO zen / citrix is even worse than a poor starving dog.

Cheers
jimbo

jimbo45 said:

Excellent !!! @SlicEnDicE

You can only pass thru devices from a Linux Host ( Virtualisation using KVM/QEMU) if they are not being used by the host or other VM's currently powered on. You if passing through a GPU need a SEPARATE video card (or port) on it. You might / might not need IOMMU settings any more -- that depends on the "Bios" in the Mobo / video card.

If using CITRIX / ZEN for virtualisation --then I've no idea but IMO zen / citrix is even worse than a poor starving dog.

Cheers
jimbo

Click to expand...

Don't worry,I have more than one GPU. I have 3 gpus. The ones you see below :

root@marietto-133:/usr/ports/www/chromium # lspci

00:00.0 Host bridge: Intel Corporation 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S] (rev 0d)

00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 0d)

00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) (rev 0d)

00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02)

00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)

00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev 10)
00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)
00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)
00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-V (rev 10)

01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)

02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)

03:00.0 Non-Volatile memory controller: Micron/Crucial Technology P1 NVMe PCIe SSD[Frampton2] (rev 03)

What I'm doing is to choose (from the BIOS) this gpu and I assign it to the host :

00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02)

at the same time,I reserve the RTX 2080 ti to a guest os (Linux or Windows),by declaring this parameter inside the file /boot/loader.conf :

pptdevs="2/0/0 2/0/1 2/0/2 2/0/3"

Take in consideration that it works inside the Linux vm,but NOT in the Windows vm,because there is some kind of conflitcs between resources,that I need to understand.

This is what I read on the conflicts / shares after writing : msinfo32

I would like to understand what are the conflicts that I'm looking for. Maybe they are between the IRQ3 and IRQ4,the Geforce RTX 2080 ti and the Microsoft default video adapter ? So there are multiple conflicts ?

Maybe even the picture below can be useful to understand what's wrong :

I'm not a developer,but I suspect that a patch is needed for bhyve,because Windows needs ACPI. I have disabled it between the bhyve parameters and this what happens.



The other option could be to disable ACPI between Windows ?