Trying to passthru my RTX 2080 ti from FreeBSD (14) to a Windows 11 vm (it generates a resource conflict and requires additional installation)


ziomario

Active member
Local time
11:41 PM
Posts
14
OS
windows 11
Hello to everyone.

What I would like to understand if why,when I try to passthru my RTX 2080 ti from FreeBSD to Windows 11,it won't do it,causing the error 12. It says that it generates a resource conflict and requires additional installation).

I'm trying to debug the error. Below you see the IOMMU group of my Nvidia Geforce RTX 2080 ti passed in a Windows 11 / bhyve VM and of Windows 11 installed physically. I'm not able to understand if there are inconsistencies.


VM :

Code:
Group: 30.0.15.1123 - Device: NVIDIA GeForce RTX 2080 Ti (Device ID: PCI\VEN_10DE&DEV_1E04&SUBSYS_250319DA&REV_A1\3&61AAA01&0&48)

Group: 4.39.0.0 - Device: NVIDIA Virtual Audio Device (Wave Extensible) (WDM) (Device ID: ROOT\UNNAMED_DEVICE\0000)

Group: 1.46.831.832 - Device: NVIDIA USB Type-C Port Policy Controller (Device ID: PCI\VEN_10DE&DEV_1AD7&SUBSYS_250319DA&REV_A1\3&61AAA01&0&4B)

Group: 10.0.22000.2360 - Device: Controller host NVIDIA USB 3.10 eXtensible - 1.10 (Microsoft) (Device ID: PCI\VEN_10DE&DEV_1AD6&SUBSYS_250319DA&REV_A1\3&61AAA01&0&4A)


Physical installation :


Code:
Group: 31.0.15.5222 - Device: NVIDIA GeForce RTX 2080 Ti (Device ID: PCI\VEN_10DE&DEV_1E04&SUBSYS_250319DA&REV_A1\4&94764DC&0&0009)

Group: 4.49.0.0 - Device: NVIDIA Virtual Audio Device (Wave Extensible) (WDM) (Device ID: ROOT\UNNAMED_DEVICE\0000)

Group: 1.50.831.832 - Device: NVIDIA USB Type-C Port Policy Controller (Device ID: PCI\VEN_10DE&DEV_1AD7&SUBSYS_250319DA&REV_A1\4&9476
4DC&0&0309)

Group: 10.0.22000.2360 - Device: NVIDIA USB 3.10 eXtensible Host Controller - 1.10 (Microsoft) (Device ID: PCI\VEN_10DE&DEV_1AD6&SUBSY
S_250319DA&REV_A1\4&94764DC&0&0209)

As an experiment I've passed only this device from FreeBSD to WIndows 11 :

Code:
pptdevs="2/0/0"

I have excluded "2/0/1 ; 2/0/2 ; 2/0/3". I want to show you what means these addresses :

Code:
02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)

What happened when I launched the vm using these parameters ?

Code:
bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-hd,/mnt/$vmdisk0'p2'/bhyve/img/Windows/Windows11.img,bootindex=1 \
-s 9:0,passthru,2/0/0 \
-s 11,hda,play=/dev/dsp,rec=/dev/dsp \
-s 13,virtio-net,tap18 \
-s 29,fbuf,tcp=0.0.0.0:5918,w=1600,h=950,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:18 < /dev/null & sleep 2 && vncviewer 0:18

this error :

Assertion failed: (!err), function hda_init, file /usr/corvin-src-releng-140/usr.sbin/bhyve/pci_hda.c, line 353.

The error is caused by this parameter :

Code:
-s 11,hda,play=/dev/dsp,rec=/dev/dsp \

I have excluded it and Windows 11 booted. Now,inside it I still see the error 12 :

The device PCI\VEN_10DE&DEV_1E04&SUBSYS_250319DA&REV_A1\3&61aaa01&0&48 generates a resource conflict and requires additional installation.

If I pass to the vm only -s 9:0,passthru,2/0/0 \ ; it does not work anyway,because AFAIK,MAYBE there is one and only IOMMU group,so I should pass everything ? But if I pass everything,the error 12 is there anyway.

I would like to understand if there is a problem with the IOMMU thing,if there is maybe there is the needing to develop a patch like this one : GitHub - benbaker76/linux-acs-override for FreeBSD.

For an experienced developer should not be so hard to understand which devices conflict. I want to ask some help to understand how to debug the error. If we will be able to understand which devices conflict,maybe I can explain this to the bhyve developer with which I'm in contact and he can develop a patch. So,we will be able to use every modern nvidia gpus inside a Windows VM with bhyve under FreeBSD ! That's a nice goal,isn't it ? So,please ask me more informations that you need to understand the origin of the error.
 
Windows Build/Version
10.0.22000.2538
Last edited:

My Computer

System One

  • OS
    windows 11
    Computer type
    PC/Desktop
    CPU
    I9 9900k
    Motherboard
    Gigabyte Aorus Pro
    Memory
    32
    Graphics Card(s)
    RTX 2080 ti
If you run msinfo32 and then check the resource sharing and conflicts tab then it should show which devices are conflicting with each other.
 

My Computer

System One

  • OS
    Windows 11, Windows 10, Linux Fedora Cinnamon
You can't pass through the GPU if it's used by the host. Or you can but as the setup told you, there's a conflict and requires additional software. Support must exist for both host and client. Another way is to dedicated the GPU entirely to the VM, but then you need something that can drive the host screen. Intel iGPU could be used for that...
 

My Computers

System One System Two

  • OS
    Windows 11 Pro 23H2 build 10.0.22631.3296 (Release Channel) / Linux Mint 21.3 Cinnamon
    Computer type
    Laptop
    Manufacturer/Model
    Lenovo A485
    CPU
    Ryzen 7 2700U Pro
    Motherboard
    Lenovo (WiFi/BT module upgraded to Intel Wireless-AC-9260)
    Memory
    32GB
    Graphics Card(s)
    iGPU Vega 10
    Sound Card
    Realtek
    Monitor(s) Displays
    14" FHD (built-in) + 14" Lenovo Thinkvision M14t (touch+pen) + 32" Asus PB328
    Screen Resolution
    FHD + FHD + 1440p
    Hard Drives
    Intel 660p m.2 nVME PCIe3.0 x2 512GB
    PSU
    65W
    Keyboard
    Thinkpad / Logitech MX Keys
    Mouse
    Logitech MX Master 2S
    Internet Speed
    600/300Mbit
    Browser
    Edge (Chromium)
    Antivirus
    Windows Defender
    Other Info
    SecureBoot: Enabled
    TPM2.0: Enabled
    AMD-V: Enabled
  • Operating System
    Windows 11 Pro 23H2 build 10.0.22631.3296(Release Preview Channel)
    Computer type
    PC/Desktop
    Manufacturer/Model
    Custom
    CPU
    i7-7700k @4.8GHz
    Motherboard
    Asus PRIME Z270-A
    Memory
    32GB 2x16GB 2133MHz CL15
    Graphics card(s)
    EVGA GTX1080Ti FTW 11GB
    Sound Card
    Integrated
    Monitor(s) Displays
    32" 10-bit Asus PB328Q
    Screen Resolution
    WQHD 2560x1440
    Hard Drives
    512GB ADATA SX8000NP NVMe PCIe Gen 3 x4
    PSU
    850W
    Case
    Fractal Design Define 7
    Cooling
    Noctua NH-D15 chromax.black
    Mouse
    Logitech MX Master 2S
    Keyboard
    Logitech MX Keys
    Internet Speed
    600/300Mbit
    Browser
    Edge (Cromium)
    Antivirus
    Windows Defender
    Other Info
    AC WiFi Card
  • Like
Reactions: OAT
You can't pass through the GPU if it's used by the host. Or you can but as the setup told you, there's a conflict and requires additional software. Support must exist for both host and client. Another way is to dedicated the GPU entirely to the VM, but then you need something that can drive the host screen. Intel iGPU could be used for that...
Excellent !!! @SlicEnDicE

You can only pass thru devices from a Linux Host ( Virtualisation using KVM/QEMU) if they are not being used by the host or other VM's currently powered on. You if passing through a GPU need a SEPARATE video card (or port) on it. You might / might not need IOMMU settings any more -- that depends on the "Bios" in the Mobo / video card.

If using CITRIX / ZEN for virtualisation --then I've no idea but IMO zen / citrix is even worse than a poor starving dog.

Cheers
jimbo
 

My Computer

System One

  • OS
    Windows XP,7,10,11 Linux Arch Linux
    Computer type
    PC/Desktop
    CPU
    2 X Intel i7
Excellent !!! @SlicEnDicE

You can only pass thru devices from a Linux Host ( Virtualisation using KVM/QEMU) if they are not being used by the host or other VM's currently powered on. You if passing through a GPU need a SEPARATE video card (or port) on it. You might / might not need IOMMU settings any more -- that depends on the "Bios" in the Mobo / video card.

If using CITRIX / ZEN for virtualisation --then I've no idea but IMO zen / citrix is even worse than a poor starving dog.

Cheers
jimbo

Don't worry,I have more than one GPU. I have 3 gpus. The ones you see below :

root@marietto-133:/usr/ports/www/chromium # lspci

00:00.0 Host bridge: Intel Corporation 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S] (rev 0d)

00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 0d)

00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) (rev 0d)

00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02)

00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)

00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev 10)
00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)
00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)
00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-V (rev 10)

01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)

02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)


03:00.0 Non-Volatile memory controller: Micron/Crucial Technology P1 NVMe PCIe SSD[Frampton2] (rev 03)

What I'm doing is to choose (from the BIOS) this gpu and I assign it to the host :

00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02)

at the same time,I reserve the RTX 2080 ti to a guest os (Linux or Windows),by declaring this parameter inside the file /boot/loader.conf :

pptdevs="2/0/0 2/0/1 2/0/2 2/0/3"

Take in consideration that it works inside the Linux vm,but NOT in the Windows vm,because there is some kind of conflitcs between resources,that I need to understand.
 

My Computer

System One

  • OS
    windows 11
    Computer type
    PC/Desktop
    CPU
    I9 9900k
    Motherboard
    Gigabyte Aorus Pro
    Memory
    32
    Graphics Card(s)
    RTX 2080 ti
This is what I read on the conflicts / shares after writing : msinfo32

I would like to understand what are the conflicts that I'm looking for. Maybe they are between the IRQ3 and IRQ4,the Geforce RTX 2080 ti and the Microsoft default video adapter ? So there are multiple conflicts ?

Istantanea_2024-05-09_17-39-55.png
 

My Computer

System One

  • OS
    windows 11
    Computer type
    PC/Desktop
    CPU
    I9 9900k
    Motherboard
    Gigabyte Aorus Pro
    Memory
    32
    Graphics Card(s)
    RTX 2080 ti
Maybe even the picture below can be useful to understand what's wrong :

Istantanea_2024-05-09_17-56-58.png
 

My Computer

System One

  • OS
    windows 11
    Computer type
    PC/Desktop
    CPU
    I9 9900k
    Motherboard
    Gigabyte Aorus Pro
    Memory
    32
    Graphics Card(s)
    RTX 2080 ti
I'm not a developer,but I suspect that a patch is needed for bhyve,because Windows needs ACPI. I have disabled it between the bhyve parameters and this what happens.

Istantanea_2024-05-09_18-06-03.png

The other option could be to disable ACPI between Windows ?
 

My Computer

System One

  • OS
    windows 11
    Computer type
    PC/Desktop
    CPU
    I9 9900k
    Motherboard
    Gigabyte Aorus Pro
    Memory
    32
    Graphics Card(s)
    RTX 2080 ti

Latest Support Threads

Back
Top Bottom