LXD infiniband devices

A somewhat uncommon device type as few have Infiniband hardware and even fewer are using that hardware in actual Infiniband mode.
But here is how to configure Infiniband devices and pass them to containers and virtual machines, including SR-IOV support!
RESOURCES:
Infiniband devices: documentation....
Github: github.com/can...
Website: ubuntu.com/lxd
Community forum: discourse.ubun...

[@ewenchan1239]

Great video!
Two questions:
1) In your video, you mentioned that the virtual functions were "tied to the host driver", but then the SR-IOV weren't available to containers.
However, in the demo, you showed that you were able to attached them as either type=physical (which is the virtual function) or as type=sriov, so I'm a little bit confused by your statement in regards to what you said about them not being able to be used by the container.
Can you maybe clarify this a little bit further?
2) You also mentioned in the video that you were using Ubuntu 22.04 LTS and that you had to install or enable the Infiniband kernel driver because it isn't installed/enabled (in the linux kernel) by default.
Would you happen to drop the commands that you used to install said IB kernel driver?
Your help is greatly appreciated.
Thank you.

[@ewenchan1239]

i just tested this and it doesn't really work.
When I tried to add the Infiniband SR-IOV to a VM, this is the error message that I get when I try to start up the VM:
"Failed setting up device via monitor: Failed setting up device "ib0": Failed adding NIC device: Monitor is disconnected"
When I passed the SR-IOV VF as a physical Infiniband NIC, it doesn't recognise the VF, and the graphical console is stuck on the EFI/UEFI boot loading page (but no UEFI prompt though).