I realize that I am stumbling across this tutorial a little over a year after it was published. Better late then never is what I have heard. Merci / thank you so much for sharing your time and expertise with us. By the way, we are practically neighbors. I'm from Montréal as well.
Some of you may notice that this is a re-upload. For some reason TH-cam never processed the HD version of this video so we ended up re-uploading it as a new video which then processed just fine... Sorry for the duplicate notification!
For the record, some of the commands changed in LXD 5.0.1 / Ubuntu 22.04: To use distrobuilder I needed to sudo apt install libwin-hivex-perl wimtools # to create vm in zfs storage pool lxc init win11 --vm --empty -s my-storage-pool # if parent folder does not exist error occurs: zfs create -o mountpoint=legacy my-storage-pool/virtual-machines # set resource limits lxc config set win11 limits.cpu=4 limits.memory=8GiB # instead of lxc config device override win11 root size=80GiB # use lxc config device set win11 root size=80GiB # instead of lxc config device add win11 vtpm tpm path=/dev/tpm0 # use lxc config device add win11 vtpm tpm # add image to virtual drive lxc config device add win11 install disk source=/path/to/win11.lxd.iso boot.priority=10 # to remove image from virtual drive lxc config device remove win11 install
this is a great tutorial and it has me most of the way there! I'm sure something has changed within windows since this video, though. I created a default bridge network when I initialized lxd, but the windows VM can't find a network and won't let me past the setup stage without it.
Thank you for the tutorial 🙏 I have an issue, when installing Win11 the VM can't find network connection and installation process can't continue. "Lxc list" shows that the VM is using an IPv6 address, I think this is the problem... how can I fix it? 🤔
Can I run and use any games and any engineering software on it near native windows? Actually I'm curious about can LXD use 3d capable of gaming RTX GPUs?
You could if you were able to pass in a physical GPU. LXD will let you do it, but you can't share a GPU between the host and guest, so you need a separate dedicated GPU just for the VM and a host system that allows VFIO with IOMMU support.
Depends on the machine and how large a Windows VM you need. For a minimal VM with just 4GB of RAM and a couple of CPU cores, you could likely pack 4 of those on a normal laptop. But then I have servers here with 64 cores, 128 threads and over 1TiB of RAM, those beasts could easily run over 200 such VMs.
ChromeOS ships with LXD 3.14 which predates VM support. Google is getting ready to move to LXD 4.0.7 which would support VMs (minus the TPM) but this will ultimately depend on whether they'll include QEMU in their installation, early sign suggests that they are not and that LXD will therefore be limited to containers only for now. Also worth noting that because ChromeOS uses a VM layer underneath LXD, this would effectively be nested virtualization. This works quite well in our experience on X86 systems but ARM64 does not support nested virtualization, so even if Google does include QEMU in their build, it will only work on X86 ChromeBooks.
Yes, that's normal and will happen until we either get the LXD agent working on Windows (waiting for vsock and virtiofs drivers to work reliably) or we implement support for fetching as much as we can from the host side (which we have plans to do over the next 6 months or so).
dropping some commands that resolved some issues for me sudo snap set lxd ui.enable=true sudo systemctl reload snap.lxd.daemon lxc config set win11 raw.qemu="-cpu host"
Does anybody knows if it’s possible to run this LXD Win11 installed on an Ubuntu machine remotely from a MacOS machine? Please, any guidance is welcome.
Salut Stefan. Is it possible to install FreeBSD, or GhostBSD etc. on a lxc VM? I had no luck so far getting it to finish the installation. It's crashing and throw me out of the shell. Would be great if you could provide a full HowTo sometime. Thanks very much for your great videos.
LXD VMs support GPU passthrough with the `gpu` device type. However this will almost certainly not do what you want as you need a physically dedicated GPU per VM. That's because GPUs cannot be shared between VMs or with the host system. So passing in your GPU will have it get unplugged from the host system which in most cases will lead to a crash of the operating system on the host. If you do have a dedicated GPU that you want to use for a VM, then you need to: - Make very sure that no drivers are attached to it on the host system (you may need modprobe.d blacklist entries or force the vfio-pci driver to take over that GPU) - Enable IOMMU on your system (VT-d or similar in BIOS and iommu=pt or similar kernel boot option) - Confirm that the GPU is alone in its IOMMU group (lspci can help with that) - Finally, pass it to your VM with `lxc config device add VM-NAME DEVICE-NAME gpu gputype=physical pci=PCI-ADDRESS`
Currently this is limited to Windows where we support Windows 10/11 as well as most versions of Windows server (2019, 2016, 2012). MacOS would be fun to get working though I don't think it'd be quite as trivial as injecting a few drivers in an install media. Last I checked, there weren't Darwin drivers for most of the devices that we rely on in LXD VMs, so that'd be the first blocker. Other solutions get around this issue by using non-virtio QEMU devices instead, effectively emulating traditional hardware which macOS knows how to handle. This isn't something we expose with LXD VMs as we use a very minimal build of QEMU that primarily relies on virtio devices (limiting attack surface). So we'll have to see. If someone gets a full set of virtio drivers for Darwin and we can inject those into a macOS install media, then this is something we could add to distrobuilder, but that seems quite unlikely to me. Not to mention the current move to ARM isn't going to make this any easier. And lastly, my understanding is that you are only (legally) allowed to virtualize macOS when running on Apple hardware, so this would only really work if you were to be running Linux on a Mac and then would want to run macOS on it inside a VM. A pretty limited usecase :)
It could be a version thing or be related to exactly how they built EDK2. github.com/lxc/lxd-pkg-snap/blob/latest-edge/snapcraft.yaml has the details on how we build qemu, edk2, swtpm and libtpm which are the main components involved there. I don't think we're particularly picky about the version of any of those, except that there are some known issues with qemu 7.2 so we've been sticking to 7.1. If you need firmware measurements through the TPM, it's important that edk2 is built with TPM2 support, maybe that's missing on Arch.
This was running on my desktop machine, an AMD Ryzen 7 5700G. LXD performs CPU passthrough to the guest so as long as the host is compatible with Windows 11, so will the guest.
So many typos doesn't raise confidence. English needs a little polishing and a note: source path doesn't work with symlink which is just poor programming.
Wow, I didn't even know LXD was capable of running virtual machines. This is super cool to see.
I realize that I am stumbling across this tutorial a little over a year after it was published. Better late then never is what I have heard. Merci / thank you so much for sharing your time and expertise with us. By the way, we are practically neighbors. I'm from Montréal as well.
Some of you may notice that this is a re-upload. For some reason TH-cam never processed the HD version of this video so we ended up re-uploading it as a new video which then processed just fine... Sorry for the duplicate notification!
For the record, some of the commands changed in LXD 5.0.1 / Ubuntu 22.04:
To use distrobuilder I needed to sudo apt install libwin-hivex-perl wimtools
# to create vm in zfs storage pool
lxc init win11 --vm --empty -s my-storage-pool
# if parent folder does not exist error occurs:
zfs create -o mountpoint=legacy my-storage-pool/virtual-machines
# set resource limits
lxc config set win11 limits.cpu=4 limits.memory=8GiB
# instead of
lxc config device override win11 root size=80GiB
# use
lxc config device set win11 root size=80GiB
# instead of
lxc config device add win11 vtpm tpm path=/dev/tpm0
# use
lxc config device add win11 vtpm tpm
# add image to virtual drive
lxc config device add win11 install disk source=/path/to/win11.lxd.iso boot.priority=10
# to remove image from virtual drive
lxc config device remove win11 install
Thanks, was looking for this since a time , great Job…keep going
LXD is amazing.
this is a great tutorial and it has me most of the way there! I'm sure something has changed within windows since this video, though. I created a default bridge network when I initialized lxd, but the windows VM can't find a network and won't let me past the setup stage without it.
Very helpful thank you.
very good video
Thank you for the tutorial 🙏
I have an issue, when installing Win11 the VM can't find network connection and installation process can't continue.
"Lxc list" shows that the VM is using an IPv6 address, I think this is the problem... how can I fix it? 🤔
Did you manage to fix this? I have no network connection either, so cannot proceed with the install
@@magnusmaynard Did you find a solution for this issue. I also cannot continue windows installation without network connection.
Thanks for the explanation sir, I have a problem, my Ipv4 didn't appear, how can I fix it?
Can I run and use any games and any engineering software on it near native windows?
Actually I'm curious about can LXD use 3d capable of gaming RTX GPUs?
You could if you were able to pass in a physical GPU.
LXD will let you do it, but you can't share a GPU between the host and guest, so you need a separate dedicated GPU just for the VM and a host system that allows VFIO with IOMMU support.
Valuable information
How many windows 11 on LXD ?
I mean that # of multiple windows VMs on single machine.
Depends on the machine and how large a Windows VM you need. For a minimal VM with just 4GB of RAM and a couple of CPU cores, you could likely pack 4 of those on a normal laptop.
But then I have servers here with 64 cores, 128 threads and over 1TiB of RAM, those beasts could easily run over 200 such VMs.
Great video. Worked for me on ubuntu. Has anyone tried this on a ChromeOS machine?
ChromeOS ships with LXD 3.14 which predates VM support.
Google is getting ready to move to LXD 4.0.7 which would support VMs (minus the TPM) but this will ultimately depend on whether they'll include QEMU in their installation, early sign suggests that they are not and that LXD will therefore be limited to containers only for now.
Also worth noting that because ChromeOS uses a VM layer underneath LXD, this would effectively be nested virtualization. This works quite well in our experience on X86 systems but ARM64 does not support nested virtualization, so even if Google does include QEMU in their build, it will only work on X86 ChromeBooks.
Can you run Adobe Products on this?
Nice video, but I can't get the resources usage data like CPU from LXD host for windows guests.. Kepp the great work and improve:)
Yes, that's normal and will happen until we either get the LXD agent working on Windows (waiting for vsock and virtiofs drivers to work reliably) or we implement support for fetching as much as we can from the host side (which we have plans to do over the next 6 months or so).
my Windows 11 VM doesn't have internet and it sucks. I've tried everything to no avail.
dropping some commands that resolved some issues for me
sudo snap set lxd ui.enable=true
sudo systemctl reload snap.lxd.daemon
lxc config set win11 raw.qemu="-cpu host"
Does anybody knows if it’s possible to run this LXD Win11 installed on an Ubuntu machine remotely from a MacOS machine? Please, any guidance is welcome.
Comparing to QEMU/KVM stack, does it rely on KVM and kind of replaces QEMU?
LXD drives QEMU so it can be seen as a replacement to something like libvirt instead.
@@LXD thank you! that gives some understanding
Salut Stefan. Is it possible to install FreeBSD, or GhostBSD etc. on a lxc VM? I had no luck so far getting it to finish the installation. It's crashing and throw me out of the shell. Would be great if you could provide a full HowTo sometime. Thanks very much for your great videos.
Can you please add a step to passthrough Intel (or cuda) GPU
LXD VMs support GPU passthrough with the `gpu` device type.
However this will almost certainly not do what you want as you need a physically dedicated GPU per VM.
That's because GPUs cannot be shared between VMs or with the host system.
So passing in your GPU will have it get unplugged from the host system which in most cases will lead to a crash of the operating system on the host.
If you do have a dedicated GPU that you want to use for a VM, then you need to:
- Make very sure that no drivers are attached to it on the host system (you may need modprobe.d blacklist entries or force the vfio-pci driver to take over that GPU)
- Enable IOMMU on your system (VT-d or similar in BIOS and iommu=pt or similar kernel boot option)
- Confirm that the GPU is alone in its IOMMU group (lspci can help with that)
- Finally, pass it to your VM with `lxc config device add VM-NAME DEVICE-NAME gpu gputype=physical pci=PCI-ADDRESS`
How can I choose where to put the root disk for windows?
Does distrobuilder have the ability to also perform an iso repack for other OS's like Mac OSX?
Currently this is limited to Windows where we support Windows 10/11 as well as most versions of Windows server (2019, 2016, 2012).
MacOS would be fun to get working though I don't think it'd be quite as trivial as injecting a few drivers in an install media. Last I checked, there weren't Darwin drivers for most of the devices that we rely on in LXD VMs, so that'd be the first blocker. Other solutions get around this issue by using non-virtio QEMU devices instead, effectively emulating traditional hardware which macOS knows how to handle.
This isn't something we expose with LXD VMs as we use a very minimal build of QEMU that primarily relies on virtio devices (limiting attack surface).
So we'll have to see. If someone gets a full set of virtio drivers for Darwin and we can inject those into a macOS install media, then this is something we could add to distrobuilder, but that seems quite unlikely to me. Not to mention the current move to ARM isn't going to make this any easier.
And lastly, my understanding is that you are only (legally) allowed to virtualize macOS when running on Apple hardware, so this would only really work if you were to be running Linux on a Mac and then would want to run macOS on it inside a VM. A pretty limited usecase :)
Hi, I can get it to work with Ubuntu but I can't get it to work on Arch or Manjaro. Is there any special trick for the TPM or the OVMF?
It could be a version thing or be related to exactly how they built EDK2.
github.com/lxc/lxd-pkg-snap/blob/latest-edge/snapcraft.yaml has the details on how we build qemu, edk2, swtpm and libtpm which are the main components involved there.
I don't think we're particularly picky about the version of any of those, except that there are some known issues with qemu 7.2 so we've been sticking to 7.1.
If you need firmware measurements through the TPM, it's important that edk2 is built with TPM2 support, maybe that's missing on Arch.
Is sound working?
What host are you using?
This was running on my desktop machine, an AMD Ryzen 7 5700G.
LXD performs CPU passthrough to the guest so as long as the host is compatible with Windows 11, so will the guest.
@@stephane.graber Thank you.
Is the procedure same with windows servers iso?
Yeah, distrobuilder supports converting Windows Server images, so the exact same procedure will work there.
So many typos doesn't raise confidence. English needs a little polishing and a note: source path doesn't work with symlink which is just poor programming.