Thank you so much for sharing your knowledge. I have to add that, for a non-English speaker like me, your cadence in speaking and your pronunciation help a lot to understand what you are saying. Good job. 👍
none of what you showed worked at all, web UI is a complete mess, when i try to create lxc container, the next button greyed out, NO MATTER what i do, i have given it hostname/passwork, even found pub rsa key file to paste in, NOTHING works. "next" simply CAN NOT be clicked to proceed.
LXC is very useful. I've ran both containers and VMs using LXC (yes including a windows VM) issues is that it's it's great for creating and testing systems locally but it's intentionally a total pain in the posterior to "expose" a container to the network. I did use a rpi3b running alpineOS to run a few basic containers I could access from the internet to test things.
This is so scary every time I’m working with any product, Jim comes around the corner and makes a video about it. Superb! I would really want a guide to install ansible awx on a single node k3s LXC without much hassle, is this even possible? I tried multiple guides with no luck
Hi Jim, I've just stumbled across your videos, and this looks really useful. You mentioned about default SSH keys from when you spun up Proxmox. Could you point me to the video you mention this? I can't see anything obvious at the moment but I'll keep digging
From a management and monitoring perspective, what do you prefer to use, Docker or LXC? I like the idea of using LXC as each service will be visible/exposed directly in the Proxmox GUI, but I suppose the Portainer, Rancher etc. could solve the monitoring part when using Docker.
Nice! My only concern here is how the lifecycle is. When using docker I have watchtower for many of my containers and for the critical ones it is just a manual docker compose pull (or the Portainer GUI). Let's say a move them all to individual LXC containers. Now what? I've seen many of these videos, but no one addresses this important fact. Cheers!
@@Jims-Garage that is ok. That could be the best of both worlds. I have read different opinions on installing docker on LXC, but I guess it is a matter of trying it. Do you use it like that or have docker on top of a VM? In my case if I am going to have most of my services in a single environment, it could be just a VM. Since I am not splitting out too much else, why bother. I haven't reached the kubernetes world yet, but I'm looking forward to it. Cheers
@@fedefede843 I predominantly use Kubernetes on VMs and have a solo docker VM. I am likely to shift over to LXC in the near future as I want to share my GPU with all containers. I'll do a video on the journey.
Good video, as much as I know another difference is that VMs restart automatically after migration (cluster) and LXC container have to be started manually, right?
Would you consider doing a video on utilising LXC container for something like Radarr using the helper templates - I tried this but had lots of issues with access rights writing to shared storage ?
Tried it all commands worked but still read only. I think I have an issue making sure the LXC uses the right user account when running - it does mention username depends on application
11:24 AFAIK your LXCs will use the swap space of your Proxmox VE host, if it doesn't have swap set up, the LXC will likely get killed trying to allocate any swap.
One problem you may hit if you are on AMD and passing hardware acceleration to jellyfin or frigate , to pass your GPU with PCIe passthrough you blacklist nvidia and radeon , amdgpu drivers etc. So stopped the hardware acceleration from the GPU until I removed the radeon & amdgpu from the blacklist.
@@pabloszi you don't , BUT if you do pass a full PCI-e device to a VM don't disable the inbuilt iGPU at the same time. I pass iGPU hardware encoding / decoding to a Frigate & Jellyfin LXCs , but have a VM with my RTX2070 super to use for tdarr.
@@peteradshead2383 That's right. It works as you described with VM, but LXC isn't VM. LXC is Linux container, and you can't passthru PCI cards to LXC. Instead of this you can share devices between Proxmox host and LXCs.
I have started with Proxmox with VMs only as it was much easier to work with (for the newbie like me). But now, I move almost everything to LXC. Lighter but most importantly: I feel much better with IP management - rather than assigning MACVLAN in docker (in VM). Sharing the storage sometimes is an issue but once you know how to it is easy. But still there is a place for VMs there.
Excellent. Question: how much thinner is an LXC vs a VM? That is to say, if a Debian install as a VM is X megabytes, how much smaller is the LXC install?
Well I have a SMB server with Webmin on 81k of ram , netboot-xyz on 81k , and jellyfin on 374k , so most programs only need 512k or less , try running a running jellyfin on a 512k VM .
So can I run docker in lxc and do the same setup you do on a vm setup like for jellyfin radar sonar and so on for it? But would you want to do on a lxc this way since I heard it’s not as secured as doing it on a VM
You'll need to deploy a desktop image and then use a remote desktop application. It doesn't give a great user experience though (I find Linux remote desktops not great with Windows).
@@Jims-Garage Thanks for the reply! In your 1 gpu 4 LXC it seems like you have a GUI LXC. Basically I'd like a web browser in their own container. I have this now but use VMs. What if I deploy a desktop image to the LXC and then use a remote desktop application like spice or something? Or do I need to use a VM to deploy multiple browsers like I'm doing now (seems kind of inefficient)? Thanks!
@@Jims-Garage Okay, I'll check it out! Would the docker containers be unprivileged and secure? How did you get jellyfin running in the LXC containers? I watched the video, I'm wondering if I missed that part. If you have time I'm also wondering about a Proxmox system with a 5950x, 3090 and Intel Arc a380 or a750. Could I passthrough the 3090 to a VM and try to split the Intel Arc (running as the main GUI) between containers or VMs. I tried passing through the 3090 to a VM but since i dont have integrated graphics or a second graphics card the VM would not start. Thanks!
I'm root on the LXC, but in a separate namespace on the host. If privileged you're using the host namespace which is bad. You can create another user on the LXC like any regular Linux distro
@@Jims-Garage The other way I thought of this was like the root inside the LXC is alias of another user of the host which automatically generates upon LXC creation. It just happens to have the root name.
Interesting observation. I'm from the middle of the UK originally and nowhere else in the country sounds like that to me. I now live somewhere very different 😂
none of what you showed worked at all, web UI is a complete mess, when i try to create lxc container, the next button greyed out, NO MATTER what i do, i have given it hostname/passwork, even found pub rsa key file to paste in, NOTHING works. "next" simply CAN NOT be clicked to proceed.
Started watching on the OPNsense setup video and it was very helpful, thank you! I've been wanting to setup Jellyfin in an LXC for GPU transcoding, but it seems like a hassle. Would love a video on it like you mentioned!
I recently set up an dedicated game server for the game Enshrouded in Proxmox, alas it's pretty resource hungry. My first try in a VM was very disappointing. I have an Ryzen 4300G running my homelab and i gave the VM 8 vCPUs (so one for each physical thread) and 16 GB RAM. The gaming experience for one connected player was O.K'ish but for three ore more it was unusable. I then switched to a (privileged) LXC also 8 vCPUs but only 8 GB RAM (the game is more CPU than memory hungry). The gaming experience is much(!) better. Now gaming with 3 or 4 players is possible. Nonetheless the Enshrouded dedicated Server is pretty bad performing it consumes almost half of my Homelab which was not the case with other game servers like Valheim e.g.! But yes there are performance gains when using LXC in such "extreme" situations!
Hi Good video and teaching. Question (maybe a leter video) can i create 1 lxc and install 1 instance of docker and within this environment have multiple applications such as pihole and tailscale? or will i need to make seperate lxc?
Why do people always think in "better" or "worse" terms? Please do not implement clickbite titles on your channel. Things are just different, something can be more efficient in some kind situations but not in certain use cases. Besides this good educational video, I will share this to my students. 🙏🙏
Appreciate the feedback. The title is reflective of a question that is frequently asked and I do believe there is a better choice in every situation, hopefully I called that out. E.g., internal Vs external, resources, performance.
Yes, albeit I think we need to be careful with wording. My understanding is that you're giving the LXC access to the device, not passing it through like a VM. As such, the host can see it as well as other LXCs if you allow it. This is how I'm able to share a single GPU with 3 LXCs.
@@Jims-Garage i was also able to pass through my nvidia gpu to lxc using cgroups in the .conf file, the thing is the same method would not work for disks for some reason, the disk just wont be listed under /dev/disk/by-id/ , if you had any luck please pass the knowledge.
none of what you showed worked at all, web UI is a complete mess, when i try to create lxc container, the next button greyed out, NO MATTER what i do, i have given it hostname/passwork, even found pub rsa key file to paste in, NOTHING works. "next" simply CAN NOT be clicked to proceed.
@@Jims-Garage i am not sure what you mean by that, you mean enable it from the proxmox web interface? not the host machine right? how exactly should I do that? everything else was followed in this video, and proxmox was up to date through ssh before any attempt to create new container
@@Jims-Garage if you either give me instruction, or link to the doc, i will read it, other than that, this is what happened, and i followed every steps of your videos and proxmox is all up to date
@@Jims-Garage wait i know, i was creating vms on the same machine, vms are easy, and it works, its just sluggish so much, so yes, i believe the virtualisation IS enabled
none of what you showed worked at all, web UI is a complete mess, when i try to create lxc container, the next button greyed out, NO MATTER what i do, i have given it hostname/passwork, even found pub rsa key file to paste in, NOTHING works. "next" simply CAN NOT be clicked to proceed.
![[TH] FS vs BME - VCT Ascension Pacific - Lower Bracket Final](http://i.ytimg.com/vi/wnygqsR2Fig/mqdefault.jpg)
Thank you so much for sharing your knowledge. I have to add that, for a non-English speaker like me, your cadence in speaking and your pronunciation help a lot to understand what you are saying. Good job. 👍
Thanks, I really appreciate the feedback
none of what you showed worked at all, web UI is a complete mess, when i try to create lxc container, the next button greyed out, NO MATTER what i do, i have given it hostname/passwork, even found pub rsa key file to paste in, NOTHING works. "next" simply CAN NOT be clicked to proceed.
To be fair we invented the language so going back to the source makes sense 👍👍 completely unbiased of course! 😂
The proxmox forums helped me a ton when I was manually setting up a sandbox container from a custom buildroot image
I love this Proxmox videos! Keep up the good work!
Thanks, I have a few more to tick off at least
LXC is very useful. I've ran both containers and VMs using LXC (yes including a windows VM) issues is that it's it's great for creating and testing systems locally but it's intentionally a total pain in the posterior to "expose" a container to the network. I did use a rpi3b running alpineOS to run a few basic containers I could access from the internet to test things.
Was looking forward to this video, your explanations are great!
Thanks, I hope it was helpful
This is so scary every time I’m working with any product, Jim comes around the corner and makes a video about it. Superb!
I would really want a guide to install ansible awx on a single node k3s LXC without much hassle, is this even possible? I tried multiple guides with no luck
K3S is possible on LXC, I'll have a look at it.
I hear: "Hey everybody...." and I click like! Jim's Garage should have 100k subscribers
Haha, thanks 👍
Thanks. Great video
Glad you liked it!
Thank you for this
You're welcome
Hi Jim, I've just stumbled across your videos, and this looks really useful. You mentioned about default SSH keys from when you spun up Proxmox. Could you point me to the video you mention this? I can't see anything obvious at the moment but I'll keep digging
@@GeorgeHirst93 I have a video on cloud init templates, that's probably the right one
@@Jims-Garage amazing! Thanks
From a management and monitoring perspective, what do you prefer to use, Docker or LXC?
I like the idea of using LXC as each service will be visible/exposed directly in the Proxmox GUI, but I suppose the Portainer, Rancher etc. could solve the monitoring part when using Docker.
I prefer docker in a VM (albeit most of my stuff is in Kubernetes)
@@Jims-Garage Maybe an idea for another video, detailing pros and cons, your preferences and why, management etc. :)
Yes do a video on igpu passthru
It's on the way
Nice!
My only concern here is how the lifecycle is. When using docker I have watchtower for many of my containers and for the critical ones it is just a manual docker compose pull (or the Portainer GUI).
Let's say a move them all to individual LXC containers. Now what? I've seen many of these videos, but no one addresses this important fact.
Cheers!
No, install docker on LXC. Use docker as normal
@@Jims-Garage that is ok. That could be the best of both worlds. I have read different opinions on installing docker on LXC, but I guess it is a matter of trying it.
Do you use it like that or have docker on top of a VM? In my case if I am going to have most of my services in a single environment, it could be just a VM. Since I am not splitting out too much else, why bother. I haven't reached the kubernetes world yet, but I'm looking forward to it.
Cheers
@@fedefede843 I predominantly use Kubernetes on VMs and have a solo docker VM. I am likely to shift over to LXC in the near future as I want to share my GPU with all containers. I'll do a video on the journey.
Can you show a video on the iGPU and jellyfin. I have this setup on a NUC got all the configuration working but jellyfin cant transcode.
It should be exactly the same but 128 instead of 129. Have you tried that?
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/fb0 dev/fb0 none bind,optional,create=file
lxc.idmap: u 0 100000 65536
lxc.idmap: g 0 100000 44
lxc.idmap: g 44 44 1
lxc.idmap: g 45 100045 61
lxc.idmap: g 106 103 1
lxc.idmap: g 108 100108 65428
@@Jims-Garage
Yes I have@@Jims-Garage
Thank you for your video.
How to create an LXC container with docker or k3s?
Docker , just install as I did. K3S, coming soon
Docker is easy, docker swarm is the problem with nfs shares
Good video, as much as I know another difference is that VMs restart automatically after migration (cluster) and LXC container have to be started manually, right?
I'll have to check. I know they can auto start on creation.
Would you consider doing a video on utilising LXC container for something like Radarr using the helper templates - I tried this but had lots of issues with access rights writing to shared storage ?
Probably not as TH-cam isn't a fan of those topics... I suspect it's a simple case of permissions though.
How about making a shared drive available to a container. My issue is being asked multiple times without a good answer 😊
@@meandthemrs896 check here for a cifs example - forum.proxmox.com/threads/tutorial-unprivileged-lxcs-mount-cifs-shares.101795/page-4
Tried it all commands worked but still read only. I think I have an issue making sure the LXC uses the right user account when running - it does mention username depends on application
11:24 AFAIK your LXCs will use the swap space of your Proxmox VE host, if it doesn't have swap set up, the LXC will likely get killed trying to allocate any swap.
Good to know, I'll have to experiment. Thanks for the comments.
One problem you may hit if you are on AMD and passing hardware acceleration to jellyfin or frigate , to pass your GPU with PCIe passthrough you blacklist nvidia and radeon , amdgpu drivers etc.
So stopped the hardware acceleration from the GPU until I removed the radeon & amdgpu from the blacklist.
Hmmm... With LXC you don't need to passthru PCI-e devices. You can share it between your desired LXC, Proxmox OS and other LXCs you want.
@@pabloszi you don't , BUT if you do pass a full PCI-e device to a VM don't disable the inbuilt iGPU at the same time.
I pass iGPU hardware encoding / decoding to a Frigate & Jellyfin LXCs , but have a VM with my RTX2070 super to use for tdarr.
@@peteradshead2383 That's right. It works as you described with VM, but LXC isn't VM. LXC is Linux container, and you can't passthru PCI cards to LXC. Instead of this you can share devices between Proxmox host and LXCs.
I have started with Proxmox with VMs only as it was much easier to work with (for the newbie like me).
But now, I move almost everything to LXC.
Lighter but most importantly: I feel much better with IP management - rather than assigning MACVLAN in docker (in VM).
Sharing the storage sometimes is an issue but once you know how to it is easy.
But still there is a place for VMs there.
Absolutely, both have strengths and weaknesses. As always, a blend is usually best.
Excellent. Question: how much thinner is an LXC vs a VM? That is to say, if a Debian install as a VM is X megabytes, how much smaller is the LXC install?
About 20x smaller
Well I have a SMB server with Webmin on 81k of ram , netboot-xyz on 81k , and jellyfin on 374k , so most programs only need 512k or less , try running a running jellyfin on a 512k VM .
I've run the turnkey-gitea template and I've found the they tend to leg behind in software updates, by a few versions.
Could I run the a arr stack on a lxc container and docker with trafik and stuff like you did with the VMs in the previous series?
So can I run docker in lxc and do the same setup you do on a vm setup like for jellyfin radar sonar and so on for it? But would you want to do on a lxc this way since I heard it’s not as secured as doing it on a VM
That is what I demonstrated, Jellyfin on Docker with GPU passthrough in a unprivileged LXC.
@@Jims-Garage oh ok I just wanted to make sure so ima just follow this thanks 😊
Hi there! How can I get an unprivileged LCX with a GUI setup? Thanks!
You'll need to deploy a desktop image and then use a remote desktop application. It doesn't give a great user experience though (I find Linux remote desktops not great with Windows).
@@Jims-Garage Thanks for the reply! In your 1 gpu 4 LXC it seems like you have a GUI LXC. Basically I'd like a web browser in their own container. I have this now but use VMs. What if I deploy a desktop image to the LXC and then use a remote desktop application like spice or something? Or do I need to use a VM to deploy multiple browsers like I'm doing now (seems kind of inefficient)? Thanks!
@@codescholar7345 you could deploy chrome in docker if you prefer github.com/browserless/browserless
@@Jims-Garage Okay, I'll check it out! Would the docker containers be unprivileged and secure? How did you get jellyfin running in the LXC containers? I watched the video, I'm wondering if I missed that part. If you have time I'm also wondering about a Proxmox system with a 5950x, 3090 and Intel Arc a380 or a750. Could I passthrough the 3090 to a VM and try to split the Intel Arc (running as the main GUI) between containers or VMs. I tried passing through the 3090 to a VM but since i dont have integrated graphics or a second graphics card the VM would not start. Thanks!
@@codescholar7345 jump on Discord, all of this is doable.
At 7:58 you ve mentioned you were going to be using a non root user. At 13:25 you re logging in as root.
I'm root on the LXC, but in a separate namespace on the host. If privileged you're using the host namespace which is bad. You can create another user on the LXC like any regular Linux distro
@@Jims-Garage The other way I thought of this was like the root inside the LXC is alias of another user of the host which automatically generates upon LXC creation. It just happens to have the root name.
I don't know why but so many Englishmen seem to have this exact voice.
Interesting observation. I'm from the middle of the UK originally and nowhere else in the country sounds like that to me. I now live somewhere very different 😂
this is the worst i ever seen, why do you skip so many details?
RTFM?
none of what you showed worked at all, web UI is a complete mess, when i try to create lxc container, the next button greyed out, NO MATTER what i do, i have given it hostname/passwork, even found pub rsa key file to paste in, NOTHING works. "next" simply CAN NOT be clicked to proceed.
excellent easy to follow guide as always. i would be interested to see you setup k3s using lxc
Thanks, it's coming
I woud like to see K3s on LXC
It's in the works!
Started watching on the OPNsense setup video and it was very helpful, thank you! I've been wanting to setup Jellyfin in an LXC for GPU transcoding, but it seems like a hassle. Would love a video on it like you mentioned!
Thanks, I might do it next. Watch this space.
I've overtime moved all of my VMs to be LXC. They just do everything so quick. Boot in seconds so small easy to back up and migrate.
Yes, they're extremely performant. It's a great benefit.
Didn't think I'd be learning from a muckle (MCC) rider! Thanks for sharing your knowledge, see you on the road!
Haha, thanks! Keep it shiny side up 🚲
Where can I find my SSH Key?
It's in the root folder on Proxmox. Alternatively you can generate and use your own.
Jump to 5:00
You're welcome
Haha, thanks 😂
Thanks for this video and the lovely face
The way you explain things is top notch bro thank you
Thank you, appreciate the feedback.
Thank you James, As always, excellent and with comprehensive explanations.👌 👍
Very welcome
I recently set up an dedicated game server for the game Enshrouded in Proxmox, alas it's pretty resource hungry. My first try in a VM was very disappointing. I have an Ryzen 4300G running my homelab and i gave the VM 8 vCPUs (so one for each physical thread) and 16 GB RAM.
The gaming experience for one connected player was O.K'ish but for three ore more it was unusable.
I then switched to a (privileged) LXC also 8 vCPUs but only 8 GB RAM (the game is more CPU than memory hungry). The gaming experience is much(!) better. Now gaming with 3 or 4 players is possible.
Nonetheless the Enshrouded dedicated Server is pretty bad performing it consumes almost half of my Homelab which was not the case with other game servers like Valheim e.g.! But yes there are performance gains when using LXC in such "extreme" situations!
That's great feedback, thanks. Definitely something to consider if you're looking for high performance.
1:09 This is true, however, it is possible to run a Linux userland in a FreeBSD jail :^)
If I wanted to spin up a wordpress website, hosting around 80-400 gb per month, VM or Container ?
VM, better isolation IMO
Hi Good video and teaching. Question (maybe a leter video) can i create 1 lxc and install 1 instance of docker and within this environment have multiple applications such as pihole and tailscale? or will i need to make seperate lxc?
Check the next videos 😁
Why do people always think in "better" or "worse" terms? Please do not implement clickbite titles on your channel.
Things are just different, something can be more efficient in some kind situations but not in certain use cases.
Besides this good educational video, I will share this to my students. 🙏🙏
Appreciate the feedback. The title is reflective of a question that is frequently asked and I do believe there is a better choice in every situation, hopefully I called that out. E.g., internal Vs external, resources, performance.
Very good, as usual 👍🏻.
Thanks, I appreciate the feedback
Amazing guide as usual.
One question, is it possible to pass through individual disks to an lxc, not just mount point, the entire hard disk?
Yes, albeit I think we need to be careful with wording. My understanding is that you're giving the LXC access to the device, not passing it through like a VM. As such, the host can see it as well as other LXCs if you allow it. This is how I'm able to share a single GPU with 3 LXCs.
@@Jims-Garage i was also able to pass through my nvidia gpu to lxc using cgroups in the .conf file, the thing is the same method would not work for disks for some reason, the disk just wont be listed under /dev/disk/by-id/ , if you had any luck please pass the knowledge.
excelent!
Thanks
Reminds me of Solaris Zones
lovely face 😄
Is there a way to upgrade an OS in a container? I don't think do-distr upgrade works
It does
none of what you showed worked at all, web UI is a complete mess, when i try to create lxc container, the next button greyed out, NO MATTER what i do, i have given it hostname/passwork, even found pub rsa key file to paste in, NOTHING works. "next" simply CAN NOT be clicked to proceed.
Have you enabled virtualisation in the bios?
@@Jims-Garage i am not sure what you mean by that, you mean enable it from the proxmox web interface? not the host machine right? how exactly should I do that? everything else was followed in this video, and proxmox was up to date through ssh before any attempt to create new container
@@Jims-Garage was there anything to do with the cli ? that i ssh into proxmox host, or you are purely speaking about its web UI
@@Jims-Garage if you either give me instruction, or link to the doc, i will read it, other than that, this is what happened, and i followed every steps of your videos and proxmox is all up to date
@@Jims-Garage wait i know, i was creating vms on the same machine, vms are easy, and it works, its just sluggish so much, so yes, i believe the virtualisation IS enabled
none of what you showed worked at all, web UI is a complete mess, when i try to create lxc container, the next button greyed out, NO MATTER what i do, i have given it hostname/passwork, even found pub rsa key file to paste in, NOTHING works. "next" simply CAN NOT be clicked to proceed.
Did you download a template? Try rewatching from 6:00