Excellent video Tim. I liked what you said - "I am a developer trying hard to be a sysadmin".. This resonates very well with me. I have similar setup too but at far far smaller level.. Keep up the good work. I will now be watching your videos every week.
I have been looking for a video like this. It’s so helpful to see. Going through r/homelab can be so daunting even with the wiki. So seeing it all laid out is great. Thank you.
I've been in IT for about 25 years... grew up when bare metal was the only game in town. So I never really understood the need of VMs until I saw these home lab vids. It provided my ah-ha moment of why I should move my 12 core workstation from Virtualbox VMs to Proxmox VMs.
What a great comment! Thank you so much! I am glad this helped you! After you move to VMs, you might then find that Docker can help consolidate even more! Start with VMs though and then decide later if Docker might be a good fit for some of your services! Thank you again, so glad this helped you! th-cam.com/video/pxwUXJmAER4/w-d-xo.html
This setup is SICK! But elegant. Everything that you utilize is sewn together very well. NICE to have a powerful couple of servers to get all these VMs and containers humming along seamlessly. Great vid!!!
Really enjoying these videos as I build out services on my own lab. No formal computing g background so learning everything the hard way. Would be really interested in an overview of all of the firewalls you talked about as well as why you decided on sophos. I'm gravitating towards Pfsense for now primarily because of the great documentation as well as lots of how to videos.
As always, good stuff bro. FYI, I noticed you get better results in Windows if you pass the controllers thru via PCI for every device possible. I use to get freezing when I initiated a file transfer between my Windows VM and my FreeNAS VM via a quad gigabit adapter I had configured in a bond, until I passed through a single port PCI E network adapter I had laying around, solely for my Windows VM. Now it feels like a native setup.
Thanks for making this video. It's super timely for me as well, since I'm setting up a rack very similar to yours and this setup seems like it would solve a lot of the issues I've been having with unraid.
One reason you need vlans is to separate your public facing VM's and containers from the rest of your network. If someone manages to hack into one of your public facing VM's/containers he/she will be able to use that as a starting point to attack the rest of your network. However if your public facing stuff is in a separate vlan, a hacker compromising one of your public VM's will be able to attack your other public facing stuff, but not the rest of your network.
Great Video Tim!!! I love the way you teach stuff. It's easy to understand, but somehow you manage to fit every aspect of a topic in one 20 min video. Awesome. I have a video idea for you: Since you already did a tutorial on Proxmox and are all about managing Kubernetes clusters with Rancher, why not make a video about Monitoring Proxmox with the built in "Export to External Metric Server" feature and setting up a database such as InfluxDB with some Visualization using Grafana. You could split it up into 2 parts, since databases and Grafana are a ratter big topic, but I'm sure you're gonna figure out some way to do it! Thanks I really appreciate you making these tutorials!
Hey Tim, awesome videos. I'm also a software engineer and need to build a homelab to for training purposes. I discovered your channel while researching the topic of building a homelab. I'm glad I found your channel, because it helped to clarify some open questions. I will start the coming weekend with the setup of my homelab. Two questions I have nevertheless. - Your setup also includes an ups, what did you have to configure for that? - Are the VMs that Proxmox manages LXC containers? - Is proxmox dependent on ZFS, or can it also work with a harware RAID 10 based on xfs? Somehow I have the impression that the discussions on this topic in context of homelabs strongly opinionated, almost religious. Is somewhat hard for a linux noob to figure out the way to go. Any I am curious how my journey in this topic will proceed
Hey Tim, would love to see a series on how you setup your Sophos firewall...the hardware and software side of things....also how that can be pair with PiHole to make the most of both. Thanks!
Thanks for the tour! But now I have more questions haha. Curious how you perform backups for proxmox and why you chose VM over container. Would love to see videos on how you setup each service as well. Cheers.
Have to say I like the astronomy theme to your naming convention. Also I think I may have stumbled into an answer I was looking for inregards to my own home lab server rack setup. I was going to go with a NetApp disk shelf like the one you have and a Dell R710 that I already have. It has a Perc-6i controller rather than the fancier H700 and my question is if I could pass through an HBA card to Proxmox and still use the Perc-6i for local drives. Looks like I can. And that is a monster help and means I can run one less physical machine :) The First 7 minutes of this video alone have been incredibly helpful. Subbed, and now to finish watching this video.
Thanks! That's exactly what I do. I use my H700 RAID for all my VMs on Proxmox, then pass through my HBA to my FreeNAS install. th-cam.com/video/iva4DmOmSTc/w-d-xo.html
I am blown away by the content you have up on youtube. Do you have any notes or something to share about your garage door check and sending notifications?
Wow nice coverage of your homelab! You fit the devops mentality perfectly. Curious how you're handling ingress and load balancing. Do you assign a cert with a DNS record, or are you using layer2 like metallb? I've been using rancher 2 with multiple loadbalancers mapped to DHCP reservations, which is a bit annoying, but gives me some firewall flexibility in routing my services. I should probably figure out multus and macvlan to have better layer 2 management.
Good question! I am not exposing my external services yet on Rancher 2 but on Rancher 1 (without kubernetes) I used Rancher's load balancer to do this.
I am actually doing a dissertation on network firewalls mostly virtualised firewalls such as the Sophos XG and how to optimise it within a enterprise enviroment
Great video! Thanks for introducing me to Rancher!! I have an, I guess, security related question: why are you running the docker daemon on a VM instead of an unprivileged LXC container? Is there any reason for this besides maybe that KVM has "complete" isolation? Thanks!
Update: I played around with it (Proxmox 7), and sadly came to the conclusion that - probably only kubelet - relies on certain kernel modules and mount points which are not available in unprivileged LXC containers ("cannot found mountpoint for cpu" I think was the error). I would either use privileged containers or VMs as hosts. If I were to do this at home, I'd choose a privileged LXC container. However, since I'm not doing this at home but rather on a dedicated server that has to be somewhat production-ready and open the least amount of possibilities for intruders, I should try it out with a couple VMs. Thank you anyways!
I really could use that Wemo garage door setup... onetime I went to work and just happened to forget to close it. Luckily my neighbor let me know and shut it.
Impressive. Would love to see a break down of how you back all this up. Redundancy...etc. A lot of interconnected dependencies, would like to see your strat for how to rebuild and restore if/when it goes down from hardware failure and others.
Hi Tim, excellent video. I noticed in your Proxmox host that under “Hardware” for your VMs, you have the Machine Type set to i440fx. If you change this to q35, your VM performance will use significantly less resources
Just discovering your channel, great video very inspiring for me to build home server My current "server" is just an old laptop with ubuntu server. not much work just serving some docker container for file sharing, blog / notes, and ebook sharing.
We were trying to track down a performance issue on some of our virtual machines at work and had a conversation with the people running VMware. They commented that we had over allocated CPUs for our machines given the CPU usage. It seemed counter intuitive at first until they explained that if we have allocated 16 cores to our VM then VMware needs to wait until it has 16 cores available before our processes could be run. Maybe allocating 24 cores to several VMs is not the best performance unless you really need them. Otherwise your virtual machines could be actually blocked waiting for CPU availability.
I'd say it's a bit dumb to have a DMZ without it being its own VLAN, since the point usually is to secure and separate in a sense those potentially exposed servers from internal servers and other systems. With separate subnets you don't have any security if a VM in DMZ gets exposed, since potential hacker could just add the subnets to it. Also without VLANs, they do hear each others broadcasts, which means there's extra clutter for all of the machines. Also you can't have DHCP as easily (without static leases). I personally use VLANs obviously in my server environment, but also even at home, where I've got separate VLANs for desktops, IoT (chromecasts, phones), NoT (Not-internet of Things), management (all networking gear have their mgt here), two internet connections, storage network, guest network, development network, VPN/tunneled network, etc. All in all, even at home I have around 15 VLANs. Many more at my homelabbity-environment which is actually 8U of co-location at my work. TL;DR: VLAN's are good and everybody should be using them.
jaketus thanks for the feedback. My DMZ is 100% virtualized and no other device physically connect to it and my router has a firewall between the two networks. May set up VLANs at some point. Thank you! I have nothing against them, just haven’t crossed that bridge yet.
Any reason you don't use LXC Containers? I used to run most services as VM's or docker containers inside a VM but recently I'm using Proxmox LXC's more and more. Even running Docker inside an LXC is easy (just need to tick some features under Options), but I don't really like nesting (not sure it impacts performance though). Setting up a new LXC is super easy and fast, then controlling everything through the Proxmox interface and managing backups is also convenient.
Long story short, Docker is widely use for cloud computing, in the enterprise, and for software engineering. Also Kubernetes uses it too. Also, I like to keep my Proxmox clean and keep my containers on a guest vm so it’s portable and scalable. Hope that helps.
Pretty cool setup! I have gone slightly differently - Ubuntu, ZFS and just plain Docker and deployment via Ansible. I have lost the ability to host vms/containers like iris, but it is also a way to go if you don’t build a cluster at home. Anyway checkout the frigate! It upgrades the ip cams to another level.
TH-cam just suggested this video to me. I think I pretty much copy some of your infrastructure. Especially rancher: Right now, I'm still messing around manually with docker-compose and my amount of containers currently is at 29... Managing all via console is getting - well - annoying :D.
Thank you for this video Tim. Off the back of this I'm going to take a look at the Sophos XG Firewall. I'm currently using pfSense with the FauxAPI addon. This allows me to easily enable/disable firewall rules through Home Assistant. This allows me to disable access to the internet on the kids tablets when they should be doing homework! Do you (Or anyone reading this) know if the same thing can be done with the XG Firewall? There is probably a better place to ask questions like this. If so, where?
my iot is isolated along with my guest network. My other public services are completely physically separated (well, virtual nics, networks, switches). vlans coming soon! Thank you!
No LXC containers on Proxmox? Curious as to why you don't run docker from a container in Proxmox rather than a VM. Awesome video, I thought I knew everything about homelab until I found your channel...
Thanks for sharing Tim. I'm a new subscriber... Can you do a video explaining why you favour proxmox? 👀 You may have covered this already, but curious how others come to this choice. Thanks!
Forgive my lack of knowledge if I am incorrect, but another resource suggested you need a minimum of 3 nodes to run a HA cluster in ProxMox; is that the case? It was stated by Craft Computing that each node votes on which is the primary or secondary and if you have only 2 they each vote for the other and the mechanism doesn't work.
Great JOB for a home setup. How noisy the desk shelf you are using?? How many Power Supplies are you using? How long are you Virtualizing FreeNAS although it is not recommended???
not too noisy, I have redundant power supplies for my server, FreeNAS runs fine on virtual hardware if done right. They even have documentation on how to do it. Here's how th-cam.com/video/iva4DmOmSTc/w-d-xo.html
Great video, this like watching myself from a few years back when I was limited by a 14u rack.. now I have a 42u rack.. my advise, stick with a smaller rack :)
Nice tour. Makes me wanna try proxmox again and I'm definately putting Rancher on my server now. Don't you have a huge power bill with all that equipment running? Or maybe electricity is just so much cheaper in the US? (I'm located in Denmark)
I run unraid in a vm on my vmware cluster. Only thing you need to remember is that USB stick with the license bound to it. I just passthrough the usb to the vm
What are the reasons you chose to pass through your LSI controller to TrueNAS as opposed to just creating the logical volume(s) in ProxMox and then just assigning that storage to the TrueNAS VM? I'm new to Proxmox so go easy on me :D Cheers for the video. Crazy stuff you got there.
I have a remarkably similar setup using an HPE DL380 Gen9 and a repurposed gaming system both running Proxmox. I would be interested to hear your thoughts about running them in a Proxmox cluster. I have gone back and forth but, without any great need for replication/high-availability (or a 3rd machine for quorum issues), am currently running them independently like you. That said, I think clustered servers offer some benefits for storage. What do you think?
Curious on how your FreeNAS VM performs. I've been wrestling with a virtualized instance for a while, but have found the performance really poor. It has 64 GB of RAM and is mostly used for SMB sharing, and consistently slows down after a couple of days of use. I was thinking about running it bare metal just to rule out any possible virtualization issues, but that would require a lot of server and hardware shuffling. So I'm just sitting on a slow VM for now.
Mine runs great. I reboot mine only about every 60 days or so when there’s an update. How many users? Do you have a cache disk? This might help. Doubting it’s virtualization.
@@TechnoTim Thanks for the reply. I did try adding a cache disk, but performance was unchanged. I've been the only user so far, and my research send to point to either a physical fault in my network or some sort of bugginess with the SMB protocol. Good to hear that yours works - that gives me hope that I can get this working at some point!
@@TechnoTim Just wanted to do a follow-up since I did more testing. Got a server from work to play with for a couple of weeks and virtualized FreeNAS on it in a very similar setup to my home setup, and it runs great. So now I'm thinking I either have a bad disk, or the LOM on my server is bad. But viva la virtualization!
Tim, have a question for you bud. You said in a video your home internet speed was around 300 something. How is it you are able to run so many websites within dockers. Did you upgrade your internet speed or do you just shutdown unnecessary apps and dockers? I only ask because I'm thinking of migrating my wordpress websites from Amazon and deploying them on my home server. I'm new to whole home server thing. Thanks in advance.
Sick setup! I have so many questions, but I'm gonna try to hit u up on twitch. I think the main question I would have is: Why you decided to virtualize FreeNas, and did you give FreeNas direct control of the hard drives (since that is usually recommended)? If so, how did you do that? If not, why? Great video!
@@TechnoTim found your channel this morning, and definitely top tier. I’ve been trying to get the courage to launch a channel for a while lol, thanks for the guidance in “How I built a tech TH-cam Channel” th-cam.com/video/SivkS9Ug5dw/w-d-xo.html . I’ll definitely have to check out your live streams on twitch, I have some questions about the home lab, disk array shelf and a few nuanced questions. Crazy how google shows some for $30k and some for $300… lol
wait wait are you playing on those VMs for twich/mixer or how are you piping your stream to them for compression/streaming? I also ran a FreeNAS VM on proxmox and passedthrough a sata controller. Nice setup overall, was just wondering how those 2 windows VMs actually work.
I don't use it to stream my game play, I have an always on channel that streams other content. I showed me in the video because I didn't have the rights for all the content on TH-cam.
Hi Tim, I'm new to your channel but I am addicted to it, thank you so much for taking the time to make videos! I know it takes lots of hours, I have a channel too. Anyways, I wasn't able to find a video on unifi on Docker, I had tried running it in my house and I wasn't able to get it to discover my 2 access points. If you can, could you make a video about it? Thank you again
Pretty nice. quick question, does you servers also have a public IP address? you mentioned that you were running a webpage, so I would like to know if you are actually just posting to a web server or if you actually run the web server at home. thanks in advance
Hi Tim, what do you think think about openmediavault? There are lots of easy tutorials are out there and has good portainer/docker integration. Would you recommend that? I see you are running Freenas. Also what is the advantage of running rancher on one machine (master) only vs portainer/docker combo?
Hey! I don't know enough about OMV however I use FreeNAS just as a NAS. I then use Rancher (running on Ubuntu VM) which gives me Docker and Kubernetes. I found this combo to work great, separate my concerns.
Hi Tim, great content! I'm new to virtualisation and I'd like to ask, how does provisioning CPU cores and RAM work? Your Proxmox 1 build has 24 cores, does assigning CPU cores to a service mean they get reserved for that service and subtracted from the total available? Say I assign 8 cores to one service, so now I only have 16 cores to split with the others? Or is the service only allowed to use 8 cores and I can still assign up to 24 cores for each service so long as it doesn't overload the server? Is this the same for RAM? And is this the same for other virtualisation software (VMWare, VirtualBox etc.)? I'm guessing its the latter but I just want to make sure.
Hi! Thank you! You can assign your servers total cores to every VM, this is called over provisioning and is totally acceptable. With RAM you may want to consider ballooning if your guest supports installing the agent.
Nice setup. I am starting to move toward a rack kind of system myself but I am not a developer. I want to run Gaming VMs and Dockers for UniFi, UNMS, and other services such as Pi-Hole and Plex. Currently I am in a Caselabs M8 with water cooling but I am STRONGLY considering going to rackmounted cases. Do you have any recommendations?
hello tim can't wait your stream today i got a little question for you and i hope you can help me well i choosed proxmox as a graduation project (i know it's not a big shot project :p but i'm from morocco even hyper-Vis a BIG HUGE project) so my question is how can i have access to my proxmox management page from another network or another domain
💪 Which services are you running at home?
👋 Oh hi! if you're new here, welcome! Be sure to subscribe for more content like this!
Done, subscribed. Love the content!
can you do a setup guide on these servers please?
i alredy subbed with notzafackeyshunz
Very informative thank you for sharing
Hello when I tried to install proxmox on hp z820 , I get error , can't display this video , issue with resolution.
Any solution
Tim, your journey with the firewall would be great to see.
How about this for now? th-cam.com/video/hdoBQNI_Ab8/w-d-xo.html
I would like to see that too.
UPDATE: Here's where I am at now in my firewall journey! th-cam.com/video/sWvEiVM9Yfw/w-d-xo.html
Yes indeed!
Definitely would love to hear more about your journey through the virtualization of network firewalls!
Here it is so far! th-cam.com/video/hdoBQNI_Ab8/w-d-xo.html
Cool video! I always like hearing what people actually DO with their homelabs, because I like looking for ideas.
Glad you enjoyed!
Thanks Tim!!! Keep Growing this channel. Great Editing and Videos.
Thank you so much!
Excellent video Tim. I liked what you said - "I am a developer trying hard to be a sysadmin".. This resonates very well with me. I have similar setup too but at far far smaller level.. Keep up the good work. I will now be watching your videos every week.
Glad it was helpful!
I have been looking for a video like this. It’s so helpful to see. Going through r/homelab can be so daunting even with the wiki. So seeing it all laid out is great. Thank you.
You are so welcome!
As a field service engineer for Dell/Emc, it’s always nice to see a well kept home lab. Some people’s PROFESSIONAL setups.... *flashbacks ensue*
Dude, you don't want to see our one IDF with the EMC's in it. No one has touched it in 5 years and it looks like spiderman shot Cat5-6 everywhere.
@@evalyer That's every professional network rack I've seen.
I've been in IT for about 25 years... grew up when bare metal was the only game in town. So I never really understood the need of VMs until I saw these home lab vids. It provided my ah-ha moment of why I should move my 12 core workstation from Virtualbox VMs to Proxmox VMs.
What a great comment! Thank you so much! I am glad this helped you! After you move to VMs, you might then find that Docker can help consolidate even more! Start with VMs though and then decide later if Docker might be a good fit for some of your services! Thank you again, so glad this helped you! th-cam.com/video/pxwUXJmAER4/w-d-xo.html
Beautiful. Great demonstration for moving away from hardware for everything.
Glad you liked it!
This setup is SICK! But elegant. Everything that you utilize is sewn together very well. NICE to have a powerful couple of servers to get all these VMs and containers humming along seamlessly. Great vid!!!
Thank you so much!
Really enjoying these videos as I build out services on my own lab. No formal computing g background so learning everything the hard way.
Would be really interested in an overview of all of the firewalls you talked about as well as why you decided on sophos. I'm gravitating towards Pfsense for now primarily because of the great documentation as well as lots of how to videos.
Thank you! Here's my journey up to PFSense th-cam.com/video/hdoBQNI_Ab8/w-d-xo.html
Im just starting out with my homelab (just built my whitebox server) really like your homelab video tours and guides as it really helps me out
Thank you so much!
"Somewhat Tested" I felt that as a python dev haha.
As always, good stuff bro. FYI, I noticed you get better results in Windows if you pass the controllers thru via PCI for every device possible. I use to get freezing when I initiated a file transfer between my Windows VM and my FreeNAS VM via a quad gigabit adapter I had configured in a bond, until I passed through a single port PCI E network adapter I had laying around, solely for my Windows VM. Now it feels like a native setup.
Thanks for the tip!!
Thanks for the demo and info it's nice to see the setup and how you utilize all these services. Have a great day
Thank you!
Keep it up this channel has so much potential - greetings from Switzerland! My homelab ideas get more and more expensive but it‘s totally worth it. 🙃
Thank you! Welcome from the US!
you really should have more subs.. amazing content!!
Thank you!
Review of firewalls would be much appreciated!
I don't like how Techno Tim is so good at Javascript but he doesn't get much views,he deserves more
Thank you for the kind words. Subs and shares help! Maybe someday!
Thank you for the video, i needed this kind of explanation!
Glad it was helpful!
I've just watch this at Nov 2021, its a very great project though, I will look forward for your next project ,
Thank you!
Thanks for making this video. It's super timely for me as well, since I'm setting up a rack very similar to yours and this setup seems like it would solve a lot of the issues I've been having with unraid.
Glad it was helpful! Let me know how it goes! Hopefully I'll have some more relevant videos soon!
Nice Setup! Greetz from Germany ✌
Great video! Love the homelab!
Thanks for visiting and thanks for a great product!!!
One reason you need vlans is to separate your public facing VM's and containers from the rest of your network. If someone manages to hack into one of your public facing VM's/containers he/she will be able to use that as a starting point to attack the rest of your network. However if your public facing stuff is in a separate vlan, a hacker compromising one of your public VM's will be able to attack your other public facing stuff, but not the rest of your network.
Now going back to manage my humble Proxmox server on my old i7 laptop :D :D :) :) .
Thanks, great video as always
Thank you! Sounds like a fine system!
Great Video Tim!!! I love the way you teach stuff. It's easy to understand, but somehow you manage to fit every aspect of a topic in one 20 min video. Awesome.
I have a video idea for you: Since you already did a tutorial on Proxmox and are all about managing Kubernetes clusters with Rancher, why not make a video about Monitoring Proxmox with the built in "Export to External Metric Server" feature and setting up a database such as InfluxDB with some Visualization using Grafana. You could split it up into 2 parts, since databases and Grafana are a ratter big topic, but I'm sure you're gonna figure out some way to do it! Thanks I really appreciate you making these tutorials!
Thank you and great idea! Great feedback!
Hey Tim, awesome videos. I'm also a software engineer and need to build a homelab to for training purposes.
I discovered your channel while researching the topic of building a homelab. I'm glad I found your channel, because it helped to clarify some open questions. I will start the coming weekend with the setup of my homelab. Two questions I have nevertheless.
- Your setup also includes an ups, what did you have to configure for that?
- Are the VMs that Proxmox manages LXC containers?
- Is proxmox dependent on ZFS, or can it also work with a harware RAID 10 based on xfs? Somehow I have the impression that the discussions on this topic in context of homelabs strongly opinionated, almost religious. Is somewhat hard for a linux noob to figure out the way to go.
Any I am curious how my journey in this topic will proceed
Hey Tim, would love to see a series on how you setup your Sophos firewall...the hardware and software side of things....also how that can be pair with PiHole to make the most of both.
Thanks!
Thanks for the idea!
Continue with your Fantastic work 👏👏👏👏👏
Thanks!
Thanks for the tour! But now I have more questions haha. Curious how you perform backups for proxmox and why you chose VM over container. Would love to see videos on how you setup each service as well. Cheers.
Hi! Thank you! I actually prefer containers over VMs. VMs just end up being my Docker/Rancher host th-cam.com/video/pxwUXJmAER4/w-d-xo.html
Also, I have most of the videos. Check out the Homelab playlist :)
Have to say I like the astronomy theme to your naming convention. Also I think I may have stumbled into an answer I was looking for inregards to my own home lab server rack setup. I was going to go with a NetApp disk shelf like the one you have and a Dell R710 that I already have. It has a Perc-6i controller rather than the fancier H700 and my question is if I could pass through an HBA card to Proxmox and still use the Perc-6i for local drives. Looks like I can. And that is a monster help and means I can run one less physical machine :) The First 7 minutes of this video alone have been incredibly helpful. Subbed, and now to finish watching this video.
Thanks! That's exactly what I do. I use my H700 RAID for all my VMs on Proxmox, then pass through my HBA to my FreeNAS install. th-cam.com/video/iva4DmOmSTc/w-d-xo.html
Great stuff, very nice walkthrough 👌
Thank you!
I am blown away by the content you have up on youtube.
Do you have any notes or something to share about your garage door check and sending notifications?
Thank you! Not yet but I can open source it. Do you Wemo, that’s the inly way it will help!
I figured that you like the color gold!
What about monitoring and notifying? Like Kuma Uptime?
I also would like to see your firewall journey.
Wow nice coverage of your homelab! You fit the devops mentality perfectly.
Curious how you're handling ingress and load balancing. Do you assign a cert with a DNS record, or are you using layer2 like metallb?
I've been using rancher 2 with multiple loadbalancers mapped to DHCP reservations, which is a bit annoying, but gives me some firewall flexibility in routing my services. I should probably figure out multus and macvlan to have better layer 2 management.
Good question! I am not exposing my external services yet on Rancher 2 but on Rancher 1 (without kubernetes) I used Rancher's load balancer to do this.
@@TechnoTim I just watched your latest video on exposing services. Man you nailed it! Nice work Tim!
I am actually doing a dissertation on network firewalls mostly virtualised firewalls such as the Sophos XG and how to optimise it within a enterprise enviroment
Great video! Thanks for introducing me to Rancher!!
I have an, I guess, security related question: why are you running the docker daemon on a VM instead of an unprivileged LXC container? Is there any reason for this besides maybe that KVM has "complete" isolation?
Thanks!
Update: I played around with it (Proxmox 7), and sadly came to the conclusion that - probably only kubelet - relies on certain kernel modules and mount points which are not available in unprivileged LXC containers ("cannot found mountpoint for cpu" I think was the error). I would either use privileged containers or VMs as hosts. If I were to do this at home, I'd choose a privileged LXC container. However, since I'm not doing this at home but rather on a dedicated server that has to be somewhat production-ready and open the least amount of possibilities for intruders, I should try it out with a couple VMs. Thank you anyways!
Update #2: I've been very successful with k0s! It's very nice and easy to use. Would recommend for beginners.
Would love to hear more about your firewall setup and journey
Here's my latest! th-cam.com/video/sWvEiVM9Yfw/w-d-xo.html
I really could use that Wemo garage door setup... onetime I went to work and just happened to forget to close it. Luckily my neighbor let me know and shut it.
Haha! Nice! This is exactly why I wrote the code to check mine all day long! I forget sometimes too when mowing the lawn...
Supercool! Thanks for sharing. By the way, what about a video about Let's encrypt?
Great suggestion! Maybe soon!
@@TechnoTim Good! I'll be waiting for it.
Impressive. Would love to see a break down of how you back all this up. Redundancy...etc. A lot of interconnected dependencies, would like to see your strat for how to rebuild and restore if/when it goes down from hardware failure and others.
Thanks for the idea!
Try a Watchguard Virtual Firewall. I liked the Sophos, but i feel the Watchguard Firebox VM is a freaking awesome network firewall.
Very nice content! 10/10 :)
Thanks! 😁
Whoops I am late I need to see every videos off yours to make my own
Awesome video. Thank you
No problem 👍
Great content Tim. What do you use to create your Thumbnails?
thank you! phontoshop and paint dot net
@@TechnoTim Thank you. What camera and lenses do you use?
See my amazon store in the description!
Hi Tim, excellent video. I noticed in your Proxmox host that under “Hardware” for your VMs, you have the Machine Type set to i440fx. If you change this to q35, your VM performance will use significantly less resources
Thank you!
my nas is named andromeda too.. what the?? lol, great vids as always
Just discovering your channel, great video very inspiring for me to build home server
My current "server" is just an old laptop with ubuntu server. not much work just serving some docker container for file sharing, blog / notes, and ebook sharing.
Sick setup sir
Thank you!
We were trying to track down a performance issue on some of our virtual machines at work and had a conversation with the people running VMware. They commented that we had over allocated CPUs for our machines given the CPU usage. It seemed counter intuitive at first until they explained that if we have allocated 16 cores to our VM then VMware needs to wait until it has 16 cores available before our processes could be run. Maybe allocating 24 cores to several VMs is not the best performance unless you really need them. Otherwise your virtual machines could be actually blocked waiting for CPU availability.
Good call! Yeah, it doesn't protect against noisy neighbors!
I'd say it's a bit dumb to have a DMZ without it being its own VLAN, since the point usually is to secure and separate in a sense those potentially exposed servers from internal servers and other systems. With separate subnets you don't have any security if a VM in DMZ gets exposed, since potential hacker could just add the subnets to it. Also without VLANs, they do hear each others broadcasts, which means there's extra clutter for all of the machines. Also you can't have DHCP as easily (without static leases).
I personally use VLANs obviously in my server environment, but also even at home, where I've got separate VLANs for desktops, IoT (chromecasts, phones), NoT (Not-internet of Things), management (all networking gear have their mgt here), two internet connections, storage network, guest network, development network, VPN/tunneled network, etc. All in all, even at home I have around 15 VLANs. Many more at my homelabbity-environment which is actually 8U of co-location at my work.
TL;DR: VLAN's are good and everybody should be using them.
jaketus thanks for the feedback. My DMZ is 100% virtualized and no other device physically connect to it and my router has a firewall between the two networks. May set up VLANs at some point. Thank you! I have nothing against them, just haven’t crossed that bridge yet.
I second this. You should really add them when you find time. There is no segmentation between anything on that switch otherwise.
Wow dude, are u sure u don’t need a few more services ?? 😁😁
Great video 👍
Just spun up another that you'll see in my next video!
Any reason you don't use LXC Containers? I used to run most services as VM's or docker containers inside a VM but recently I'm using Proxmox LXC's more and more. Even running Docker inside an LXC is easy (just need to tick some features under Options), but I don't really like nesting (not sure it impacts performance though).
Setting up a new LXC is super easy and fast, then controlling everything through the Proxmox interface and managing backups is also convenient.
Long story short, Docker is widely use for cloud computing, in the enterprise, and for software engineering. Also Kubernetes uses it too. Also, I like to keep my Proxmox clean and keep my containers on a guest vm so it’s portable and scalable. Hope that helps.
Pretty cool setup! I have gone slightly differently - Ubuntu, ZFS and just plain Docker and deployment via Ansible. I have lost the ability to host vms/containers like iris, but it is also a way to go if you don’t build a cluster at home.
Anyway checkout the frigate! It upgrades the ip cams to another level.
Nice!
TH-cam just suggested this video to me. I think I pretty much copy some of your infrastructure. Especially rancher: Right now, I'm still messing around manually with docker-compose and my amount of containers currently is at 29... Managing all via console is getting - well - annoying :D.
Great minds think alike!
This is so Awesome!!!
When you compare Sophos XG Firewall and Netgate pfSense, it's very different UI.
Would love a bit more on the firewall.
Thank you for this video Tim. Off the back of this I'm going to take a look at the Sophos XG Firewall. I'm currently using pfSense with the FauxAPI addon. This allows me to easily enable/disable firewall rules through Home Assistant. This allows me to disable access to the internet on the kids tablets when they should be doing homework! Do you (Or anyone reading this) know if the same thing can be done with the XG Firewall? There is probably a better place to ask questions like this. If so, where?
@1:24 - not using VLANs is a very, very bad decision.
my iot is isolated along with my guest network. My other public services are completely physically separated (well, virtual nics, networks, switches). vlans coming soon! Thank you!
No LXC containers on Proxmox? Curious as to why you don't run docker from a container in Proxmox rather than a VM. Awesome video, I thought I knew everything about homelab until I found your channel...
Thanks for sharing Tim. I'm a new subscriber... Can you do a video explaining why you favour proxmox? 👀 You may have covered this already, but curious how others come to this choice. Thanks!
Great suggestion! I do talk about this quite a bit on my live stream, check it out and ask there!
Forgive my lack of knowledge if I am incorrect, but another resource suggested you need a minimum of 3 nodes to run a HA cluster in ProxMox; is that the case?
It was stated by Craft Computing that each node votes on which is the primary or secondary and if you have only 2 they each vote for the other and the mechanism doesn't work.
Great JOB for a home setup. How noisy the desk shelf you are using?? How many Power Supplies are you using? How long are you Virtualizing FreeNAS although it is not recommended???
not too noisy, I have redundant power supplies for my server, FreeNAS runs fine on virtual hardware if done right. They even have documentation on how to do it. Here's how th-cam.com/video/iva4DmOmSTc/w-d-xo.html
That is an amazing t-shirt you are wearing kind sir
Thank you so much!
I cannot find much information about that Aquarius. What's it actual running?
What a lovely and organised setup.
Just subbed would love to see you make proxmox cluster happening (For maintaining your HW node without down time of your service instance?)
Thanks for the sub! Check this out th-cam.com/video/GoZaMgEgrHw/w-d-xo.html
@@TechnoTim
Thanks! i already watched that video. I am using 4 nodes proxmox cluster with ceph storage at my workplace :)
Can you make video how to build and deploy custom service? :)
Great video, this like watching myself from a few years back when I was limited by a 14u rack.. now I have a 42u rack.. my advise, stick with a smaller rack :)
That is awesome!
love to see a video on the cloud flare docker setup
Sounds good!
Nice tour. Makes me wanna try proxmox again and I'm definately putting Rancher on my server now.
Don't you have a huge power bill with all that equipment running? Or maybe electricity is just so much cheaper in the US? (I'm located in Denmark)
Great video! Thanks for making it. I see you run FreeNAS as a VM which leads me to ask you, can Unraid run as a VM in Proxmox?
Inception! Not sure, haven’t run Unraid yet. If it did, nested virtualization may not. Also thanks!
I try to run Unraid as a VM on Vmware and it didn't work. I think it's because you can only install Unraid with the os mount on an USB key
I run unraid in a vm on my vmware cluster. Only thing you need to remember is that USB stick with the license bound to it. I just passthrough the usb to the vm
What are the reasons you chose to pass through your LSI controller to TrueNAS as opposed to just creating the logical volume(s) in ProxMox and then just assigning that storage to the TrueNAS VM? I'm new to Proxmox so go easy on me :D
Cheers for the video. Crazy stuff you got there.
Thank you! That way I don't have to pass individual disks though
Good lot of work here. How do you get the display from the vm's to your workspace physically? Thunderbolt cable to dock to displays? RDP? curious.
All headless. RDP, ssh, or proxmox console.
bruh i gotta study this video religiously
What was that Heimdal (if I heard correctly). Where you see your all services and some searchbar was top of page?
I have a remarkably similar setup using an HPE DL380 Gen9 and a repurposed gaming system both running Proxmox. I would be interested to hear your thoughts about running them in a Proxmox cluster. I have gone back and forth but, without any great need for replication/high-availability (or a 3rd machine for quorum issues), am currently running them independently like you. That said, I think clustered servers offer some benefits for storage. What do you think?
Curious on how your FreeNAS VM performs. I've been wrestling with a virtualized instance for a while, but have found the performance really poor. It has 64 GB of RAM and is mostly used for SMB sharing, and consistently slows down after a couple of days of use. I was thinking about running it bare metal just to rule out any possible virtualization issues, but that would require a lot of server and hardware shuffling. So I'm just sitting on a slow VM for now.
Mine runs great. I reboot mine only about every 60 days or so when there’s an update. How many users? Do you have a cache disk? This might help. Doubting it’s virtualization.
@@TechnoTim Thanks for the reply. I did try adding a cache disk, but performance was unchanged. I've been the only user so far, and my research send to point to either a physical fault in my network or some sort of bugginess with the SMB protocol. Good to hear that yours works - that gives me hope that I can get this working at some point!
@@TechnoTim Just wanted to do a follow-up since I did more testing. Got a server from work to play with for a couple of weeks and virtualized FreeNAS on it in a very similar setup to my home setup, and it runs great. So now I'm thinking I either have a bad disk, or the LOM on my server is bad. But viva la virtualization!
@8:49 NTP server is running in virtualized container? Seems like you not concerned in NTP at all ;)
haha I know, I know. It syncs with other servers. I know that a hardware clock is better but it's fun to play with and learn.
@@TechnoTim Oh, you will learn, definitely. :-)
Tim, have a question for you bud. You said in a video your home internet speed was around 300 something. How is it you are able to run so many websites within dockers. Did you upgrade your internet speed or do you just shutdown unnecessary apps and dockers? I only ask because I'm thinking of migrating my wordpress websites from Amazon and deploying them on my home server. I'm new to whole home server thing. Thanks in advance.
Thanks for the great video. When you get time please consider why you have changed from pfsense to other products.
Possibly! In the meantime, check out my PFSense one, it's everything leading up to XG Firewall. Also, I answer this quite a bit in my live stream!
Can you explain a bit more about how you did networking from Sophos XG to your VM's. Do the other VM's get dhcp from sophos XG?
Sick setup! I have so many questions, but I'm gonna try to hit u up on twitch. I think the main question I would have is: Why you decided to virtualize FreeNas, and did you give FreeNas direct control of the hard drives (since that is usually recommended)? If so, how did you do that? If not, why? Great video!
Thank you! I'd love to answer it on stream but until then, here's the video! th-cam.com/video/iva4DmOmSTc/w-d-xo.html
Love the name and the content, what made you choose proxmox over esxi?
Thanks! For me, it's open source with no limitations.
@@TechnoTim found your channel this morning, and definitely top tier. I’ve been trying to get the courage to launch a channel for a while lol, thanks for the guidance in “How I built a tech TH-cam Channel” th-cam.com/video/SivkS9Ug5dw/w-d-xo.html . I’ll definitely have to check out your live streams on twitch, I have some questions about the home lab, disk array shelf and a few nuanced questions. Crazy how google shows some for $30k and some for $300… lol
good job , but what kind of Sophos do You use ? (hardware Appliance ?) control center it is free ?
Software, virtual install, free for home!
Id like more info on your firewall setup :)
Thanks!
Great work please make videos and tutorials about XG FIREWALL 👍
Thank you!
wait wait are you playing on those VMs for twich/mixer or how are you piping your stream to them for compression/streaming? I also ran a FreeNAS VM on proxmox and passedthrough a sata controller. Nice setup overall, was just wondering how those 2 windows VMs actually work.
I don't use it to stream my game play, I have an always on channel that streams other content. I showed me in the video because I didn't have the rights for all the content on TH-cam.
7:56 which server is this? like which software / os does he use here... couldnt understand it
Heimdall? th-cam.com/video/PA01Z6-z8Qs/w-d-xo.html
Hi Tim, I'm new to your channel but I am addicted to it, thank you so much for taking the time to make videos! I know it takes lots of hours, I have a channel too.
Anyways, I wasn't able to find a video on unifi on Docker, I had tried running it in my house and I wasn't able to get it to discover my 2 access points. If you can, could you make a video about it?
Thank you again
Thank you! Glad you like them! I may create a video otherwise join the discord!
Pretty nice. quick question, does you servers also have a public IP address? you mentioned that you were running a webpage, so I would like to know if you are actually just posting to a web server or if you actually run the web server at home. thanks in advance
Runs at home!
Hi Tim, what do you think think about openmediavault? There are lots of easy tutorials are out there and has good portainer/docker integration. Would you recommend that? I see you are running Freenas. Also what is the advantage of running rancher on one machine (master) only vs portainer/docker combo?
Hey! I don't know enough about OMV however I use FreeNAS just as a NAS. I then use Rancher (running on Ubuntu VM) which gives me Docker and Kubernetes. I found this combo to work great, separate my concerns.
Thank you awesome vid
Thanks for stopping by! See you soon ;)
Hi Tim, great content! I'm new to virtualisation and I'd like to ask, how does provisioning CPU cores and RAM work? Your Proxmox 1 build has 24 cores, does assigning CPU cores to a service mean they get reserved for that service and subtracted from the total available? Say I assign 8 cores to one service, so now I only have 16 cores to split with the others? Or is the service only allowed to use 8 cores and I can still assign up to 24 cores for each service so long as it doesn't overload the server? Is this the same for RAM? And is this the same for other virtualisation software (VMWare, VirtualBox etc.)?
I'm guessing its the latter but I just want to make sure.
Hi! Thank you! You can assign your servers total cores to every VM, this is called over provisioning and is totally acceptable. With RAM you may want to consider ballooning if your guest supports installing the agent.
@@TechnoTim Thank you Tim!
Nice setup. I am starting to move toward a rack kind of system myself but I am not a developer. I want to run Gaming VMs and Dockers for UniFi, UNMS, and other services such as Pi-Hole and Plex. Currently I am in a Caselabs M8 with water cooling but I am STRONGLY considering going to rackmounted cases. Do you have any recommendations?
I have lots of suggestions in my video description. See my Amazon store too!
hello tim can't wait your stream today
i got a little question for you and i hope you can help me
well i choosed proxmox as a graduation project (i know it's not a big shot project :p but i'm from morocco even hyper-Vis a BIG HUGE project)
so my question is how can i have access to my proxmox management page from another network or another domain
Diid you pay for you electricity? In Portugal I have a nuc i3 15 tb on external storage with Debian and docker. Firewall is the mikrotik one.
Yes I pay for electricity.
You've been a naughty boy, Techno Tim. 60 objectionable websites seen???
It classifies gaming sites as questionable 😀
@@TechnoTim uhuhhh...