I've been wanting to do something like this for a good while now for my own home server/lab, and with this video you answered most of the questions I had! Thanks a bunch!
nice work For 6 months I have been searching for ways to implement this work. Please continue and show us all the ways that lead to the work of cluster and supercomputers.
The things you did are good for a first setup but i have some tips: - make use of projects to add minecraft servers. In a project configuration you can tell how much resources a project may take. We have projects for our buildserver and management seperate so you can give for example someone access to the buildserver project but not to it's management project. - do for example the minecraft server in it's own namespace. If you do that it becomes easier to filter for it.
I didn't have a clue what you went over on this video (b/c im new to all of this myself) but it looks very interesting. Keep making these type of videos because you seem to be more knowledgeable than what you put yourself to be.
Setting up the storage as RWO was totally right. The only thing you got wrong was that you'll need to create a StatefulSet(sts) instead of a Deployment. That's because with a StatefulSet a pod replica is bound to a PVC. So when you do a rollout restart it will wait for the PVC to be unmounted before starting the new container. By setting the PVC to rwx you'll also lose a lot of performance, because it will share the volume via NFS...
With stateful sets there is a different issue: you _have to_ use several replicas. K8s guarantees that in a stateful set there will never be 2 pods pointing to the same PVC. So if a node dies, k8s will never replace the old pod, because it can't successfully terminate it. This is fine if your app natively supports clustering. For example, if you can run 3 replicas of minecraft server, each with its own unique disk storage, but with synchronized state, so if one of them dies, the remaining 2 will continue handling requests, and clients won't even notice. Databases usually have support for this mode. I'm not sure if minecraft server does. For stateful workloads that don't support clustering natively I'm yet to figure out something better than a deployment with a single replica.
I bought 4 of the M75Qs and loved them so much I bought two more and then four M75Q G2s for the new production cluster. Four of them are as powerful as my single Epyc 7402p that has 180W TDP, so they actually use less power and cost much less! I use them for kubernetes clusters that I deploy with PXE booting using my unify/synology setup and configure with ansible.
Many Nodes R/W can become a problem if the node is not completely down, but still running and just the network is disconnected. Then MC on the disconnected Node could still write into the volume, but also another MC pod would try to write there and in the end you would have data corruption. That's one of the bigger problems with Kubernetes and HA with persistent services :)
is that possible? splitting a programs execution over 2 sets of hardware, how would a program run over 2 CPUs? I have one of those lenovo M910q looking to get another for my proxmox running homelab.
Thanks! Probably not unfortunately, at least not on the channel. Maybe that could be a patreon/member only idea or something. It’s just hard making off topic videos
Cool, I'm setting up with HP mini pc also a Proxmox setup like you. Question, I'm using Home Assistant and what a HAHA home assistant with this setup since redundancy is getting more and more important for me since 70% of the house is now smart, from AC, heatpimps, central heating (in Germany you use water), lights, window shades, PIRs, PV and battery, energy management and also CCTV with frigate and I did see you did install in one of your videos a google coral acceraltor in the wifi interface. Here is the question, should I have in all 3 PC with an coral acceraltor? I guess so but wanted to be sure 😊
Have you ever experienced that the Thinkcentre shutdowns/reboot when removing the monitor? I have a M630e which does this regardless of using HDMI or Displayport. And it doesn't boot backup before adding the monitor again. Kind a defeat the purpose of a server :)
I would want to do this. Both save old broken hardware and get them working as one giant cluster for things like blender rending and video gaming. Wonder what that could look like if I ever get my act together enough to learn to do those.
I've done the same thing with 3 HP Elite Desk mini pc. However, I created a Proxmox HA cluster then I created 6 VM's for k3s. 3 master nodes, and 3 worker nodes.
Hello, I've just made a corosync and pacemaker cluster between two debian 12 machines and I want to install CasaOS on the cluster so the load is balanced between the machines. Is it possible to install CasaOS on the cluster and if it is, how?
Well, I'm currently training for the RHCE and the whole thing is about ansible. This could be an application for my elitedesk minis once I pass the exam.
If you really wanted to setup a high avelebility minecraft server you would use something like multipaper with multiple waterfall proxies with a load balancer plugin and redisbungee and a redis cluster.
I've been using a computer like those as mi personal local server, and I can say that those devices are immortal, and consumes very little power compared to the most of PCs. Thanks for the video!
Every time you say "mc" my mind adds "hammer" and I have to suppress a chuckle to not start seeming even more nuts to myself. Hammertime! diidiidiidi-didi-didi.
just wanna say that rook is probably not the best choice for most use cases. It's pretty slow and pretty dang bad on 1GiB/s. I'd much rather use drbd with the piraeus operator that does reads locally. So you'll get full NVMe speeds on reads and really fast writes thanks to drbd9. It's also very fault tolerant and survives a node drain unlike Longhorn...
I still prefer using Proxmox or Incus as base system, running k8s in VMs. You can get a lot more value from servers than just running fancy k8s distros.
Hey, I'd like to do a K3S HA cluster between different NATs (mine and my grandparent's house), since the most risky part is the internet connection. I'm considering using Tailscale for the communication between the nodes and then have 1 node run my app at all times, and when that starts it tells cloudflare to set the A record to my current IP. Is this a good or flawed plan?
Hey, I get an error: Task [K3s/POST: Apply Metallb CRS] *** FAILED - Retrying: [192.168.1.45]: Apply Metallb CRS (5 Retris Left). what could be the problem?
Loved this video. I am always curious on how this equipment works. I have not done a deep dive into it at all. But the day I do, good bye wallet. I enjoyed the video and subbed immedietaly.
this shit should be plug'n'play by today, it should be a bios option like "be slave machine" and so on, you shouldnt even need an os on the machines that are not the master one
Id be careful with notion. Its freeware with a subscription additive, aka the most frequent business strategy to die and their stuff is not in plaintext so if they go down one day youre losing your brain maps and organization systems. There are opensource alternatives.
Bro I'm in dog stage of all this, just setup my homeserver and have to learn all this stuff. Got a retropie up and running tomoght, something like this is next
I'm an expert on Kubernetes, so if you have any questions, just ask me. Also, trust me, 90% of home labs don't need Kubernetes, docker/docker compose/portainer/nomad are sufficient.
you want regular desktop cases in terms of refurbs - more expand-ability and ease of use - refurb pc are value kings - no question - having a reg sized case will aloow you to add better networking and gpu options and stay away largely from vendor proprietary parts - you might use a bit more power - that is totally ok since you will probably have more better cpu and ram - biz wants cluster tech for HA but also since they can scale out easily - just add a node or two
@@shephusted2714 Then why not specify that in your original comment? I see it says edited, not exactly sure for what though, wasn't to correct the "aloow" instead of allow. So what was the edit? And why couldn't you clarify in the original or add it during the edit?
I get it, but you’re still making assumptions. You’re assuming that people prefer expandability and performance over size and efficiency. In this instance, I 100% prefer the 1L cases over something larger and I don’t really care for >1Gbps since I’m not transferring a ton of data. So buying bulkier machines would’ve made no sense. And I don’t think I’m a unicorn. I think you could easily have said, “I would prefer and probably get larger form factors for the expandability and higher clocked CPUs” and been 100% correct. Just stop making blanket statements based on your opinion/needs/use case.
Thanks for the shout out! Happy to help! Also, Techno Tim sent me to a real expert!
😂😂😂
Heyyyy
I always get so excited when I get the notification that Hardware Haven has posed a new video
Well hopefully it doesn’t disappoint haha
Same
@@HardwareHaven it never does
Well, these guys made me realized I'm not even subscribed! I've just been binging this channel. SMH
I've been wanting to do something like this for a good while now for my own home server/lab, and with this video you answered most of the questions I had! Thanks a bunch!
what for, it seems cool, but like completely useless in my opinion, give me legitimate 5 reasons why id want this
Your production values are always great. I haven’t figured out a need for Kubernetes yet. Maybe one day.
Thanks! And yeah me too honestly haha
NIce, glad to see you showing how its all done, and that you followed some of Tim's stuff !! Good work !
Thanks 👍
My man! Just ordered 2 of these on Wednesday and will use this video to build my cluster 🙏
Nice! Just know you need at least 3 for HA
Public service announcement: Jeff Geerling is giving away his ansible book. Literally the best way to learn.
nice work
For 6 months I have been searching for ways to implement this work. Please continue and show us all the ways that lead to the work of cluster and supercomputers.
The things you did are good for a first setup but i have some tips:
- make use of projects to add minecraft servers. In a project configuration you can tell how much resources a project may take. We have projects for our buildserver and management seperate so you can give for example someone access to the buildserver project but not to it's management project.
- do for example the minecraft server in it's own namespace. If you do that it becomes easier to filter for it.
I didn't have a clue what you went over on this video (b/c im new to all of this myself) but it looks very interesting. Keep making these type of videos because you seem to be more knowledgeable than what you put yourself to be.
Setting up the storage as RWO was totally right. The only thing you got wrong was that you'll need to create a StatefulSet(sts) instead of a Deployment. That's because with a StatefulSet a pod replica is bound to a PVC. So when you do a rollout restart it will wait for the PVC to be unmounted before starting the new container. By setting the PVC to rwx you'll also lose a lot of performance, because it will share the volume via NFS...
Ah, thanks! Appreciate the help!
With stateful sets there is a different issue: you _have to_ use several replicas.
K8s guarantees that in a stateful set there will never be 2 pods pointing to the same PVC.
So if a node dies, k8s will never replace the old pod, because it can't successfully terminate it.
This is fine if your app natively supports clustering.
For example, if you can run 3 replicas of minecraft server, each with its own unique disk storage, but with synchronized state, so if one of them dies, the remaining 2 will continue handling requests, and clients won't even notice.
Databases usually have support for this mode. I'm not sure if minecraft server does.
For stateful workloads that don't support clustering natively I'm yet to figure out something better than a deployment with a single replica.
Great video as always
Thanks Leo!
You got Notion to sponsor you! Congrats dude!
Thanks! 😁
First early access video :) love the content
Nice! And thanks!
Thanks!
No, thank you!
Thanks for this very hands on video. I stumbled on to K0S which is "supposedly" even lighter than K3S and just as easy to use.
I bought 4 of the M75Qs and loved them so much I bought two more and then four M75Q G2s for the new production cluster. Four of them are as powerful as my single Epyc 7402p that has 180W TDP, so they actually use less power and cost much less! I use them for kubernetes clusters that I deploy with PXE booting using my unify/synology setup and configure with ansible.
Many Nodes R/W can become a problem if the node is not completely down, but still running and just the network is disconnected. Then MC on the disconnected Node could still write into the volume, but also another MC pod would try to write there and in the end you would have data corruption.
That's one of the bigger problems with Kubernetes and HA with persistent services :)
Oh yeah that makes sense. I really want to get a permanent k8s setup to mess around with and learn more haha
it would be cool if you made a proxmox cluster on these, and combine their resources, to run one VM
is that possible? splitting a programs execution over 2 sets of hardware, how would a program run over 2 CPUs? I have one of those lenovo M910q looking to get another for my proxmox running homelab.
GREAT IDEA BEEN THINKING ABOUT THIS FOR A WHILE
It's hilarious to see you walking through Techno Tim videos when you look like Techno Tim yourself and doing kinda same stuff on the channel 😄
I just need more RGB!!!
Your videos are getting ready high quality!
I really enjoyed watching this
Thanks! Also, I saw your message on patreon. Feel free to use my tracks, just link the video 👍🏻
Great video!🎉
Glad you liked it!
Pretty cool you ordered "parts" and got working PCs. 👍
Haha with a bit of work but yeah
Videos like these is why I am subbed to this channel.
Hi! I'm a new youtube member! I'm always happy when you put out a new video. So the least I could do is the membership on TH-cam.
Welcome aboard! And thanks!!
another awesome video Hh 🎉 thanks. Can you please do a video on Notion 🙏🏽 and how you use it for productivity flow
Thanks! Probably not unfortunately, at least not on the channel. Maybe that could be a patreon/member only idea or something. It’s just hard making off topic videos
Cool, I'm setting up with HP mini pc also a Proxmox setup like you. Question, I'm using Home Assistant and what a HAHA home assistant with this setup since redundancy is getting more and more important for me since 70% of the house is now smart, from AC, heatpimps, central heating (in Germany you use water), lights, window shades, PIRs, PV and battery, energy management and also CCTV with frigate and I did see you did install in one of your videos a google coral acceraltor in the wifi interface. Here is the question, should I have in all 3 PC with an coral acceraltor? I guess so but wanted to be sure 😊
Tough day, but your vids always get me inspired! Love it!
Love to hear that ❤️
didn't understand a single keystroke, but well worth A LIKE.
Thanks for the demo and info, have a great day
Have you ever experienced that the Thinkcentre shutdowns/reboot when removing the monitor? I have a M630e which does this regardless of using HDMI or Displayport. And it doesn't boot backup before adding the monitor again. Kind a defeat the purpose of a server :)
why didn`t you use statefullset instead of manually creating pvcs and attaching it to pods ? anyway good video :)
Because I still have a lot to learn about kubernetes haha
@@HardwareHaven Welcome to a world of confusing stuff, multiple ways to achieve goals and new problems you didn`t know you had ;)
need a workflow for k8s on baremetal and with out longhorn.
I did this several years ago with a pile of free thinclients. Twas quite hilarious 😂
Docker Swarm is also a great option as K8s can be very overkill for a home lab
Might be worth a video!
@@HardwareHaven please yes!
Hashicorp nomad too :)
I'm always excited when I get notified that you've uploaded a new video
Any interest in using n100 mini pc to run this cluster? Which I think is a good idea too, newer and lighter, draws much less power.
What is on that tablet in front of your monitors around the 2:20 mark?
Would be great to know from where you got BIOS/UEFI images ready to flash using programmer
New subscriber, your channel is awesome please don't be so hard on yourself!
Thanks! And I feel like I have a fairly healthy balance of confidence and self criticism haha
I would want to do this. Both save old broken hardware and get them working as one giant cluster for things like blender rending and video gaming. Wonder what that could look like if I ever get my act together enough to learn to do those.
Watched :45 seconds. Best intro ever. Not kidding. Here is my LIKE! :D
Bookmarking to watch the rest soon.
Awesome 👌
Sence you know inside and out of these 715q's, can you tell me how to connect external power button?
😊thx 😊
Other than soldering to the PCB, I'm not really sure
@HardwareHaven the opposite side of the power button, there are a lot of pins.
What are they?
I found out about notion last week. It's already changing my life. You should do a video on how you use it and ideas for others.
I just built this in my homelab! Ended up with 3 master + 2 agent nodes on HP 705 G3 machines i got for super cheap.
I've done the same thing with 3 HP Elite Desk mini pc. However, I created a Proxmox HA cluster then I created 6 VM's for k3s. 3 master nodes, and 3 worker nodes.
Did you put all 6 VMs on one node? Or did you split master and worker onto different nodes?
@@cryptolobo2273 I split them into different nodes.
can you provide the process of programming the bios on the PCs since i have one sitting at my home with same problem
Felt a long time since I watched your channel 😅
Great video Colton!!
Yeah sorry haha. That one took a while
Hello, I've just made a corosync and pacemaker cluster between two debian 12 machines and I want to install CasaOS on the cluster so the load is balanced between the machines. Is it possible to install CasaOS on the cluster and if it is, how?
do you know if its also possible to do the same but with docker installed on the pcs ? so the container has linux
Cool beans...I really need to do this as well..pretty darn cool!
Keep em coming!!!!
Thanks! Will do!
Love that intro. Came from another video; subscribed ✔️
Well, I'm currently training for the RHCE and the whole thing is about ansible. This could be an application for my elitedesk minis once I pass the exam.
Hi, what about benchmarking all your CPUs and do a load test on them so they all receive the same load so you can use all the CPU at one time
great viedo even though it is not a tutorial it is fun to watch
If you really wanted to setup a high avelebility minecraft server you would use something like multipaper with multiple waterfall proxies with a load balancer plugin and redisbungee and a redis cluster.
Hello, could these pc run jellyfin in docker? I mean with gpu pass and hw encode/decode? or better a solution with the i5 8th?
Fantastic - great video, enjoyed the video
Can you share which speakers you have on your desk?
Presonus Sceptre S6
Jeff geerling make his ansible book free. He did a video about it with the link.
I've been using a computer like those as mi personal local server, and I can say that those devices are immortal, and consumes very little power compared to the most of PCs.
Thanks for the video!
what do you use it at home for?
Every time you say "mc" my mind adds "hammer" and I have to suppress a chuckle to not start seeming even more nuts to myself. Hammertime! diidiidiidi-didi-didi.
I wish there was a tutorial for rock cluster
Seems way above my level, but worth looking into!
It disappeared from Distrowatch. Very strange.
just wanna say that rook is probably not the best choice for most use cases. It's pretty slow and pretty dang bad on 1GiB/s. I'd much rather use drbd with the piraeus operator that does reads locally. So you'll get full NVMe speeds on reads and really fast writes thanks to drbd9. It's also very fault tolerant and survives a node drain unlike Longhorn...
Its a crime what you don't have more subscribers
I still prefer using Proxmox or Incus as base system, running k8s in VMs. You can get a lot more value from servers than just running fancy k8s distros.
Hey, I'd like to do a K3S HA cluster between different NATs (mine and my grandparent's house), since the most risky part is the internet connection. I'm considering using Tailscale for the communication between the nodes and then have 1 node run my app at all times, and when that starts it tells cloudflare to set the A record to my current IP. Is this a good or flawed plan?
Let me know if you get something like this going.
And it worked? I want to do the same setup
@@Disturbed_face Have been putting it of, but be sure to report your findings. Do let me know in this thread
I am quite impressed with your intro
Your content is so awesome!!
Keep it up!!
Don't forget us when you get even more famous!!
Thanks! And no way haha
This is all over my head but I enjoy the video and maybe I can learn something thanks.
Hey,
I get an error:
Task [K3s/POST: Apply Metallb CRS] ***
FAILED - Retrying: [192.168.1.45]: Apply Metallb CRS (5 Retris Left).
what could be the problem?
This might help: github.com/techno-tim/k3s-ansible/discussions/247
what's the power consumption on these?
22:15. There are timestamps on all my videos to try and make finding stuff easier just FYI. Hope it helps in the future!
Great tute for Kewbs, shame these pcs here in Australia (even broken) are around $120 each
Loved this video. I am always curious on how this equipment works. I have not done a deep dive into it at all. But the day I do, good bye wallet. I enjoyed the video and subbed immedietaly.
this shit should be plug'n'play by today, it should be a bios option like "be slave machine" and so on, you shouldnt even need an os on the machines that are not the master one
Best intro ever 😂😂 trust future me!
I try install rancher but error pulling image, any solution?
Error message?
@@HardwareHaven invalid-token-authorization-failed
Is there anything other than that?
@@HardwareHaven failed to copy: httpReadSeeker: failed open: server message: invalid_token: authorization failed
Id be careful with notion. Its freeware with a subscription additive, aka the most frequent business strategy to die and their stuff is not in plaintext so if they go down one day youre losing your brain maps and organization systems. There are opensource alternatives.
Great guide though why use Debian instead of Alpine? Anyways, looking forward to your next video!
Just because I'm familiar with it and wanted to limit the number of things that I could mess up lol
Bro I'm in dog stage of all this, just setup my homeserver and have to learn all this stuff. Got a retropie up and running tomoght, something like this is next
I'm an expert on Kubernetes, so if you have any questions, just ask me.
Also, trust me, 90% of home labs don't need Kubernetes, docker/docker compose/portainer/nomad are sufficient.
Nice video, very informative :)
i would've named them "lenonode-X" or "lenode-X" btw
This whole time I thought you and technoTim were the same person lol
Have you ever seen us in the same room?
@@HardwareHaven All kidding aside, you guys do look ridiculously similar. Or maybe its just me.
sir can use 3 pcs performance in one
Jeff have opensource his ansible for devops just saying
It is great resource.
What a nice day watching watching Haven video on morning
cool video
The Hardware Haven drought is over!
5:25 oooo look how bent that ssd is, yikes
i think the camera angle or lighting makes it look worse than it is
Yeah definitely looks a bit worse than it actually is
@@HardwareHaven cameras have a habit of doing that. They also like to make a little bit of dust look like 20 lbs of dirt.
NEW HARDWARE HAVEN VIDEO 😎😎😎
new video LET'S GOOOOOOOOOOOOO
Systems such as kubernetes and docker are already destroying processor performance, if you want a real cluster, you should use mosix or openmosix
Awesome
🔥
I also want to learn Kubernetes.
why are clusters so hard and contra intuitive to do?
Pretty sure my doctor prescribed me lenovonode the other day
you want regular desktop cases in terms of refurbs - more expand-ability and ease of use - refurb pc are value kings - no question - having a reg sized case will aloow you to add better networking and gpu options and stay away largely from vendor proprietary parts - you might use a bit more power - that is totally ok since you will probably have more better cpu and ram - biz wants cluster tech for HA but also since they can scale out easily - just add a node or two
No I don’t. At least not always. You might though! 👍🏻
@@HardwareHaven I am talking to users and smb sector here not you specifically - they don't want compact and micro pc generally for reasons
@@shephusted2714
Then why not specify that in your original comment?
I see it says edited, not exactly sure for what though, wasn't to correct the "aloow" instead of allow.
So what was the edit?
And why couldn't you clarify in the original or add it during the edit?
Doesn’t look like the main points changed at all, so I wouldn’t get too upset.
I get it, but you’re still making assumptions. You’re assuming that people prefer expandability and performance over size and efficiency. In this instance, I 100% prefer the 1L cases over something larger and I don’t really care for >1Gbps since I’m not transferring a ton of data. So buying bulkier machines would’ve made no sense. And I don’t think I’m a unicorn. I think you could easily have said, “I would prefer and probably get larger form factors for the expandability and higher clocked CPUs” and been 100% correct. Just stop making blanket statements based on your opinion/needs/use case.
I looked at the thumbnail i thought it was a comparison
I remeber NileRed when I look to this guy
I get that a lot.. haha