A Three Server HomeLab for less than $1,000!
ฝัง
- เผยแพร่เมื่อ 4 ต.ค. 2020
- Thanks to LEXAR for sponsoring this video! Check out the Lexar NM610 here: bit.ly/2craftnm610
Building your first homelab is always fraught with challenges. Do you build or buy? What platform? Will your spouse divorce you? If you've asked yourself any of these questions, this might be the video for you.
But first... What am I drinking???
Ft. George Brewing Vortex IPA. Just a classic NW IPA. Hop forward, lots of hop oils will leave you tasting nothing but IPA for the next hour. Some people don't like that... I see it as a plus, because I get to taste nothing but IPA for the next hour.
Links to items below may be affiliate links for which I may be compensated
Check out the Rosewill RSV-Z2700 on Amazon: amzn.to/33vlltl
Other parts from this build:
Machinist X79 uATX motherboard: ebay.to/34oJeBV ali.ski/W2Gvc8
Intel Xeon E5-2648L: ebay.to/2F1mxuZ
8GB DDR3-REG 1866MHz: ebay.to/2F1ekqx ali.ski/_KDx3
Sunbow 32GB m.2 SATA SSD: amzn.to/34noL0l ali.ski/YyZV_
Even better deal for Video - ATI HD 2400 LP card: ebay.to/30AAck7
PowerMan 350W Power Supply: ebay.to/3niUfNO
24-Pin PSU Extension: amzn.to/2HRIu0p
8-Pin PSU Extension: amzn.to/3cZSErB
10pk LiCB CR2032 PC Batteries: amzn.to/36AHeJs
Find the parts I recommend on my Amazon store: www.amazon.com/shop/craftcomp...
Follow me on Twitter @CraftComputing
Support me on Patreon and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long. / craftcomputing
Music:
Hot Swing by Kevin MacLeod
Link: incompetech.filmmusic.io/song...
No Good Layabout by Kevin MacLeod
Link: incompetech.filmmusic.io/song...
License: creativecommons.org/licenses/b... - วิทยาศาสตร์และเทคโนโลยี
The swearing while mounting shows this guy knows what he is doing.
sublockdown Yep. I believe that's standard procedure.
Pretty much a sign of qualification really
Part of the standard 3-step process. Every computer you TRULY work on involves at least 2 out of 3 of the following: blood, sweat, and tears. The swearing is just using the proper banter, so that you clearly can see someone is a pro. ;-)
@@matthewclarke8313 Chuckle; the only thing that made me cringe was not power on the box before putting it in; lord knows Murphy should have put the failed box on the bottom and then he'd be fighting to get it installed again
Yep. If your not swearing while installing cage nuts, your doing it wrong! :-)
Pro installation tip, take two of your least favorite screwdrivers and insert it through the Server Ear Eye Holes of your choice on both sides (e.g. bottom left and bottom right hole) of the rack to pin it in place through the appropriately aligned rack holes that will eventually have screws in them too. The screwdriver bears the weight while you fiddle with the screws. The largest diameter screwdriver that will fit through both sets of holes will glean you the least fiddling as it will achieve the best temporary alignment, they also act as handy levers for slight alignment adjustments pre tightening.
If you like this idea, bend the pair into hook shapes and they won't ever slide out until you work them out.
Nice tip! (I know...that's what she said). Seriously, that is a good tip to know. I'll try it on my next install.
Honestly I prefer just swearing profusely trying to balance a 2U appliance with one hand while bracing my forearm on the switch below it and hoping I don't cause anything to power cycle. XD
@@CyberdyneSystemsSkynet or tear your biceps
Even easier, just put a pair of rack screws in the ru below and hang the server off that. You'd be surprised how much wait it takes
@@dirkdeboer6569 this is how I have done it in the past but not with a cage nut rack; prethreads make that approach so much easier; almost impossible to get cage nuts in place after the server is mounted even loosely; my trick is to put a shelf in below the insert, inverted and let it hold the weight
I ordered the case, but it did not include free cat. Did I order the wrong one?
Romnipotent I think so
That was last month's promo
@@Jahades Damn and I ordered it just for the cat.
You have to select the "Cat-1e" checkbox for the free cat
Guess I can’t order the case now
I love this channel. Thanks so much! Brings back memories of when I worked for a small server system builder for 10 years. The number of cheap Chinese cases like this that we had to find work-arounds for to get all the cables connected and everything to fit was insane. I was so glad when they fully switched over to pure Supermicro, although even that came with its challenges when the customer wanted non-standard kit in the build.
This channel was an easy subscribe!
I have been doing homelab for over 4 years now, and I just want to say that you are my absolute favourite youtuber. I was in a bit of a homelab rut in the past couple of months, and you have reignited my love for networking. Thanks for the videos!
Don't lie, your case decision was solely based on the free cat.
You got me
Rosewill, such a great company. Cheap server chassis have saved my life a few times lol. I'm very tempted to follow this and make a Proxmox Cluster!
Great video, man. I love seeing builds like this where you're making due with low-cost hardware and not pretending like everyone needs a Threadripper.
Just ordered the parts. Man am I excited !!! youre great man! keep up all the awesome homelab/server vids! :))
back in the day (1980's) you could get plastic feet for MB mount holes that did not line up to any mount points in the case (you can probably still get them). This would solve your wobbly MB corner
still have half a gross of these for those oddball cases/betwixed form factor mb's (bought a gross back in 2001), especially helpful for those instances where 20/24 pin connectors are hanging out in space.
I need some of these for my server, as im running an xlATX board in a standard atx case and im using some packing foam to hold up the top
@@elikirkwood4580 I found their proper names: 10mm Flush Mount PCB Supports
Old school checking in, a 'standard' nylon mouse pad cut into 2 inch by 1/2 inch are the perfect height to place underneath motherboard and provide adequate insulation and dampening, that's what i've always used without failure.. just glue or place adhesive tape on back
21:17 *Dell R710... So both on-camera Jeff and editor Jeff are idiots this week.
I like my 710 with 5675s and only paid 160 for it
I picked up a R720xd for 350. so far I love it. its quiter that the R710 and I got 6 more drive bays for my data hoarding needs
If someone is looking for a cheap 1U server, R420s CAN be had pretty cheap and low power :)
I bought 2 used servers a hp dl360e g8 and a r610 for 320$ (total 56gb ddr3 48threads aprox. 2ghz)
So i guess i stole my R730 for $25 then...
Quick way to double the cost of all these servers replace all the 80mm fans with Noctua NF-A8s
how much do they cost?
@@JoaoSilva-gs5jb ~$16-$30
Magnetic-tipped screwdrivers.... are not a backup LMAO
Also, Rambo was helping LOL.
Tip on getting the significant other on board: point out that if you don't have a home lab, you'll be screwing around with VMs on the same system that your family photo backup is on.
And she'll say "But honey. Isn't that in the cloud?"
To overcome the airflow issue with ATX power supplies in 2U chassis, I used an SFX to ATX adapter and mounted an SFX power supply inside the chassis. This will leave about 1cm of space between the case wall and the fan for the power supply to breathe.
I considered that as well, but it wasn't worth nearly tripling the price of the power supply. These supplies aren't exactly going to be stressed, with 100/350W being used.
@@CraftComputing Maybe some DIY "speed holes" could be added to the PSU with a drill? Lol.
Also, don't think i don't see you avoiding my question about those cheap HGST drives...
Hahaha, caught them on an eBay BIN. I bought all they had left 🤫
I'm looking forward to videos about what you decide to use your brand new servers for. Homelab fascinates me, but always seems like too much hardware for my needs.
I'm looking forward to seeing your next video about these servers.
Really nice work! I see some Proxmox HA in your future! Can't wait to see what goes on them!
Keep looking at building my own NAS and that chassis is the exact one I keep looking at so good to actually see a build in 1 for once.
Thanks for the video
"It was cryptocurrency e-waste, babe! Oh - those? I'm...just...recycling... these old Xeons and chipsets."
See, once you have a full server stack like me, you can just say "Oh that one? It was at the bottom of the rack, that's why you didn't see it. I just moved it up a bit. Yeah, that's it!"
Venting for standard PSUs is why I prefer 3U cases. And room for full height cards. Totally in agreement when talking about fan noise.
This is why I have things shipped to work. Happy wife, happy life. And I get new toys.
OMG. Free cats are the best bonus item! Love you Rambo
Also would like to point out that proper prior planning would take into account what you'd need in a year from now. It's pretty easy to build something that doesn't support IPMI or SR-IOV and regret it later.
Well, it wasn't 3 servers without virtualization, but a Threadripper 1920x for $199 and a $239 ASROCK Phantom Gaming 6 motherboard using both ethernet ports and Wifi would be sub $1000 too. ;) I know, not really fair, but it's another approach and would allow for greater performance overall. Still, I can see the benefit to multiple machines when really building a network "lab", so well done and nicely executed.
Nice, that's almost exactly what the 2U case for my first homelab looks like. I used it as a router for a long time with Untangle and then as my Peppermint Plex Server
Nice video!! I would like to see an option to have an add-on card or something to have IPMI interface for remote management of the server for a complete remote management
For anyone complaining about the PSU. When you draw very little power like these servers, it's a non-issue. Generic PSUs are decent at running such low loads. They "blow up" once you put a overclocked CPU or a gaming GPU on them.
Bingo.
Here's a tip for hanging servers onto rails to avoid the trouble you were having at 18:50
Get 2x 2-inch long M6 bolts and cut the heads off, or better still, just buy some M6 threaded studding. With a hacksaw, cut a deep slot to fit a flat screwdriver in the end. You've just made 2 locator studs.
Screw these into the upper holes/cage nuts. Then offer up the server/switch onto the studs, which then hold the weight while you screw in the two lower permanent rack machine screws. Once they are in, remove the slotted studs one at a time and replace them with the rack screws.
I used to carry 2 of these locator studs in my belt pouch and used a coin to screw them in and out.
My buddy just skipped the slot and used mole grips to insert and remove the studs (and also to open the beer)
Oh, I really shouldn't watch this... The wallet is in trouble.
You can also get ATX powersupplies with an 80mm fan pulling air through the powersupply front to back so that would remove the need for top ventilation in this case.
Difficult, while avoiding total junk and things that used to be good once but it was very nearly 20 years ago.
@Craft Computing You should 3D print some 1.75"x1.75"x1.75" blocks to use as supports when mounting something with a 1U gap between it and the thing below it. Stack them as needed for multiple U spacing.
I got that exact rosewill case for my 2u server, works really well
A NUC based home lab would make a lot more sense for most, and as Wife Approval Rating seems to be a factor, a far easier to sell.
between energy efficiency, space efficiency, heat output, noise, and flexibility of placement, they make a lot more sense. Plus companies are putting out multi-NIC solutions with 3rd party mods and 3rd party NUC-Alts like the Asus PN50.
no.1 issue - convincing my significant other i need a homeLab - subscribed
You almost made me cry trying to screw stuff in :) It's always happening when someone is watching ... :)
I felt the dropping screws anger in my soul, one server tech to another.
I've never felt sadness till i'm working on the back of a 4 post and the screws fall into the abyss
Thats why I go in the bullpen and grab a helpdesk guy. I get a set of hands, they get off the phones and maybe learn something. Everyone wins.
Thanks. Realizing I need to build a homelab for my homelab myself
Would love to see some virtual network set up in your new proxmox test environment. Maybe also setting up an ansible server to handle the maintenance of these test vms! (Keeping them up to date, easier software setup) I've seen some demos for ansible but most never get past "your first playbook"
My first true server was an IBM PS/2 9595A with a type Y complex. It was OLD and slow, but it worked. It had a 100mhz Pentium and several 10gb SCSI-2 drives. This was back in ~2005, so that was a fair bit of space. I actually hosted my website off that server for almost 10 years.
Today I have a Dell PowerEdge T320 with a 2TB hard drive and a 64GB SSD. I picked up a PowerEdge T420 that should arrive in just days along with 2 Xeon E5-2470V2 chips. I cant wait! Next month I pick up some drives and finally have a decent network storage option.
I'm surprised how much you know about server hardware and virtualization and all that. You look really young but you speak like you've been doing this since the dawn of time. I remember you saying you did this for work for the last ~14 years which is also amazing. Hope TH-cam proves to be very fruitful for you
Man! I just loved when you showed your cat. Very interesting project. You have my suscription. Cheers from México.
cg on new lab, please make a tuto on camera surveillance, blue iris etc
If I hadn't come up with a homelab plan literally 2 hours before I saw this video, I'd be looking at this in desktop form. Nonetheless, it's good to learn I don't have to pay an arm and a leg for hardware.
Glad to see I'm not the only adult running an MC server for friends and family 😅
Ebay is the gold market for second hand enterprise equipment 👌🏼
Hah! Did enjoy the bonus cat.
Found your channel through the LTT video and been really enjoying your videos! New subscriber :)
Instead of mounting ATX psu inside 2U, I recommend SFX one with ATX to SFX bracket (sometimes included with PSU's). PSU will have some clearance for air and you'll have free space under it allowing you to fit an full ATX board.
18:43 sorry to laugh at your pain. But dropping those screws killed me!
I guess it's just that the same sort of thing happens to me. Lol thanks
I'll never forget my data center admin days mounting a top of rack Cisco switch with one hand praying I don't fumble it before mounting a screw on each ear. Enterprise Cisco switches are very expensive, as many know. I loved working in the data center.
I worked with some $25,000 1U Aruba switches. I totally understand 😁
Got a good laugh watching you mount that server. Now I know what I looked like dropping screws while mounting mine.
Well Jeff, you asked for video requests so here are some things I'd like to see you do with your new servers:
1. Setup centralized authentication with Port389 LDAP and MIT Kerberos (Active Directory is cool, but the cost of Windows Server licensing is not.)
2. Having 3 servers puts you in a nice "bare minimum" situation to play with Gluster, and tuning it to get decent enough performance to mount your Linux ~/ might be a cool video.
3. IDK if you do any programming in your free time, but it could be cool to get some high throughput computing software (HTCondor as an example) set up on these machines so that you can run cluster jobs (using singularity containers to run kernel jobs, running some openai simulations, who the heck knows)
4. If you wanna try out some hacking, throw up a metasploitable VM and try breaking into it with Kali, or setup some vulnerability scanners like clair and openVAS and get a report about which services have vulnerabilities delivered to your email inbox.
Just throwing out some ideas, your content has definitely improved a lot over the past few years (not that it was ever terrible cringe) keep it up!
I just started using openvas. Amazing software. Luckily for me the only severe vulnerabilities are the honeypots I have.
Great job my fellow MB!
hey, jeff! Thank you for the really good manual. I have build a Home-Server due to your manual. But i have changed it a little bit. I put the maximum amound of RAM(64GB) because i need it for my programs. Also i have changed the Chassis to a mATX Tower because i have no rack. I have added an old Graphics Card that i had laying arround(Also an old ATI Model). And i have added an 1Gbit NIC because the Motherboard you recommended has only 100Mbit onboard NIC and i am using my server for NAS too, so i need the Bandwidth. Thanks a lot for this great manual. Have a nice day
Hello Kevin, Good Day. Could you please help me to find a refurbished/used server with up to : 2CPUs/ 64GB RAM / No OS with all other basic features . This is not for business,this is for education and training for a group of students in India. Please help with information useful to purchase . Thank you so much.
i cant wait to see the secret pc your working on with someone else :) not gunna spoil it. keep watching this space
Very interesting. So if just messing around and want something for a playground, or to get started - use X79 knockoffs. In your other video where you upgraded your production Promox, you mentioned, that you didn't want to risk using a cheap motherboard so you nabbed a Gigabyte board from eBay. Makes sense, if you just want to try out different features then having a bunch of NVM's around with these boards so that you can hop between projects, or at least that's another form of backup. ;] ESXi set, Proxmox set, XCP-NG set, and ??? set. Ooh, TrueNAS replication too. Now I have to decide if it'll be production with full WAF, or a playground. ;] Thanks Jeff!
I'd like to see HA virtualised pfsense VM 's in there to offer some network redundancy of your nodes.
Invest in some rails! I always look for rails for any servers I buy or build, even if they are shallow depth it makes a world of difference.
Hey Craft, I'd be very interested what power draw these boards idle at without the HDDs connected. Also if you downclock or undervolt or set the power profile to power save.
Ah, dropping screws off the old screwdriver brings back such memories. That's why I'll resort to the old trick of using scotch tape to temporarily stick the screw to the screw driver in awkward situations like this. Informative as always!
I have supercharged my screwdrivers. Most powerful magnet you can find (or a whole bag of nice neodymium ones), plonk it on the side, and slowly drag it up to the tip. Repeat several times. No more falling off screws, ever. But sometimes the screwdriver will just hop out of my hand and cling to a piece of steel :D Fortunately it's much harder to lose a screwdriver than a screw!
Should the tradeoff be undesired, blu-tac works.
Proxmox tends to work best in 3 node clusters. Ceph storage is also interesting to play with.
I swear when you said you'll have to figure the last one yourself in my head I said damn😂😂
Question
What about add rails to the cases? Any recommendations?
Your server rack is looking great 👍.
Hi, Jeff. Great video as always! Are you running the rack mostly for home-needs or also use it for work-related tasks (freelance projects aside from TH-cam)? What about some Docker stuff (educational tutorials as an idea for your next video) ;)
cat: if I fits, I sits.
Just started watching your channel and now I'm hooked... Sam and Max music ;-)
This is great. Can you share the monitor and keyboard configuration you have at the ending of the video. I have been looking for something simple and not too much.
Did I miss something, or is there still zero air flow in to the PSU on the bottom two servers?
I believe there were plastic spacers to populate holes in the mobo if there was no standoff in the case. They don't secure to the case but they will support the mobo.
Would love to you install and set up PFsence and set up Vlans for a home network..(cause I might be working on setting that up at the moment......)
I lost my lab in a flood but I used Sun Fire X2200s. Everyone was dumping Sun gear when they were bought by Oracle so I picked up 3 dual quad core optron for 100 dollars each and spent another 100 bucks buying the ram to boost them all to 32gb of ram. 100 bucks for the keyboard, mouse, monitor that folded down into the rack. The rack, KVM, switch and Netapp NAS were throwaways I got for free.
Loved the free cat. I want it too!
Great video cannot wait to see the collab with ltt
Jeff is a very talent and fun to watch
As a fellow tech as soon as you were racking the servers I said let's see many times he drops the screws. Every tech's nightmare.
And that children is why we use rack mount kits instead of juggle and hope for the best. 😆
Excellent work. Using custom hardware is king. I would have chosen used small servers like the dell R210, at cheap prices, minimum power consumption, and easy assembly, but that is me 😂.
The r210 will only handle processors with what max 4 cores? I don’t see how that would make as good of a low powered virtualization platform.
Right around 'cat depth' ... made my day :)
Hi Jeff, great video! I followed along and have one small question. Do the Super Micro heatsinks require an M4 standoff in the mobo? The ones I bought look very similar with the same mounting brackets. I'm just trying to figure out how the active coolers attach to the board and it's the only issue holding up my home lab. Thank you so much!
nice video. Any plans to add IPMI capabilities to these servers? I love to see that.
What thermal paste did you use with those cpu fans?
Security Onion Install! Great content!
Seeing as you already have a pihole server you should look into adguard home, unbound, dnscrypt, knot resolver, and technitium. Some ofnthese services accomplish one or more of the same task but there is very little info on which do them better.
my problem is what about the issue around the psu cooling? how did you manage to solve that
Jokes on you i don't have girlfriend..
Recently I built my first "server".
I got cheap x99 ud4 mobo - 2 of the pins are missing but all memory and pcie slots are working. - 20€ (I got it localy)
128gb ECC Reg 2133mhz cl16 - 150€ (I got it localy)
E5 2620 v3 - 13€ (from Aliexpress)
Artic Freezer 34 - 27€ (New)
PSU beQuiet 400W 80+ Bronze - 15€ (I got it localy)
And some old case I got laying around (airflow is good enough)
Also one thing I found this mobo (Gigabyte x99 ud4) is that it supports headless mode.
I've used HP's sff desktops for similar usage. Only 4c8t on similar aged hardware tho, but really cheap and no building required.
Also: wouldn't it make sense to mount your chassis wifh about 1/3 U space for the PSUs?
Also very low power draw. I've got an 8300 SFF that I've been using, but upgrading to 2 Ryzen 2700X's in rack for serious play.
In this type of rack setup the psu fan is not blocked by the case of the server above?
I’m using a pi as I thought that a regular server would be a power guzzler. Only 100W max? That’s insane! Once I move into my own home, I’m in an appartment right now, I’ll be moving to traditional servers with the same Ubuntu OS I’m currently running
hehe I use books to hold a server steady while screwing it in :) Amazing video dude :D
Damn nice! The server that im gonna do will be based on B450 and will have a Ryzen 7 1700, as the cores are a bit more powerful and can be useful for my requirements, will need to run CCTV, Home domain, minecraft server hosting and family storage server
Uses can be setting one up as a Linux server for a password manager/NAS/Windows VM machine/code compiler. It's in my project todo list.
Great video
I’d like to see you install truenas on top of proxmox. Connect it to network storage, and failover from one proxmox server to the other. I’d also like to see a kubernetes or a docker swarm demo with the multiple hosts.
Great minds think alike :)
Maybe connect a disk shelf. Truenas on proxmox cluster for HA
@Craft Computing What brand are the 1U snap in cover plates you are using? Those are nice.
One Tipp: You can mount a regular racktray upside down, in the slot below whatever you want to install. You can then place your device you want to install on top of that tray, allowing you to screw it in without having to hold up the system. After screwing it in, you can then remove the tray. Just a lifehack from an audio guy.
Hi Jeff! nice builds you got there! i'm currently building my own proxmox cluster with an MSA(fast storage server) as a backend storage for the data. Since you know probably a lot from proxmox, i would like to see how i can utilize my nvidia quadro 600 in my plex VM (i know that's not redundant) but also how redundancy works just by unplugging "power / network"-failure and somethings about recovery (like; which virtual harddisk format do you use, i like VMDK for example because it's widely supported) and backup solutions. In other words; show us how your vm or vm's will never be down! ;) you probably thought of this yourself but i'd mentioned it anyway. Thanks!
So how will the PSU's breathe? I had hoped you would use the blanking plates as spacers between the servers, also knowing they are 14$ a piece and no ventilation they seem like a fire hazard.
was my tought!!!
Yeah, that's what I was waiting for, how he manages the cooling of the PSUs. At least spacing them out in the rack. But I would rather find a PSU which only uses 80mm fan at the back.
I'm cooling the PSUs by only drawing 100W.
@@CraftComputing Fair enough, but I would still make sure it's not getting hot by monitoring it with a temperature probe for a few days.
Take the tops off the power supplies, the case has fans at the front so there is air flow in the case. Simple.
Can i ask what your doing with the pretty sangoma box? Would love to know, love the video’s thx