My first cheap crimp Patch Panel. Spent so much time debugging and rewiring only to realize that the problem wasn't my crimping, but the patch panel itself 🤦🏻♂ I've upgraded to a Keystone one and it's been smooth sailing since.
The other comment complaining about this video disappeared - but I wanted to tell that person that the occasional hardware video is appreciated as well. Not all of us are pure software fans, so there is an audience for content of this kind.
I'd like to second your comment as i commented below the now deleted comment. I appreciate both, specific homelabbing content dealing with software and services and more general content like this video.
You can't have the software without hardware. Anyone who develops software will be able to write more reliable and efficient code when they understand hardware and networking.
Tim’s channel is primarily a Homelab channel. A huge chunk of this hobby is hardware. If you’re just interested in software solutions, I’d say you are not into Homelabbing: you’re into whatever software solution you’re trying to achieve. And you know what? That’s perfectly fine, but it’s only part of Tim’s (and most of us’) hobby (and channel). ¯\_(ツ)_/¯
Actually with the new firmware Version on the UDM Pro you can assign both SFP+ Ports to be LAN or WAN - I am so happy they finally added that feature !!
Yeah, the new feature is really handy. In the EA firmware it’s even better. But you have to be weary of how the ports are wired. To get from any of the ports except for between the gigabit switch ports you have to go through the CPU so I’d probably skip using them if you want high speed networking across the two 10G ports - they aren’t a switch.
The network setup is nice. MicroTik is my go-to because it packs every enterprise tool you could need for a home or small business into an extremely reliable appliance.
And at 5:55 we can see 50% of the available memory bandwidth not being used. Move the two modules in the black slots over to the two blue slots on the other side of the socket. Then you will use all 4 available memory channels instead of just 2. I myself have recently gotten into setting up my own home network/computing lab for better or worse... Currently though just have one 2U server and no rack for it yet. But one has to start somewhere...
@@TechnoTim Information regarding channel layout tends to exist in the motherboard/system manual. Usually alongside a "recommended population order." Some motherboards also print: A1, A2, B1, B2, C1, C2, D1, D2. Or A1, B1, A2, B2, A3, etc. Or other fancy numbering systems. But if the naming/numbering doesn't make much sense, then it is best to check the manual. Though, then there is some cheap 2011(-3) boards that only uses 2 out of the four channels regardless, despite often have 4 memory slots...
To be honest, I rarely regret anything in life. I think, when you make a choice, you make it based on what you know, the resources you have, and what IS AVAILABLE at the time (any choice in life), so you don't really have much of a choice, because 100 times out 100 times, you would choose 'that option' FOR THE 1st TIME, based on that specific situation. We learn from our mistakes!
And I think learning from these mistakes is also good to know what not to buy in the future. As long as you don't waste money, but make a rational decision to begin with, you either pay for a good product or pay for a lesson.
Appreciate you giving your thoughts on the equipment decisions you've made over time. For me, and I know I'm not alone in this, I like to skip straight to the "perfect setup" to avoid some of the regrets and pain points. That said, these kinds of decisions are so subjective because everyone's needs, goals, and resources are different. At the end of the day, the things we look back on as mistakes are often the most valuable aspects of the learning process. Experience is the greatest teacher, so while it is wise to learn from other's mistakes, we shouldn't look to avoid the growth that comes from making our own. Congrats on the channel growth, and thank you for bringing us along with you on your journey.
2 comforting thoughts for me are that 1) everything is always changing: your needs, your knowledge, (software and hardware) products available in the market, energy prices and the time you have available for this. And 2) bad choices you make in only software (so not hardware, which is by its nature static) will have to be “fixed” with you spending time, perhaps your most valuable resource. Therefore it is best to not always want it to be perfect (because, in terms of continuity, that is impossible) but rather see it as a piece of art at which you can chisel away, learning stuff along the way.
A big issue with "perfect setup" from my current journey, is that the time it takes to afford and get everything technology is already passing you by and you are losing that ability to experience home labbing. I am a victim of this and i wish i could just dive into home labbing. For me its the networking side that is going to cost the most while the initial case for storage is my other hurdle everything else i can pick and chose as needed
With the UDM Pro you are able to change the 10G wan to a Lan port. Click on that port, and it will bring you to a screen where you can change it to a LAN port. It works in 1.12.22
Cool to see your rack, but it would be interesting to have seen the price you paid then versus now as a part of the "keep" or "upgrade". Cost and Energy Usage are two factors that add up over time.
Most of my “upgrades” were due to power consumption and I paid retail for most things which hasn’t changed. I have them all listed in the gear recommendations link!
I remember you recently made a NUT video about being able to safely power down all your device connected to the UPS, but I saw in this video you said you only have 10 min of runtime before you had to hurry and power down everything. Are some of your servers/components not connected through USB/etc. to NUT so they power down automatically?
I just installed a pass-thru patch panel like you and it is working just fine. I had Unifi network gear and was great for about 8 months and then nothing but grief and after 2 months of no joy, removed the Unifi and put back my Cisco SG200/300 switches and everything is stable. Two of my friends who also bought Unifi had some similar and other issues that I did not have. We are all quite well versed with networking. Glad that Unifi is working for you, but I could never recommend the Unifi line.
I prefer to buy older end of life data center switches and network equipment. I bought a Juniper 48 port switch for $50, and all 48 ports are PoE+ 1gig ports, plus the unit has two 10gig SFP+ port. I did the same thing for my UPS, I bought a used APC smt2200rm2u for $200 and it came with brand new batteries installed. I prefer my UPS to be compatible with APCUPSD so that all my equipment automatically shuts down when the UPS battery gets down to a certain level that I can set for each device during a power outage. I'm even thinking about buying another end of life switch right now too, as I can get a Celestica Seastone DX010 32-Port 100G QSFP28 switch for less than $500 on eBay right now! I have NO idea what I would use 100gig networking for, but each port can also be broken out to two 50gig ports, or four 25gig ports, so I could probably make use of it somehow!
@@vincentnthomas1 slightly, yes. But not as much as you'd think. If you compare new switches with as many ports and the same features (such as poe), the used older switches only use a tiny bit more power.
This small touch screen looks very easy to use, your explanation is very attentive, very careful, and very friendly to beginners, MicroTik is my go-to！
I like your patch panel, but I as of yet do not have any experience crimping RJ-45s . Some day I plan on getting a crimper and trying but is going to have to wait right now.
I have a tip! if you need more rack space, extend the rack to the max length and put stuff on the back side. things like patch panel or PDU or shorter switches can use the same U front/back
I do have same thought couple times a month. Should i start all over again with my Proxmox Server and all VMs. A lot of things i would do in different way and more optimised. But then i realise that current setup is already a refresh i did 10 - 12 months ago when i had same thought to start from 0 and do better. Any setup is a great setup as long as you can easily find problems and fix them.
Worth noting you can also run multiple battery backups. With multiple battery backups you can separate items by importance. My main network runs on my bigger backup and my less important stuff runs on the smaller unit. I do split my redundant power supplies on my servers between both backups though.
Nice video as always! If you do decide to upgrade your rack for a few more slots you have probably seen the StarTech 25U rack is real popular in the homelab community and isn't horribly expensive. I have been using it for over a year and it's been great.
6:00 there are some cool thunderbolt cards and even ones that use a riser card to grab more than on pcie slot and then pipe the 40Gb+ of each slot out a thunderbolt connection to what ever enclosure you want. Its a meathod I employ in my 1U to add 10G and a HBA card that connects my disk array.
Do you have a link to these Thunderbolt Cards? Particularly the ones that use riser cards to combine output from multiple PCI-E slots.... If that is what they do. Also, I assume this is on supported platforms only? (By this I mean, "Can you do this on older non-TB3 Platforms like AMD AM3 for example?")
@@longnamedude3947 I think so. There are some more main stream enclosures that are used to add expansions to macs. The card can be a little harder but ill edit this comment with one when I find it.
Curious what you would choose for your disk shelf now. Go the 45drive route? I kind of want to buy one of their cases and backplane to build my own but i doubt that is affordable in disk shelf terms.
I have a patch panel similar to that where I use keystone couplers and I don't regret it one bit. If I was running something more than gig I might care, but it's so convenient. you can even swap out those for other keystones like USB or display if you want.
Hi Tim - On early access for Unifi Dream Machine Pro, you can re-assign WAN-2 SFP to LAN. Follow Port Configuration > Configure Interface > Assign to Lan
For those that dont need huge amount of drives I use the EMC KTN-STL3 and it idles with no drives at 30w. Each drive adds around 7watt either sas or sata.
I personally regret not getting a better UPS that's actually rack based. Using the keyholes to wall mount a UPS probably isn't the best plan. Any rack mount UPS recommendations? Is Li-Ion worth it?
I have decided to go TP-Link Omada. The functionality for the most part is very comparable (central management) and the WiFi APs as well as managed/POE+ switching is far cheaper. I am just not throwing caution to the wind and buying the best-of-the-best right out of the gate, as Omada is fairly new when compared to Unify and TP-Link are adding new hardware all of the time, so I do plan on upgrades in the future as I cycle older switching components out of the rack as I buy new Omada stuff. I have limited myself to a 22U rack and will NOT upgrade it.
@@antoarre not that I’m aware. Tim has said soon for sharing that info but when will that be I can’t say. I’m hoping the next video or the video after that.
@@josephmyers5956 Thanks for the info. I'll be sure to double check the upcoming videos for it. I have been torn getting a disk shelf, but knowing the power draw, I'm interested in alternatives. Thanks again.
Really nice to see a hardware video once in a while! The more I see your homelab content, the more I wish I bit the bullet and went with unifi gear. Had an old xeon system though so I cobbled together an opnsense box and chucked it on a shelf with a regular pc case. Highly tempted to swap the cooling solution and get a rackmount case after seeing several of your videos showing your rackmount case for your system now.
I have several used DELL servers. My latest one is an R730 with 2x 16-core XEON CPUs (E5-2698 v3 @ 2.30GHz), 256G DDR4 ECC RAM., 2xSPF+, 2x 1G NIC, 8x 2.5" SAS Drive bays for about $1700. I purchased SSDs separately and don't remember how much those cost.
This makes me happy with the rack I bought, a 24U rack. I'm still working on filling it though as I have just started buying equipment for my rack. 24U feels like It'll be plenty for what I want to do, though if certain things take off (game servers)...I may need to add a second one.
I wish I hadn't run cat6a to my office for 10gig, those SFP+ RJ45 modules get really hot! Also I think you can reassign that 10gig port on your UDM Pro to LAN with a beta firmware.
I always go full size rack. The unused space can be used for shelfs. To put in unused stuff like laptops and cable. So i don´t have the decision between sizes.
Fun video Tim, looking forward to seeing the upgrades in the future. I really don’t have any regrets on building my lab, as I’ve learned so much from my mishaps and the money spent it’s a wash for me.
Hi Tim; I'm wondering why you would ever need more than say 2 VMs on your home lab server. I do like the choices you made for your equipment; especially units which have blinking or rolling lites. Give the whole rack unit some bling. I'm currently using the synology 918+ which fits my needs fine, however I did want to install a VM so I could run a version of windows outside my regular mac OS. I couldn't seem to git it configured completely and maybe it might be due to the location I'm currently living in.
Hey Tim if your UPS unit ever needs batteries I can confirm that LiFePO4 replacements are fantastic. I recently had a 5-hr power outage and my 900W APC (2x 12v 18Ah batteries) lasted nearly 30 min with a constant 320W load. There's a price premium for the pre-made batteries (as opposed to DIY) but they work so much better and they'll last so much longer than SLAs. 😃
Love your presentations. FYI, generally there is a reason they don't have half height panels. If you stack 24 port panel, 48 port switch, 24 port panel, then the top 24 ports fit perfectly with 6" patch cables going up and the bottom 24 fit going down. This is of course with replaceable jacks, which are necessary because a cat 6a+ copper cable is very thick, like 3/8". It's very difficult to punch those down while maintaining high speed network tolerances in such tight quarters. In a home lab though, you might want to direct cable three+ Proxmox servers to each other in a loop with dual ten gigabit network ports per server then route everything between them without using a switch at all. You can still have the low speed built in networking go to the Internet switch, but that can be a typical pfsense low cost low port mini micro machine. :)
I saw a server that was 2U maybe 3U, It’s split into four quadrants on the back and each quadrant can be pulled out and have a complete server built into it and then it slides in. So in theory you can run for servers in two units of rack space. I think if I were to start over again I would’ve saved some money on my ubiquity gear by buying the dream machine pro SE over the dream machine pro, and I would have started with the 24port enterprise switch instead of starting with the 24 port switch than the 24 port pro switch then moving to the 24port enterprise switch. The latter of which gives me POE and a handful of 2.5 Gb ports as well. Currently my steam machine and my server or connected up with 2.5 Gb
I am currently thinking about getting a server for my lab. I'll get a few 16TB Wxynos drives for cheap in a few months and am not sure if I should get an old 2x xeon dl380 gen8 with 128 or 256 gb of ram or if I should invest in a 128 gb build with new parts because that'll save me some power since the newer parts are more efficient. Another plus would be that I could easier use a custom cooling solution since noise would be a problem... decisions decisions..
That Tripp Lite can be monitored via USB or Serial. Things like NUT in like pfsense or TrueNAS or plain Linux can then be used to trigger a safe shutdown of HW.
Hey Tim appreciate all the great content. On the 1u servers have you considered adding in a pcie card with a couple of intel optane drives and moving to RAID10 with dedupe enabled and using the optane drives for the dedupe vdev? It could be an affordable way to get more space. CraftComputing recently talked about this type of setup for his home gaming VM rig. I am in the process of setting up a similar setup. It would also be great to see a video on Technitium DNS as a potential pi-hole replacement. It has all the blocking features plus can replicate zones to other hosts for that HA DNS setup you have in place.
So what would you upgrade the disk shelf to? You made mention that it's oversized for your current needs and pulls a lot of power. I'm curious if that meant downgrading to something more relevant to the size you need and therefore more power friendly... Or if it meant going in some other direction.
Great video Tim . I am personally not a fan of the pass through keystone patch panel, but if you’re not use to terminating cables this is a good way to go . The UDM pro is will fully feature wireguard and OpenVPN in the future ( just need to wait a little more time ) 😊
@@Gunzy83 I love keystone patch panels that’s all I use. What is being used here is keystone pass through so you’re connecting ethernet cable to the back and front with rj45 ends.
I'm surprised you do a manual shutdown on power failure, wouldn't it be kind of easy to use something like Network UPS Tools to auto-shutdown on power failure?
Your SSD layout, can you back up the VMs, destroy the pool, and do a pool of mirrors? Raidz2 is great starting at 6 drives. If you're running 5 do 2 mirrored pairs and a hot spare. You'll get a TON more IO too. Love the videos. Labbing is all about lessons learned and you're crushing it.
BTW you can add that 10gig wan to a lan (Ports/Port Management/click on port 8 set as wan2 click on port 10 aka 10gbs wan set as lan) and done you have 10gbs lan ethernet ports
Thanks for the upload! Looking forward to more awesome educational videos. You are great at explaining stuff so it's easy to follow for even noobs like myself. Hoping to see a pfsense as a router and firewall video soon. It would be great to see a video from you that explains how to run all your homelab services but only using a simple PC with enough RAM and storage. Use, opensource software solutions where possible instead of purchasing expensive enterprise level switches or firewalls. Nothing against them but not all homelabbers can afford them.
You can actually use the 2nd WAN port now as a LAN port. You have to go to Network -> UniFI devices -> UDMP -> Port Management -> Click WAN2 and change. I am on Network version 7.2.91 and Device version 1.12.24
Great video Tim! Do you have thoughts on the time horizon for future-proofing, for when you might be trading off technological improvements over time? Like for your switch regrets/recommendation, if I think I will need more than 24 ports but maybe not for 5(?) years or so, is it better to buy the 48 port now that will fit my needs in 5 years, or buy 24 now and either another 24 or a 48 replacement in 5 years, when the tech has improved by 5 years, when maybe at that point all the ports are 10gig with some 100gig uplinks? What sort of time frame do you consider to be too far into the future where the future-proofing becomes compromising?
This is a tough call. What I learned is that adding capacity is relatively cheap if you do it up front but expanding it later can be expensive (You have to rebuy) Adopting new tech is also expensive. I try to balance this out. I wish there were a great answer to this but I just try to do the calculus when I buy. For instance, there is a 48 port switch is is PoE and also 2.5 Gbs. I seriously considered it but then looked at the price. It was a 1/3 more in price for a tech I probably won't use, especially if I am plugging in PoE devices which are typically 1Gbs at most. It's always a case by case basis and I will be sure to continue to share my findings in the future!
Interesting video. I enjoyed it. I was in economy mode when I built my rack. I went with a 20U version and I've not regretted it. I started with the Unifi SW 16 PoE and I would definitely do it differently if I had it to do over again. Given where I was back then, I would go with the 8 Enterprise and 24 Pro (non-PoE) at this point. That's $479 for the SW 8 Enterprise and $399 for the SW 24 Pro compared to $699 for the SW 24 Pro you bought ($179 upgrade). That'll give me 2.5G PoE ports for the U6 Enterprise APs if I ever decide to upgrade them. Since I didn't have that option available back then, I may end up just spending more and buying a SW 24 Enterprise though. I opted for the Switch Aggregation though for 8 10G ports and that has been super nice for only $279.
I really wish you could get those 1U SM servers because I have tried to no avail. I cant seem to find a comparable 1U single socket server that will accept any CPU past 4c8t.
I was waiting to hear the make and model of the PC case and the server rack while I was listening to ( not watching) the video, and they never came. Anyone know the make model of these items ?
instead of the supermicro 1U's, it sounds like a couple of used Dell R630's would have been a better fit for you. Dual socket 2011-3, 32 DIMM slots, tons of 2.5" hot swap at the front. They're pretty cheap, you might still think about getting them.
On the 1U super micro servers. They are awesome boxes - but that particular model maybe isn’t the best for a home lab. You’re only running SSDs so having full size HDD slots is a bit of a waste. SM has versions of the chassis with 8 (or 10) 2.5 inch hotswap bays. There is also a version of your same motherboard you use with enough SATA/SAS ports use all those hotswaps as well as 10Gbe on board (the “CTF” version of the same board). And you still have the single PCIe slot open. Maybe not something you’d need to replace but if making recommendations to people about to follow your journey this is definitely a place where I would suggest they spend a bit more up front.
Ok - so it looks like I brain faulted... The motherboard i was thinking of with 10gbe and lots of SATA is an lga3647 (xeon scaleable) model. X11spi-tf. Not a version of yours. Overall comment stands though. For labs following your example I'd still recommend spending a bit more and getting the 8x or 10x 2.5inch 1U chassis.
You kind of glanced over the noise of the 1U chassis - do you have them in a space where the noise isn’t an issue? Or are you able to spin down fans to reduce noise but keep noise down?
I just stumbled upon your channel and want to thank you! I'm planning to wire up my new home and have an old PC running as a Plex server (hoping that running cable will help the performance on this old hardware) but you've given me an end game goal! Any tips you can give for Plex server hardware, etc would be greatly appreciated!
i would raise eveything up to the empty spot take ups out add extra battery cables and add batterys in the now empy rack below this is what i intend to do with my server rack when i get my man cave
UniFi added support to switch wan ports to lan ports. On the port section at the bottom you should have a link for configure interfaces. You can change port 10 to lan. however I don’t think you an LAG it with port 11.
What bothers me the most is the LED strip: You can see the single light spots because they are so far apart from each other. In my rack, I've used a 6000K (cold white) strip with COB LEDs where the elements are much closer to each other - so it's like one long light source.
Couldn't you "just" automate the shutdown in case of a power outage? May be with a threshold percentage to brigde short power losses. I remember my old APM USP allowing this. Only had to boot up everything again manually.
Regarding the UPS. It should be possible for the servers to monitor the UPS input power state. Most UPS can give out this information. And when external power disapears the servers shutdown themself before the 10min runtime of the UPS runs out.
Thanks so much for this video. I'm looking for a completed inventory that can help me buy everything I need to build my own server. Can you share those infos?
For me I view Ubiquity as the Apple of networking. Which you may call a compliment but I don;t mean it like that. I mean it as its the locked down walled garden ecosystem that I cannot stand to use. I'm just getting started into homelabbing right now and haven't gotten any real equipment yet, I only have a basic prosumer grade router but I want to custom build a 1U box to run pfsense. And then I want to custom build a 4U server with the front covered in drive bays to install TruNas and run any docker containers and VMs directly on that too.
I recently dropped my server from the rack in favor of my AWS account. I can essentially do anything I want for the flavor of the month something something I am working on. And I can dictate exactly what I need.
I cant see spending $1100 on a 48 port POE switch when 4x+ 10gSPF+ and more bandwidth you could imagine in a lab from a retired enterprise switch like a ICX 6450 for less than $200. POE++ is the only thing I dont ever see on old switches but that is more for PTZ cameras anyway.
I will trade you my StarTech 25U rack for that 18U any day. Been wanting to size it down some, as I am not using all the space in it, and keep cutting out old and unneeded equipment.
Yoy may want to consider switch Mikrotik Routers, they support multiple VPN protocols such as L2TP/IPSec, IPSec IKEv1 and IKEv2, SSTP, OVPN, WireGuard and ZeroTier!. I would use a L3 switch for interVLAN routing without doing this at the router level!
I personally regret actually buying Dell servers and not sticking to consumer hardware like I should have and just getting some rack mount cases. Really the only nice thing is the redundant power supply and the IPMI interface. I however don't regret the purchase of a 42U rack, but I'm not even using hal;f of it. So TechnoTim, if you want. I'm willing to trade you racks. Mines a 42U startech open frame rack.
What's one thing you regret buying for your HomeLab??? 😭
Budget always not enough
I regret only this, I buy UDM pro and after 4 days later UDM SE has been announced :D
My first cheap crimp Patch Panel. Spent so much time debugging and rewiring only to realize that the problem wasn't my crimping, but the patch panel itself 🤦🏻♂ I've upgraded to a Keystone one and it's been smooth sailing since.
@@LegionInfanterie no open refund?
Not getting a bigger appartment so i could have space for a rack :*(. For now i got to love with tower servers
The other comment complaining about this video disappeared - but I wanted to tell that person that the occasional hardware video is appreciated as well. Not all of us are pure software fans, so there is an audience for content of this kind.
I'd like to second your comment as i commented below the now deleted comment. I appreciate both, specific homelabbing content dealing with software and services and more general content like this video.
You can't have the software without hardware. Anyone who develops software will be able to write more reliable and efficient code when they understand hardware and networking.
Tim’s channel is primarily a Homelab channel. A huge chunk of this hobby is hardware. If you’re just interested in software solutions, I’d say you are not into Homelabbing: you’re into whatever software solution you’re trying to achieve. And you know what? That’s perfectly fine, but it’s only part of Tim’s (and most of us’) hobby (and channel). ¯\_(ツ)_/¯
Thank you!
@@levifig 100% agree. Its not a homelab without a rack of your own hardware . And we like the content.
Actually with the new firmware Version on the UDM Pro you can assign both SFP+ Ports to be LAN or WAN - I am so happy they finally added that feature !!
Yeah, the new feature is really handy. In the EA firmware it’s even better. But you have to be weary of how the ports are wired. To get from any of the ports except for between the gigabit switch ports you have to go through the CPU so I’d probably skip using them if you want high speed networking across the two 10G ports - they aren’t a switch.
O0o0o0. I didn't know that. Must try it
The network setup is nice. MicroTik is my go-to because it packs every enterprise tool you could need for a home or small business into an extremely reliable appliance.
And at 5:55 we can see 50% of the available memory bandwidth not being used. Move the two modules in the black slots over to the two blue slots on the other side of the socket. Then you will use all 4 available memory channels instead of just 2.
I myself have recently gotten into setting up my own home network/computing lab for better or worse... Currently though just have one 2U server and no rack for it yet. But one has to start somewhere...
Thank you! Since then all slots are now filled but good to know!
@@TechnoTim Information regarding channel layout tends to exist in the motherboard/system manual. Usually alongside a "recommended population order."
Some motherboards also print: A1, A2, B1, B2, C1, C2, D1, D2. Or A1, B1, A2, B2, A3, etc. Or other fancy numbering systems. But if the naming/numbering doesn't make much sense, then it is best to check the manual.
Though, then there is some cheap 2011(-3) boards that only uses 2 out of the four channels regardless, despite often have 4 memory slots...
You know you can change the SFP+ 10gbe WAN point on the UDM Pro to a LAN port? It’s a setting in the GUI
If so, that must be a brand new feature, as I see no such option on mine, but I also haven't updated the firmware in a little while.
@@Blooest there’s a specific firmware update that enables it to be switched
To be honest, I rarely regret anything in life. I think, when you make a choice, you make it based on what you know, the resources you have, and what IS AVAILABLE at the time (any choice in life), so you don't really have much of a choice, because 100 times out 100 times, you would choose 'that option' FOR THE 1st TIME, based on that specific situation.
We learn from our mistakes!
And I think learning from these mistakes is also good to know what not to buy in the future. As long as you don't waste money, but make a rational decision to begin with, you either pay for a good product or pay for a lesson.
As of the newest Unifi Update, you can now reassign that WAN port to be a LAN port!
Now this is cool! Thanks for the info
Appreciate you giving your thoughts on the equipment decisions you've made over time. For me, and I know I'm not alone in this, I like to skip straight to the "perfect setup" to avoid some of the regrets and pain points. That said, these kinds of decisions are so subjective because everyone's needs, goals, and resources are different. At the end of the day, the things we look back on as mistakes are often the most valuable aspects of the learning process. Experience is the greatest teacher, so while it is wise to learn from other's mistakes, we shouldn't look to avoid the growth that comes from making our own.
Congrats on the channel growth, and thank you for bringing us along with you on your journey.
2 comforting thoughts for me are that 1) everything is always changing: your needs, your knowledge, (software and hardware) products available in the market, energy prices and the time you have available for this. And 2) bad choices you make in only software (so not hardware, which is by its nature static) will have to be “fixed” with you spending time, perhaps your most valuable resource. Therefore it is best to not always want it to be perfect (because, in terms of continuity, that is impossible) but rather see it as a piece of art at which you can chisel away, learning stuff along the way.
A big issue with "perfect setup" from my current journey, is that the time it takes to afford and get everything technology is already passing you by and you are losing that ability to experience home labbing. I am a victim of this and i wish i could just dive into home labbing. For me its the networking side that is going to cost the most while the initial case for storage is my other hurdle everything else i can pick and chose as needed
Thanks
Wow! Thank you so much!!!!
With the UDM Pro you are able to change the 10G wan to a Lan port. Click on that port, and it will bring you to a screen where you can change it to a LAN port. It works in 1.12.22
About your NetApp, look up the difference in power supplies on it, the newer PSU's draw much less power.
Cool to see your rack, but it would be interesting to have seen the price you paid then versus now as a part of the "keep" or "upgrade". Cost and Energy Usage are two factors that add up over time.
Most of my “upgrades” were due to power consumption and I paid retail for most things which hasn’t changed. I have them all listed in the gear recommendations link!
I remember you recently made a NUT video about being able to safely power down all your device connected to the UPS, but I saw in this video you said you only have 10 min of runtime before you had to hurry and power down everything. Are some of your servers/components not connected through USB/etc. to NUT so they power down automatically?
I just installed a pass-thru patch panel like you and it is working just fine. I had Unifi network gear and was great for about 8 months and then nothing but grief and after 2 months of no joy, removed the Unifi and put back my Cisco SG200/300 switches and everything is stable. Two of my friends who also bought Unifi had some similar and other issues that I did not have. We are all quite well versed with networking. Glad that Unifi is working for you, but I could never recommend the Unifi line.
I prefer to buy older end of life data center switches and network equipment. I bought a Juniper 48 port switch for $50, and all 48 ports are PoE+ 1gig ports, plus the unit has two 10gig SFP+ port. I did the same thing for my UPS, I bought a used APC smt2200rm2u for $200 and it came with brand new batteries installed. I prefer my UPS to be compatible with APCUPSD so that all my equipment automatically shuts down when the UPS battery gets down to a certain level that I can set for each device during a power outage. I'm even thinking about buying another end of life switch right now too, as I can get a Celestica Seastone DX010 32-Port 100G QSFP28 switch for less than $500 on eBay right now! I have NO idea what I would use 100gig networking for, but each port can also be broken out to two 50gig ports, or four 25gig ports, so I could probably make use of it somehow!
Wouldn’t it be less efficient in power?
@@vincentnthomas1 slightly, yes. But not as much as you'd think. If you compare new switches with as many ports and the same features (such as poe), the used older switches only use a tiny bit more power.
This small touch screen looks very easy to use, your explanation is very attentive, very careful, and very friendly to beginners, MicroTik is my go-to！
I like your patch panel, but I as of yet do not have any experience crimping RJ-45s . Some day I plan on getting a crimper and trying but is going to have to wait right now.
I have a tip! if you need more rack space, extend the rack to the max length and put stuff on the back side. things like patch panel or PDU or shorter switches can use the same U front/back
I do have same thought couple times a month. Should i start all over again with my Proxmox Server and all VMs. A lot of things i would do in different way and more optimised. But then i realise that current setup is already a refresh i did 10 - 12 months ago when i had same thought to start from 0 and do better.
Any setup is a great setup as long as you can easily find problems and fix them.
Worth noting you can also run multiple battery backups. With multiple battery backups you can separate items by importance. My main network runs on my bigger backup and my less important stuff runs on the smaller unit. I do split my redundant power supplies on my servers between both backups though.
Nice video as always! If you do decide to upgrade your rack for a few more slots you have probably seen the StarTech 25U rack is real popular in the homelab community and isn't horribly expensive. I have been using it for over a year and it's been great.
6:00 there are some cool thunderbolt cards and even ones that use a riser card to grab more than on pcie slot and then pipe the 40Gb+ of each slot out a thunderbolt connection to what ever enclosure you want. Its a meathod I employ in my 1U to add 10G and a HBA card that connects my disk array.
Do you have a link to these Thunderbolt Cards?
Particularly the ones that use riser cards to combine output from multiple PCI-E slots.... If that is what they do.
Also, I assume this is on supported platforms only?
(By this I mean, "Can you do this on older non-TB3 Platforms like AMD AM3 for example?")
@@longnamedude3947 I think so. There are some more main stream enclosures that are used to add expansions to macs. The card can be a little harder but ill edit this comment with one when I find it.
Curious what you would choose for your disk shelf now. Go the 45drive route? I kind of want to buy one of their cases and backplane to build my own but i doubt that is affordable in disk shelf terms.
Press F to pay respect to the old Dell R710 that became an awesome empty slot. Where did it go Tim?
I gave it a new good home, and sold it to a new, young homelabber willing to learn in the Mpls area!
I have a patch panel similar to that where I use keystone couplers and I don't regret it one bit. If I was running something more than gig I might care, but it's so convenient. you can even swap out those for other keystones like USB or display if you want.
Hi Tim - On early access for Unifi Dream Machine Pro, you can re-assign WAN-2 SFP to LAN. Follow Port Configuration > Configure Interface > Assign to Lan
For those that dont need huge amount of drives I use the EMC KTN-STL3 and it idles with no drives at 30w. Each drive adds around 7watt either sas or sata.
I personally regret not getting a better UPS that's actually rack based. Using the keyholes to wall mount a UPS probably isn't the best plan. Any rack mount UPS recommendations? Is Li-Ion worth it?
I’m wondering about this too
Here's the one I have! amzn.to/2XGN6yt
@@TechnoTim thank you. I have been looking at this one though I keep seeing people mention getting one from APC but they are definitely pricer.
I have decided to go TP-Link Omada. The functionality for the most part is very comparable (central management) and the WiFi APs as well as managed/POE+ switching is far cheaper. I am just not throwing caution to the wind and buying the best-of-the-best right out of the gate, as Omada is fairly new when compared to Unify and TP-Link are adding new hardware all of the time, so I do plan on upgrades in the future as I cycle older switching components out of the rack as I buy new Omada stuff. I have limited myself to a 22U rack and will NOT upgrade it.
What would you get instead of the disk shelf then to house your drives?
That is what I wondering too. I’m in the process of looking for a disk shelf so I’m really curious
Did Tim hint at what he would have replaced the shelf with?
@@antoarre not that I’m aware. Tim has said soon for sharing that info but when will that be I can’t say. I’m hoping the next video or the video after that.
@@josephmyers5956 Thanks for the info. I'll be sure to double check the upcoming videos for it. I have been torn getting a disk shelf, but knowing the power draw, I'm interested in alternatives. Thanks again.
@@antoarre No problem. I'm in the same boat too. I have been thinking of getting one too but would like to know what the alternative would be too.
Really nice to see a hardware video once in a while! The more I see your homelab content, the more I wish I bit the bullet and went with unifi gear. Had an old xeon system though so I cobbled together an opnsense box and chucked it on a shelf with a regular pc case. Highly tempted to swap the cooling solution and get a rackmount case after seeing several of your videos showing your rackmount case for your system now.
I have several used DELL servers. My latest one is an R730 with 2x 16-core XEON CPUs (E5-2698 v3 @ 2.30GHz), 256G DDR4 ECC RAM., 2xSPF+, 2x 1G NIC, 8x 2.5" SAS Drive bays for about $1700. I purchased SSDs separately and don't remember how much those cost.
This makes me happy with the rack I bought, a 24U rack. I'm still working on filling it though as I have just started buying equipment for my rack. 24U feels like It'll be plenty for what I want to do, though if certain things take off (game servers)...I may need to add a second one.
I wish I hadn't run cat6a to my office for 10gig, those SFP+ RJ45 modules get really hot!
Also I think you can reassign that 10gig port on your UDM Pro to LAN with a beta firmware.
It's in the released firmware now.
I always go full size rack. The unused space can be used for shelfs. To put in unused stuff like laptops and cable. So i don´t have the decision between sizes.
Fun video Tim, looking forward to seeing the upgrades in the future. I really don’t have any regrets on building my lab, as I’ve learned so much from my mishaps and the money spent it’s a wash for me.
Hi Tim; I'm wondering why you would ever need more than say 2 VMs on your home lab server. I do like the choices you made for your equipment; especially units which have blinking or rolling lites. Give the whole rack unit some bling. I'm currently using the synology 918+ which fits my needs fine, however I did want to install a VM so I could run a version of windows outside my regular mac OS. I couldn't seem to git it configured completely and maybe it might be due to the location I'm currently living in.
Hey Tim if your UPS unit ever needs batteries I can confirm that LiFePO4 replacements are fantastic. I recently had a 5-hr power outage and my 900W APC (2x 12v 18Ah batteries) lasted nearly 30 min with a constant 320W load. There's a price premium for the pre-made batteries (as opposed to DIY) but they work so much better and they'll last so much longer than SLAs. 😃
So you mentioned upgrade to the disk shelf. But with what? Another full fledge server?
Love your presentations. FYI, generally there is a reason they don't have half height panels.
If you stack 24 port panel, 48 port switch, 24 port panel, then the top 24 ports fit perfectly with 6" patch cables going up and the bottom 24 fit going down. This is of course with replaceable jacks, which are necessary because a cat 6a+ copper cable is very thick, like 3/8". It's very difficult to punch those down while maintaining high speed network tolerances in such tight quarters.
In a home lab though, you might want to direct cable three+ Proxmox servers to each other in a loop with dual ten gigabit network ports per server then route everything between them without using a switch at all. You can still have the low speed built in networking go to the Internet switch, but that can be a typical pfsense low cost low port mini micro machine. :)
I saw a server that was 2U maybe 3U, It’s split into four quadrants on the back and each quadrant can be pulled out and have a complete server built into it and then it slides in. So in theory you can run for servers in two units of rack space.
I think if I were to start over again I would’ve saved some money on my ubiquity gear by buying the dream machine pro SE over the dream machine pro, and I would have started with the 24port enterprise switch instead of starting with the 24 port switch than the 24 port pro switch then moving to the 24port enterprise switch. The latter of which gives me POE and a handful of 2.5 Gb ports as well. Currently my steam machine and my server or connected up with 2.5 Gb
I think on the latest unifi update you can now swap the 10gBe port to LAN now!
I am currently thinking about getting a server for my lab. I'll get a few 16TB Wxynos drives for cheap in a few months and am not sure if I should get an old 2x xeon dl380 gen8 with 128 or 256 gb of ram or if I should invest in a 128 gb build with new parts because that'll save me some power since the newer parts are more efficient. Another plus would be that I could easier use a custom cooling solution since noise would be a problem... decisions decisions..
That Tripp Lite can be monitored via USB or Serial. Things like NUT in like pfsense or TrueNAS or plain Linux can then be used to trigger a safe shutdown of HW.
Hey Tim appreciate all the great content. On the 1u servers have you considered adding in a pcie card with a couple of intel optane drives and moving to RAID10 with dedupe enabled and using the optane drives for the dedupe vdev? It could be an affordable way to get more space. CraftComputing recently talked about this type of setup for his home gaming VM rig. I am in the process of setting up a similar setup. It would also be great to see a video on Technitium DNS as a potential pi-hole replacement. It has all the blocking features plus can replicate zones to other hosts for that HA DNS setup you have in place.
So half height 1u patch panels. I've got two of those from Trendnet. But you have to use a punch down tool because it's too compact fire keystones.
Thanks for the tip!
So what would you upgrade the disk shelf to? You made mention that it's oversized for your current needs and pulls a lot of power. I'm curious if that meant downgrading to something more relevant to the size you need and therefore more power friendly... Or if it meant going in some other direction.
Soon!
What would you replace the disk shelf with? another server that just has a lot of slots?
Soon!
@@TechnoTim soon as in yes? or soon as in there is a video about this coming soon? :P
Great video Tim . I am personally not a fan of the pass through keystone patch panel, but if you’re not use to terminating cables this is a good way to go .
The UDM pro is will fully feature wireguard and OpenVPN in the future ( just need to wait a little more time ) 😊
Hey and thank you! Yeah, you're a pro at doing this! It would seriously take me 2 days and I would still have half of them that wouldn't work!
All of you always say you don't like the Keystone passthrough patch panels but never say why... Can you elaborate?
@@Gunzy83 I love keystone patch panels that’s all I use. What is being used here is keystone pass through so you’re connecting ethernet cable to the back and front with rj45 ends.
would have been nice to know what you paid for and what you got freebee.... and total cost if purchased.
I paid for 100% of everything in this rack. No freebies
I'm surprised you do a manual shutdown on power failure, wouldn't it be kind of easy to use something like Network UPS Tools to auto-shutdown on power failure?
The new Unifi Network allows you to reassign that port. Also port 8 can be added to the mix on UDM Pro.
Your SSD layout, can you back up the VMs, destroy the pool, and do a pool of mirrors? Raidz2 is great starting at 6 drives. If you're running 5 do 2 mirrored pairs and a hot spare. You'll get a TON more IO too. Love the videos. Labbing is all about lessons learned and you're crushing it.
Thank you! I mean to say that I have 2 mirrored pairs! Only 4 drives, kind of like radi1+0 I think! Thank you!
fwiw, 37U racks fit through a door even on casters. It made moving from my old apartment easier.
Great tip!
get solar pannels charge up the batteries and you have extra batteries :) for the whole house or just for the servers
I'm sure others have already commented, but recent UDMP OS release allows you to reassign port 10 to be a second sfp+ LAN port...
BTW you can add that 10gig wan to a lan (Ports/Port Management/click on port 8 set as wan2 click on port 10 aka 10gbs wan set as lan) and done you have 10gbs lan ethernet ports
Thanks for the upload! Looking forward to more awesome educational videos. You are great at explaining stuff so it's easy to follow for even noobs like myself. Hoping to see a pfsense as a router and firewall video soon. It would be great to see a video from you that explains how to run all your homelab services but only using a simple PC with enough RAM and storage. Use, opensource software solutions where possible instead of purchasing expensive enterprise level switches or firewalls. Nothing against them but not all homelabbers can afford them.
You can actually use the 2nd WAN port now as a LAN port. You have to go to Network -> UniFI devices -> UDMP -> Port Management -> Click WAN2 and change. I am on Network version 7.2.91 and Device version 1.12.24
Thank y ou!
Great video Tim! Do you have thoughts on the time horizon for future-proofing, for when you might be trading off technological improvements over time? Like for your switch regrets/recommendation, if I think I will need more than 24 ports but maybe not for 5(?) years or so, is it better to buy the 48 port now that will fit my needs in 5 years, or buy 24 now and either another 24 or a 48 replacement in 5 years, when the tech has improved by 5 years, when maybe at that point all the ports are 10gig with some 100gig uplinks? What sort of time frame do you consider to be too far into the future where the future-proofing becomes compromising?
This is a tough call. What I learned is that adding capacity is relatively cheap if you do it up front but expanding it later can be expensive (You have to rebuy) Adopting new tech is also expensive. I try to balance this out. I wish there were a great answer to this but I just try to do the calculus when I buy. For instance, there is a 48 port switch is is PoE and also 2.5 Gbs. I seriously considered it but then looked at the price. It was a 1/3 more in price for a tech I probably won't use, especially if I am plugging in PoE devices which are typically 1Gbs at most. It's always a case by case basis and I will be sure to continue to share my findings in the future!
There is still place in that rack .. you could use the Front and the back. Switches and Patchpanels dont need to be at the front.
What wattage are we talking about Tim?
Interesting video. I enjoyed it. I was in economy mode when I built my rack. I went with a 20U version and I've not regretted it. I started with the Unifi SW 16 PoE and I would definitely do it differently if I had it to do over again. Given where I was back then, I would go with the 8 Enterprise and 24 Pro (non-PoE) at this point. That's $479 for the SW 8 Enterprise and $399 for the SW 24 Pro compared to $699 for the SW 24 Pro you bought ($179 upgrade). That'll give me 2.5G PoE ports for the U6 Enterprise APs if I ever decide to upgrade them. Since I didn't have that option available back then, I may end up just spending more and buying a SW 24 Enterprise though. I opted for the Switch Aggregation though for 8 10G ports and that has been super nice for only $279.
with the latest firmware for the udm pro you can erasigne the outher wan port as a lan port
Looking at previous posts, do you still use your Sophos XG firewall or did you completely go with the UDM Pro firewall?
Nope, check out my latest tour!!
I really wish you could get those 1U SM servers because I have tried to no avail. I cant seem to find a comparable 1U single socket server that will accept any CPU past 4c8t.
Tim, what desktop case is that on the table at 0:15 seconds?
Check the link in the description to all my gear!
@@TechnoTim Thanks Tim, but I just don't see the desktop case listed in any of the gear lists or the documentation page?
What disk shelf would you recommend?
Soon!
I was waiting to hear the make and model of the PC case and the server rack while I was listening to ( not watching) the video, and they never came.
Anyone know the make model of these items ?
My rack is a 25U startech. Wouldn't you know it, a Dell Poweredge R720XD followed me home from ebay the other day.
I hate it when that happens! 😉
@@TechnoTim Considering that they are not that loud when not under full load, I'm looking to adopt another one soon.
instead of the supermicro 1U's, it sounds like a couple of used Dell R630's would have been a better fit for you. Dual socket 2011-3, 32 DIMM slots, tons of 2.5" hot swap at the front. They're pretty cheap, you might still think about getting them.
Why is it called a hOme lab? Not a home network rack?
I’m setting mine up now. Unifi is that patch panel with couplers? That’s what I’m doing
See my video on what is a home lab, and welcome!
On the 1U super micro servers. They are awesome boxes - but that particular model maybe isn’t the best for a home lab. You’re only running SSDs so having full size HDD slots is a bit of a waste. SM has versions of the chassis with 8 (or 10) 2.5 inch hotswap bays. There is also a version of your same motherboard you use with enough SATA/SAS ports use all those hotswaps as well as 10Gbe on board (the “CTF” version of the same board). And you still have the single PCIe slot open. Maybe not something you’d need to replace but if making recommendations to people about to follow your journey this is definitely a place where I would suggest they spend a bit more up front.
Thank you! Would love to see this model!
Ok - so it looks like I brain faulted... The motherboard i was thinking of with 10gbe and lots of SATA is an lga3647 (xeon scaleable) model. X11spi-tf. Not a version of yours.
Overall comment stands though. For labs following your example I'd still recommend spending a bit more and getting the 8x or 10x 2.5inch 1U chassis.
You kind of glanced over the noise of the 1U chassis - do you have them in a space where the noise isn’t an issue? Or are you able to spin down fans to reduce noise but keep noise down?
They aren't that loud. They are in my basement too, which isn't where I hang out
Great video as usual...
8:58 do you know the model number of the other UPS?
10:52 there is and error in "Subscribe" ;-)
I just stumbled upon your channel and want to thank you! I'm planning to wire up my new home and have an old PC running as a Plex server (hoping that running cable will help the performance on this old hardware) but you've given me an end game goal!
Any tips you can give for Plex server hardware, etc would be greatly appreciated!
why isnt the cisco network switch screwed in???
It's going on an extended vacation soon!
i would raise eveything up to the empty spot take ups out add extra battery cables and add batterys in the now empy rack below this is what i intend to do with my server rack when i get my man cave
I would love to see a video where you have everything in the rack run an agent that gracefully shuts down if the UPS goes into battery mode.
Check out! th-cam.com/video/vyBP7wpN72c/w-d-xo.html That's where I set it up, but what you mentioned is coming soon!
Fibrain makes a .5U patchpanel that uses casettes. Just a thought
UniFi added support to switch wan ports to lan ports. On the port section at the bottom you should have a link for configure interfaces. You can change port 10 to lan. however I don’t think you an LAG it with port 11.
Look every network you build you learn and think of things you could improve,sometimes it will not work out most times it will mostly work out.
8:30 I'd just put a 2U blank panel there to fill the gap. Costs about 10 bucks and looks neat.
Speaking of neat looking: Rackstuds! 🤓
What bothers me the most is the LED strip: You can see the single light spots because they are so far apart from each other. In my rack, I've used a 6000K (cold white) strip with COB LEDs where the elements are much closer to each other - so it's like one long light source.
The wan SFP+ Wan can be changed to a lan port, infact both SFP+ ports and the rj45 wan port can be configured for wan or lan.
I really like the idea of a rack... and the cool factor. It's just so much more expensive than a handful of thrift-store PCs and Odroids strewn about.
Couldn't you "just" automate the shutdown in case of a power outage? May be with a threshold percentage to brigde short power losses. I remember my old APM USP allowing this. Only had to boot up everything again manually.
@Astrocat 3D Alright, its way too obvious to not have that! xD
Regarding the UPS. It should be possible for the servers to monitor the UPS input power state. Most UPS can give out this information. And when external power disapears the servers shutdown themself before the 10min runtime of the UPS runs out.
ive read that UDM Pro WAN can be changed into LAN.
Thanks so much for this video.
I'm looking for a completed inventory that can help me buy everything I need to build my own server.
Can you share those infos?
kit.co/TechnoTim
For me I view Ubiquity as the Apple of networking. Which you may call a compliment but I don;t mean it like that. I mean it as its the locked down walled garden ecosystem that I cannot stand to use.
I'm just getting started into homelabbing right now and haven't gotten any real equipment yet, I only have a basic prosumer grade router but I want to custom build a 1U box to run pfsense. And then I want to custom build a 4U server with the front covered in drive bays to install TruNas and run any docker containers and VMs directly on that too.
What server do you plan to use?
Are you running a cp server with all rhat equipment
I recently dropped my server from the rack in favor of my AWS account. I can essentially do anything I want for the flavor of the month something something I am working on. And I can dictate exactly what I need.
I cant see spending $1100 on a 48 port POE switch when 4x+ 10gSPF+ and more bandwidth you could imagine in a lab from a retired enterprise switch like a ICX 6450 for less than $200. POE++ is the only thing I dont ever see on old switches but that is more for PTZ cameras anyway.
I will trade you my StarTech 25U rack for that 18U any day. Been wanting to size it down some, as I am not using all the space in it, and keep cutting out old and unneeded equipment.
Yoy may want to consider switch Mikrotik Routers, they support multiple VPN protocols such as L2TP/IPSec, IPSec IKEv1 and IKEv2, SSTP, OVPN, WireGuard and ZeroTier!. I would use a L3 switch for interVLAN routing without doing this at the router level!
Your supermicro boards with the DIMM's are populated wrong, consult the manual of the X10sri. A1 B1 C1 D1 should be populated ;) (blue slots)
I’ve since filled them up! Good eye!
Thank you Tim. Awesome video as always
I personally regret actually buying Dell servers and not sticking to consumer hardware like I should have and just getting some rack mount cases. Really the only nice thing is the redundant power supply and the IPMI interface. I however don't regret the purchase of a 42U rack, but I'm not even using hal;f of it. So TechnoTim, if you want. I'm willing to trade you racks. Mines a 42U startech open frame rack.
Unifi Dream machine firmware was just updated to allow this! "UDMP Port 10 SFP+ can now be assigned to LAN! (Running 7.1.66)"
😱