Building a 176TB JBOD SAS Disk Array for Chia Farming, Supermicro 3U 16-Bay
ฝัง
- เผยแพร่เมื่อ 2 ส.ค. 2024
- Featured Products: (affiliate links)
HGST 10TB SAS Drives... ebay.us/fNd9jH
Seagate 18TB Drives... ebay.us/FCls0W
More Cheap Drives... shop.digitalspaceport.com/
Dell 5R10N SAS Expander... ebay.us/y8K8Tk
LSI 9200-8e SAS Controller... ebay.us/8zLMdi
EVGA 510BP Power Supply... amzn.to/3EweaQZ
SFF-8088 Cable... amzn.to/42Q7MBa
SFF-8087 Breakout Cables... amzn.to/3nFcDkW
Today, I'll be expanding my 160TB Chia crypto-currency farming setup by adding another 176TB of storage space. I'll be using SAS drives and converting this old 3U 16-bay server chassis into a power-efficient JBOD disk array! This ended up consuming much less power than I was anticipating.
Chapters:
00:00 Introduction
00:14 New Hard Drives!
01:30 Supermicro Enclosure
04:05 EVGA 510BP Power Supply
05:28 Dell 5R10N SAS Expander
07:25 SAS Expander Power Connection
08:07 SAS Backplane Power Connection
11:51 Drive Cooling Fans
12:30 SAS Drive Cabling
13:25 SAS Expander Cooling
13:50 Power-On Testing
14:32 All Drives Installed!
14:55 LSI SAS9200-8e Controller
16:02 Host Server Connection
16:52 First Power-On
17:26 Does It Work?!
18:29 Final Thoughts/Conclusions
Recommended Stores: (Using these links helps support this channel)
Current Connected... store.currentconnected.com/?r...
Signature Solar... signaturesolar.com/?ref=lithi... ($50 Discount Code "LITHIUMSOLAR50")
Best LFP Source... www.18650batterystore.com/?rf...
SanTan Solar... www.santansolar.com/?ref=lsolar
Battery Hookup... batteryhookup.com/discount/BA...
Batrium BMS... www.batrium.com/
Contact Info:
My business email is lithiumsolardiy@gmail.com. You may contact me for sponsorships, product reviews, business-related questions, or any similar reason. I am not available for personal project questions or consultation. I will not respond to these emails.
NEW Twitter Page... / lithiumsolaryt
Disclaimers and Statements:
► I receive a small commission on purchases made using my affiliated links shared the video description and comments section. The views and opinions expressed here are my own, unbiased, and not influenced by this commission in any way.
► My videos are in no way intended to be instructional "how-to" lessons. I am simply documenting my project for informational purposes. Property damage, personal injury, or death may result, even when following manufacturer's instructions. I cannot be held liable for such damage or injury. It is YOUR OBLIGATION to ensure that you are complying with any local and federal laws as well as code and permit requirements. - วิทยาศาสตร์และเทคโนโลยี
HGST 10TB SAS Drives... ebay.us/fNd9jH (affiliate links)
Seagate 18TB Drives... ebay.us/FCls0W
More Cheap Drives... shop.digitalspaceport.com/
This is one of the most satisfying mods and builds of a storage system I've ever seen. Good work!
Very nice construction! I love it that you re-use and re-purpose. I was thinking, man what a nice ZFS array that would make.
This is such a great wealth of knowledge on building out a custom JBOD. Many thanks for putting this together!
Really enjoyed watching this and to see you reuse an old enclosure!
Great Tutorial, loving it. Didn´t think this was possible but you made it happen. and the power consumption is decent, too.
Very nice! We are running ~ 2 PB using netapp enclosures and just converted to running it all on solar.
This is an excellent practical tutorial on modifying both chassis, and power supply to suit your project. Thank you.
Best drive array setup using point to point in a custom low powered server hardware I’ve seen. Great way to repurpose old tech for say a NAS or Plex Server.
I must say I like this a lot, You have helped me with my next steps for my home server. Need to move from a giant computer case holding 16 drives to something rack mounted that will let me have more room to grow.
A great solution for sure. I'm working on building a 3rd of these currently. Chia farm keeps expanding...!
This is awesome! Im using some NetApp shelfs for my plots. Its a great system (with the iom6 controller) but they are louder and use up to 275 watts for 24 drives. Great video my dude!
That's funny, I'm a IT professional and a battery hobbyist and now I'm seeing IT content on a channel I follow because of my hobby. It was a pleasant surprise and you did well!
I've built my iSCSI storage using three of these exact 16-bay Supermicro cases way back in the day, one contained the server and the two others were JBODs exactly like yours. There even is an original Supermicro part to convert a server enclosure into a JBOD, it's a small board that accepts a 24-Pin ATX connector, connects the front panel so you retain functionality of the LEDs and power switch and provides some fan headers, it's pretty neat. But obviously your solution is fine as well, I almost did it just like you because these JBOD boards were pretty hard to find.
As you've noticed yourself, staggered spinup is a thing and it would have been fine with the normal molex power cables, even with multiple connectors on the same string. And good job that you didn't use the sata power connectors unlike many other people! They are usually only specced to carry 4.5A while the molex connectors are specced up to 8.5A and using sata connectors for general power applications could become nasty very quickly, especially with dodgy adapters which can carry even less amps.
If you want to add a second enclosure I'd use the second port of the LSI controller before starting a daisy chain of expanders, unless you start dual linking the first enclosure, using both controller ports. I assume this Chia business doesn't need a massive bandwidth so dual linking is not important anyway.
How do you hook up 512 drives with only 2 connectors?
@@Mack_Dingo You use SAS expanders, I talked about one at 5:33 and showed all of the connections in the video.
Such a small board to use an enclosure only to attach many hard drives sounds very interesting, as I'm in the process of building and selling similar things. I couldn't find such a product, do you have a name for it?
I love your video; this is exactly what I had in mind. Thank you very much for sharing.
The most amazing aspect of this build is how quiet it is compared to other shelves.
It's because I replaced the fans. The original fans it came with were the typical high-speed server fans that sound like a jet engine.
@@LithiumSolar Going to build one for these from a BayNetworks Wellfleet 10mbps 48 port switch enclosure to attach to my R730xd. Great video thanks for posting this !!!
@@LithiumSolar is it possible to boot into sas drive with a sas card?
using a mobo without a built in sas controler
Very neat and thoughtful workmanship.
Thanks! This makes my hunt to build a JBOD easier just hunt/look for a faulty server that has caddies and backplate, and well all that space I could add more drives!
My friend you made look easy. It was a very good watch as well. I'm currently looking at hub option until I found this
Just wanted to say I loved this video, and I love ur mods.
Thank you :) I just finished building out my 3rd one!
Thank you for the explanation. Your video was very well done! Even a rookie like me could understand.
Got a tutorial and learned much more than I expected.
Only thing I see I'd change . You have two holes for 80mm fans right behind the SAS expander.A couple more silenx fans wouldn't hurt.
Thanks for the video. I enjoyed the build. Would love to see more hypervisor stuff as well. VMware vs XCP-ng or whatever. Or, running local disks vs TruNas.
Exactly what i was looking for, thanks a lot!
Now this is AMAZING! Great build and PERFECT explanation.
access
So it seems LTO-9 is in my future.
Impressive ! Very well explained.
I liked the last video you did on this, looked into the subject matter and thought it would be dead before I broke even.
Great vid bud, thanks for teaching me something new!
Ready to switch to Linux yet? ;)
@@LithiumSolar 🤣😜🤣😜🤣😜
oi oi oi pete, where u been hiding lately??? r u still in plandemic prison?
Awesome.
You are very intelligent brother.
Thanks for sharing
Awesome video. Wanting to do something similar for a bunch of disks I've been given. Appreciate the content.
Thank you for posting this video. You have a new subscriber. Keep up the great 👍 work. This is an amazing 🤩 build.
Wow that was really interesting, thanks a lot.
Nice presentation!
Amazing explanation 👌🏼
This is awesome, building one!
It's not my cup of tea but I watched anyways because your videos are very detailed and informative.
this is quite impressive and Please can you do more of this,particularly on some of the onboard power connector conversions you made there
I moved most of my server content to a separate channel, please see youtube.com/@HomeSysAdmin
Great video!👍
Nice job, Thanks.
Great, Great video. well done.
You sir may have solved a very unrelated but good problem for a dev rig and I love the fact its 128Watts, i do wonder if software raid is possible, keep up the good work I love these tutorials :)
Yes! You can software raid, I use mdadm on mine but pretty much anything works. It's just a pile of individual disk to the OS. PS: I moved most of my server content to a separate channel www.youtube.com/@HomeSysAdmin there's a more recent video there where I built another with a 4U chassis.
Building one.. love it...
This guy is like the Chris Fix for IT but more janky. I love it :D
Do I take "janky" as a compliment? Or... LOL :)
@@LithiumSolar Definitely a compliment. Reusing electrical equipment and making it look this good is awesome.
Cool, if only a shorter depth chassis existed similar to this one. It looks enormous. I wonder if cutting it with a dremel would be possible.
Hey Mr Lithium/Solar man.. I'd be interested to see a video on cutting out the DC-to-AC-to-DC conversion losses.. what would it take in the realm of boost/buck converters etc to run this kind of compute/storage rig directly off of a 12V/24V/48V lithium battery bank?
On a similar note, I was thinking this yesterday.
@LithiumSolar Can you make a DIY DC-DC UPS? I want a solution to power my home router and modem when the power goes out (I need 12VDC, ~3A & 19.5VDC, ~35W). Maybe something that fits in an ammo can? :) Thanks!
I used to do enterprise dell stuff back in the day so yea this is is a decent build using hackdell equipment. Its usally better to find an expander that is compatible with the backplane you are running so you can turn on the notification lights from the os. Those are raid hardware controled (or in the HBA case, software controlled). If the backplane shows up in linux, you usally have to do some configuring on what drive address goes where. It sounds like a hassle, but if you swap the sas plug in the back, the drives detect in a different order and its annoying having to replace: P Might be seeing alot of supermicro stuff soon. Yahoo converted all their stuff to it to save money and I am sure they will be upgrading in the next year.
Oh and technically they are hot swapable. I say technically, as I have done enclosure swaps. As long as all the drives are unmounted and you know how to reset the controler (or rescan) using the os. Its sketchy though and only did it when the customer was "WE CANNOT TAKE IT DOWN!" kind of mad.
In other words, `right-click`>`safely remove` and the just unplug with both the JBOD and host server still powered on?
This is a solution i been looking for in forever but never had any server hardware experience, I wish u mentioned pricing on the server rack gear and expander cards plus cables, the cable joinr the host pc to the server rack cost a fortune doesn't it?? I would be good if u can do a few more vids about this and somewhere to source the hardware.
I linked to most of the parts in the description of the video. The SFF-8088 cable is about $15. They're cheap now. I remember 10-15 years ago they were expensive. The only components I don't have prices for are the chassis (I've had it for years and Chia drove the cost through the roof right now) and a server rack (I don't have one).
You inspire me build one of this, all I need to think about is how I could buy those hardware :). Kidding aside, can you make a video update about your Chia farm? Is it still up?
I have a lot of updates, I just moved them to a new channel. See here... www.youtube.com/@HomeSysAdmin :)
@@LithiumSolar Thank you, I'm a subscriber on that channel as well :D I just realized now that they're yours or they're different channels
nice video. nice
How long does it take to break even? Love all the technology.
I was wondering that to
I really don't know... Probably about 2 years with the current price of Chia if you were to farm and sell. I don't intend to sell my coins though. I plan to just leave them sit and see where the market is in 5+ years.
@@LithiumSolar Maybe you should convert one of those drives to mine virtual to save the video cards
would it be possible to get an update on your Chia farm setup and progress/plans?
I moved the Chia content to my server channel located here www.youtube.com/@HomeSysAdmin/videos there are a few videos that came after the one you're commenting on. I'll probably have more soon. I'm working on replotting with Gigahorse currently.
@LithiumSolar I just recently got back to chia I have a server rack just like yours and would like to move the plotted drives into it, but do not know how to connect the server rack to the computer and set them up for farming, If you could make a video on that or if you would help me out that would be great. Nice video too!
great great video
I need a link for the chassis including the power supply, please.
And any other parts you recommend too, please.
Would it be possible to run a freeNAS or some sort of NAS in a VM in host machine, and then make the drives available from a SAN, because having them on fiber channel would be kinda nice.
Great work!! Gave me an idea to build something like this as a NAS server with UnRaid. As that will allow me to spin down the disks for my purposes can i ask how much power does the case use when all the disks are spun down please?
I don't have an exact measurement; however, 3.5" drives are usually around 3W idle, so at 16 drives that's 48W. Add in another 15W between the expander card and fans, you're at 63W with a 90% efficient PSU, you can expect about 70W with drives spun down :)
@@LithiumSolar thank you.
I have a EMC KTN-STL4 sitting around. You think I could gut it out and do with it like you did with this supermicro chassis?
Did you ever find out for sure if SFF-8087 is in fact hot-swappable? Do you know the maximum length of run that can be used for SFF-8087? I'm trying to build a JBOD chassis like this, but I might want to keep it in the basement for noise and heat reasons.
No. I've read conflicting information online and haven't dared try it. I just shut it down the couple of times I made changes. I also keep mine in the basement - great for cooling and noise. I can't comment on the max length, also see conflicting information there as well (sorry, can't be of much help).
can i add 1 node controller to connect sas expender with 8088 cable? failover with dual controller storage ?? thank sir
Do you know any way I can get this to work on a win 10 host machine
how much will you make ?
I think Chia is a conspiracy started by drive manufacturers to sell more drives! 😂
Maybe, except that both Bram and Gene are from the US, not China lol.
@@LithiumSolar ? where did China come into this? Seagate and WD are US companies.
@@d-katalyst3732 Clearly I misread in my half-asleep busy working state 😂 Oooops.
How did you bypass the control panel with pwr button on front of the chassis?
Discussed at the 10:17 timestamp in the video :)
@@LithiumSolar I totally missed the part where you jumped the 24 pin connector to for the "always on" feature. I've watched this video twice all the way through which makes it even more embarrassing. Thanks and this is much appreciated. BTW, just wanted to let you know that you might save a little bit of time/money on your second CSE-836 by swapping the backpane to the BPN-SAS2-EL1 which has a SAS Expansion board built in for all 16 drives. This is what I have after buying a CSE-836 based off watching your video! It has a single SAS SFF-8087? connector that you can run right to your HBA and you don't have to worry about SATA cables everywhere.
How did you attach that fan to the heatsink on the JBOD controller?
And did you flip it around so it's not just blowing the hot air right back on it?
I used two small zip ties, nothing fancy...
👍
I have 24 bay Jbod, it's working fine with ubuntu at the beginning but the hd become unmount after start mining. could you please give some advice. Thanks
Two things I would check
1) Make sure all cables are connected properly, no bent/damaged pins, no corrosion on connectors anywhere
2) Check the system log file, it will likely have some sort of explanation as to what's occurring. On Ubuntu, this would be /var/log/syslog.
Do you have any pictures from after the house fire?
Nope, but if you tell me where your house is, I can come take pictures
@LithiumSolar it was a joke bro chill 😎
What are the drive temps like in this case
They stay within the high 20's to low 30's (Celsius) depending on time of year. Nothing even close to a concerning level.
you could've just bought sc jbod board... and have front panel working. (also could have used velcro, no need for trilling/cutting.)
Uses a 510W power supply … just to use 120W in total? Why not a 200W ps?
I was think you doing only battery and electricity things lol
I have a lot of hobbies that interest me, just don't usually make videos of them :)
are people still doing this? you can buy JBods for not that much, but you will use a lot more power my 15bay uses around 400w I think. but I have 12gbs on a 10gbe lan connect.
I think you may be able to do it cheaper with older FBC hardware?
also can you not just pull out the cage for the bays and them mount them all themselves into the chassis from both your servers?
Yes. I have 3 of these now. And no, the drive cage is not removable that way.
@@LithiumSolar here is a little secret for you.
All standard computer cases have a standard Gap. Old hp ml 310 back planes fit directly to the drives and have one standard molex connection.
There in groups of four, and can be found for a few pound as there the desktop version of there severs.
Better using a modular power supply. cable will be more less.
HBA + Expander could be ~16W easily
And I have difficulty wiring up a Raspberry Pi.
Every time I see a dell product, its using non-standard things, even though it could use a standard.
Like my computer with non-ATX PSU, even though, it could use ATX, or this square molex instead of normal molex.
Did you actually make any money on this?
I am hoarding my tokens for now. I have not sold any, so I've made $0 so far.
😁👌🖖✌👍😎
> Epoxied the SAS expander to the case with standoffs
OK nice..
> Mounts the HBA adapter without PCI bracket
*shudder
It came with a low profile bracket. I need to find/order a high profile bracket.
You say before chia coin was a thing. It was never a thing. It still isn't a thing. But thanks for showing off the jbod
I don't know what you're getting at. It became "a thing" when it was released in March 2021.
@@LithiumSolar no it didn't.
K bro, whatever you say
@@LithiumSolar most of crypto never became a thing. Bitcoin, doge, and Ethereum being a few of the exceptions. You doing crypto is raising prices on parts i need for storage.
......that is NOT an expander backplane. that's a TQ
I don't know what you're saying. This video features a SAS backplane AND a SAS expander. They are two different parts.
How cheap can a person go?😅😅
Even the monitor is defective with blue lines
LOL yeah that laptop has seen better days. That's my shop/garage laptop where I'm doing things that may result in unexpected damage.
Great video tho❤🎉
Isn't chia mining dead already?
No, though price is continuously dropping, so...
@@LithiumSolar WHY DO YOU LIMIT TO ONE CRYPTO ANYWAY
@@RonPaulgirls In the same server he could add some GPU's to do some other crypto's as well.
I would find out at what load the PSU becomes the highest efficient, and aim for that mark if doing the GPU mining, though. Get as much in return as possible for the power consumed. Even if using Solar generated power.
you crazy... just buy netapp and hba!!
13 bucks for terabyte ;-) ;-) I bought Dell-Oracle HDD rack with disks, each disks costs to me 9$ (4tb HDD 3.5) or 1Tb costs $2.25. Even with this price I think what Chia is biggest scam.
4TB disks are impractical for large-scale storage.
@@LithiumSolar For $2.25 per TB it is most practical HDD on the planet Earth. At least for right now. But I bought it from former Data Center, that is why so cheap. Of course - it is not brand new.
Ok, well you enjoy those :)
Chia skam🤣🤣
Chia farming?? I don't see why you need so much space.....sounds more like a nice bittorrent store
Chia farming is a proof-of-storage crypto that requires storing large amounts of "plot" files - different from proof-of-work like Bitcoin and Ethereum.
really don't get chia farming "oh lets destroy some perfectly good drives for some magic barley known currency that's going down in value" Like at least bitcoin and euterium weren't as destructive.
No, Bitcoin and Ethereum just burn up graphics cards to no end so people whom need them have to pay 10x of practicality to get one. I still can't get a good graphics card for video editing and refuse to pay the insanely-marked up prices.
Mining does not damage graphics cards like chia destroys drives on plot creation. drives have a limited number of TBW before they fail, be an ssd or HDD, as long as there half decently cooled a GPU will survive. Current GPU shortages are not really down to mining, but just industry shortages. Sucks for us all, but I find most crypto mining to be incredibly wasteful.
Pretty much everyone I know whom is mining with GPUs have loaded custom BIOS's for overclocking and that sort of thing. They get burned up after a few years. I'm aware SSDs have a TBW rating but that's just a rating for warranty purposes. They'll last much longer - in fact, the 1TB NVMe I've been plotting with is over 4x it's rating and still runs fine. I wouldn't put anything critical on it though, exactly the same as I wouldn't rely on a GPU that's been mined with for a few years.
@@LithiumSolar I think those "custom bios's" that you mention are undervolting bios which make the card last longer because they will use less power for similar performance (do they get a quicker return because it's more efficient per watt),or to adjust memory timings for the "cold" cypto (that doesent really use the GPU core, but more the memory). Almost every mining platform will recommend some sort of underclock, weather it be a simple "turn down the power limit" from nicehash or some really detailed guides from other company's. the main failure is the fans, which is a cheep and easy fix. eitherway were talking less of 2 evils here. Just ewaste production. The difference between the 2 is this. You do not want an ex Chia drive for pritty much anything other than chia. there ewaste really, even if they don't fail before chia is unprofitable, they cannot be relied on and even using them as game storage is just a ticking time bomb of you having to reinstall hundreds of gb's of games. A used GPU can be sold as a ex mining GPU at a lower cost. Even if the thing did fail, you having lost anything other than that GPU. any artifacting can sometimes be solved by a simple underclock. Once again it's the less of 2 evils. Both accelerate the failure of the hardware. Either way, $12 USD a month is decent amount so you enjoy your chia operation :)
You really have no idea what you're talking about. The drives for farming are written to 1 Time period! That's it! Even 5 months later Drive diagnostics show no more writes were done on the drive. One 16TB drive gets about 16TB's of writes (that's 1 write) when filling the drive with plots and that's it! They sit idle 90% of the time as only 1 out of 512 Plots are checked per a 10 minute block. Reads are even less because only on 9 encrypted keys are checked per validation and 64 sequential keys to validate a winning Block. On 16TB drives I don't even have 1TB of total reads yet.
As for burning out NVMe's and that's probably what you meant. All of my NVMe's used to create 4200 Plots still at over 80% life left and I only used MadMax for 10 plots. Also for several months now you can create plots in memory using Madmax and a RAM drive so even NVMe's are getting very minimal wear now making plots.
As for value. Chia was only predicted for months to be worth $20 a coin. People pumped it up to $1500 but most people that knew what they were doing in Crypto converted their Chia into more stable crypto like I did when it was overbought. Since you posted Chia is actually up 30% not down. I still think it's oversold some. Chia farming has paid for my entire 420TB rig 3 times over already.
A fool and his money are soon parted. You really think this will be worth anything in 40 years? 100 years? What a joke.
@@RadiationNetwork Money making, the only people making money are the people selling storage to fools who do chia
@@RadiationNetwork Stable crypto.. lol, good one..
@@strandvaskeren If you think about it. Stablecoins are a type of crypto. I think I’m going to invest in Terra. The most super duper stablecoin in existence :)
eehhhhh. chia is something I don't want to touch with a 10 yard stick.
like at all.
This is totally a bad idea. Dislike!
How did you get to the command screen to see the hdd info using tge host computer ?
Man, bro is taking chia to the max like any simple person would just say just buy a server with everything included, and just need to put the drives in but no this guy is DIY 😂🫡
Yeah man, less power overhead, cheaper, better storage density. If you're willing to DIY. If you think this one is big, check the new 40-drive version I built th-cam.com/video/tmqkUI_ZXSs/w-d-xo.html