@Pi Man A 1TB NVMe drive can only handle around 600TB of data written to it before it dies. These drives can take 22 Petabytes, or around 37x more data before failing.
My room is a mini home lab. No rack though, just a table with a lot of computer hardware on it (a minitower home server/router, a high-end gaming PC, two monitors, 5 external hard drives, 16 port enterprise switch, Ubiquiti AP and some other stuff)
@@WangGanChang I picked one of these up for under £100, and an F40 for like £75, now the only one I see is £175 because it has CHIA in the listing title
Back in 1988 i purchased a complete large scale server setup from the Boeing company surplus sale and tried the server thing for about 3 years then sold the setup to a small company that wanted to get up and running fast and needed a turn-key system. all i can say is its a lot of fun to experiment with new tech and today i have a entire room i can use for computer experiments and given all the advancements to software and hardware the choices are endless. CHEERS!
For those looking for alternatives, there are old Intel enterprise U.2 NVME MLC SSDs also being tossed out which only need a PCI-E adapter to work in a normal PC.
one of the reasons I didn't grab one of these was that I was reading that they use a fair amount of power. It would be interesting to run some tests to see how much they actually use.
@@joonasfi Koolance offers a really fancy ~1" square block which I thermal epoxied on my LSI 9260-8i years ago. It's still solid to this day, and connecting a water line is easy if you have a water cooled gpu right next to it. That block might work on one of these too, and the thermal epoxy works great
Is there a name for this effect yet? I know the "Techmoan Effect" is commonly used for rare audio gear Techmoan showcases, but it also applies to just about any electronics/tech channel that shows off some cool hardware at good prices. I got my F80 a couple years ago, so, yeah. Although now if I ever want or need another I'll probably have to pay a stinkin' premium... Thanks a lot! :P
Managed to buy one just before this video was posted....and the writes on these drives are VERY CONSISTENT. I ran this in IT mode on Unraid and striped them in RAID0 and transferred a folder with 60gb of videos and it was a consistent 800MB/s per second over 10gbe. Not bad
We (Sun/Oracle) used these in Exadata and other engineered systems. We now offer Intel 6.4TB Cache drives that show up as a single 6.4TB drive. The original cards was an F20, 96GB, 4 24GB flash modules with a ESM (capacitor) to keep the power if the system lost power.
You are literally the second place I've ever seen floppy disk coasters. I once got some as a handout at a County fair made of black plastic with a aluminum slider stuck on both sides and a home printed label.
or you could have simply looked up ssd to buy on google like I did and this was one of the first 5 pages on the list lol.I saw this thing a long time ago too I just wanted to see performance.
Ive got a handful of these F80s (1 in IT mode for unraid cache, another flashed into raid0 for a windows drive). Also have a F160 (1.6TB) thats being used for a couple of VM drives (4x400gb OS drives in IT mode). They are nice when they come up on eBay for the right price.
I bought one of those cards about a year ago for the same reason. Got it flashed as you did, and right now I currently use it in my unRAID server and pass it through to a windows VM as a cheap way to improve my VMs disk speed.
I picked up a Fusion ioScale 3.2TB PCIe SSD shortly after finding out I can use it as a storage drive. Got it for $210 shipped with 64% life left. Was about to build a new VM server and pulled it from my system to move to the server, and at the same time, I picked up a SanDisk SX350-6400 for $600 new, shipped. So that's now my new storage drive in my gaming/productivity server and the Fusion ioScale host my VM's and SFTP. These old enterprise drives are a great buy for average users.
It's not very often I miss features of a full ATX system but when I see like this I do start to question going ITX. On the upside the few friends I do go and see this means I can take my machine with me with it not feeling like it's totally impractical.
Just a heads up: 1. These drives run VERY hot (and by that I mean: you cannot touch them right after you have shut down your machine, as you may get burned), so if you're thinking on running them on a compact/low ventilation chassis, just don't. Remember, these were designed for server use (tons of cooling). 2. Right out of the box, they are NOT compatible with ESXi, particularly direct or hardware passthrough. Not sure if flashing will help as I didn't get to flash mine (see below). I don't remember if they work on Proxmox. 3. The price difference soon will not make those worth it IMO. You'd be better off with standard enterprise SSDs (unless it's QLC or if you are out of SATA ports and PCIe is your only option). 4. No, they will not last forever. Mine only had about 200 hours of use when I got it and still it died miserably last year for no reason (less than 2 years total use). Source: I bought one about two years ago, back when it was recommended by www.serverbuilds.net Nice video though.
Now as these have doubled in price on eBay, here's a good alternative: FusionIO IODrives. These come in at about 10c per GB ($60 for 600GB, $120 for 1.2TB) and require no flashing and have readily available Windows drivers. They're also 2GB/s read and 1GB/s write like these. Unfortunately the Linux/FreeBSD support isn't quite as good and I haven't been able to get mine running in TrueNAS, TrueNAS scale or UnRAID yet.
I’ve had a similar issue with the “sas address checksum error” when I was flashing my dell h200 into it mode for my truenas box, I think when you use the sbr tool it removes or doesn’t read the sas address of the card as you can see 12:46 “Sasaddr = 0xffffffffffffffff” so most of the time on one of the stickers on the card is the unique sas address but if you can’t find that you should be able to “make” your own sas address to get rid of the error. The sas address, the way I understand it is it’s like a MAC address for storage cards where every card has a slightly different address so you can differ the cards. Hope this helps and let me know if that fixes your issue :)
The IT mode 4 x 200 were sun/oracle OEM, you can find 4x 512 with IR nativ. I was one of the engineers that tested these cards back in the day. I can tell you more but I don't know when My NDA is up. If you get a Nytro Megaraid it has 2 x 512 and an onboard sas 2 socket. They also come with a native ability to create a ssd cache disk, one thats large then you are able to create with cachecaid tm. Stay away from the wildcat/ optimus drives they were bad. The 2.5inch drives were good but the lsi ver. didnt work so well.
The moment you said "and they're just $60 in ebay" I thought "yeah not now they aren't". Still great value even at $200 though, considering their endurance.
I bought one for $60 USD from Japan today. Too bad the shipping was $70. Still a great value and will be replacing my tiny 256gb cheap Kingston cache I have now.
He isn't joking about the temperature, these start throttling at 73C and basically stop at 76C, definitely need a fan on them. In my 68F room they were idling with no air flow at 64C on a benchtop open case. Also when doing the final format, both times I had to do it twice (Reboot and try again), both times second time took, not sure why the first go round failed. I was able to find 2 for $70 each, prices are inflated now, I will consider myself lucky. Thanks for this awesome guide and bringing this to our attention!
mine are in active use (few MB/s) at 55C in an r710 (using as a cache drive for an rclone mount). Was using rust before as the cache drive, but the rust was a raid5 array and the metadata latency (i.e. deletion times) was abysmal, which has been my experience on fragmented files on raid5 which as rclone mount is going to do, as it just creates sparse files and fills them. Performance is now much more normalized.
Indeed, those Americans are so lucky in that regard. It's pretty hard to find a SAS HBA or something for a decent price over here, even used. It's as if there's no market here for second hand enterprise hardware.
@@InsideAlan maybe, but what about HBAs, enterprise CPUs and ECC memory? I had to move heaven and earth to obtain the latter two for my home server and still haven't found a decent HBA, neither new nor second hand.
This is fun for sure. Sadly prices have doubled as others have said. I just bought a 1tb Sata SSD (older PC that still uses sata) for 80 bucks it should do the job for me.
I got a intel dc6300 ssd, the 1.6tb version for $160 with Christmas money, it’s a great ssd. Intels M.A.S utility is easy to use and shows real time stats. If you can find one I would recommend buying it
Test an ioDrive2 - 785G vs these drives ... Would be interesting to see which performs better as cache-drive. The ioDrives have almost double the IOPS to the Sun F80 and almost 25% more IOPS to the Micron P320.
I've had one of these boards as a 4x200 GB drive in a test machine back in the lab for a number of years. Enough space to install an OS and try out a few things in each of the cells. If you're looking for a secondary source for these cards, check out the older pre-Oracle Sun server rackmount hardware. I recall these cards were used in redundant pairs in Sun X4500 and X4540 servers. But, be warned, those boxes are BEASTS, perfect for that collection of 48 3.5" SATA drives you have laying around that need a home... kinda surprised that Jeff doesn't have one in his rack at home. :)
Great video, I have been using IBM LSI 2008 cards for my freenas server and freenas server backup for several years. I have never re-flashed the bios in Linux as the writeups were for win os. Your use is different so I understand for using in IR mode, but I have always used IT mode to get the full benefit control and error checking of ZFS. I always have at least one extra LSI card Just in Case.... Thanks for the video
Buy a shield for $6 or a printer for $200+. Hmmmm, choices, choices. (Believe it or not, many people still haven't a need for 3D printers. They may be handy, but they're not for everyone.) Hell, I could carve one out of sheet metal too, but there's diminishing returns to think about.
if you can get them, Micron P320h drives have much lower latency, higher speed, and can deliver more IOPS comparatively, though they are definitely more expensive and are becoming pretty rare.
There is also a Fusion-IO 1.2 GB app accelerator which works well as a cache or SSD storage option as well. I hope you look at more of these high end durable NVMe storage options since many of them are very cheap on eBay!
Perfect timing. I literally just walked back in from getting my mail (which contained 2 Sun F80s that I ordered before your video drove prices up and availability down) So it looks like I need to start flashing. Thanks for the tutorial! Any plans on similar videos for the Fusion IO devices out there?
You know what I would like to see? A PCI card that can use DDR4 memory for storage. There use to be cards like that made, but no one does it anymore and I have no ideal why. It is awesome having a dedicated chunk of memory that you know 100% will only be used as a junk drive. You can do the same thing with some programs that use your system memory to create a temp drive, but they don't always work so great or in every operating system like a PCI card can.
I'm thinking about grabbing a couple for my "big" dual opteron libvirt box for VM storage. Its currently running on spinning rust with a nvme Samsung 960 EVO 250GB lvmcache. So its not GREAT but not SUPER TERRIBLE. A couple of these enterprise cards would kick it up several notches.
I managed to pick up an optane 900p before the 905p were the only thing you could find. At the time it was only like 260 for a 280GB capacity - best zil and l2arc ever
Nicely explained, very nicely presented, and deftly executed. I'll be looking at other videos from your channel, and will likely subscribe after that. I'm none to keen on Windows, but don't resent that you use Windows tools when they suit your purpose. Pop OS has piqued my curiosity. Thanks for that. EDIT: as Wtdtd observed, yeah... prices have shot up more than double since this video was posted. There goes a cost-effective solution, swept away by market forces. Eh. It happens. Still a good video, and, who knows? The price may drop after the initial rush wears off.
"So we don't have to wait, here's one that I prepared earlier." You... yep you did it. I'm glad I'm not the only one that uses nano. I feel like literally everyone else uses vim or emacs.
@@JarrodCoombes Vim is way faster for actual text-editing work, and works well as an IDE (not as well as EMACS, but still better than most of the paid bloatware IDEs out there). For the use here, Nano works just as well. Learning to use "text objects" correctly was what hooked me on Vim, although I am considering moving to Evil EMACS, which is EMACS with Vim bindings. Best of both worlds. I find that Nano is good for those who have only been exposed to GUI text-editors before, due to its familiar control-scheme and keybindings listed at the bottom.
e: I also got the SAS address checksum error, but my F40 is now working as expected after loosely following the same guide you did. Think I caught you slackin, Jeff. You said ServeTheHome, but you're referencing a ServerBuilds.net post. At any rate, can you add the links to those posts to the description? I've had an F40 laying around for a few months at this point that claims to have something like 100 million reallocated sectors, so I've decided to dust off and try and reconfigure thanks to this video. And I scooped an F80 for good measure.
I just picked up a F40 and was wondering if the process for it is exactly the same as the one listed for the F80 or if any other changes or files are required? Thanks
@@Thetruth-di2ic It's not super different, but there are a few extra steps. I used the F80_Files zip package from the ServerBuilds post because it had all the tools available already, but the Kasilag post ( kasilag dot me/warpdrive ) has direct links to the F40 firmware. You need to split the firmware as described in the Kasilag post (requires trimming a bit in the beginning and a decent amount towards the end of the file), when updating the sbr.cfg file to change the PCI IDs, you use SubsysVID = 0x1000 and SubsysPID = 0x0581 instead of SubsysVID = 0x1000 and SubsysPID = 0x0504, and then you'd swap out the references to the F80 firmware with the F40 firmware. I'm probably going to wind up writing a guide specifically for the F40 anyways if that's not clear enough.
Reminds me of when IBM M1015's were dirt cheap and great for ZFS. Might have to grab one of these to play around with but I suspect a bunch just sold =D
Great video, too bad they're all gone now tho. Back in like October? I was able to score a 1.6 TB Intel P3605 for $160 been an amazing boot drive for my desktop!
I have yet to find a better deal than the p3520 I got for 125 bucks. Itd been listed for 2 months at some enormous price, and slowly lowered by 20 bucks at a time. I guess they eventually gave up and just dropped it to 125 bucks - i immediately picked both of them up, and even with their less than PB endurance, i just dont see me using all that endurance before needing more space. And better performance too!
Had i not just put together a 4x110GB Optane raid card for caching i'd have jumped on this, as it serves the same purpose two drives for write caching/additional protection against power loss, and two drives for read cache/array load reduction
"FLASH ACCELERATOR" I've never heard about them ^^ Interesting you can store the components on a hard drive (like a 16bit cartridge) and then compile them into a quad system to work together, or build a game with different components and after compile
neat little cards, those things. I have a pair of F40s in my Dell R710 I use for VMs. kinda wish I'd gone for bigger ones but oh well. I seem to only get like sata 2 speeds out of a single VM. might be something to do with LVM. idk. every once in a while I might notice a GB/s write speed for something. they do get a bit hot though. think I got my fans spinning at 3k to keep them happy.
I bought these floppy disk coasters too, they're nice. And then I realized I have a massive box with hundreds of dead floppies... Guess I can throw a really big party with them.
I don't really have a use for a home lab right now, but i did order one of these and a bracket that ill be using for a game drive, $65 after accounting for the bracket is really nothing for such a nice drive, even if it isn't as fast as modern nvme drives.
Jeff, be on the lookout for a tiny little lady wildly swinging a 9 iron....Lately my better half has become quite irritated with you, (and ebay, paypal, UPS, FedEx, Amex etc...) Now to figure out how I can extend a PCIe port to a remote location in my NR12000 so i don't have to give up my 4port NIC.
The format did not work for me and just gave me an error so after redoing the process - getting the same error I just rebooted into Windows. In Windows I used the ddcli to format and this time it did work. Yay 800gb ez ssd!
I don't know if it was mentioned, but you might also want Linux to help with the install. I guess you could use the win 10 command line as it takes bash commands, good luck though re-writing the software.
this is awesome value for a nas on a 10gb network as a cache as it can still saturate the 10gb net bandwith :) thanks for the video, I might need to have a look at this
I got one of these a couple years ago for around $90. I'm currently using it as VM storage on a Proxmox server, with the four SSDs configured as a ZFS striped array. Works great. In my case having it in IT mode is exactly what I wanted. You can also get 400GB versions (the F40) for a bit less. Also, there's the F20, which has only around 100GB of storage (wonder why it wasn't called the F10...). F20s are usually quite inexpensive (often less than $25) and can serve as a good ZIL for a ZFS array (since ZFS by default flushes every 5 seconds anyway, so you don't need much storage for a ZIL) If you search for LSI Nytro, you'll also find 1.6TB and I believe even 3.2TB versions of this which have up to 8 SSDs onboard to achieve that capacity. (I imagine one of those cards could easily achieve the SSD speeds we currently see from PCIe 4.0 NVMe drives!) This concept was actually quite popular for a while. OCZ's RevoDrive series uses the same concept (putting multiple SATA-based SSDs on a single card intended to be RAIDed together). FusionIO cards also are flash-based accelerated storage. Of course, today we are seeing the same approach but with NVMe SSDs - it's possible to get >20GB/sec with an 8-slot PCIe x16 HBA supporting NVMe drives...
yeah i bought a sun f20 with the hopes of using it as a boot drive but it doen't work as a boot drive and idk why, the data sheet says its suposed to be bootable
@@firenado4295 It may be possible to boot from it, the main issue is the F20 is based on the SAS1068 chipset which, among other limitations, doesn't support native booting on UEFI systems if I recall. On BIOS based systems it will present a BIOS extension that you can access to configure booting from it. The F80 is based on the later SAS2008 chipset commonly seen in PERC H series cards popular among homemakers, so it can boot a UEFI system just fine (might need to flash a UEFI boot ROM, but I know I've successfully done it as an experiment)
@@fdmillion ah cool at the moment i have my f20 in a 2006 dell dimentions c512 and i think thats old enough to just use a bios and not uefi. but idk what to do really starting to regret this perches to be honest lol
@@firenado4295 yes that system almost certainly uses traditional BIOS. When you power on do you see a few messages showing the card initializing (it will usually show the four modules and a copyright notice for the firmware - either LSI or Avago depending on revision)? If so it should give you a key you can press to access the cards configuration, where you should be able to setup boot ability. If you don't see the init messages at all go into your regular BIOS setup (F2 on most Dells) and look for an option related to "Option ROMs" and make sure they are enabled. Hope this helps!
they advertise em as memory buffers, cache, and that no one is really looking for, I bought 3 cards cheap years ago because the info I found said I could use em as a drive, not just cache.
The power of Jeff, some guy had 20 of them on a new listing for $89. By the time I had switched browsers he'd sold 16 0f them. I bought one and I suspect they're all gone now. I bet he didn't even know why his entire stock blew out on eBay in under an hour. I'm just glad I got one at only 50% over what you indicated. Please keep on making content around old enterprise hardware. My test server has to live in the living room with us so I have to re-case and re-fan to keep the noise down. It's running TrueNAS (FreeNAS) and it's just a hoot playing around with what it can do. I'm probably going to use it split as dedicated cache for two arrays.
I just realized I made a stupid mistake. I bought one of these SSDs to use as a cache-drive for my home-server and when I wanted to install it I realized I don't have any unused PCIe slots...
this particular hardware didn't interest me much so just skipped to 18:00 for the drink :) there is a Ward 8 bar/restaurant in Boston near North Station. used to walk by it when going to work. and yes, it was for the state legislature. there is a wikipedia article on it though only mentions 1 teaspoon of grenadine
Really depends on your board orientation, if the Mobo is mounted flat(horizontal), there is really no need for the cover, if it is mounted vertically (like normal tower case) then it adds support, these x8 cards don't lock in like x16 cards so the PCIE end cover secures it in place, without it, it could fall out as the pcie contact spring force is pretty weak and the only thing holding on to the card.
....Aaaaand they're gone. Prices are now double of what you said.
$200 now
Lmao it literally took 1 hour
Just bought 2 @ $100/ea. There are a few left still
Why buy these when I can get 1 TB SSD for $90 to $115 and not have to do all the extra work he had to do.
@Pi Man A 1TB NVMe drive can only handle around 600TB of data written to it before it dies. These drives can take 22 Petabytes, or around 37x more data before failing.
"Not everyone wants to dedicate an entire room in their house to a home lab..." Challenge accepted good sir.
Wait, 18 hours ago
*Something's wrong I can feel it*
@@jackson8753 join the discord
@@FancyBro oof, got it
Yes - home datacentre when? :3
My room is a mini home lab. No rack though, just a table with a lot of computer hardware on it (a minitower home server/router, a high-end gaming PC, two monitors, 5 external hard drives, 16 port enterprise switch, Ubiquiti AP and some other stuff)
This video should be called " How to drive up the price on an Ebay item by creating the appearance of high demand "
I'm sure Chia mining has nothing to do with it.
*sips tea*
@@danlewin2260 was excited, then look at ebay... $220.... Perfectly good drive, now all going be to destroyed by chia...
@@WangGanChang I picked one of these up for under £100, and an F40 for like £75, now the only one I see is £175 because it has CHIA in the listing title
i feel this drove the cost of the card up a lot, knowing that they're at least $140 on ebay now
While you arent entirely wrong, youre only partially correct. Chia got big at about the same time.
pluss hidden shipping fees that only exist in the sellers imagination...
Yep, I just saw this video today and looked at them, and the cheapest one I see is $80 shipped from Hong Kong, the rest are well over $100.
Back in 1988 i purchased a complete large scale server setup from the Boeing company surplus sale and tried the server thing for about 3 years then sold the setup to a small company that wanted to get up and running fast and needed a turn-key system. all i can say is its a lot of fun to experiment with new tech and today i have a entire room i can use for computer experiments and given all the advancements to software and hardware the choices are endless. CHEERS!
I miss the surplus store, many good/fun deals back in the day (Tube Tektronix O'scope, vacuum tubes, etc).
For those looking for alternatives, there are old Intel enterprise U.2 NVME MLC SSDs also being tossed out which only need a PCI-E adapter to work in a normal PC.
one of the reasons I didn't grab one of these was that I was reading that they use a fair amount of power. It would be interesting to run some tests to see how much they actually use.
Yeah, the need for cooling and wasted electricity go hand in hand. It's always a trade-off..
@@joonasfi Koolance offers a really fancy ~1" square block which I thermal epoxied on my LSI 9260-8i years ago. It's still solid to this day, and connecting a water line is easy if you have a water cooled gpu right next to it. That block might work on one of these too, and the thermal epoxy works great
Or you could of just scalped it like everyone else lol
Yep! They're gone! Only $99.99 and above now.
Looks like $190 and above now.
@@AB-hh1xo A handful of them at around $155 after shipping, i ordered mine a couple days ago so i got lucky.
May I present parts.metservers.com/7069200-sun-oracle-rev.02-800gb-wrap-drive-flash-accelerator-f80-pcie-ssd
@@nathanhamman418 Still not a bad deal for the capacity
@@Beltonius you get less capacity and speed when compared to a similarly priced nvme drive, however durability isn't even comparable on them.
The microsecond this video came out, prices tripled. Awesome.
maybe make a "Don't buy these drives video" to drive the prices back down?
Is there a name for this effect yet? I know the "Techmoan Effect" is commonly used for rare audio gear Techmoan showcases, but it also applies to just about any electronics/tech channel that shows off some cool hardware at good prices.
I got my F80 a couple years ago, so, yeah. Although now if I ever want or need another I'll probably have to pay a stinkin' premium... Thanks a lot! :P
Darn... They are like 200 bucks now. 💔 It's okay, dear credit card. You'll get some action soon.
As I opened the video 1 hour after release I already realized they'd probably cost twice as much by now... Oh well, at least there's the video.
Managed to buy one just before this video was posted....and the writes on these drives are VERY CONSISTENT. I ran this in IT mode on Unraid and striped them in RAID0 and transferred a folder with 60gb of videos and it was a consistent 800MB/s per second over 10gbe. Not bad
We (Sun/Oracle) used these in Exadata and other engineered systems. We now offer Intel 6.4TB Cache drives that show up as a single 6.4TB drive.
The original cards was an F20, 96GB, 4 24GB flash modules with a ESM (capacitor) to keep the power if the system lost power.
It's like Steve from GN he made a spot about how affordable those Arctic aios were now they cost more then corsair by nearly $100 regularly.
You are literally the second place I've ever seen floppy disk coasters. I once got some as a handout at a County fair made of black plastic with a aluminum slider stuck on both sides and a home printed label.
"60$ you say??"
*checks ebay*
"Yikes, that was quick."
$200 on ebay MIN and with shipping much more. That for a sub tbit drive is expensive so why waste your time ?
at those prices I'll just stick to hoarding Fusion-io cards I suppose... tho a few F80's to play with would be fun, oh well, not at those prices....
$500 now
@@Dimondminer11 are there such cards with higher capacities? Like 1TB+? Also what are other options that may be better than these cards?
@@DebateKing69 Nah, their down to about $150 US average
Another benefit to being on the exclusive Discord ... we were talking about these a month ago when they were $60.
or you could have simply looked up ssd to buy on google like I did and this was one of the first 5 pages on the list lol.I saw this thing a long time ago too I just wanted to see performance.
Ive got a handful of these F80s (1 in IT mode for unraid cache, another flashed into raid0 for a windows drive). Also have a F160 (1.6TB) thats being used for a couple of VM drives (4x400gb OS drives in IT mode). They are nice when they come up on eBay for the right price.
Is it possible to run the F160 in a different mode? 🤔
I bought one of those cards about a year ago for the same reason. Got it flashed as you did, and right now I currently use it in my unRAID server and pass it through to a windows VM as a cheap way to improve my VMs disk speed.
Love you bro. I actually used linux to do this 95%. I did the final ddcli format step within windows.
I picked up a Fusion ioScale 3.2TB PCIe SSD shortly after finding out I can use it as a storage drive. Got it for $210 shipped with 64% life left.
Was about to build a new VM server and pulled it from my system to move to the server, and at the same time, I picked up a SanDisk SX350-6400 for $600 new, shipped. So that's now my new storage drive in my gaming/productivity server and the Fusion ioScale host my VM's and SFTP.
These old enterprise drives are a great buy for average users.
It's not very often I miss features of a full ATX system but when I see like this I do start to question going ITX. On the upside the few friends I do go and see this means I can take my machine with me with it not feeling like it's totally impractical.
I really like the way you type everything out and explain as you go. Great work!
Dude you are so good in showing everyone the true saving and speed used gear
"Take a drink"
"Make yourself another drink"
If you say so...
"because making this cocktail might take a while, we'll be drinking a lovely boutique IPA only available locally while I make this cocktail" 🤣
Father Jack has entered the chat. "DRINK!"
Just a heads up:
1. These drives run VERY hot (and by that I mean: you cannot touch them right after you have shut down your machine, as you may get burned), so if you're thinking on running them on a compact/low ventilation chassis, just don't. Remember, these were designed for server use (tons of cooling).
2. Right out of the box, they are NOT compatible with ESXi, particularly direct or hardware passthrough. Not sure if flashing will help as I didn't get to flash mine (see below). I don't remember if they work on Proxmox.
3. The price difference soon will not make those worth it IMO. You'd be better off with standard enterprise SSDs (unless it's QLC or if you are out of SATA ports and PCIe is your only option).
4. No, they will not last forever. Mine only had about 200 hours of use when I got it and still it died miserably last year for no reason (less than 2 years total use).
Source: I bought one about two years ago, back when it was recommended by www.serverbuilds.net
Nice video though.
so no ESXI compatible at all ?
Glad I got one the same day of the video release still $89.95, the techmoan effect is nuts
Now as these have doubled in price on eBay, here's a good alternative: FusionIO IODrives. These come in at about 10c per GB ($60 for 600GB, $120 for 1.2TB) and require no flashing and have readily available Windows drivers. They're also 2GB/s read and 1GB/s write like these.
Unfortunately the Linux/FreeBSD support isn't quite as good and I haven't been able to get mine running in TrueNAS, TrueNAS scale or UnRAID yet.
I’ve had a similar issue with the “sas address checksum error” when I was flashing my dell h200 into it mode for my truenas box, I think when you use the sbr tool it removes or doesn’t read the sas address of the card as you can see 12:46 “Sasaddr = 0xffffffffffffffff” so most of the time on one of the stickers on the card is the unique sas address but if you can’t find that you should be able to “make” your own sas address to get rid of the error. The sas address, the way I understand it is it’s like a MAC address for storage cards where every card has a slightly different address so you can differ the cards. Hope this helps and let me know if that fixes your issue :)
The IT mode 4 x 200 were sun/oracle OEM, you can find 4x 512 with IR nativ. I was one of the engineers that tested these cards back in the day. I can tell you more but I don't know when My NDA is up. If you get a Nytro Megaraid it has 2 x 512 and an onboard sas 2 socket. They also come with a native ability to create a ssd cache disk, one thats large then you are able to create with cachecaid tm. Stay away from the wildcat/ optimus drives they were bad. The 2.5inch drives were good but the lsi ver. didnt work so well.
you wiped the the sas address of the card you reprogram it with lsiutil
The moment you said "and they're just $60 in ebay" I thought "yeah not now they aren't".
Still great value even at $200 though, considering their endurance.
I bought one for $60 USD from Japan today. Too bad the shipping was $70. Still a great value and will be replacing my tiny 256gb cheap Kingston cache I have now.
no
I would love to have a list of cool older enterprise parts that would be good to look for.
Yes and put into categories, like beginner home nas, enterprise practice, tinkerer. Not sure on the exact names but some sort of categorization.
He isn't joking about the temperature, these start throttling at 73C and basically stop at 76C, definitely need a fan on them. In my 68F room they were idling with no air flow at 64C on a benchtop open case. Also when doing the final format, both times I had to do it twice (Reboot and try again), both times second time took, not sure why the first go round failed.
I was able to find 2 for $70 each, prices are inflated now, I will consider myself lucky. Thanks for this awesome guide and bringing this to our attention!
mine are in active use (few MB/s) at 55C in an r710 (using as a cache drive for an rclone mount). Was using rust before as the cache drive, but the rust was a raid5 array and the metadata latency (i.e. deletion times) was abysmal, which has been my experience on fragmented files on raid5 which as rclone mount is going to do, as it just creates sparse files and fills them. Performance is now much more normalized.
Thank you for this tutorial, I just followed it and other then I had to take into account I was booting ubuntu from a usb stick, it went pretty smooth
did you get it to boot from the f80 or is that not what you are using it for?
@@firenado4295 I'm using it as cache in a unraid server
@@electrohacker a fair enough, im trying to use an f20 as a boot drive for my server but not having much luck getting it to be bootable so far
Here in Europe those things cost about the same as a Gen4 NVMe from what I saw with a quick Ebay search… ouch…
Indeed, those Americans are so lucky in that regard. It's pretty hard to find a SAS HBA or something for a decent price over here, even used. It's as if there's no market here for second hand enterprise hardware.
Different data regulations. It's a lot harder to wipe these drives to a European standard.
@@InsideAlan good point.
@@InsideAlan maybe, but what about HBAs, enterprise CPUs and ECC memory? I had to move heaven and earth to obtain the latter two for my home server and still haven't found a decent HBA, neither new nor second hand.
@@RudyBleeker try Techbuyer_computing on eBay, or drop me a DM with what you're looking for!
Makes perfect sense that the Sun F80 was in IT mode. Sun created ZFS, a device like this would be a perfect L2Cache red cache or a write SLOG.
This is fun for sure. Sadly prices have doubled as others have said. I just bought a 1tb Sata SSD (older PC that still uses sata) for 80 bucks it should do the job for me.
Another good option is to use it as SLOG device in raid10 for slow ZFS array with spinning drives. F20 is even better because is with SLC-NAND.
I got a intel dc6300 ssd, the 1.6tb version for $160 with Christmas money, it’s a great ssd. Intels M.A.S utility is easy to use and shows real time stats. If you can find one I would recommend buying it
Test an ioDrive2 - 785G vs these drives ... Would be interesting to see which performs better as cache-drive. The ioDrives have almost double the IOPS to the Sun F80 and almost 25% more IOPS to the Micron P320.
really cool if you have ZFS raid-Z and not enough sata connectors for extra cache drives(which you will want to be in raid as well)
I've had one of these boards as a 4x200 GB drive in a test machine back in the lab for a number of years. Enough space to install an OS and try out a few things in each of the cells. If you're looking for a secondary source for these cards, check out the older pre-Oracle Sun server rackmount hardware. I recall these cards were used in redundant pairs in Sun X4500 and X4540 servers. But, be warned, those boxes are BEASTS, perfect for that collection of 48 3.5" SATA drives you have laying around that need a home... kinda surprised that Jeff doesn't have one in his rack at home. :)
bro how did you get an os to boot of one of these drives? i have a f20 and it refuses to boot
Great video, I have been using IBM LSI 2008 cards for my freenas server and freenas server backup for several years. I have never re-flashed the bios in Linux as the writeups were for win os. Your use is different so I understand for using in IR mode, but I have always used IT mode to get the full benefit control and error checking of ZFS. I always have at least one extra LSI card Just in Case....
Thanks for the video
Ah wonderful! The head of the ‘F80 Owners Club’ made a video about it :)
Neither do I have a Sun F80, nor do I currently have a PC with linux on it atm. But I followed all the steps that don't require both of these.
Cheers.
13 months later.....Hey Jeff, are you still rocking these cards? any hiccups with updates?
That is absolutely phenomenal for a metadata/dedupe drive for TrueNAS
Enterprise drives are like the best! Picked up a HPE IODriveii 3.2tb for like $200 bucks best of all 37PB of endurance.
3d print your I/0 shields works fine with hign/low shield bracket problems.
My thought exactly. I would 3D-print and just use wire to ground to chassis just to be sure.
Buy a shield for $6 or a printer for $200+.
Hmmmm, choices, choices.
(Believe it or not, many people still haven't a need for 3D printers. They may be handy, but they're not for everyone.)
Hell, I could carve one out of sheet metal too, but there's diminishing returns to think about.
@@TheTalonts quite obvious that print if you have printer 🤦
if you can get them, Micron P320h drives have much lower latency, higher speed, and can deliver more IOPS comparatively, though they are definitely more expensive and are becoming pretty rare.
There is also a Fusion-IO 1.2 GB app accelerator which works well as a cache or SSD storage option as well. I hope you look at more of these high end durable NVMe storage options since many of them are very cheap on eBay!
A friend of mine that has some said support in TrueNAS/FreeNAS for these is lacking.
Perfect timing. I literally just walked back in from getting my mail (which contained 2 Sun F80s that I ordered before your video drove prices up and availability down) So it looks like I need to start flashing. Thanks for the tutorial! Any plans on similar videos for the Fusion IO devices out there?
@4:53, i love how you use a floppy drive as a drink coaster. this alone earned this video a like.
Watch starting at about 19:50
@@coletraintechgames2932 awww what a let down LoL
Thats not a floppy disk, its a save icon 🤣
You know what I would like to see? A PCI card that can use DDR4 memory for storage. There use to be cards like that made, but no one does it anymore and I have no ideal why. It is awesome having a dedicated chunk of memory that you know 100% will only be used as a junk drive. You can do the same thing with some programs that use your system memory to create a temp drive, but they don't always work so great or in every operating system like a PCI card can.
Thanks for showing us how to set up $100+ drives. Hope they go back down in price.
I'm thinking about grabbing a couple for my "big" dual opteron libvirt box for VM storage. Its currently running on spinning rust with a nvme Samsung 960 EVO 250GB lvmcache. So its not GREAT but not SUPER TERRIBLE. A couple of these enterprise cards would kick it up several notches.
Jeff... You need a 3d printer. No need to source brackets when you can print them! I've had to do this with some 10gb Intel Ethernet cards.
I managed to pick up an optane 900p before the 905p were the only thing you could find. At the time it was only like 260 for a 280GB capacity - best zil and l2arc ever
Nicely explained, very nicely presented, and deftly executed. I'll be looking at other videos from your channel, and will likely subscribe after that. I'm none to keen on Windows, but don't resent that you use Windows tools when they suit your purpose. Pop OS has piqued my curiosity. Thanks for that.
EDIT: as Wtdtd observed, yeah... prices have shot up more than double since this video was posted. There goes a cost-effective solution, swept away by market forces. Eh. It happens. Still a good video, and, who knows? The price may drop after the initial rush wears off.
"So we don't have to wait, here's one that I prepared earlier." You... yep you did it.
I'm glad I'm not the only one that uses nano. I feel like literally everyone else uses vim or emacs.
VIM is for sadists in my opinion. Nano works great for me as well.
I don't need to do lots of work in a terminal editor, nano is simple and ubiquitous, I like it.
I basically always use nano on the cli unless I'm forced into vim.
@@JarrodCoombes Vim is way faster for actual text-editing work, and works well as an IDE (not as well as EMACS, but still better than most of the paid bloatware IDEs out there). For the use here, Nano works just as well.
Learning to use "text objects" correctly was what hooked me on Vim, although I am considering moving to Evil EMACS, which is EMACS with Vim bindings. Best of both worlds.
I find that Nano is good for those who have only been exposed to GUI text-editors before, due to its familiar control-scheme and keybindings listed at the bottom.
15:40 2 updates are available for your system.
just FYI.
Why people would downvote this video I have no idea, this is technical stuff, why are you even watching?
e: I also got the SAS address checksum error, but my F40 is now working as expected after loosely following the same guide you did.
Think I caught you slackin, Jeff. You said ServeTheHome, but you're referencing a ServerBuilds.net post. At any rate, can you add the links to those posts to the description? I've had an F40 laying around for a few months at this point that claims to have something like 100 million reallocated sectors, so I've decided to dust off and try and reconfigure thanks to this video. And I scooped an F80 for good measure.
I just picked up a F40 and was wondering if the process for it is exactly the same as the one listed for the F80 or if any other changes or files are required? Thanks
@@Thetruth-di2ic It's not super different, but there are a few extra steps. I used the F80_Files zip package from the ServerBuilds post because it had all the tools available already, but the Kasilag post ( kasilag dot me/warpdrive ) has direct links to the F40 firmware. You need to split the firmware as described in the Kasilag post (requires trimming a bit in the beginning and a decent amount towards the end of the file), when updating the sbr.cfg file to change the PCI IDs, you use SubsysVID = 0x1000 and SubsysPID = 0x0581 instead of SubsysVID = 0x1000 and SubsysPID = 0x0504, and then you'd swap out the references to the F80 firmware with the F40 firmware. I'm probably going to wind up writing a guide specifically for the F40 anyways if that's not clear enough.
Reminds me of when IBM M1015's were dirt cheap and great for ZFS. Might have to grab one of these to play around with but I suspect a bunch just sold =D
That is freaking ridiculous... The prices have nearly doubled... 🤦
We call it the Jeff EBay Effect
"I'm a influencer"
@@CraftComputing Jeffluencer
@@CraftComputing I would actually be honored if you would start these kinds of videos with "I'm Enterprise Hardware Price Influencer Jeff."
On Ebay in the EU/UK, these are £120+, or $165 USD. And the F40 model is about £90 / $124 USD. So you probably got lucky to find yours for $60...
Still a bargain for an fast MLC SSD.
May want to check again just ordered a F40 for £55 including shipping
Great video, too bad they're all gone now tho.
Back in like October? I was able to score a 1.6 TB Intel P3605 for $160 been an amazing boot drive for my desktop!
I have yet to find a better deal than the p3520 I got for 125 bucks. Itd been listed for 2 months at some enormous price, and slowly lowered by 20 bucks at a time. I guess they eventually gave up and just dropped it to 125 bucks - i immediately picked both of them up, and even with their less than PB endurance, i just dont see me using all that endurance before needing more space. And better performance too!
I know you'll love the news from nvidia allowing GPU pass through now 😎 expecting some cool videos to come out from you on it all now :P
That's so cool!
@@nathanscarlett4772 ikr, I was surprised as hell to hear it. Not sure of any restrictions around it- but it'll make a great video :)
@@buddybleeyes indeed good sir! This is the stuff the common man needs to switch to Linux full time...and still be able to play video games 😃
@@nathanscarlett4772 agreed there dude, Linux is sounding better and better with this news :P I'm heavily considering doing the switch in a few months
@@buddybleeyes the paranoid person in me, is also cautious because it is Nvidia, they nickel and dime everyone for everything.
They're 150 dollars on ebay now! Guess they noticed the price of uptick in sales since your video was posted
4:30 you sipped - I noticed that.
My coaster is a 5¼-inch floppy disk - IBM DOS v.3.30.
Have you tried the Performance overprovision setting? The performance difference might be worth depending on how much storage it removes.
how would to do that on a sun f20? i got one of those and it has 4 32gig drives but they have like 4gigs unused on each becaus of over provisioning
having that radiator exhausting right next to intake fans makes no sense to me
Thanks a lot for the guide, found this old tech about 33$ here in Indonesia
It's crazy to see that an used card like that has a price tag of US$ 1,663.59 in Brazil and US$ 60~140 in the United States
Had i not just put together a 4x110GB Optane raid card for caching i'd have jumped on this, as it serves the same purpose two drives for write caching/additional protection against power loss, and two drives for read cache/array load reduction
Why do people using AIO water cooling usually have just as many or more fans than people using air cooled heatsinks?
"FLASH ACCELERATOR" I've never heard about them ^^ Interesting you can store the components on a hard drive (like a 16bit cartridge) and then compile them into a quad system to work together, or build a game with different components and after compile
neat little cards, those things. I have a pair of F40s in my Dell R710 I use for VMs. kinda wish I'd gone for bigger ones but oh well. I seem to only get like sata 2 speeds out of a single VM. might be something to do with LVM. idk. every once in a while I might notice a GB/s write speed for something.
they do get a bit hot though. think I got my fans spinning at 3k to keep them happy.
I bought these floppy disk coasters too, they're nice. And then I realized I have a massive box with hundreds of dead floppies... Guess I can throw a really big party with them.
I don't really have a use for a home lab right now, but i did order one of these and a bracket that ill be using for a game drive, $65 after accounting for the bracket is really nothing for such a nice drive, even if it isn't as fast as modern nvme drives.
Jeff, be on the lookout for a tiny little lady wildly swinging a 9 iron....Lately my better half has become quite irritated with you, (and ebay, paypal, UPS, FedEx, Amex etc...)
Now to figure out how I can extend a PCIe port to a remote location in my NR12000 so i don't have to give up my 4port NIC.
The format did not work for me and just gave me an error so after redoing the process - getting the same error I just rebooted into Windows. In Windows I used the ddcli to format and this time it did work. Yay 800gb ez ssd!
These drives are now $135... Magic of sharing something on TH-cam.
I don't know if it was mentioned, but you might also want Linux to help with the install. I guess you could use the win 10 command line as it takes bash commands, good luck though re-writing the software.
this is awesome value for a nas on a 10gb network as a cache as it can still saturate the 10gb net bandwith :) thanks for the video, I might need to have a look at this
I got one of these a couple years ago for around $90. I'm currently using it as VM storage on a Proxmox server, with the four SSDs configured as a ZFS striped array. Works great. In my case having it in IT mode is exactly what I wanted.
You can also get 400GB versions (the F40) for a bit less. Also, there's the F20, which has only around 100GB of storage (wonder why it wasn't called the F10...). F20s are usually quite inexpensive (often less than $25) and can serve as a good ZIL for a ZFS array (since ZFS by default flushes every 5 seconds anyway, so you don't need much storage for a ZIL) If you search for LSI Nytro, you'll also find 1.6TB and I believe even 3.2TB versions of this which have up to 8 SSDs onboard to achieve that capacity. (I imagine one of those cards could easily achieve the SSD speeds we currently see from PCIe 4.0 NVMe drives!)
This concept was actually quite popular for a while. OCZ's RevoDrive series uses the same concept (putting multiple SATA-based SSDs on a single card intended to be RAIDed together). FusionIO cards also are flash-based accelerated storage. Of course, today we are seeing the same approach but with NVMe SSDs - it's possible to get >20GB/sec with an 8-slot PCIe x16 HBA supporting NVMe drives...
yeah i bought a sun f20 with the hopes of using it as a boot drive but it doen't work as a boot drive and idk why, the data sheet says its suposed to be bootable
@@firenado4295 It may be possible to boot from it, the main issue is the F20 is based on the SAS1068 chipset which, among other limitations, doesn't support native booting on UEFI systems if I recall. On BIOS based systems it will present a BIOS extension that you can access to configure booting from it. The F80 is based on the later SAS2008 chipset commonly seen in PERC H series cards popular among homemakers, so it can boot a UEFI system just fine (might need to flash a UEFI boot ROM, but I know I've successfully done it as an experiment)
@@fdmillion ah cool at the moment i have my f20 in a 2006 dell dimentions c512 and i think thats old enough to just use a bios and not uefi. but idk what to do really starting to regret this perches to be honest lol
@@firenado4295 yes that system almost certainly uses traditional BIOS. When you power on do you see a few messages showing the card initializing (it will usually show the four modules and a copyright notice for the firmware - either LSI or Avago depending on revision)? If so it should give you a key you can press to access the cards configuration, where you should be able to setup boot ability. If you don't see the init messages at all go into your regular BIOS setup (F2 on most Dells) and look for an option related to "Option ROMs" and make sure they are enabled. Hope this helps!
Would a typical 120mm fan wall in a 4u chassis be enough to cool this card?
I would assume so. Like I said, it doesn't take much airflow to keep them cool.
they advertise em as memory buffers, cache, and that no one is really looking for, I bought 3 cards cheap years ago because the info I found said I could use em as a drive, not just cache.
The power of Jeff, some guy had 20 of them on a new listing for $89. By the time I had switched browsers he'd sold 16 0f them. I bought one and I suspect they're all gone now. I bet he didn't even know why his entire stock blew out on eBay in under an hour. I'm just glad I got one at only 50% over what you indicated. Please keep on making content around old enterprise hardware. My test server has to live in the living room with us so I have to re-case and re-fan to keep the noise down. It's running TrueNAS (FreeNAS) and it's just a hoot playing around with what it can do. I'm probably going to use it split as dedicated cache for two arrays.
stayed to the end cause it was interesting... then found out this guy also teaches how to make a new drink ! nice
I usually scream about how terrible IPAs are at the screen, but sometimes he makes an awesome cocktail or pours a stout.
I know this is a pretty boring request, but do you have any recommendations for rack enclosures and what good/bad things to look out for
Man, this guy is great. I love the drinks tech :)
I love that small IT-blurbs just before the beer reviews, cocktail showcases and such :)
Nice addition!
I just realized I made a stupid mistake.
I bought one of these SSDs to use as a cache-drive for my home-server and when I wanted to install it I realized I don't have any unused PCIe slots...
So sell it for a decent price on any homelab site. If you didn't overpay there will probably be people to buy at your cost.
going to one for each of my poweredge servers..great find! I'll be using them in IT mode..:)
this particular hardware didn't interest me much so just skipped to 18:00 for the drink :) there is a Ward 8 bar/restaurant in Boston near North Station. used to walk by it when going to work. and yes, it was for the state legislature. there is a wikipedia article on it though only mentions 1 teaspoon of grenadine
could you leave the bracket off and just use a pcie cover like for when you don't have anything inserted in the slot?
Really depends on your board orientation, if the Mobo is mounted flat(horizontal), there is really no need for the cover, if it is mounted vertically (like normal tower case) then it adds support, these x8 cards don't lock in like x16 cards so the PCIE end cover secures it in place, without it, it could fall out as the pcie contact spring force is pretty weak and the only thing holding on to the card.
Drive age, not usage is the determining factor in drive health. You should read the google study on SSD's
The brackets are 3d printable, i just got done making an rx 480 bracket that prints flat.