The genuine hilarity that ensued, and my own catharsis at having experienced that same "I'm not a smart man" moment in my own tech misadventures was utterly priceless.
Linus Tech Tips can make a vid 20 minutes long about a bigger, way more expensive unit and I'll probably watch half. Your video is more than an hour and I'll watch it all like a fine movie. You're a special kind of TH-camr, keep the good work!
LTT at the end of the day is really entertainment. There's a lot to learn here, whereas watching Linus might get you some laughs but not much knowledge.
@@DrKratso I love LTT videos but I feel like over the past year or so especially their videos about this kind of stuff really moved away from the homelab/home user level and into the enterprise grade scale level. I mean while it's neat and what they as a company need it kind of now leaves a gap for the regular joe server/NAS/DAS/JBOD builds which videos like this fill in amazingly well. I almost kind of look at it like CRD as Home-User/Hobbyist level, somebody like Jayz2Cents as sort of Power-User/Small Home Business level and then LTT & a few others as basically Enterprise level stuff.
@@DrKratso I've been watching a random assortment of LTT videos over the last year for no particular reason other than they keep showing up as the next autoplay video and I've noticed just how badly their videos age. Production quality has barely improved in 5 years. Not the video quality, the content. They handle everything like its okay to throw things around and hit them with hammers. You are definitely right in it being only entertainment, its the tech version of EBLR.
i can't believe that I've watched 1 1/2 hours of a guy talking about a video server in so many unnecessary details, going about half an hour of history before getting to the original point i love it, if i had the will of making videos, it would be like this, 100℅, i never, and i really mean it NEVER felt like i was watching a video of someone that was like me, all of the thought process, the battery discovery, the complains about servers in general, pointing out the path that you needed to follow in order to get to the bottom of it i truly love it, this is only the 2nd video of yours that I'm watching, and i already love it, i wish i had some more money to afford a patreon membership, i just LOVED every second of the video
Hi, is it normal that I don't understand 95% of the tech that you talk about but still love watching your videos ? I am a dumb airline pilot in Lima , Peru. Love your work , keep it up !!!
I was so nervous that you wouldn't include a "two of them" in this video. finally, over 80 minutes in, I am satisfied. I loved this video, I thought it'd just be good for background noise, but you actually covered some things I didn't know existed, and I got totally sucked in
I will have a lot these videos playing while doing house work. No matter what I’m doing, if I hear “two of them” I look up to see if the meme is there. Edit: JUST SAW IT I’m so happy
I am three minutes in and already hooked. Let's see where this rabbithole goes. EDIT: 1 1/2 hours later, and now I have the strong urge to upgrade our home NAS. great Video, as always!
Been a long time user of Freenas (now truenas) for many years. And WD Red drives(5400rpm models). power efficient, reliable as heck and performance is not too bad really. And doing automated backups has been pretty easy, doing snapshots of changed data every day takes far less time and is often just a few seconds to a minute unless I have done something significant. Than its a little longer. And yes I mentioned my nas drives are 5400 rpm, and I do have 10g network. But as a small family its always been quite sufficient. I also use SLOG and L2Arc (flash based read and write sync caches) and the nas has 48 gigs of ram. So that itself does a lot of heavy lifting. Loading in a couple Tibbs of data for backup isn't too bad when it goes at about 600mb/s Truenas and ZFS also have paranoid levels of paranoia when it comes to data integrity. And as long as you stick to recommended practices then that data will always be safe almost no matter what.
I am extremely sad now that I've realize that we've gotten to the point where it needs to be specified when a link is NOT an affiliate link but I greatly appreciate the integrity in providing them. There is inherent bias in providing affiliate links and I never feel comfortable buying from anyone who gets paid when I make a purchase
@@pyrioncelendil yes but that defeats the purpose of why you don't buy from someone who posts an affiliate link. If it is an affiliate link the prospect of money is the motivator of posting it and you cannot be sure they truly recommend the item because it's just as easy to post and say you like it and still get money. Deleting the affiliate info onto stops them from getting paid for your click, it can't change their reason for posting the link in the first place. By the creator *posting* a non affiliate link, they show that money could not be a motivator since they do not expect payment.
Dude, I could seriously listen to you just talk all day. Not just because I'm a tech nerd myself, but your presentation and enthusiasm for the subject matter is so genuine. I also love how you cover all the bases and thought processes someone would would realistically go through to reach the final... Uh, result of whatever we're doing in the video.
28:23 For the dock I'd suggest you 3D Print a removeable spacer for the drives to support the connector. Or get somebody to print one for you. It would be an easy design & print for anyone who does their own designs.
Was going to say the same 😁. Literally 15 minutes to model and maybe an hour to print. Could also, just as simply, make adapters to deal with The T9 drives not fitting into the Small Rig clamp.
@Cathode Ray Dude [CRD] I just caught this video now. I'd do it for him. No charge. Just have a chicken and egg of not having the drive or kingston bay/hub to measure. lol Could probably work off just a caddy and drive model. @Cathode Ray Dude [CRD]
To be honest, the dock looks like a prosumer version of those usb copy stations they sometimes have at merch booths to have concert recordings for sale shortly after they ended.
I loved how you reacted when you discovered the battery door. Such genuine laughs. I'm glad that you didn't hide your mistakes and show your true self after discovering that you did something wrong for years for basically not even doing the basic RTFM. The contrast of a noobish mistake and the general well researched information dense documentary type you usually do was just perfect. This makes you even more genuine and lovable as a TH-camr.
Just wanted to complement your going the SFP route in the network interconnect for 10GbE, it has lower latency than 10 gig twisted pair so nice little bonus there. Also direct attach cables really do feel like a cheat and we use them extensively inside and in-between racks; there's a whole world of those cables now, including thinner versions to improve airflow, colored and plenum-rated jackets... congratulations on your workflow upgrades, cheers!
SFP modules are just dumb unless you need copper or fibre for longer installations. DAC is the bees knees. They require much less power (less then 1% of a 10gbps ethernet sfp). Both optic and ethernet/copper takes alot of power to drive. Not a big issue with 2 connectors, but when you haev 24 or 48 in a rack the power demand is significant.
@@VCD-Channel A lot of little things. 1) In a commercial environment, where you're going to have potentially lots of SFP modules packed in to a dense switch port cluster, the transceivers we have today require too much power, and get too hot. It breaks the limits of the SFP+ spec, and while most implementations can handle staggered deployment (so every other port, or something like that), it's just really hacky. 2) We've kinda pushed twisted-pair signaling to the edge of its usefulness here. It's a fine topology for baseband, or 10Mb, 100Mb, or even 1Gb networking. But at the bandwidth required for 10Gb, something like coax (which, ironically, we abandoned for UTP long ago in conventional LANs) is much better suited. It's difficult to guarantee reliability at the traditional Ethernet specification of 100 meters, and even that's assuming you're using a recent plant with quality cabling that was installed carefully. Real-world cable plants are often going to be CAT-5E or CAT-6, with similarly-rated jacks and patch panels, and a lot of potential weaknesses like kinks, cinches from over-zealous zip ties, too-long runs, crappy patch cables on the ends, etc. 3) Given the density and integrity issues, many professional vendors haven't focused on 10Gb-T, so your options for 10Gb-T switches are limited compared to SFP+. This is my bread and butter, so I used to sell a lot of switches like the Juniper EX4550, which was a 32-port SFP+ switch with two modular bays where you could install an expansion card with (your choice of) 8x 10Gb-T ports, an additional 8x SFP+ ports, 2x QSFP+ ports, or 2x chassis stacking ports. Super versatile! But, that was discontinued, and there just aren't a lot of models out there where you can casually add a handful of 10Gb-T ports, without committing to an entire switch full of them. This is particularly annoying when there are just like one or two appliances that need 10Gb, but don't have any SFP+ slots. That's the worst. A lot of people need or want a healthy dose of 1Gb ports anyway for legacy or low-bandwidth appliances, or out-of-band management interfaces on their high-bandwidth appliances. So you _could_ just buy a 48-port 10Gb-T switch and run most of the ports at 1Gb, but that's going to cost a lot more than a similar 1Gb-T switch, so your only true recourse is to populate some of your valuable SFP+ slots with 10Gb-T modules -- and that just feels like the IT-industry equivalent of sharing prescription medication.
Such an amazing video! Inspired me to change my approach to scratch disks, 'negatives' and backups. I especially liked all the preemptive "yes, you might be able to do it better, but this is how I do it and this is how I like it" comments. Definitely shows years of reading YT comments that contain simplistic and sometimes condescending suggestions to complicated problems 😁 I have a fairly similar setup at the moment (Atomos Ninja V recording to a 1TB SATA SSD), but I use the "negatives" for editing (love that term). Sometimes I would actually move the files to my NAS (with two NVMe SSDs in a RAID0 configuration) and edit off of that, but both of these workflows are very fragile and prone to data loss. I'm thinking about adopting a similar setup that you have, and since I already have a NAS, this should be relatively easy. Once again, I really appreciate and respect the effort that you put into making this huge (and very insightful) video.
Thanks for not leaving out your past mistakes! Personally, I love my server rack even though there isn't a server in it :P I use it for my 19'' audio gear and some battery backups for my PC. I also put a shelf on it to store random PC accessories.
I used a two post rack for my home theatre gear, Crestron gear, a laser printer and UPS for 10+ years. It was hidden away, so it wasn't visible from the main room. Unfortunately in the new house, it's not possible so I had to get a regular cabinet.
When you started talking about flashing the PERC RAID controller to IT mode, I was like, "god I hope he shows the bootloader for the flashing utility" and at 1:17:00 I started busting out in a fit of laughter and clapping. Its so unbelievably cursed and I'm glad you shared it with the rest of the world.
I subscribe to several channels where the creator really cares about video quality. At least enough to mention how much they care about it. I watch everything in 480p and lower. Now, if a creator has a terrible mic, it's painful to have an interesting video distracted by awful audio.
Well, sure. But the 480p version of the 6K footage looks better than the 480p version of the 1080p* camera from the 2000s, noticeably so if I may add. Hell, any High Production Cost Video will still look ridiculously good, even in a low resolution. I mean, look at the LTT videos nowadays.
Camera quality is actually pretty independent from TH-cam quality setting. You can tell when the creator has a high quality camera even when viewing at 480p. Its sharper, the colors are better, there is less noise. So many other benefit than just resolution. Not that you are wrong, the audio quality is way more important. But once a creator has good quality audio, they might as well start improving video. Bad video might not drive people away as much as bad quality audio, but the viewer still notices and responds slightly better to high quality video.
@@DoubleMonoLR Shooting in high resolution allows you to do things in post that would be...messy at a lower resolution. That doesn't mean the end consumer needs that max resoultion, like they don't need the content delivered in 1 gibagit/sec Prores
@@LocalAitch True... we did a music video once that emulated video look and I wanted it as close to SD as possible and exported it as 576p50 (as I'm in Europe). Looked the part... TH-cam made it 480p25 which doesn't make any sense and looks horrible. Had to upscale it to 1080p50 :D
Video pro here. Your desire to do things right is something I admire with this project. Have you considered a Proxy workflow? The reason why it was originally created was because even though most workstations (even mobile) were able to playback full resolution media the storage attached to them couldn't handle the size and data rates of high quality intra frame and raw codecs. Just going from your ProRes 422 HQ to ProRes Proxy would mean 1100Mbps -> ~225Mbps and that's before you reduce your resolution. Modern Nvidia, AMD, and Intel (including integrated. Quick Sync is shockingly good) GPUs can playback h.264 footage with great performance. If you go down that route, you will only need a few dozen extra GBs with your media for your entire project to have lower res high performance playback. Resolve 18 went stable and had a point release already. It has a revamped proxy workflow that you should take a look at (see Resolve 18 reference manual pages 188 - 192). It can now automatically detect and link proxies on and after import and comes with a little proxy generator app complete with watch folders. Generating proxies is something that can be automated part of ingest too, you may already be familiar with ffmpeg, so it can be done server side too. In addition, if you want something a little less fiddly than an entire server to administrate for the sake of having more space for video production, consider a Blackmagic Cloud Store/Pod.
Fun Fact: 25 years ago, I worked in a post production house. A place to edit ads for tv, and corporate videos. To edit the videos they used AVID in Quadra 900. The amount of storage in the external disck was 32GB. I Guess this amount of space give us something like 6 hours of video storage at that time.
Reason for the 2 connectors on the drive backplane: SAS supports a feature called Dual Port, it's basically a second cable path to reach the drives for better reliability. It might be the multiplexer (or whatever Dell calls it) has a fault that makes it choke on DP.
As a community college ICT teacher, I have a whole bunch of these kind of 2013-2017 era servers (we get them for free from companies that get rid of their old equipment) and never know what to do with them, but this gives me some ideas. I can make a project where the students have to make a solution to store footage from a security camera
I just recently found your channel, and I have to say, your presentation is excellent. You're one of the shockingly few people I will listen to for an hour and a half straight and not get bored. In fact, as I write this, I am about 2.5 hours in after watching your video on that absurdly long IBM portable with the attachable printer. I'm really glad you've dedicated yourself to this.
Yes! New CRD video! Edit: I too started with a Sony handycam (cx240e) which I still use and I hate it. When I bought it I didn't know alot about camcorders but one of my criteria's was 50p/60p 1080p video so it was a good starter camera, so when I saw one for 65$ I bought it. I might mod it for a mic input because the internal mic is not that great.
Long time viewer, first time commenter. The fact that your workflow is this complex doesn’t surprise me - high quality and fastidious attention to detail are your hallmark at this point. What does surprise me is how vast the delta between your workflow and mine actually is. ~200 10-15 min 4k videos (across my multiple channels) all render out of my 256 gb m1 Mac mini each year and make their way onto TH-cam. Really enjoyed the insight, but I can’t say I’m not glad to have stayed clear of this rabbit hole! 😬
1:05:01 Modern fiber can take a lot of abuse, way more than copper. I’ve slammed cabinet doors on x connects all the time in a data center and they work just fine! You can also get bend insensitive fiber that can be tied in loops without much attenuation. Btw you can get 40G and 56G infiniband hardware for pretty cheap on eBay once you outgrow 10G Ethernet
So, I actually purchased two servers from the community college I graduated from, and each of them were quite interesting. They both were Wide Orbit Radio Automation Servers, which were used for the college's radio broadcasting station, and one came with a 1TB HDD with songs and Licenses for the station to be able to broadcast those songs, and the other came with three 2TB HDDs, two of which were chalked full of songs as well, and the other had licenses for each of the songs on the other two 2TB HDDs, as well as each containing a Samsung EVO 850 250GB SSDs (I'm guessing so they can boot quickly), each of them also had a PNY GeForce 8400 1GB DDR3 PCIe 2.0 GPU (with a video port that looks quite strange to me). One of the servers had an i5 - 4570, with 8GB of DDR3 Ram (1x 8GB, Single Channel), and a good amount of SATA cables sprawling from the five (5) drive bays to many of the SATA ports on the Motherboard. But, the second server that I purchased, which was the same brand (Wide Orbit) and server model, came with an i7 - 4790, 16GB of DDR3 Ram (4x 4GB, Dual Channel), and a PCIe card that is quite similar to the one that 01:20:26, with the plug that has four SATA cables coming off of it, which were attached to each drive bay, with the drive bay containing the SSD having its own SATA cable plugged directly into port 0. I know the hardware is old, but I still can't help but think that I got the deal of the century with these beauties!
I'm so glad I opted to watch this instead of some arbitrary Netflix filler tonight. Got to admit I was impressed at how much self-awareness and hard honesty was used, but it helped define the problem so much better, and was great to see such a satisfying solution!
Love iDRAC and its equivalents. I mandated that all new servers at work have licenses for it, they were cutting it out to save a few hundred bucks per server. I am a firm believer that the cost is well worth the amount of time you save vs having to physically go to your server or mess with a network KVM. Nothing like installing the OS on a server from your couch at home!
When I was first learning server shit at work I discovered one of our servers had an iDRAC license and instantly fell in love with the features and it's saved me so much farting around. Now that I'm actually in charge of specing out our servers, we don't have a single one without that license.
I so wish Intel's AMT was more widely available too, it's not full iDRAC/iLO functionality (especially if you have an i3, but some of the i5+ only features can be unlocked with MeshCommander). I have a standard HP elitedesk 800 G2 mini PC running as a server that I can just pop open a serial-over-lan terminal or boot an ISO image over the network and it is SO. COOL.
Hate iDRAC, other integrated server management hardware/softwares are better. Never seen so many iDRAC hardware failures over the years. I have had to replace 100s of Dell PowerEdge motherboards over the years, simply due to iDRAC hardware failures. For whatever reason. iDRACs are prone to failure. There are ways to reprogram the iDRAC firmware by externally attaching a programmer to the iDRAC firmware chip or even reprogram the iDRAC via sd card, but it is more painful than simply replacing the motherboard. iDRAC blows, but hard to have hardware lever remote server management without it. The iDRAC free verson works, but the Enterprise version is much better, with more features (and btw is easily hacked to become free, lol).
A video this long has no right to be as entertaining as this was! Super interesting topic that had me hooked right from the start and till the very end.
Since you have ZFS, you might improve the archival copying by using snapshots and sending them instead of using rsync. You snapshot the scratch which gives you an *exact* state in time and can zfs send/recv to the archive. You can also mark the recv'd dataset as read only and avoid messing with the permissions. Look into snapshots and sending datasets; I think you might find some value there.
He also might want to consider playing with TrueNAS, possibly scale if he'd like to stick with Debian over FreeBSD and have native containerization and a better hypervisor. Doing ZFS on raw Linux is fine, but given how developed TrueNAS is as an appliance, it's worth him checking out. I know he says he doesn't want to learn anything, but with what he's doing, he's just getting access to a gui form of what he's doing in the command line. Also, the irony of him saying that TrueNAS might break is that manually configuring ZoL will mean it's more likely he could accidentally break something. I'm sure he's competent in the command line, but there's a reason OSes and Applications have GUIs. Unless he was a NixOS kid, then I'd say he's fine configuring ZFS entirely from the command line haha.
@@metaleggman18 Maybe some double checking if recursive datasets work fine in the containerized environment. Proxmox had a bug when dealing with recursive sets that is just now being patched with some updated code. But I agree using a hypervisor instead of running it directly on the bare metal would be a good idea. I used to be pretty gung-hoe about everything running on the bare metal until I had to come to terms that I'd like to not deal with some packages with pedantic requirements that can break the whole system, and instead just build a container for that service and leave it alone until I want to maintain it instead of being forced to do it during a universe upgrade.
Specifically, just get sanoid/syncoid and it can all be automated quickly: Sanoid can manage snapshots, taking snapshots daily/hourly/weekly/monthly/whatever and pruning old ones according to rules. Syncoid can sync those snapshots from a source pool to a target pool easily.. I do something similar: Mirrored NVMe drives hosting VMs, taking a daily snapshot and keeping 2 days of snapshots, and syncoid doing a daily sync from the NVMe drives to some spinning rust, where it holds more than 2 days worth of snapshots. Sanoid is basically just automatic. Syncoid is just going to replace rsync in his workflow.
Thank you for actually saying "couldn't care less". Also, very informative video. This has really helped as someone looking to increase my 4K video capacity and backup years of photos to something better than a random external HDD connected by USB.
This is awesome, dude. Ten years ago, I was teaching myself basically everything you cover in this vid. Had I continued down that path, this is exactly the hardware and knowledge base that I would have achieved. As life goes, directions change along with needs and priorities and such a beautiful solution never manifested for me. You may have inspired my to tear into that mountain of n-egg boxes, drives and cases hidden in the closet. I'll have to consult my cardiologist first LoL. Great job on the system and video!
I have loved every video on ltt about the ridiculous servers ,networking and the nitty gritty of their production, but they are criminally short... So having a lengthy explanation of your workflow and the fantastic description of the trials and tribulations you had to go to achieve this is delightful! Thank you for this fabulous production, it is truly enjoyable, entertaining and informative. P.S. I hope we get an update once you get the transcoding directly in the server.
LTT desparately needs a professional admin. Someone who will tell them "no". They're not stupid, and they all have clear technical ability. But professional administration is another thing entirely.
This is good. Respecting your own process and building to work with it. Also I spent two years in an office with one of those severs at the end and although I became immune to its screams, people who called me on the phone or tried to have a conversation rarely failed to remark on them.
This is probably one of your best videos so far, my dude. As much as I love your retrospectives, histories, and general deep dives into bizarre and interesting pieces of technology, your aptness for storytelling mixed with personal problem solving made this entire video a trip. It certainly didn't feel its length in the best way possible - I was with you every step of the way and was excited to hear what came next again and again. I normally don't comment, but I have a terrible feeling this video won't perform like your others because TH-cam is garbage - but know that I, personally, enjoyed this video immensely.
or print your own drive bay with a few right angle usb-c cables going to a pcie usbc controller in the server, plug them in and the script copies the stuff on connect!
I stayed to the end . I learned a number of things to USE and to LOOK OUT FOR . I can't use EVERYTHING here, but there WERE some things that will really help ! THANKS !
As someone who is not very interested in storage management solutions, this was a very entertaining video to watch. I loved how you detailed your chronicles in the sphere of content creation, and your explanations were simple and apt so that I could follow along. Great video!
This Rube Goldberg setup to use those SSDs with your blackmagic camera is hilarious to me. It feels like a professional version of my very first video setup, my very first """"""camcorder""""" was a webcam and a cheap computer mic taped together and wired to a laptop that I carried around in a backpack
@@nikGhost1 Three rolls of Velvia 100 and Velvia 50, one roll of Fujicolor 100, Puyo Sun for the PS1, Sega Saturn, and N64, and a Minolta Weathermatic 35. Oh, also a Puyo Sun guidebook.
A very entertaining watch on youtube. I was thinking about getting a two rack system, but, the noise would keep the neighbors awake, and also me and my wife, since we have not got any basement where we live. Very good information you give out with the problems that come up with these servers. Thank you for doing this.
Experience is the best teacher, and you teach by sharing the experience! You go deep in the weeds, but anyone watching your videos can avoid having to go through the same pain building a NAS!
I actually had a segate rosewood (it's basically the data equivalent of a hard drive with a bomb with a random number generator attached)(also very common hard drive) and SMART actually saved my data before anything catastrophic happened, at least i checked crystal disk info and it was throwing a yellow caution and I was able to purchase a new drive and move the contents before it ruined anything. only had about 20k hours because it started to fail.
This video has SAVED ME from learning about the Crucial BX500 the hard way. I placed an order for one, and then the feint memory of this video started coming back to me. I came back to this video to check what SSD was causing your problem and lo and behold it was the same one I just bought. I cancelled my order and have my money back. Thank you Mr. Dude for sharing your technical trials and tribulations
I knew there was underlying reasons I didn't have a youtube channel. It's not just laziness, lack of interest, or too tired, it's because I would not know the first thing about storage and making videos. Yet I start off watching your videos thinking I will move on when I get fidgety, but no, I actually like your videos even though I understand about 2 per cent of what your saying. I still find not understanding interesting, because of the way you present stuff. Keep up the good work it's very very interesting stuff. One last thought, you must have a very good computer very costly and extremely fast internet speeds, but then again it would not have to be super fast to be faster than Australian internet speeds. But yes I can imagine your super dooper desktops and laptops doing such wonderful video work. Only the best, but you deserve this your videos are again very good and entertaining.
They make CFAST TO SATA3 Convertors for the Black Magic Cameras. Since CFAST uses the sata bus it is just a connector changer. That is always a good way to get more storage. You can also do the usbc to nvme enclosures and put a 4TB ssd there as well.
i love your content! your delivery is incredibly entertaining. Time flies when I turn on one of your videos! Great pacing, derivations and tangents are awesome, and your passion is obvious. One of my current favorites.
Congrats on getting something working that meets your needs. It's comforting knowing I'm not the only one that seems to be cursed with the kinds of issues you had to go through lol
as an amateur video do-er working with a whole bunch of gigabytes of high-quality game-capture video, this video makes me feel better about the relatively low-key and yet still innumerable problems i impose upon myself and also gives me dark thoughts about making my problems worse great video, thank you!
1:23:24 haha, I've actually bought three of these over the past few weeks, for two different machines, with different problems to the ones you're facing. It might seem like they're designed for you, but nah they have plenty of uses. E.g.some 1U machines put a Mini-SAS connector on the motherboard pointing right up into the air. If you put in a straight one you can't even close the server. Some other servers have a similar issue, except they're 2u and it's the air-guide plastic thing that's blocking them. Or other things like "we're using what we got, so the back of this server is right up against the back door of the rack or a wall, so we need to use 90 degree connectors (thermals? lol).
I love how unapologetic you are about your process. I imagine a lot of problems would be avoided if people were more honest to themselves about just how inconsistent they really are. And oh yeah, LOMs rock.
the thing that gets me about content creation in the modern era is just how much further that original setup of a phone, a decent cheap mic, and a free video editor can really get you! I mean look at DankPods; he shoots, or at least shot, all of his videos on a iPhone, records all his audio into garageband on his iPad, etc. Now granted, his quality has improved immensely since the first video, but man! its amazing how within arms reach of most people is the ability to make some damn good quality content without much trouble!
One of the best videos I've watched in months. I was working with Dell servers of this vintage back in the day myself. I also used to be the guy that needed to try all the server stuff at home myself. I'm saying to myself "I remember that" and "I've done that too!" whilst watching this video. Great watch.
Love your channel, keep on keeping on man! My NAS journey went from pulling my hair out with FreeNAS many many years ago, moving to synology, and am now on Unraid. Hoping they add official ZFS support at some point, but so far I am quite happy with managing my storage, dockers, and VMs. edit: If ARK doesn't work out, a 1650 Super or similar may fix your encoding woes! Encoder is nearly identical between all models of similar release. The 1650 Super is the cheapest (and lowest power!) card with the 3000 series encoder. I use it for all my Linux ISO transcoding needs :)
If you don’t mind me asking, how many years ago was it? I tried it in I think 2015 and it was very easy and I love it. But I know they had a lot more CLI setup a few years prior to that.
Just a small correction: the 1650 has the Titan V encoder, which is a bit older but still very usable. I was actually gonna suggest a low-end Ampere quadro, like the RTX A2000! (which only uses 70W)
@@SegaSaturnSubs The 1650 Super is upgraded though with turing encoder, but the 1650 (normal) is not. That's why I picked it up; absolute cheapest modern encoder card I could find!
ZFS is a great option for storage. It works great, I've been using it for well over a decade now in both a personal and business setting and would recommend it for any sort of storage solution where reliability and speed is a concern.
ZFS is really grate and all. But jeez is it a pain to deal with. It is rocket science levels of work to use it. Or you pay/trust a 3th party to solve it for you. Or you use software that is not really clear if they are going to be around in years to come. But OpenZFS is honestly the only file system I can trust data to. Edit: lol. Yea Ubuntu ZFS is basically the ticket to have a decent time. But I'm not dealing with Ubuntu or Windows. So FreeBSD it is for me :/ It is hard enough dealing with Linux. But I rather deal with open source then WIndows 8+ The distro support for ZFS with Ubuntu solves allot of problems with OpenZFS on Linux.
It is an interesting filesystem, but it is not optimal for servers using hardware RAID. In the case of Dell PowerEdge (like in the video) you have to disable that by flashing the card with a LSI firmware to enable some weird IT mode, which will present individual drives to the OS.
The second heard "HP" and 2011... I instantly thought "Thats gonna have a Turion in it isnt it... This isnt gonna end well." I'll defend the Phenom IIs desktop CPU from the era being quadcores you could get for dual core prices. But Turions IIs were hot garbage that for some reason HP kept using using.
HP used them for the same reason dell and gateway uses cheap junk. (or proprietary hardware) it makes them more money and once you've handed over your money it's your problem, not theirs.
A note on SSDs - since many of them are now using QLC flash, Samsung SSDs aren't necessarily the best choice for high endurance scenarios. There are many high quality brands that are still using TLC, so that would be my recommendation for building out the storage on your server further.
Also a function of size, also multiple level cells can be used in lower level mode (i.e. qlc can be used as slc) by the firmware iirc for e.g. caching. Best way for scratch drives is too buy high-tbw server-grade ssds which are suitable for say DB use. If course mirror them still. And eventually switch to better stuff like u.2 kioxia drives
"[...] probably wasn't as exciting as my usual stuff [...]" Aw man, it was SO exciting and entertaining! It felt really good not only to have a guided tour through an enthusiast's process throughout a long standing project, but also so validating to hear you acknowledge that it's sometimes just not realistic to say you're just gonna live with the compromises because you A) know you're prone to break that half-hearted promise to yourself, but also B) "you deserve better"! Really enjoyable, I can't even put into words exactly why. Great job! 😊❤
This was a fascinating exploration. Thank you for putting this together and sharing your workflow - it's always great to see how the videos I enjoy so much are made.
You can always get a "converged network card" that supports storage (usually by iSCSI and/or fiber channel) plus 20gb network all in one. They present to the OS as both a storage device and a NIC
That server might be the ultimate reason to buy that Intel A40 small Alchemist GPU, no gpu power connectors required. Intel seems to have open source drivers for Linux so it should be fine
I’m a teenager still, and this past summer I spent my time building a server VERY similar to this. I bought an older Poweredge r510 and put the best cpu’s that server supports, upgraded to 64gb ram and I also slapped 35tb of storage inside of it. My server is compatible in computing power to a lot of r710’s. I run ESXI 6.7 as my host os hypervisor, and a combination of CentOS8 and ubuntu VM’s under that. After investing in gigabit internet for my family and myself, my Plex server is SO MUCH FASTER. Like it’s unreal how fast. Going from 15mbps upload to over 950mbps upload is insane. I currently use about 15tb of that 35tb total storage with a combination of music, movies, and ripped game isos (ps2, ps3). How I directly access the storage from my main (linux, and windows) pc is via SSHFS. It uses ssh to mount the file system locally on my pc. And on windows using sshfs means that each 9tb drive shows up as separate disks, along with the 4tb and 1tb OS and misc drives. All in I spent right around $400 on the server, raid cards (h200, and h700), CPU’s (2x Xeon x5670), ram (8x8gb). I love the system and recommend it to anyone with an interest in tech and a few hundred dollars laying around. And remember… EBAY IS YOUR BEST FRIEND!! The r510 cost me $85! This video was very interesting and also relatable in the sense that I went through a very similar process as the CRD. Even down to the LOUD fan speeds on boot.
Was planning to watch 15 minutes and come back for the rest (as it was getting late and I needed to be fresh for work in the morning). Instead I watched the full episode and I loved it! I don't know what it is exactly, but your video's are always entertaining, informative and your sense of humour is just pure art at this point. Every time there was a cutaway to the editingbay I was like "which show will he refer to next?" Love your video's man, keep up the awesome work!
Somebody must have designed a 3D printed adapter for the T5 to fit nicely in that Kingston dock? Heck, a drill press, some wood, a sander, and a couple hours will make some workable adapters.
That was one hell of an adventure! I don't do video editing, but I move a TON of data around my network. Having used Synology NAS in the past it fit the bill perfectly. On a side note -- don't be afraid of fiber patch cords... they're much more durable than you realize. I've pulled hundreds of thousands of feet through AT&T offices and dirty connectors is a bigger prob over breakage.
I'll always be a bit nostalgic for the R720; it was the first server install I did as an IT apprentice (replacing a 2950). Learning about iDRAC was mind-blowing at the time.
Great vid and a real deep dive into the woes of storage arrays and the issues they come with. I implemented huge arrays in my last job - think the biggest was 1.4pb addressable for 120 edit bays for a large media firm. Worth watching for anyone wanting a grounding on almost everything that can go bad, including the FC (Real world networkig) vs IP stateful. Good stuff!
Direct attach is great until you're trying to shove a couple dozen of those SFP ends through the wire management in your racks, then you'll wish you sprang for the fiber. If you want fast low latency network storage you have to go iSCSI. All of the file protocols assume the network is crap and trade speed for reliability.
@@dragonheadthing Most iSCSI use cases involve many simultaneous read/write operations and that recommendation is to reduce fragmentation on platters that consumes time. Irrelevant on SSD and also not applicable to your single user set-up so you can safely ignore the advice. However it does require different HBAs, not sure what cost those would be refurbished but new they're expensive. And although I know th drivers are in Windows Server I don't know if they're in the desktop editions.
Why don't you record in ProRes 422 or LT? Visually, you won't see any difference, but it will be in terms of space. Example with 256GB: ProRes HQ 41 min ProRes 422 68 min ProRes LT 98min Even RAW 8:1 in 6K can accommodate 41min and it is definitely better than ProRes HQ for my taste. By the way, the recording quantities on the website are calculated at 30p, at 60p this doubles. Does it have to be 60p?
I agree, recording 24/30 fps prores lt in hd will probably be more than good enough for these videos. Just make sure you do the fake upscale to 4k for the output file to youtube, so its compression algo doesnt completely destroy it.
@@DoubleMonoLR admittedly, owning a 3D printer makes me want to find excuses to use it, like this, so yeah I might have a slight Impulse to overcomplicate it
I have had several conversations in the past 3 weeks that covered most of what you have now discovered and most of the problems you mentioned are NOT rare or unusual.. One comment is that there are two versions of SFP that are mechanically interchangeable but not electrical. SFP is for 1 GB, SFP+ is for 10GB. Where you mentioned SFP it should be called SFP+ instead.
Everyone I know uses the term "SFP" to generically cover SFP, SFP+, SFP28, QSFP+, QSFP28, XFP, and whatever other standard or quasi-standard plugs into the slot on a networkable device. Unless, of course, the context of the exact specification matters. But if it doesn't, it's "An SFP," and that's good enough to convey what you meant.
I knew somebody would have already pointed out the difference between SFP and SFP+. To complicate things more some but not all SFP+ ports are backwards-compatible with 1G SFPs.
One reason for some drives getting worse is due to the chip shortage in 2020. Some drives even got worse than their earlier runs. Even after the chip shortage was over, at least a few companies found that it worked really well to launch drives with premium parts, then slowly replace things like controller chips and DRAM chips with slower and less capable/slower spec versions. The initial reviews would say the drive was fast and would bury the newer reviews talking about how slow they were. Lately Amazon has been prioritizing showing me reviews made in the same year, though, so that might specific trick might not be nearly as useful to them, but capitalism will always be a race to the bottom in terms of quality in order to maximize profits. :/
It's lovely hearing you rave about the features on server grade kit. I've been a SysAdmin since the late 90s, and all our kit had remote management. It was over serial cables, but it worked. I'd love to have (be able to afford) LOMs on all my compute widgets.
"Oh, Dell really kept the design of their servers for some time" - I got out of IT in... 2013! Solved that problem. Worked at a tech uni 2009-2013 and we used Dells for the most part servicing ~450 users. The 500 Mhz Poweredge in the basement cable closet (and under the radar storage room of tech junk for me) did serve us fine for e-mail until 2010-2011 when central admin and other paper pushers made us join the centralized M$ solution they had payed good money for. We upgraded our poweredge 2950:s to R710/720 (I think) for VM:s. ILO-systems are great stuff! Esp if you, like I did at a previous job, were dealing with hundreds or even thousands of blade servers. That was before the company got into VM and just provided blades for customers. This video did bring me back to fun times, but I left for good reasons and are happy where I am. Thank you for your videos, I stay for every minute!
But I completely agree with you on enterprise gear at home. They are incredibly loud and eat a lot of space and power. Been on that route, vintage machines mostly. Running a Compaq Proliant Dual Pentium Pro for just DNS while your Sparcstation with added external 1,3 GB FULL HEIGHT SCSI HDD serves web pages in 2009 is cool in a nice way, but hey.. At least I stopped before I actually used my Microvaxes for something, :) The Sun E450 with its 20 HDD:s made a good TV bench at least.
![A CD-R, that's only partly a CD-R [Ricoh Encryptease]](http://i.ytimg.com/vi/_5ucImqdKbY/mqdefault.jpg)
![A CD-R, that's only partly a CD-R [Ricoh Encryptease]](/img/tr.png)
![I eliminated CD swaps FOREVER. [Nakamichi MJ-5.16]](http://i.ytimg.com/vi/oRuhRfvIkn0/mqdefault.jpg)
![[TH] 2024 PMGC League | Survival Stage วันที่ 1 | PUBG MOBILE Global Championship](http://i.ytimg.com/vi/FVFT-PFwU80/mqdefault.jpg)
That battery discovery was hilarious!
His enthusiastic laughter on the discovery had me crack up too, so very relatable and hilarious!
Keep up the awesome work CRD!
The deep full bodied laugh, gorgeous and inspiringly vulnerable
The genuine hilarity that ensued, and my own catharsis at having experienced that same "I'm not a smart man" moment in my own tech misadventures was utterly priceless.
That part was awesome. You can't plan stuff like that!
It made the video.
Watching for a second time 1 year later and still enjoy the "oh my God, the battery's removable " part. So glad you kept that in.
Linus Tech Tips can make a vid 20 minutes long about a bigger, way more expensive unit and I'll probably watch half. Your video is more than an hour and I'll watch it all like a fine movie. You're a special kind of TH-camr, keep the good work!
LTT at the end of the day is really entertainment. There's a lot to learn here, whereas watching Linus might get you some laughs but not much knowledge.
@@DrKratso 'hehehehe look at me i dropped an expunsive gee pee you! hahaha im so funny haha typical linus he drops things hahahahaha'
@@DrKratso I love LTT videos but I feel like over the past year or so especially their videos about this kind of stuff really moved away from the homelab/home user level and into the enterprise grade scale level. I mean while it's neat and what they as a company need it kind of now leaves a gap for the regular joe server/NAS/DAS/JBOD builds which videos like this fill in amazingly well.
I almost kind of look at it like CRD as Home-User/Hobbyist level, somebody like Jayz2Cents as sort of Power-User/Small Home Business level and then LTT & a few others as basically Enterprise level stuff.
@@DrKratso I've been watching a random assortment of LTT videos over the last year for no particular reason other than they keep showing up as the next autoplay video and I've noticed just how badly their videos age. Production quality has barely improved in 5 years. Not the video quality, the content. They handle everything like its okay to throw things around and hit them with hammers. You are definitely right in it being only entertainment, its the tech version of EBLR.
I agree!!
i can't believe that I've watched 1 1/2 hours of a guy talking about a video server in so many unnecessary details, going about half an hour of history before getting to the original point
i love it, if i had the will of making videos, it would be like this, 100℅, i never, and i really mean it NEVER felt like i was watching a video of someone that was like me, all of the thought process, the battery discovery, the complains about servers in general, pointing out the path that you needed to follow in order to get to the bottom of it
i truly love it, this is only the 2nd video of yours that I'm watching, and i already love it, i wish i had some more money to afford a patreon membership, i just LOVED every second of the video
Exactly the same for me! I was about to write a very similar comment, but now I can just say, "Same here!"
@@tux9656I agree thirdly.
Hi, is it normal that I don't understand 95% of the tech that you talk about but still love watching your videos ? I am a dumb airline pilot in Lima , Peru. Love your work , keep it up !!!
I was so nervous that you wouldn't include a "two of them" in this video. finally, over 80 minutes in, I am satisfied. I loved this video, I thought it'd just be good for background noise, but you actually covered some things I didn't know existed, and I got totally sucked in
I will have a lot these videos playing while doing house work. No matter what I’m doing, if I hear “two of them” I look up to see if the meme is there.
Edit: JUST SAW IT I’m so happy
I love Technology Connections' "And through the magic of buying two of them, I've got one disassembled here to look at."
@@mndlessdrwer beat me to it by a month...
I am three minutes in and already hooked. Let's see where this rabbithole goes.
EDIT: 1 1/2 hours later, and now I have the strong urge to upgrade our home NAS. great Video, as always!
He was so ready to shit on that battery 🤣
I deffinitly have a strong need to build a NAS watching this, one of those too-long-delayed projects
Been a long time user of Freenas (now truenas) for many years. And WD Red drives(5400rpm models). power efficient, reliable as heck and performance is not too bad really. And doing automated backups has been pretty easy, doing snapshots of changed data every day takes far less time and is often just a few seconds to a minute unless I have done something significant. Than its a little longer.
And yes I mentioned my nas drives are 5400 rpm, and I do have 10g network. But as a small family its always been quite sufficient. I also use SLOG and L2Arc (flash based read and write sync caches) and the nas has 48 gigs of ram. So that itself does a lot of heavy lifting. Loading in a couple Tibbs of data for backup isn't too bad when it goes at about 600mb/s
Truenas and ZFS also have paranoid levels of paranoia when it comes to data integrity. And as long as you stick to recommended practices then that data will always be safe almost no matter what.
I am extremely sad now that I've realize that we've gotten to the point where it needs to be specified when a link is NOT an affiliate link but I greatly appreciate the integrity in providing them. There is inherent bias in providing affiliate links and I never feel comfortable buying from anyone who gets paid when I make a purchase
So strip out the affiliate information in the URL. Amazon you can strip out everything past the DP/# code and it'll still work.
@@pyrioncelendil yes but that defeats the purpose of why you don't buy from someone who posts an affiliate link. If it is an affiliate link the prospect of money is the motivator of posting it and you cannot be sure they truly recommend the item because it's just as easy to post and say you like it and still get money. Deleting the affiliate info onto stops them from getting paid for your click, it can't change their reason for posting the link in the first place. By the creator *posting* a non affiliate link, they show that money could not be a motivator since they do not expect payment.
@@SonicKiwi123 exactly
Dude, I could seriously listen to you just talk all day. Not just because I'm a tech nerd myself, but your presentation and enthusiasm for the subject matter is so genuine. I also love how you cover all the bases and thought processes someone would would realistically go through to reach the final... Uh, result of whatever we're doing in the video.
Yeah. He could read a twenty year old newspaper and I would listen.
I agree totally, I start off thinking I will watch this guy for a few minutes only to find I have watched it for a few hours to the end.
28:23 For the dock I'd suggest you 3D Print a removeable spacer for the drives to support the connector. Or get somebody to print one for you. It would be an easy design & print for anyone who does their own designs.
Was going to say the same 😁. Literally 15 minutes to model and maybe an hour to print. Could also, just as simply, make adapters to deal with The T9 drives not fitting into the Small Rig clamp.
Or jam a piece of cardboard in the slot with the drive.
@Cathode Ray Dude [CRD] I just caught this video now. I'd do it for him. No charge. Just have a chicken and egg of not having the drive or kingston bay/hub to measure. lol Could probably work off just a caddy and drive model. @Cathode Ray Dude [CRD]
I would go even farther :):):) Why not suggest to buy a 3d printer and build it on itself. :):):)
To be honest, the dock looks like a prosumer version of those usb copy stations they sometimes have at merch booths to have concert recordings for sale shortly after they ended.
I loved how you reacted when you discovered the battery door. Such genuine laughs.
I'm glad that you didn't hide your mistakes and show your true self after discovering that you did something wrong for years for basically not even doing the basic RTFM. The contrast of a noobish mistake and the general well researched information dense documentary type you usually do was just perfect.
This makes you even more genuine and lovable as a TH-camr.
Just wanted to complement your going the SFP route in the network interconnect for 10GbE, it has lower latency than 10 gig twisted pair so nice little bonus there. Also direct attach cables really do feel like a cheat and we use them extensively inside and in-between racks; there's a whole world of those cables now, including thinner versions to improve airflow, colored and plenum-rated jackets... congratulations on your workflow upgrades, cheers!
I've never heard of these, but color me intrigued I'll say that much.
As a professional in IT I love that you chose SFP modules and not 10G-base-T
SFP modules are just dumb unless you need copper or fibre for longer installations. DAC is the bees knees. They require much less power (less then 1% of a 10gbps ethernet sfp). Both optic and ethernet/copper takes alot of power to drive.
Not a big issue with 2 connectors, but when you haev 24 or 48 in a rack the power demand is significant.
@@jmkhenka sfp dac?
DACs are great, but I don't care. I'm 100% with the OP here: At least it wasn't 10Gb-T.
@@nickwallette6201 What's wrong with 10Gb-T?
@@VCD-Channel A lot of little things.
1) In a commercial environment, where you're going to have potentially lots of SFP modules packed in to a dense switch port cluster, the transceivers we have today require too much power, and get too hot. It breaks the limits of the SFP+ spec, and while most implementations can handle staggered deployment (so every other port, or something like that), it's just really hacky.
2) We've kinda pushed twisted-pair signaling to the edge of its usefulness here. It's a fine topology for baseband, or 10Mb, 100Mb, or even 1Gb networking. But at the bandwidth required for 10Gb, something like coax (which, ironically, we abandoned for UTP long ago in conventional LANs) is much better suited. It's difficult to guarantee reliability at the traditional Ethernet specification of 100 meters, and even that's assuming you're using a recent plant with quality cabling that was installed carefully. Real-world cable plants are often going to be CAT-5E or CAT-6, with similarly-rated jacks and patch panels, and a lot of potential weaknesses like kinks, cinches from over-zealous zip ties, too-long runs, crappy patch cables on the ends, etc.
3) Given the density and integrity issues, many professional vendors haven't focused on 10Gb-T, so your options for 10Gb-T switches are limited compared to SFP+. This is my bread and butter, so I used to sell a lot of switches like the Juniper EX4550, which was a 32-port SFP+ switch with two modular bays where you could install an expansion card with (your choice of) 8x 10Gb-T ports, an additional 8x SFP+ ports, 2x QSFP+ ports, or 2x chassis stacking ports. Super versatile! But, that was discontinued, and there just aren't a lot of models out there where you can casually add a handful of 10Gb-T ports, without committing to an entire switch full of them. This is particularly annoying when there are just like one or two appliances that need 10Gb, but don't have any SFP+ slots. That's the worst. A lot of people need or want a healthy dose of 1Gb ports anyway for legacy or low-bandwidth appliances, or out-of-band management interfaces on their high-bandwidth appliances. So you _could_ just buy a 48-port 10Gb-T switch and run most of the ports at 1Gb, but that's going to cost a lot more than a similar 1Gb-T switch, so your only true recourse is to populate some of your valuable SFP+ slots with 10Gb-T modules -- and that just feels like the IT-industry equivalent of sharing prescription medication.
Such an amazing video! Inspired me to change my approach to scratch disks, 'negatives' and backups. I especially liked all the preemptive "yes, you might be able to do it better, but this is how I do it and this is how I like it" comments. Definitely shows years of reading YT comments that contain simplistic and sometimes condescending suggestions to complicated problems 😁
I have a fairly similar setup at the moment (Atomos Ninja V recording to a 1TB SATA SSD), but I use the "negatives" for editing (love that term). Sometimes I would actually move the files to my NAS (with two NVMe SSDs in a RAID0 configuration) and edit off of that, but both of these workflows are very fragile and prone to data loss. I'm thinking about adopting a similar setup that you have, and since I already have a NAS, this should be relatively easy.
Once again, I really appreciate and respect the effort that you put into making this huge (and very insightful) video.
Thanks for not leaving out your past mistakes! Personally, I love my server rack even though there isn't a server in it :P I use it for my 19'' audio gear and some battery backups for my PC. I also put a shelf on it to store random PC accessories.
I used a two post rack for my home theatre gear, Crestron gear, a laser printer and UPS for 10+ years. It was hidden away, so it wasn't visible from the main room. Unfortunately in the new house, it's not possible so I had to get a regular cabinet.
When you started talking about flashing the PERC RAID controller to IT mode, I was like, "god I hope he shows the bootloader for the flashing utility" and at 1:17:00 I started busting out in a fit of laughter and clapping. Its so unbelievably cursed and I'm glad you shared it with the rest of the world.
I love the scene from Office Space, like "this is what's about to happen to your machine"
Lmao do you even download that??
"you finna get flashed" I died
This was phenomenal. Every minute. I loved the back and forth with the "editing bay", the humor, the data, the geeky stuff... Thank you!
I subscribe to several channels where the creator really cares about video quality. At least enough to mention how much they care about it. I watch everything in 480p and lower. Now, if a creator has a terrible mic, it's painful to have an interesting video distracted by awful audio.
Well, sure. But the 480p version of the 6K footage looks better than the 480p version of the
1080p* camera from the 2000s, noticeably so if I may add. Hell, any High Production Cost Video will still look ridiculously good, even in a low resolution. I mean, look at the LTT videos nowadays.
Camera quality is actually pretty independent from TH-cam quality setting.
You can tell when the creator has a high quality camera even when viewing at 480p. Its sharper, the colors are better, there is less noise.
So many other benefit than just resolution.
Not that you are wrong, the audio quality is way more important. But once a creator has good quality audio, they might as well start improving video. Bad video might not drive people away as much as bad quality audio, but the viewer still notices and responds slightly better to high quality video.
If only TH-cam could deliver 480p60 video
@@DoubleMonoLR Shooting in high resolution allows you to do things in post that would be...messy at a lower resolution. That doesn't mean the end consumer needs that max resoultion, like they don't need the content delivered in 1 gibagit/sec Prores
@@LocalAitch True... we did a music video once that emulated video look and I wanted it as close to SD as possible and exported it as 576p50 (as I'm in Europe). Looked the part... TH-cam made it 480p25 which doesn't make any sense and looks horrible. Had to upscale it to 1080p50 :D
Video pro here. Your desire to do things right is something I admire with this project.
Have you considered a Proxy workflow? The reason why it was originally created was because even though most workstations (even mobile) were able to playback full resolution media the storage attached to them couldn't handle the size and data rates of high quality intra frame and raw codecs.
Just going from your ProRes 422 HQ to ProRes Proxy would mean 1100Mbps -> ~225Mbps and that's before you reduce your resolution.
Modern Nvidia, AMD, and Intel (including integrated. Quick Sync is shockingly good) GPUs can playback h.264 footage with great performance. If you go down that route, you will only need a few dozen extra GBs with your media for your entire project to have lower res high performance playback.
Resolve 18 went stable and had a point release already. It has a revamped proxy workflow that you should take a look at (see Resolve 18 reference manual pages 188 - 192). It can now automatically detect and link proxies on and after import and comes with a little proxy generator app complete with watch folders.
Generating proxies is something that can be automated part of ingest too, you may already be familiar with ffmpeg, so it can be done server side too.
In addition, if you want something a little less fiddly than an entire server to administrate for the sake of having more space for video production, consider a Blackmagic Cloud Store/Pod.
Fun Fact:
25 years ago, I worked in a post production house. A place to edit ads for tv, and corporate videos. To edit the videos they used AVID in Quadra 900. The amount of storage in the external disck was 32GB. I Guess this amount of space give us something like 6 hours of video storage at that time.
Reason for the 2 connectors on the drive backplane: SAS supports a feature called Dual Port, it's basically a second cable path to reach the drives for better reliability. It might be the multiplexer (or whatever Dell calls it) has a fault that makes it choke on DP.
One hour into this and I'm fully conversant in quality jargon regarding the world of NAS. I appreciate the articulated trailblazing of CRD.
As a community college ICT teacher, I have a whole bunch of these kind of 2013-2017 era servers (we get them for free from companies that get rid of their old equipment) and never know what to do with them, but this gives me some ideas. I can make a project where the students have to make a solution to store footage from a security camera
You could use them for folding
@@deepspacecow2644 You've never paid and electric bill have you?
@@MatthewHolevinski What gave is away, suggesting using hundreds of watts of electricity on old inefficient cpus and high power draw fans?
@@deepspacecow2644 I didn't realize that came off sounding like a jerk, I'm sorry, I meant that to be funny not mean.
@@MatthewHolevinski it was funny, I am just salty because it's true.
I just recently found your channel, and I have to say, your presentation is excellent. You're one of the shockingly few people I will listen to for an hour and a half straight and not get bored. In fact, as I write this, I am about 2.5 hours in after watching your video on that absurdly long IBM portable with the attachable printer. I'm really glad you've dedicated yourself to this.
I like this guy. His intelligence and self-awareness seem rare in Americans, and make for excellent presentations.
Yes! New CRD video!
Edit: I too started with a Sony handycam (cx240e) which I still use and I hate it. When I bought it I didn't know alot about camcorders but one of my criteria's was 50p/60p 1080p video so it was a good starter camera, so when I saw one for 65$ I bought it. I might mod it for a mic input because the internal mic is not that great.
Long time viewer, first time commenter. The fact that your workflow is this complex doesn’t surprise me - high quality and fastidious attention to detail are your hallmark at this point. What does surprise me is how vast the delta between your workflow and mine actually is. ~200 10-15 min 4k videos (across my multiple channels) all render out of my 256 gb m1 Mac mini each year and make their way onto TH-cam. Really enjoyed the insight, but I can’t say I’m not glad to have stayed clear of this rabbit hole! 😬
1:05:01 Modern fiber can take a lot of abuse, way more than copper. I’ve slammed cabinet doors on x connects all the time in a data center and they work just fine! You can also get bend insensitive fiber that can be tied in loops without much attenuation. Btw you can get 40G and 56G infiniband hardware for pretty cheap on eBay once you outgrow 10G Ethernet
So, I actually purchased two servers from the community college I graduated from, and each of them were quite interesting.
They both were Wide Orbit Radio Automation Servers, which were used for the college's radio broadcasting station, and one came with a 1TB HDD with songs and Licenses for the station to be able to broadcast those songs, and the other came with three 2TB HDDs, two of which were chalked full of songs as well, and the other had licenses for each of the songs on the other two 2TB HDDs, as well as each containing a Samsung EVO 850 250GB SSDs (I'm guessing so they can boot quickly), each of them also had a PNY GeForce 8400 1GB DDR3 PCIe 2.0 GPU (with a video port that looks quite strange to me). One of the servers had an i5 - 4570, with 8GB of DDR3 Ram (1x 8GB, Single Channel), and a good amount of SATA cables sprawling from the five (5) drive bays to many of the SATA ports on the Motherboard.
But, the second server that I purchased, which was the same brand (Wide Orbit) and server model, came with an i7 - 4790, 16GB of DDR3 Ram (4x 4GB, Dual Channel), and a PCIe card that is quite similar to the one that 01:20:26, with the plug that has four SATA cables coming off of it, which were attached to each drive bay, with the drive bay containing the SSD having its own SATA cable plugged directly into port 0.
I know the hardware is old, but I still can't help but think that I got the deal of the century with these beauties!
I'm going to guess the port on the 8400 is dms-59
I'm so glad I opted to watch this instead of some arbitrary Netflix filler tonight. Got to admit I was impressed at how much self-awareness and hard honesty was used, but it helped define the problem so much better, and was great to see such a satisfying solution!
Love iDRAC and its equivalents. I mandated that all new servers at work have licenses for it, they were cutting it out to save a few hundred bucks per server. I am a firm believer that the cost is well worth the amount of time you save vs having to physically go to your server or mess with a network KVM.
Nothing like installing the OS on a server from your couch at home!
When I was first learning server shit at work I discovered one of our servers had an iDRAC license and instantly fell in love with the features and it's saved me so much farting around. Now that I'm actually in charge of specing out our servers, we don't have a single one without that license.
I so wish Intel's AMT was more widely available too, it's not full iDRAC/iLO functionality (especially if you have an i3, but some of the i5+ only features can be unlocked with MeshCommander). I have a standard HP elitedesk 800 G2 mini PC running as a server that I can just pop open a serial-over-lan terminal or boot an ISO image over the network and it is SO. COOL.
I think the common generic term is IPMI (Intelligent Platform Management Interface). That's what I tend to use when not OEM specific.
Hate iDRAC, other integrated server management hardware/softwares are better. Never seen so many iDRAC hardware failures over the years. I have had to replace 100s of Dell PowerEdge motherboards over the years, simply due to iDRAC hardware failures. For whatever reason. iDRACs are prone to failure. There are ways to reprogram the iDRAC firmware by externally attaching a programmer to the iDRAC firmware chip or even reprogram the iDRAC via sd card, but it is more painful than simply replacing the motherboard. iDRAC blows, but hard to have hardware lever remote server management without it. The iDRAC free verson works, but the Enterprise version is much better, with more features (and btw is easily hacked to become free, lol).
A video this long has no right to be as entertaining as this was! Super interesting topic that had me hooked right from the start and till the very end.
A interesting journey from beginning to end, with that hilarious camcorder-battery-gaff at the beginning kicking things off wonderfully
Also, thank you Daria 🙏 for your help in the studio and logistics!
Since you have ZFS, you might improve the archival copying by using snapshots and sending them instead of using rsync.
You snapshot the scratch which gives you an *exact* state in time and can zfs send/recv to the archive. You can also mark the recv'd dataset as read only and avoid messing with the permissions.
Look into snapshots and sending datasets; I think you might find some value there.
He also might want to consider playing with TrueNAS, possibly scale if he'd like to stick with Debian over FreeBSD and have native containerization and a better hypervisor. Doing ZFS on raw Linux is fine, but given how developed TrueNAS is as an appliance, it's worth him checking out.
I know he says he doesn't want to learn anything, but with what he's doing, he's just getting access to a gui form of what he's doing in the command line. Also, the irony of him saying that TrueNAS might break is that manually configuring ZoL will mean it's more likely he could accidentally break something. I'm sure he's competent in the command line, but there's a reason OSes and Applications have GUIs. Unless he was a NixOS kid, then I'd say he's fine configuring ZFS entirely from the command line haha.
@@metaleggman18 Maybe some double checking if recursive datasets work fine in the containerized environment. Proxmox had a bug when dealing with recursive sets that is just now being patched with some updated code. But I agree using a hypervisor instead of running it directly on the bare metal would be a good idea. I used to be pretty gung-hoe about everything running on the bare metal until I had to come to terms that I'd like to not deal with some packages with pedantic requirements that can break the whole system, and instead just build a container for that service and leave it alone until I want to maintain it instead of being forced to do it during a universe upgrade.
Specifically, just get sanoid/syncoid and it can all be automated quickly:
Sanoid can manage snapshots, taking snapshots daily/hourly/weekly/monthly/whatever and pruning old ones according to rules.
Syncoid can sync those snapshots from a source pool to a target pool easily.. I do something similar:
Mirrored NVMe drives hosting VMs, taking a daily snapshot and keeping 2 days of snapshots, and syncoid doing a daily sync from the NVMe drives to some spinning rust, where it holds more than 2 days worth of snapshots.
Sanoid is basically just automatic. Syncoid is just going to replace rsync in his workflow.
Thank you for actually saying "couldn't care less". Also, very informative video. This has really helped as someone looking to increase my 4K video capacity and backup years of photos to something better than a random external HDD connected by USB.
This is awesome, dude. Ten years ago, I was teaching myself basically everything you cover in this vid. Had I continued down that path, this is exactly the hardware and knowledge base that I would have achieved. As life goes, directions change along with needs and priorities and such a beautiful solution never manifested for me. You may have inspired my to tear into that mountain of n-egg boxes, drives and cases hidden in the closet. I'll have to consult my cardiologist first LoL. Great job on the system and video!
I have loved every video on ltt about the ridiculous servers ,networking and the nitty gritty of their production, but they are criminally short... So having a lengthy explanation of your workflow and the fantastic description of the trials and tribulations you had to go to achieve this is delightful!
Thank you for this fabulous production, it is truly enjoyable, entertaining and informative.
P.S. I hope we get an update once you get the transcoding directly in the server.
LTT desparately needs a professional admin. Someone who will tell them "no". They're not stupid, and they all have clear technical ability. But professional administration is another thing entirely.
Just finished the whole thing. I found it all fascinating! That battery door realization is going down in this channel's history. 🤣
This is good. Respecting your own process and building to work with it. Also I spent two years in an office with one of those severs at the end and although I became immune to its screams, people who called me on the phone or tried to have a conversation rarely failed to remark on them.
I’m not sure why, but I love the contemptuous way that you say “rig”
This is probably one of your best videos so far, my dude. As much as I love your retrospectives, histories, and general deep dives into bizarre and interesting pieces of technology, your aptness for storytelling mixed with personal problem solving made this entire video a trip. It certainly didn't feel its length in the best way possible - I was with you every step of the way and was excited to hear what came next again and again. I normally don't comment, but I have a terrible feeling this video won't perform like your others because TH-cam is garbage - but know that I, personally, enjoyed this video immensely.
how to cure depression:
hysterically laughing CRD
Así es :3
CRD's Joker moment
Add hysterically laughing LGR as needed
You could probably 3d print an insert for that Kingston drive bay thing to make the T5's fit more securely
I thought about the insulation stuff you stick around your windows
or print your own drive bay with a few right angle usb-c cables going to a pcie usbc controller in the server, plug them in and the script copies the stuff on connect!
What about a magnet and some double-sided tape? Surely there’s a metal panel somewhere on the camera…
Yeah, that's what I thought immediately. This is the kind of stuff I design and send the printer almost without thinking about it these days.
@@NageebTheAverage He's not talking about the drive bracket on the camera, he's talking about the Kingston dock that they go in when he gets home.
I stayed to the end .
I learned a number of things to USE and to LOOK OUT FOR .
I can't use EVERYTHING here, but there WERE some things that will really help !
THANKS !
As someone who is not very interested in storage management solutions, this was a very entertaining video to watch. I loved how you detailed your chronicles in the sphere of content creation, and your explanations were simple and apt so that I could follow along. Great video!
This Rube Goldberg setup to use those SSDs with your blackmagic camera is hilarious to me. It feels like a professional version of my very first video setup, my very first """"""camcorder""""" was a webcam and a cheap computer mic taped together and wired to a laptop that I carried around in a backpack
All my stuff came in from Japan and CRD uploads. EVERYTHING IS COMING UP MILLHOUSE!
What you got?
@@nikGhost1 Three rolls of Velvia 100 and Velvia 50, one roll of Fujicolor 100, Puyo Sun for the PS1, Sega Saturn, and N64, and a Minolta Weathermatic 35.
Oh, also a Puyo Sun guidebook.
I remember connecting the gravis Stinger to my compaq iPaq so I could play NES games on it
A very entertaining watch on youtube. I was thinking about getting a two rack system, but, the noise would keep the neighbors awake, and also me and my wife, since we have not got any basement where we live. Very good information you give out with the problems that come up with these servers. Thank you for doing this.
Experience is the best teacher, and you teach by sharing the experience! You go deep in the weeds, but anyone watching your videos can avoid having to go through the same pain building a NAS!
I actually had a segate rosewood (it's basically the data equivalent of a hard drive with a bomb with a random number generator attached)(also very common hard drive) and SMART actually saved my data before anything catastrophic happened, at least i checked crystal disk info and it was throwing a yellow caution and I was able to purchase a new drive and move the contents before it ruined anything. only had about 20k hours because it started to fail.
Your laugh about the battery had me laughing so hard 😂 it was genuine and authentic I love it
The Black magic's touch screen actually being better than the physical buttons on the previous camera was an M. Night Shamylan level twist.
This video has SAVED ME from learning about the Crucial BX500 the hard way.
I placed an order for one, and then the feint memory of this video started coming back to me. I came back to this video to check what SSD was causing your problem and lo and behold it was the same one I just bought. I cancelled my order and have my money back.
Thank you Mr. Dude for sharing your technical trials and tribulations
I knew there was underlying reasons I didn't have a youtube channel. It's not just laziness, lack of interest, or too tired, it's because I would not know the first thing about storage and making videos. Yet I start off watching your videos thinking I will move on when I get fidgety, but no, I actually like your videos even though I understand about 2 per cent of what your saying. I still find not understanding interesting, because of the way you present stuff. Keep up the good work it's very very interesting stuff. One last thought, you must have a very good computer very costly and extremely fast internet speeds, but then again it would not have to be super fast to be faster than Australian internet speeds. But yes I can imagine your super dooper desktops and laptops doing such wonderful video work. Only the best, but you deserve this your videos are again very good and entertaining.
They make CFAST TO SATA3 Convertors for the Black Magic Cameras. Since CFAST uses the sata bus it is just a connector changer. That is always a good way to get more storage. You can also do the usbc to nvme enclosures and put a 4TB ssd there as well.
Or even 8TB, if you're willing to drop $1100 on the SSD alone (the enclosure is only about $50).
i love your content! your delivery is incredibly entertaining. Time flies when I turn on one of your videos! Great pacing, derivations and tangents are awesome, and your passion is obvious. One of my current favorites.
Congrats on getting something working that meets your needs. It's comforting knowing I'm not the only one that seems to be cursed with the kinds of issues you had to go through lol
huge fan of the random cut-back lines
finally getting returned to higurashi after all these years
was about time imo
subscribed.
Your "we now return you to [totally unrelated media]" bit from post never gets old. Please never stop.
as an amateur video do-er working with a whole bunch of gigabytes of high-quality game-capture video, this video makes me feel better about the relatively low-key and yet still innumerable problems i impose upon myself and also gives me dark thoughts about making my problems worse
great video, thank you!
1:23:24 haha, I've actually bought three of these over the past few weeks, for two different machines, with different problems to the ones you're facing. It might seem like they're designed for you, but nah they have plenty of uses. E.g.some 1U machines put a Mini-SAS connector on the motherboard pointing right up into the air. If you put in a straight one you can't even close the server. Some other servers have a similar issue, except they're 2u and it's the air-guide plastic thing that's blocking them.
Or other things like "we're using what we got, so the back of this server is right up against the back door of the rack or a wall, so we need to use 90 degree connectors (thermals? lol).
thermals? lol
@@CathodeRayDude thermals? lol
I love how unapologetic you are about your process. I imagine a lot of problems would be avoided if people were more honest to themselves about just how inconsistent they really are.
And oh yeah, LOMs rock.
the thing that gets me about content creation in the modern era is just how much further that original setup of a phone, a decent cheap mic, and a free video editor can really get you! I mean look at DankPods; he shoots, or at least shot, all of his videos on a iPhone, records all his audio into garageband on his iPad, etc. Now granted, his quality has improved immensely since the first video, but man! its amazing how within arms reach of most people is the ability to make some damn good quality content without much trouble!
One of the best videos I've watched in months. I was working with Dell servers of this vintage back in the day myself. I also used to be the guy that needed to try all the server stuff at home myself. I'm saying to myself "I remember that" and "I've done that too!" whilst watching this video. Great watch.
Love your channel, keep on keeping on man! My NAS journey went from pulling my hair out with FreeNAS many many years ago, moving to synology, and am now on Unraid. Hoping they add official ZFS support at some point, but so far I am quite happy with managing my storage, dockers, and VMs.
edit: If ARK doesn't work out, a 1650 Super or similar may fix your encoding woes! Encoder is nearly identical between all models of similar release. The 1650 Super is the cheapest (and lowest power!) card with the 3000 series encoder. I use it for all my Linux ISO transcoding needs :)
You should give truenas scale a shot... it's nice, supports dockers out of the box, runs your vm's etc; and a doofus can set it up.
If you don’t mind me asking, how many years ago was it? I tried it in I think 2015 and it was very easy and I love it. But I know they had a lot more CLI setup a few years prior to that.
1650 super is generally just a good cheap low-power gpu anyways
Just a small correction: the 1650 has the Titan V encoder, which is a bit older but still very usable. I was actually gonna suggest a low-end Ampere quadro, like the RTX A2000! (which only uses 70W)
@@SegaSaturnSubs The 1650 Super is upgraded though with turing encoder, but the 1650 (normal) is not. That's why I picked it up; absolute cheapest modern encoder card I could find!
ZFS is a great option for storage. It works great, I've been using it for well over a decade now in both a personal and business setting and would recommend it for any sort of storage solution where reliability and speed is a concern.
ZFS is really grate and all. But jeez is it a pain to deal with. It is rocket science levels of work to use it. Or you pay/trust a 3th party to solve it for you. Or you use software that is not really clear if they are going to be around in years to come. But OpenZFS is honestly the only file system I can trust data to.
Edit: lol. Yea Ubuntu ZFS is basically the ticket to have a decent time. But I'm not dealing with Ubuntu or Windows. So FreeBSD it is for me :/ It is hard enough dealing with Linux. But I rather deal with open source then WIndows 8+ The distro support for ZFS with Ubuntu solves allot of problems with OpenZFS on Linux.
@@TheDiner50 This is why you use FreeNas/TrueNas/etc. Let it deal with the zfs management. Couldn't be easier.
It is an interesting filesystem, but it is not optimal for servers using hardware RAID. In the case of Dell PowerEdge (like in the video) you have to disable that by flashing the card with a LSI firmware to enable some weird IT mode, which will present individual drives to the OS.
@@openworked hardware raid is a scam anyways, and should be disabled.
@@snowwsquire why? Please enlighten us
The second heard "HP" and 2011... I instantly thought "Thats gonna have a Turion in it isnt it... This isnt gonna end well."
I'll defend the Phenom IIs desktop CPU from the era being quadcores you could get for dual core prices.
But Turions IIs were hot garbage that for some reason HP kept using using.
HP used them for the same reason dell and gateway uses cheap junk. (or proprietary hardware) it makes them more money and once you've handed over your money it's your problem, not theirs.
I've seen those turions microservers around and figured there was a reason they're so cheap.
A feature-length chat about video production and storage. Thumbs up, it's jam-packed.
I can't see anyone noticed it in the comments, but the imperial vs metric joke at 16:15 is well delivered.
A note on SSDs - since many of them are now using QLC flash, Samsung SSDs aren't necessarily the best choice for high endurance scenarios. There are many high quality brands that are still using TLC, so that would be my recommendation for building out the storage on your server further.
Tlc is better, right?
yes @@KOTYAR0
The more bits per cell the lower the write endurance. TLC is Three bits per cell and QLC is Four bits per cell.
Also a function of size, also multiple level cells can be used in lower level mode (i.e. qlc can be used as slc) by the firmware iirc for e.g. caching.
Best way for scratch drives is too buy high-tbw server-grade ssds which are suitable for say DB use. If course mirror them still.
And eventually switch to better stuff like u.2 kioxia drives
Very interesting, I have a similar issue with my videos/amount of footage and how to store all of it. Good to know the experiences you’ve had
"[...] probably wasn't as exciting as my usual stuff [...]"
Aw man, it was SO exciting and entertaining!
It felt really good not only to have a guided tour through an enthusiast's process throughout a long standing project, but also so validating to hear you acknowledge that it's sometimes just not realistic to say you're just gonna live with the compromises because you A) know you're prone to break that half-hearted promise to yourself, but also B) "you deserve better"!
Really enjoyable, I can't even put into words exactly why. Great job! 😊❤
This was a fascinating exploration. Thank you for putting this together and sharing your workflow - it's always great to see how the videos I enjoy so much are made.
You can always get a "converged network card" that supports storage (usually by iSCSI and/or fiber channel) plus 20gb network all in one. They present to the OS as both a storage device and a NIC
Great video! I'd love to see you talk more about the servers, the hardware, how you built them and software tutorials too.
the moment you found out about the battery door 🤣🤣🤣
and with a laugh like this, this man is a national treasure
One of the more wholesome things in TH-cam.
That server might be the ultimate reason to buy that Intel A40 small Alchemist GPU, no gpu power connectors required. Intel seems to have open source drivers for Linux so it should be fine
I’m a teenager still, and this past summer I spent my time building a server VERY similar to this. I bought an older Poweredge r510 and put the best cpu’s that server supports, upgraded to 64gb ram and I also slapped 35tb of storage inside of it. My server is compatible in computing power to a lot of r710’s. I run ESXI 6.7 as my host os hypervisor, and a combination of CentOS8 and ubuntu VM’s under that. After investing in gigabit internet for my family and myself, my Plex server is SO MUCH FASTER. Like it’s unreal how fast. Going from 15mbps upload to over 950mbps upload is insane. I currently use about 15tb of that 35tb total storage with a combination of music, movies, and ripped game isos (ps2, ps3). How I directly access the storage from my main (linux, and windows) pc is via SSHFS. It uses ssh to mount the file system locally on my pc. And on windows using sshfs means that each 9tb drive shows up as separate disks, along with the 4tb and 1tb OS and misc drives. All in I spent right around $400 on the server, raid cards (h200, and h700), CPU’s (2x Xeon x5670), ram (8x8gb). I love the system and recommend it to anyone with an interest in tech and a few hundred dollars laying around. And remember… EBAY IS YOUR BEST FRIEND!! The r510 cost me $85!
This video was very interesting and also relatable in the sense that I went through a very similar process as the CRD. Even down to the LOUD fan speeds on boot.
Was planning to watch 15 minutes and come back for the rest (as it was getting late and I needed to be fresh for work in the morning). Instead I watched the full episode and I loved it! I don't know what it is exactly, but your video's are always entertaining, informative and your sense of humour is just pure art at this point. Every time there was a cutaway to the editingbay I was like "which show will he refer to next?"
Love your video's man, keep up the awesome work!
This video is like manna from heaven for a camera nerd like me
It's never a reliable solution if humans can interfere. I learned that the hard way in two decades in the IT business.
Somebody must have designed a 3D printed adapter for the T5 to fit nicely in that Kingston dock? Heck, a drill press, some wood, a sander, and a couple hours will make some workable adapters.
2:47 and on is literally my favorite moment on TH-cam. I keep coming back to this because I love it so much.
That was one hell of an adventure! I don't do video editing, but I move a TON of data around my network. Having used Synology NAS in the past it fit the bill perfectly. On a side note -- don't be afraid of fiber patch cords... they're much more durable than you realize. I've pulled hundreds of thousands of feet through AT&T offices and dirty connectors is a bigger prob over breakage.
I'll always be a bit nostalgic for the R720; it was the first server install I did as an IT apprentice (replacing a 2950). Learning about iDRAC was mind-blowing at the time.
where did you learn about it at? any good resources? im using one now for my home nas and was wanting some info lol
What impresses me is that big Clive still uses a phone for much of his videos and gets decent results.
Yes but his video are all the same overhead shots so much less demanding on a camera, he does use open camera to boost the bitrate up a lot as well
@mipmipmipmipmip Not much different from CRD, is it?
He uses an iPad sir
@@psychoacer he also uses a moto g7 power.
Great vid and a real deep dive into the woes of storage arrays and the issues they come with. I implemented huge arrays in my last job - think the biggest was 1.4pb addressable for 120 edit bays for a large media firm. Worth watching for anyone wanting a grounding on almost everything that can go bad, including the FC (Real world networkig) vs IP stateful. Good stuff!
Having not the greatest night then seeing you discover the battery came out made it all better.
Excellent information, great humour!
Great presentation/with Gfriend behind the camera combo with you doing the tech talking.
Quality video!
Direct attach is great until you're trying to shove a couple dozen of those SFP ends through the wire management in your racks, then you'll wish you sprang for the fiber.
If you want fast low latency network storage you have to go iSCSI. All of the file protocols assume the network is crap and trade speed for reliability.
I'd love to use iscsi, but I keep reading that the drives shouldn't get more than 50% full when using it. That's too much wasted space for me.
@@dragonheadthing Most iSCSI use cases involve many simultaneous read/write operations and that recommendation is to reduce fragmentation on platters that consumes time. Irrelevant on SSD and also not applicable to your single user set-up so you can safely ignore the advice.
However it does require different HBAs, not sure what cost those would be refurbished but new they're expensive. And although I know th drivers are in Windows Server I don't know if they're in the desktop editions.
@@calmeilles Aaah! Thanks for the info.
Why don't you record in ProRes 422 or LT? Visually, you won't see any difference, but it will be in terms of space. Example with 256GB:
ProRes HQ 41 min
ProRes 422 68 min
ProRes LT 98min
Even RAW 8:1 in 6K can accommodate 41min and it is definitely better than ProRes HQ for my taste.
By the way, the recording quantities on the website are calculated at 30p, at 60p this doubles. Does it have to be 60p?
I have to agree with the question of does it have to be 60.
I agree, recording 24/30 fps prores lt in hd will probably be more than good enough for these videos. Just make sure you do the fake upscale to 4k for the output file to youtube, so its compression algo doesnt completely destroy it.
I wonder if you could 3D print an insert for that Kingston card reader dock thingy to make them fit smaller drives better.
I thought about that too. To make things simple we could print just a couple of cubes to pad the sides, and not have to measure the curves.
@@DiThi A chamfer on the outer edges to avoid the curves would still make it one piece and thereby easier to use.
Had the same thought, probably just needs an extra "push", either from something breaking or someone insisting/sending them to him XD
@@DoubleMonoLR admittedly, owning a 3D printer makes me want to find excuses to use it, like this, so yeah I might have a slight Impulse to overcomplicate it
I love how every correction you cut to in post ends with “We now return you to (different show-title every single time)”
easily one of my fav youtubers to just nerd out about cameras and now sysadmin related things. thanks buddy.
A 3D printer to make some adapters for the T4/T5 SSDs for the USB dock and the camera clamps would solve a lot of issues from what I hear.
I have had several conversations in the past 3 weeks that covered most of what you have now discovered and most of the problems you mentioned are NOT rare or unusual.. One comment is that there are two versions of SFP that are mechanically interchangeable but not electrical. SFP is for 1 GB, SFP+ is for 10GB. Where you mentioned SFP it should be called SFP+ instead.
Everyone I know uses the term "SFP" to generically cover SFP, SFP+, SFP28, QSFP+, QSFP28, XFP, and whatever other standard or quasi-standard plugs into the slot on a networkable device. Unless, of course, the context of the exact specification matters. But if it doesn't, it's "An SFP," and that's good enough to convey what you meant.
I knew somebody would have already pointed out the difference between SFP and SFP+. To complicate things more some but not all SFP+ ports are backwards-compatible with 1G SFPs.
One reason for some drives getting worse is due to the chip shortage in 2020. Some drives even got worse than their earlier runs. Even after the chip shortage was over, at least a few companies found that it worked really well to launch drives with premium parts, then slowly replace things like controller chips and DRAM chips with slower and less capable/slower spec versions. The initial reviews would say the drive was fast and would bury the newer reviews talking about how slow they were. Lately Amazon has been prioritizing showing me reviews made in the same year, though, so that might specific trick might not be nearly as useful to them, but capitalism will always be a race to the bottom in terms of quality in order to maximize profits. :/
It's lovely hearing you rave about the features on server grade kit. I've been a SysAdmin since the late 90s, and all our kit had remote management. It was over serial cables, but it worked. I'd love to have (be able to afford) LOMs on all my compute widgets.
"Oh, Dell really kept the design of their servers for some time" - I got out of IT in... 2013! Solved that problem. Worked at a tech uni 2009-2013 and we used Dells for the most part servicing ~450 users. The 500 Mhz Poweredge in the basement cable closet (and under the radar storage room of tech junk for me) did serve us fine for e-mail until 2010-2011 when central admin and other paper pushers made us join the centralized M$ solution they had payed good money for. We upgraded our poweredge 2950:s to R710/720 (I think) for VM:s. ILO-systems are great stuff! Esp if you, like I did at a previous job, were dealing with hundreds or even thousands of blade servers. That was before the company got into VM and just provided blades for customers. This video did bring me back to fun times, but I left for good reasons and are happy where I am. Thank you for your videos, I stay for every minute!
But I completely agree with you on enterprise gear at home. They are incredibly loud and eat a lot of space and power. Been on that route, vintage machines mostly. Running a Compaq Proliant Dual Pentium Pro for just DNS while your Sparcstation with added external 1,3 GB FULL HEIGHT SCSI HDD serves web pages in 2009 is cool in a nice way, but hey.. At least I stopped before I actually used my Microvaxes for something, :) The Sun E450 with its 20 HDD:s made a good TV bench at least.