Having worked with many versions of LTO drives/tapes over the last 20 years I know all too well that they can/do fail, don’t fall into the trap of thinking that the data/tapes won’t degrade over time or get chewed up by a bad tape drive leader. I highly recommend you create two tapes for each backup and keep a good backup of your backup library master data base, so you don’t have to rescan all the tapes to rebuilt it. It’s also important that you follow the correct storage guidelines (temperature, humidity etc)
I was thinking the same thing, ... I also worked with tapes and they are not always reliable... storage is an important part; I would keep the tapes in an airtight box keeping it away from heat.
I too had worked with tapes in the past with a large 50+ tape machines and was thinking the same thing. At least once a week a tape would fail requiring backups to be ran almost constantly. Plenty of multi tape systems out there which could allow easy striping as just a backup... Of the backup.
As an IT guy all I have to say is "MAKE SURE YOUR BACK UP YOUR CATALOG DATABASE" this is the database that tells you what is on each tape and test its restore every so often on a test machine. This is so important and no one thinks about backing this data up. Make sure the backup is automated and is taken off site and ensure you have multi revisions of this database. If you don't have this catalog all your tapes are useless or you will have to re catalog all you tapes. If you only have 4 not an issue but once you get to a 100... you could spend a month cataloging your tapes!
@@jirehla-ab1671 I'm sorry you want to reuse your SSD or HDD after it's repaired? If you have any issue with any media you 100% replace it and recover the data to a new medium if it's important
@@steve_main cause I already backed up the data to the cloud i am also planning to backup it to my other google drive, do you think it is also better to seek replacement through a warranty or just buy 2 new hdd / ssd instead?
@@jirehla-ab1671 If it's under warranty then have a replaced under warranty. why would you buy new hardware if it's under warranty? 100% replace it if you've had any issues with it. if not, any data you put on it expect it to be gone
@@steve_main cause for me I accidentally fall it to the floor and I don't know if there's already physical damage to the external hdd and western digital said that the warranty will be VOIDED if there is a physical damage and will NOT be replaced
If you really care to preserve everything then 3-2-1 backups are critical. Tape can be a key part of this, but it can't be the only thing. 3 - Total copies 2 - Different types of media 1 - Copy at a different physical location
plus 1 - Copy offline. If you delete a file unintentionally and the deletion is immediately mirrored to all your backups you are screwed. Better have one backup run from time to time so you can go back to your backup from last week or so, where the file has not been deleted yet.
for example: a file saved on your computer and mirrored to the cloud plus backup every x days to an external hard drive fullfills all of those criteria.
As has been said below already, make sure you have two copies. One on-site and one off-site. The fact that you haven't lost data is more of a miracle than anything else. Also tapes do suffer from bit rot as well (they are magnetic as well) Insure you store them in a stable environment, while they are great for archive storage the archive part only works if they are treated as per their specs.
Yes, now he has 45 tapes, time to get a second set of 45, a second drive and make a copy of each one as it is done. That will at least give single redundancy. Then a nice data safe that is rated for at least 3 hours fire resistance to store them in, preferably in a room above flood level, and with non flammable walls, and temperature control. That 30 year life assumes storage at constant 20C and 50%RH, which is only achievable in a massive data centre, and also assumes that you use no compression, and maximum error correction on the tapes.
While this is a big improvement vs random hard drives, as someone who archives data for a living I'd caution you against just putting everything on these tapes once and assuming they'll be safe for 30 years. That guarantee is more like a warranty than a specification, and I've had tapes fail much earlier than their rated lifespan. An immediate solution would be to use some form of parity or mirroring to ensure every file can't be lost with a single tape failing, but longer term I'd highly recommend backing up your archives using multiple forms of storage, in multiple locations, especially since this is your livelihood and if something like a flood hit wherever you store your tapes, you'd lose everything. A cheap option would be to make a JBOD server using your existing HDDs you just moved your data off of. JBOD is short for just a bunch of disks, and is exactly what it sounds like. It would also let you use a filesystem that automatically checks for and fixes bitrot, like ZFS, and mean you would have access to your entire archive without having to plug in a certain tape, or even multiple as LTO tapes can store a single file across multiple drives to ensure you're using all the storage it offers. This is why they have the barcodes as standard, as many commercial archives using LTO tapes will have a tape library, which are basically jukeboxes with tape instead of vinyl, and the software can automatically switch tapes to retrieve data spread across multiple drives.
Not to forget that that spec only applies if you store the tapes and drives dust-free and climate controlled (temperature and humidity). That's a lot to ask even for a small business.
To repeat what others already said... But since it's important and could avoid data loss. Do NOT expect 30 years out from those tapes. LTO is by no means safe from data/bit rot and for important data you should rotate every 5-10 years depending on storage conditions. The 30 years they market them with is during very specific conditions when it comes to temp and humidity, treat it as marketing buzz. Also, do NOT throw them around in luggage and avoid X-rays, strong magnetic fields and so on. And for the love of God, do NOT drop them. They are fragile and you just have to disassemble one to realize exactly how fragile they are. Have had more then one eaten by tape drives in the past (LTO4/5). You should have two copies of all critical data. If possible, use LTFS to avoid being locked into one vendors proprietary db's and system and keep backups of any eventual databases. But yes! When dealt with properly and when knowing about the quirks and limitations and gotchas LTO is very cost effective and nice. But like everything, LTO have its limitations and should not be treated as a indestructible bullet proof Jesus-medium that will save us all. 😎👍
I've worked with the LTO tape drives for a long time, although I worked primarily with Tape Library machines, that is, robots that you load dozens of tape into and has more than one drive. The backup software has a master database of all the files and tapes, and does backups, automatically changing the tape when the one in the drive becomes full. As other people have said you need *two* backups of everything, and store one copy in another site, preferably that is climate controlled. You should also periodically verify your backups. In the long term, you should also consider budgeting to change storage mediums every 5-10 years. Just like you are going from Hard Drives to LTO Tapes, in 5 - 10 years, you should be looking at the next great thing...maybe SSDs, or maybe holographic storage...
I've used LTO-4 for 12 years, and suddenly tape drives fail and HPE has forgotten how to repair the old drives, apparently throwing away all the repair manuals. That plus the nightmare of Indian support outsourcing plus only about 2 companies left actually making LTO products, despite originally introduced as a monopoly-free category as DLT had become a Quantum monopoly. So, an old suggestion that I sadly ignored, is to purchase a spare drive (hugely expensive), test it and store it with the off site backups. Once you need the spare, immediately transfer all backups to a new physical format that can be secured with the same measures. Oh, and in case you missed the small print: New LTO versions cannot read tapes from previous versions, except for one overhyped situation. Now I'd need to purchase an otherwise useless old LTO-5 drive to read the old tapes, while also investing in a new format to store new backups and copies of the old ones.
@@johndododoe1411 I totally agree with everything you said, especially the backup drive unit! But I would rotate it in every once in a while since it is bad for equipment to sit unused for long periods of time.
Magnetic tape is so cool, i'm still shocked after i found out that a vhs can hold gigabytes of digital data depending on the length. 18tb in such a small package is unbelievable data density
Theoretically yes. vhs is helical scan, the data is on a constant spiral, not linear. it doing so because vhs is optimised for analogue video, each scan line is physically present on the width of the tape. data however, is linear in nature, thats why we have data tapes like lto are always store in linear fashion. makes reading and writing much faster and searching easier.
@@gernhartreinholzen3992 Back? Tape never left at least in industrial and professional enviroments. Lto is a backup standard for decades now with 9 generations of tape and more are planned so its not obsolete by any means.
@@gernhartreinholzen3992 We never stopped using tape. A lot of supercomputers have physical tape storage libraries where an arm chooses a tape and inserts it into the drive for the computer
Just don't fall into the trap of routinely cleaning an LTO drive, clean it when the drive requests a cleaning by itself or in case performance takes a sudden dive.
Specifically, the Single Character Display (technical term for that amber 7-segment display) will display "C" when it wants a cleaning tape. Unless there is a serious problem, it will wait for you to eject the tape that's in there first before displaying it. As well, the drive has a brush built into it. Every once in a while, you will notice it will stall for a few minutes while it unloads the tape, brushes the head, reloads the tape, and then goes back to the point it left off. Another thing: If you are writing a lot of small files with LTFS, consider using the sync_type=unmount option, so that it's not spending a bunch of time shoeshining between the index and data partitions.
I'm just going to echo what other people are saying and strongly consider duplicating your backups and storing them off site somewhere. Redundancy is just as important as backing up the data itself. It doesn't do you much good to back up all your data if it's all sitting in one location waiting to get destroyed in a fire or a tornado. You have a lot of very valuable data and you're now stepping into the world of enterprise level solutions. I would find an off-site backup location in another city, or state if you can mange it. Redundancy, redundancy, redundancy!!!
Of all the TH-camrs, your channel has been the one that has been in the back of my mind constantly. as someone who's a tech enthusiast, knowing how much raw data you generate every video and trying to come up with a solution had to come in occasional mental exercise for myself. I knew tape would probably be your archival solution but I didn't know the technology had progressed to the point where the write speeds would actually be faster than most hard drives. Your probably still needing a Linus grade storage server for, effectively, a scratch disk during editing but now you can of-load one your done and can stop having to build servers. Sweet
Love that you now have more cohesive way to store your vids, wondering what happened with the NAS LTT helped you out with. Love all the vids! Cannot stress enough that you should always have AT LEAST 2 copies of all your data (or anybody else reading this comment)
just wish the tape drives weren't so expensive. Or it was easier to rent them or something like that. since archived storage that doesnt need to be accessed you dont need the drive all the time.
The NAS would be great for more short term storage when he needs access to data but his local computer doesn't have enough storage/he might want to reference later. The tapes are great for the long term storage and archiving of his footage.
For someone who only genuinely enjoys the content and knows nothing about the logistics and workings of your results, its really cool to see this. As much as I wish you both were super close to have more time in this channel, life moves on and project priorities change i am glad you keep slowmoguys still going!
I have 1.04PB of storage which is backed up to large archive drives. I thought about using tapes, but they're likely to degrade at the same speed as a hard drive. The fundamental difference is that with a hard drive you can repair the fading magnetic fluxes. So every few years, just copy off the data, reformat the drive and then rewrite it. This is something that would be more difficult to do with tape. So as someone has pointed out... Don't think of tapes as a very long term backup solution. Not unless you're willing to maintain them too. Also bear in mind that unlike a hard drive, your tape machine is going to wear over time and you also run the risk of tape getting chewed up. These issues are less of a problem to a hard drive as it is a self-contained unit. Moreover, tapes will be much more sensitive to things like moisture and temperature than a HD because the platters in a HD are sealed. You do run the risk of motors failing in spinning discs, but this is more likely through use, rather than sitting idle and certainly less than the degrading of a LTO machine. So always have two drives and make sure they're refreshed at a reasonable interval.
You can repair the magnetic flux as you say, if the rest of your drive still works. Thats the thing right there, these tapes dont have any subsystems that will fail and entomb the data in a dead box. If the tape drive dies, you get a new one. As this is LTO 9 you could argue it may be cheaper to try and move a dead HDD's platters to another compatible drive, if you have someone who has a working drive to transplant them into. Plus the tapes are very immune to fading like old reel to reel audio tape can be because they are made of very different stuff. You are more likley to have problems with tape damage over anything to do with the magnetic fields.
@@dlarge6502 I've used tapes, lots of them and the magnetic fluxes of tape degrade at about the same rate as a HD. It's slightly higher for tapes because tape layers are wound upon each other. HDs die from use, not usually from sitting idle. They die either from motor failure or the controller dies which is why they have a MTBF indication. Each HD is self contained so the MTBF is spread over the number of drives for the archive, but a tape machine is used for all the media, so it's failure rate is significantly higher as is the chance of it chewing or having a r/w failure. The storage requirements in terms of moisture and temperature is also significantly higher for tape. Although admittedly, moving platters between HDs is extremely tricky, it can be done as a last resort. But get a chewed tape and it's game over. The only real benefit a tape gives you over a HD is from shock resistance, if it's a spinning disc and size. This can be eliminated with SSD storage though, obviously at a higher cost. Add to that the random access of a HD coupled with the higher transfer speed, especially from SSD, the HD always ends up on top in terms of long term storage.
@@thewelder3538 Well, at least your second comment made the argument sound more factual and not just a HDD fanboy making stuff up (your first comment gave that impression).
@@johndododoe1411 Agreed, it was a little wishy-washy. Although I do enterprise backups, it's hard to find the level between trying to sound like the ultimate authority on the subject and someone who's just a fan-boy.
0:46 Some fun facts: 720p at 82,000 frames per second is equivalent to 8K video at 2,278 frames per second, and an hour-long 720p@82,000 video that is actually 1 hour long and not a 90 second slo-mo stretched to 1 hour would be 324TB, and he would only be able to store 2.5 1hr long "720p" videos in his 810TB drive. Even if you somehow found the capacity to store a video that big, good luck 1. finding storage that is fast enough to show 82,000fps 1280x720 video (90GB/s is 12x faster than the fastest consumer-grade NVMe storage that is available today, and that $850,000 1 petabyte 100GB/s storage server that Linus showcased in his video a couple weeks ago (normally $1,000,000 but ignoring the $250,000 GPU cluster with 8x Nvidia A100 GPUs that is irrelevant to the storage portion of it that it includes) would only *theoretically* be able to handle it), 2. and *then*, good luck finding a CPU fast enough to decode (play back) such a video in real time. This is not a perfect comparison because of different codecs of varying quality, but knowing that my M1-powered 13" MacBook Pro can't quite play back 8K@30 VP9 video on TH-cam without buffering, you'd need a CPU 76x faster than an M1 to be able to decode an 8K@2,278 VP9 video uploaded to TH-cam (assuming TH-cam even allows that video to be uploaded). Even if you bought 2 of the just announced $12,000 flagship AMD 96-core Zen 4 Epyc server processor, theoretically it would only take you 40% of the way there (30x faster than M1).
Until I retired, I was heavily involved in the design of truly large scale mission-critical IT systems. I was also the first in the company to exploit LTO (generation 1). It proved to be a good bet on the technology, and we eventually had massive robotic libraries which went through generations of LTO. However, one single copy is not enough. If you are going to use if for archive, then do keep multiple copies. That's a lot of data to lose in one go when a tape fails. Also, one copy should be kept off-premises. Also, be prepared to do a media conversion with new generation of LTO, and it's very good practice to periodically refresh anyway, to prove you can read a tape and then you can deal with a failed tape using one of your alternative copies. This also means a single drive isn't going to do the job either. Note that an LTO can write data a lot faster than it can be read off a single HDD. What you do not want to get into is a situation where you LTO drive is not streaming and you are in write, run out of data, rewind a bit and start writing again. It massively slows throughput and is very bad for the tape mechanism and tapes if it's extreme. LTO drives will throttle down write speed to some extent. It it's a problem, then stage the data to an SSD first before backing up. Of course writing large amounts of data to an SSD will affect it's life too, but at least it would be done once. In our case, we had massive enterprise array storage systems which striped data over RAID sets and could keep up (whilst still keeping systems going), but even then it could still be a problem to keep the LTO drives streaming efficiently. Also, if LTO is used for archival purposes, the tapes must be kept in a temperature and humidity controlled environment free of dust. Nothing too drastic, but something to be careful of. Extreme heat would be bad for them. There is quite a lot more to backup and archive on tape then there might appear to be, and it really requires experience. Of course, if you are keeping tapes and the original disk drives that does limit some of the risks, but this all needs a lot of care.
Gav has always been so good at making the tech side as interesting as the demonstration side. Cheers Gav! I am from Bristol! :D I live in Louisiana now, I miss the UK :(
The barcodes were used for larger units which had mechanical arms which auto loaded the tapes into the reader/writer all enclosed into a cabinet which would live in the rack room. I remember those days...
Just to chime in here: definitely keep a second storage backup if it's world-ending if you lose the raw. Tapes are pretty hardy but the chance of a file corrupting or at least degrading in that time span is pretty high (though better than a HDD). Would reccomend a second tape backup for any files you definitely need, and a third backup in a different storage medium (HDD, cloud, etc.) for anything absolutely critical. 2 tapes will do you for 30 years, but not if there's a fire where they're stored. Off-site backup in a different medium is airtight for anything critical and gets the best results :) But if it's just nice to have, one tape will keep the majority of your stuff for the 30 years bar any catastrophe
That's bonkers! I didn't know such efficient storage even existed! What also surprised me was that 8mm film converter on the shelf there; I didn't expect you of all people to have one of those, lol!
I remember fixing a pretty big tape library machine a while ago (a tiny gear had cracked), it was super cool to see how it worked with a moving, mechanical arm on a pulley system to move the tapes around and stuff inside
I find it crazy that these machines that cost sometimes as much as a house use any plastic gears in them. I suppose they are expected to be decommissioned and superseded long before the plastic gears degrade, but that's clearly not always the case.
@@jamescollins6085 Hehe, the one I worked on wasn't quite a house in terms of price, but it sure saved the client probably a good few thousand euros. Bought a pack of gears from ebay, used a soldering iron to melt it onto the shaft as it was smooth.. and a few attempts later it worked just fine
Another benefit of tape is how natural is to have offsite and offline backups (the first protects you from disasters like fire or crime and the second one protects you from ransomware)
DO NOT take the tapes on a plane, the scanners may wipe or corrupt them. When you are copying, you need to use file verification. Nothing worse than thinking you have a file, only to find it was corrupted right off the start. Take the original drives and store them at another location. Also, you may want to create some parity files (.par2) just in case it gets corrupted later or as a later verification of integrity.
Scanners won't do anything to modern tapes. Maybe 30 years ago the motors that drive the conveyor belt might have caused problems for low-density tapes. High density tapes like LTO need a huge amount of flux to cause issues. Scanners won't do it.
I've always found these types of videos interesting, but since I started properly working as a video editor and working with VR video a year ago these are absolutely amazing. That much storage is a dream
A few notes on LTO as someone who's used it extensively for video backup: 1) Generational compatibility -- while they tapes may be guaranteed to last upwards of 30 years, the same definitely isn't true of the drives. A new generation is released roughly every 3-4 years. So LTO-10 is due in about 2 years. And LTO-9 drives will likely still be available and in warranty for a couple of years after that. And LTO-10 will read LTO-9 tapes. But once LTO-11 is on the scene your days are numbered when it comes to being readily able to read LTO-9 cartridges. 2) Tape failure -- they are not perfect. Probably more reliable than 3.5" drives in cold storage, but not 100% reliable. Our backup plans always called for duplicate copies, stored in physically separate locations. On restores we did see failures. Also, while they're a great low-cost high-reliability storage, tapes are slow and annoying in many ways. I've typically preferred to keep media backed up on 3.5" drives also, as a first-instance archive that I can generally access quickly and with little effort.
Thanks for the tech update. It's always interesting seeing storage problems solved, that us mere mortals (in a data size sense of the word) will never face!
Former IT gal here. I can't stress hard enough how right all the other IT professionals and data archivists commenting on here are. You need *_at least_* two copies, one stored safely offsite.
Our biggest fear in our computer workshop and data storage room (at a university) was that one day, someone would come in with a degaussing wand and destroy all our backup tapes. We were so paranoid that we had an off-site secure storage area that only 3 people knew of.
That's a bit much don't you think? Wouldn't that storage room only be able to be accessed by verified people anyway? Or was it not locked down. Sorry, not trying to come off as rude just genuinely want to know why.
@@johndenver6769 The uni didn't quite have the budget or sense to provide a secure server room. So we took our own measures. Losing the data of thousands of off-campus distance ed students wouldn't have been a good thing. Not to mention study materials etc. We looked after data for the publishing section as well.
@@johndenver6769On a campus, IT and Facilities are often forced to share spaces. An off site colocation facility goes a long way to having dedicated space for IT to have sole access to.
Part of the reason the BBC wipe over recordings was cost (magnetic tapes cost much much more than they do now), another key reason was contractually in some cases they were not allowed to reuse recordings, there was no reuse infrastructure or licensing agreements back then. Many historical recordings were destroyed just because they were not allowed to reuse it, even on film as opposed to on tape.
"All in the same format, and I only need one device." I would suggest buying three units, one of which is tested and repacked safely, and that you save the data on at least three tapes. And then make Dan keep that third copy of the tapes at his house, in case there's a fire. Also, keep the original hard drives as a last ditch solution. If you could copy the data to clean fresh disks, that'd probably be good too.
It is better to copy the data to tapes and then after checking that the data is fine format the drives and give it away / use it for something different. Hard drives are not the best long term storage solution and can be used. It is a huge opportunity cost to keep them around.
Tapes can definitely have bit rot. And these tapes are guaranteed to last 30 years, just like LED lights are guaranteed to last ten years -- I.e., not. Tapes are a fine backup medium on top of other mechanisms, but I'd never use tape as my only backup. Not even in a 3-2-1 pattern. Disclaimer: I was a technical reviewer of the O'Reilly book "Backup & Recovery" by my friend and former co-worker, Curtis Preston.
If I have learned anything about specialty hardware from my time working in IT, it's that you should go ahead and buy another one of the symply drives. No matter how good the company seems to be doing now, that unit could fail when you really need it and there will be none available
@@HydroHUN I'd say hard drives experience mechanical failure ("click of death") long before bit rot. The HDD in my IBM Model 30 from 87 still works and that machine was rescued from a damp container at the recycling center
@@sylvester4207 It is semi manual, he inserts the tape drive and it just goes in the background. As far as I understand the file movement is automatic as long as the drive is not full
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive. So no, an LTT server isn't necessarily the best choice.
Make sure you do regular catalog backups. You should also consider using a tape vault service. That way its offsite for DR redundancy, and they will keep it in a humidity/temp controlled storage to keep them in the best condition so they last for a long time
It's awesome to see how things evolve. It would be cool to see what your production path looks like from recording to ingestion to process to archive. Do you still have the 45 Drives server as well from Linus? Great video!
Very helpful! We’re going to need a long term backup solution for our observatory data, and this is exactly what we’ve been looking for. Definitely going to back up with tape! It makes it so easy to back up off site
@@Bruh-zx2mc yup! We’re going to have our NAS store the main sets of data, and then tape on site and off site for long term storage. Our current NAS is 36 tb but I’m sure we will get it closer to 120 tb. We will save up for tape when we can afford it
This, this and 100% this! Please make sure you have an off-site backup of your entire library; a theft, fire, flood can wipe out everything in minutes...
As many people have noted, tape drives are a "quantity over quality" upgrade. You sure are getting a lot of storage, but that storage may or may not be less reliable than a mechanical drive or SSD. Mechanical drives at least have the advantage of warning you when they start to die, unlike tape and SSDs.
This was basically just an advert, but very interesting and nice to see Gavin is getting more help in the inevitable data hoarding that this job requires. Linus tried, but the Slow Mo Guys are on another level of storage needs.
@@akyhne They gifted him the drive and tapes in exchange for him creating this video in which their brand is mentioned to hundreds of thousands of people. That's literally what an advert is.
I've somehow managed to cram my whole computer life into 1tb up till now. Finally had to order a 2nd TB last week though, because my old 1TB drive is completely full and I don't have anything disposable to free up space. Managing almost a petabyte of data sounds hard.
This is what Linus should have supplied during your collab, tape is the obvious solution to your problems. When I interned at IBM in a datacenter position, they had a machine with hundreds and hundreds of these tapes. Unit had a robot arm to read the barcodes in the library of tapes and load the appropriate tape on demand. Had to be petabytes of data on them. I remember the 4 tape heads on the high speed recorders were named John, Paul, Ringo and George lol. Someone there had a sense of humor. Then a guy would manually load them from the high speed output to the reading library with all the others, and add the barcodes to their spines.
@@jshklsn Linus is good for general education and entertainment. But his approach o industry standard equipment is usually quite lacklustre and backwards. His use of storage systems is quite painful to watch.
To be clear, I'm on the side of using things for what they are designed. Need to store massive data over quite a few hours? Tape. Need to store data real fast, multi-user? SAN or something local. I just think tape better fits this use case.
I recommend not relying on tape alone. Make sure to get some new hard drives, create a ZFS RAID7 (triple redundancy) on them and copy all data you already shot onto it. You can spin it up like once a project is finished, put everything on it and power it down again. Tape on the other hand should be just a backup, so make two of them and store them at two different locations. Make sure to write to two tapes for each backup location and alternate them. Meaning: You write currently onto tape A and tape B on backup location X, and tape A and tape B on backup location Y. But you fetch tape A from backup location X and tape B from backup location Y. So both tapes have completely different content. You then write the current project to tape A and the previous project to tape B and bring them back to the backup locations. On the next backup cycle you alternate and fetch tape B from X and tape A from Y and so on.
Didn't you get a 130TB storage server with LinusTechTips? Or was that never used? Edit: Thanks guys for the update. I didn't know it filled up that fast !
a tape backup is usually a 3rd level backup, not a primary, normally you would have a live source, a hot backup, normally on instant access HDD (NAS/iSCSI), and then cold storage, aka tape. then if you are wanting to achieve it, you would normally have two independent copies, plus a tape backup.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive. I'm pretty sure, they wouldn't use tapes, if they sucked!
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Storing on harddrives, would be incredibly expensive.
Tapes absolutely are susceptible to bit rot, whether the tape is damaged by environmental conditions (i.e. literal rot), or the signal fades over time as neighboring magnetic charges interact.
I agree it's tragic that so many shows and pieces of footage are lost forever just because some corporation was too lazy to store a copy. I love to watch old entertainment from back in the pre-internet ages. I often wonder if the old lesser known cartoons I used to watch still exist in a vault somewhere, or if they only exist on random people's VHS tapes.
If you're curious about how much it'd cost to store 810TB in the cloud while keeping access to it, you're looking at a monthly fee of about $3,000. As a backup solution (accessible once or twice per year) it's a tenth of that.
I had a job at a datacenter once where I was managing daily backups and they used these tapes in this bigass IBM tape ROBOT. It was basicly a couple of serverracks made into an enclosed tape station where it had like 8 tape stations that all writing simulatiously, an "arm" like a printer or something where it would move between the tape writers and this huge array/cluster of available drives, just sitting in slots all over the place. I think they're called "IBM Storage robots" or something like that. One of the daily tasks was to have it eject the tapes that needed to be put in a remote location, and swap them out with the old drives that needed to be erased and put back into the array.
Every time I look into tape I am shocked at the prices of drives. Been sticking to HDD and a HDD dock and basically using the HDDs like tapes. I need to completely revamp my backups though so I can have more automated versioning. Right now it's kinda manual and is just me choosing what drive to put in to update to latest backup.
I'm shocked not only by the prices of the drives, but of the tapes as well. In many cases, you aren't saving much over the cost of a comparable hard drive, and you have the added inconvenience of packing and unpacking all the data every time you want to access any part of it from the tape. I also don't understand the obsession with keeping source material that so many TH-cam creators seem to have when the bitrate that it will be compressed to when streaming is hundreds of times lower than source anyway. I'm sure he could re-encode the footage using the new AV1 encoding format and save mountains of space with no decrease in visual fidelity. Perhaps like with photography, it is necessary to keep the files in their raw form to aid in post-processing. Either way, it is his footage to treat in any way he sees fit.
@@jamescollins6085 The tapes are brand new, this is literally the first time I've seen one outside of a product sheet, thus they will be pricey for a few years. I'm totally shocked that anyone would pay $1000 for a phone too.
I remember being interested in digital tape storage as a kid, back when I was backing up my Windows 98 files on 250mb ZIP Discs. I had no idea they were still used, and had grown to that capacity. From what I (barely) remember back then, the largest tapes you could buy in stores were like 40/80gb. This probably would have been around 2001 or 02.
It will be interesting, As a former main frame computer engineer form the days of huge tape files, in general a tape reel would be at least a mile long and often twice that. We used to run archive tapes from end to end periodically in fast forward to make sure they wouldn't crosstalk between tape layers with time. Tape can and does deteriorate over time, but good luck. The bigger the tape the more you loose when it goes bad! Add to that access is serial (because it's a tape,) getting to a file at the other end can take an appreciable time where as solid state or HD is much faster to access.
I was where you are now back in the 90's when i got hold of a jazz drive and a ton of media. Youll find in 5 years when your the only guy with that machine you will move all your data back onto the standard format of the day. With jazz it was serial ports disapearing from the market, you just cant risk never being able to access your data again. Its no good for that lifetime storage feel thats inspiring you. But yup feel your pain, know the glory lol
0:05 It technically is a paid video. Paid with a free product that allows content and gives them free advertising, so check that box. If you could send the tape drive over here when you're done that would be great ;o)
Old IT guy here. Unless you have 3 copies in 3 different locations, you don’t have a failsafe backup. I use LTO but external threats like fire and magnetic fields are still a risk. S#!t happens. If the data are critical, then at least find a deep storage option online like Glacier storage or at least another LTO copy in a temperature controlled vault offsite is called for. Also, test the backup media occasionally to see if you can restore it. I’ve been bitten badly in my early career trusting the software saying “no problem” then have it fail when we actually needed to restore the data. Even the vendor couldn’t recover the files.
The benefit of tape is the cost per GB at scale - the system Gavin has, with 810TB, comes to 0.024 per GB, physical HDDs would be about 0.035 per GB, and the cheapest AWS storage tier (Glacier Deep Archive) would be 0.046 per GB over 5 years (assuming no data is ever actually retrieved).
So, we're back to where we were forty years ago. Tape on cartridges/cassettes for data storage. Gives the micro-tapes from the original Star Trek new credibility.
No, we are not going "back". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
I just saw a short video on TH-cam, from CERN. They explained, that they are connected to 170 data centers around the globe, to process the data from the LHC. They also have their own computer on site, with over a million cores.
My favourite lesson about backing up data is how many businesses in the World Trade Center lost their backups. They were in the other tower. Emphasis needs to go to the *off site* part of the 321 backup strategy being completely off site because the incident could affect more than just one of the places your backup is, especially if, for example, your off site backup is _next door._
I hadn't heard about this tapes, but it sounds a lot like a miracle product. I think it would be best to use them as one more backup besides the HDDs and SSDs.
Tapes are absolutely affected by bitrot and the requirement to exercise the tapes. I have worked in backups since 97 dealing with one of the major backup softwares and I can tell you that I've seen more than one customer that realized their tapes from 5+ years ago are not as reliable as they had hoped they would be.
older generation of lto drives have been falling in prices like a rock, I personally use a HP tape library that holds 23 + 1 cleaning cartridge, 2U rack unit holding 2 drives that can be used at the same time, it supports both LTFS which I believe is what you are using: makes the tape look like a flash drive (just dont try to use it like one) and more traditional backup software like Veritas Backup Exec which is what I use with a client software on all my VM Servers and desktop workstations, the barcodes are actually used by the library;s builtin barcode reader to scan in order to know which slot it took the tape from and where to put it back or determine if there is a tape in the slot. Been using it for years and super happy with it. these are 2 Fibre Channel drives,, don't let that freak you out, all you need is a cheap dual port card that syncs up to 8Gbps depending on the generation of drive, just remember to make a rotation policy so not all the tapes are at the same physical location due to theft, vandalism, fire etc...except mines is non portable mounted in a cabinet and requires 2 people to move safely. If it is a HP and you use LTFS, all files copied onto tape gets indexed, so as long as the tape is labelled, you can use the provided utility go search which tape the file is on, load and restore; all very economical and cheapest per GigaByte. The same hardware would have cost an arm and a leg when new! just wait till lto14 comes 570TB - 1440TB depending on how compresssable your files are! Could probably buy a economic car for the price of one of those units!
We are not going "backwards". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
One of the main reasons why those old BBC tapes were written over with no thought of preserving them was the cost of magnetic tape was really expensive at the time, something that you've pointed out is definitely not the case now. So like so many people have pointed out in other comments back up your backup!
No, we are not going "back". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
Whoever told you tape isn't susceptible to bit rot lied to you. High density tape has an even shorter shelf life than the cheapest hard drives. And good luck finding that one file you're looking for when you ran it in a job spread across 10 tapes. It's not exactly on-demand data retrieval. And don't even get me started on parity. I've done the math and, byte for byte, HDD is cheaper and getting even moreso, and if you're leaving em unplugged on a shelf they'll last longer. Though for your particular use case you're probably long past the point of just needing a software-defined petabyte NAS so you can upgrade capacity on the fly. But hey, if you're satisfied, more power to ya. At least you're right that tape weighs less.
Have you thought about what will happen to all your old hard drives long term? The British library or the national film archives could look after them as you are getting to the “national treasure” stage!
I know this video is old, but it's always a good idea to follow rule 3-2-1 of backup - 3 copies, 2 different mediums, 1 on remote location. It's the baseline of not losing your data no matter what
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
Having worked with many versions of LTO drives/tapes over the last 20 years I know all too well that they can/do fail, don’t fall into the trap of thinking that the data/tapes won’t degrade over time or get chewed up by a bad tape drive leader. I highly recommend you create two tapes for each backup and keep a good backup of your backup library master data base, so you don’t have to rescan all the tapes to rebuilt it. It’s also important that you follow the correct storage guidelines (temperature, humidity etc)
I was thinking the same thing, ... I also worked with tapes and they are not always reliable... storage is an important part; I would keep the tapes in an airtight box keeping it away from heat.
Exactly. At least 2 copies of each file is necessary.
I keep 9 copies of each tape incase the first 8 fail on me. Thinking of going to 10 soon.
I too had worked with tapes in the past with a large 50+ tape machines and was thinking the same thing. At least once a week a tape would fail requiring backups to be ran almost constantly.
Plenty of multi tape systems out there which could allow easy striping as just a backup... Of the backup.
Never forget about 3-2-1, even with tapes.
As an IT guy all I have to say is "MAKE SURE YOUR BACK UP YOUR CATALOG DATABASE" this is the database that tells you what is on each tape and test its restore every so often on a test machine. This is so important and no one thinks about backing this data up. Make sure the backup is automated and is taken off site and ensure you have multi revisions of this database. If you don't have this catalog all your tapes are useless or you will have to re catalog all you tapes. If you only have 4 not an issue but once you get to a 100... you could spend a month cataloging your tapes!
If my ssd or hdd failes, is it better to replace it than repairing without considering data recovery?
@@jirehla-ab1671 I'm sorry you want to reuse your SSD or HDD after it's repaired? If you have any issue with any media you 100% replace it and recover the data to a new medium if it's important
@@steve_main cause I already backed up the data to the cloud i am also planning to backup it to my other google drive, do you think it is also better to seek replacement through a warranty or just buy 2 new hdd / ssd instead?
@@jirehla-ab1671 If it's under warranty then have a replaced under warranty. why would you buy new hardware if it's under warranty? 100% replace it if you've had any issues with it. if not, any data you put on it expect it to be gone
@@steve_main cause for me I accidentally fall it to the floor and I don't know if there's already physical damage to the external hdd and western digital said that the warranty will be VOIDED if there is a physical damage and will NOT be replaced
If you really care to preserve everything then 3-2-1 backups are critical. Tape can be a key part of this, but it can't be the only thing.
3 - Total copies
2 - Different types of media
1 - Copy at a different physical location
plus 1 - Copy offline. If you delete a file unintentionally and the deletion is immediately mirrored to all your backups you are screwed. Better have one backup run from time to time so you can go back to your backup from last week or so, where the file has not been deleted yet.
for example: a file saved on your computer and mirrored to the cloud plus backup every x days to an external hard drive fullfills all of those criteria.
oh and TEST YOUR BACKUPS!!!!
@@cobaltno51 Mutahar (Some Ordinary Gamers) recently said his system makes a backup every other day so at most he loses one day of data.
@@cobaltno51 A plan you don't test is no plan at all.
Big props to those tapes for ensuring that Dan stays the most photographed person on earth for the next years.
Sounds like a Guinness record
He probably is!
Thats just one afternoon for an instagram influencer.
If you count each frame as a photo, he’s almost certainly the most photographed person by a couple of orders of magnitude 😁
As has been said below already, make sure you have two copies. One on-site and one off-site. The fact that you haven't lost data is more of a miracle than anything else.
Also tapes do suffer from bit rot as well (they are magnetic as well)
Insure you store them in a stable environment, while they are great for archive storage the archive part only works if they are treated as per their specs.
Yes, now he has 45 tapes, time to get a second set of 45, a second drive and make a copy of each one as it is done. That will at least give single redundancy. Then a nice data safe that is rated for at least 3 hours fire resistance to store them in, preferably in a room above flood level, and with non flammable walls, and temperature control. That 30 year life assumes storage at constant 20C and 50%RH, which is only achievable in a massive data centre, and also assumes that you use no compression, and maximum error correction on the tapes.
Knowing how tape worked, I always thought they would demagnetize. I guess the only surefire cold storage is archival m-disc
(And also Sony's proprietary optical disc cartridge)
The rule now is three backups, two of which can be on-site, but at least one of which must be off-site.
That's the 3-2-1 rule.
@@shubinternetwhat about concerns about earthquakes? Tsunami?
While this is a big improvement vs random hard drives, as someone who archives data for a living I'd caution you against just putting everything on these tapes once and assuming they'll be safe for 30 years. That guarantee is more like a warranty than a specification, and I've had tapes fail much earlier than their rated lifespan. An immediate solution would be to use some form of parity or mirroring to ensure every file can't be lost with a single tape failing, but longer term I'd highly recommend backing up your archives using multiple forms of storage, in multiple locations, especially since this is your livelihood and if something like a flood hit wherever you store your tapes, you'd lose everything. A cheap option would be to make a JBOD server using your existing HDDs you just moved your data off of. JBOD is short for just a bunch of disks, and is exactly what it sounds like. It would also let you use a filesystem that automatically checks for and fixes bitrot, like ZFS, and mean you would have access to your entire archive without having to plug in a certain tape, or even multiple as LTO tapes can store a single file across multiple drives to ensure you're using all the storage it offers. This is why they have the barcodes as standard, as many commercial archives using LTO tapes will have a tape library, which are basically jukeboxes with tape instead of vinyl, and the software can automatically switch tapes to retrieve data spread across multiple drives.
Interesting, which companies/brands would you recommend?
Tape guaranteed for 30 years, and you get a replacement if it fails. But not the data that was on the tape.
It sounds like he plans on ditching HDD storage and simply copying SSD to tape. Safety of the data will not have improved much.
This Might be one of the Most informative comment ever posted
Not to forget that that spec only applies if you store the tapes and drives dust-free and climate controlled (temperature and humidity). That's a lot to ask even for a small business.
To repeat what others already said... But since it's important and could avoid data loss. Do NOT expect 30 years out from those tapes. LTO is by no means safe from data/bit rot and for important data you should rotate every 5-10 years depending on storage conditions. The 30 years they market them with is during very specific conditions when it comes to temp and humidity, treat it as marketing buzz. Also, do NOT throw them around in luggage and avoid X-rays, strong magnetic fields and so on. And for the love of God, do NOT drop them. They are fragile and you just have to disassemble one to realize exactly how fragile they are. Have had more then one eaten by tape drives in the past (LTO4/5). You should have two copies of all critical data. If possible, use LTFS to avoid being locked into one vendors proprietary db's and system and keep backups of any eventual databases. But yes! When dealt with properly and when knowing about the quirks and limitations and gotchas LTO is very cost effective and nice. But like everything, LTO have its limitations and should not be treated as a indestructible bullet proof Jesus-medium that will save us all. 😎👍
To be fair the raw footage is not critical at all, it's just "nice to have", even the edited versions are, because the last backup is on YT itself.
Any competitors making better ones? (than LTO)
I've worked with the LTO tape drives for a long time, although I worked primarily with Tape Library machines, that is, robots that you load dozens of tape into and has more than one drive. The backup software has a master database of all the files and tapes, and does backups, automatically changing the tape when the one in the drive becomes full. As other people have said you need *two* backups of everything, and store one copy in another site, preferably that is climate controlled. You should also periodically verify your backups. In the long term, you should also consider budgeting to change storage mediums every 5-10 years. Just like you are going from Hard Drives to LTO Tapes, in 5 - 10 years, you should be looking at the next great thing...maybe SSDs, or maybe holographic storage...
I've used LTO-4 for 12 years, and suddenly tape drives fail and HPE has forgotten how to repair the old drives, apparently throwing away all the repair manuals. That plus the nightmare of Indian support outsourcing plus only about 2 companies left actually making LTO products, despite originally introduced as a monopoly-free category as DLT had become a Quantum monopoly.
So, an old suggestion that I sadly ignored, is to purchase a spare drive (hugely expensive), test it and store it with the off site backups. Once you need the spare, immediately transfer all backups to a new physical format that can be secured with the same measures.
Oh, and in case you missed the small print: New LTO versions cannot read tapes from previous versions, except for one overhyped situation. Now I'd need to purchase an otherwise useless old LTO-5 drive to read the old tapes, while also investing in a new format to store new backups and copies of the old ones.
@@johndododoe1411 I totally agree with everything you said, especially the backup drive unit! But I would rotate it in every once in a while since it is bad for equipment to sit unused for long periods of time.
Magnetic tape is so cool, i'm still shocked after i found out that a vhs can hold gigabytes of digital data depending on the length. 18tb in such a small package is unbelievable data density
Theoretically yes. vhs is helical scan, the data is on a constant spiral, not linear. it doing so because vhs is optimised for analogue video, each scan line is physically present on the width of the tape. data however, is linear in nature, thats why we have data tapes like lto are always store in linear fashion. makes reading and writing much faster and searching easier.
I'd never thought we'd go back to tape someday.
@@gernhartreinholzen3992 Back? Tape never left at least in industrial and professional enviroments. Lto is a backup standard for decades now with 9 generations of tape and more are planned so its not obsolete by any means.
Makes me think everyone's been distracted by "Big Pharma" when the _real_ villain of our age has been "Big Hard Drive" all this time!
@@gernhartreinholzen3992 We never stopped using tape. A lot of supercomputers have physical tape storage libraries where an arm chooses a tape and inserts it into the drive for the computer
Very interesting. Thanks for taking the time to go through the process of migrating large amounts of data from hdd's to LTO tape.
How did u send this 1 hour ago I cl I cked on it when it was posted
@@AbsentNinjawaffles4 your notification was 1 hour late?
I have TH-cam notify me of videos just posted, but a few days after I watched them!
@@dlarge6502 same there were were ekss than a hundred comments all were seconds and minutes just thought the glitch was funny that I saw it
Don't forget a cleaning tape or two for the drive! They need to be cleaned every so often (40+ hours of writing, for example) for best operation.
As I'm using older equipment I tend to avoid cleaning tapes as all they do is sandpaper the tape path. I like to open the driven and manually clean.
Just don't fall into the trap of routinely cleaning an LTO drive, clean it when the drive requests a cleaning by itself or in case performance takes a sudden dive.
Specifically, the Single Character Display (technical term for that amber 7-segment display) will display "C" when it wants a cleaning tape. Unless there is a serious problem, it will wait for you to eject the tape that's in there first before displaying it.
As well, the drive has a brush built into it. Every once in a while, you will notice it will stall for a few minutes while it unloads the tape, brushes the head, reloads the tape, and then goes back to the point it left off.
Another thing: If you are writing a lot of small files with LTFS, consider using the sync_type=unmount option, so that it's not spending a bunch of time shoeshining between the index and data partitions.
I'm just going to echo what other people are saying and strongly consider duplicating your backups and storing them off site somewhere. Redundancy is just as important as backing up the data itself. It doesn't do you much good to back up all your data if it's all sitting in one location waiting to get destroyed in a fire or a tornado. You have a lot of very valuable data and you're now stepping into the world of enterprise level solutions. I would find an off-site backup location in another city, or state if you can mange it. Redundancy, redundancy, redundancy!!!
Of all the TH-camrs, your channel has been the one that has been in the back of my mind constantly. as someone who's a tech enthusiast, knowing how much raw data you generate every video and trying to come up with a solution had to come in occasional mental exercise for myself. I knew tape would probably be your archival solution but I didn't know the technology had progressed to the point where the write speeds would actually be faster than most hard drives. Your probably still needing a Linus grade storage server for, effectively, a scratch disk during editing but now you can of-load one your done and can stop having to build servers. Sweet
Love that you now have more cohesive way to store your vids, wondering what happened with the NAS LTT helped you out with. Love all the vids!
Cannot stress enough that you should always have AT LEAST 2 copies of all your data (or anybody else reading this comment)
just wish the tape drives weren't so expensive. Or it was easier to rent them or something like that. since archived storage that doesnt need to be accessed you dont need the drive all the time.
Preferably 3 copies tbh, for stuff that's irreplaceable
The NAS would be great for more short term storage when he needs access to data but his local computer doesn't have enough storage/he might want to reference later. The tapes are great for the long term storage and archiving of his footage.
@@ge2719 er if thats expensive to you, what are the cheaper options? 18tb for 170$ is way cheaper than hard drives.
@@hailgod1 He's not talking about the tapes, which are cheap. He's talking about the drives, the reader/writer device, which is expensive.
Love the breakdown, it’s obviously not just a hobby for you guys! Keep bringing us that sweet delicious slo-mo greatness ♥️
Oh and give Dan some more explosives
For someone who only genuinely enjoys the content and knows nothing about the logistics and workings of your results, its really cool to see this. As much as I wish you both were super close to have more time in this channel, life moves on and project priorities change i am glad you keep slowmoguys still going!
I’m a data scientist in the banking industry and I train models with tape archives all the time. Super reliable form of cold storage.
I have 1.04PB of storage which is backed up to large archive drives. I thought about using tapes, but they're likely to degrade at the same speed as a hard drive. The fundamental difference is that with a hard drive you can repair the fading magnetic fluxes. So every few years, just copy off the data, reformat the drive and then rewrite it. This is something that would be more difficult to do with tape. So as someone has pointed out... Don't think of tapes as a very long term backup solution. Not unless you're willing to maintain them too. Also bear in mind that unlike a hard drive, your tape machine is going to wear over time and you also run the risk of tape getting chewed up. These issues are less of a problem to a hard drive as it is a self-contained unit. Moreover, tapes will be much more sensitive to things like moisture and temperature than a HD because the platters in a HD are sealed. You do run the risk of motors failing in spinning discs, but this is more likely through use, rather than sitting idle and certainly less than the degrading of a LTO machine. So always have two drives and make sure they're refreshed at a reasonable interval.
You can repair the magnetic flux as you say, if the rest of your drive still works. Thats the thing right there, these tapes dont have any subsystems that will fail and entomb the data in a dead box.
If the tape drive dies, you get a new one. As this is LTO 9 you could argue it may be cheaper to try and move a dead HDD's platters to another compatible drive, if you have someone who has a working drive to transplant them into.
Plus the tapes are very immune to fading like old reel to reel audio tape can be because they are made of very different stuff. You are more likley to have problems with tape damage over anything to do with the magnetic fields.
@@dlarge6502 I've used tapes, lots of them and the magnetic fluxes of tape degrade at about the same rate as a HD. It's slightly higher for tapes because tape layers are wound upon each other. HDs die from use, not usually from sitting idle. They die either from motor failure or the controller dies which is why they have a MTBF indication. Each HD is self contained so the MTBF is spread over the number of drives for the archive, but a tape machine is used for all the media, so it's failure rate is significantly higher as is the chance of it chewing or having a r/w failure. The storage requirements in terms of moisture and temperature is also significantly higher for tape. Although admittedly, moving platters between HDs is extremely tricky, it can be done as a last resort. But get a chewed tape and it's game over.
The only real benefit a tape gives you over a HD is from shock resistance, if it's a spinning disc and size. This can be eliminated with SSD storage though, obviously at a higher cost.
Add to that the random access of a HD coupled with the higher transfer speed, especially from SSD, the HD always ends up on top in terms of long term storage.
@@thewelder3538 Well, at least your second comment made the argument sound more factual and not just a HDD fanboy making stuff up (your first comment gave that impression).
@@johndododoe1411 Agreed, it was a little wishy-washy. Although I do enterprise backups, it's hard to find the level between trying to sound like the ultimate authority on the subject and someone who's just a fan-boy.
There is something about tape that I'm happy to see back.
0:46
Some fun facts: 720p at 82,000 frames per second is equivalent to 8K video at 2,278 frames per second, and an hour-long 720p@82,000 video that is actually 1 hour long and not a 90 second slo-mo stretched to 1 hour would be 324TB, and he would only be able to store 2.5 1hr long "720p" videos in his 810TB drive.
Even if you somehow found the capacity to store a video that big, good luck
1. finding storage that is fast enough to show 82,000fps 1280x720 video (90GB/s is 12x faster than the fastest consumer-grade NVMe storage that is available today, and that $850,000 1 petabyte 100GB/s storage server that Linus showcased in his video a couple weeks ago (normally $1,000,000 but ignoring the $250,000 GPU cluster with 8x Nvidia A100 GPUs that is irrelevant to the storage portion of it that it includes) would only *theoretically* be able to handle it),
2. and *then*, good luck finding a CPU fast enough to decode (play back) such a video in real time.
This is not a perfect comparison because of different codecs of varying quality, but knowing that my M1-powered 13" MacBook Pro can't quite play back 8K@30 VP9 video on TH-cam without buffering, you'd need a CPU 76x faster than an M1 to be able to decode an 8K@2,278 VP9 video uploaded to TH-cam (assuming TH-cam even allows that video to be uploaded). Even if you bought 2 of the just announced $12,000 flagship AMD 96-core Zen 4 Epyc server processor, theoretically it would only take you 40% of the way there (30x faster than M1).
Until I retired, I was heavily involved in the design of truly large scale mission-critical IT systems. I was also the first in the company to exploit LTO (generation 1). It proved to be a good bet on the technology, and we eventually had massive robotic libraries which went through generations of LTO.
However, one single copy is not enough. If you are going to use if for archive, then do keep multiple copies. That's a lot of data to lose in one go when a tape fails. Also, one copy should be kept off-premises. Also, be prepared to do a media conversion with new generation of LTO, and it's very good practice to periodically refresh anyway, to prove you can read a tape and then you can deal with a failed tape using one of your alternative copies. This also means a single drive isn't going to do the job either.
Note that an LTO can write data a lot faster than it can be read off a single HDD. What you do not want to get into is a situation where you LTO drive is not streaming and you are in write, run out of data, rewind a bit and start writing again. It massively slows throughput and is very bad for the tape mechanism and tapes if it's extreme. LTO drives will throttle down write speed to some extent.
It it's a problem, then stage the data to an SSD first before backing up. Of course writing large amounts of data to an SSD will affect it's life too, but at least it would be done once. In our case, we had massive enterprise array storage systems which striped data over RAID sets and could keep up (whilst still keeping systems going), but even then it could still be a problem to keep the LTO drives streaming efficiently.
Also, if LTO is used for archival purposes, the tapes must be kept in a temperature and humidity controlled environment free of dust. Nothing too drastic, but something to be careful of. Extreme heat would be bad for them.
There is quite a lot more to backup and archive on tape then there might appear to be, and it really requires experience. Of course, if you are keeping tapes and the original disk drives that does limit some of the risks, but this all needs a lot of care.
Gav has always been so good at making the tech side as interesting as the demonstration side.
Cheers Gav!
I am from Bristol! :D
I live in Louisiana now, I miss the UK :(
I note you didn't say you miss Briz tho lol
If I'd grown up in the UK I, too, would miss the NHS if I ever moved to the US.
>I miss the UK :(
LMAO why would you
@@xamanto he downgraded to the US
@@upsilondiesbackwards7360 *upgraded
The barcodes were used for larger units which had mechanical arms which auto loaded the tapes into the reader/writer all enclosed into a cabinet which would live in the rack room. I remember those days...
Super nice video to see genuine appreciation for each other. Every company in this video is well intended and represented.
Just to chime in here: definitely keep a second storage backup if it's world-ending if you lose the raw. Tapes are pretty hardy but the chance of a file corrupting or at least degrading in that time span is pretty high (though better than a HDD). Would reccomend a second tape backup for any files you definitely need, and a third backup in a different storage medium (HDD, cloud, etc.) for anything absolutely critical.
2 tapes will do you for 30 years, but not if there's a fire where they're stored. Off-site backup in a different medium is airtight for anything critical and gets the best results :)
But if it's just nice to have, one tape will keep the majority of your stuff for the 30 years bar any catastrophe
Flashbacks to my old job. So much time spent with LTO tapes.
IKR, tape is so cool
That's bonkers! I didn't know such efficient storage even existed!
What also surprised me was that 8mm film converter on the shelf there; I didn't expect you of all people to have one of those, lol!
new LTO drives cost between 5K-12K😭 and they are limited to own and 2 prior generations of the tapes that it can read
I remember fixing a pretty big tape library machine a while ago (a tiny gear had cracked), it was super cool to see how it worked with a moving, mechanical arm on a pulley system to move the tapes around and stuff inside
I find it crazy that these machines that cost sometimes as much as a house use any plastic gears in them. I suppose they are expected to be decommissioned and superseded long before the plastic gears degrade, but that's clearly not always the case.
@@jamescollins6085 Hehe, the one I worked on wasn't quite a house in terms of price, but it sure saved the client probably a good few thousand euros.
Bought a pack of gears from ebay, used a soldering iron to melt it onto the shaft as it was smooth.. and a few attempts later it worked just fine
Love a slo mo guys 2nd channel video! So glad this channel is still active
Another benefit of tape is how natural is to have offsite and offline backups (the first protects you from disasters like fire or crime and the second one protects you from ransomware)
DO NOT take the tapes on a plane, the scanners may wipe or corrupt them.
When you are copying, you need to use file verification. Nothing worse than thinking you have a file, only to find it was corrupted right off the start. Take the original drives and store them at another location.
Also, you may want to create some parity files (.par2) just in case it gets corrupted later or as a later verification of integrity.
Scanners won't do anything to modern tapes. Maybe 30 years ago the motors that drive the conveyor belt might have caused problems for low-density tapes. High density tapes like LTO need a huge amount of flux to cause issues. Scanners won't do it.
Maybe if you’re traveling with them all the time and are exposed to the X-rays multiple times…otherwise, no this isn’t entirely correct
I've always found these types of videos interesting, but since I started properly working as a video editor and working with VR video a year ago these are absolutely amazing. That much storage is a dream
The real slo mo storage.
Wrong.
@@_GhostMiner Why? Linear storage is much slower than HDDs.
I was going to give this post a like, but it was at a nice number so I didn't :)
@@Scoots1994 77?
@@hitsmike2046 because he said the speed is ≈300MB/s
A few notes on LTO as someone who's used it extensively for video backup:
1) Generational compatibility -- while they tapes may be guaranteed to last upwards of 30 years, the same definitely isn't true of the drives. A new generation is released roughly every 3-4 years. So LTO-10 is due in about 2 years. And LTO-9 drives will likely still be available and in warranty for a couple of years after that. And LTO-10 will read LTO-9 tapes. But once LTO-11 is on the scene your days are numbered when it comes to being readily able to read LTO-9 cartridges.
2) Tape failure -- they are not perfect. Probably more reliable than 3.5" drives in cold storage, but not 100% reliable. Our backup plans always called for duplicate copies, stored in physically separate locations. On restores we did see failures.
Also, while they're a great low-cost high-reliability storage, tapes are slow and annoying in many ways. I've typically preferred to keep media backed up on 3.5" drives also, as a first-instance archive that I can generally access quickly and with little effort.
Thanks for the tech update. It's always interesting seeing storage problems solved, that us mere mortals (in a data size sense of the word) will never face!
Former IT gal here. I can't stress hard enough how right all the other IT professionals and data archivists commenting on here are. You need *_at least_* two copies, one stored safely offsite.
Our biggest fear in our computer workshop and data storage room (at a university) was that one day, someone would come in with a degaussing wand and destroy all our backup tapes. We were so paranoid that we had an off-site secure storage area that only 3 people knew of.
That's a bit much don't you think? Wouldn't that storage room only be able to be accessed by verified people anyway? Or was it not locked down.
Sorry, not trying to come off as rude just genuinely want to know why.
@@johndenver6769 The uni didn't quite have the budget or sense to provide a secure server room. So we took our own measures. Losing the data of thousands of off-campus distance ed students wouldn't have been a good thing. Not to mention study materials etc. We looked after data for the publishing section as well.
@@johndenver6769On a campus, IT and Facilities are often forced to share spaces. An off site colocation facility goes a long way to having dedicated space for IT to have sole access to.
Nice to see tape is still alive and well
I hope this is the next step in your Blu-ray journey! 🤞
me thinking tape storage was dead, and you come along and show me this thing, Heck
Part of the reason the BBC wipe over recordings was cost (magnetic tapes cost much much more than they do now), another key reason was contractually in some cases they were not allowed to reuse recordings, there was no reuse infrastructure or licensing agreements back then. Many historical recordings were destroyed just because they were not allowed to reuse it, even on film as opposed to on tape.
Yesh, all European broadcasters were doing the same thing.
LOVE these videos on the technical background behind Slow Mo Guys! Thanks for sharing!
"All in the same format, and I only need one device."
I would suggest buying three units, one of which is tested and repacked safely, and that you save the data on at least three tapes. And then make Dan keep that third copy of the tapes at his house, in case there's a fire. Also, keep the original hard drives as a last ditch solution. If you could copy the data to clean fresh disks, that'd probably be good too.
It is better to copy the data to tapes and then after checking that the data is fine format the drives and give it away / use it for something different. Hard drives are not the best long term storage solution and can be used. It is a huge opportunity cost to keep them around.
"Phantom Operator" is the best job title I've ever heard of 🤘🏿
Hey Cedders! Make sure you get one of those fireproof media storage cabinets. Would be a shame if who ever stole your flamethrower returned.
Tapes can definitely have bit rot. And these tapes are guaranteed to last 30 years, just like LED lights are guaranteed to last ten years -- I.e., not.
Tapes are a fine backup medium on top of other mechanisms, but I'd never use tape as my only backup. Not even in a 3-2-1 pattern.
Disclaimer: I was a technical reviewer of the O'Reilly book "Backup & Recovery" by my friend and former co-worker, Curtis Preston.
Interesting how I was just researching LTO tape this morning, and then this video came up! Nice job Gav haha
If I have learned anything about specialty hardware from my time working in IT, it's that you should go ahead and buy another one of the symply drives. No matter how good the company seems to be doing now, that unit could fail when you really need it and there will be none available
Truth, keep another unit tucked away that isn't open, maybe even 2 or 3 of them if you have the money.
7:03 Gav: Correction. Tape is also subject to bitrot. Just like a HDD.
No storage type is flawless, eventually the data will start to rot. Though HDDs will experience bit rot way sooner than tape drives
@@HydroHUN but if you have a HDD server it can be done automatically, he has to do it manually
@@HydroHUN I'd say hard drives experience mechanical failure ("click of death") long before bit rot.
The HDD in my IBM Model 30 from 87 still works and that machine was rescued from a damp container at the recycling center
@@DecibelAlex damp containers won't inherently hurt the HDD. It's not a physical rot
@@sylvester4207 It is semi manual, he inserts the tape drive and it just goes in the background. As far as I understand the file movement is automatic as long as the drive is not full
the channel is so old were starting to see the salt and pepper in gavs hair, cant wait for more wise and wisdom from this channel.
Linus tech tips be like: "but can you make a 2 Petabyte server?"
1500 exabyte sever time :D
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
So no, an LTT server isn't necessarily the best choice.
Make sure you do regular catalog backups. You should also consider using a tape vault service. That way its offsite for DR redundancy, and they will keep it in a humidity/temp controlled storage to keep them in the best condition so they last for a long time
It's awesome to see how things evolve. It would be cool to see what your production path looks like from recording to ingestion to process to archive. Do you still have the 45 Drives server as well from Linus? Great video!
Very helpful! We’re going to need a long term backup solution for our observatory data, and this is exactly what we’ve been looking for. Definitely going to back up with tape! It makes it so easy to back up off site
I'll just forward what others have been saying - keep three total copies of everything, two different types of media and one copy offlocation.
@@Bruh-zx2mc yup! We’re going to have our NAS store the main sets of data, and then tape on site and off site for long term storage. Our current NAS is 36 tb but I’m sure we will get it closer to 120 tb. We will save up for tape when we can afford it
You might want to make a second run of tapes for of-site backup. Just in case the house burns down or something like that.
This, this and 100% this! Please make sure you have an off-site backup of your entire library; a theft, fire, flood can wipe out everything in minutes...
$170 per tape times 45 equal $7650. Real pricey but I guess worth it depending on how much he values his files.
As many people have noted, tape drives are a "quantity over quality" upgrade. You sure are getting a lot of storage, but that storage may or may not be less reliable than a mechanical drive or SSD. Mechanical drives at least have the advantage of warning you when they start to die, unlike tape and SSDs.
I just imagine Gavin being like 80 years old just watching back all the footage
This was basically just an advert, but very interesting and nice to see Gavin is getting more help in the inevitable data hoarding that this job requires. Linus tried, but the Slow Mo Guys are on another level of storage needs.
Just an ad?
Why is it, that every time someone mentions a brand, it's definitely an ad in someone's eyes?!
@@akyhne They gifted him the drive and tapes in exchange for him creating this video in which their brand is mentioned to hundreds of thousands of people.
That's literally what an advert is.
I've somehow managed to cram my whole computer life into 1tb up till now. Finally had to order a 2nd TB last week though, because my old 1TB drive is completely full and I don't have anything disposable to free up space.
Managing almost a petabyte of data sounds hard.
I love learning about this side of the whole process. Things we as viewers may take for granted
This is what Linus should have supplied during your collab, tape is the obvious solution to your problems. When I interned at IBM in a datacenter position, they had a machine with hundreds and hundreds of these tapes. Unit had a robot arm to read the barcodes in the library of tapes and load the appropriate tape on demand. Had to be petabytes of data on them. I remember the 4 tape heads on the high speed recorders were named John, Paul, Ringo and George lol. Someone there had a sense of humor. Then a guy would manually load them from the high speed output to the reading library with all the others, and add the barcodes to their spines.
Please dont tell me they have Linus "advising" them...?
@@jshklsn Linus is good for general education and entertainment. But his approach o industry standard equipment is usually quite lacklustre and backwards. His use of storage systems is quite painful to watch.
To be clear, I'm on the side of using things for what they are designed. Need to store massive data over quite a few hours? Tape. Need to store data real fast, multi-user? SAN or something local. I just think tape better fits this use case.
I recommend not relying on tape alone. Make sure to get some new hard drives, create a ZFS RAID7 (triple redundancy) on them and copy all data you already shot onto it. You can spin it up like once a project is finished, put everything on it and power it down again.
Tape on the other hand should be just a backup, so make two of them and store them at two different locations. Make sure to write to two tapes for each backup location and alternate them. Meaning: You write currently onto tape A and tape B on backup location X, and tape A and tape B on backup location Y. But you fetch tape A from backup location X and tape B from backup location Y. So both tapes have completely different content.
You then write the current project to tape A and the previous project to tape B and bring them back to the backup locations.
On the next backup cycle you alternate and fetch tape B from X and tape A from Y and so on.
Didn't you get a 130TB storage server with LinusTechTips? Or was that never used?
Edit: Thanks guys for the update. I didn't know it filled up that fast !
He used it. 130tb Storage didn't last long however
he filled that up before the video was put out.. you see he did a little update at the end showing it full.
You use such a server more for live projects. So dump tapes to the server, or hold a catalog of lower quality videos for searching.
I remember him tweeting that it got filled up in about 5 months
Thanks guys for the update. I didn't know it filled up that fast !
a tape backup is usually a 3rd level backup, not a primary, normally you would have a live source, a hot backup, normally on instant access HDD (NAS/iSCSI), and then cold storage, aka tape. then if you are wanting to achieve it, you would normally have two independent copies, plus a tape backup.
I've worked with tapes in the past and never trusted them to be reliable. I just hope this setup works out for Gav.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
I'm pretty sure, they wouldn't use tapes, if they sucked!
I didn’t even realize tape format was still being used for storage, that’s awesome
Yep, this is the latest generation LTO9 and its the first time I've seen it out in the wild
yep reliable and secure :)
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive.
Storing on harddrives, would be incredibly expensive.
Tapes absolutely are susceptible to bit rot, whether the tape is damaged by environmental conditions (i.e. literal rot), or the signal fades over time as neighboring magnetic charges interact.
Yea LTO what an interesting medium. Funny how the drives cost so much.
I agree it's tragic that so many shows and pieces of footage are lost forever just because some corporation was too lazy to store a copy. I love to watch old entertainment from back in the pre-internet ages.
I often wonder if the old lesser known cartoons I used to watch still exist in a vault somewhere, or if they only exist on random people's VHS tapes.
If you're curious about how much it'd cost to store 810TB in the cloud while keeping access to it, you're looking at a monthly fee of about $3,000. As a backup solution (accessible once or twice per year) it's a tenth of that.
S3 Glacier Deep Archive cost (monthly): 822.89 USD
I had a job at a datacenter once where I was managing daily backups and they used these tapes in this bigass IBM tape ROBOT. It was basicly a couple of serverracks made into an enclosed tape station where it had like 8 tape stations that all writing simulatiously, an "arm" like a printer or something where it would move between the tape writers and this huge array/cluster of available drives, just sitting in slots all over the place. I think they're called "IBM Storage robots" or something like that.
One of the daily tasks was to have it eject the tapes that needed to be put in a remote location, and swap them out with the old drives that needed to be erased and put back into the array.
Every time I look into tape I am shocked at the prices of drives. Been sticking to HDD and a HDD dock and basically using the HDDs like tapes. I need to completely revamp my backups though so I can have more automated versioning. Right now it's kinda manual and is just me choosing what drive to put in to update to latest backup.
I'm shocked not only by the prices of the drives, but of the tapes as well. In many cases, you aren't saving much over the cost of a comparable hard drive, and you have the added inconvenience of packing and unpacking all the data every time you want to access any part of it from the tape. I also don't understand the obsession with keeping source material that so many TH-cam creators seem to have when the bitrate that it will be compressed to when streaming is hundreds of times lower than source anyway. I'm sure he could re-encode the footage using the new AV1 encoding format and save mountains of space with no decrease in visual fidelity. Perhaps like with photography, it is necessary to keep the files in their raw form to aid in post-processing. Either way, it is his footage to treat in any way he sees fit.
Its because the users of tape have IT budgets and we throw money about to companies who know we have the money to play with.
@@jamescollins6085 The tapes are brand new, this is literally the first time I've seen one outside of a product sheet, thus they will be pricey for a few years. I'm totally shocked that anyone would pay $1000 for a phone too.
@@dlarge6502 Agreed. After a year, most phones will cost roughly half of what they were initially sold for at launch anyway.
I remember being interested in digital tape storage as a kid, back when I was backing up my Windows 98 files on 250mb ZIP Discs. I had no idea they were still used, and had grown to that capacity.
From what I (barely) remember back then, the largest tapes you could buy in stores were like 40/80gb. This probably would have been around 2001 or 02.
What is the oldest recording you own, gav? Like, the oldest thing pre-slo mo guys?
The missed opportunity not naming this channel “The Slow Mo(re) Guys”
Can these go through x-ray at the airport?
It will be interesting, As a former main frame computer engineer form the days of huge tape files, in general a tape reel would be at least a mile long and often twice that. We used to run archive tapes from end to end periodically in fast forward to make sure they wouldn't crosstalk between tape layers with time. Tape can and does deteriorate over time, but good luck.
The bigger the tape the more you loose when it goes bad!
Add to that access is serial (because it's a tape,) getting to a file at the other end can take an appreciable time where as solid state or HD is much faster to access.
Whatever happened to Linus’s backup system?
I remember gavin tweeting 5 months later that it got full
I was where you are now back in the 90's when i got hold of a jazz drive and a ton of media.
Youll find in 5 years when your the only guy with that machine you will move all your data back onto the standard format of the day.
With jazz it was serial ports disapearing from the market, you just cant risk never being able to access your data again. Its no good for that lifetime storage feel thats inspiring you.
But yup feel your pain, know the glory lol
0:05 It technically is a paid video. Paid with a free product that allows content and gives them free advertising, so check that box. If you could send the tape drive over here when you're done that would be great ;o)
Old IT guy here.
Unless you have 3 copies in 3 different locations, you don’t have a failsafe backup.
I use LTO but external threats like fire and magnetic fields are still a risk. S#!t happens.
If the data are critical, then at least find a deep storage option online like Glacier storage or at least another LTO copy in a temperature controlled vault offsite is called for.
Also, test the backup media occasionally to see if you can restore it. I’ve been bitten badly in my early career trusting the software saying “no problem” then have it fail when we actually needed to restore the data. Even the vendor couldn’t recover the files.
3-2-1 Backup.
3 copies, 2 media, 1 copy offsite.
I dare say we're not seeing the entire back-up strategy here and LTO is just one media
Glad they provided you the things you need. $5200 for just the drive is expensive!
The benefit of tape is the cost per GB at scale - the system Gavin has, with 810TB, comes to 0.024 per GB, physical HDDs would be about 0.035 per GB, and the cheapest AWS storage tier (Glacier Deep Archive) would be 0.046 per GB over 5 years (assuming no data is ever actually retrieved).
So, we're back to where we were forty years ago.
Tape on cartridges/cassettes for data storage.
Gives the micro-tapes from the original Star Trek new credibility.
No, we are not going "back". Tape has always been essential for large capacity storage.
20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers.
Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
@@akyhne I did not know that.
Are those still magnet tapes? How are they able to have that high storage capacity?
I just saw a short video on TH-cam, from CERN.
They explained, that they are connected to 170 data centers around the globe, to process the data from the LHC.
They also have their own computer on site, with over a million cores.
My favourite lesson about backing up data is how many businesses in the World Trade Center lost their backups. They were in the other tower. Emphasis needs to go to the *off site* part of the 321 backup strategy being completely off site because the incident could affect more than just one of the places your backup is, especially if, for example, your off site backup is _next door._
I hadn't heard about this tapes, but it sounds a lot like a miracle product. I think it would be best to use them as one more backup besides the HDDs and SSDs.
Tapes are absolutely affected by bitrot and the requirement to exercise the tapes. I have worked in backups since 97 dealing with one of the major backup softwares and I can tell you that I've seen more than one customer that realized their tapes from 5+ years ago are not as reliable as they had hoped they would be.
what happened to linus' server
Probably was only used for new footage, here he's mostly talking about old footage, pre ltt server
older generation of lto drives have been falling in prices like a rock, I personally use a HP tape library that holds 23 + 1 cleaning cartridge, 2U rack unit holding 2 drives that can be used at the same time, it supports both LTFS which I believe is what you are using: makes the tape look like a flash drive (just dont try to use it like one) and more traditional backup software like Veritas Backup Exec which is what I use with a client software on all my VM Servers and desktop workstations, the barcodes are actually used by the library;s builtin barcode reader to scan in order to know which slot it took the tape from and where to put it back or determine if there is a tape in the slot. Been using it for years and super happy with it. these are 2 Fibre Channel drives,, don't let that freak you out, all you need is a cheap dual port card that syncs up to 8Gbps depending on the generation of drive, just remember to make a rotation policy so not all the tapes are at the same physical location due to theft, vandalism, fire etc...except mines is non portable mounted in a cabinet and requires 2 people to move safely. If it is a HP and you use LTFS, all files copied onto tape gets indexed, so as long as the tape is labelled, you can use the provided utility go search which tape the file is on, load and restore; all very economical and cheapest per GigaByte. The same hardware would have cost an arm and a leg when new! just wait till lto14 comes 570TB - 1440TB depending on how compresssable your files are! Could probably buy a economic car for the price of one of those units!
It's so cool to me that going "backwards" is such a great leap forward
We are not going "backwards". Tape has always been essential for large capacity storage.
20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers.
Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
One of the main reasons why those old BBC tapes were written over with no thought of preserving them was the cost of magnetic tape was really expensive at the time, something that you've pointed out is definitely not the case now. So like so many people have pointed out in other comments back up your backup!
So that server that Linus built you is totally full?
It got filled up in around 5 months.
@@akyhne yeah. Sounds about right. Slow-Mo footage takes up quite a bit of storage.
Remember to store tapes upright and in the proper humidity/temperature.
I love it. We've brought back the flip phone, we've brought back disposable cameras, and now we've brought back the granddaddy of them all... the VHS!
No, we are not going "back". Tape has always been essential for large capacity storage.
20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers.
Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
tape never left dawg
Whoever told you tape isn't susceptible to bit rot lied to you. High density tape has an even shorter shelf life than the cheapest hard drives.
And good luck finding that one file you're looking for when you ran it in a job spread across 10 tapes. It's not exactly on-demand data retrieval. And don't even get me started on parity.
I've done the math and, byte for byte, HDD is cheaper and getting even moreso, and if you're leaving em unplugged on a shelf they'll last longer. Though for your particular use case you're probably long past the point of just needing a software-defined petabyte NAS so you can upgrade capacity on the fly.
But hey, if you're satisfied, more power to ya. At least you're right that tape weighs less.
Have you thought about what will happen to all your old hard drives long term? The British library or the national film archives could look after them as you are getting to the “national treasure” stage!
This would be very cool
Seeing all that data without any backups makes my palms sweaty. This has been long overdue!
8:30 Would the baggage scanners have any effect on the drives/data?
I know this video is old, but it's always a good idea to follow rule 3-2-1 of backup - 3 copies, 2 different mediums, 1 on remote location. It's the baseline of not losing your data no matter what
Can safely say I never thought I’d hear anyone say they’re upgrading to tape for their data storage, the more I learn the more I learn I don’t know!
It's not really something consumers generally do. Usually happens more in larger companies.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.