A lot of people are having trouble understanding that yes, a single URE can indeed cause the entire array to fail to rebuild. Because of the nature of the parity system, it is NOT the same as having a URE on a standalone drive, where only that single file is lost. You need it ALL to rebuild it all. You might be able to send the drives to a data recovery service who could potentially rebuild the array and only lose the files directly using the lost parity data, but that would be very expensive and time consuming, and not necessarily a guarantee. Not to mention that because array rebuilds take so long, and are very very intensive, it drastically increases the chances of having yet another URE in that time, which could completely ruin any chances of any recovery at all. You can google something like "Raid 5 rebuild failure probability calculator" if you don't believe me on the odds. But using 10^14 URE rate drives is extremely risky and you actually are VERY likely to experience a rebuild failure even with small arrays. In any case, when looking to buy drives for a NAS, my suggestion is ONLY buy a drive with a URE rate equal to or better than 1 per 10^15 bits. This can be found by googling for the 'Data Sheet' of the specific model drive you're looking for. It will be called something like "Non-recoverable read errors rate", "Error Rate (non-recoverable)", "Unrecoverable Bit Error Rate (UBER)", or something similar, probably listed along with the 'reliability' specs in the data sheet. I haven't really been able to even find any HDDs with a URE rate better than 10^15. However, with SSDs you can usually find better URE rates. Data sheets for consumer desktop SSDs (Like samsung's EVO/PRO SSD line or Seagate's Barracuda SSD line) don't list URE rates. But Seagate's "Ironwolf" NAS SSD line actually have URE rates of 1 in 10^17, so 100 times better than any HDD I've seen. However those cost several times more than an HDD (right now it's $750 for just a 3.8TB drive).
Jeeeez rebuild issues where the whole thing dies is when old RAID cards would just give up. Software doesn't, could lose a few files around that failure (which isn't as likely as any of these calculators say. Rebuilds read from multiple disks too...
I have a question. Given your example you have 14TB with 7 drives in RAID 5 parity is split evenly across every one. So that means that each drive has only 2TB/7 dedicated for parity. When rebuilding a failed drive does it mean you need to read remaining DATA plus Parity ( whole 12TB ) or not ( maybe a smaller amount is enough )? What i want to ask is basically how does rebuilding work?
I understand that, but according to a thread: "most hardware RAID will abort the reconstruction and some will also mark the array as failed, bringing it down. The rationale is that if an URE happens during a RAID5 rebuild it means some data are lost, so it is better to completely stop the array rather that risking silent data corruption. Note: some hardware RAID (mainly LSI based) will instead puncture the array, allowing the rebuild to proceed while marking the affected sector as unreadable (similar to how Linux software RAID behaves). linux software RAID can be instructed to a) stop the array rebuild (the only behavior of "ancient" MDRAID/kernels builds) or b) continue with the rebuild process marking some LBA as bad/inaccessible. The rationale is that it is better to let the user do his choice: after all, a single URE can be on free space, not affecting data at all (or affecting only unimportant files); ZRAID will show some file as corrupted, but it will continue with the rebuild process (see here for an example). Again, the rationale is that it is better to continue and report back to the user, enabling him to make an informed choice." meaning that its not a big deal if I use Linux software RAID 5? right?
I would argue that the bigger lesson here isn't to get overly hung up over RAID5 vs RAID6, but to remember not to fully trust either of them and always have a proper backup. A good backup system turns a nightmare into an inconvenience.
100%. A Raid is not a substitute for a backup, no matter how you stripe or parity it. Even raid 60 could have an internal power surge or something and toast all those spinning drives at once... your data NEEDS to be elsewhere, off the raid, if it matters.
This is 6 years old, but is as misunderstood now as it was then, and when this was first floated around 2009. The HDD data sheet specs for bit read errors are a statistical expression of failure rates [and a poor way to express it], not a performance specification. RAID 5 is absolutely fine - drives fail at around 1% PA for the cohort of drives used in such arrays. Another way to look at is [also misleading but uses the same logic as the RAID 5 is dead concept] if 1 HDD from a 6 HDD R5 array fails - that event itself is way way beyond the URE rate that 6 drives collectively would have ever experienced if we believe a plain text reading of the URE spec.
What a load of BC. First of all, explanation for an average home user goes like this: If you build a redundant array with: 2 drives - your only meaningful choice is RAID1 3 drives - your only choice is RAID5 4 drives - RAID5 for capacity vs RAID10 for redundancy. Only starting with 5 drive array you should consider using RAID6. And, of course, you don't lose ALL data with RAID5 disk loss and URE. You lose one file. It might be critical for an enterprise environment, IF they don't make backups. For an average homeuser Joe it means one corrupted photo, or movie. Not the end of the world. And even Joe should make backups of critical data.
If you have 3 drives and have to put them in an array, you buy another drive to do so. Who even needs to RAID 3 drives in which kind of environment for what purpose anyway?
+Colton Blumhagen Well, I'm not familiar with the error rates of SSDs. If they have very low error rates (if at all), then it might be safe. This video kind of assumes the hard drives have a URE of 10^14, but if you have one with 10^15 or even 10^16, you could probably get away with RAID5.
Yeah, this video goes hand in hand with the "how long do hard drives last" video. But that one doesn't go into the difference between sata, sas and ssd drives and what vendors make the higher quality drives. Unfortunately because of all the low quality production these days all of this makes a difference.
+ThioJoeTech i was going to suggest that as a video for you but you beat me to it. Hard drives can give you lots of directions for multiple videos too I think.
RAID1/10/5/6 is about availability of data not safety of data. I.e. minor failures not causing an interruption of service, not protection from loss of data from drive failure, server failure, building burning down floods threat explosions etc etc etc.
Good source of misinformation. If one drive goes you still have access to the logical disk even if another drive has a read error. Take a backup of the logical drive using Acronis or similar software, replace the defective drives, rebuild the RAID and restore your backup. You wouldn't lose everything as this video states unless you have two full drive failures, but then you should be doing regular backups anyways. RAID is about keeping your data available all the time , not protecting it from loss.
@@jabetajones the more disks you use the higher chance of a double fault can happen and loss of data (or all data if 2 disks fail) that's where raid6 should be used, but size of the disks also matter as well over say 4TB you should use RAID6 as well (unless you don't mind restoring your data from backup if you have one) if you don't have a backup you must use RAID6 minimise risk of total data loss but you should have critical information backup up (like pictures, personal videos, and documents) The problem with raid 5 it's not very forgiving when your rebuilding the array after a failed disk has been replaced because if another fault happens 1, because of a data error witch will result in Data loss (normally you can still access the remaining unaffected data) 2, a second disk fails and you lose all data with bi way of recovery The statistical probability of 3 faults happening when you got RAID 6 setup is very unlikely (but not impossible just unlikely, but gets more likely as each disks get larger) but raid 6 good enough for quite a long time even with 16Tab disks as 2 disk failures at the same time are rare, as raid controllers and nas boxes do a weekly or monthly raid patrol read and data consistently checks (depending on how they are configured) witch normally weed out problems disks before they suddenly fail
For small ammount of disks (like 4 or 6) usefull space / used space ratio is too small to consider RAID 6, you just need to stick with WD Red, Seagate (not Barracudas!) Ironwolf / Constellation and similar kind of disks and you'll be fine. For larger ammounts, you use ZFS anyway to avoid bit rot. The only really dangerous scenarios i see are some instant power failures and thus you need either RAID controller with battery or UPS.
People have different storage needs. There isn't a one size fits all solution, and an absolute statement like "RAID 5 is bad, never use it" is just silly even today. RAID 6 _is_ safer for large arrays where more disks means more points of failure, but for smaller arrays RAID 5 is still fine. It may even be preferable because it has better storage efficiency. Besides, you should be backing up data that's important anyway. RAID is only meant for minimizing down time when a drive dies. If you're ever using it by itself as a substitute for a true backup then you're an idiot.
I can confirm that what he is saying is correct. This has just happened to me on Raid 5. Good thing is that I still backup each week, but HE IS ABSOLUTELY CORRECT!!!
I use raid 1 (plus cloud backup). Sure you don't get the storage size advantage of raid 5, but IMO anything worth keeping is worth keeping 3 copies of.
Good and proper explanation, except i have to wonder about your reasoning why RAID6 is not a futureproof concept (or as drives get bigger). Unless you get two UREs in the same exact sector in both parity segments, you will always be able to rebuild your lost drive by using the readable sector from the other parity segment and this problem does not scale with drive size.
I've been using a 12 drive Linux Software RAID5 setup the past 10 years. Gradually growing and replacing drives (from 8x200GB to my current 12x2TB). Although I agree with you that RAID6 would be safer, I've never experienced the problems you're describing. I've had 1 drive failing periodically, and replaced maybe 4-5 drives in total (due to failed/dead drives) during the same time period. This is data that I am comfortable loosing, although I would prefer not to. The worst problem I've encountered during this time, is the failing of a controller, where symptoms indicate that multiple drives are lost. But thanks to Software RAID, I'm not bound to a specific hardware disk controller. Which makes it easy to rebuild or even move the entire array to different hardware. So if I encounter the bad sector issue I'll report back when that happens, although I haven't seen a bad sector since the mid 90s
I have yet to see a RAID controller that stopped rebuilding a disk because of ONE read error. So far I only managed to get a corrupted file as a result but the rest was recovered nicely. So the big question is: Is ONE defective file (which is possible, but not guaranteed to happen) worth an extra disk (and a bigger NAS system in case you need more than 4 drives)?
Raid 6 doesn't sound like a good option if you're looking to build a four drive array like I am. I'm looking to build a four drive setup using 3, 4 or 5tb drives, but I don't want to loose the storage of two drives, so raid 5 is really the only option. If I was fine with only having two drives worth of space, may as well go raid 10.
Tripple partiy will never come. With the increase in disk sizes we are running into the limits of what RAID can do with parity. Rebuilding huge arrays simply takes too long and is no longer worth it. If rebuilding takes over a week it simply takes too long. RAID5 is not a bad choice, depending on your use case. You've taken a very specific use case where the capacity of the array equals the URE to say that you are statistically certain that one of the drives will have a URE. While I did not study statistics in school, I do doubt you can simply state it like that. You have 6 remaining drives, each with 10^14 bits so it wouldn't surprise me that you can say in this case that the actual value would be 6*10^14. And what would you do with the new 14TB drives that got just announced. If they also have a URE of 10^14 you statistically will always have an URE on the drive. Not use it? (welcome to the wonderfull world of statistics). And to add a few more things: - The URE is the avg over the lifespan of a drive. It might be possible that you read 1000TB of a drive without a single URE and then all of a sudden you get a multitude of them resulting in the avg of once every 12TB - The URE is a single bit in a sector. Every sector has a CRC that gets updated on every write (how else would you know about the error?) The URE can even be in the CRC data. Most bit errors happen on write anyway and the system will always read back written data to confirm it is written OK, further diminishing the chance of having bad data due to a bad bit. - On modern drives with 4k sectors if you write small amounts of information the drive will pad the sector with the same information, as long as it is smaller than 2k, thus making a copy of the same file making it more likely that the information can be recovered. And lastly, let's not forget: RAID is not a substitute for backups. If you do come across a URE, you should still be able to recover the unrecoverable data from your backups.
I rarely have dead drives to begin with. One or two in the last fifteen years. Just periodically replace them. Virtualization has made the vast majority of my hardware super easy to replace. Move guests to different box. Box can be turned off.
@@mesaber86 it sounds like periodically is being used to define 1/2 failures, not uncovering hidden secret failures. Although I could be wrong, only Kevin de Bie really knows how many drives have failed.
"I use RAID 5 for all my servers and never had a issue when replacing a dead HDD." Famous last words before cold sweats, tons of sleep deprivation, and data loss.... If one of your other drives fails during recovery you're SOL......smh
I lost a primary domain controller once back in 2003. The raid 5 didn't help, neither did the secondary domain controller. It was a huge mess. I had to accept defeat and rebuild everything from scratch in two weeks with the help of a friend. Slept at the office for that time. So now I am a bit skeptical to all redundancy systems :)
For the record, raid 5 is still fine for small arrays like an executive workstation for your CEO. His premise here is that you're using a very large array like in a NAS, Backup array, or Data Center
NAS isn't "very large" ... certainly nothing like a data center. It's standard practice for people who do content creation and basic video projects. An 8TB NAS is hardly a huge amount of storage for a TH-camr/Vlogger.
I've rebuilt a few dozen 20~30TB RAID 6 arrays without a single rebuild failure. RAID 6 is solid. This has always been with Adaptec controllers (51645, 6805, 71605 etc.)
where do you buy a RAID 6 system? So 8 years later a RAID 5 is available anywhere. It can do RAID 1, 5, 10 in the same box but never 6. Have times changed?
Normally if you create a raid set you also set a spare drive so it can start rebuilding the raid asap. Depending the raid you want/need it will be better one or the other. Also keeping in mind performance of course
Hey Joe, lets us consider the following scenario. what is I have 10x8TB array of disks configured with RAID6. So even if two disks fails( Since RAID6 ). I will be having 64TB of data to be scanned for recovering the last drives. So from your URE probability calculation, there is URE happening. Then RAID6 is also not useful. What would you suggest about this... !!!! Thanks in advance
The math isn't quite that simple, because URE probabilities are independent from disk to disk. Reading 64TB across disks that each have a URE rate of 1 in 10^14 does not guarantee that a URE will happen, because a URE not happening on one disk has no bearing on whether or not a URE happens on another disk. It'd be like thinking that rolling a 6-sided die six times guarantees that you will roll a specific number at least once. You'll get _more chances_ to roll a specific number, but failing to roll that number will never make it more likely that you will succeed to roll that number in the future. All of that said, adding more drives to an array will always statistically increase the likelihood of a failure by a little bit, unless you're using RAID 1 where more drives always means more fault tolerance. And 10 drives is quite large for a RAID 6 array. You should definitely have regularly scheduled backups if you don't already. Off-site is best if it's feasible, but any backup at all is better than nothing.
This is a topic at work as we deal with a lot of data. So the answer is much more complicated. For some more to chew on: 1. When dealing with large amounts of data with many mechanical drives the answer typically comes down to ZFS RAID-Z level 2, which is somewhat similar to RAID except more check sums in more places, CoW, snapshots, etc. Plus flash based cache drives as RAID-Z is not the fastest thing in the world on mechanical drives. 2. For small arrays, a few big things are out there often favoring RAID 5. You can find reliability reports on many drives and avoid the known bad lines of drives. You can look at SMART data and pull a drive as soon as it starts going bad. As this is usually a slow process, most competent admins pull drives long before they actually fail and enterprise drive vendors usually take back drives even if they have one remapped sector. You can use hot swap as often the double fault happens when you power down a system. You can buy drives in a staggered fashion as sometimes sequential serial numbers in your array lead to drives failing close together in time, which is the last thing you want in a RAID. Flash drives if you look around will sometimes have RAIN, which is NAND level RAID. Flash also will often fail into read only mode, giving you a chance to copy over to a new drive, but not always. However flash in a RAID tends to get beat on more both because RAID controllers don't support TRIM and if you are using throwing money into a flash array, it is probably because you beat on the system a lot more than the average bear. However this wear concern can be addressed with higher endurance flash at a higher cost. For example my main home flash array with Crucial MX500 drives should burn out its flash in about 6 years at the rate I am currently grinding away at it, which is about how long my older flash drives on low use systems failed due to old age, however for twice the cost I could get SSDs with 10-100x the endurance and never come close to burning them out before they failed due to old age. 3. With any data storage system, you should not completely rely on the RAID to save you. Someone might delete important files and RAID does not stop file deletion. Someone might try updates that go horribly wrong. A cooling fan may fail and the whole array bakes to death. The computer with RAID array might get stolen or lost to a fire. Or like the other day with a software RAID 5 a power loss caused the array to scramble while the hardware RAID on the same system was fine. I have also seen arrays lost when the power supply in the box (often cheap NAS) fails and fries all of the electronics in the process. I have seen all of the hard drives seize at the same time after many months of running because the drive manufacture used the wrong lubricant, so once the drives stopped spinning, in this case routine maintenance, they never spun again.
100%. A Raid is not a substitute for a backup, no matter how you stripe or parity it. Even raid 60 could have an internal power surge or something and toast all those spinning drives at once... your data NEEDS to be elsewhere, off the raid, if it matters.
I am not fully convinced by you. If errors are so rare, then when doing recovery in RAID 5, after one disc goes dead - we would loose maybe few files, and it doesn't mean that whole recovery would go wrong. Right?? (In RAID 1, after we lose one disc, the second one also may have some errors, and it doesn't mean that the whole recovery wouldnt work...) My concern about RAID 5 is: maybe you're right, if the discs are encrypted? Could you please refer to this message ThioJoeTech?? Perhaps I misunderstood your video, sorry :(
OK, what I don't get with parity 5 is if you lose drive 0 (2:48) what happens to 'Dp'? is that copy of the parity lost forever or is it actually copied onto one of the other drives as well?
I understand RAID 6 has a slower write speed than RAID 5. What about RAID 5 plus a hot-swap drive? That's what I'm doing right now. So if a drive goes bad the hot-swap kicks in and it rebuilding the array automatically. I replace the bad drive with a new one and it becomes my new hot-swap. You get part of the benefit of RAID 6 (although not the exact same) without the performance hits of 6 vs 5.
all that means is less time is required to discover that a drive has died. If you have any errors during a raid 5 rebuild, data is toast. Raid 6 allows the data to be protected during a raid 5 rebuild
Thanks, good information. I guess my only question is: How do you archive this much data when you do not need it on a local drive? Such as old youtube videos. I want to keep them, but not locally when I use up my 20-30TB NAS.
not sure if you are still monitoring this feed, but any updates on RAID systems and/or tech on this subject. Dusting off an old project and want to finish it this winter. Thanks
I'm not sure this advice is particularly helpful for a SOHO user. His implication is that that your RAID array somehow forms part of your backup strategy (or perhaps even *is* your backup strategy) which in itself is a poor message to convey. I really can't see too much issue with RAID 5 + proper backups. How many SOHO environments need 2 levels of redundancy unless in special situations?
There is something I don't understand about this: Assume you were to use 12TB HDDs with one URE at every 10^14 bits, your probability of an URE would approach 100% during a RAID-5 rebuild. However it would also approach 100% during a RAID-6 rebuild, because every HDD itself would likely have at least one URE. So no matter how many parity HDDs you use, you are always hosed and thus RAID-6 would give you anything over RAID-5. What am I missing here?
I have read that for a raid with multiple hard drives next to each other, you should only use NAS or Enterprise hard drives as they have vibration detection. Because cheap hard disks without vibration detection are actually only intended to be alone in the PC case or individually in an external case.
Nope, but funny. Subject to Satire, being part of your sense of humour? No but it is like phoning tech support and speaking to two technicians, a day apart. Explain the Lion? Since you need the attention? :) Then go back to Facebook, please. :)
You are aware that you lose performance in a raid 6 array. I work for an MSP that specializes in dental I.T. These places due to Canadian standards must keep data for patients for 10 years. Every one of our 2000 servers in play all have a raid 5 arrry and for the most part there are no issues. Now keep on mind even with a raid array we still run a cloud and on site back up of the host and the DC VM and Server VM. You should never %100 rely on you raid array.
It-s not. My home build 4 disk NAS, primarily used for archiving is build on ZFS which has one big advantage over RAID5. It is protected against bit-rot.
how is it a big advantage? bit-rot is not undetectable and between smart and filesystem protection you'd have to be incredibly unlucky to end up with a corrupted file - you DO have backups right? and i have never heard of a raid array failing due to bit-rot. but go on, all you zfs ppl just keep saying it over and over, it's not going to make zfs any better you know.
th00ht Every harddisk has already some pretty good ECC algorithm against bitrot, i really think it's most likely a ZFS marketing strategy and not a real problem for the majority of people.
I know this was a few years ago, but correct me if I'm wrong. This really depends on the number of drives that you have? Right? What if I'm only using 4 drives instead of 7? This will certainly change my chances a bit. I agree with what's said, but the scenario certainly depends on a few factors.
Vince Cropani If you listen to what he is actually saying, he is correct. The parity drive can only make up for failure or errors on one drive. If you have a drive failure and then find reading errors on another, you will likely lose data. RAID 6 is designed to help overcome this.
Thanks for the education. If I'm using SSD drives and do very little writing (mostly just archives) on a 12TB array (2TB x 6) Should I go for RAID 6 or SHR2? or just go with SHR or RAID 5?
i am planning to setup a NAS for the first time. i know you talking about drive failures. what happen if the raid controller fails? lets just say if i got a qnas enclosure and setup a raid 6. if qnas dies will my whole raid 6's data lost?
eh... Delving into a grey area here... For most home users, Raid 5 is just fine, even if the array is larger than 12 TB. Lets say worse case scenario, and you loose a disk, and error correction failed to detect the URE, and during re-build the URE is discovered hence fails to rebuild. There is a another option that Joe fails to mention. Actually, there's 4 or 5 other options, but I'm not going to get into them right now. The easiest thing to do if your Raid array fails to rebuild due to a URE, is to copy off the data. Yes, that means you will likely have to buy a couple of huge drives, and stand up temporary storage, which is a pain, but you will be able to copy the data off. Folks... Don't freak out over URE. Yes, the risk is there, but if you're buying quality drives the chances for URE is even smaller than what Joe stated. If you're Enterprise or your data is that important I figure you're going to go with 6 or Raid 10 anyway..... So, my point is just geared to the folks at home that may not have critical data and don't want to loose that additional disk space by going to Raid 6.
What are the other options he fails to mention? I’m trying to find a good safe way to back up my data and thought raid was the answer. After reading the comments, however, I’m seeing that it is not.
What you just said there is no simple task for home user, where if you had been using RAID6 to begin with a URE won't be a problem (or second disk fail) just replace the disk and let it rebuild It be nice if Synology used 5 bay NAS by default (no 4 bay nas's) from 2 bay ones to cover the extra disk needed for RAID6/SHR2
Also UREs can just be skipped over. Unlikely you will get more than a handful. So worst case you rebuild the array with 5 files missing. Obviously if you have a backup then its fine also. Easily encounter same problem with RAID 6 with UREs. So video isn't exactly telling the whole truth. Rebuilds can be hard on drives so possible it can lead to other disk failures during it, unlikely but can happen. This video hides the full facts and tells only a narrative that he wants you to hear
A few years after acquiring a NAS box with 5 disks I set it up in RAID-5. Within a week one drive failed and during the rebuild a second drive failed probably as it was loaded rebuilding the raid array. So I lost data. Nothing serious as I hadn't been using this for long. Since then I replaced this with a RAID-6 setup and have had drives fail since then but have been able to rebuild the array without issues. So yes depending on a single disk for parity can be risky especially as the load regenerating the full array can trigger a second fatal failure.
@@leexgx are you serious? Raid 10 is stripping+ mirroring so it is more secure and more fast, of course at expense of more disks and less space, it what's it is used in critical infrastrutures... trust me ... it's my work loool
So, the TLDR is you realistically have options of RAID 5 and RAID 10, and I would recommend RAID 10. The longer version. RAID 5 means you stripe data across 3 discs and use the 4th for parity. This will give you a total effective storage of 42 Tb. In RAID 10 you stripe the data across 2 discs and use the other 2 to maintain a completely independent copy of the first 2. So in RAID 10 you won't have as much total storage, only 28 Tb, since you use 2 discs for redundancy instead of 1. Raid 5 will also get you more read/write performance from your array. Since the data is striped across 3 discs instead of 2, you could notionally get up to 50% better speed from the RAID 5 setup compared to RAID 10, although that will probably be limited by RAID overhead and network bottlenecks. Now, as to UREs. WD Ultrastart drives are rated for an average of 1 URE per 125 Tb of data read (10^15 bits). That means during a rebuild (if we assume the array is basically full) each individual disc has about a 1 in 18 chance of encountering a URE (14 Tb/125 Tb x 0.5). The chances that 3 drives manage to rebuild without a URE are then about 5 in 6. So, if you have a drive failure with RAID 5, there is a 1 in 6 chance that the array fails to rebuild due to a URE and you suffer data loss. In RAID 10, you maintain a completely independent copy of the data at all times. So your data is protected without having to rebuild the array. In my opinion, a 1 in 6 chance of data loss if I have to rebuild the array is too high, its worth sacrificing the extra 14 Tb of capacity for better fault tolerance.
@@jordanbanko5347 I'd go Raid 5 and look into Crashplan. for $10 a month it backs up what folders you ask it to with unlimited space. Easy to restore from and keeps file versions so you can go back to a file version from last week if you want to.
I had a major RAID 5 failure. I lost 15TB worth of data 5 disks. I've now got 3 NAS with a total of 85TB each running RAID 6 and being backed up in realtime to each other. When my RAID 5 did crash and I went to install the replacement drive that is when the shit got real and the whole RAID died. I had to use a RAID recovery software to retrieve my data most of it but not all. Enough to get me out of hot water. There are two kinds of people on earth people who have lost data and learned from it and other who have yet to learn. Peace from Sydney Australia
OnTarget100 More fool you for not having a backup. I use two RAID 5 arrays, one backs up to the other nightly. The first array also backs up to cloud storage as soon as I write to it. I do this because I don't want to loose data, not because I learned the hard way
Must have been some valuable data lost to over compensate like that. I’ve lost data too, usually when upgrading OSS. By my own mistake of thinking I’ve completely copied data over , then resetting the disks for the new os. I’ve never lost data due to a disk failure. I also never use cheap disks. Only HGST hard disks and Samsung SSDs. I’ve had hard drives fail but only WS and toshiba. You’re smart and have the budget to have not only redundancy for disks but also separate enclosures. Very cool. I’m in the process of replacing all my bad Disha from 8 4TB to 10TB. At 350$ each, it’s damn expensive. But, like you, I have valuable or priceless (old work files I can never get back), so I get it. For sure. Photos fade, digitals do not ... music leaves popularity thus impossible download, same with films and tv shoes. What bad devices did you do with?
As usual very interesting. I have an external drive with 2 X 2 GB drives in RAID 1 it meets my needs and budget. Perhaps in future you could talk about building an external RAID array.
When you want to be extremely save you even need to build a backup archive... Because bad data can be copied over time ? So you keep a backup drive f.e. every 3 years... After 30 years you have an archive with 10 drives... When ever you find out a data is broken. Maybe you will load it from an older drive... Is that true what I am writing ?
What do you guys recommend me for a torrent movie library? (Don't judge me, where I live Blu rays are not sold and streaming quality is not on par for OLED TVs) I want to have around 50TBs in RAID5 or 6. As far as I know HGST is the best brand. Is it still the same today? And what NAS Brand?
@Thios.Joe Tech. I just picked up a 4 dri.. It just seems so wasteful because RAID 5 practically speaking gives you 2 drive redundancy but you lose a lot of storage, I don't need more than about 8 T. I thought to use 4 4 T drives which would give me 12 T of storage in RAID 5....but only 8 with RAID 6.....seems like wasting a lot of storage. I have also heard that one should routinely scrub the drives i a NAS....not sure if that is more than SMART or some other technology.
HW RAID10 -> No. Linux software RAID10 -> Yes and No. RAID5 achieves 95% bare hardware read speed. RAID6 a fraction of that. RAID5 and RAID6 takes a tremendous hit on write speed. Much more so on RAID6. RAID6 is not worth it. RAID5 for upto say 6 drives, further than that RAID50. Much higher performance, same level of redundancy. RAID != Backup.
That depends on your RAID controller card. My Areca cards that I use can run RAID 6 at nearly the same speed as they can run RAID 5. I've tested it, and with 8 drives it can read/write at ~750Mb/s in RAID 6. Cheap RAID controllers, however, won't be able to do that. The drives independently will only do around 130Mb/s, so it's losing very little to overhead if you figure that 6 drives (not counting parity) X 130 = 780Mb/s (max hardware speed) and I'm getting 750.
I just bought QNAP TVS-472XT 4 bay Thunderbolt 3 4GB (for Editing FCPX on my MacBook Pro) with four (4) Samsung 1TB 860 EVO SATA SSDs... can I configure them more than RAID 0 like RAID 5 or 6 or 10?!?
Actually RAIDS is the same thing as aids, but you get it from rape, that's what the "R" is for. Don't trust this guy's channel, ThioJoe will make a video explaining it.
I don't agree, in many cases (at home, etc..) a NAS with 3 drives RAID 5 it's good enough. If you want to assure to don't lose data you schedule a regular backup of your NAS into an external drive. This is the best practice
I have an Asustor 2404TE, one of the very early line of Ausstor NAS that had an early Atom CPU. It had a 4x3TB setup (WD Reds) at RAID-5. The 3pin cable conencted to it wore and there was a power surge that caused damage to one of my drives. Eventually it was reporting a lot of rear errors on the drive, which started small then began to build up. I don't know if they were UREs, but there were many. The system rebuilt the raid twice with no issues during this time. I then replaced the faulty drive (4) with another, the RAID rebuilt. That was back in 2017 and there's been no issues since, though the other drives will need replacing soon. So while there's a risk of URE it's incredibly low, and your drive will likely start reporting errors before this stage, at which point you should replace the faulty drive ASAP. Raid 6 has more redundancy, but I don't think it makes up for the exorbitant cost to get the same level of storage. I am planning to get a new ASUSTOR with a 4x4TB setup. Obviously my rebuild risk is higher, but I don't think it's worth me forking out another $400ish just for the same level of storage.
Never use RAID5. If you are an IT tech chances are you will: 1. Get a call at some point in time to go deal with a RAID5 disaster 2. Never get a call about a RAID1 or RAID10 disaster
I always use Raid 10. Raid 5 rebuild is complicated. I've seen where it never finishes rebuilding. Raid 10 rebuilds usually finish in a couple hours. I know R10 is pricey, but it is 100% worth it. FYI, Setup some monitoring of your servers as well so you don't have multiple hard drive failures before noticing.
Raid 6 on 4 Drives cuts drives in half like Raid 1, right? so Raid 5 then… if Drives are more then four (>4) then Raid 6... did I understand that correct?
im about to buy a new NAS, it will have 8 drives that are 4TB each. is RAID 6 still effective enough to have a low chance of data loss if a drive should fail?
Of course backups are a must unless you enjoy playing Russian roulette. As a former network administrator, over 20 years, we had 4 drive failures in a raid 5 setup. Luckily, we never had a problem with an URE. But if we had, are drives were backed up to tape drive and other hard drives. URE's are probably rare, but when they happen, it can be disastrous. Will the average home user have a failed drive in their raid 5 and a URE? Probably not, but is it worth taking the chance over.
Hi. I want to build a Raid 10 with 4 HDDs. But now I ask myself what if my motherboard breaks down at some point ? Can I simply connect the 4 HDDs to a new and more modern MB that can also run Raid 10 and it will continue to run (plug & play) as before ? Or do I have to set up and format Raid 10 and then have to upload the data from a last backup ? Thank you
I know this video is a million years old in TH-cam years but there is one downside to raid 6, but you have to have minimum 4 drives to do that. Shout out to anyone watching this video in 2020
Joe. Thanks for the Vid. Question: If you get an URE on a 4 disk RAID 5 during rebuild, is that disk array still readable, assuming a second disk hasn't crashed? i.e can you still copy off the data onto a backup?
The filesystem will likely survive, but an individual file will become corrupt and will need to be restored from backup. It's also possible that the corrupt sector will be in free space, in which case nothing bad will happen. The biggest concern is that when rebuilding the array, all of the other drives are under a lot of stress, so it's likely that a second drive will fail during the rebuild and all data will need to be restored from backup.
Question: let's say if one drive fails in RAID5 array, can I just remove the bad drive and access the data, I mean, before adding a new drive and rebuilding the array? If yes, then I can move all the data to another storage system. In this case, if one of the remaining drives has URE, then I guess I will just loose very few data but not all of them?
As long you don t replace the drive you should have still access to your data. And in my opinion the best way is to do a backup before you start the raid rebuild.
@@theduck001 right, that's also what I understand, which says if I can got a temporary place to hold all my data, I don't need to worry about the rebuild failure risk.
@@diwu4575 Yeah but the lower the rebuild sucess is the more new backup possibilty of same capacity you will need. And the read error can also be when doing backup, but you have more tries and you can do file by file if there's a problem. But I agree. It is flexible, if a hdd breaks you just need to buy capacity to do a new backup. (I would not overwrite one of my existing 2 backups or an archive in the unsecure state that you have a broken raid with your real time data...). Actually i recommand to have minimum 2 backups and 1 old archive. Backups protect you against raid fail. The archive protects you against encryption virus you will recognize too late or copied errors...
personaly i know about a risk of loosing data from RAID5 during rebuild cuz drives are 100% loaded ( work+raid rebuilding) and have bigger chance to fail at this time, so basicly you can loose data on RAID5 during rebuild process, hard to tell if its cuz of URE or cuz something else, but RAID5 is ok basicly to 4-5 drives max, everything above is for RAID6 or if you can afford it, i belive none on common users cant( and dont even need it :D) then something like RAID60 or w/e
i plan to use raid 6 for backup. but if i need to back up my storage offline then maybe raid 5 or just jbod is okay. because i dont do server for people. my server/nas is only for me and few member of family. downtime was never be issue for me. how to backup the storage is the true issue
A lot of newer harddrives like the ironwolf pro have one URE in 10^15 bits instead of 10^14. So that means 125 tb of storage where one bit will fail you.
hey guys i have heard that RAID 5 can tolerate multiple disk failures if we do something to parity group, is it possible this question is haunting me can any one explain thanks...
I used raid 0 to install my steam and origin games on, since saves aren't usually stored in the install directory, and I have no data cap. it wouldn't be a big issue if the raid failed, and it's a lot cheaper than buying a SSD big enough
I understand the "You can not rebuild" but if i run Raid 5 and 1 drive fails, i will still have access to the data even though the raid is degraded. Would it not be possible to do a full backup of the NAS before trying the rebuild. ?Also i run Raid 5, but i also have another NAS with a single disc, where i have an Automatic backup job set up so i have the data on that disc as well.Also the Raid 5 NAS is in my locked shed, and the other disc is inside my house.This way it is really unlikely that both Buildings would burn Down (Shed is not next to the house) and also it is unlikely that both NAS'es would be stolen. The thief would set off the alarm when entering any of the 2 Buildings and would not have time to rob both.
You can use a degraded RAID the same way that you can use a normally-functioning disk or RAID, which means it can still be backed up, but you still run the risk of losing your data if another drive dies before the backup completes. The risk is also compounded by the fact that a degraded RAID will take a hit to read/write performance, meaning the backup will take more time. It's best to have a scripted backup that runs daily or at least a few times each week, so that you can prioritize repairing a RAID if it does become degraded while also being able to recover your data if the whole RAID crashes before you can repair it.
Good video. Possible little issue I saw with this logic- the 10^14 URE rating is for EACH drive. So, each drive will have to read back 2TB to rebuild in this example (assuming the storage is full, right?) This is a ~17% probability that each drive will encounter a URE, right? I forgot my stats to calculate what the total probability of a failure would be but I think it would be 17% *6= 102%. So still almost a definite failure but the logic is different.
the probability of no errors is (1-.17) ^ 6 or .83 ^ 6 or .32 (for no errors) so for 1 or more errors its 1-.32 or .68 or 68% (assuming 6 drives like I think you did)
A lot of people are having trouble understanding that yes, a single URE can indeed cause the entire array to fail to rebuild. Because of the nature of the parity system, it is NOT the same as having a URE on a standalone drive, where only that single file is lost. You need it ALL to rebuild it all. You might be able to send the drives to a data recovery service who could potentially rebuild the array and only lose the files directly using the lost parity data, but that would be very expensive and time consuming, and not necessarily a guarantee.
Not to mention that because array rebuilds take so long, and are very very intensive, it drastically increases the chances of having yet another URE in that time, which could completely ruin any chances of any recovery at all.
You can google something like "Raid 5 rebuild failure probability calculator" if you don't believe me on the odds. But using 10^14 URE rate drives is extremely risky and you actually are VERY likely to experience a rebuild failure even with small arrays.
In any case, when looking to buy drives for a NAS, my suggestion is ONLY buy a drive with a URE rate equal to or better than 1 per 10^15 bits. This can be found by googling for the 'Data Sheet' of the specific model drive you're looking for. It will be called something like "Non-recoverable read errors rate", "Error Rate (non-recoverable)", "Unrecoverable Bit Error Rate (UBER)", or something similar, probably listed along with the 'reliability' specs in the data sheet.
I haven't really been able to even find any HDDs with a URE rate better than 10^15. However, with SSDs you can usually find better URE rates. Data sheets for consumer desktop SSDs (Like samsung's EVO/PRO SSD line or Seagate's Barracuda SSD line) don't list URE rates. But Seagate's "Ironwolf" NAS SSD line actually have URE rates of 1 in 10^17, so 100 times better than any HDD I've seen. However those cost several times more than an HDD (right now it's $750 for just a 3.8TB drive).
Jeeeez rebuild issues where the whole thing dies is when old RAID cards would just give up. Software doesn't, could lose a few files around that failure (which isn't as likely as any of these calculators say. Rebuilds read from multiple disks too...
So how does a raid 6 help with ures?
I have a question. Given your example you have 14TB with 7 drives in RAID 5 parity is split evenly across every one. So that means that each drive has only 2TB/7 dedicated for parity. When rebuilding a failed drive does it mean you need to read remaining DATA plus Parity ( whole 12TB ) or not ( maybe a smaller amount is enough )? What i want to ask is basically how does rebuilding work?
@@dennisrkb Watch the video AGAIN!!
I understand that, but according to a thread:
"most hardware RAID will abort the reconstruction and some will also mark the array as failed, bringing it down. The rationale is that if an URE happens during a RAID5 rebuild it means some data are lost, so it is better to completely stop the array rather that risking silent data corruption. Note: some hardware RAID (mainly LSI based) will instead puncture the array, allowing the rebuild to proceed while marking the affected sector as unreadable (similar to how Linux software RAID behaves).
linux software RAID can be instructed to a) stop the array rebuild (the only behavior of "ancient" MDRAID/kernels builds) or b) continue with the rebuild process marking some LBA as bad/inaccessible. The rationale is that it is better to let the user do his choice: after all, a single URE can be on free space, not affecting data at all (or affecting only unimportant files);
ZRAID will show some file as corrupted, but it will continue with the rebuild process (see here for an example). Again, the rationale is that it is better to continue and report back to the user, enabling him to make an informed choice."
meaning that its not a big deal if I use Linux software RAID 5? right?
I would argue that the bigger lesson here isn't to get overly hung up over RAID5 vs RAID6, but to remember not to fully trust either of them and always have a proper backup. A good backup system turns a nightmare into an inconvenience.
100%. A Raid is not a substitute for a backup, no matter how you stripe or parity it. Even raid 60 could have an internal power surge or something and toast all those spinning drives at once... your data NEEDS to be elsewhere, off the raid, if it matters.
This is 6 years old, but is as misunderstood now as it was then, and when this was first floated around 2009. The HDD data sheet specs for bit read errors are a statistical expression of failure rates [and a poor way to express it], not a performance specification. RAID 5 is absolutely fine - drives fail at around 1% PA for the cohort of drives used in such arrays. Another way to look at is [also misleading but uses the same logic as the RAID 5 is dead concept] if 1 HDD from a 6 HDD R5 array fails - that event itself is way way beyond the URE rate that 6 drives collectively would have ever experienced if we believe a plain text reading of the URE spec.
What a load of BC.
First of all, explanation for an average home user goes like this:
If you build a redundant array with:
2 drives - your only meaningful choice is RAID1
3 drives - your only choice is RAID5
4 drives - RAID5 for capacity vs RAID10 for redundancy.
Only starting with 5 drive array you should consider using RAID6.
And, of course, you don't lose ALL data with RAID5 disk loss and URE. You lose one file. It might be critical for an enterprise environment, IF they don't make backups. For an average homeuser Joe it means one corrupted photo, or movie. Not the end of the world. And even Joe should make backups of critical data.
Andris Orinskis and beside that, if the data is so important you need an offsite backup, so RAID 6 is not needed
Thanks for th einfo. If the controller restored a degraded raid array, will it report any URE's (and affected files?)?
Andris Orinskis what if you loose an important file that would go un noticed until necessory for use
If you have 3 drives and have to put them in an array, you buy another drive to do so. Who even needs to RAID 3 drives in which kind of environment for what purpose anyway?
Exactly what I thought also when watching the video. But nevertheless I'm thankful the guy made the video. It's not wrong, just not applicable for me.
I know some nutjob who uses 8 SSDs in raid zero. That dude's crazy.
+Colton Blumhagen Well, I'm not familiar with the error rates of SSDs. If they have very low error rates (if at all), then it might be safe. This video kind of assumes the hard drives have a URE of 10^14, but if you have one with 10^15 or even 10^16, you could probably get away with RAID5.
Bombersnomore Haha. This guy gets it.
Yeah, this video goes hand in hand with the "how long do hard drives last" video. But that one doesn't go into the difference between sata, sas and ssd drives and what vendors make the higher quality drives. Unfortunately because of all the low quality production these days all of this makes a difference.
+Ebel Ray I might have a new video coming soon about different types of drives ;)
+ThioJoeTech i was going to suggest that as a video for you but you beat me to it. Hard drives can give you lots of directions for multiple videos too I think.
it doesn't matter if the raid fails because you should have a back up of the raid itself.
You'd be surprised/depressed how many people think RAID = backups.
Having an array fail is kind of a big deal, even with backups.
RAID1/10/5/6 is about availability of data not safety of data. I.e. minor failures not causing an interruption of service, not protection from loss of data from drive failure, server failure, building burning down floods threat explosions etc etc etc.
+Joseph Lucas: Only if the downtime is costing you a lot of money. Otherwise, it's not a big deal at all. A mild inconvenience at best.
Good source of misinformation. If one drive goes you still have access to the logical disk even if another drive has a read error. Take a backup of the logical drive using Acronis or similar software, replace the defective drives, rebuild the RAID and restore your backup. You wouldn't lose everything as this video states unless you have two full drive failures, but then you should be doing regular backups anyways. RAID is about keeping your data available all the time , not protecting it from loss.
2 drives use raid 1,
3-4 drives use Raid 5,
5+ drives use raid 6.
I use Raid 5 for 10 SAS drives
Raid 5 can have u to 16 drives
@@jabetajones the amount disks is limited by the nas or server your using
@@leexgx oh ok im kinda new to pcs
@@jabetajones the more disks you use the higher chance of a double fault can happen and loss of data (or all data if 2 disks fail) that's where raid6 should be used,
but size of the disks also matter as well over say 4TB you should use RAID6 as well (unless you don't mind restoring your data from backup if you have one) if you don't have a backup you must use RAID6 minimise risk of total data loss but you should have critical information backup up (like pictures, personal videos, and documents)
The problem with raid 5 it's not very forgiving when your rebuilding the array after a failed disk has been replaced because if another fault happens
1, because of a data error witch will result in Data loss (normally you can still access the remaining unaffected data)
2, a second disk fails and you lose all data with bi way of recovery
The statistical probability of 3 faults happening when you got RAID 6 setup is very unlikely (but not impossible just unlikely, but gets more likely as each disks get larger) but raid 6 good enough for quite a long time even with 16Tab disks
as 2 disk failures at the same time are rare, as raid controllers and nas boxes do a weekly or monthly raid patrol read and data consistently checks (depending on how they are configured) witch normally weed out problems disks before they suddenly fail
For small ammount of disks (like 4 or 6) usefull space / used space ratio is too small to consider RAID 6, you just need to stick with WD Red, Seagate (not Barracudas!) Ironwolf / Constellation and similar kind of disks and you'll be fine. For larger ammounts, you use ZFS anyway to avoid bit rot. The only really dangerous scenarios i see are some instant power failures and thus you need either RAID controller with battery or UPS.
People have different storage needs. There isn't a one size fits all solution, and an absolute statement like "RAID 5 is bad, never use it" is just silly even today. RAID 6 _is_ safer for large arrays where more disks means more points of failure, but for smaller arrays RAID 5 is still fine. It may even be preferable because it has better storage efficiency.
Besides, you should be backing up data that's important anyway. RAID is only meant for minimizing down time when a drive dies. If you're ever using it by itself as a substitute for a true backup then you're an idiot.
Whats the "safety limit" of raid5? at which number of drives it makes more sense to got with raid6? honest question
I can confirm that what he is saying is correct. This has just happened to me on Raid 5. Good thing is that I still backup each week, but HE IS ABSOLUTELY CORRECT!!!
This is because you are using consumer devices
@@TheWayOfTheHeart NOPE! Using enterprise drives and Areca Chasis.
are you using raid card too? 🤔
i'm thinking to use some LSI MegaRAID for RAID 5 btw.
I use raid 1 (plus cloud backup). Sure you don't get the storage size advantage of raid 5, but IMO anything worth keeping is worth keeping 3 copies of.
Good and proper explanation, except i have to wonder about your reasoning why RAID6 is not a futureproof concept (or as drives get bigger).
Unless you get two UREs in the same exact sector in both parity segments, you will always be able to rebuild your lost drive by using the readable sector from the other parity segment and this problem does not scale with drive size.
But how do I download more raids?
+Clark Epic You download a copy of Windows and make as many (virtual) RAID's as you like.
+Clark Epic i think you can use ramdisk
you can download it from amd
Unfortunately you can't download extra raids like you can with more rams
+ThioJoeTech Wrong channel to reply from, should have been main channel (would have made more sense) :)
You need to write a letter to the president asking for a download link. Also works for free money.
Thank you for this vid. Now I backup my data 'cause of bad sector, tried raid 5 twice and it always have error, will try this raid setup :D
I've been using a 12 drive Linux Software RAID5 setup the past 10 years. Gradually growing and replacing drives (from 8x200GB to my current 12x2TB). Although I agree with you that RAID6 would be safer, I've never experienced the problems you're describing. I've had 1 drive failing periodically, and replaced maybe 4-5 drives in total (due to failed/dead drives) during the same time period.
This is data that I am comfortable loosing, although I would prefer not to.
The worst problem I've encountered during this time, is the failing of a controller, where symptoms indicate that multiple drives are lost. But thanks to Software RAID, I'm not bound to a specific hardware disk controller. Which makes it easy to rebuild or even move the entire array to different hardware. So if I encounter the bad sector issue I'll report back when that happens, although I haven't seen a bad sector since the mid 90s
I have yet to see a RAID controller that stopped rebuilding a disk because of ONE read error. So far I only managed to get a corrupted file as a result but the rest was recovered nicely. So the big question is: Is ONE defective file (which is possible, but not guaranteed to happen) worth an extra disk (and a bigger NAS system in case you need more than 4 drives)?
Raid 6 doesn't sound like a good option if you're looking to build a four drive array like I am. I'm looking to build a four drive setup using 3, 4 or 5tb drives, but I don't want to loose the storage of two drives, so raid 5 is really the only option. If I was fine with only having two drives worth of space, may as well go raid 10.
Tripple partiy will never come. With the increase in disk sizes we are running into the limits of what RAID can do with parity. Rebuilding huge arrays simply takes too long and is no longer worth it. If rebuilding takes over a week it simply takes too long. RAID5 is not a bad choice, depending on your use case. You've taken a very specific use case where the capacity of the array equals the URE to say that you are statistically certain that one of the drives will have a URE. While I did not study statistics in school, I do doubt you can simply state it like that. You have 6 remaining drives, each with 10^14 bits so it wouldn't surprise me that you can say in this case that the actual value would be 6*10^14. And what would you do with the new 14TB drives that got just announced. If they also have a URE of 10^14 you statistically will always have an URE on the drive. Not use it? (welcome to the wonderfull world of statistics).
And to add a few more things:
- The URE is the avg over the lifespan of a drive. It might be possible that you read 1000TB of a drive without a single URE and then all of a sudden you get a multitude of them resulting in the avg of once every 12TB
- The URE is a single bit in a sector. Every sector has a CRC that gets updated on every write (how else would you know about the error?) The URE can even be in the CRC data. Most bit errors happen on write anyway and the system will always read back written data to confirm it is written OK, further diminishing the chance of having bad data due to a bad bit.
- On modern drives with 4k sectors if you write small amounts of information the drive will pad the sector with the same information, as long as it is smaller than 2k, thus making a copy of the same file making it more likely that the information can be recovered.
And lastly, let's not forget: RAID is not a substitute for backups. If you do come across a URE, you should still be able to recover the unrecoverable data from your backups.
I use RAID 5 for all my servers and never had a issue when replacing a dead HDD. Touch Wood.
Touch SSD
I rarely have dead drives to begin with. One or two in the last fifteen years. Just periodically replace them.
Virtualization has made the vast majority of my hardware super easy to replace.
Move guests to different box. Box can be turned off.
@@kevinsky86 How odd you rarely have any dead drives if you periodically replace em. *insert meme*
@@mesaber86 it sounds like periodically is being used to define 1/2 failures, not uncovering hidden secret failures.
Although I could be wrong, only Kevin de Bie really knows how many drives have failed.
"I use RAID 5 for all my servers and never had a issue when replacing a dead HDD."
Famous last words before cold sweats, tons of sleep deprivation, and data loss.... If one of your other drives fails during recovery you're SOL......smh
I lost a primary domain controller once back in 2003. The raid 5 didn't help, neither did the secondary domain controller. It was a huge mess. I had to accept defeat and rebuild everything from scratch in two weeks with the help of a friend. Slept at the office for that time. So now I am a bit skeptical to all redundancy systems :)
For the record, raid 5 is still fine for small arrays like an executive workstation for your CEO. His premise here is that you're using a very large array like in a NAS, Backup array, or Data Center
NAS isn't "very large" ... certainly nothing like a data center. It's standard practice for people who do content creation and basic video projects. An 8TB NAS is hardly a huge amount of storage for a TH-camr/Vlogger.
So for a 4bay NAS, would Raid 1+0 be a better option than raid 6?
I've rebuilt a few dozen 20~30TB RAID 6 arrays without a single rebuild failure. RAID 6 is solid. This has always been with Adaptec controllers (51645, 6805, 71605 etc.)
where do you buy a RAID 6 system? So 8 years later a RAID 5 is available anywhere. It can do RAID 1, 5, 10 in the same box but never 6. Have times changed?
Normally if you create a raid set you also set a spare drive so it can start rebuilding the raid asap. Depending the raid you want/need it will be better one or the other. Also keeping in mind performance of course
Hey Joe, lets us consider the following scenario. what is I have 10x8TB array of disks configured with RAID6. So even if two disks fails( Since RAID6 ). I will be having 64TB of data to be scanned for recovering the last drives. So from your URE probability calculation, there is URE happening. Then RAID6 is also not useful. What would you suggest about this... !!!! Thanks in advance
The math isn't quite that simple, because URE probabilities are independent from disk to disk. Reading 64TB across disks that each have a URE rate of 1 in 10^14 does not guarantee that a URE will happen, because a URE not happening on one disk has no bearing on whether or not a URE happens on another disk. It'd be like thinking that rolling a 6-sided die six times guarantees that you will roll a specific number at least once. You'll get _more chances_ to roll a specific number, but failing to roll that number will never make it more likely that you will succeed to roll that number in the future.
All of that said, adding more drives to an array will always statistically increase the likelihood of a failure by a little bit, unless you're using RAID 1 where more drives always means more fault tolerance. And 10 drives is quite large for a RAID 6 array. You should definitely have regularly scheduled backups if you don't already. Off-site is best if it's feasible, but any backup at all is better than nothing.
This is a topic at work as we deal with a lot of data. So the answer is much more complicated. For some more to chew on:
1. When dealing with large amounts of data with many mechanical drives the answer typically comes down to ZFS RAID-Z level 2, which is somewhat similar to RAID except more check sums in more places, CoW, snapshots, etc. Plus flash based cache drives as RAID-Z is not the fastest thing in the world on mechanical drives.
2. For small arrays, a few big things are out there often favoring RAID 5. You can find reliability reports on many drives and avoid the known bad lines of drives. You can look at SMART data and pull a drive as soon as it starts going bad. As this is usually a slow process, most competent admins pull drives long before they actually fail and enterprise drive vendors usually take back drives even if they have one remapped sector. You can use hot swap as often the double fault happens when you power down a system. You can buy drives in a staggered fashion as sometimes sequential serial numbers in your array lead to drives failing close together in time, which is the last thing you want in a RAID. Flash drives if you look around will sometimes have RAIN, which is NAND level RAID. Flash also will often fail into read only mode, giving you a chance to copy over to a new drive, but not always. However flash in a RAID tends to get beat on more both because RAID controllers don't support TRIM and if you are using throwing money into a flash array, it is probably because you beat on the system a lot more than the average bear. However this wear concern can be addressed with higher endurance flash at a higher cost. For example my main home flash array with Crucial MX500 drives should burn out its flash in about 6 years at the rate I am currently grinding away at it, which is about how long my older flash drives on low use systems failed due to old age, however for twice the cost I could get SSDs with 10-100x the endurance and never come close to burning them out before they failed due to old age.
3. With any data storage system, you should not completely rely on the RAID to save you. Someone might delete important files and RAID does not stop file deletion. Someone might try updates that go horribly wrong. A cooling fan may fail and the whole array bakes to death. The computer with RAID array might get stolen or lost to a fire. Or like the other day with a software RAID 5 a power loss caused the array to scramble while the hardware RAID on the same system was fine. I have also seen arrays lost when the power supply in the box (often cheap NAS) fails and fries all of the electronics in the process. I have seen all of the hard drives seize at the same time after many months of running because the drive manufacture used the wrong lubricant, so once the drives stopped spinning, in this case routine maintenance, they never spun again.
100%. A Raid is not a substitute for a backup, no matter how you stripe or parity it. Even raid 60 could have an internal power surge or something and toast all those spinning drives at once... your data NEEDS to be elsewhere, off the raid, if it matters.
I am not fully convinced by you. If errors are so rare, then when doing recovery in RAID 5, after one disc goes dead - we would loose maybe few files, and it doesn't mean that whole recovery would go wrong. Right?? (In RAID 1, after we lose one disc, the second one also may have some errors, and it doesn't mean that the whole recovery wouldnt work...) My concern about RAID 5 is: maybe you're right, if the discs are encrypted? Could you please refer to this message ThioJoeTech?? Perhaps I misunderstood your video, sorry :(
can you make a video showing how to download more cpu cores?
HAHA U GET REDDIT WOOSH
lol
OK, what I don't get with parity 5 is if you lose drive 0 (2:48) what happens to 'Dp'? is that copy of the parity lost forever or is it actually copied onto one of the other drives as well?
I understand RAID 6 has a slower write speed than RAID 5. What about RAID 5 plus a hot-swap drive? That's what I'm doing right now. So if a drive goes bad the hot-swap kicks in and it rebuilding the array automatically. I replace the bad drive with a new one and it becomes my new hot-swap. You get part of the benefit of RAID 6 (although not the exact same) without the performance hits of 6 vs 5.
all that means is less time is required to discover that a drive has died. If you have any errors during a raid 5 rebuild, data is toast. Raid 6 allows the data to be protected during a raid 5 rebuild
Thanks, good information. I guess my only question is: How do you archive this much data when you do not need it on a local drive? Such as old youtube videos. I want to keep them, but not locally when I use up my 20-30TB NAS.
Ok so I can use RAID 5. Thanks
not sure if you are still monitoring this feed, but any updates on RAID systems and/or tech on this subject. Dusting off an old project and want to finish it this winter. Thanks
I'm not sure this advice is particularly helpful for a SOHO user. His
implication is that that your RAID array somehow forms part of your
backup strategy (or perhaps even *is* your backup strategy) which in
itself is a poor message to convey. I really can't see too much issue
with RAID 5 + proper backups. How many SOHO environments need 2 levels
of redundancy unless in special situations?
Mark Browell what do you use besides raid to backup your data?
The best backup plan is to have 2 identical NAS using HDD from 2 different companies.
There is something I don't understand about this: Assume you were to use 12TB HDDs with one URE at every 10^14 bits, your probability of an URE would approach 100% during a RAID-5 rebuild. However it would also approach 100% during a RAID-6 rebuild, because every HDD itself would likely have at least one URE. So no matter how many parity HDDs you use, you are always hosed and thus RAID-6 would give you anything over RAID-5.
What am I missing here?
The parity file is duplicated. That's the critical part.
awesome video!, but I think you should of also include RAID 10 as well. because it gives you best of both RAID 0 & 1.
I have read that for a raid with multiple hard drives next to each other, you should only use NAS or Enterprise hard drives as they have vibration detection. Because cheap hard disks without vibration detection are actually only intended to be alone in the PC case or individually in an external case.
Wait is this the non troll channel?
Adolf Hitler Okay, thanks Hitler. You always got my back.
I'm Not Chase I enjoyed this conversation immensely ^^^
Nope, but funny. Subject to Satire, being part of your sense of humour? No but it is like phoning tech support and speaking to two technicians, a day apart. Explain the Lion? Since you need the attention? :) Then go back to Facebook, please. :)
You are aware that you lose performance in a raid 6 array.
I work for an MSP that specializes in dental I.T. These places due to Canadian standards must keep data for patients for 10 years. Every one of our 2000 servers in play all have a raid 5 arrry and for the most part there are no issues. Now keep on mind even with a raid array we still run a cloud and on site back up of the host and the DC VM and Server VM. You should never %100 rely on you raid array.
ZFS is the future of drive arrays.
It's total overkill for desktop systems or small servers.
It-s not. My home build 4 disk NAS, primarily used for archiving is build on ZFS which has one big advantage over RAID5. It is protected against bit-rot.
how is it a big advantage? bit-rot is not undetectable and between smart and filesystem protection you'd have to be incredibly unlucky to end up with a corrupted file - you DO have backups right? and i have never heard of a raid array failing due to bit-rot. but go on, all you zfs ppl just keep saying it over and over, it's not going to make zfs any better you know.
I've had my share of corrupted files on large storage (2TB) devices.
th00ht Every harddisk has already some pretty good ECC algorithm against bitrot, i really think it's most likely a ZFS marketing strategy and not a real problem for the majority of people.
I know this was a few years ago, but correct me if I'm wrong. This really depends on the number of drives that you have? Right? What if I'm only using 4 drives instead of 7? This will certainly change my chances a bit. I agree with what's said, but the scenario certainly depends on a few factors.
i can no longer take this guy seriously i just assume its bullish
why are so many disabled people in the youtube comments
who can??!
Right, but who's making the distinction...
WOAH I am literally just discovering this fact!!! Why isn't this video fake????
Vince Cropani If you listen to what he is actually saying, he is correct. The parity drive can only make up for failure or errors on one drive. If you have a drive failure and then find reading errors on another, you will likely lose data. RAID 6 is designed to help overcome this.
Thanks for the education. If I'm using SSD drives and do very little writing (mostly just archives) on a 12TB array (2TB x 6) Should I go for RAID 6 or SHR2? or just go with SHR or RAID 5?
Great video. I'll be using RAID 6 on my new storage!
Many thanks for your effort,
cheers!
i am planning to setup a NAS for the first time. i know you talking about drive failures. what happen if the raid controller fails? lets just say if i got a qnas enclosure and setup a raid 6. if qnas dies will my whole raid 6's data lost?
eh... Delving into a grey area here... For most home users, Raid 5 is just fine, even if the array is larger than 12 TB. Lets say worse case scenario, and you loose a disk, and error correction failed to detect the URE, and during re-build the URE is discovered hence fails to rebuild. There is a another option that Joe fails to mention. Actually, there's 4 or 5 other options, but I'm not going to get into them right now. The easiest thing to do if your Raid array fails to rebuild due to a URE, is to copy off the data. Yes, that means you will likely have to buy a couple of huge drives, and stand up temporary storage, which is a pain, but you will be able to copy the data off.
Folks... Don't freak out over URE. Yes, the risk is there, but if you're buying quality drives the chances for URE is even smaller than what Joe stated. If you're Enterprise or your data is that important I figure you're going to go with 6 or Raid 10 anyway..... So, my point is just geared to the folks at home that may not have critical data and don't want to loose that additional disk space by going to Raid 6.
What are the other options he fails to mention? I’m trying to find a good safe way to back up my data and thought raid was the answer. After reading the comments, however, I’m seeing that it is not.
What you just said there is no simple task for home user, where if you had been using RAID6 to begin with a URE won't be a problem (or second disk fail) just replace the disk and let it rebuild
It be nice if Synology used 5 bay NAS by default (no 4 bay nas's) from 2 bay ones to cover the extra disk needed for RAID6/SHR2
Also UREs can just be skipped over. Unlikely you will get more than a handful. So worst case you rebuild the array with 5 files missing. Obviously if you have a backup then its fine also. Easily encounter same problem with RAID 6 with UREs. So video isn't exactly telling the whole truth. Rebuilds can be hard on drives so possible it can lead to other disk failures during it, unlikely but can happen. This video hides the full facts and tells only a narrative that he wants you to hear
A few years after acquiring a NAS box with 5 disks I set it up in RAID-5. Within a week one drive failed and during the rebuild a second drive failed probably as it was loaded rebuilding the raid array. So I lost data. Nothing serious as I hadn't been using this for long. Since then I replaced this with a RAID-6 setup and have had drives fail since then but have been able to rebuild the array without issues. So yes depending on a single disk for parity can be risky especially as the load regenerating the full array can trigger a second fatal failure.
well you can just use RAID 10, simple, efective and reliable.. i know you have less space.... but its more secure for critical data ;)
And faster too
RAID10 doesn't use parity so it's technically less secure then RAID6 (RAID10 is for some redundancy + speed)
@@leexgx are you serious? Raid 10 is stripping+ mirroring so it is more secure and more fast, of course at expense of more disks and less space, it what's it is used in critical infrastrutures... trust me ... it's my work loool
I bought 4 drives 14 TB WD Ultrastart and QNAP what raid can I apply to have more space and not experience UTE .Thank you
So, the TLDR is you realistically have options of RAID 5 and RAID 10, and I would recommend RAID 10.
The longer version. RAID 5 means you stripe data across 3 discs and use the 4th for parity. This will give you a total effective storage of 42 Tb. In RAID 10 you stripe the data across 2 discs and use the other 2 to maintain a completely independent copy of the first 2. So in RAID 10 you won't have as much total storage, only 28 Tb, since you use 2 discs for redundancy instead of 1. Raid 5 will also get you more read/write performance from your array. Since the data is striped across 3 discs instead of 2, you could notionally get up to 50% better speed from the RAID 5 setup compared to RAID 10, although that will probably be limited by RAID overhead and network bottlenecks.
Now, as to UREs. WD Ultrastart drives are rated for an average of 1 URE per 125 Tb of data read (10^15 bits). That means during a rebuild (if we assume the array is basically full) each individual disc has about a 1 in 18 chance of encountering a URE (14 Tb/125 Tb x 0.5). The chances that 3 drives manage to rebuild without a URE are then about 5 in 6. So, if you have a drive failure with RAID 5, there is a 1 in 6 chance that the array fails to rebuild due to a URE and you suffer data loss. In RAID 10, you maintain a completely independent copy of the data at all times. So your data is protected without having to rebuild the array. In my opinion, a 1 in 6 chance of data loss if I have to rebuild the array is too high, its worth sacrificing the extra 14 Tb of capacity for better fault tolerance.
That is why we use Raid 5 and 6, but need to have a Backup. I see your point. But I cannot see your conclusion. RAID and backup are not the same thing
Agree, no need for 3 parity drives. you need RAID 5 if you use something like 3 to 5 disks, or Raid 6 if you are using 5 to 32 disks AND your Backup.
Christoph Esch then what do you use as a backup?
@@jordanbanko5347 I'd go Raid 5 and look into Crashplan. for $10 a month it backs up what folders you ask it to with unlimited space. Easy to restore from and keeps file versions so you can go back to a file version from last week if you want to.
Is all the data really lost though, with the read error? Wouldn't you have like one or two corrupt files, but still get the rest back?
I had a major RAID 5 failure. I lost 15TB worth of data 5 disks. I've now got 3 NAS with a total of 85TB each running RAID 6 and being backed up in realtime to each other. When my RAID 5 did crash and I went to install the replacement drive that is when the shit got real and the whole RAID died. I had to use a RAID recovery software to retrieve my data most of it but not all. Enough to get me out of hot water.
There are two kinds of people on earth people who have lost data and learned from it and other who have yet to learn. Peace from Sydney Australia
+OnTarget100 Dude thanks, you just crystallized my decision between RAID 5 and RAID 1.
OnTarget100 More fool you for not having a backup. I use two RAID 5 arrays, one backs up to the other nightly. The first array also backs up to cloud storage as soon as I write to it. I do this because I don't want to loose data, not because I learned the hard way
If you follow the 3-2-1 rule you shouldn't ever lose data.
Must have been some valuable data lost to over compensate like that. I’ve lost data too, usually when upgrading OSS. By my own mistake of thinking I’ve completely copied data over , then resetting the disks for the new os.
I’ve never lost data due to a disk failure. I also never use cheap disks. Only HGST hard disks and Samsung SSDs. I’ve had hard drives fail but only WS and toshiba.
You’re smart and have the budget to have not only redundancy for disks but also separate enclosures. Very cool.
I’m in the process of replacing all my bad Disha from 8 4TB to 10TB. At 350$ each, it’s damn expensive. But, like you, I have valuable or priceless (old work files I can never get back), so I get it. For sure.
Photos fade, digitals do not ... music leaves popularity thus impossible download, same with films and tv shoes.
What bad devices did you do with?
but with a professional 4 or 8 bay server rigg it should be fine right? For raid 5?
What is your opinion using SHR-1 with BTRFS with four (4) 6TB NAS (Iron Wolf) disk drives in a Synology DS918+?
trust issues.
Hi from 2019 - what do you think is the new standard now Joe?
As usual very interesting. I have an external drive with 2 X 2 GB drives in RAID 1 it meets my needs and budget. Perhaps in future you could talk about building an external RAID array.
When you want to be extremely save you even need to build a backup archive... Because bad data can be copied over time ? So you keep a backup drive f.e. every 3 years... After 30 years you have an archive with 10 drives... When ever you find out a data is broken. Maybe you will load it from an older drive... Is that true what I am writing ?
What's wrong with raid one 🤷🏻♂️
Usable space cost
What do you guys recommend me for a torrent movie library? (Don't judge me, where I live Blu rays are not sold and streaming quality is not on par for OLED TVs) I want to have around 50TBs in RAID5 or 6. As far as I know HGST is the best brand. Is it still the same today? And what NAS Brand?
excellent content from the Michael Cera of Tech :D
@Thios.Joe Tech. I just picked up a 4 dri.. It just seems so wasteful because RAID 5 practically speaking gives you 2 drive redundancy but you lose a lot of storage, I don't need more than about 8 T. I thought to use 4 4 T drives which would give me 12 T of storage in RAID 5....but only 8 with RAID 6.....seems like wasting a lot of storage. I have also heard that one should routinely scrub the drives i a NAS....not sure if that is more than SMART or some other technology.
Is RAID 10 faster than RAID 5 and 6?
yes raid 10 is actually 2 raid 1 in raid 0... it's actually 1+0
HW RAID10 -> No.
Linux software RAID10 -> Yes and No.
RAID5 achieves 95% bare hardware read speed. RAID6 a fraction of that.
RAID5 and RAID6 takes a tremendous hit on write speed. Much more so on RAID6.
RAID6 is not worth it. RAID5 for upto say 6 drives, further than that RAID50. Much higher performance, same level of redundancy.
RAID != Backup.
That depends on your RAID controller card. My Areca cards that I use can run RAID 6 at nearly the same speed as they can run RAID 5. I've tested it, and with 8 drives it can read/write at ~750Mb/s in RAID 6. Cheap RAID controllers, however, won't be able to do that. The drives independently will only do around 130Mb/s, so it's losing very little to overhead if you figure that 6 drives (not counting parity) X 130 = 780Mb/s (max hardware speed) and I'm getting 750.
130 Mb/s sounds quite slow. My drives do around 120 MiB/s Thats around 8.3 times as fast. I would check your drives.
I just bought QNAP TVS-472XT 4 bay Thunderbolt 3 4GB (for Editing FCPX on my MacBook Pro) with four (4) Samsung 1TB 860 EVO SATA SSDs... can I configure them more than RAID 0 like RAID 5 or 6 or 10?!?
RAIDs? Isn't that something you do in video games?
lmao
+ThioJoe I thought RAIDs was an std
Yes u raid in minecraft factions
I THOUGHT RAID WAS A BUG REPELLANT!!
Actually RAIDS is the same thing as aids, but you get it from rape, that's what the "R" is for. Don't trust this guy's channel, ThioJoe will make a video explaining it.
I don't agree, in many cases (at home, etc..) a NAS with 3 drives RAID 5 it's good enough. If you want to assure to don't lose data you schedule a regular backup of your NAS into an external drive. This is the best practice
Wait wait, is this his joke channel or his real one lol?
I have an Asustor 2404TE, one of the very early line of Ausstor NAS that had an early Atom CPU. It had a 4x3TB setup (WD Reds) at RAID-5. The 3pin cable conencted to it wore and there was a power surge that caused damage to one of my drives. Eventually it was reporting a lot of rear errors on the drive, which started small then began to build up. I don't know if they were UREs, but there were many. The system rebuilt the raid twice with no issues during this time. I then replaced the faulty drive (4) with another, the RAID rebuilt. That was back in 2017 and there's been no issues since, though the other drives will need replacing soon.
So while there's a risk of URE it's incredibly low, and your drive will likely start reporting errors before this stage, at which point you should replace the faulty drive ASAP. Raid 6 has more redundancy, but I don't think it makes up for the exorbitant cost to get the same level of storage. I am planning to get a new ASUSTOR with a 4x4TB setup. Obviously my rebuild risk is higher, but I don't think it's worth me forking out another $400ish just for the same level of storage.
Just use 6 drives in raid 0 for maximum craziness :) :) :), I live dangerously...
how about home environment with a total of 3-4 drives?
Never use RAID5. If you are an IT tech chances are you will:
1. Get a call at some point in time to go deal with a RAID5 disaster
2. Never get a call about a RAID1 or RAID10 disaster
I always use Raid 10. Raid 5 rebuild is complicated. I've seen where it never finishes rebuilding. Raid 10 rebuilds usually finish in a couple hours. I know R10 is pricey, but it is 100% worth it. FYI, Setup some monitoring of your servers as well so you don't have multiple hard drive failures before noticing.
Raid 6 on 4 Drives cuts drives in half like Raid 1, right? so Raid 5 then… if Drives are more then four (>4) then Raid 6... did I understand that correct?
N A S
im about to buy a new NAS, it will have 8 drives that are 4TB each. is RAID 6 still effective enough to have a low chance of data loss if a drive should fail?
Do you ever blink?
Of course backups are a must unless you enjoy playing Russian roulette. As a former network administrator, over 20 years, we had 4 drive failures in a raid 5 setup. Luckily, we never had a problem with an URE. But if we had, are drives were backed up to tape drive and other hard drives. URE's are probably rare, but when they happen, it can be disastrous. Will the average home user have a failed drive in their raid 5 and a URE? Probably not, but is it worth taking the chance over.
i thought this guy is a troll what happen to him?
Mustafa Hashem he does troll.
Hi. I want to build a Raid 10 with 4 HDDs. But now I ask myself what if my motherboard breaks down at some point ? Can I simply connect the 4 HDDs to a new and more modern MB that can also run Raid 10 and it will continue to run (plug & play) as before ? Or do I have to set up and format Raid 10 and then have to upload the data from a last backup ?
Thank you
I know this video is a million years old in TH-cam years but there is one downside to raid 6, but you have to have minimum 4 drives to do that.
Shout out to anyone watching this video in 2020
Sir, i f i may ask, if you copy or transfer files while Raid 5 is rebuilding, will it affect the system?
your out to lunch!!! you don't know what your talking about!
How about solid State drives? Is it the same thing Or a whole other thing
Leo Warren, what SSD's do you use that die??
Joe. Thanks for the Vid. Question: If you get an URE on a 4 disk RAID 5 during rebuild, is that disk array still readable, assuming a second disk hasn't crashed? i.e can you still copy off the data onto a backup?
Hi Joe. Thanks for the interesting video, but please tell what filesystem cannot survive 1 bit error in 12TB aproximately?
The filesystem will likely survive, but an individual file will become corrupt and will need to be restored from backup. It's also possible that the corrupt sector will be in free space, in which case nothing bad will happen. The biggest concern is that when rebuilding the array, all of the other drives are under a lot of stress, so it's likely that a second drive will fail during the rebuild and all data will need to be restored from backup.
Isn't RAID 6 going to be much slower? I'm using a QNAP TS-453 to stream media and for home theater purposes, don't I want higher performance?
Question: let's say if one drive fails in RAID5 array, can I just remove the bad drive and access the data, I mean, before adding a new drive and rebuilding the array?
If yes, then I can move all the data to another storage system. In this case, if one of the remaining drives has URE, then I guess I will just loose very few data but not all of them?
As long you don t replace the drive you should have still access to your data. And in my opinion the best way is to do a backup before you start the raid rebuild.
@@theduck001 right, that's also what I understand, which says if I can got a temporary place to hold all my data, I don't need to worry about the rebuild failure risk.
@@diwu4575 Yeah but the lower the rebuild sucess is the more new backup possibilty of same capacity you will need. And the read error can also be when doing backup, but you have more tries and you can do file by file if there's a problem. But I agree. It is flexible, if a hdd breaks you just need to buy capacity to do a new backup. (I would not overwrite one of my existing 2 backups or an archive in the unsecure state that you have a broken raid with your real time data...). Actually i recommand to have minimum 2 backups and 1 old archive. Backups protect you against raid fail. The archive protects you against encryption virus you will recognize too late or copied errors...
personaly i know about a risk of loosing data from RAID5 during rebuild cuz drives are 100% loaded ( work+raid rebuilding) and have bigger chance to fail at this time, so basicly you can loose data on RAID5 during rebuild process, hard to tell if its cuz of URE or cuz something else, but RAID5 is ok basicly to 4-5 drives max, everything above is for RAID6 or if you can afford it, i belive none on common users cant( and dont even need it :D) then something like RAID60 or w/e
i plan to use raid 6 for backup. but if i need to back up my storage offline then maybe raid 5 or just jbod is okay. because i dont do server for people. my server/nas is only for me and few member of family. downtime was never be issue for me. how to backup the storage is the true issue
A lot of newer harddrives like the ironwolf pro have one URE in 10^15 bits instead of 10^14. So that means 125 tb of storage where one bit will fail you.
Wouldn't a read error be eliminated by a journalled system like ext4?
how can i implement RAID using NAS Synology
hey guys i have heard that RAID 5 can tolerate multiple disk failures if we do something to parity group, is it possible this question is haunting me can any one explain thanks...
I used raid 0 to install my steam and origin games on, since saves aren't usually stored in the install directory, and I have no data cap. it wouldn't be a big issue if the raid failed, and it's a lot cheaper than buying a SSD big enough
Yemto What is the best way to setup your system with say 100 tries with 15 external tide together with sub 3.0 hub?
I understand the "You can not rebuild" but if i run Raid 5 and 1 drive fails, i will still have access to the data even though the raid is degraded.
Would it not be possible to do a full backup of the NAS before trying the rebuild. ?Also i run Raid 5, but i also have another NAS with a single disc, where i have an Automatic backup job set up so i have the data on that disc as well.Also the Raid 5 NAS is in my locked shed, and the other disc is inside my house.This way it is really unlikely that both Buildings would burn Down (Shed is not next to the house) and also it is unlikely that both NAS'es would be stolen. The thief would set off the alarm when entering any of the 2 Buildings and would not have time to rob both.
You can use a degraded RAID the same way that you can use a normally-functioning disk or RAID, which means it can still be backed up, but you still run the risk of losing your data if another drive dies before the backup completes. The risk is also compounded by the fact that a degraded RAID will take a hit to read/write performance, meaning the backup will take more time.
It's best to have a scripted backup that runs daily or at least a few times each week, so that you can prioritize repairing a RAID if it does become degraded while also being able to recover your data if the whole RAID crashes before you can repair it.
Good video. Possible little issue I saw with this logic- the 10^14 URE rating is for EACH drive. So, each drive will have to read back 2TB to rebuild in this example (assuming the storage is full, right?) This is a ~17% probability that each drive will encounter a URE, right? I forgot my stats to calculate what the total probability of a failure would be but I think it would be 17% *6= 102%. So still almost a definite failure but the logic is different.
the probability of no errors is (1-.17) ^ 6 or .83 ^ 6 or .32 (for no errors) so for 1 or more errors its 1-.32 or .68 or 68% (assuming 6 drives like I think you did)
With RAID 5 and a hot spare don't you get the same result as RAID 6 except you have less probability for URE because you have less drives in the RAID?
Can you make a raid 6 with a ssd caching drive without nested raid cards?