This is the reason I stick with Synology. When people complain about their sub-par hardware and weird hdd compatibility, my primary data is just too important and it is always my first priority that my family photos and such are kept safe. When my family put their trust in me with their data... I don't mess around, and neither should Synology. Good work.
SpaceRex keeps his checksums in order. A periodic yet continuous incorruptible datastream of useful Synology advice. Ought to set up that backup soon...
Your comments on the kids knocking your NAS off of the shelf brings back a rather horrifying image to me. 3 years ago, I was undergoing a major home renovation. My Synology DS920+ was on its shelf in the basement with all of my other network equipment and panel. I didn't realize how close the contractors were to this with their equipment. The next thing I hear is someone yelling my name and to come downstairs. I see that my 920+ has fallen 3 feet off of its shelf and 3 of the 4 drives have spilled out onto the dusty floor. I picked up the pieces, used my compressor to blow out the dust, reassemble and reboot ( in a safer location ) and it came online like nothing even happened. I don't recommend this as a way to physically collision test your NAS but it does say something about them being not quite as fragile as we might think.
True story: I have a DS1821+ that I initially set up with three 20TB drives using BTFS SHR. I decided, later, that I wanted to add a fourth 20TB drive. So, after physically adding the drive, I had the Synology add it to the volume, but this process takes days. When it got to about 40% we had a storm come through and the power went out for about 1 minute. When the Synology rebooted, the volume process picked up right where it left off with absolutely zero data corruption. I thought, for sure, my volume and data would be hosed, but instead, I was amazed.
Dang that’s lucky. I have an ups that can run my nas for about an hour in the event of a power outage because I’m paranoid of data corruption if my nas had a hard shutdown from power failure. Even though it can run for an hour I have it set to just auto shutdown if it switches to ups power. Anything im doing can wait til the power is back on.
First time I have an issue with my NAS and my volume is in critical state (1 HDD dead ; 2 HDDs filesystem error)… Fortunately Synology support is wonderful and we are recovering partially my data through BTRFS recovery. Still crossing my fingers :) Best part is that I just bought another Synology to replicate my data. I was about to receive HDDs when the disaster happened.
I'm still wary of BTRFS after having some bad experiences with it in the past but it's good to know it's gotten better since then. And yeah, always have a backup of anything critical.
SpaceRex, you always tell us to backup our data, and most of us (at least I) do it. I use HyperBack-up to backup the Synology NAS to a remote QNAP NAS and also to an on-site external HDD (again, with HyperBackup). What some of us don't actually know, is how can we recover that data if the Synology NAS breaks? Lets say a kid pushes it over and everything is shattered. I assume I have to buy another Synology NAS, but should I buy the same HDDs, or can I have bigger ones or other RAID configuration? And then, how do I actually recover the data (from the external HDD or from the remote NAS)? I would highly appreciate a tutorial, or at least a reply to my comment if you think that is enough, but will be really helpful. Love what you are doing! Keep up the good work!
Ooo, fascinating topic. I always strived for a system with btrfs which my old synologies didnt support. Now that I got a more powerful one, I enabled btrfs right away
A NAS is not inherently a backup. A RAID is not either. It ONLY protects you against a single HD failure... That is all. The 3,2,1 rule is the bare minimum for data that is important. For my important stuff I keep 5 copies. you might think that is over kill.... Until a storm blows out your electrical killing all your electronics and a subsequent flood destroys your backup.
Checksum and scrubbing is probably the best thing you can do to maintain data integrity. It's good that Synology have an automated process for data scrubbing, but you can also do the same using other open source tools such as cshatag, which runs checksum on all files and store the checksum on the file's tag metadata so that it can compared later. There are lots of other checksum tools, but a NAS RAID on its own is not designed to protect data integrity (it's designed for speed and cheap mass storage). To protect data integrity, you need frequent checksum and frequent backups.
bit rot is data very slowly eroding. I personally believe most of the eroding is from environmental background radiation. I used to have my NAS up high on a shelf, and over the years, I would see sectors errors across all drives appearing and slowly getting worse. But in the last 6 years, I put my servers near the ground inside a makeshift faraday cage and have yet to see a single sector fail. Hard-drives only notice bad data when there comes a time it wants to read that section, and mirroring drives only works if you schedule "Data Scrubbing". I setup data scrubbing every 6 months on all my servers, because its actually quite intensive, doing data scrubbing too often could have a negative effect.
@spacerex Can you PLEASE do a video on enabling data checksum for the HOMES folder? Seems to be impossible??? Yet so much important data is stored there, like Synology photos? Thanks!
Any chance there's a similar capability for Windows? My backup is a cheap 4 bay USB RAID enclosure attached to a Windows PC. It would suck to find your backup had files on it that deteriorated over time.
great video as always. just curious where did you find that keyboard whist support that holds your keyboard? maybe there is one for the Logitech mx mini.
Ideally, i like to start seeing single drive NAS's but sold in two's, both acting as a single server but continues functioning normally when one of them goes offline. When back online, automatically catching up/resyncing. I know it can be done already, but is a headache, wish there was more seamless solutions.
@@SpaceRexWill Could you go into more detail on this? The DSM 7.2 KB page on data scrubbing says: "Data scrubbing is only supported on Btrfs volumes or storage pools of the following RAID types: SHR (consisting of three or more drives), RAID 5, RAID 6, or RAID F1." This leaves me wondering whether this should be interpreted as "Data scrubbing is only supported on [Btrfs volumes] or [storage pools of the following RAID types: SHR (consisting of three or more drives), RAID 5, RAID 6, or RAID F1]." or as "Data scrubbing is only supported on [Btrfs volumes or storage pools] of the following RAID types: SHR (consisting of three or more drives), RAID 5, RAID 6, or RAID F1." As a 923+ owner with two drives in SHR, I'm wondering if I need to add at least one more drive to be protected by data scrubbing.
using zfs would be better, but for the normal user zfs as it's disadvantage too when it comes to expending a pool . I've had a QNAP nas completely die on me that the only way was remove the disks and get the same model to put the disks in , which did work and get all my data back.
@@nadtz looking at a blog post from arstechnica dated 6/15/2021 , "ZFS fans, rejoice-RAIDz expansion will be a thing very soon" , So not holding my breath on that one.
The data scrubbing only works with raid 5-6 and an another one and of course shr The Raid 1 doesn’t work because you need a checksum and you need 3 disks
On Synology it runs only the btrfs scrub part (you need to run about 2-4 scrubs before it checks both half's of the mirror) Not watched all the video yet to see if he glosses over the fact that Synology still does not enable checksum by default when you're creating the share folders you have to tick the box all the data scrub With it not ticked it doesn't do anything apart from check the metadata (and sync the raid if you have 3 or more drives, there is no scrub for raid it just syncs the data to the parity even if the data is corrupted, it's why it runs btrfs scrub first)
Real shame that Synology forces customers buy their stupid branded label drives for larger systems like the 12 bay desktop units. I don't buy their explanations why this is supposedly a good idea. They just want to make a profit selling drives. Currently looking for alternatives to Synology.
Massive shout out to Dalton for his test!: daltondur.st/syno_btrfs_1/
This is the reason I stick with Synology. When people complain about their sub-par hardware and weird hdd compatibility, my primary data is just too important and it is always my first priority that my family photos and such are kept safe. When my family put their trust in me with their data... I don't mess around, and neither should Synology. Good work.
SpaceRex keeps his checksums in order. A periodic yet continuous incorruptible datastream of useful Synology advice. Ought to set up that backup soon...
Your comments on the kids knocking your NAS off of the shelf brings back a rather horrifying image to me. 3 years ago, I was undergoing a major home renovation. My Synology DS920+ was on its shelf in the basement with all of my other network equipment and panel. I didn't realize how close the contractors were to this with their equipment. The next thing I hear is someone yelling my name and to come downstairs. I see that my 920+ has fallen 3 feet off of its shelf and 3 of the 4 drives have spilled out onto the dusty floor. I picked up the pieces, used my compressor to blow out the dust, reassemble and reboot ( in a safer location ) and it came online like nothing even happened. I don't recommend this as a way to physically collision test your NAS but it does say something about them being not quite as fragile as we might think.
Dang, just reading that gave me shivers 😬
True story: I have a DS1821+ that I initially set up with three 20TB drives using BTFS SHR. I decided, later, that I wanted to add a fourth 20TB drive. So, after physically adding the drive, I had the Synology add it to the volume, but this process takes days. When it got to about 40% we had a storm come through and the power went out for about 1 minute. When the Synology rebooted, the volume process picked up right where it left off with absolutely zero data corruption. I thought, for sure, my volume and data would be hosed, but instead, I was amazed.
Dang that’s lucky. I have an ups that can run my nas for about an hour in the event of a power outage because I’m paranoid of data corruption if my nas had a hard shutdown from power failure. Even though it can run for an hour I have it set to just auto shutdown if it switches to ups power. Anything im doing can wait til the power is back on.
First time I have an issue with my NAS and my volume is in critical state (1 HDD dead ; 2 HDDs filesystem error)… Fortunately Synology support is wonderful and we are recovering partially my data through BTRFS recovery. Still crossing my fingers :)
Best part is that I just bought another Synology to replicate my data. I was about to receive HDDs when the disaster happened.
I'm still wary of BTRFS after having some bad experiences with it in the past but it's good to know it's gotten better since then. And yeah, always have a backup of anything critical.
Same here. I'm not going to migrate into btrfs. I'll stick with ext4
SpaceRex, you always tell us to backup our data, and most of us (at least I) do it. I use HyperBack-up to backup the Synology NAS to a remote QNAP NAS and also to an on-site external HDD (again, with HyperBackup). What some of us don't actually know, is how can we recover that data if the Synology NAS breaks? Lets say a kid pushes it over and everything is shattered. I assume I have to buy another Synology NAS, but should I buy the same HDDs, or can I have bigger ones or other RAID configuration? And then, how do I actually recover the data (from the external HDD or from the remote NAS)? I would highly appreciate a tutorial, or at least a reply to my comment if you think that is enough, but will be really helpful. Love what you are doing! Keep up the good work!
Ooo, fascinating topic. I always strived for a system with btrfs which my old synologies didnt support. Now that I got a more powerful one, I enabled btrfs right away
Good timing... i uninstalled a synology package and hit delete all associated user data. It wiped my nas. I am using recovery tools now.
I always wondered what the data scrubbing info I get was about, but not enough to research it in any way - thanks for the explanation.
Very informative as usual. Keep up the great work!
A NAS is not inherently a backup. A RAID is not either. It ONLY protects you against a single HD failure... That is all.
The 3,2,1 rule is the bare minimum for data that is important.
For my important stuff I keep 5 copies. you might think that is over kill.... Until a storm blows out your electrical killing all your electronics and a subsequent flood destroys your backup.
you make us smarter, thank you : )
Checksum and scrubbing is probably the best thing you can do to maintain data integrity. It's good that Synology have an automated process for data scrubbing, but you can also do the same using other open source tools such as cshatag, which runs checksum on all files and store the checksum on the file's tag metadata so that it can compared later. There are lots of other checksum tools, but a NAS RAID on its own is not designed to protect data integrity (it's designed for speed and cheap mass storage). To protect data integrity, you need frequent checksum and frequent backups.
Can you make a video about bit rot
bit rot is data very slowly eroding. I personally believe most of the eroding is from environmental background radiation. I used to have my NAS up high on a shelf, and over the years, I would see sectors errors across all drives appearing and slowly getting worse. But in the last 6 years, I put my servers near the ground inside a makeshift faraday cage and have yet to see a single sector fail.
Hard-drives only notice bad data when there comes a time it wants to read that section, and mirroring drives only works if you schedule "Data Scrubbing". I setup data scrubbing every 6 months on all my servers, because its actually quite intensive, doing data scrubbing too often could have a negative effect.
One question... I didn't know that my DS420+ does not support ECC when i bought it. In terms of data integrity, how much am I missing out?
@spacerex Can you PLEASE do a video on enabling data checksum for the HOMES folder? Seems to be impossible??? Yet so much important data is stored there, like Synology photos? Thanks!
Any chance there's a similar capability for Windows? My backup is a cheap 4 bay USB RAID enclosure attached to a Windows PC. It would suck to find your backup had files on it that deteriorated over time.
So SHR1 is still risky ? Cause I only use two drives in mirror + btrfs. Should I add another or is this an unlikely scenario ?
great video as always. just curious where did you find that keyboard whist support that holds your keyboard? maybe there is one for the Logitech mx mini.
Thanks for this 🙏
Ideally, i like to start seeing single drive NAS's but sold in two's, both acting as a single server but continues functioning normally when one of them goes offline. When back online, automatically catching up/resyncing. I know it can be done already, but is a headache, wish there was more seamless solutions.
Synology has this exactly, with Synology high availability
But how come in a 2bay nas you can activate data scrubbing but in Synology's website says it needs raid 5 thus a 3bay? So confusing
With BTRFS you can
@@SpaceRexWill Could you go into more detail on this? The DSM 7.2 KB page on data scrubbing says: "Data scrubbing is only supported on Btrfs volumes or storage pools of the following RAID types: SHR (consisting of three or more drives), RAID 5, RAID 6, or RAID F1." This leaves me wondering whether this should be interpreted as "Data scrubbing is only supported on [Btrfs volumes] or [storage pools of the following RAID types: SHR (consisting of three or more drives), RAID 5, RAID 6, or RAID F1]." or as "Data scrubbing is only supported on [Btrfs volumes or storage pools] of the following RAID types: SHR (consisting of three or more drives), RAID 5, RAID 6, or RAID F1." As a 923+ owner with two drives in SHR, I'm wondering if I need to add at least one more drive to be protected by data scrubbing.
Makes me wonder why I even bother with RAID when a simple power failure can corrupt hundreds or thousands of files anyway.
using zfs would be better, but for the normal user zfs as it's disadvantage too when it comes to expending a pool .
I've had a QNAP nas completely die on me that the only way was remove the disks and get the same model to put the disks in , which did work and get all my data back.
Single drive expansion is *finally* coming to ZFS. It's taken forever but it's going to be huge once it trickles into Linux/FreeBSD/Truenas/Unraid.
@@nadtz looking at a blog post from arstechnica dated 6/15/2021 , "ZFS fans, rejoice-RAIDz expansion will be a thing very soon" , So not holding my breath on that one.
The data scrubbing only works with raid 5-6 and an another one and of course shr The Raid 1 doesn’t work because you need a checksum and you need 3 disks
On Synology it runs only the btrfs scrub part (you need to run about 2-4 scrubs before it checks both half's of the mirror)
Not watched all the video yet to see if he glosses over the fact that Synology still does not enable checksum by default when you're creating the share folders you have to tick the box all the data scrub
With it not ticked it doesn't do anything apart from check the metadata (and sync the raid if you have 3 or more drives, there is no scrub for raid it just syncs the data to the parity even if the data is corrupted, it's why it runs btrfs scrub first)
when i hear BTRFS only one thing comes in my mind - "buggy" :D everything else after that does not matter
Real shame that Synology forces customers buy their stupid branded label drives for larger systems like the 12 bay desktop units. I don't buy their explanations why this is supposedly a good idea. They just want to make a profit selling drives. Currently looking for alternatives to Synology.
16:35 Yeah! Packet loss and I still see your uncorruptible face
first
This guy does not understand ext4. What has been said about it is not accurate. Disappointing.