NAS Storage for Photos & Video: PROTECT YOUR DATA!
ฝัง
- เผยแพร่เมื่อ 7 มิ.ย. 2024
- Use this link to get $25 off your PPA membership www.ppa.com/join/tony-chelsea...
Tony Northrup (a former IT nerd and storage expert) teaches you how to properly backup your photos and videos - because your external USB drive IS NOT ENOUGH! It's a good start, but it doesn't protect your photos and videos in the event of natural disasters (fire, flood) or theft. It also won't protect you from malware and ransomware.
He discusses cloud storage solutions, NAS (Network Attached Storage), including a description of his geographically distributed Synology DS1817+ configuration with over 400TB of storage in two different locations.
Tony also describes bitrot, which causes old files to gradually degrade even if they are backed up. - วิทยาศาสตร์และเทคโนโลยี
TELL ME YOUR STORAGE HORROR STORY! And use this link to get $25 off your PPA membership www.ppa.com/join/tony-chelsea-northrup/mar2022
True story: Adobe Lightroom mobile early days, circa 2015ish they lost my entire online catalog (which was relatively small at that time, only around 800 raw files + edits). I was a paying member, was in between main desktop catalog machines and ADOBE had data loss with NO enterprise recovery or backup abilities at the time! All I got was a slew of uninformed tech support reps on the phone and a few very apologetic emails.
I've run a synology nas for years......might be time to double check it's doing what it should be doing :)
ALWAYS at least 3 different back-ups on 3 different types of harddrives/SSDs, NEVER use the same type of drives ! (a fault on a drive will be on ALL drives if it the same type !), so buy 3 different drives at 3 different times to be shure. ALWAYS check each back-up and NEVER delete/format the memory cards before you are sure you have 3 different back-ups that work well ! All drives must be offline when not in use ! When you do back-up, turn the internet OFF ! Do not use "NAS", if you do use 3 different types.
A mirrored zfs array will automatically heal those corrupted files every time the file doesn't match the checksum.
Just came across a picture today that had bitrot. Luckily the copy on my desktop was still in good shape.
I have been an IT person for over 20 years. I shot my first wedding for a coworker 7 years ago. I used a Canon 5D with 2 memory cards. After the wedding I backed up all my memory cards to my computer and then to an external hard drive. The next morning I left on vacation to go camping. I received a call from my neighbor at 2:00 am telling me that my house was on fire and the fire department was cutting a hole in my roof to let the smoke out.
They busted out all my windows and threw all my computers and electronics out of the windows. When i got home my Mac was sitting on a pile of debris with water dripping out of it. My external hard drive and memory cards were in another soaking wet pile. I was devastated. I lost everything I owned except for the clothes I took on vacation. I found all the memory cards and my 2 hard drives while raking through the debris. I put them in a bag of rice for a week. I had to call my coworker and tell her that all of her wedding pictures were in the house during the fire.
After a week i connected the hard drives to a USB device and amazingly I was able to get 100% of my data. Even the memory cards still worked. I learned a huge lesson that I should have been following, being an IT person for over 20 years! Get those files offsite ASAP. The 3-2-1 backup strategy simply states that you should have 3 copies of your data (your production data and 2 backup copies) on two different media with one copy off-site for disaster recovery.
OMG. How did the fire start?
@@saganandroid4175 not sure. Undetermined.
Oh man have I been there done that! Replicated corrupted data is a nightmare. Offline solid state back ups plus data integrity checked Time Machine backups, plus cloud galleries that are unsynced and static. Long term data management is a deep rabbit hole. I so feel your pain TonyTone!
Think you mean offline as in not internet connected? Having the the backup air gaped (no internet access) is a good idea.
But if you leave a SSD unplugged for long enough the taped electrons that makes a "1" in solid-state storage will leak over time and flipping that bit to a "0".
How long that takes varies wildly. and i would guess older SSDs are likely to suffer from this than newer ones.
HDD, DVD / blu-Ray have a not so good "shelf life". The reason why tape-drives are still a thing for on the shelf data storage.
Once a year , I go through all my kids pictures on my drives and make an old fashioned hard copy photo book and have backed up pictures on different drives - Thanks for this video lost memories sucks - I am in my 50’s and getting to the age I think more about memories and not the things we have acquired - When we are on our death bed I want to remember those moments while having headphones listening to music - life is all about memories with your family friends and moments -
This is such a good summing up of the situation. I have made 5 family albums but discovered some corrupted images in the process, which is incredibly sad. I laid out the albums in Affinity Publisher, exported the high res PDFs as well as email quality, and distributed these to family who can decide to print from the high res versions if they want. But after watching this, I realise even more the need to back up twice, in different locations. Thank you Tony!
Great video, Tony - thank you very much! So sorry for your loss (we felt your pain). In a seminar years ago, an IT expert advised about backing up digital media - get in the habit (at least annually) of migrating your data to the next-best most secure storage available and have an off-site location. ChronoSync software really helps as you syncronize your drives. Your recommended use of battery backup power conditioners is spot on. I use 5 (from APC). Electrical spikes and sags can wreak havoc on sensitive micro electronics. The videos you and Chelsea produce are great!
In my early days of selling computer stuff, I heard and remembered a quote "There's data you've backed up, and data you haven't lost yet". I try to backup my photos on my desktop, laptop, wife's laptop, Google Photos, and photobooks I've created and had printed. Only saving grace in my case is my total storage is under 1TB total. Oh, and when I worked at an actual office vs home, I kept a backup drive there too.
Anyway, great video. Thanks very much. As for a (lame sauce) horror story, I lost the photos I'd taken for my brother-in-law's wedding a few years after the event, and it was maybe another decade later my brother-in-law found the CD backup I'd made for him at the time and I was able to get the photos back.
So sorry for this event! Thank you for sharing with everyone!
it is quite remarkable how often your photos get corrupted, especially on external drives. I had corrupted files very soon after copying them on the drive. Once I even discovered that whole directories just disappeared
Oh damn I was thinking about my backups and boom your video pops up!
Same here!
Great info, as always, Tony. I think all of us - pro or enthusiast, must find the best way (that suits the individual) to address backup, recovery, and loss. I don't think there's any one way that's the correct way. Each photographer should find what works best -- from a perspective of value, cost, time, and importance.
I'm not a fan of cloud storage -- but I'm only an enthusiast. For me, it's simple -- I don't store any photos on my desktop (or laptop) PC. After every shoot, I copy the contents of the memory cards to an external drive (connected to my desktop). Note - If I'm in the field for an extended shoot, I bring an external drive and copy the contents of the days shoot to that external drive, connected to my laptop, each evening. The external drive is backed up to my local Synology DS 920+, and a make a backup of the external to a 2nd external (then store the 2nd external offsite). At this point I have 3 sources of the photos (not counting the memory cards). I don't format the memory cards until I've confirmed the files on the Synology and the 2 external drives.
I do all Lr/Ps post processing from the desktop-attached external drive -- not the fastest method, but then again, I'm not a fast editor. And, I replace the external drives every 3 years. Is it perfect? Nope - not even close. Do I sleep well? As well as one can, I suppose. I end up with my complete NAS backup - original and edited files; another copy of the edited and original files on the desktop-connected external drive; and a copy of just the original raw files on the non-connected external drive.
For me - this sysem works.
Like most of us, I don't go back and view the hundreds of thousands of old images on my drives, with any regularity. I just don't have the time. But I hope that keeping my external drives fresh, on a 3 year rotation minimizes the chances of storing corrupted files.
Thanks again for the insight, detail, and passion. Truly appreciated.
This is why I got into Freenas (now truenas) which uses ZFS which uses scrubbing to protect against bit rot.
Had a programming teacher who said, “Blessed are those who make backups of backup”. I would always have a running backup and one stored in the bank vault. Macs use a concurrent backup. But to be safe, have a separate folder the Mac never uses. So, basically you have a concurrent backup on the backup drive and a separate folder that contains the photos. (X2). Much harder if you take thousands of photos weekly but it that’s your income, then invest. Memory is cheap cheap cheap compared to 20 years ago.
Also IT background. I've been running a ZFS storage array with two drive redundancy (RAIDZ2) for years now and it has been rock solid. I run pool scrubbing monthly and it has from time to time detected and corrected bit rot. One of the Amazon services I've been taking advantage of is free photo storage for Amazon Prime customers. This includes RAW photos too. My workflow works something along the lines of: Import from SD to laptop. The Amazon Photos app runs in background monitoring my pictures folder. I also sync to my ZFS NAS. On another system, I have the Amazon Photos app download to another system. I think run a hash calculation on both the NAS and the downloaded photos. When cleaning an SD card, I hash those files and if that hash exists on BOTH Amazon and NAS I'll delete just that file.
I don't know enough about jpeg compression to really understand, but since it looks like the corruption on the images "starts" some ways down the images, I'd be really curious if there was some way to go into the image and manually start flipping bits in the data associated with the specific pixel it "starts" at (assuming that can be identified), and see if it can be fixed.
I'd also be curious if raw files, presumably being uncompressed, would be affected the same way, or if a flipped bit would instead just appear as a messed up single pixel. (And if so, maybe compressed raw would be affected though)
No, that's not what's happening here. Jpegs use transformation matrices and coordinate transformations. So, if you lose a bit of that coordinate matrix, you're screwed...sort of. What's happening here is that the magnetic strength of the stored data is not readable using the conventional mechanics of the arm in the HDD. He needs to use the software that I used to recover my files. That software will cause the drive to read files backward and sort of simulating what you were hinting at but not quite. It should work on most of the files. There may be a couple that are really that bad where nothing will work but the software won't freeze and continue trying to recover every file.
Some JPEG recovery programs can get most of the file back. I have only seen that damage on JPEG files for which there was no RAW copy but I had analogue originals.
Not a single file is safe from what tony is saying. Pretty much everything gets corrupted over time. Same with RAW files. At the end of the day it's a matter of luck. If in jpeg a byte is changed in specific area it could potentially ruin whole image or just a single pixel, really depends on where that is going to happen. But where it's going to happen is unpredictable. The HDD's looses magnetism and SSD's looses charge so not a single storage is safe over time. Best thing to do is probably cloud storage. Also Tony has mentioned making videos with slides of images which is also a pretty good idea since to corrupt a video file it would require much more effort than with JPEG's or RAW files. In general video files are much more robust to glitches. Even if glitch would happen during the video it would still continue to play and recover at some part where the image is still good.
His is examples use JPEGs but this can also happen to RAWs and most other files. JPEGs were probably used for this example as most apps can display them and show the impacts (bitrot, or corruption) he was trying to show. But all files of any type are succeptible to this.
@@PaganiTypeR what about using .par files when making backups? I remember those from the old Usenet days. Random parts of the file would get corrupted, so redundant information was sent so that the file could be repaired.
Thank you for sharing, great suggestions. I am 65 and have been preparing negatives, slides, prints, (to digital) to be passed on to the next generation. As they are backed up, clouded, ... I need to make sure my kids have the codes and directions to all the files...thank you for motivation to take care of business.
I back up only new files or files where the date or size has changed. That means an error inside the file will not be duplicated to the backups.
Wise words from a wise man. Thank you for all the videos you've made over the years, I just started about a year ago with the DSLR and can't imagine how difficult it would have been for me to piece together all the information that you put in your videos.
Thank you.
Excellent video that takes the world of IT and crosses it with the wonderful world of photography. I currently use the Adobe cloud to backup pictures and Acronis cloud to backup machines along with my own Synology NAS which has pictures. I don't have anything close to what you have in terms of TBs, but even 1 photo is worth a lot.
Spot on Tony, and timely too. Only two weeks ago I had a massive storm where a very close lightning strike meant we lost power for just over two days, and flooding caused all sorts of other problems. I have at least one UPS on almost all my pc and tv equipment, but I was only using it for power, not for the network cables, so the lightning strike ended up frying my WAN port on the router (connected from the modem by a long network cable, which picked up the lightning RF), so once I got power back it took a while to work out the issue. One very scary part was that without the router, I couldn't know if I lost my 12TB NAS box, as it was hard to get it to boot, and I didn't know if I lost it all for ages. I'm currently organising my online backup, which will be terribly expensive, but I can't afford to lose it all. Your bitrot reminder is very scary though!
Thanks for the reminder and sorry abut the lost files. I set my Lumix G9 to record to both SD slots and save the files to two external HDDs attached to a desktop PC. Cloud storage would be a good tertiary option, but I could also periodically save to a third external HDD that I stored in another location (I'm not dealing with quite the volume you are). Years ago my travelling companion and I had all of our camera gear and over a dozen rolls of exposed but undeveloped film from a trip to Utah and Colorado stolen (apparently by a parking valet at a high-end hotel in Denver, but the issue was never resolved). It still bothers me so I can imagine how frustrated you feel, but at least we have the memories (but of course we still want the images)!
One of the main reasons I'm using ZFS for storage at the moment, with scrubs every other month to check everything for corruption and to correct it if needed.
With spinning disks I've found that basically since around the time they introduced shingled magnetic recording reliability has gone downhill (even on non SMR disks).
I have old 1-2 TB drives that have been running for years and return less corrupted data than brand new drives of similar size.
Using a filesystem that can recover corruption combined with regular scrubbing of the disks, and to basically not sync files older than a few weeks from your PC (that probably uses a filesystem that won't catch corruption) to the storage server anymore really helps prevent issues for me.
Also I don't know the current state, but iirc btrfs did have issues with raid5 or 6 type usage when I last looked into it allowing data to get corrupted.
Also, don't have the backup server "mounted" on your system as network drive or network source by default, set the server up to initiate the connection to your PC to get the data. That really reduces the chances of ransomware getting to the data
@@TDCIYB77 It kinda makes me wonder why Synology would even want you to use BTRFS for archiving. (I don't know their products from experience, but I assume they will have proper documentation for filesystem choices).
Ive been eyeing BTRFS for ages as replacement for my own home server/NAS, specially because it allows you to spread data properly over existing striped disks when you add another one. something ZFS doesn't allow. (last time I checked anyway). But iirc it was even on their own wiki that RAID-5/6 like setups were called "experimental" (or something along those lines). So NAS brands allowing usage of it without clear warnings about hat is kinda shocking to me.
Funny that you made this video after I had an issue with memory cards. Fortunately I didn't lose any pictures. My issue was that I started out shooting on a Canon APS-C camera and of course those only have 1 memory card slot. Last year I bought an R6. Of course those have 2 card slots. I use 2 Sony Tough cards in it. Last Saturday I was shooting and my camera stopped taking pictures. I thought one of the card became corrupt. I was smart enough to have listened to you Tony about keeping more cards handy. So I swapped the cards out and kept shooting. I got home and checked the cards. Come to find out my second Tough card was full! I had never deleted the old photos from it because I didn't realize I had to switch between cards to delete everything. Luckily I have everything backed up on an external HDD and 2 cloud serviced. So no more lost pictures. Thank you Tony and Chelsea for drilling the 'carrying the extra cards' into me all these years!!
Excellent video, Tony, thank you for the tips!
Sorry to hear, like you its just happened to me. 20 years of work and memories. I will now place in multi locations. thanks for a great video
Tony thank you for sharing this story and sorry you lost precious memories. I burned out my Dell XPS hard drive and home media server. Took to PC Fix who recovered almost all images. Upgraded PC memory and new WD EX4100 16TB NAS server. Use Amazon cloud backups. Bit behind on memory books but now motivated to continue from your story.
Great video, thanks!! With your RAID (and hopefully file-checksums), how often do you schedule data-scrubbing? I still hope that with Synology's data-scrubbing+checksums, the bit-rot problems might at least get recognized before I overwrite my backups. But I'm interested if you think monthly/quarterly scrubbing is enough. Thx
Hi Tony: I had an external HD become corrupt this past summer and because I back-up each night to a second locate drive it also became corrupt. As a result I couldn't access my LrC Catalogue or images. I never let my HD capacity go over 80% full so this wasn't the issue. We had some power outages over the summer and I think that these events contributed to my issues. Fortunately, I also store all of my images to Back Blaze with the 1-year version option selected so I had them send me a new HD with my 235,000 images on it. I dodged a bullet but I am still very careful when I look at them back-up strategy. Great PSA video. Everyone should be mindful of this topic. It is not a matter of if my HD fails, it is a matter of when. Cheers, Keith
I haven't had a hard drive failure in quite a while (knocking furiously on my wooden desktop). I have had memory cards fail, but not with anything on them (again - knock wood). They just suddenly couldn't be recognized by the equipment or computer. I do make redundant backups of my memory cards as part of my work flow - a back up to external USB drives, a backup in the cloud, and two copies on my working drive in separate folders and working only from one of those folders. Memory cards are too expensive for me to keep around, so the handful that I have get reformatted between projects once all the back ups are done.
So sorry this happened to you. Thank you for teaching.
Automated backups are good for short term (1 or so years), and decent manual backups to bare drives are good for important older data. I store those bare drives in a fire/water resistant safe. For short term backups, a DIY NAS setup, will have options for periodic automated scans for corruption, especially for many ZFS setups.
On my PC, I have 6 hard drives, 2 M.2 SSDs, and 2 SATA SSDs. Since I also backup to a NAS, I use more of a JBOD setup on the PC. Then for data that is not easy to replace, I keep manual bare drive backups, which are easy to do with a USB 3.1 SATA dock.
For automated backups, make sure you have versioning enabled, and have it for bulk media and basic documents, have a separate backup pool that enforces versioning without deleting older versions. Since they are unlikely to be changed much, it will not cause space issues.
Use a NAS that uses ZFS as a file system, since zfs has built-in bit rot and data integrity features. We build large enterprise storage SANs with sensitive data, all built on ZFS. In 18+ years haven’t had a single corruption issue with high io storage devices. I don’t remember the last time I had to do data recovery.
Yes, but make sure to use a mirror or similar as well as an online backup. A mirror allows for healing the rot.
Modern filesystems that protect against bit rot and data integrity are definitely the way to go. I'm building a new setup with two NAS-es now (haven't had any data loss in 20 years, but trying to beat my bad luck here). Primary NAS will be TrueNAS Core running ZFS. Secondary NAS will be Synology running BTRFS. I also have backup in the cloud for everything except video (too expensive and slow to backup and restore).
I'll also use snapshots at least for TrueNAS/ZFS, and I'm hoping to do this with BTRFS on Synology too. Snapshots every hour for a day, every day for two weeks, etc.
Even so, I'm still constantly worried about bit rot or crypto virus. I'd like to incorporate some form of checking integrity automatically, perhaps using hashing, but I'm not really sure about the best way to go about this...
Maybe if there were good ways to declare parts of backups immutable, in order to prevent them from being overwritten...?
Specifically make sure ZFS scrubbing is setup to repair the rot. Just having RAIDZ, etc. is not enough as it won't actively check for errors until something is accessed.
Linus Tech Tips just had this exact issue when they weren't actively checking their old data.
Tony said, that he's using Btrfs for his backup system, though?
Currently I just have external harddrive, now I need to rethink of my backup storage. Thank you for this informative video.
Last year I lost a 1TB drive with lots of media on it. I don't yet go to the extreme of off-site storage or fire/waterproof safes, but I decided to make multiple copies of files past a certain age and spread them across various drives (Laptop, NAS, Desktop, Google Photos and older drives that I leave unplugged in a closet). Since HDDs always get larger, every time I upgrade, I make a new copy from a verified-good backup. It's pretty manual, but it hopefully prevents the automatic backup of corrupted files.
man so sorry to hear this Tony. You've really got me thinking about this ....
I'd like to start doing annual photo books this year to counteract this. Even with triple back-up updated regularly, I have no trust in digital storage anymore, and physical media is much more gratifying to go through. I used to do individual prints of my best photos, but I have so many envelopes of prints that photo books seem like a better option in the future.
No one will go through your hard drives when you die. They'll be happy to leaf through a photo book, though.
That is true. You can even print a few and give them to friends and family in the event you lose yours.
Excellent comment Christian! And like my reply to Harry, will there be available, devices to be able to read today’s digital media, by tomorrow’s generation?
This is a great idea, and one I definitely subscribe to. Love having 'coffee table' books people will look at, that I can also put other info in, like family tree or history/stories. And I can revise and get it reprinted, and give away the old copy.
" *No one will go through your hard drives when you die. They'll be happy to leaf through a photo book, though* "
OH MY GOSH, so true!
Great tutorial on backups. Although I do multiple drive backups, I do plan to get a small Synology system. I also do cloud backup on Google Drive for client files. I don't have as much as data as Tony and Chelsea but still enough for multiple TB backups. For important shoots, I keep multiple SDs with original files of images until backups are done before re-formating SDs. Sandisk Pro SDs are cheap at 32GB and 64GB so I buy enough. The more SDs you have, the less each is over written over time which increases life of SD. I have had a few corrupted SDs before.
What a nightmare. I'm so sorry this happened to you guys. I use a similar backup system to yours: multiple Synology DiskStations, BTRFS, SHR-2, multiple locations. I don't think most people appreciate the importance of using enterprise-grade drives like the WD (HGST) UltraStar models. The error rate on them is exponentially lower than consumer drives. Instead of the expansion units, I use three DS1821+ units which I regularly sync to each other using GoodSync with checksums to ensure data integrity. I physically transport one of the DS1821+ units back and forth to a second location to sync with a third DS1821+ to keep the units air-gapped. Whenever I replace the drives in a DiskStation, I take the old drives and copy data onto them with the intention that they will never be touched again except in case of emergency. If I had unlimited funds, I'd set up snapshot replication...but I don't.
If I dealt with as much data as you, I'd also look at LTO tape. The tapes are surprisingly archival. The drives are expensive but the tapes aren't bad. They're ransomware proof too.
Thank you for this video. You did a great job demonstrating that even if you practice the 3-2-1 method of data backup, you can still have bad data replicating through your backup system. Did you ever figure out where the source of the corruption occurred? Did it happen long ago when you put photos on external HDD's? Then when you upgraded to a more sophisticated NAS you were just copying bad data? Is there a way to review all your pictures in bulk to make sure that they retain their original data integrity? I am naive to this, but I would think that each file would have a CRC check or something equivalent that would alert you that the data is corrupted.
I‘ve lost a film roll from the wedding of my very best friends, who relied on me taking good photos from that event-I absolutely know what you are talking about. I‘ve also seen several hard drives suddenly crash. It’s not the question if such things happen, it’s only about when-and you’ll never know that in advance. By the way, having one redundant backup system is not enough. I have seen two redundant raid-5 systems electronically fail in two days, so before the first of them was repaired … 1 TB of photos needed te be recovered from tape.
Film photographers also would send half the film from one of the two cameras to be processed, wait for the results before sending the remainder of film. Also one of my instructors would process one sheet of 4x5 film at the beginning of his shoot before continuing to shoot.
Did this always for commercial shoots.
I used to send my film from 2 different cameras to 2 different film processing houses.
This was very comprehensive. Excellent content.
Thanks Tony for the wake-up call. I had never heard of bit rot.
Very goodmorning Tony& Chelsea ..i used to watch your videos , they are really helping me to improve my photography skills ..thanks for that .... recently ibought a 5d mark4 ( used one ) ..i have 6d mark 1 too ..even though 6d is fantastic camera the focusing abilities are not that good ..thats y i bough 5d4, while comparing both 5d mark4 and 6d raw files (13,1/125,iso2500)..i noticed my 5d files are noisy compred to 6d ...am i missing some thing in 5d4 settings ?.... 6d image quality is better than 5d4 ..? or due to over usage of sensor my 5d4 lost its image quality ?..kindly help me ....
Hi Tony, Great topic and probably overlooked 99% of the time. Suggestion, get a nice computer rack to house all your network equipment. You always do everything first class.
You've convinced me to move a few hundred GB of photos onto my one drive 365 subscription considering I pay for it every year, So far I've only had a handful of individually corrupt images over the years
Extremely helpful video, thanks!
In synology btrfs file system, did you enable “data scrubbing” which will check the data integrity occasionally?
I'm sorry to hear about that! Data loss is a real nightmare. I feel the urge to check on my files asap... After my NAS and the cloud, my third way to backup photos is actually Blu-Ray discs! Optical media rated for a 100 years. If my files are intact at the point I burn them to the disc, bitrot is rather unlikely. I keep the discs in my basement which is basically a bunker, so they are relatively safe from fire or other disasters. Of course this is only for long term storage and not for files I use regularly.
In some ways, that is where old fashioned film photography is superior to digital. I still have a very thick album full of negatives from 30 years ago still intact. Nothing beats making prints, and keep them somewhere safe. Favorite past time might just be having to look for that one photo, only to be flooded by the many memories from THAT album you look through every now and then.
Very good advice there Tony, also coming from an IT rich background I’ve seen plenty of the old writable CD and DVD’s turn to custard after a few years , very dependant on brand. Ironically the slides my folks took in the 1950’s of us kids still look good so long as all those other catastrophes you mention don’t befall them. Fully agree that as you get older and people pass away it is great to be able to look back. Cheers Chris from NZ
Optical media just sucks as a backup medium. Apart from preventing deletions and modifications, it just doesn't do anything compelling and is a massive pain in the backside to check.
Best thing is a hard disk based backup system with online backup and automated file verification. Preferably using a filesystem that can detect and repair bitrot, but as long as you detect it while you still have a good copy, that's OK.
@@SmallSpoonBrigade agree entirely, I haven’t backed up using CD’s since usb drives were affordable. These CD’s were backups we made in the office in the 90’ s and claimed to be “archival” 😀😀 our first Pc with a ten mb hard drive for my engineering team plus a dot matrix printer, green mono screen, plus a pen plotter in 1985 cost $47,500 in todays money ($33,000 US) I think my toaster has more computing power and a single image from my mirrorless camera would fill the hard drive five times over and yet back then we though we would never fill it. In the mid 90’s ram for my 486 was $100 a megabyte !!!
That is why I still use slide film for important photos. I’ve lost too many digital files and almost no slides.
@@trevorsowers2202 Interesting approach and of course I’ve still got my parents Kodachrome ll slides from the 1950’s still in great condition. I did have a complete failure with Agfa slide film I used in Australia in 1980 much of it now has an orange tinge. Of course easy enough to scan all that old stuff. Printing is my long term archiving option plus photo-books of course. Not sure I could go back to film after spending 35 years in that medium but you are correct in thinking long term digital storage no matter how carefully you are hasi it’s pitfalls and I wonder what will become of all the images taken by todays youth taken on phones when they want to show the grand kids in years to come. ☹️
I do have a mistrust of online services and there subscription models that’s for sure.
@@Chris-NZ some films are more stable than others for sure. Kodachrome was very stable but early E6 slides were not great but the later E6 is great. I have lots of Ektachrome that is 30+ years old and looks as good as new. I’m currently using Ektachrome E100 and I’m very satisfied with that film. I use a EOS 1V and a 5D MKIII so I can carry one set of lenses and speedlites and shoot digital or film as the situation warrants but the more important photos are usually done with Ektachrome as I have more faith in that medium
I'm old so I can speak from personal experience to back in the film days when we used to have two photographers to cover any wedding giving us multiple views but also two rolls of film from two different cameras cause not only were there film process failures but camera failures on site happened as well. We had a rule of backups of equipment in extra flashes and camera bodies for each photographer and backups for film with multiple camera backs, backups were always needed and are needed now more than ever. Oh and also one other point. If we lost a roll of film due to processing we only lost 15 frames (120 format film) out of the hundreds that were shot on a full day, not at all like the unimaginable risk of a single 256mb flash memory card with the complete day on it.
Thanks Tony, a nightmare indeed. I have two external hard drives and two SSD drives backed up with the same content. Should I leave my external disk drives unplugged from the computer until needed or have power to them and unplug USB cables to protect against Ransome ware?
Sorry for your loss. A hard lesson to learn.
Thanks for the video Tony!
But I think you did not tell us how someone can prevent corrupted stuff from being backed up and also corrupting the backups.
Agree ... with all of the backup processes and equipment you have in place ... how did this happen? We could all really use a more detailed assessment. Just got thru doing archive scans of 3000+ slides and tossed the originals. Do agree about printing ... printing ... printing
Fantastic content! Professional Photographers of Washington just published an article I wrote about "Digital Decay." With the usage of cellphones and reliance on social media for photos, I strongly encourange people to print their pix and backup. I dropped a 4Tb hard drive--bye bye to all client pictures prior 2017! I now backup to 2 external harddrives. A lesson was learned!!
Ha! One of my cats recently knocked an 18TB hard drive onto the floor. Not all that far, either, it was on top of the PC, and she decided she ought to be on top of the PC instead. It's warm there, but it's the drive that's toast :-( Aside from cat-proofing one's office, this is a good example of not having a single copy of anything.
So sorry to hear, heartbreaking. I use M-Discs as my ultimate archives. Future plans are for a zfs based NAS.
M-Discs are the best stable data backup solution and it’s cheap.
Very good information.
By the way why don't you lock off your focus? Wouldn't it stop the pulsing in your videos?
Good and sound advice. I use DAT (Direct attached storage) which will back up to my on line Backblaze account. For less than $10 a month you can get unlimited storage for one computer and all its directly attached storage. My DAT is a 5 bay Drobo set to double redundancy. That unit is backed up locally to a Sinology NAS on my internal network at the house which also is set to double redundancy. Everything in three places - the two local storage units using RAID set to double redundancy.
Great video.
I have exactly the same setup using two Synology NAS devices (onsite+offsite) sync together and cloud backup to iDrive using Hyperbackup (Versioning enabled).
In addition, my data is physically located on two different continents.
Hi Tony, Thank you for the good information. I do have couple of questions. 1. How do I recognize a faulty memory card? ( I wonder if by checking pictures in my camera as I take them). 2. Can pictures that I have already saved in my pc that are perfectly fine go bad after time?. Thank you.
Thanks Tony for this.
I don't quite understand what the solution is for Bitloss though if backing up alone is not.
Testing not corrupted by opening sample photos in Edit?
Retaining all previous backups as does Time Machine with a big enough hard drive?
Thanks for reminding me - I have bought enough space on G-Drive to at least upload a couple of thousand of my "best" images as RAW - but have to be honest forgot to do it. I will now actually dedicate a couple of hours in the weekend to finally do it! 😊👍
Use mirrored zfs along with online backups.
Really sorry for what happened, Its a real bummer loosing those early pictures. The older the images become the more precious they are. Grandfather, father and son copies, one of them off site is the golden rule of digital backup from my early days in IT. Would advise separating and keeping JPG copies if cost of storage is an issue. Better a lower resolution image than no image.
Under Windows I use the system tool RoboCopy to copy my images from one hard drive to a different one. That does not copy unchanged files, so it does not "copy corruption". Corrupted files usually have same size and time stamp so they are not detected as new.
i must have missed the part where you avoid backing up corruption. does you Synology system do some sort of special verification? if your files are corrupted does it alert you?
For some reason, right from the very beginning I took having a backup routine very seriously, probably to the point of overdoing it, if that's possible. Then again I only have about 15 TB currently. I started with CDs when they were the most affordable solution, then external drives, DVD's, BluRay and cloud services. All of my data is backed up multiple hard drives (on and off site), optical media and the cloud. I haven't lost a file, to my knowledge, in almost 30 years. Hard drives DO fail and optical media does go bad but so having multiple backups is critical. That being said, in all that time, I've only had one or two occasions where I found the back up to be corrupted. Even my oldest optical back ups, while I wouldn't rely on them still seem to be in good shape.
Lost all my son's grad pictures from Simon Fraser University in BC. A suggestion for videos like this which is basically aimed at pros. You mention the one thing I now advise my non-photography friends to do 1) use the picture books offered by Facebook 2) use other picture book services. Make a picture book at the very least every five years for those photos that are important to you. I would suggest your suggestions for the non-pro's appear earlier in the video before you nerd out on equipment. I almost went to sleep when you were showing all your cool gear, I just hung in there to hear the "cheap" suggestions. I bet most non-pro photographers won't make it to the end of this video. But suggestions for non-pro photographers earlier in your video and totally nerd out at the end for your pro subscribers.
So sorry to hear of your data loss, its always the cobblers with no shoes. My Synology NAS suffered a drive failure due to an electrician flipping the wrong breaker. After replacing the bad drive I added a UPS.
Are the any software's that will recognise new unique photos? Also do you have an tips regarding re-writing backup drives from the originals ever so often?
I feel bad for you Tony. You said you cried at the beginning and I believe it. I can still see the hurt on your face even in the video. Thanks for sharing your story and helping us become better photographers.
I some how don't think this was "For real". Obviously someone like Tony and most other enthusiasts would have multiple backups (you really should if you value your files). HOwever it's likely one of his backups was corrupt, but I doubt he lost "everything" he claimed. BUT this story does give good advice that you should check your backups to make sure you're not backing up corrupt files. For me, once a file is backed up, I disconnect the device and leave the cloud backup alone once the file integrity has been verified. No sense in resyncing files that don't change, or you could end up with what Tony has describe if some how your onsite backup gets corrupt and then passes that corrupt data to the cloud backup, which is likely what happens to people.
Thanks Tony but I missed the solution. How do you stop a backup of a corrupt file whether local or cloud based?
Thank you for this great video. I’ve been going to do all this for years but haven’t done it yet. I’ve taken millions of pictures as a hobby and don’t have a backup of most of them. Most of them are only on old SD-cards. I’ve been using them since 2004. Only one of them has been broken so far. First step that I have been trying to do is copy all those SD-cards (maybe 100 cards or so) to my 6 Tb external hard drive that I bought 3 years ago. But it’s still almost empty. It takes about 2 hours to copy one 128 Gt SD-card so it’s just too boring and too much work so I haven’t done it yet. I’d rather walk around in woods with my camera (or cameras) always when I have free time. It’s easy for me to search for wildanimals for 10 hours in forest but difficult to copy files for 2 hours. :)
Some great advice there. You can never have too many backups. I too keep photos on my memory cards until I`m certain they are backed up. The importance of a good power supply is very true. I did lose some images one time, photos of a friend with his classic car which he no longer has. They just disappeared from my hard drive! How I don`t know but fortunately I came across them on an old CD I had backed them up to using my old Amiga computer! Don`t know if those CDs are readable now but at least they are backup up on other devices now. Thank you again for valuable advice.
I came across an old CD burned while I worked at Commodore, back when the "gold" CD blanks cost $50 each and we had a dedicated station for buring CDs. It was still readable, at least five years ago. That's pretty lucky, though if they're kept in a cool, dry place they can last for decades. There are, of course, better options today.
@@DaveHaynie wow that is great to know. So honoured to get a reply from you. Still have my A1200 tower and AmigaOne XE, both work. I kept my CDs out of the light, will have to try them. Thank you.
I’ve had that corruption issue working in music production. We had a producer come over to start working on a song after all the basic tracking was done. Pull it up for listen back and the whole project is corrupted!
Embarrassing and annoying. So we had to record it all again. And that was backed up, but the backup was corrupted as well.
The best advice you have given, fortunately, although I had a laptop that suddenly failed, I was able to remove the hard disk and transfer. Not had any failure of cards, though now I am tempting fate. I always transfer SD cards to iMac or SSD Sandisk drive. Still have thousands of negatives though that are in their protective slide sheets. However, will invest in further back up, i Cloud looks promising. I did know, after being a geek like you, that digital storage can get corrupted over time. All the best, thanks for the reminder! Share you sadness, that is a photographers nightmare, it is like robbed and having all your equipment stolen when insurance could not be afforded. Yep that happens.
I actually discovered some photo corruption when I started using Google Photos to back up the photos on my PC and it complained about some corrupt images that it couldn’t back up. I dug up my backups and they had the same corruption.
Fortunately I had even older backups which were okay. All my photos go to two external drives plus cloud backup. But I have two older external drives that have been filled up and aren’t actively written to anymore. I keep them around for cases like this.
I should probably spin them up more frequently though. Hard drives that are stored for years might succumb to stiction.
Hello Tony .... any suggestions on duplicate image removing software ... thanks
As a fellow nerd who has layers of media, devices, locations providing redundant copies (card, drive, external drive, synology, cloud), the one that still worries me is the bit rot problem. Are you aware of any tools to identify (and perhaps even correct) the development of bitrot? I know that in theory techniques like hashes comparisons can do it. But I'm wondering if there are either file system level tools or automatable utilities that can constantly scan a repository and alert when something unexpectedly changes?
Can you explain again what can be done to check if there are bit rot corrupt files before they get overwritten to the backup ? Many thanks !
Easily the most useful video I’ve ever seen… didn’t know Bit Rot was even a thing.
I have been there. The backup solution I use now is an incremental backup. Only new photos are backed up and are copied to an external drive and a NAS. So if I run in to bitrot, at least one of the two backups should have a clean file since once the photo has been backed up, it doesn’t get copied again on the next backup. After watching this, I am going to ad a cloud backup to the mix also.
A great video. I have 3 independent back ups. I have back up drives connected to my computer all the time. Soon as I get back from a photo shoot I put everything on the external back up drives. I also every few weeks back up those drives to some more drives which are only connected to the computer during back up. I have a bank safe so 1set of drives goes into the safe, another set I have at home for easy access in case of a natural disaster or fire so can grab them easily, if I am here at the time it happens that is. I never store any photos on my computer's they are all on external drives,, not connected to the computer except the one used to initially put the photos on from the cameras memory card. So even if my computer was hacked, broken or even destroyed I would not lose my photos. I also every year or two buy new drives to replace the oldest ones for more peace of mind. I also don't use cloud services as anything connected to the internet has potential to be stolen, no matter how secure they are supposed to be. If you photos are that important do not ever use cloud services.
Tony! So sorry you lost a lot of those photos! Thank you for this video, and I've got to say, you and Chelsea DO NOT AGE! You guys look amazing! Curious question: Which camera did you use to make this video? My assumption is your Sony A1. If I were go guess lens, I'd say 50mm f1.2? I could be totally wrong, but just guessing. Reason I ask is due to the hunting in the AF in sporadic points of your video. I know Sony's AF is amazing, but if I'm wanting to open the aperture to F1.2 on any camera, is the AF performance just destined to struggle? Also, I wonder if the bright background had anything to do with it? I'm so curious! I'll run tests with my R5 to see if the same thing happens. Thank you again for the video!
I came out OK, but despite living in the burbs, out of flood, earthquake, and forest fire zones an urban wildfire blew up here (Louisville, CO) at the end of 2021 forcing me to evacuate with about 30 minutes notice. I grabbed two backup drives (and a bunch of gear), but missed another drive and all my printed photo albums. I was looking into cloud storage when Tony posted this, good timing. Upshot: Backup, but have a plan and checklist for what to grab if you have to leave at a moment's notice.
Interesting how the digital age has made things like this more complicated and costly than in the analog age. Great benefits with digital, to be sure; I would not be using a camera now but for today's digital options. This is a worthwhile presentation, Tony. Thanks so much for creating and sharing it.
Its less costly and less complicated. Stored, processed film can go bad too. And making film backups of film is far more expensive than making digital backups..
@@veganpotterthevegan It is mostly a moot point for me. The question is, do we live in the past, or do we live in the present...the present being the only time we actually have to live in? There is enough happening now to occupy our full attention everyday. I don't really agree with the cost and complication thing though. I think it best to print your best work onto archival media, display it, and enjoy it today and each day...while you can.
@@AliasJimWirth We live how we want, it's person to person... odd to draw that comment from your own original comment though🙃 As for printing and displaying, a lot of us take far too many photos to do that. Or can't afford enough frames to do that.
*Some people have piss poor memory. Some were too young to remember really old occurences, and sometimes other people that weren't there would like to see those images. I'd love to see photos from my mother's upbringing. But she was far too poor(in the Philippines) and there are less than 10 photos of her before she got out of her teens. My dad has one photo of his parents, they all burned in a fire while he was in college and his parents were already dead by then.
@@veganpotterthevegan In this life and in this world there are and will be many tragedies and sad moments and experiences. I am sorry for your losses and the general losses of your family. Not to make slight of that, I would point out that the time we have here on this earth, at best, is short of one hundred years. A more significant concern is the question of where each of us will spend eternity, which is not merely one hundred years, but forever. With that in mind, our photos and mementos, have little real and enduring value. Will we remember this life in eternity...I think not.
@@AliasJimWirth There's definitely no life after death which is why it's important to do what you want(so long as you're not hurting anyone). Many people want to look into the past. Nothing is wrong with that. That said, looking at printed photos you hung on a wall is living in the past too.
Tony I generally load all my photos straight to Dropbox which syncs with my Synology NAS, it’s not as big as your setup but also I pay extra for Dropbox to keep my files for 1 year after deletion.
Computer Information Systems degree, but professional cinematographer and wedding cinematographer here.
This is so key, personal and business. My personal/business setup includes a QNAP 72TB ZFS NAS system with a UPS as well.
As a business owner capturing footage that cannot be re-shot, a data backup is probably the most important aspect of your business. There is way too much on the line.
The QNAP has Hybridbackupsync that allows you to back up footage to multiple online providers like Google Drive, Backblaze, and Onedrive.
Once the footage has been shot, I upload it immediately to a Google Drive Workspace account. $12 per month per user and unlimited data.
That way, if anything happens to the server locally, that data is up on Google's servers and can be re-downloaded at any time.
From a personal perspective, it is true that we should be printing our photos and making albums.
Happened to me last year, lost all the holiday tour pictures on a new external HD. Tried recovery software, it found everything except those pictures. Bought a small synology NAS with two used drives (RAID configured). First WD drive failed in the first week, replaced that. Went to a wedding, came back a week later, NAS reporting the other WD drive had failed. Like you I have taken to leaving the images on both the X-T2 cards instead of formatting each time. If I had done that I would have last years images.
How do you sync your NAS at one location with the NAS at the other location? I need exactly that solution!
I need to do some work to cleanup and backup my files better. I have 3 copies but they are all in the same place. I manually save copies, no automatic backups as they provide limited value by copying errors etc.. I don't have a cloud backup.
Very interesting to think about the ideal solution.
Hey Tony... I recently had something similar happen to my Lightroom catalog. I can’t prove it, but I suspect Lightroom Face Detection. Just curious if you were scanning faces or doing something that would update the metadata in large parts of your catalog?
I had a hard drive crash in 2013, and I didn't have a serious backup strategy. Now I have a 100T off line back up facility where I back up photos and videos at least twice a month, and often once a week. I lost All the photos that I took that year. Fortunately I had prior photos backed up already through dumb luck on another computer. Now i have a system, and that system seems to work pretty well. I'm thinking of doubly backing up photos, but haven't done that yet.
We use Synology racks now. We just upgraded to backups too. Data integrity is super important.
Are there checksum or even parity-bit solutions to combat bit rot?
I once wanted to insert a sd card in the reader... And it broke in the middle... Now I have a camera with 2 sd card slots.
I tend to make a backup every month and store once a year on a separate drive that will probably never be touched again.
Lastly I have a small (100GB) storage server rented for the best.
Is there any easy checksum tool that can just generate (and check) sidecar checksum files of a whole folder if not already available. That would at least allow you to catch bad files before backing them up and it's independent from the file system.
What's causing your camera to lose focus a few times, esp. near the end?
OK but how do I protect myself against the bit-rot? I use a Synology NAS (only four disks with some type of RAID) and I keep copies on multiple computers.
But as you say I am not going to know until I open them years later.
What about Synology's snapshot Replication functionality? Could it be another safeguard against bit rot in the system?