One of the best channels to learn about ZFS, TrueNAS Scale and network… watched many videos, while trying to replicate the ideas on my server (play, stop, play,…) Thank you for the great content❤
A word of caution regarding snapshots and malware. In general, I would NOT depend solely on snapshots for protection against malicious changes (e.g., ransomware). Because the snapshot is read-only and only changes are recorded from the point of the snapshot, the snapshot necessarily must remain online and accessible, unlike a traditional backup/imaging to remote storage (e.g., USB enclosure turned on only for those purposes). That means there's always the potential that your snapshot could be corrupted as well! Yeah, I know, in theory that's NOT supposed to happen, but malware gets more malicious and capable by the day. And as long as any data is online, it's a target. Snapshots are best suited for protection against *accidental* changes. When it comes to malicious changes, the best approach remains an offline backup/image. That's not to say I wouldn't use snapshots; they're still very useful. I use them w/ VirtualBox as well, but NOT because of concerns about malware. But you need to understand how they work to appreciate their limits.
I'm 79 for ZFS I prefer the raw experience. If I want restore an old version of a file, I use the file manager to go to the .zfs directory, go to the folder of the snapshot and than I copy and paste the file I want to restore to the required directory :) TrueNAS seems to complicate a very basic operation. If I create a dataset, I use the CLI and I make sure that the .zfs folder is visible and I set the mount-point of my main datasets to /media/dataset-name, which on Linux and Windows makes the dataset visible in the file manager like a mounted disk :)
If you're 79, you're probably used to doing things predominantly with a CLI. CLI sucks for grabbing multi-select files whilst not grabbing others (vs. it's faster to just use CTRL+click to selective select the files that you want to restore/manipulate/etc.). IF you type something wrong in the CLI (e.g. rm dataset *file.7z vs. rm dataset* file.7z) -- you get VERY different results. ^THAT, BTW, happened (to me) and resulted in a 3-day data recovery effort because the files that I was manipulating were the pictures of our kids when they were really young (0-3 years) -- pictures from my wife's phone that we can't get back (cuz they're not 0-3 anymore). The CLI is VERY unforgiving if you screw up. Snapshotting and snapshots are a safety net for said screw ups.
@@0r_1x It REALLY depends on what you do/what you're doing. If the files that you have are named in such a way that you can script it, then yes, you can use a 4-line long shell command or shell script to perform said multi-select via the CLI. But the problem with that are the assumptions of: 1) You have a priori knowledge with respect to how to compose said shell command (or shell script), which SIGNIFICANTLY varies) and 2) again, it assumes that the file names are programmatic enough such that you CAN script it. But if you're working with randomly generated file names, and you want to dedup your files, that lack of programmability means that going through the GUI is going to be significantly faster, and/or if you don't know how to write said shell command, using the GUI is DEFINITELY going to be significantly faster for that group of users in the Venn diagram of all users. i.e. if you ignore the fact that most people AREN'T YOU, then yes, your statement would be correct. But that's the thing -- most people AREN'T you, and therefore; don't know what you know to be able to do what you said in your reply.
@@ewenchan1239 RSI leads me to operate under the mantra of "mouse bad, just use keyboard". Which honestly, 9/10 times in any GUI I'm hitting super key and searching for what I want. Crap moves in a UI interface way too often or what I want is buried under layers of sub-menus. I just think your point of someone being 79 and using CLI for ZFS commands "because they're used to the old way" is odd. ZFS CLI commands are the same across any distro, ZFS UI commands are not.
Very nice video, I didn't setup snapshots yet unfortunately but I will. Thanks for this video, it reminded me of doing this. Snapshots here work the same way when I do snapshots in VirtualBox for the VMs.
Good tutorial videos, I've been looking for tutorials and haven't found much deep dive information with demo and break down on how it works. Im looking into changing from Unraid to TrueNAS Scale. What size of NVME drive you recommend for the OS to live on. I currently have a 2tb NVME I was using as Plex metadata only. Writing everything directly to the pool.
Great video!!! I recognise that there is a significantly more technical video that explains what happens if you delete a snapshot in the middle. My question is since space is finite, the thing that I have never understood about ZFS snapshotting is what happens if you tell it to keep 2 weeks worth of snapshots, and then it starts purging old(er) snapshots. Does it delete that data or does that differential become "promoted" to be a part of the "actual"/"original" dataset? If you don't mind explaining this a little further, that would be greatly appreciated. Thank you.
@@LAWRENCESYSTEMS So....I am a little bit confused. If you delete the snapshot, does that mean that you are deleting/losing the data? Or does that differential data gets "promoted" to the original dataset to begin with? And if deleting the snapshot = deleting the data, then why would people use snapshots since space is finite, and every time you delete a snapshot, you are deleting the data that's contained inside said (differential) snapshot. That's the part that I never fully grasped how snapshots and snapshotting worked. (i.e. I don't want to lose my data. And if deleting a snapshot means that I am going to be deleting my data, then it begs the question "why would I implement something that will result in deleting data vs. rolling those changes into the original dataset?")
@@ewenchan1239 As I said in the video, snapshots are read only versions of your data at that point in time, you only lose the ability to roll back to that point in time when you delete the snapshot.
Great content as always! I learned that Windows can restore snapshots! Is it the case that Windows has ZFS support? Or is this baked into the SMB protocol?
One question I have a Dataset called App_Data and inside 6 datasets for each of the 6 apps. the App_Data shows as used space 54,02GB (Data Written 151.13 KiB and Children 54.02 GiB). If I do a snapshot of the App_Data, the snapshot referenced is 151.13 KiB. So do I have to do snapshots of child dataset or the parent snapshot also contain the child dataset, because judging from the size on the snapshot, it only reference the parent.
I didn’t know I couldn’t restore ix applications and my entire applications folder is not bricked, all my apps have been deleted for some reason. Should I proceed to restore all ix applications? My kubernetes service cannot be started as well. How should I get my data back or start all over given the state of the kubernetes services with missing files
A quick question with snapshots the first snapshot how big will it be if you got a 20tb of storage and you have 5tb of data will the first snapshot be 5tb?
@@user-hc6uo5fp8n linux timeshift works like this too. Either because it's doing filesystem-level snapshots (on btrfs, when set inbtrfs mode), or because it's hardlinking the same files so they are not stored multiple times (on any other file system, set in rsync mode). It's still trying to act like snapshots. The behaviour you describe is how incremental backups work. The first backup is the same size of your data, and then all incremental backups add the changes on top of it
@@user-hc6uo5fp8n now, maybe your file manager is confused and counts the sizes of the snapshotted folders as if they were not hardlinked, but the free disk space should not change after you do a snapshot with timeshift
One of the best channels to learn about ZFS, TrueNAS Scale and network… watched many videos, while trying to replicate the ideas on my server (play, stop, play,…) Thank you for the great content❤
A word of caution regarding snapshots and malware. In general, I would NOT depend solely on snapshots for protection against malicious changes (e.g., ransomware). Because the snapshot is read-only and only changes are recorded from the point of the snapshot, the snapshot necessarily must remain online and accessible, unlike a traditional backup/imaging to remote storage (e.g., USB enclosure turned on only for those purposes). That means there's always the potential that your snapshot could be corrupted as well! Yeah, I know, in theory that's NOT supposed to happen, but malware gets more malicious and capable by the day. And as long as any data is online, it's a target. Snapshots are best suited for protection against *accidental* changes. When it comes to malicious changes, the best approach remains an offline backup/image. That's not to say I wouldn't use snapshots; they're still very useful. I use them w/ VirtualBox as well, but NOT because of concerns about malware. But you need to understand how they work to appreciate their limits.
I'm 79 for ZFS I prefer the raw experience. If I want restore an old version of a file, I use the file manager to go to the .zfs directory, go to the folder of the snapshot and than I copy and paste the file I want to restore to the required directory :) TrueNAS seems to complicate a very basic operation.
If I create a dataset, I use the CLI and I make sure that the .zfs folder is visible and I set the mount-point of my main datasets to /media/dataset-name, which on Linux and Windows makes the dataset visible in the file manager like a mounted disk :)
That's exactly how I did it with NetAPP storage, which is essentially the same thing as ZFS
If you're 79, you're probably used to doing things predominantly with a CLI.
CLI sucks for grabbing multi-select files whilst not grabbing others (vs. it's faster to just use CTRL+click to selective select the files that you want to restore/manipulate/etc.).
IF you type something wrong in the CLI (e.g. rm dataset *file.7z vs. rm dataset* file.7z) -- you get VERY different results.
^THAT, BTW, happened (to me) and resulted in a 3-day data recovery effort because the files that I was manipulating were the pictures of our kids when they were really young (0-3 years) -- pictures from my wife's phone that we can't get back (cuz they're not 0-3 anymore).
The CLI is VERY unforgiving if you screw up.
Snapshotting and snapshots are a safety net for said screw ups.
@@ewenchan1239 GUI is rarely faster, use wildcards, matching, and get an overall better directory organization.
@@0r_1x
It REALLY depends on what you do/what you're doing.
If the files that you have are named in such a way that you can script it, then yes, you can use a 4-line long shell command or shell script to perform said multi-select via the CLI.
But the problem with that are the assumptions of:
1) You have a priori knowledge with respect to how to compose said shell command (or shell script), which SIGNIFICANTLY varies) and
2) again, it assumes that the file names are programmatic enough such that you CAN script it.
But if you're working with randomly generated file names, and you want to dedup your files, that lack of programmability means that going through the GUI is going to be significantly faster, and/or if you don't know how to write said shell command, using the GUI is DEFINITELY going to be significantly faster for that group of users in the Venn diagram of all users.
i.e. if you ignore the fact that most people AREN'T YOU, then yes, your statement would be correct.
But that's the thing -- most people AREN'T you, and therefore; don't know what you know to be able to do what you said in your reply.
@@ewenchan1239 RSI leads me to operate under the mantra of "mouse bad, just use keyboard". Which honestly, 9/10 times in any GUI I'm hitting super key and searching for what I want. Crap moves in a UI interface way too often or what I want is buried under layers of sub-menus.
I just think your point of someone being 79 and using CLI for ZFS commands "because they're used to the old way" is odd. ZFS CLI commands are the same across any distro, ZFS UI commands are not.
Another excellent content in ZFS, thank you Tom!
Glad you enjoyed it!
I love rollback. I do not allow empty - I do hourly snapshots but for 2 days on my NAS, then I do Dialy I keep for 2 weeks. It's served me well!
Very nice video, I didn't setup snapshots yet unfortunately but I will. Thanks for this video, it reminded me of doing this. Snapshots here work the same way when I do snapshots in VirtualBox for the VMs.
Great vid Lorry... ZFS is black magic. love it!
Thank you Tom for this great explanations on ZFS.
Glad it was helpful!
Tom, is there a GUI method to view these files and act on them, similar to a “file explorer”, rather than doing it all via the command line? Thanks
th-cam.com/video/XCouS6Zw5vA/w-d-xo.htmlsi=WNCFeoMKtu8Suq6_
Good tutorial videos, I've been looking for tutorials and haven't found much deep dive information with demo and break down on how it works. Im looking into changing from Unraid to TrueNAS Scale. What size of NVME drive you recommend for the OS to live on. I currently have a 2tb NVME I was using as Plex metadata only. Writing everything directly to the pool.
This is awesome, Thanks, Tom!
Glad you liked it!
Super informative, thanks Tom
Very welcome
Great video!!!
I recognise that there is a significantly more technical video that explains what happens if you delete a snapshot in the middle.
My question is since space is finite, the thing that I have never understood about ZFS snapshotting is what happens if you tell it to keep 2 weeks worth of snapshots, and then it starts purging old(er) snapshots.
Does it delete that data or does that differential become "promoted" to be a part of the "actual"/"original" dataset?
If you don't mind explaining this a little further, that would be greatly appreciated.
Thank you.
Yes, as you delete snapshots you get back the differentials space they take.
@@LAWRENCESYSTEMS
So....I am a little bit confused.
If you delete the snapshot, does that mean that you are deleting/losing the data?
Or does that differential data gets "promoted" to the original dataset to begin with?
And if deleting the snapshot = deleting the data, then why would people use snapshots since space is finite, and every time you delete a snapshot, you are deleting the data that's contained inside said (differential) snapshot.
That's the part that I never fully grasped how snapshots and snapshotting worked.
(i.e. I don't want to lose my data. And if deleting a snapshot means that I am going to be deleting my data, then it begs the question "why would I implement something that will result in deleting data vs. rolling those changes into the original dataset?")
@@ewenchan1239 As I said in the video, snapshots are read only versions of your data at that point in time, you only lose the ability to roll back to that point in time when you delete the snapshot.
@@LAWRENCESYSTEMS
Gotcha.
Thank you.
I guess I'll have to play around with it to learn more about how it actually works.
Thank you.
Great content as always! I learned that Windows can restore snapshots! Is it the case that Windows has ZFS support? Or is this baked into the SMB protocol?
SMB has support for Shadow Copies, and Samba translates between that and ZFS
Not sure which came first, ZFS or SMB, but that is nice parity between the ZFS interface and the SMB protocol 😊
One question I have a Dataset called App_Data and inside 6 datasets for each of the 6 apps. the App_Data shows as used space 54,02GB (Data Written
151.13 KiB and Children 54.02 GiB). If I do a snapshot of the App_Data, the snapshot referenced is 151.13 KiB. So do I have to do snapshots of child dataset or the parent snapshot also contain the child dataset, because judging from the size on the snapshot, it only reference the parent.
Replying to myself. SORRY, you already explain check the Recursive box.
I didn’t know I couldn’t restore ix applications and my entire applications folder is not bricked, all my apps have been deleted for some reason. Should I proceed to restore all ix applications?
My kubernetes service cannot be started as well. How should I get my data back or start all over given the state of the kubernetes services with missing files
If you set the apps up properly the data should be in the datasets you chose.
are shadow copies available for iSCSI?
That is block storage so that is up to the system connecting to that block storage.
Does this now support block(iscsi), unless am wrong in core only nfs/smb were supported.
I don’t know about core, but scale does support snapshots for zvol’s, so you should be covered whether your iSCSI is backed by a zvol or a file.
Yes, both Core & Scale support ZVOL & iSCSI and snapshots.
A quick question with snapshots the first snapshot how big will it be if you got a 20tb of storage and you have 5tb of data will the first snapshot be 5tb?
The snapshot is the difference in change of that dataset since the snapshot
eg 0, first snapshot takes no space, until you change your files
@@bababababa Thank you for this I thought snapshot were like timeshift on linux thank you for clearing this up.
@@user-hc6uo5fp8n linux timeshift works like this too. Either because it's doing filesystem-level snapshots (on btrfs, when set inbtrfs mode), or because it's hardlinking the same files so they are not stored multiple times (on any other file system, set in rsync mode).
It's still trying to act like snapshots. The behaviour you describe is how incremental backups work. The first backup is the same size of your data, and then all incremental backups add the changes on top of it
@@user-hc6uo5fp8n now, maybe your file manager is confused and counts the sizes of the snapshotted folders as if they were not hardlinked, but the free disk space should not change after you do a snapshot with timeshift
The personal company ad grows bigger and bigger. At some point in the future it will probably be bigger than the whole clip.
It's the same it has been for almost 2 years, would you rather it be for some food subscription service?
@@LAWRENCESYSTEMS No problem. I cancelled my sub with your channel.
¯\_(ツ)_/¯
@@DocMacLovin weak
I am the super nerd😂
I knew people would look!
Immediately knew it had to say something when I saw it 😅🙈
Just found your channel and I'm lovin' it! You have a new sub!
Thanks!