@@hassanzahin7852 That was not why CDDL was made and this has been hashed many times by the people that created ZFS and got Sun to try to open source their stuff. The suits didn't like the fact that the GPL didn't preserve certain rights to the business, and the Apache License while good wasn't quite enough to make the suits feel better so they made the CDDL which was the GPL and MIT licenses mashed together with all the bits that made the suits feel nervous stripped. They also say hind sight being 2020 they regret not fighting harder sooner to get it licensed into an existing opensource license. The FUD that they made the CDDL as some sort of genius mastermind plot because they hated the GPL is just that, FUD
@Richard Clutterbuck Yeah that's right on that. Ubuntu is basically saying by precedent currently a user can choose to do whatever they want, so "ZFS is no different than loading the NVidia drivers, fite me" in effect. Nobody really wants to take up that fight because if they win they basically kick Linux hardware support in the shin, if they lose they weaken the GPL.
@@Lead_Foot you are not suppoed to change it out with XFS or ext4 afterwards, it is a learning process that you'll stick with for the rest of your life, so you have time to learn :-)
I'd love a video on ZFS tuning. So far, all articles etc. on this topic have been a complete mystery to me, the documentation is just not very helpful to me in this regard. It only talks about hundreds of tunables without any comprehensive explanation (that a person without a deep understanding of the source code could understand) what they mean.
Create a pool and a dataset. Run 'zpool get all | less', run 'zfs get all | less'. Read through all the flags and properties, Google them and learn about them in the manuals 'man zfs', 'man zpool'. Read openzfs.org/wiki/Performance_tuning and start tweaking the settings with 'zpool set' and 'zfs set'. Once you're comfortable tweaking through the ZFS commands, you can start tweaking your OS' kernel parameters and the ZFS services, if need be.
@@erisdiscordia5547 Yep it's basically what happened to me, they were all these fake WD Red 6Go NAS HDDs, i bought them before the issue was exposed...
I would check that they aren't on this list, if they are, return them now and get a product not on this list. The list: www.ixsystems.com/community/resources/list-of-known-smr-drives.141/
2:50 oh the days we've had SUN-BLADE workstations with Solaris OS , SPARC processors and the rest .... such nice systems to work on ... some still survive till today (almost 20 years later)... only a few of us original IT guys know how to use and administer them so their days are numbered (in preeeeeeetty small number).
I use ZFS everyday, we use it on most of our storage devices, and I'm still learning new stuff about it on a regular basis. Sun really did some great work.
Good stuff Wendell. For months I poured over reviews of raid cards to run a raid 5 array. Then I was introduced to FreeNAS and ZFS, and when I found out all was needed was a decent hba, I was sold. ZFS is crucial for data protection.
This is the best reason. Redundancy brought to the masses with inexpensive HBA cards, and all you really need is a bunch of RAM to run it, which we have. ECC is not even required, its just important to have. Also, its open source and we can tell what its doing. Using something like FreeNAS will teach you ZFS and BSD at the same time, if you decide to use the command line, and take control of your own data destiny.
2 years later--ZFS is still rock solid and I have to say the only time I have lost data, is when I tried to use the command line to move data instead of TrueNas native data transfer. Probably error on my part. I understand more now about ZFS, and love freeBSD which natively supports it and so I have not done anything with BTRFS. Great Informative video !
Wow - thanks for the shot at 13:00 of your old offices. It made me realize how long I've actually been watching your videos! Boy, how time flies...thanks, Wendell!
Thank you for this. Having someone like you explain it is so much more digestible than reading a ton of technical documents.. Would love to see a video like this for *XFS* ! But if you don't want to that's totally understandable.
I live in a country with 2 to 14 power-fails/week and I had a lot of garbled music files, so I'm very happy with ZFS and its Copy On Write. I even have my dataset with music stored with copies=2, basically introducing a mirror for that dataset in an otherwise striped datapool.
@amitranaware9356 That is why I have a 1200W Avtek Surge Protector. But in general power supplies and motherboards die more often, because of power fails.
I enjoy these types of videos way more than tech news on the other channel, I would really like if the focus would shift towards teaching material such as this!
I started using ZFS a few days ago, and I was really blown away at how incredibly simple it was. I wanted to configure four identical disks in RAID 10, and this was the only command I had to use: $ sudo zpool create NAME mirror VDEV1 VDEV2 mirror VDEV3 VDEV4 And that was it. No need for formatting or lenghty initialization processes. It just worked right away, and performance is excellent. I'm sure ZFS can get really complicated for more intricate setups with caching and tiering and tuning, but for my simple use case it was refreshing to see how easy it was.
22:30 You can always se the "copies" property to e.g. 2. By doing so you're telling the filesystem that it should store every file within the given dataset 2 times, on 2 different physical location on the disk. So if one of them is on a location with full of bad sectors, the fs will still be able to retrieve the file from the other location.
I have a 12 bay ZFS file server appliance, and i have experienced bit rot, but thanks to ZFS Z3 i have never actually suffered a problem do to it, and it fixed the problem, if i get more than one on a drive, I'll do a scrub to make sure no other drives have corruptions, then swap the drive with a replacement
It sounds like ZFS works best with a large number of slow storage drives. Could you use like … 48 micro SD cards with ZFS to create one really nice and reliable total storage area? (Not sure what the technical name for a combined virtual storage drive is. Is there a generic term, or does it change from file system to file system?) Actually, how would you use an silly number of micro SD cards as storage for a single computer?
Love ZFS as a gamer. I took my gaming laptop with two ~500GB SATA SSDs, created a mirrored bpool, and then created a striped rpool. Of course, I recommend no one run this way, but it’s fast. Also, with built in compression, potential exemption of datasets from snapshots, and dataset level case sensitivity, I can create separate sets for flatpak, Steam, and Lutris that are fully under my control with increased operability of case-insensitive programs :) No idea how to use it on a server level, but as an insane home user, I love it.
Checksums give you 1 bit of information - whether or not there is an error somewhere. Error-correcting codes tell you where the error is. With binary data, when you know where the error is, you can invert it - therefore removing the error.
Great high-level overview video Wendell! I wonder if you do a next installment could you fit unRAID (XFS/BTRFS) in there with a similar approach? Like basic overview, differences, applications?
Obviously ZFS is more mature than BTRFS, because you can’t tell me that whoever came up with the name wasn’t thinking “butterface” as in a lady with a nice body, but an ugly face-which isn’t exactly the worst description of BTRFS as a file system…
I realize this video is 4 years old now, but I want to share my 2024 experiences with ZFS on my Fedora 40 desktop. I run a redundant ZFS Mirror on root and 4 SATA SSDs (Crucial MX500 4TB) in a Zpool with my data. While transferring data from my nvme root mirror to my SATA SSD pool, I easily get 1.3 - 1.5GB/sec transfer speed. Small files are finished before my computer tells me the transfer has started. It is ridiculously fast. Same thing to my TrueNAS device over my 10G home network. Insane transfer speeds with ZFS at both ends.
TLDR: Basic Raid: If a file corrupts, even when there is a non corrupted file on a another drive, you can't get it (unless you send the drives to recovery) ZFS: automatically gets the non corrupted file from the mirror device Also ZFS hasn't caught upto NVME yet (@24:22), to be fair RAID is an array of *"Inexpensive"* drives
There is debate back and forth about whether ECC memory is required or an optional extra. ZFS has been the file system of TrueNAS (formerly FreeNAS) for many years. iX Systems, the maintainers of TrueNAS, only sell computers with ECC memory. Draw whatever conclusion you wish from that. Personally, I think if you’re going to go to the trouble of running ZFS to protect your data, and you’re buying the hardware to run it (rather than using what you’ve already got) it makes sense to spend whatever extra it costs to get ECC memory. But I’ve lost data to bit rot and to having a backup hard drive fail at the same time as my primary drive failed, so perhaps I’m more cautious than other people? But that’s why my primary storage is on a TrueNAS server using ZFS. It’s built with a used SuperMicro server motherboard using ECC memory. My first backup is on a duplicate of that server. My second backup is online at Backblaze. My third backup is on a Windows box in a different city. And just for the fun of it, I also keep extra copies of my entire family photo archive on Google and iCloud. There are only two kinds of computer users. Those who have lost data, and those who will.
Here is an example of bitrot: you pay for google photos and notice that really really old photos start to have gray lines or half of the picture is gray. It doesn’t what device you use to try to view those photos. Has Googles datacenter experienced bitrot? I have since added Nextcloud with a zfs backend to the place I backup, no more trusting in google to ensure your data is reliable.
Regarding scrub speed - 24 drive storage: CentOS 7, OpenZFS 0.8.4-1.el7, 1 pool, 1 drive pool global spare, 2 vdevs raidz2 (11 and 12 diskas), 90TiB of data ~= 2 days for a scrub (while other machines are writing about 50MiB/s data to it over the network)
OMG, PBS couldn't have done a better job at explaining this. I was with DEC back when ... This was the type of information that was available daily. It's great to see it again. Hit SUBSCRIBE, "YES"
I'm interested in how the Windows port of ZFS turns out. Recently I've been tinkering with the next best thing (kind of) - exporting a ZFS ZVOL on FreeBSD as an iSCSI target and mounting it on a Windows PC. Effectively NTFS on top of ZFS. Surprisingly easy to set up, too.
Why would you place NTFS on top of ZFS? I'm asking out of interest, I am not questioning your decisions, I don't have a very good understanding of ZFS or Filesystem's. I hope this makes sense :) Thanks :)
@@longnamedude3947 ZVOLs are a little different from ZFS datasets. A ZFS dataset is a dynamically allocated chunk of storage space with a mountpoint that the OS sees as a UNIX-like filesystem (inodes, users, groups, permissions, etc.) with some properties like "is the data compressed?" or "is the data deduplicated?". A ZVOL is like telling to ZFS "I need X amount of raw storage no strings attached" and then ZFS creates a new (virtual) device with X amount of storage space, but it's not formatted with ANY filesystem: the main use for this feature is to use these chunks of raw data as disk space for virtual machines, but nothing prevents you to set them as an iSCSI target. iSCSI is basically SCSI over Ethernet and Windows can access iSCSI targets (aka iSCSI data volumes) and see them as regular disks and so format them in NTFS.
Can you also do the same for btrfs? Do you recommend it for traditional desktop usage, opensuse uses btrfs as its default filesystem. What do you think about the compression feature of btrfs?
In a round about way ZFS was a pioneer in software defined storage where managing storage at a low level went from hardware to software. You now see the same principles applied in many other applications and technologies like HCI for instance.
I recently built a storage box and I couldn't decide if I wanted to go with NTFS or ZFS (Windows vs FreeNAS (at the time)). I really liked FreeNAS's interface, and after I got over the learning curve coming from an all-Windows background, I was all set to use it except 1) I didn't like the idea that I couldn't add a single drive to the pool. As an individual on a budget, being able to add a single drive here and there is of great benefit to me, and 2) at the time I was still under the impression that using RAID was better than using the disks individually and FreeNAS wanted me to break up the RAID into individual disks. In the end I decided to go with NTFS but now I'm starting to second-guess myself. Can anyone tell me what the minimum number of disks I can add to a vdev pool in ZFS and can each pool be aggregated into one larger pool of storage?
I used to go to a swap meet in CA with my dad on weekends where he would buy things like 8" floppy drives and huge 68000 cpu-based servers that were basically rescued from a landfill. Two or three weekends later he would triumphantly show me a working command line on an amber monochrome monitor. I miss my dad.
Thank you Wendell, this is one of the best introduction to ZFS, it's clear and concise, it's easy to understand from a beginner's perspective. Would love to see more of this type of video!
There is error correction encoding at the driver level, so the driver can usually tell, if the data is corrupt. Of course, this may fail too, or just the electronics can fail, but it is not so simple for a drive to fail without notifying it.
Im amazed on how simple you manage to explain this complex topic. Thanks for all the effort you put in to your videos! Btw: I think its pronounced ZFS ..
4 ปีที่แล้ว +1
While other people go for 19:9 video aspect for phone users, you are going the other direction (1.66:1). Just shows how special this channel is :)
Okay, so is there a plug-n-use ZFS backup system? Just seems to overly complicated, I just want to plug-in a backup system, throw in some hard drives, and then it does its thing.
If i get an off site backup do i still need raid redundancy? Its not super important to get things running fast if something fails. I dont need things up and running fast if anything fails, just need to know when it happens and never lose data.
Cool, that corrupted picture you used was of an FRC competition from 2015 if I remember correctly. Winnovation was one of our regional teams when I was in high school.
i've got a thing for SLC SAS SSdrives, you know, the petabytes drives.. Presently on a HP zx2660 (with the 2.5GHz 4 core chips, 32GBmem, GT730video) that has SAS 500GB limitation per drive,LVM Anti-X, PLIANT(149G) raid 5 [3]+PLINT406S+HP480GHDD-DP10K(4) and I saw the option on installation for ZFS and will try them on the platters.. So thanks for the information. I'll try this on my second machine with HGST HUSSL 40200S500 drives. They worked, but had a bad drive so I switched them out on the presently running system. In the past this system had faulted out LVM install on SATA SSD raid5, twice and refuses to do a encrypted install.(buggy old controllers? ). The drive controller battery is dead, direct write trough, maybe that is part of the problem. They are sort of loud.. They live in the basement..
So... how does ZFS handle entire drive dying? Is it like raid, so I can replace the broken drive and it rebuilds it or ... not? Or should I put raid on top of ZFS? Or under it? Or they shouldn't even be used together?
ZFS is so much better than the filesystems of old that It's not even funny. Sure, It's not the single fastest or most efficient but It's no doubt the most intelligent, reliable and useful storage solution you can easily setup on your desktop or server today. It does use more resources than say ext4 but the benefits are well worth the cost in RAM and CPU time. That being said, It can also be set up to run in a resource constrained environment without much hassle. You should also have mentioned copy-on-write, snapshots, clones, zvols, compression, encryption and the seemingly endless number of configuration options available. Learn about ZFS and use it, It will solve problems you did not know you had.
What I am interested about is actually the development of bcachefs. It's built on top of bcache but (oversimplified) want to be a ZFS with the performance of an ext4. It's pretty young (with few developer(s)) and isn't mainlined, but can be mainlined into the Linux kernel, is actually stable (compared to btrfs) and a pretty "clean" (for a FS) codebase. Definitely something we should keep an eye on.
Great video. I just got my self a new dell Precision 5750 with two NVMe drives and want to do a system wide raid1 setup on gentoo. Was wondering would people recommend zfs for this type of setup? Am just getting my head into zfs at this point and love any tips. Thanks
Is MS server ReFS, microsofts version of ZFS? I am debating whether to switch from [MS Server 2019] to [TrueNas core or source] - it is simply for file server in Video Production? any recommendations?
I use ZFS with Ubuntu on desktop and laptop. I have a backup server with Pentium 4 HT (3.0 GHz); 1.25 GB DDR (400 MHz); 1.2 TB with 2 IDE HDDs and 2 SATA-1 laptop HDDs. That one has been designed for Microsoft Windows 2000 Pro and Windows 98 and it now runs 32-bits FreeBSD 12.1 with ZFS for say one hour/week.
Here I am almost a year later watching this video. Why? I bought a QNap NAS to play with, and I'll eventually buy another one to back up to. Thank you once again Wendell for putting out these, and I'm now subbed to this channel in addition to the L1T channel. :)
One of the developer of Free NAS which uses ZFS F/S is adamant about using ECC RAM.with ZFS. Why is ZFS more vulnerable than other File Systems on non-ECC RAM, as opposed to EXT4 or BTRFS which do not necessitate the use of ECC RAM? I would appreciate a response from the video maker or someone who knows exactly why?
A performance hit for ZFS because of checksumming and other features ZFS has was mentioned. And it said EXT4 was faster. Does that hold true for write speed only or is ZFS slower on read speed as well?
Why can't i stretch the screen to full on my LG V40. Wendell, what did you do. Did you lock the resolution on your video again. I still don't get it. Going full screen on a smartphone shouldn't be blocked when watching any video.
Totally - Been running ZFS as a nas for myself. Work and all my customers. It's virtually bullet proof. I have had corruption once though that lost the entire pool. A poorly designed power supply caused it. Remember it's NOT about the wattage of a PSU it's the amount of power on the rails basically 1200W GOLD was not enough to keep my drives alive. A smaller better spread out 700W had no issues. Most modern PSU's have loads of power for the GPU but not for multiple drives as more people move to SSD's and CLOUD.
@@df3yt Ty for the answer. What would you use as Raid there, something like a Raid 5 with one "spare" HDD? I thought about using a small APU with the upcoming renoir chips since I want to avoid Intel if possible.
Hey Level1! I’m about to be brave and give ZFS a shot! Using 5 drives of which 2 are redundancy. Got 2 NVME drives for cache (Unraid). Can you please give some advice on the file system I should use for my cache? ChatGPT said “the smart people at Unraid picked BTFS for their cache for a reason!” LOL! What do you think, BTFS cache and ZFS HDDs? Any advice would be really appreciated!!!
What is the speed of the BIOS chip vs RAM vs NVMe SSD? What would happen if the flash memory in an SSD was treated as flash memory, and not as a simulated HDD?
ZFS doesn't catch memory corruption though right? For instance If I move a file and it caches in ram, It can be corrupted in ram and then written to disk. ZFS has no idea that happened and neither do I. Short of buying ECC memory what would you recommend?
If you're moving a file the copy will probably be checked against the original before the original gets deleted. It would potentially be a problem if you're writing changes to a file.
If your data is important enough to you that you are worried about memory corruption, spend the money for ECC ram. It doesn't matter what file system you use, all of them are susceptible to memory corruption.
Would it be possible for hard drive manufacturers to make each drive as a RAID 0 were the data would be stripped across the platters? I have wondered why hard drives cap at 200Mbps whole density kept going up. And since the whole drive dies at the same time, so the drawback of raid 0 doesn’t seem to make a difference.
ZFS is cool, I have contact with Oracle Appliance with a shelf of 400 drives. A metadata server is required for its operation. Added to this is the HSM server and client. Greetings!
Oracle acquiring Sun is classic example of some evil corp buying a really good company and destroy it.
Actually Sun Microsystems intentionally chose a license that isn't GPL compatible so that Linux can't add zfs drivers in it's kernel.
@@hassanzahin7852 isnt the legal opinion on this divided? Strictly talking about openzfs.
@@hassanzahin7852 That was not why CDDL was made and this has been hashed many times by the people that created ZFS and got Sun to try to open source their stuff. The suits didn't like the fact that the GPL didn't preserve certain rights to the business, and the Apache License while good wasn't quite enough to make the suits feel better so they made the CDDL which was the GPL and MIT licenses mashed together with all the bits that made the suits feel nervous stripped.
They also say hind sight being 2020 they regret not fighting harder sooner to get it licensed into an existing opensource license. The FUD that they made the CDDL as some sort of genius mastermind plot because they hated the GPL is just that, FUD
@Richard Clutterbuck Yeah that's right on that. Ubuntu is basically saying by precedent currently a user can choose to do whatever they want, so "ZFS is no different than loading the NVidia drivers, fite me" in effect. Nobody really wants to take up that fight because if they win they basically kick Linux hardware support in the shin, if they lose they weaken the GPL.
Oracle watched 300 and thought that classic line was brilliant.
Oracle: "Our errors... will blot out the Sun!"
I love how a 30+ minute video is a brief primer on this channel.
No, I'm not being sarcastic, I actually love that.
Right!? Some of us have a longer attention-span than a goldfish...
ZFS is not very beginner friendly.
@@Lead_Foot you are not suppoed to change it out with XFS or ext4 afterwards, it is a learning process that you'll stick with for the rest of your life, so you have time to learn :-)
There's another channel that has an introduction to ZFS video that's 90 minutes long. Just the introduction.
Others do a 12 minutes video and title it "everything you want to know about zfs".
It's pronounced ZFS not ZFS.
>Americans can't even pronounce ZFS as ZFS
What on earth are you talking about, CLEARLY its ZFS. I mean come on, ZFS just sounds... silly
It's aladeen not aladeen 🙏
I read every one of those as "zed eff ess".
@@tin2001 Me too
Wendell:"Brief primer"
Me: "Really? Brief?"
Wendell: "30 minutes"
Me: "Oh, ok! That's very brief for ZFS!"
I was thinking the same thing. 30 min video 😅
I’m so glad people like you are working so hard to educate us; I owe you. If you are ever in London, my house is your house.
I'd love a video on ZFS tuning. So far, all articles etc. on this topic have been a complete mystery to me, the documentation is just not very helpful to me in this regard. It only talks about hundreds of tunables without any comprehensive explanation (that a person without a deep understanding of the source code could understand) what they mean.
+1
Yeah same
The books Wendell recommended at the end are very good. Try them.
Create a pool and a dataset. Run 'zpool get all | less', run 'zfs get all | less'. Read through all the flags and properties, Google them and learn about them in the manuals 'man zfs', 'man zpool'. Read openzfs.org/wiki/Performance_tuning and start tweaking the settings with 'zpool set' and 'zfs set'. Once you're comfortable tweaking through the ZFS commands, you can start tweaking your OS' kernel parameters and the ZFS services, if need be.
go to ZFS subreddit, it's pretty good
I love ZFS, it is the best FS I've used, at work and at home.
I really like the snapshot functionality
Time to take notes!
Excellent, this comes up literally the same day I buy three 6TB drives for my first ZFS setup. :D
Hope you didn't buy any of the SMR drives :D
@@erisdiscordia5547 Yep it's basically what happened to me, they were all these fake WD Red 6Go NAS HDDs, i bought them before the issue was exposed...
I would check that they aren't on this list, if they are, return them now and get a product not on this list.
The list:
www.ixsystems.com/community/resources/list-of-known-smr-drives.141/
Hopefully I'm not too late. Save yourself a lot of time and headache. Get another 6TB drive, and set up a pool of mirrors. RAID5/RAIDZ is dead.
better hope none of them are SMR drives!
2:50 oh the days we've had SUN-BLADE workstations with Solaris OS , SPARC processors and the rest .... such nice systems to work on ... some still survive till today (almost 20 years later)... only a few of us original IT guys know how to use and administer them so their days are numbered (in preeeeeeetty small number).
I was selling Sun Ultras (off lease) from Ford ... to the tune of 300 x Month
I use ZFS everyday, we use it on most of our storage devices, and I'm still learning new stuff about it on a regular basis. Sun really did some great work.
As a viewer from the overseas. I appreciate your Aloominum vs. Al-u-min-i-um - I mean Zed vs. Zee distinction.
As an American, I do to. Not sure why, but I do.
American men dont wear bonnets. I would never drive a car with a wind screen, I prefer a wind shield
Nick Norton Everyone in US says Aluminum. Even our Aloominum foil in grocery stores has it printed on the box.
Good stuff Wendell. For months I poured over reviews of raid cards to run a raid 5 array. Then I was introduced to FreeNAS and ZFS, and when I found out all was needed was a decent hba, I was sold. ZFS is crucial for data protection.
This is the best reason. Redundancy brought to the masses with inexpensive HBA cards, and all you really need is a bunch of RAM to run it, which we have. ECC is not even required, its just important to have. Also, its open source and we can tell what its doing. Using something like FreeNAS will teach you ZFS and BSD at the same time, if you decide to use the command line, and take control of your own data destiny.
...yep, can't have data-protection without zfs!
@@touristguy87 I can't see it any other way
2 years later--ZFS is still rock solid and I have to say the only time I have lost data, is when I tried to use the command line to move data instead of TrueNas native data transfer. Probably error on my part. I understand more now about ZFS, and love freeBSD which natively supports it and so I have not done anything with BTRFS.
Great Informative video !
This is probably the best ZFS explanation I've ever encountered in my life.
Wow - thanks for the shot at 13:00 of your old offices. It made me realize how long I've actually been watching your videos! Boy, how time flies...thanks, Wendell!
Thank you for this. Having someone like you explain it is so much more digestible than reading a ton of technical documents..
Would love to see a video like this for *XFS* ! But if you don't want to that's totally understandable.
I live in a country with 2 to 14 power-fails/week and I had a lot of garbled music files, so I'm very happy with ZFS and its Copy On Write. I even have my dataset with music stored with copies=2, basically introducing a mirror for that dataset in an otherwise striped datapool.
@amitranaware9356 That is why I have a 1200W Avtek Surge Protector. But in general power supplies and motherboards die more often, because of power fails.
I enjoy these types of videos way more than tech news on the other channel, I would really like if the focus would shift towards teaching material such as this!
8:21 drives. ZFS START: 17:41
I started using ZFS a few days ago, and I was really blown away at how incredibly simple it was. I wanted to configure four identical disks in RAID 10, and this was the only command I had to use:
$ sudo zpool create NAME mirror VDEV1 VDEV2 mirror VDEV3 VDEV4
And that was it. No need for formatting or lenghty initialization processes. It just worked right away, and performance is excellent. I'm sure ZFS can get really complicated for more intricate setups with caching and tiering and tuning, but for my simple use case it was refreshing to see how easy it was.
Thanks for this great fireside chat Wendel, you are the best
Thanks!
22:30 You can always se the "copies" property to e.g. 2. By doing so you're telling the filesystem that it should store every file within the given dataset 2 times, on 2 different physical location on the disk. So if one of them is on a location with full of bad sectors, the fs will still be able to retrieve the file from the other location.
12:29 It’s a *proprietary* computer on a card. And an inflexible one at that. Which is why I try to avoid hardware RAID.
This may be the only logical thing you've said. This argues for using ZFS instead of hardware RAID. Since ZFS is open source now.
Any hope for more enterprise TH-cam videos?
I have a 12 bay ZFS file server appliance, and i have experienced bit rot, but thanks to ZFS Z3 i have never actually suffered a problem do to it, and it fixed the problem, if i get more than one on a drive, I'll do a scrub to make sure no other drives have corruptions, then swap the drive with a replacement
Would love to hear a comparison between ZFS, WAFL, and Nimble CASL
I put butter on everything, just not my file system. BTRfs is catchy, like you can't say "I can't belive its not ZFS" it doesn't roll off the tounge
Once Redhat abandoned BTRFS as an option, it was doomed...
It sounds like ZFS works best with a large number of slow storage drives. Could you use like … 48 micro SD cards with ZFS to create one really nice and reliable total storage area?
(Not sure what the technical name for a combined virtual storage drive is. Is there a generic term, or does it change from file system to file system?)
Actually, how would you use an silly number of micro SD cards as storage for a single computer?
Love ZFS as a gamer. I took my gaming laptop with two ~500GB SATA SSDs, created a mirrored bpool, and then created a striped rpool. Of course, I recommend no one run this way, but it’s fast.
Also, with built in compression, potential exemption of datasets from snapshots, and dataset level case sensitivity, I can create separate sets for flatpak, Steam, and Lutris that are fully under my control with increased operability of case-insensitive programs :)
No idea how to use it on a server level, but as an insane home user, I love it.
Checksums give you 1 bit of information - whether or not there is an error somewhere. Error-correcting codes tell you where the error is. With binary data, when you know where the error is, you can invert it - therefore removing the error.
Thanks! Been doing flashes of OpenBSD & Ubuntu, both of which offer 'experimenting' with ZFS. Gonna try them out today
Great high-level overview video Wendell! I wonder if you do a next installment could you fit unRAID (XFS/BTRFS) in there with a similar approach? Like basic overview, differences, applications?
Obviously ZFS is more mature than BTRFS, because you can’t tell me that whoever came up with the name wasn’t thinking “butterface” as in a lady with a nice body, but an ugly face-which isn’t exactly the worst description of BTRFS as a file system…
Thank you, chill background music.I enjoyed this.
"Brief"
31:49
Wendell!
Awesome video and explanation of ZFS. Now I have the bug to want to learn more. Thanks so much. 👍
I realize this video is 4 years old now, but I want to share my 2024 experiences with ZFS on my Fedora 40 desktop. I run a redundant ZFS Mirror on root and 4 SATA SSDs (Crucial MX500 4TB) in a Zpool with my data. While transferring data from my nvme root mirror to my SATA SSD pool, I easily get 1.3 - 1.5GB/sec transfer speed. Small files are finished before my computer tells me the transfer has started. It is ridiculously fast. Same thing to my TrueNAS device over my 10G home network. Insane transfer speeds with ZFS at both ends.
TLDR:
Basic Raid: If a file corrupts, even when there is a non corrupted file on a another drive, you can't get it (unless you send the drives to recovery)
ZFS: automatically gets the non corrupted file from the mirror device
Also ZFS hasn't caught upto NVME yet (@24:22), to be fair RAID is an array of *"Inexpensive"* drives
What are your thoughts on ECC RAM being absolutely mandatory when running with ZFS as the filesystem?
There is debate back and forth about whether ECC memory is required or an optional extra. ZFS has been the file system of TrueNAS (formerly FreeNAS) for many years. iX Systems, the maintainers of TrueNAS, only sell computers with ECC memory. Draw whatever conclusion you wish from that.
Personally, I think if you’re going to go to the trouble of running ZFS to protect your data, and you’re buying the hardware to run it (rather than using what you’ve already got) it makes sense to spend whatever extra it costs to get ECC memory.
But I’ve lost data to bit rot and to having a backup hard drive fail at the same time as my primary drive failed, so perhaps I’m more cautious than other people? But that’s why my primary storage is on a TrueNAS server using ZFS. It’s built with a used SuperMicro server motherboard using ECC memory. My first backup is on a duplicate of that server. My second backup is online at Backblaze. My third backup is on a Windows box in a different city. And just for the fun of it, I also keep extra copies of my entire family photo archive on Google and iCloud.
There are only two kinds of computer users. Those who have lost data, and those who will.
Great insights. Have you done vid on how long solid state storage lasts versus hard drive (spinning rust)?
Here is an example of bitrot: you pay for google photos and notice that really really old photos start to have gray lines or half of the picture is gray. It doesn’t what device you use to try to view those photos. Has Googles datacenter experienced bitrot? I have since added Nextcloud with a zfs backend to the place I backup, no more trusting in google to ensure your data is reliable.
You have a good voice for explaining. Kudos.
Regarding scrub speed - 24 drive storage: CentOS 7, OpenZFS 0.8.4-1.el7, 1 pool, 1 drive pool global spare, 2 vdevs raidz2 (11 and 12 diskas), 90TiB of data ~= 2 days for a scrub (while other machines are writing about 50MiB/s data to it over the network)
OMG, PBS couldn't have done a better job at explaining this. I was with DEC back when ... This was the type of information that was available daily. It's great to see it again. Hit SUBSCRIBE, "YES"
Ha! I did the raid 5 recovery last week, that was fun.
I'm interested in how the Windows port of ZFS turns out.
Recently I've been tinkering with the next best thing (kind of) - exporting a ZFS ZVOL on FreeBSD as an iSCSI target and mounting it on a Windows PC. Effectively NTFS on top of ZFS. Surprisingly easy to set up, too.
Why would you place NTFS on top of ZFS?
I'm asking out of interest, I am not questioning your decisions, I don't have a very good understanding of ZFS or Filesystem's.
I hope this makes sense :)
Thanks :)
@@longnamedude3947 ZVOLs are a little different from ZFS datasets. A ZFS dataset is a dynamically allocated chunk of storage space with a mountpoint that the OS sees as a UNIX-like filesystem (inodes, users, groups, permissions, etc.) with some properties like "is the data compressed?" or "is the data deduplicated?". A ZVOL is like telling to ZFS "I need X amount of raw storage no strings attached" and then ZFS creates a new (virtual) device with X amount of storage space, but it's not formatted with ANY filesystem: the main use for this feature is to use these chunks of raw data as disk space for virtual machines, but nothing prevents you to set them as an iSCSI target. iSCSI is basically SCSI over Ethernet and Windows can access iSCSI targets (aka iSCSI data volumes) and see them as regular disks and so format them in NTFS.
Can you also do the same for btrfs? Do you recommend it for traditional desktop usage, opensuse uses btrfs as its default filesystem. What do you think about the compression feature of btrfs?
You know ZFS does compression too right? :)
@@Kludgedean now I do! That is cool!
In a round about way ZFS was a pioneer in software defined storage where managing storage at a low level went from hardware to software. You now see the same principles applied in many other applications and technologies like HCI for instance.
Can we get the 10 hour complete ZFS explanation? I just started using FreeNAS and I'm still getting used to the nuances of how it all works.
Excellent high level video. Just migrated from a Synology to my home build Ubuntu, and needed to understand what the ZFS hype was all about.
17:55 *Hard drive can be a silent carrier: ASYMPTOMATIC !!!!*
LOLOLOLOLOLOLOLOLO
+1
I recently built a storage box and I couldn't decide if I wanted to go with NTFS or ZFS (Windows vs FreeNAS (at the time)). I really liked FreeNAS's interface, and after I got over the learning curve coming from an all-Windows background, I was all set to use it except 1) I didn't like the idea that I couldn't add a single drive to the pool. As an individual on a budget, being able to add a single drive here and there is of great benefit to me, and 2) at the time I was still under the impression that using RAID was better than using the disks individually and FreeNAS wanted me to break up the RAID into individual disks. In the end I decided to go with NTFS but now I'm starting to second-guess myself. Can anyone tell me what the minimum number of disks I can add to a vdev pool in ZFS and can each pool be aggregated into one larger pool of storage?
I used to go to a swap meet in CA with my dad on weekends where he would buy things like 8" floppy drives and huge 68000 cpu-based servers that were basically rescued from a landfill. Two or three weekends later he would triumphantly show me a working command line on an amber monochrome monitor. I miss my dad.
I wonder where can I find the document 9:18 you showed about the ZFS details from jrssystemnet, I looked for it and could not find it 🤔
love ZFS in FreeNas/FreeBSD. when BTRFS gets more reliable I will learn and use it.
Thanks for the video W. :)
Thanks Wendell, I love these, now when is the next one? ;-)
Thank you for the excellent video. I am considering switching from Windows 11 to TrueNAS for my home file backup server.
Great video ofc, but would love to have a similar video about btrfs
24:00 except scanning isn't 200 MB/s... it MUCH more like 20 to 40 MB/s and 60 MB/s if you're really lucky and dealing with HUGE continuous files.
Taciturn is the perfect word to describe the speaker in this video! We are all wondering, when will he come out of his shell and speak more at length?
you said asymptomatic in such a way I just knew this was uploaded this year
Thank you Wendell, this is one of the best introduction to ZFS, it's clear and concise, it's easy to understand from a beginner's perspective. Would love to see more of this type of video!
There is error correction encoding at the driver level, so the driver can usually tell, if the data is corrupt. Of course, this may fail too, or just the electronics can fail, but it is not so simple for a drive to fail without notifying it.
This is so informative and well-explained! Respect man!
22:11 can you recover data with ZFS, if you have a single disk with 2 ZFS partitions?
Thank you for this video, you did a good job dimystifying zfs
Im amazed on how simple you manage to explain this complex topic. Thanks for all the effort you put in to your videos! Btw: I think its pronounced ZFS ..
While other people go for 19:9 video aspect for phone users, you are going the other direction (1.66:1). Just shows how special this channel is :)
This was so incredibly helpful and educational. Thank you so much!
Okay, so is there a plug-n-use ZFS backup system? Just seems to overly complicated, I just want to plug-in a backup system, throw in some hard drives, and then it does its thing.
I am wondering if you would give us an update with some details of the latest OpenZFS updates to the ZFS system
Have you looked at the features that are coming out with the latest TrueNAS OS from iXsystems?
If i get an off site backup do i still need raid redundancy? Its not super important to get things running fast if something fails. I dont need things up and running fast if anything fails, just need to know when it happens and never lose data.
Cool, that corrupted picture you used was of an FRC competition from 2015 if I remember correctly. Winnovation was one of our regional teams when I was in high school.
I got the basic idea, but I think I'm gonna need to try it practically to get it. Too bad I don't have a lot of drives to create a pool out of
i've got a thing for SLC SAS SSdrives, you know, the petabytes drives.. Presently on a HP zx2660 (with the 2.5GHz 4 core chips, 32GBmem, GT730video) that has SAS 500GB limitation per drive,LVM Anti-X, PLIANT(149G) raid 5 [3]+PLINT406S+HP480GHDD-DP10K(4) and I saw the option on installation for ZFS and will try them on the platters.. So thanks for the information. I'll try this on my second machine with HGST HUSSL 40200S500 drives. They worked, but had a bad drive so I switched them out on the presently running system. In the past this system had faulted out LVM install on SATA SSD raid5, twice and refuses to do a encrypted install.(buggy old controllers? ). The drive controller battery is dead, direct write trough, maybe that is part of the problem. They are sort of loud.. They live in the basement..
So... how does ZFS handle entire drive dying? Is it like raid, so I can replace the broken drive and it rebuilds it or ... not? Or should I put raid on top of ZFS? Or under it? Or they shouldn't even be used together?
ZFS is so much better than the filesystems of old that It's not even funny. Sure, It's not the single fastest or most efficient but It's no doubt the most intelligent, reliable and useful storage solution you can easily setup on your desktop or server today. It does use more resources than say ext4 but the benefits are well worth the cost in RAM and CPU time. That being said, It can also be set up to run in a resource constrained environment without much hassle. You should also have mentioned copy-on-write, snapshots, clones, zvols, compression, encryption and the seemingly endless number of configuration options available. Learn about ZFS and use it, It will solve problems you did not know you had.
This patrician gets it.
Can you do one on IPFS?
What I am interested about is actually the development of bcachefs.
It's built on top of bcache but (oversimplified) want to be a ZFS with the performance of an ext4.
It's pretty young (with few developer(s)) and isn't mainlined, but can be mainlined into the Linux kernel, is actually stable (compared to btrfs) and a pretty "clean" (for a FS) codebase.
Definitely something we should keep an eye on.
Great video. I just got my self a new dell Precision 5750 with two NVMe drives and want to do a system wide raid1 setup on gentoo. Was wondering would people recommend zfs for this type of setup? Am just getting my head into zfs at this point and love any tips. Thanks
Thank you so much. Eye opener!
Very well explained, insightful video. Thank you for this.
Is MS server ReFS, microsofts version of ZFS? I am debating whether to switch from [MS Server 2019] to [TrueNas core or source] - it is simply for file server in Video Production? any recommendations?
I use ZFS with Ubuntu on desktop and laptop. I have a backup server with Pentium 4 HT (3.0 GHz); 1.25 GB DDR (400 MHz); 1.2 TB with 2 IDE HDDs and 2 SATA-1 laptop HDDs. That one has been designed for Microsoft Windows 2000 Pro and Windows 98 and it now runs 32-bits FreeBSD 12.1 with ZFS for say one hour/week.
Here I am almost a year later watching this video. Why? I bought a QNap NAS to play with, and I'll eventually buy another one to back up to. Thank you once again Wendell for putting out these, and I'm now subbed to this channel in addition to the L1T channel. :)
More zfs videos. Can you make a gluster plus zfs video? Or another scalable zfs solution
One of the developer of Free NAS which uses ZFS F/S is adamant about using ECC RAM.with ZFS. Why is ZFS more vulnerable than other File Systems on non-ECC RAM, as opposed to EXT4 or BTRFS which do not necessitate the use of ECC RAM?
I would appreciate a response from the video maker or someone who knows exactly why?
A performance hit for ZFS because of checksumming and other features ZFS has was mentioned. And it said EXT4 was faster. Does that hold true for write speed only or is ZFS slower on read speed as well?
Why can't i stretch the screen to full on my LG V40. Wendell, what did you do. Did you lock the resolution on your video again. I still don't get it. Going full screen on a smartphone shouldn't be blocked when watching any video.
Very interesting explanation.
Would ZFS be a good file system for a DIY NAS with 4+1 drives with 12TB each?
Totally - Been running ZFS as a nas for myself. Work and all my customers. It's virtually bullet proof. I have had corruption once though that lost the entire pool. A poorly designed power supply caused it. Remember it's NOT about the wattage of a PSU it's the amount of power on the rails basically 1200W GOLD was not enough to keep my drives alive. A smaller better spread out 700W had no issues. Most modern PSU's have loads of power for the GPU but not for multiple drives as more people move to SSD's and CLOUD.
@@df3yt Ty for the answer. What would you use as Raid there, something like a Raid 5 with one "spare" HDD? I thought about using a small APU with the upcoming renoir chips since I want to avoid Intel if possible.
Hey Level1! I’m about to be brave and give ZFS a shot! Using 5 drives of which 2 are redundancy. Got 2 NVME drives for cache (Unraid). Can you please give some advice on the file system I should use for my cache? ChatGPT said “the smart people at Unraid picked BTFS for their cache for a reason!” LOL! What do you think, BTFS cache and ZFS HDDs? Any advice would be really appreciated!!!
What is the speed of the BIOS chip vs RAM vs NVMe SSD? What would happen if the flash memory in an SSD was treated as flash memory, and not as a simulated HDD?
is the audio/fps screwy, or something funky going on on my end? strange
ZFS doesn't catch memory corruption though right? For instance If I move a file and it caches in ram, It can be corrupted in ram and then written to disk. ZFS has no idea that happened and neither do I. Short of buying ECC memory what would you recommend?
If you're moving a file the copy will probably be checked against the original before the original gets deleted. It would potentially be a problem if you're writing changes to a file.
If your data is important enough to you that you are worried about memory corruption, spend the money for ECC
ram. It doesn't matter what file system you use, all of them are susceptible to memory corruption.
Love your videos, Wendell! Thanks!
Would it be possible for hard drive manufacturers to make each drive as a RAID 0 were the data would be stripped across the platters?
I have wondered why hard drives cap at 200Mbps whole density kept going up. And since the whole drive dies at the same time, so the drawback of raid 0 doesn’t seem to make a difference.
ZFS is cool, I have contact with Oracle Appliance with a shelf of 400 drives. A metadata server is required for its operation. Added to this is the HSM server and client.
Greetings!