ZFS 101: Leveraging Datasets and Zvols for Better Data Management

แชร์
ฝัง
  • เผยแพร่เมื่อ 15 ต.ค. 2024

ความคิดเห็น • 80

  • @GeoffSeeley
    @GeoffSeeley ปีที่แล้ว +109

    Tom's status: Degraded

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว +85

      Having two arms does offer me some level of redundancy 😜

    • @beardoftears
      @beardoftears ปีที่แล้ว +48

      The Other Arm is not a backup. You need an off-site prosthetic just in case.

    • @Yandarval
      @Yandarval ปีที่แล้ว +11

      However, he looks like he is resilverd.

    • @XtianApi
      @XtianApi 4 วันที่ผ่านมา

      Talk about a slow rebuild

  • @mattwhite532
    @mattwhite532 ปีที่แล้ว +3

    Great video! Thank you for taking time out of your recovery to do this. I don't want to overstep, but as a healthcare professional, I'd be remiss not to suggest getting more of a cushion under the strap, up by your neck. The weight of your arm on your shoulder/neck area over time can cause additional undo pain and discomfort. I'm so grateful to you for your time w/ these vids! Believe it or not, they've saved me from making many silly/stupid tech choices.

  • @mithubopensourcelab482
    @mithubopensourcelab482 ปีที่แล้ว +8

    Wish you a very fast recovery. Your dedication to work is very much appreciated. You are a special person indeed.

  • @ajhieb
    @ajhieb ปีที่แล้ว +5

    I hope you have a speedy recovery and thanks for all the great content!

  • @JohnAngelmo
    @JohnAngelmo ปีที่แล้ว +9

    Thanks for a great video and take care! You will get back on your bike in no time!

  • @olegurvingurigard8373
    @olegurvingurigard8373 ปีที่แล้ว +4

    Hope you get back on the bike quick Tom, I love riding bikes in the woods too.
    Nothing more relaxing, currently running a drz400sm but wouldn't mind a spare set of knobbies for better traction!

    • @zacharysandberg
      @zacharysandberg 5 หลายเดือนก่อน

      DRZs are great bikes! I also had a 400SM

  • @Felix-ve9hs
    @Felix-ve9hs ปีที่แล้ว +3

    I once built a TrueNAS CORE server with all flash storage and used it for testing with four VMware ESXi (one for vCenter, three for VMs).
    It was actually really fast and ended up running rock solid for over 100 days because I upgraded to a never release.

  • @ryanmalone2681
    @ryanmalone2681 5 หลายเดือนก่อน

    Adventure riding has broken a fair few bones of mine too. Hope you’re back on by now. Appreciate all these videos about the TrueNAS concepts as I plan for a migration from Unraid.

  • @itssoaztek4592
    @itssoaztek4592 ปีที่แล้ว +4

    Tom, thank you for yet another great video. Most of what I know about Truenas and ZFS I learned on your YT channel.
    Suggestion for another video: What are recommended steps to test a backup of a system drive? I do not want to overwrite a functioning system drive with a new backup just to test if the backup works fine. Is the only solution to test the backup by writing it to a new physical drive and swapping that drive with the existing system drive?
    Sorry to hear about your unfortunate bike accident. I wish for a speedy recovery!

    • @bikerchrisukk
      @bikerchrisukk ปีที่แล้ว

      Are you talking about a boot drive or zpool?

    • @itssoaztek4592
      @itssoaztek4592 ปีที่แล้ว

      @@bikerchrisukk Sorry, I realize my post was not well written and confusing. I meant it to be a suggestion for Tom to consider making a general video on a proper testing procedure for backups of system drives. Don't know if such a tutorial can be made general enough to cover a variety of systems (TrueNAS systems, system drives of regular PC) or if it would be more advisable to limit the scope and make it for a specific system only.
      In my particular case, I am least worried about the TrueNAS boot drive, slightly more worried about the zpools and most worried about the backup of a Windows PC boot drive. If my thinking is correct, the TrueNAS boot drive can be easily recovered without fear of data loss (and without the need for an additional disk) as long as I make sure to save a configuration file. I have two NAS (one for backups). So I guess I can use the second NAS to store a replicate of a zpool of the first NAS and restore it for testing purposes to a new zpool on the second NAS (provided that the second NAS has sufficient storage capacity). At least in theory. My NAs is fairly new, so I have not actually tried it yet to restore a backup. The system drive on my PC is a much worse case and a totally different story for various reasons. Would be quite a revelation for me to see how people test their backups on a Windows PC.

  • @darylnd
    @darylnd 6 หลายเดือนก่อน

    Great videos, Tom!
    I've been binge watching, including revisiting a few I've seen before. I'm rebuilding my unRAID home server after multiple drive failures (two HDD, one SSD, and the boot USB had to be repaired--so much for the supposed improbability of losing two drives at once) and switching to TrueNAS Scale. I did have some experience with FreeNAS.
    The box itself is modest: it's the first-generation Ryzen 5 I built, years ago, as a desktop system, 16MB RAM (for now). I'll be using four, 4TiB Seagate Ironwolf HDDs in RAID-Z2, and a SATA SSD for boot.There's no NVME, just a m.2 slot that uses one of the six SATA lanes, so no need to spend on that. That leaves one SATA lane open. I'm thinking of pulling the now unnecessary video card, plugging a Sun F80 into that slot, and making a pool of two mirrored stripe VDEVs for virtualization.
    Hope you've healed up by now. Falling off isn't important: getting back on is.

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  6 หลายเดือนก่อน +2

      Thanks, all healed up and riding again!

  • @jayjarrett732
    @jayjarrett732 ปีที่แล้ว +4

    I feel for you bro. I wish you a speedy recovery. Thanks for another great informative video. You are my go too guy!

  • @m3ellis
    @m3ellis ปีที่แล้ว +2

    Oof, been there before as far as the clavicle is concerned (cycling accident). Hope it heals fast. Thanks for all the content 🙏.

  • @MactelecomNetworks
    @MactelecomNetworks ปีที่แล้ว +2

    Get well soon Tom!

  • @77axwell
    @77axwell หลายเดือนก่อน +1

    Was just what I needed. Thank you!

  • @joshua_lee732
    @joshua_lee732 ปีที่แล้ว +7

    Hope your arm is ok and gets better than it was. Have no context on what happened and won't inquire further. I am curious though on how this would be configured via command line rather than TruNAS. I always feel limited when I look at the TruNAS interface.

    • @RaulCastillo
      @RaulCastillo ปีที่แล้ว

      I guess you didn't watch the video

    • @joshua_lee732
      @joshua_lee732 ปีที่แล้ว

      @@RaulCastillo I made the comment in the first 15 seconds.

  • @Tsuroerusu
    @Tsuroerusu 11 หลายเดือนก่อน

    Watching in early November, 2023 and I hope your recovery is coming along well, thanks for the video. Best wishes from Denmark.

  • @nid274
    @nid274 ปีที่แล้ว +5

    great video...and hoping a speedy recovery!

  • @markpreece1565
    @markpreece1565 ปีที่แล้ว +1

    Get well soon !

  • @mattvisaggio
    @mattvisaggio ปีที่แล้ว

    Would love a video for setting up wg-easy for remote access using TrueNAS app but then general next steps with router and phone app to access it. No one has done one yet on YT that I can find...

  • @austinwilson930
    @austinwilson930 ปีที่แล้ว

    Tom, how is your ix_applications dataset encrypted? I had a pool set to be encrypted, and truenas would never encrypt the auto created ix_applications dataset.

  • @prahe86
    @prahe86 11 หลายเดือนก่อน

    Great video, get well soon

  • @3k3k3
    @3k3k3 ปีที่แล้ว +1

    Ouch Tom.. be careful out there, you have a family of 294.000 that is worried about you :)
    What is you take on Scale vs Core, it seems as if Scale is supporting more hardware? But can and should i run it instead of core, and expect it to be just as reliable?

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว +1

      For NAS functions Scale is fine

    • @3k3k3
      @3k3k3 ปีที่แล้ว

      @@LAWRENCESYSTEMS so where does Core fit in?

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว

      @@3k3k3 As a very stable NAS that has been around for a long time.

  • @SnizzleTille
    @SnizzleTille ปีที่แล้ว

    Does compression do something for you if you have a thick provisioned zvol with compression enabled? I tried that a few days ago and I see any advantage in doing so.

    • @entelin
      @entelin ปีที่แล้ว

      Yes, it works, depending on the kind of data you should see some performance improvement on spinning disks. Keep in mind that the filesystem on the zvol will not itself see the saved space, but rather the zfs pool under it.

  • @chinchye
    @chinchye ปีที่แล้ว

    crazy you are still doing videos in sling! Get well soon!

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว +1

      Thanks, I guess you could say I'm doing these videos.. single-handedly 😜

  • @salvanation
    @salvanation ปีที่แล้ว

    Hi
    I am using TrueNas Scale and can you tell something about datasets and the Case Sensitivity option in a next video?
    My client is Windows and because I chose SMB when creating all datasets, all datasets are on case insesitive. Now I would additionally work with Linux and I don't know if I can leave everything as it is or should I better redo all datasets with case sensitive and choose generic as share? Or what happens if I switch completely to Linux and then want to use NFS or should I stay with SMB and make additional settings for SMB so that I can work with windows and linux at the same time?
    The manual also mentions the option "mixed" but in the GUI I can only choose between sensitive and insensitive.

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว +1

      Whenever I use a Dataset for an SMB share I use Case Sensitivity:ON

    • @salvanation
      @salvanation ปีที่แล้ว

      @@LAWRENCESYSTEMS Thx!
      Strange if I create a new dataset and choose Share Typ SMB, the Case Sensitivity is changing to Insesitive and the box is now greyed out?
      Is this a new behavior? Looks like I need to leave the share typ at generic at first to create the dataset.

  • @wpoole10
    @wpoole10 ปีที่แล้ว

    Tom or ANYONE, How can I setup a permissions in a dataset to where I (dad builtin_admin) has full control and want my other users (wife and kid builtin_users) to be able to read and write files if they want, but not be able to delete anyone's files ONLY the ones they created/own? I am able to have each user write files to the dataset but the user that creates the file cannot delete their own file. If I give them modify permissions they can delete anything in the dataset. How do I fix this???

    • @wyattarich
      @wyattarich 10 หลายเดือนก่อน

      look up setuid and setgid bits

  • @gjkrisa
    @gjkrisa ปีที่แล้ว +2

    Never really noticed you had long hair. Kinda look like the Gibson from hackers or his tag the plaque

  • @glmchn
    @glmchn ปีที่แล้ว

    8:37 "we don't need sync on this" please why ? doesn't seems that obvious to me

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว

      Sync is a more complex topic and is covered here th-cam.com/video/fo3EGuCszpE/w-d-xo.html

  • @DarrolKHarris
    @DarrolKHarris ปีที่แล้ว +1

    get better soon

  • @GXShade
    @GXShade ปีที่แล้ว

    I hope recovery goes well :D

  • @Nostang3
    @Nostang3 ปีที่แล้ว

    I broke my collarbone once so I feel your pain.

  • @d3wy
    @d3wy ปีที่แล้ว +2

    Ouch on the arm :(

    • @d3wy
      @d3wy ปีที่แล้ว

      Hope you have a speedy recovery Tom, take your physio seriously.

  • @danielchristie6546
    @danielchristie6546 ปีที่แล้ว +3

    Why are you in a sling tom?

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว +2

      It's explained in the video

    • @NickyNiclas
      @NickyNiclas ปีที่แล้ว +1

      13:27

    • @danielchristie6546
      @danielchristie6546 ปีที่แล้ว

      @@LAWRENCESYSTEMS i know, so funny the way you said that at the start

  • @ewenchan1239
    @ewenchan1239 ปีที่แล้ว +2

    Sorry to hear about your arm. I hope that you will feel better soon and I wish you a speedy recovery!
    Three questions:
    1) If the dataset is used either "by application" or because you want to set different permissions "by dataset" level -- but if you have only one application (dumb file server), for one user, and therefore; one set of permissions -- would I be correct in that use case/usage scenario - datasets wouldn't really offer much in the way of benefits?
    2) If snapshots only record the incremental data, will the total volume of disk space used by the data and its snapshot be equal to each other (i.e. you have initial set of data, take a snapshot, add more data, take another snapshot, therefore; the total volume of disk space used: data + differential = data@snapshot + incremental@snapshot?)
    I'm still a little fuzzy in regards to the comment that you made in the video: "you can create as many of these as you have space for".
    And what I am still unsure of is whether the total volume of disk space consumed would group regardless of whether you're doing snapshotting or not, or if there is a little bit more of an "overhead" in terms of the total volume of disk space consumed would be a little bit more than the data itself.
    Your help in clarifying that is greatly appreciated.
    3) If I have a ZFS dataset - how would I back that up to LTO-8 tape?
    Thank you.

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว +1

      1) if you don't need them then don't use them they're not required if you don't have a use case.
      2) correct that snapshots are only the difference from the previous snapshot therefore only take up that much data. Creating data sets does take up a few kilobytes which means you could create several thousand of them without a problem on a reasonably sized system
      3) I'm not aware of anything in ZFS that automatically backs up to lto

    • @ewenchan1239
      @ewenchan1239 ปีที่แล้ว

      ​@@LAWRENCESYSTEMS
      Thank you.
      1) Some people were trying to educate me about the benefits that datasets brings beyond those that you've mentioned in your video, so I was looking to get your insights/perspectives in regards to this kind of a recommendation.
      2) Thank you. This is good information to have and know.
      3) I haven't really figured out a particularly great way to be able to backup block devices (e.g. zvols) to tape because the files aren't exposed to the TrueNAS host (unlike datasets, because I can see the files/folders in a dataset, on the host system).
      Right now, I am just using rsync to send the files/folders over to tape (over NFS), but that's causing a high CPU load average, so I didn't know if there might be a better way to do this vs. my current process.

  • @randleqgod
    @randleqgod ปีที่แล้ว +1

    Get better. Saw your LinkedIn post. Definitely a freak accident.

  • @Mikesco3
    @Mikesco3 ปีที่แล้ว

    I wonder if any of your clients freak out whenever you have an accident 😂

  • @dooetboog
    @dooetboog ปีที่แล้ว +2

    get well quick... resilver quick.

  • @jGRite
    @jGRite ปีที่แล้ว

    Well, at least you messed you up, and no one else messed you up.

  • @eidodoos
    @eidodoos ปีที่แล้ว +2

    he unmount his hand. hope he can mount it back sooner.

  • @rpWeb3
    @rpWeb3 ปีที่แล้ว

    I also broke my collar bone on my kx125 .. good luck sleeping.. ugh

  • @VisionNick-n5f
    @VisionNick-n5f หลายเดือนก่อน

    Wilson Michelle Martinez Linda Martin Frank

  • @richardbennett4365
    @richardbennett4365 ปีที่แล้ว

    ZFS is a ROW System not a 🐮 cow.

    • @LAWRENCESYSTEMS
      @LAWRENCESYSTEMS  ปีที่แล้ว

      If you're looking for a more specific answer it's a COW for general writing and a ROW for snapshots

    • @richardbennett4365
      @richardbennett4365 ปีที่แล้ว

      @@LAWRENCESYSTEMS, copy-on-write requires three I/Os per write; however, and it is why btrfs and ZFS are redirect on write systems, only performs one. These systems are trying to save writing, so that's why they redirect on write instead of copying on write.

  • @justisstudding2101
    @justisstudding2101 ปีที่แล้ว +1

    I was just recently involved in a Hit n Run on my bike coming home from work. Broad side. Smashed my left foot. Broken Heel, Ankle, and now the Ankle is having serious healing issues. I have to return to get antibiotics, pain meds. Driver is still unknown. Good news though. I got my equipment in the post office. New drivws to set up my TrueNAS Scale. I'm excited...

    • @bikerchrisukk
      @bikerchrisukk ปีที่แล้ว

      Sorry to hear that and hope you recover eventually (it can be a long road, but stick with it, do physio, etc.)

    • @justisstudding2101
      @justisstudding2101 ปีที่แล้ว

      @@bikerchrisukk thanks.