DUDE I have used Linux off and on for a decade, and started daily driving on all my machines a year ago. I'm glad I'm still finding cool stuff like this I didn't know. Thanks!
Hi Chris. An addition ere: LVM doesn't necessarily have to use partitions. It can be built on any set of any block devices: partitions, raw drives, RAID devices, files, memory, etc. Things can go really crazy with it.
So if I have a second hard drive that has some data on it but I want to combine it with another one will that data be lost when I add it to a new group
@@InspiredInsights4U Yes. Basically, when you extend a partition, you'll also extend a filesystem to that drive, which practically means reformatting it.
Regarding the drawbacks you mentioned about LVM, one way to deal with them is to have a RAID layer below LVM. That way you used the logical drives from the RAID configuration instead of using "real" drives as PVs. Its very hard to lose data that way.
Really great video, Chris. I had largely avoided lvm, because I hadn't seen it all laid out in one place and wasn't exactly sure I knew what I would be doing. This makes the process accessible. Thanks much!
I watched. Wow, I wish you would have been the Instructor that I had during one of the early Linux classes. How you explain it and simplify it out, is what more Instructors need to do. Also, the split screen -- with the commands in the background -- is incredibly helpful; also, as you work in the Terminal windows, is what I wish would have been presented when I was in courses for this. It would have helped drive it home. Most of our stuff was just walk through some tutorial steps, and it really didn't deepen the knowledge. It was rote learning, instead of "real learning" like this. Now, I feel more comfortable with disk organization. [Also, I plan on becoming a Patreon too for you.]
Thanks Chris for teaching me how to extend the LVM. I knew how to setup LVM's but never knew how to extend them with more drives. Another good thing about LVM are creating snapshots using Timeshift. If you have LVM configured using BTRFS it is super powerfull with creating snapshots.
Thank you very much for this, I've come back to this multiple times in the last years. It has made my life so much more easier. Also thanks for your other vids, they are really a great help.
Passing -r to the lvextend command will automatically resize the FS for the extended LV. So, in this case it would be "lvextend -l +100%FREE -r /dev/vg1/lv1" so then you don't need the final resize2fs call. Great video, thanks!
Super helpful comments section to accompany a pretty fine video 👍 1. Pass -r to lv extend; elim. resize2fs 2. Pvcreate doesn't need a f/s prerequisite or even a partition. Indeed pvcreate can ingest any block device such as a disk (partitioned or raw) or a partition (whether formatted with a f/s or not). Kindest regards, neighbours.
Three weeks ago I went totally Ubuntu. During installation I was offered to install using the LVM system. No idea what that was. But I said yes anyway.
This is how most people are introduced to it hehe. I know my introduction was doing a fedora install and I went "WTF... my home doesn't have all the space of my 120 GB drive!"
To this day I could not see it's use. I typically use one partition for /root and one for /home and the installation offered to set up lvm. No other options. Maybe make the installation offer to set up disc caching or some raid, but it just did lvm and I could not see any benefit of it, It's a click and forget thing. I assumed you could do something cool with it but ultimately when I wanted to have a system ssd and a 2x hdd raid 0 /home i went with mdadm because lvm is explained literally nowhere at all. Pointless to add it to the install and explain and do nothing with it.
@@wujekcientariposta Volume management doesn't have a lot of use for home users, but is very handy in an enterprise environment. For example, database files and their indexes can be separated into different logical volume groups and extended as required. System log files grow over time and if left unattended can cause issues. Being able to quickly extend a volume can help. Etc...
It's nice video about LVM. In enterprise area for example at IBM where I've worked we used LVM on AIX and Linux servers also. Not just at IBM. Pretty easy to manage filesystems in this way. If you have a database or a log filesystem and not possible to delete anything, then LVM is a lifesaver for that DB or application. I save your guide. This is a very well documentation for linux. Thanks for!
A few days I bought a Samsung SSD of 1TB for games specifically and other files as well. Thanks for making this video and keep up the good work Uncle Chris.
I would just mount it into the file system as Chris has done with his 2tb drive for the same purpose. You don't need LVM for that. In fact it just complicates things.
You have to use LVM if you do full disk encryption. If you use that, knowing these commands can help you get into your data if grub doesn't boot since you have to mount the LVMs to get in.
Since when? I've always done it but you should be able to use LUKS without an LVM partition. It might be more difficult if you have multiple partitions, but if you just had: /boot/efi -> EFI System Partition /boot -> Boot partition / -> Root partition It should work exactly like LVM on LUKS. The Arch wiki even has instructions for LUKS on a partition.
Oh man, doing full disk encryption on a bootable computer without using the installer or without LVM would be a pain. By putting it all under the LVM you can be sure your swap, home and root directories are encrypted. However, you are right that you can use LUKS on a drive, external drive, USB, etc., and I do that all the time for encrypted backups. Works really well.
Smac You think that's a pain? I had a setup like this once on a laptop. If you booted, it would load Windows. But if you booted with a USB drive I kept on my Keychain, it would boot into the rEFInd bootloader (because it's just better than Grub in every way IMO), and then if you selected Arch it would prompt for a LUKS password, it would use this password to decrypt an 8192 KB LUKS encrypted key file, and then use the decrypted key file and an external LUKS header to decrypt the LVM partition, then mount the USB drive at /boot, and mount the volumes. Then I could unplug the USB drive. The encrypted key file gave two factor authentication (something I had, the USB drive, something I knew, the password for the key file), and the LUKS header being external gave plausible deniability, as without it you couldn't prove the encrypted partition was encrypted and not just unallocated. That took me maybe a week to figure out, because I had to modify Arch's encrypt hook to handle it. It was the first "bare metal" Linux install I ever did, and I did it to see if I could after finding a guide to do a similar setup on Gentoo. I had a little shell script to automate the install, so I'd basically put that on the Arch live USB, move it to the laptop, run the script and reboot, if it didn't work, edit the script and start over. I still have that script on my Nextcloud I think.
@@praetorxyn I wanted to set something like that up. It really is super security. I wonder to what degree you can even tell a partition is there without the boot loader, is it entirely random without any headers? Thats good plausible deniability.
Smac As far as I understand it, I think you can tell it's a partition, if I remember right, but with an internal header you can tell it's encrypted. With an external header, there's no proof it's encrypted and not just empty space. I got the idea after reading about a situation where a court ordered a woman to decrypt something, but they were able to do it because they had a recording of her telling somebody that something they were looking for was in the encrypted place. Something like that. If they can't prove that a specific thing they are looking for is on your encrypted device, they can't order you to decrypt it. Sadly, this is all theoretical. In reality, it's more like this: xkcd.com/538/
Wow, you're a lifesaver; thanks for making this video! I had like 200MB left in my /home directory and This was my only viable solution. I am not a storage expert and do not do this stuff on a daily, weekly, or even yearly basis.
Great video Chris, used LVM about 10 years ago when I first found Linux. Also loving the System 47 LCARS screensaver in the background rather than both doing Cmatrix.
In businesses LVM is great for storing evergrowing data that can't be easily protected by the filesystem. For example a database that gets replicated to a different host. If a drive fails, you switch the production traffic to the replica and just recreate the main db host after replacing the faulty drive. This setup allows for easy expansion of space for the db while ensuring high availability. Kind of a standard practice for reliability on the cheap.
You can resize btrfs filesystem on the fly too. You just need to use "btrfs filesystem resize max /mnt/xxx" (in most common configuration) or "btrfs filesystem resize :max /mnt/xxx" (if you spread the filesystem over multiple devices). It's safe to continue using the filesystem while it's being resized. Unlike most other filesystem resizing, btrfs can only be resized while mounted (i.e. it's always online resizing). And unlike resizing ext4 partition which only supports online extension, both shrinking and extending of btrfs can be done online.
I love your channel your tutorials have helped me immensely. I'm currently working as a support tech fresh out of college just switched over to fully over to linux for work (using Ubuntu) haven't looked back XD
The way you've explained it is analogous to using LVM as a software RAID 0. There are other options though, effectively similar to normal RAID. As a sample of where it gets used extensively: Synology and QNap use LVM to define their "RAID" setups. When you delve into the finer details you'll note they also define physical and logical volumes, even for their RAID 5/6 setups. Just like a RAID 0 is hardly ever recommended, I would also never recommend a LVM volume simply spanning across several disks. Same reason, if any one of those disks develop a problem you loose all of the data across all of them. I would definitely suggest anyone attempting LVM on data they don't absolutely hate, to look at lvmraid and adding parity to your LVM groups / volumes. And even after that, make backups, RAID (not even RAID 1) is also no substitute for a true backup.
I likely misunderstood you when you said LVM is not used much. Actually. Many distros use LVM out of the box. Also, we've used LVM in the Enterprise for years. Great when you have planned ahead and need to grow LV in an emergency. Though I do miss ZFS from Solaris days. Now that was easy to use.
These days, I almost demand businesses store everything in ZFS. The only other storage I find acceptable is RAID 10, but there are a lot of requirements for RAID to be used, such as a separate controller that is a brand name and I buy everything in duplicate. Call me paranoid, but I've done several disaster recoveries and I just don't mess around with the less reliable stuff these days.
Hey the screen looks like my Commodore 128 :) Very useful video depending on what you want to do with a given computer. I found myself watching a couple of times since im experimenting with linux. Thank you.
Hi Chris, great tutorial on LVM. It would be great to see you make a part2 that would cover resizing logical volumes or replacing a physical volume. One remark: you formatted the partition with ext4 which is not needed I think because the FS of the logical volume will be used; pv create also says it is wiping the ext4 headers (and making it an LVM partition type)
When setting up the partition, you need to identify the filesystem type it is to house. Just part of the required partition meta-data. This is then happily overridden by the LVM tools with the real filesystem.
@@dingokidneys That is incorrect, the only reason Chris needed to specify the filesystem is because he's using Gparted, which is a high level tool that combines both partition management and filesystem management. If you used a lower level partitioning tool like fdisk, then you would not need to set a filesystem. Also, AFAIK filesystem type isn't part of partition metadata, but rather filesystem metadata.
There is no need to run resize2fs after extending a logical volume. You can just add -r or --resizefs to the parameters of lvextend and then the resizing of the filesystem is done automatically. you eiter use mdadm to set up some kind of raid like raid 5 with 3 to 5 maybe six disks and raid 6 with more than 6 disks. Then you make physical volumes from the /dev/md device. If you have 2 disks you can do mirroring using lvm. LVM is used a lot in enterprise setups especially when the storage comes from some kind of NAS/SAN and is shared to VM's. It allows one to resize the storagage assigned to the vm host and then be flexible with the storage in the VM's. Resizing is used a lot; destroy and recreate is not used a lot; the basic rule is that data is only moved when there is no other way to do it. And if you need to move data you can always add a new volume to the volume group and then use pvmove to move the chunks of that volume group to the new volume and emptying the old disks. This can be done on-line without downtime. I'm sure stuff like this can be done with ZFS but ZFS is a userland process that can have serious performance problems. Also ZFS has some license issues with linux so it cannot be part of the kernel unlike lvm,
Very well explained @Chris Titus Tech, that was just what I was looking for, thanks a lot for the effort! Just one doubt though, supposing I have a LVM, with multiple logical partitions, composed by 3 disks. If somehow I loose my motherboard or even one of the drives. How am I supposed to recover my files? I mean, what I should expect by plugging the two other drives in another PC setup?
Some small corrections. In partitioning the physical hd, ext4 is irrelevant. LVM needs the physical partitioned drive to be "lvm2 pv" on gparted and not "ext4". Also, now you do not need to use the pvcreate command because it just converted "ext4" to "lvm2 pv." So after gpated partitioned sdb1 to "lvm2 pv only type the following:" sudo vgextend Fedora_localhost-live /dev/sdc1 Note now "home" volume is an actual file on dev directory which is located in "/dev/Fedora_localhost-live/home." The "lvscan" command shows all the volumes. So more precise use of lvextend would be: sudo lvextend -l +100%FREE /dev/Fedora_localhost-live/home resize2fs /dev/Fedora_localhost-live/home Thanks for introducing lvm very to me as I need it for home, and for sure not for work ))))
@@ChrisTitusTech none of these are backups - just saying. might make your data last longer, but u r only putting of the innevitable loss - th-cam.com/video/l7-6m2cE6JM/w-d-xo.html
Hi, nice info! I was just about to add another disk to my system. I have some questions: 1. Can I mix SSD and HDD? Suppose that I already have a LVM partition on a SSD, can I extend the volume with the new added HDD? 2. If (1.) is possible, is it safe to do so? Any disadvantage of mixing drives? thanks ^^
I figured out that you dont have to create a single large partition with a filesystem in it, you can just use sdb as a whole, you dont even need to create a partition table. Thats why it says "1 disk" at 6:25. I also found it strange that you created a filesystem within sdb1 since thats something one would do with the devfile of the logical volume. Thats why it asks "Wipe it" at 9:20, after that there is no file system anymore. With btrfs you can also add whole disks to a file system, thats the reason why i tested this with lvm. Still, thanks for the video.
I used to do the same thing. I would LVM everything including my backup folder which I symlinked to every reinstall. That is untill I bought an M.2 drive and suddenly half my sata connections disappeared including my backup LVM partition which was made up of three hard drives. At first I didn't know what had happened so I reluctantly erased and reinstalled my backup drive. I haven't used LVM since. But still I love LVM.Could you do a part explaining the snapshot process. I watched multiple videos explaining it but I still haven't got a clue what they were on about. Great video Thanks
on many motherboards, installing a m.2 drive will disable at least 2 sata ports. depends on which m.2 port you put it in. most boards have 1 m.2 wired to the cpu, and any more wired through the chipset, along with most or all the sata ports. if you install a m.2 into the slot wired to the chipset, sata lanes will be disabled. this is most likely why your drives disappeared.
What is the difference between raid0 and LVM (in case if you lose a physical disk) will you lose all data? I used to use LVM a lot. But this thing is really bothering me
Yes, in this LVM configuration. Lose one disk and all data is gone. Backup! However you can use RAID with/within LVM for data security. You just need to set it up correctly from the start.
Nice quick tutorial. Starting a project to implement it on a distributed file system like lizardfs or glusterfs (not decided yet) with a dozen of pies with 10GB drives.
My daily driver for the past 10 years or so has been FreeBSD, mainly because it has zfs volume management which is way more reliable than lvm. How are the equivalents - btrfs & zfs on linux getting on. Any time I've looked, it has been "experimental, should be ready soon", which is not something I would want to trust my data to. At the moment I have 4 x 10TB drives on my main machine giving me 25TB of usable space, and 4 x 6TB drives on my test-bed machine giving me 15TB of usable space.
New comment /old vid... this did not work entirely as directed. Somewhere around "lvextend -l /dev/ sd#" the world essploded....but as always great tutorial.
Chris, Isn't more secure to use a raid configuration that will also take care of grouping different hard disks? Of course that limits you to same size hard drives (unless you accept to loose some space on larger HDs) Is it possible to implement a sort of Raid on top of LVM?
LVM a solution that was modern 10 years ago. I prefer ZFS (or even BTRFS) now, the concepts and the 2 commands are simpler and it would also improve the throughput of your disks by striping the data over those disks/partitions. It has a lot of other advantages like compression, encryption, raid-configurations, snapshots, clones, built-in (incremental) backups, memory and SSD caching too. Another great concept are the properties for each dataset (main folders), you can use it to define/change compression method, network file sharing (Windows or NFS), mount points, read-only, executables allowed, quotas (maximum allowed space), reservations (minimum reserved space), copies (kind of mirroring of that folder only), etc.
@@ChrisTitusTech Once in my Life I used LVM in ~2005. I had a Pentium II (400MHz) with two 2 GB disks lying around and I wanted to install Ubuntu 5.04 from a PC Magazine CD. However Ubuntu needed more than 2 GB. That was the only time, I used LVM and it was also my first look at Linux :) :) I remembered that Ubuntu experience, when I got annoyed in 2008 by a new dual core Dell laptop with a very slow disk, desperately trying to run the original Windows Vista without the service packs.
@@bertnijhof5413 Thank you for the comments 👍 esp. ZFS being preferable to LVM. Here in 2022, it seems like ZFS is still a top choice. Also some folks are looking into Longhorn's potential. Kindest regards, neighbours and friends.
Chris if we installed ubuntu without LVM and just with standard installation, than how can we do it from there, In this video you have already setup LVM and taught us that how to add and increase space, but in case we installed without LVM support, than how can we do that. Please
This is good. I was trying to do that using KDE partition manager or even with Gparted, and I could not do it. Only this worked. Why though? Thank you!
Thanks for this video. However in my understanding the partion type of the physical volume has no influence on "extend" option. Even more, You don't need to put an the partition on the disk at all. However the filesystem on the logical volume should be ext4.
I am a bit curious, what happens if I use USB stick as LVM extension on a laptop and turn on the laptop while accidentally removed the USB? Will the machine boot normally after I plug the USB back and reboot?
So volume group acts as a virtual drive, composed out of physical partitions? Say, can I use LVM over some of the physical partitions while keeping others "close to metal"?
Check out "$ man inode" or a simple google search will satisfy all you need to know. Basically, it's just the building block of the file structure. It holds data about the file it points to such as filename, ownership, permissions, where to find the data blocks, etc.
GParted: In your video I saw that you used GParted to create a partition with an ext4 file system, then immediately overwrote it with pvcreate. Note that GParted can create an empty partition by selecting "unformatted" file system. Also GParted can create a PV by selecting "lvm2 pv" file system. However as you video is concentrating on using LVM on the command line the later is probably not what you want in this case.
hi Chris, good content !! i think zfs its the way to go ?! it is evolution of LVM, zfs combine LVM and partition tools, its very powerfull, already use with freenas box. I think the evolution path is LVM, XFS and now ZFS ... i ve miss someting ? thanks to put me on the right way !? :)
ZFS is great with the down-side that most Linux distros will not boot from a ZFS drive. You would have to compile a kernel yourself with ZFS support in-built. Ubuntu is to my knowledge the only distro which will boot from ZFS.
LVM + XFS is VERY handy for cloning your boot drive to install a new larger boot drive. Run CLonezilla(or use DiskDestroyer if you dare) to easy clone your drive, then grow the partition to take up the free space.
Thanks for sharing this! Could you make a video on how to configure LVM on Proxmox VE? Just as a suggestion, I think it would be nice, what do you think?
I've always used LVM on single drives. There's no reason not to when you allocate X for each volume, then you have Y unallocated space you can allocate to grow / shrink your volumes as needed.
On a single disk I don't see the point of using LVM (except for full disk encryption) when you can just create one partition and put everything into it. You don't have to worry about growing/shrinking volumes. As far as I can see, it's only when using several physical drives that this really comes into it's own.
@@dingokidneys I've always used full disk encryption too. If I wasn't using LVM I'd at least have a separate /home partition, maybe a swap partition as well. Once, just to see if I could, I got a Windows / Arch dual boot working like this: I had a USB drive that would be mounted at /boot with an external LUKS header for the LVM partition, and a 8192 KB LUKS encrypted key file. I modified the encrypt hook so that at boot, it worked like thjs: If the USB drive wasn't inserted, you got Windows. If it was, you got rEFInd (because I've never liked Grub, and think rEFInd with a theme looks way nicer and it's easier to configure and fix) and if you selected Arch, it would prompt you for the password to decrypt the LUKS encrypted key file, then used the decrypted key file and the external LUKS header to decrypt the LVM partition and mount the volumes as normal. The key file and external /boot gave two factor authentication, and the external LUKS header gave plausible deniability because you couldn't prove it was an encrypted partition instead of unallocated space.
Now I have spent a month in Ubuntu. The only thing that went wrong once was when I rebooted and the reboot hung with a blank screen and my OS never appeared. So I put in a Linux live disk and, weirdly, it said that there was no prior OS there. Even though if I ran gparted there absolutely was my OS still there! I had set up as an LVM system install, something that I had never done before. I tried boot repair. That didn't change anything. So in the end I did a total reinstall of Linux. This time I used a normal Linux filesystem. There was no issue rebooting at all. So I think that LVM just does not work on my computer which is a bit old [2011]. Despite this I am happy that I went Ubuntu. More chance of joining the Moonies than of going back to Windows.
the disadvantage (among others) is that, for system partitions (/ or /home), you have to setup the group/volumes at system install time. also why sudo su? su should be enough...
If you use "su" instead of "sudo su", you need to know the root password. Also, I would use "sudo su -" which brings you to root in a login shell (which means that you are placed in the root home directory (instead of the home directory of the invoking user) and that the shell login scripts will be executed).
next time you forget sudo just hit: 'sudo !!' after , it will execute previous cmd with sudo ,)
I thought you were jocking but i tried it anyway. It was quite a surprise. Thanks for the tip. lol
If you are in zsh shell, press esc 2x do the same thing.
Nice feature btw
200 IQ ❤
DUDE
I have used Linux off and on for a decade, and started daily driving on all my machines a year ago. I'm glad I'm still finding cool stuff like this I didn't know. Thanks!
Bruh. U saved me hours.
I came across this video after hours of searching. This has saved me a lot more hours. Excellent. Thank You!!
Hi Chris. An addition ere: LVM doesn't necessarily have to use partitions. It can be built on any set of any block devices: partitions, raw drives, RAID devices, files, memory, etc. Things can go really crazy with it.
So if I have a second hard drive that has some data on it but I want to combine it with another one will that data be lost when I add it to a new group
@@InspiredInsights4U Yes. Basically, when you extend a partition, you'll also extend a filesystem to that drive, which practically means reformatting it.
Regarding the drawbacks you mentioned about LVM, one way to deal with them is to have a RAID layer below LVM. That way you used the logical drives from the RAID configuration instead of using "real" drives as PVs. Its very hard to lose data that way.
No need to run resize2fs. You can just give -r parameter to lvextend which will do the filesystem resizing automatically. Anyway, nice video :)
I'd prefer it's separate as one has to be careful messing with storage.
Really great video, Chris. I had largely avoided lvm, because I hadn't seen it all laid out in one place and wasn't exactly sure I knew what I would be doing. This makes the process accessible. Thanks much!
First 5 minutes are all I needed to understand in a simple way the whole point of LVM. Thanks
And I was avoiding LVM as I couldn't figure out how it works :D
Now waiting for BTRFS video
I watched. Wow, I wish you would have been the Instructor that I had during one of the early Linux classes. How you explain it and simplify it out, is what more Instructors need to do. Also, the split screen -- with the commands in the background -- is incredibly helpful; also, as you work in the Terminal windows, is what I wish would have been presented when I was in courses for this. It would have helped drive it home. Most of our stuff was just walk through some tutorial steps, and it really didn't deepen the knowledge. It was rote learning, instead of "real learning" like this. Now, I feel more comfortable with disk organization. [Also, I plan on becoming a Patreon too for you.]
Thanks Chris for teaching me how to extend the LVM. I knew how to setup LVM's but never knew how to extend them with more drives. Another good thing about LVM are creating snapshots using Timeshift. If you have LVM configured using BTRFS it is super powerfull with creating snapshots.
Thank you very much for this, I've come back to this multiple times in the last years. It has made my life so much more easier. Also thanks for your other vids, they are really a great help.
Passing -r to the lvextend command will automatically resize the FS for the extended LV. So, in this case it would be "lvextend -l +100%FREE -r /dev/vg1/lv1" so then you don't need the final resize2fs call. Great video, thanks!
LVM rules....but I don't use it so often I always need to re-learn from the scratch :D Thank you for the great video. It is very helpful.
I was just wondering all about LVM over the last few days! Thanks for reading my mind.
Super helpful comments section to accompany a pretty fine video 👍
1. Pass -r to lv extend; elim. resize2fs
2. Pvcreate doesn't need a f/s prerequisite or even a partition. Indeed pvcreate can ingest any block device such as a disk (partitioned or raw) or a partition (whether formatted with a f/s or not).
Kindest regards, neighbours.
Three weeks ago I went totally Ubuntu. During installation I was offered to install using the LVM system. No idea what that was. But I said yes anyway.
This is how most people are introduced to it hehe. I know my introduction was doing a fedora install and I went "WTF... my home doesn't have all the space of my 120 GB drive!"
@@ChrisTitusTech I have tried some of your commands in my ubuntu terminal. Looks like I have one massive partition of 2 TB ;)
To this day I could not see it's use. I typically use one partition for /root and one for /home and the installation offered to set up lvm. No other options. Maybe make the installation offer to set up disc caching or some raid, but it just did lvm and I could not see any benefit of it, It's a click and forget thing. I assumed you could do something cool with it but ultimately when I wanted to have a system ssd and a 2x hdd raid 0 /home i went with mdadm because lvm is explained literally nowhere at all. Pointless to add it to the install and explain and do nothing with it.
@@wujekcientariposta Volume management doesn't have a lot of use for home users, but is very handy in an enterprise environment. For example, database files and their indexes can be separated into different logical volume groups and extended as required. System log files grow over time and if left unattended can cause issues. Being able to quickly extend a volume can help. Etc...
@@ChrisTitusTech You remind me of Saul Goodman.
It's nice video about LVM. In enterprise area for example at IBM where I've worked we used LVM on AIX and Linux servers also. Not just at IBM. Pretty easy to manage filesystems in this way. If you have a database or a log filesystem and not possible to delete anything, then LVM is a lifesaver for that DB or application.
I save your guide. This is a very well documentation for linux.
Thanks for!
This vid is way above my GNU/Linux grade, but I still enjoyed it. Thanks for sharing Chris Titus Tech.
Thank you Chris! This was an EXCELLENT walkthrough!
A few days I bought a Samsung SSD of 1TB for games specifically and other files as well. Thanks for making this video and keep up the good work Uncle Chris.
I would just mount it into the file system as Chris has done with his 2tb drive for the same purpose. You don't need LVM for that. In fact it just complicates things.
This is by far the best explanation of LVM. thx!
You have to use LVM if you do full disk encryption. If you use that, knowing these commands can help you get into your data if grub doesn't boot since you have to mount the LVMs to get in.
Since when? I've always done it but you should be able to use LUKS without an LVM partition. It might be more difficult if you have multiple partitions, but if you just had:
/boot/efi -> EFI System Partition
/boot -> Boot partition
/ -> Root partition
It should work exactly like LVM on LUKS. The Arch wiki even has instructions for LUKS on a partition.
Oh man, doing full disk encryption on a bootable computer without using the installer or without LVM would be a pain. By putting it all under the LVM you can be sure your swap, home and root directories are encrypted.
However, you are right that you can use LUKS on a drive, external drive, USB, etc., and I do that all the time for encrypted backups. Works really well.
Smac You think that's a pain? I had a setup like this once on a laptop.
If you booted, it would load Windows. But if you booted with a USB drive I kept on my Keychain, it would boot into the rEFInd bootloader (because it's just better than Grub in every way IMO), and then if you selected Arch it would prompt for a LUKS password, it would use this password to decrypt an 8192 KB LUKS encrypted key file, and then use the decrypted key file and an external LUKS header to decrypt the LVM partition, then mount the USB drive at /boot, and mount the volumes. Then I could unplug the USB drive.
The encrypted key file gave two factor authentication (something I had, the USB drive, something I knew, the password for the key file), and the LUKS header being external gave plausible deniability, as without it you couldn't prove the encrypted partition was encrypted and not just unallocated.
That took me maybe a week to figure out, because I had to modify Arch's encrypt hook to handle it. It was the first "bare metal" Linux install I ever did, and I did it to see if I could after finding a guide to do a similar setup on Gentoo.
I had a little shell script to automate the install, so I'd basically put that on the Arch live USB, move it to the laptop, run the script and reboot, if it didn't work, edit the script and start over. I still have that script on my Nextcloud I think.
@@praetorxyn
I wanted to set something like that up. It really is super security. I wonder to what degree you can even tell a partition is there without the boot loader, is it entirely random without any headers? Thats good plausible deniability.
Smac As far as I understand it, I think you can tell it's a partition, if I remember right, but with an internal header you can tell it's encrypted. With an external header, there's no proof it's encrypted and not just empty space.
I got the idea after reading about a situation where a court ordered a woman to decrypt something, but they were able to do it because they had a recording of her telling somebody that something they were looking for was in the encrypted place. Something like that. If they can't prove that a specific thing they are looking for is on your encrypted device, they can't order you to decrypt it.
Sadly, this is all theoretical. In reality, it's more like this:
xkcd.com/538/
10/10 best explaination i've seen of this. Thank you!
Thank you very much, this is the easiest to follow video on this topic
That is the best tutorial of a linux command-set I have ever EVER seen. Subscribing Now.
Wow, you're a lifesaver; thanks for making this video! I had like 200MB left in my /home directory and This was my only viable solution. I am not a storage expert and do not do this stuff on a daily, weekly, or even yearly basis.
Great video Chris, used LVM about 10 years ago when I first found Linux. Also loving the System 47 LCARS screensaver in the background rather than both doing Cmatrix.
Yeah man, gotta say you know how to simplify and relay info to the layman. Nicely done.
Cristal clear and Excellent explanation . THANK YOU
In businesses LVM is great for storing evergrowing data that can't be easily protected by the filesystem. For example a database that gets replicated to a different host. If a drive fails, you switch the production traffic to the replica and just recreate the main db host after replacing the faulty drive. This setup allows for easy expansion of space for the db while ensuring high availability. Kind of a standard practice for reliability on the cheap.
You can resize btrfs filesystem on the fly too. You just need to use "btrfs filesystem resize max /mnt/xxx" (in most common configuration) or "btrfs filesystem resize :max /mnt/xxx" (if you spread the filesystem over multiple devices). It's safe to continue using the filesystem while it's being resized.
Unlike most other filesystem resizing, btrfs can only be resized while mounted (i.e. it's always online resizing). And unlike resizing ext4 partition which only supports online extension, both shrinking and extending of btrfs can be done online.
LVM one of the best tools ever made!! Easy mirrors, easy volume expansion!
I love your channel your tutorials have helped me immensely. I'm currently working as a support tech fresh out of college just switched over to fully over to linux for work (using Ubuntu) haven't looked back XD
The way you've explained it is analogous to using LVM as a software RAID 0. There are other options though, effectively similar to normal RAID. As a sample of where it gets used extensively: Synology and QNap use LVM to define their "RAID" setups. When you delve into the finer details you'll note they also define physical and logical volumes, even for their RAID 5/6 setups.
Just like a RAID 0 is hardly ever recommended, I would also never recommend a LVM volume simply spanning across several disks. Same reason, if any one of those disks develop a problem you loose all of the data across all of them. I would definitely suggest anyone attempting LVM on data they don't absolutely hate, to look at lvmraid and adding parity to your LVM groups / volumes. And even after that, make backups, RAID (not even RAID 1) is also no substitute for a true backup.
Hi, thank you for this really good and simple guide on using LVM it helped me setup my system the way I want it to be.
Thank you, Chris. Excellent topic.
I likely misunderstood you when you said LVM is not used much. Actually. Many distros use LVM out of the box. Also, we've used LVM in the Enterprise for years. Great when you have planned ahead and need to grow LV in an emergency. Though I do miss ZFS from Solaris days. Now that was easy to use.
These days, I almost demand businesses store everything in ZFS. The only other storage I find acceptable is RAID 10, but there are a lot of requirements for RAID to be used, such as a separate controller that is a brand name and I buy everything in duplicate. Call me paranoid, but I've done several disaster recoveries and I just don't mess around with the less reliable stuff these days.
Can't believe the difference in audio quality from the early days, so crisp and clear now!
Thanks for your explaination. I'm not English native but your speak easy to listen.
Thank you. I've been struggling with this topic for 3 years, but never give a hand. Perhaps I'm on vacation now. I'll definitely try this soon
Hey the screen looks like my Commodore 128 :) Very useful video depending on what you want to do with a given computer. I found myself watching a couple of times since im experimenting with linux. Thank you.
This was the best explanation I've seen
Hi Chris, great tutorial on LVM. It would be great to see you make a part2 that would cover resizing logical volumes or replacing a physical volume. One remark: you formatted the partition with ext4 which is not needed I think because the FS of the logical volume will be used; pv create also says it is wiping the ext4 headers (and making it an LVM partition type)
When setting up the partition, you need to identify the filesystem type it is to house. Just part of the required partition meta-data. This is then happily overridden by the LVM tools with the real filesystem.
@@dingokidneys That is incorrect, the only reason Chris needed to specify the filesystem is because he's using Gparted, which is a high level tool that combines both partition management and filesystem management.
If you used a lower level partitioning tool like fdisk, then you would not need to set a filesystem.
Also, AFAIK filesystem type isn't part of partition metadata, but rather filesystem metadata.
@@yvrelna helpful 👍 thx.
Oh man, i wouldnt dare do this on a live box. It takes some practice to be sure of what im doing. Great vid though as always 👍
Z
There is no need to run resize2fs after extending a logical volume. You can just add -r or --resizefs to the parameters of lvextend and then the resizing of the filesystem is done automatically.
you eiter use mdadm to set up some kind of raid like raid 5 with 3 to 5 maybe six disks and raid 6 with more than 6 disks. Then you make physical volumes from the /dev/md device. If you have 2 disks you can do mirroring using lvm. LVM is used a lot in enterprise setups especially when the storage comes from some kind of NAS/SAN and is shared to VM's. It allows one to resize the storagage assigned to the vm host and then be flexible with the storage in the VM's. Resizing is used a lot; destroy and recreate is not used a lot; the basic rule is that data is only moved when there is no other way to do it.
And if you need to move data you can always add a new volume to the volume group and then use pvmove to move the chunks of that volume group to the new volume and emptying the old disks. This can be done on-line without downtime.
I'm sure stuff like this can be done with ZFS but ZFS is a userland process that can have serious performance problems. Also ZFS has some license issues with linux so it cannot be part of the kernel unlike lvm,
Thank you for your comment 👍 esp. re: ZFS.
Nice video! I recently ran into a wall and wanted to understand more about LVM's.
Chris, it would be a great follow-up to manage LVMs using cockpit, say on CentOS 7 or 8. You'll need to add the RPM cockpit-storaged.
Thanks for posting this totally helped get the cobwebs out of my head
You can use -r flag with lvextend and it will automatically resize your file system
Loving the Trek theme on the back
A perfect video to discuss on my birthday Chris :)
(and btw this is Jayden Montoya)
BtrFS does support resizing, it just needs to be online / mounted. You may have to use the `btrfs` tool directly.
My 7TB Fileserver started out with a Raid 2 setup. Transitioned to LVM after that. It's been pretty trouble free ... no drive failures.
Very well explained @Chris Titus Tech, that was just what I was looking for, thanks a lot for the effort! Just one doubt though, supposing I have a LVM, with multiple logical partitions, composed by 3 disks. If somehow I loose my motherboard or even one of the drives. How am I supposed to recover my files? I mean, what I should expect by plugging the two other drives in another PC setup?
Much love to you man.... Excellent lessons...
Some small corrections. In partitioning the physical hd, ext4 is irrelevant. LVM needs the physical partitioned drive to be "lvm2 pv" on gparted and not "ext4". Also, now you do not need to use the pvcreate command because it just converted "ext4" to "lvm2 pv." So after gpated partitioned sdb1 to "lvm2 pv only type the following:"
sudo vgextend Fedora_localhost-live /dev/sdc1
Note now "home" volume is an actual file on dev directory which is located in "/dev/Fedora_localhost-live/home." The "lvscan" command shows all the volumes. So more precise use of lvextend would be:
sudo lvextend -l +100%FREE /dev/Fedora_localhost-live/home
resize2fs /dev/Fedora_localhost-live/home
Thanks for introducing lvm very to me as I need it for home, and for sure not for work ))))
Thank you 👍
I lost all my data with LVM when I reinstalled the system many years ago. Since then, I have never tried it.
LVM does a mirror option, but to be frank I never trusted any of this. If I have important data, I will be using a RAID 10 or ZFS structure.
@@ChrisTitusTech none of these are backups - just saying. might make your data last longer, but u r only putting of the innevitable loss - th-cam.com/video/l7-6m2cE6JM/w-d-xo.html
@@WeedMIC same as real life
Great explanation sir!
Hi, nice info! I was just about to add another disk to my system. I have some questions:
1. Can I mix SSD and HDD? Suppose that I already have a LVM partition on a SSD, can I extend the volume with the new added HDD?
2. If (1.) is possible, is it safe to do so? Any disadvantage of mixing drives?
thanks ^^
1. Yes you can do that
I figured out that you dont have to create a single large partition with a filesystem in it, you can just use sdb as a whole, you dont even need to create a partition table. Thats why it says "1 disk" at 6:25. I also found it strange that you created a filesystem within sdb1 since thats something one would do with the devfile of the logical volume. Thats why it asks "Wipe it" at 9:20, after that there is no file system anymore. With btrfs you can also add whole disks to a file system, thats the reason why i tested this with lvm. Still, thanks for the video.
I used to do the same thing. I would LVM everything including my backup folder which I symlinked to every reinstall. That is untill I bought an M.2 drive and suddenly half my sata connections disappeared including my backup LVM partition which was made up of three hard drives. At first I didn't know what had happened so I reluctantly erased and reinstalled my backup drive. I haven't used LVM since. But still I love LVM.Could you do a part explaining the snapshot process. I watched multiple videos explaining it but I still haven't got a clue what they were on about. Great video Thanks
on many motherboards, installing a m.2 drive will disable at least 2 sata ports. depends on which m.2 port you put it in. most boards have 1 m.2 wired to the cpu, and any more wired through the chipset, along with most or all the sata ports. if you install a m.2 into the slot wired to the chipset, sata lanes will be disabled. this is most likely why your drives disappeared.
What is the difference between raid0 and LVM (in case if you lose a physical disk) will you lose all data? I used to use LVM a lot. But this thing is really bothering me
Raid 0 does striping while LVM does not by default. LVM can do both Striping and Mirroring, but it is an option you set when doing the Volume Group.
Yes, in this LVM configuration. Lose one disk and all data is gone. Backup!
However you can use RAID with/within LVM for data security. You just need to set it up correctly from the start.
Not gonna lie I was kind of scared but it worked out great thanks man.
Nice quick tutorial. Starting a project to implement it on a distributed file system like lizardfs or glusterfs (not decided yet) with a dozen of pies with 10GB drives.
You can mirror and stripe with LVM. Also, I believe that you can add cache drives to LVM.
You are correct! this is an option when creating the Volume group I think the options are -s and -m respectively.
Hi Chris, have you used UnionFS? with Snapraid? It would seem better if there is a HDD failure.
Legend of the linux system
My daily driver for the past 10 years or so has been FreeBSD, mainly because it has zfs volume management which is way more reliable than lvm. How are the equivalents - btrfs & zfs on linux getting on. Any time I've looked, it has been "experimental, should be ready soon", which is not something I would want to trust my data to.
At the moment I have 4 x 10TB drives on my main machine giving me 25TB of usable space, and 4 x 6TB drives on my test-bed machine giving me 15TB of usable space.
I had to use your tutorial today. Still pertinent.
It's like your doing magic @Chris Titus Tech
Fantastic explanation!
New comment /old vid... this did not work entirely as directed. Somewhere around "lvextend -l /dev/ sd#" the world essploded....but as always great tutorial.
Chris,
Isn't more secure to use a raid configuration that will also take care of grouping different hard disks?
Of course that limits you to same size hard drives (unless you accept to loose some space on larger HDs)
Is it possible to implement a sort of Raid on top of LVM?
Hi Chris. You partitioned the disk with type "ext4" - ID 83. Why not ID 8e Linux logical volume manager?
LVM a solution that was modern 10 years ago. I prefer ZFS (or even BTRFS) now, the concepts and the 2 commands are simpler and it would also improve the throughput of your disks by striping the data over those disks/partitions. It has a lot of other advantages like compression, encryption, raid-configurations, snapshots, clones, built-in (incremental) backups, memory and SSD caching too. Another great concept are the properties for each dataset (main folders), you can use it to define/change compression method, network file sharing (Windows or NFS), mount points, read-only, executables allowed, quotas (maximum allowed space), reservations (minimum reserved space), copies (kind of mirroring of that folder only), etc.
I agree with you Bert! I'd prefer ZFS EVERY SINGLE TIME! There is no scenario where I would use LVM over a ZFS solution.
10 years is not really accurate. I started using LVM in 1996 under AIX. That's more than 20 years ago.
@@ChrisTitusTech Once in my Life I used LVM in ~2005. I had a Pentium II (400MHz) with two 2 GB disks lying around and I wanted to install Ubuntu 5.04 from a PC Magazine CD. However Ubuntu needed more than 2 GB. That was the only time, I used LVM and it was also my first look at Linux :) :)
I remembered that Ubuntu experience, when I got annoyed in 2008 by a new dual core Dell laptop with a very slow disk, desperately trying to run the original Windows Vista without the service packs.
@@bertnijhof5413 Thank you for the comments 👍 esp. ZFS being preferable to LVM.
Here in 2022, it seems like ZFS is still a top choice.
Also some folks are looking into Longhorn's potential.
Kindest regards, neighbours and friends.
Chris if we installed ubuntu without LVM and just with standard installation, than how can we do it from there, In this video you have already setup LVM and taught us that how to add and increase space, but in case we installed without LVM support, than how can we do that. Please
Excellent video, very helpful
Hi Chris,
love your videos.
You can also use the whole disk instead of partitions. It just has to be unpartitioned.
You can do this in windows server as well.
I just learned this in my A+ class! Ive been scrolling through comments thinking I was crazy haha
great video
This is good. I was trying to do that using KDE partition manager or even with Gparted, and I could not do it. Only this worked. Why though? Thank you!
Thanks for your excellent tutorial!
Thank you very much. This clip is really helpful.
Thanks for this video.
However in my understanding the partion type of the physical volume has no influence on "extend" option.
Even more, You don't need to put an the partition on the disk at all.
However the filesystem on the logical volume should be ext4.
Have to love this man 😁
I am a bit curious, what happens if I use USB stick as LVM extension on a laptop and turn on the laptop while accidentally removed the USB? Will the machine boot normally after I plug the USB back and reboot?
So volume group acts as a virtual drive, composed out of physical partitions? Say, can I use LVM over some of the physical partitions while keeping others "close to metal"?
Yes you can have mixed configurations: i.e. some of HD's under LVM control and others not.
@@peterdegelaen hohohohoho!!!!
*HI Chris can you explain INODE in the Linux system....and data recovery in a future video ...thanks👍👍👍*
Check out "$ man inode" or a simple google search will satisfy all you need to know.
Basically, it's just the building block of the file structure. It holds data about the file it points to such as filename, ownership, permissions, where to find the data blocks, etc.
@@dingokidneys *OK thanks 👍*
GParted: In your video I saw that you used GParted to create a partition with an ext4 file system, then immediately overwrote it with pvcreate. Note that GParted can create an empty partition by selecting "unformatted" file system. Also GParted can create a PV by selecting "lvm2 pv" file system. However as you video is concentrating on using LVM on the command line the later is probably not what you want in this case.
Great tutorial! Thanks for posting!
hi Chris, good content !!
i think zfs its the way to go ?! it is evolution of LVM, zfs combine LVM and partition tools, its very powerfull, already use with freenas box. I think the evolution path is LVM, XFS and now ZFS ... i ve miss someting ? thanks to put me on the right way !? :)
ZFS is my favorite by a long shot
ZFS is great with the down-side that most Linux distros will not boot from a ZFS drive. You would have to compile a kernel yourself with ZFS support in-built. Ubuntu is to my knowledge the only distro which will boot from ZFS.
The ZFS project (now owned by Oracle) predates LVM. Openzfs is newer.
LVM + XFS is VERY handy for cloning your boot drive to install a new larger boot drive. Run CLonezilla(or use DiskDestroyer if you dare) to easy clone your drive, then grow the partition to take up the free space.
Here in 2022.
Nice channel, yours 👍 incl. 1950s and big bear.
Good comment re LVM + XFS.
I think ZFS is a top choice, too.
Kindest regards.
You're the Life Saver!
Many thanks a lot for the vid, it was very informal for me
Thanks for sharing this! Could you make a video on how to configure LVM on Proxmox VE? Just as a suggestion, I think it would be nice, what do you think?
I've always used LVM on single drives. There's no reason not to when you allocate X for each volume, then you have Y unallocated space you can allocate to grow / shrink your volumes as needed.
On a single disk I don't see the point of using LVM (except for full disk encryption) when you can just create one partition and put everything into it. You don't have to worry about growing/shrinking volumes. As far as I can see, it's only when using several physical drives that this really comes into it's own.
@@dingokidneys I've always used full disk encryption too. If I wasn't using LVM I'd at least have a separate /home partition, maybe a swap partition as well.
Once, just to see if I could, I got a Windows / Arch dual boot working like this:
I had a USB drive that would be mounted at /boot with an external LUKS header for the LVM partition, and a 8192 KB LUKS encrypted key file. I modified the encrypt hook so that at boot, it worked like thjs:
If the USB drive wasn't inserted, you got Windows. If it was, you got rEFInd (because I've never liked Grub, and think rEFInd with a theme looks way nicer and it's easier to configure and fix) and if you selected Arch, it would prompt you for the password to decrypt the LUKS encrypted key file, then used the decrypted key file and the external LUKS header to decrypt the LVM partition and mount the volumes as normal.
The key file and external /boot gave two factor authentication, and the external LUKS header gave plausible deniability because you couldn't prove it was an encrypted partition instead of unallocated space.
thank you for the wonderful explanation.
Now I have spent a month in Ubuntu. The only thing that went wrong once was when I rebooted and the reboot hung with a blank screen and my OS never appeared. So I put in a Linux live disk and, weirdly, it said that there was no prior OS there. Even though if I ran gparted there absolutely was my OS still there! I had set up as an LVM system install, something that I had never done before. I tried boot repair. That didn't change anything. So in the end I did a total reinstall of Linux. This time I used a normal Linux filesystem. There was no issue rebooting at all. So I think that LVM just does not work on my computer which is a bit old [2011]. Despite this I am happy that I went Ubuntu. More chance of joining the Moonies than of going back to Windows.
7:37 - isn't there a dedicated Linux LVM partition type?
the disadvantage (among others) is that, for system partitions (/ or /home), you have to setup the group/volumes at system install time.
also why sudo su? su should be enough...
You are correct it is, Thanks for the correction!
@@ChrisTitusTech as you said, you were having problems with sudo... 8)
If you use "su" instead of "sudo su", you need to know the root password. Also, I would use "sudo su -" which brings you to root in a login shell (which means that you are placed in the root home directory (instead of the home directory of the invoking user) and that the shell login scripts will be executed).
@@peterdegelaen well... in my system, i stay in the path i'm typing the command, no mater if i do su or sudo su...
@@QuimChaos You have to add the "space dash" ("space minus sign") behind the "su" in "sudo su -"