legend my guy.i didnt want to really delete files on a number of drives but your usb passthrough helpped me atleast see my drives in my synology vm and backup what i wanted so i could format the drives in proxmox later
Thanks for the tutorial! Is it possible to use the drive in one container (like openmediavault), get some network address for it, and then use that address for torrents and watch the contents from it on TV?
Yea it should be possible to make a network share of a external drive. Then the network share can be access by other computers like your TV. I plan on making a video in the future about sharing storage between CTs and VMs as I see many questions about this.
Great video, i know its 2 years later but was the SMART drive data being passed through? Im looking at adding a 5 bay USB-C DAS to my NUC running proxmox
Great video mate! I learned a lot!! I want to run PBS in a VM and use a usb for store my backups. Which will be the best option to setup it? Pass through the usb port or the hhd?
Great video, thanks for the information. One question, can proxmox 8.1.3 have two virtual machines(truenas scale using a PLEX application and jellyfin installed as a virtual machine) share a hard drive(s) that is on a smb share? Would I need to use pcie sata adapter or can I identify the hard drives by there serial numbers in the configuration files? Thanks.
If you have a TrueNAS scale VM, and a Jellyfin VM that you want to access the data on TrueNAS connection, I'd setup a SMB share on the TrueNAS VM and mount SMB on the Jellyfin VM as you suggested. You should be able to identify the HDD by the serial number of the drive. With external drives that support hot swap you can also look at the list of drives, then add the external drive and see what drives shows up on the system after adding the drive. Hope this helps you with your Proxmox config.
How would I go about moving or placing a VM installed on a external hard drive to the internal hard drive of a proxmox host? P.S. i did not install the vm in that external drive. It was the first thing I mentioned, saying that it's actually not a good practive to be installing VM's on a external drive .-. now that external drive is starting to give bad signs. So I want to swap/migrate it to the actual internal hard drive of the Host itself before the whole thing crashes.
Another great video! I have passed through an internal drive to to proxnox windows vm so it’s come up as qemu. What about trim? Its shows as a thinly provisioned drive in windows. I want to be able to trim it. It’s in ntfs format. Just trim in promox directly?
Trim support really depends on how you did passthrough. If its USB passthrough, it should just work and the guest would have full control. If you passed the whole block device though, I don't think trim is supported, but need to test. If you made a virtual disk you can enable discard in the virtual disk properties, and it should make the virtual disk smaller, and that free space can be used for trim by proxmox.
So what's the correct solution if you have a Plex container on the primary proxmox HDD and you want to put your videos only on an external (ssd) drive dedicated to videos only?
I’d install the Plex containers boot drive on the primary proxmox hdd. Then I’d add the external hdd as another proxmox storage drive and add a second mount point to the container which is located on the external hdd and setup plex to use that for media.
Great video!, I've been running into a lot of issues with external hdds in proxmox. I mainly use containers and i wanted to storage data inside the external hdd from inside the xlc container, but that seems impossible, I can't change permissions, and any mountpoints from host /mnt/usb to the guest lxc containers are mounted with the user and group nobody, so i can't do anything but read. from the host proxmox machine, it works fine, i can create files and everything. I can also upload iso, and make backups of my containers. however, i can't create containers or vms that are greater than 8gb in size, after that they all timeout and it fails. Honestly it's been very annoying. If you could make a video showing how it works with containers and permissions, i would deeply appreciate it. Great content keep it up!!
Does someone know why I don't get the SMART (S.M.A.R.T.) values of my Seagate Backup Hub 12TB? In your case it seams to work with USB-Drives in general.
Some usb controllers don’t pass smart data while others do, and I don’t think there is any setting or mode that can be changed in incompatible devices. You can also try windows with crystal disk into as that seems to get the smart data on a few more usb drives.
@@ElectronicsWizardry Thanks for your advice. Thats the point, with Windows SeaTools I can see the SMART values, so i thought I need to install some driver maybe?
@@roberts.3619 I know there are some settings for smartctl in linux to see the drives of from certail USB controllers. Can you run something like lsusb to see the model of the controller? Maybe seatools is using their own tools that can see the drive here.
Thanks for the video! I used this info in my migration of a Windows host to Proxmox, I wanted to keep my NTFS drives so I just passed the disks to a Windows VM using your instructions which I'll use as a NAS. I had to use /dev/sdc, it wouldn't work when I used the device name in /dev/disk/by-id.
Seriously amazing videos! I have a question: Should be running my VMs with a disk drive holding all the files I am suing in that VM…let’s say like a Plex media server for example. Or should the VM disk only have the bootable disk and the actual data is being stored on a mapped directory off a ZFS drive on the host ? I’m trying to see the most efficient way to keep my data safe and flexible in case a VM is no longer viable in the future. Also my VM disks are all running on 2TB SSD drive….so if I keep all the data in the virtual disk I will run out of space very quickly.. Thanks again!
If you want to have shared data between vms, I'd setup one vm as a file server, and mount the share on the other VMs using NFS/CIFS/SMB. Then the server only has to hold one copy of that data, and changes make will apply to all the systems. I plan on making a video that explains storage on hypervisors a bit more and how best to share it between hosts.
@@ElectronicsWizardry I tried in the past to simply use the host as the file server and map with sshfs inside each VM the directory for the shared data. But I guess having that one big directory inside a file server VM would give me the advantage of using snapshots for that data directory… right? That must be worth doing it for. So how about using trueNAS scale as my main file server directory inside a VM so I can easily share via nfs and smb across the board ?
@@webftpmaster Running the file share in a Vm lets you make snapshots and backups easily, but if a file share on the host works for you go for it. I don't see an issue with TrueNAS here, and have used it like that before without issues.
If you setup the drive as zfs then you re stuck for Vms and CTs cretion only. Ok maybe someone wants just that. You could show the steps then to unmount this disk and mout it to another prox node. I think you have to go from datacenter -> storage and remove it (keeping data) Still present on the node side though. Do you have to destroy it from there? How you properly remove it? Edit: Well if you go to the node->Disks->zfs you can destroy it there and uncheck the boxes in order to keep contents. Afterwards if you connect it to another host you skip the part where you wipe it and initialize it with a gpt partition. You also skip the step of creating a zfs storage and go directly to datacenter and add a directory with name and directory (path) the same as the previous node. Ok the sotrage is there it shows that something it is taking space inside but you can t see the backup in order to restore it. That is where I am stuck Later on you passed the disk (usb port upon which the disk resides) to a VM. Since you had wiped the disk where all these partitions came from inside windows?
If you want to remove the drive from one host and move it to anouther proxmox system, it would be best to remove any vms using the storage, remove the storage from the datacenter, then unmount the storage. Then the drive can be plugged into a different proxmox system and mounted(you would likely want to do this in the terminal), then add the storage to the new host. Id suggest making backups of the vms and copying the backups if you want to easily move the vms between hosts, as it stores the vms and config files together. I filmed the video out of order. I had prepped the drive before filming, then filmed the passthrough section, then the wipe the drive and use ZFS section. I though reversing the order would make sense I tried going from simplest and most recommended to more complex setups.
@@ElectronicsWizardry Thanks for the answers. Apparently adding the external drive with zfs and on top of that adding a dataset in order to store Vm backups (I wasn t using it to store VMs at first place but only the backups) doesn tplay along, Instead if I create a directory (ext4 or xfs) and store the re the backups and afterwards, after correct removal move the drive to a different host mount via cli the drive and from gui adding a directory via the datacenter section makes the backup appear at backups option window. I have to try tyhe proxmox backup server now to see if i add it's storage to both nodes if I would be able to restore the VM directly to another node without the hassle of plugging mounting and unmounting external drives.
Great video, i have been struggling on how to let my plex container to see the hhd (ntfs) with my movies. Your vid give me some ideas. Thank you...lady daldom.
This channel is extremely underrated! Your content is excellent!
Thank you for sharing your knowledge with the community!
You are the best - I am toying with the idea of setting up a Proxmox server with virtual containers and your explanation is to the tee.
This is the video I was looking for!! Thnk you quick and simple!
legend my guy.i didnt want to really delete files on a number of drives but your usb passthrough helpped me atleast see my drives in my synology vm and backup what i wanted so i could format the drives in proxmox later
The trick to pass through the hdd via usb was a life saver for me. tnx :)
Thank you for taking the time to make this and all your great videos. I learned so much.
Such a comprehensive video that absolutely poops on every other video on TH-cam covering PVE storage.
this is so useful! Thank you for showing each step. Makes sense now.
Super helpful video! You are a life saver. Thank you so much!
Glad the video was helpful.
great video. Will be using an ext drive to store ISOs and backups from proxmox and hope to make some of it visible in VMs for management too.
back in and recently got soft soft again, it felt strange and i had previously just taught myself the software. Finding your videos is helping
Awesome video bro and thank you for this. This is exactly what i was looking for.
Thanks for the tutorial! Is it possible to use the drive in one container (like openmediavault), get some network address for it, and then use that address for torrents and watch the contents from it on TV?
Yea it should be possible to make a network share of a external drive. Then the network share can be access by other computers like your TV.
I plan on making a video in the future about sharing storage between CTs and VMs as I see many questions about this.
Great video, i know its 2 years later but was the SMART drive data being passed through? Im looking at adding a 5 bay USB-C DAS to my NUC running proxmox
this has been very useful tutorial for me... thanks alot
Excellent, really helpful. Thanks
Great video - very helpful!! Thank you.
Great video mate! I learned a lot!! I want to run PBS in a VM and use a usb for store my backups. Which will be the best option to setup it? Pass through the usb port or the hhd?
Great video, thanks for the information. One question, can proxmox 8.1.3 have two virtual machines(truenas scale using a PLEX application and jellyfin installed as a virtual machine) share a hard drive(s) that is on a smb share? Would I need to use pcie sata adapter or can I identify the hard drives by there serial numbers in the configuration files? Thanks.
If you have a TrueNAS scale VM, and a Jellyfin VM that you want to access the data on TrueNAS connection, I'd setup a SMB share on the TrueNAS VM and mount SMB on the Jellyfin VM as you suggested.
You should be able to identify the HDD by the serial number of the drive. With external drives that support hot swap you can also look at the list of drives, then add the external drive and see what drives shows up on the system after adding the drive.
Hope this helps you with your Proxmox config.
Thank you. I will give it a try. Thanks.
why does my VM not show "hardware" as an option???
Very usefull video. Thanks.
How would I go about moving or placing a VM installed on a external hard drive to the internal hard drive of a proxmox host? P.S. i did not install the vm in that external drive. It was the first thing I mentioned, saying that it's actually not a good practive to be installing VM's on a external drive .-. now that external drive is starting to give bad signs. So I want to swap/migrate it to the actual internal hard drive of the Host itself before the whole thing crashes.
Another great video! I have passed through an internal drive to to proxnox windows vm so it’s come up as qemu. What about trim? Its shows as a thinly provisioned drive in windows. I want to be able to trim it. It’s in ntfs format. Just trim in promox directly?
Trim support really depends on how you did passthrough. If its USB passthrough, it should just work and the guest would have full control. If you passed the whole block device though, I don't think trim is supported, but need to test. If you made a virtual disk you can enable discard in the virtual disk properties, and it should make the virtual disk smaller, and that free space can be used for trim by proxmox.
What tool did you use for io monitoring?
My favorite Linux tool for oo monitoring is iostat part of the sysstat package. It gives most info about current drive io.
If i have a 3 nodes Proxmox Server, connecting at each a 2.5" HDD, is it possible to create a CephFS cluster storage from this external hdd devices ?
How are you able to see apfs partitions in windows 10? Do you use a third party software?
So what's the correct solution if you have a Plex container on the primary proxmox HDD and you want to put your videos only on an external (ssd) drive dedicated to videos only?
I’d install the Plex containers boot drive on the primary proxmox hdd. Then I’d add the external hdd as another proxmox storage drive and add a second mount point to the container which is located on the external hdd and setup plex to use that for media.
Great video!, I've been running into a lot of issues with external hdds in proxmox. I mainly use containers and i wanted to storage data inside the external hdd from inside the xlc container, but that seems impossible, I can't change permissions, and any mountpoints from host /mnt/usb to the guest lxc containers are mounted with the user and group nobody, so i can't do anything but read. from the host proxmox machine, it works fine, i can create files and everything. I can also upload iso, and make backups of my containers. however, i can't create containers or vms that are greater than 8gb in size, after that they all timeout and it fails. Honestly it's been very annoying.
If you could make a video showing how it works with containers and permissions, i would deeply appreciate it.
Great content keep it up!!
Hey, I have spent far too much time dealing with the same issue and it's driving me up a wall. Did you ever come across a solution?
Have same problems with permission, but i can create files with samba, but dont delete it with samba method
I have exactly the same problem and it's driving me crazy. Anybody has a solution for this?
Does someone know why I don't get the SMART (S.M.A.R.T.) values of my Seagate Backup Hub 12TB? In your case it seams to work with USB-Drives in general.
Some usb controllers don’t pass smart data while others do, and I don’t think there is any setting or mode that can be changed in incompatible devices. You can also try windows with crystal disk into as that seems to get the smart data on a few more usb drives.
@@ElectronicsWizardry Thanks for your advice. Thats the point, with Windows SeaTools I can see the SMART values, so i thought I need to install some driver maybe?
@@roberts.3619 I know there are some settings for smartctl in linux to see the drives of from certail USB controllers. Can you run something like lsusb to see the model of the controller? Maybe seatools is using their own tools that can see the drive here.
Thanks for the video! I used this info in my migration of a Windows host to Proxmox, I wanted to keep my NTFS drives so I just passed the disks to a Windows VM using your instructions which I'll use as a NAS.
I had to use /dev/sdc, it wouldn't work when I used the device name in /dev/disk/by-id.
Seriously amazing videos!
I have a question:
Should be running my VMs with a disk drive holding all the files I am suing in that VM…let’s say like a Plex media server for example.
Or should the VM disk only have the bootable disk and the actual data is being stored on a mapped directory off a ZFS drive on the host ?
I’m trying to see the most efficient way to keep my data safe and flexible in case a VM is no longer viable in the future.
Also my VM disks are all running on 2TB SSD drive….so if I keep all the data in the virtual disk I will run out of space very quickly..
Thanks again!
If you want to have shared data between vms, I'd setup one vm as a file server, and mount the share on the other VMs using NFS/CIFS/SMB. Then the server only has to hold one copy of that data, and changes make will apply to all the systems.
I plan on making a video that explains storage on hypervisors a bit more and how best to share it between hosts.
@@ElectronicsWizardry I tried in the past to simply use the host as the file server and map with sshfs inside each VM the directory for the shared data.
But I guess having that one big directory inside a file server VM would give me the advantage of using snapshots for that data directory… right?
That must be worth doing it for.
So how about using trueNAS scale as my main file server directory inside a VM so I can easily share via nfs and smb across the board ?
@@webftpmaster Running the file share in a Vm lets you make snapshots and backups easily, but if a file share on the host works for you go for it.
I don't see an issue with TrueNAS here, and have used it like that before without issues.
So good thank you
If you setup the drive as zfs then you re stuck for Vms and CTs cretion only. Ok maybe someone wants just that. You could show the steps then to unmount this disk and mout it to another prox node. I think you have to go from datacenter -> storage and remove it (keeping data) Still present on the node side though. Do you have to destroy it from there? How you properly remove it?
Edit: Well if you go to the node->Disks->zfs you can destroy it there and uncheck the boxes in order to keep contents. Afterwards if you connect it to another host you skip the part where you wipe it and initialize it with a gpt partition. You also skip the step of creating a zfs storage and go directly to datacenter and add a directory with name and directory (path) the same as the previous node.
Ok the sotrage is there it shows that something it is taking space inside but you can t see the backup in order to restore it. That is where I am stuck
Later on you passed the disk (usb port upon which the disk resides) to a VM. Since you had wiped the disk where all these partitions came from inside windows?
If you want to remove the drive from one host and move it to anouther proxmox system, it would be best to remove any vms using the storage, remove the storage from the datacenter, then unmount the storage. Then the drive can be plugged into a different proxmox system and mounted(you would likely want to do this in the terminal), then add the storage to the new host.
Id suggest making backups of the vms and copying the backups if you want to easily move the vms between hosts, as it stores the vms and config files together.
I filmed the video out of order. I had prepped the drive before filming, then filmed the passthrough section, then the wipe the drive and use ZFS section. I though reversing the order would make sense I tried going from simplest and most recommended to more complex setups.
@@ElectronicsWizardry Thanks for the answers.
Apparently adding the external drive with zfs and on top of that adding a dataset in order to store Vm backups (I wasn t using it to store VMs at first place but only the backups) doesn tplay along, Instead if I create a directory (ext4 or xfs) and store the re the backups and afterwards, after correct removal move the drive to a different host mount via cli the drive and from gui adding a directory via the datacenter section makes the backup appear at backups option window.
I have to try tyhe proxmox backup server now to see if i add it's storage to both nodes if I would be able to restore the VM directly to another node without the hassle of plugging mounting and unmounting external drives.
qm set VMID --scsi1 /DEV/SDC
Great video, i have been struggling on how to let my plex container to see the hhd (ntfs) with my movies. Your vid give me some ideas. Thank you...lady daldom.
can you give a hard drive a number as a drive path instead of a letter