I was using the standard PVE backup solution for my VMs and LXCs (8 in total). I ran them every weeknight and kept the last 5 versions of the backup. It was taking up over 1TB of storage on my Unraid server. I switched to PBS and I'm taking backups every 2 hours now for all VMs/containers, and I keep the last 120 versions. The backups are only taking up 750GB on my PBS server. That's pretty amazing considering I am now taking backups 12 times a day, instead of just once every weeknight, and keeping 10 days of backups instead of just 5.
@@hawolex2341 It depends on how you integrate it. You could implement it as a kind of remote control so that you have separate hosts but direct access from one interface, similar to clusters. The backup server is integrated as a data source. Currently you can schedule, start and restore backups from Proxmox VE. This is sufficient for most operations.
@@hawolex2341 I think they meant being able to manage them from the same interface, rather than a different interface. Similar to how you can manage entire PVE clusters from the same interface.
A couple of things that don't get mentioned much: 1. PBS requires more CPU than you might expect. I have it running on an old Microserver with the Dual Core AMD CPU and only getting 25MB/s or so. The CPU does a lot of hashing and compression to get incremental backups working properly 2. PBS can be easily run in an LXC container within another PVE, no need to have it on dedicated hardware. This allows you to pass through ZFS Mount points With all that said it's a fantastic platform and it adds a lot of functionality. Live restores are so quick and easy and can really save your butt.
You do not need a CT, you can make a hybrid, this is a Debian machine. In the documentation you find how to install PBS on a PVE machine, check it out. One of my 3 PBS is a hybrid and after a year of testing I have had no problems.
as the first point, PBS doest not use a lot of CPU per se, but prefers very fast cores over lots of core. As a MSP, we have over 40 different PBS installation, the biggest one on a 40 cores / 80 threads xeon server w/ 8 nvme drives in raid 10 and guess what? Another client with a SFF PC (Intel N100) and 2 sata SSD in mirror performs nearly 2.2x better while doing backup or restores over 25Gb LAN.
Yeah....I'm not really sure how CPU intensive the backup operations actually are, on the system that's hosting said PBS. The verification tasks are CPU intensive, but as far as I can tell, the compression in preparation for the backup, appears to be running on my Proxmox server side rather than on the PBS side. And I would think that would make sense because you want to compress the backup of your VM/CT that you're backing up, BEFORE sending it over the wire to the PBS, so that you actually end up sending less data over. I don't think that it would really make sense for the PBS to be running the compression, over the wire, rather than having the Proxmox server do the compression, when you ask it to execute a backup task, and then sending the compressed result over to said PBS. This has been my observation about how it works so far.
Just what I needed as I'm at that crossroads with my backup and I wanted to go the route of using a PBS or continue doing the backups from within VE on my NUC.
A note to PBS. If you have a storage server like TrueNas, Unraid, OMV or the like, you can mount an NFS, SMB or another type, into PBS, and use that for the location for backups. That also means you can have a PBS in each of your Proxmox servers connected to the same share, or whatever you like. So you could also put your backups in a TrueNAS cluster, so you have "cluster backup" which is AWESOME. Thanks for the video, even know I already knew all of this :)
Does PBS run in a container or vm if installing on the proxmox host? I've heard others recommend not to install pbs on the actual just but another machine.
@@kevinhilton8683 I'm running PBS in a LXC container running on a QNAP NAS right now. Therefore; as such, you can absolutely install it on the same system as your Proxmox server, but if your Proxmox server dies on you, then you won't be able to access your Proxmox Backup Server Web UI, to be able to perform restores, which can cause problems for you, if that happens (and you want to try and restore/recover your Proxmox host and/or any VMs/CTs that was running on it).
In my lab I am running this and first got turned on to this backup solution from a great detailed tutorial from Jims Garage....My dedupe factor is currently 310, but I am doing some crazy stuff with taking snaps every 30min just so if I mess anything up, I just quickly revert to an old backup. Great for messing around in your lab without fear of losing anything really. Honestly no downsides. I also cannot wait for better clients in the future for better bare metal restores and backup other OS and hosts.
I do a nightly backup just via PVE,without retention, and then have a full backup routine with retention Job Prune and GC using PBS, all backing up on the NAS through a NFS share. The PBS is on a VM on the Synology (a meager DS918+ with 8gb of RAM). Setting up the PBS VM was a breeze in DSM. I used 2 cores and 2gb of ram for the initial setup, script install and updating of PBS , and then dialed it back to 1 core and 1gb for the working setup, works a charm. I'm also running a PVE VM on the Synology to run as always on as the third node in my Mini Lab 2x Mini PC cluster, that VM is set at 1 core and 2gb, no issues there so far either! Now if I could only get the Task Manager to actually run a schedule script to start and stop the PBS VM in DSM on a schedule, and an easy way to backup the PVE environments, it would be pretty damned near perfect for my modest needs.
Amazing timing. I was looking for another project and went back over your older videos and did a PBS install just two days ago. I shall watch this one to see if I've missed something important but it's working very well and I LOVE the ability to restore files instead of the whole damn thing!
I so agree with you. After just testing out the first generation of BPS I have been using it. Running 3 of them now to be able to test all of the features. One is a hybrid, a PVE and then installed on top the PBS, it’s Debian. Used the wiki/documentation for setting it up and after a year I have had no issues nor any problems.
PSA: the default vzdump backup is not a proper error checking backup, if let's say you have RAM corruption you would never know your backups are also corrupted. I learned the hard way you will not be able to restore those backups. PBS on the other hand is perfect and does backups in a way they cannot get corrupt. So when you can always go for PBS do not rely on the pve vzdumps.
I love PBS. I’ve had it in operation since setting up my first PVE host and actually used this in favor of building a cluster because I didn’t have enough stuff running to justify a cluster. With that said I’m up to 3 PVE machines but still continue to use PBS on the regular and it’s saved me several times.
I've been using it for some time and it's great and well integrated. Be aware it's much slower on HDD because of the way it works (lots of folders/small chunks). I used to backup on my SSD and synchronize with my HDD later. Unlike VMs with block devices, LXCs rely on file system which make backup & restore slower with high IO delay... All files need to be read before the backup. You can even freeze your system carelessly. Hopefully they have some options to mitigate this. I think ZFS is the way to go for LXCs.
I would just set up Veeam as they recently introduced Proxmox backups. Currently trying to set it up, tho running into some issues still. But this way I only have 1 backup server for both Windows and Linux VMs, even if I need to use their agent due to PCI passthrough. With PBS it's "technically" possible but not ideal...
I’m a huge veeam fan, but the current implementation does yet provide application aware backup. Looking forward to this coming in a future release. This is a show stopper for me implementing this in production.
@techno Tim - I noticed you started signing (American Sign Language) THANK YOU at the end of your videos. Are you doing this on purpose? Is is directed at a particular viewer(s)? What's the deal?
Hi! I've done it for years. A large company I worked for always prioritized web and app accessibility and I learned a lot about how to make software more accessible. This also gave me many opportunities to work with people who had low/no vision and also those without hearing. This gave me such a new perspective on not only software but also life. Since then, I have always tried my best to make the software I write accessible as I can, and that has also been transferred over to my videos too 😄 I try to make the best subtitles I can and while I don't know much ASL, it's one way that I try to show my appreciation as well as teach it to those curious.
@@TechnoTim Being a fully abled person, when I first saw this I was enamoured by your consideration for the less so. Great work! Can't wait for Host backups - urrgh always feels like the weak spot in my DR
I always enjoy your content. I took some time today to set up PBS to run as a VM on my Synology NAS. I'm looking forward to seeing the space-saving benefits!
Just deployed it in a LXC container running on my QNAP NAS to see how well (or how poorly) this is going to work. *edit* I wished that you had covered the use of namespaces in this video so that you can create different namespaces for the different Proxmox systems that people might be backing up to said PBS, if they have more than one Proxmox server that they want to back up. (You need to define the data store, and then you can add the namespace, and then on the Proxmox server that you want to backup, you can add the namespace to the backup task, and then back over on the PBS, you can see it by clicking on the datastore, and then clicking the namespace drop-down (which normally defaults to "root" to be able to see all of the VMs/CTs that have been backed up.) Very useful feature, I think.
My favorite feature is LIVE restore of VM. I have my pfsense router as a vm in my proxmox. More then once I have screwed up a setting and could not figure out what I did, I had to restore from back up. While I can use a back up of a config, I would have to be smart enough to back up config before I mess around. I have backups every other day on PBS, so when I screw something up, I just live restore from yesterday or day before. Now my wife and kids don't come complain about no internet while backup is being done, because ITS LIVE. I'm sure I'll find other uses sooner then later.
I have just implemented this myself, im backing up to a truenas NFS share mounted in the PBS vm, then backing those up to tape which is also passed through. Has saved me so much space!
@@Blackincolorhad this as well. There is a solution for it but had to Google it alot. Something about how the nfs is mounted and the user, backup user from memory. Reason why I can't remember totally as found nfs very very slow for some reason. Went back to just using host storage and using sync job between two pbs and all good so far.
@@Blackincolor Forum post "NFS doesn't work on PBS". Basic idea was to go to the dataset in Truenas change the permission users and group to "backup". Edit /etc/exports on true nas and add in no_root_squash to the nfs share. When you mount the nfs share on pbs make sure when mounted it has the user backup and group as backup. Create the dataset. Creating the dataset took ages though which is why I just used the host storage instead. So far my duplication factor is 8.
I run PBS directly on my proxmox homeserver, having a 18TB hdd on its own. SSDs are recommended in the documentation, but the price to backup would skyrocket, and i'm having acceptable performance for my needs. I went through a server rebuild after needing to changing drive config for zfs and it was easy as hell. install pve, add pbs repos, apt install pbs, mount hdd, point datastore to mount location, bam, vm and containers ready to restore. highly recommended
Simple solution for having this in a VM, simply install it, give it storage, then clone that VM to another node in the cluster. Then setup your backups such that the VM's aren't backing up their host nodes. Everything else can remain the same and it really works quite well.
Definitely going to add PBS. I will continue to use urbackup for all my other machines and may look into getting it working for the PVEs themselves. The combo, if successful, will give me backups for everything.
Moved to PBS a few months ago, and it's great. Saving about 2TB of disk space. One thing I do is, backup the PBS VM, within proxmox to a network share. It's only about 2GB. This way if I must restore from scratch, I can just restore the PBS VM from proxmox, then restore all the VM's from PBS. Just saves time, as I don't have to install PBS and redo it's configuration. Yes, adding PVE host backup would be the cherry on top. I look forward to the day they do this.
For those that might be reading this -- backing up the PBS VM is NOT enough to back up the actual chunks of data that constitutes the backup of your VMs and CTs. For that, you'd have to back up the .chunks folder as well as the PBS VM/CT.
I find PBS excellent for backing up my vm's, I don't really care to back up my proxmox hosts, I just do a fresh install of the host and restore the vm's, easy as pie. I'm looking forward to the day my local PBS can push a backup to a remote PBS in the cloud, so far it only works the other way with a cloud server pulling a copy of the backups on your local PBS. As for pros and cons, incremental backups are really fast to do, but rather slow to restore.
There is one major limitation with PBS - it doesn’t support “application aware backups” like Veeam on VMware. This means I can’t use proxmox or PBS with windows based application servers such as SQL and Exchange. I was hoping Veeam for PVE would resolve this but it’s still not an available feature. If this functionality is added into PBS then the solution would be awesome. If you are a Linux shop only then you won’t be affected.
I wouldn’t call that a major limitation…. You can fix it in two steps 1) ensure guest agent is installed and enabled in Proxmox 2) setup a backup job inside of the VM that exports data to a directory inside VM regularly. Step 1 ensures that any writes to disk are consistent, step 2 ensures that there is a definite known backup state available. The deduplication of PBS will still works just fine as well! Have terabytes of data in your SQL server? Any backup solution is going to be more difficult with that; however, using a backup solution inside of the VM itself isn’t really that difficult to do or manage if needed…
Being able to add windows and macOS options would be cool and even adding iOS and Android devices would make it an awesome backup solution. Maybe someday!
It is important to have at least an zfs raid. The deduplication reuses chunks between the snapshots and if one chunk is corrupted, all snapshots referencing this chunk are broken. In the worst case all/most backups for one or multiple VMs may not restorable (e.g chunk with partiton table is identical in all snapshots and corrupted). This is the reason you should always use at least zfs raid1 which will detect and repair bitrod etc. Follow zfs best practice (pass-trough disks/storage controllers directly), because it will safe you when you need the backups the most. You may want to set up an secondary remote sync on a smaller box. Schedule regular GC and Verify (there is a setting to verify snapshots after x days) runs. When the snapshots and chunk count grow over time you will notice that anything involving the dataset is getting slower each day. Then you can add an SSD for L2ARC and more RAM. Tis will speed up everything. While RAM is faster the L2ARC can be reused after a reboot (if configured to do). Also RAM is expensive.
I realize that I’m wasting tons of space on my NAS by avoiding this project. Those backups are my lifeline though. I’ve had to go to them in very hard situations multiple times and I just sleep well knowing that each backup is totally separate with no chance of snapshots corrupting one another. I’m running out of space though so I will be forced soon to pick up another mini optiplex or equivalent. Doesn’t need to be more than $200. It’s on my list.
I am using PBS as well virtualised on my second Proxmox VE. I backup my Proxmox VE with Rescuezilla. It does not take much time as I do not store any vm's on the Proxmox VE boot drive itself.
Currently i'm just using an old HDD (13y/o WD:Green) in my PVE server. when that drive dies, or i up the storage in my NAS. I'll probably going to move over to PBS virtualised in TrueNAS.
Great video as always!! Speaking of backups - Tim, how do you backup your regular ol workstations? I'm using Arch and trying to establish an incremental backup solution to my NAS, somewhat like Apple's timemachine. I've heard people use rsync but I am having some challenges with that.
Okay, so what do I do when I only have spare laptops, external HDDs? Mabye I could use a laptop for PBS with external HDD that will be syncing to my TrueNAS Core all the time?
cool. nice to see the dedup advantages but for homelab and proxmox running on an SSD wonder if the advantages of backup are useful for most. If only Truenas widely supported VM's and containers a hypervisor would not be necessary.
TrueNAS is a Hypervisor and NAS hybrid. Scale more so than Core. Way back my homelab was only my TrueNAS and some VirtualBox and KVM/QEMU on my PC and an old PC I got for free from work.
@TechnoTim, Is there a benefit to using PBS over, say, backing up to a Synology NAS? I currently backup my VMs to an NFS share on my NAS, but am considering if it would be worth my while keeping it all "in-house" and backing up to PBS instead. My existing VMs are only small - Debian 12 with mostly 16Gb of storage and at least 2Gb or at most 4Gb RAM and based on what I previously ran as physical Raspberry Pi SBCs.
I stopped using pbs cause it doesn't work well with containers. It sends the entire container over the network to the pbs server to do dedup there every time you backup. Since i have some large multi-TB volumes, it just takes far too long. pve-zsync is much better if both server and backup use zfs.
Thanks for the video. What do you think about running this in a VM in PVE, with another VM running Unraid or TrueNAS? Would not be my only proxmox host node.
It's funny that the tape backup is "what year is this even?", but they don't support network share targets unless you hack them in. My main frustration with it is needing to have another host running another VM to support it. It would be much better for me if at least some of it (mostly the dedup) was just part of PVE.
I run LTO-8 tape backup in the basement of my home. I think that the last time that I ran the numbers, at somewhere north of between 150-200 TB, it becomes cheaper (on a $/TB basis) to use tape than having HDDs, even if you've got two HDD servers, one for hot/live data, and one for cold storage of data (but still on HDDs). What I've got to figure out now though, is how to share one tape drive between Xubuntu (which runs LTFS) and PBS, so that I would be able to back up host data AS WELL AS VMs/CTs via PBS.
Adding a line to /etc/fstab is “hacking it in” ? I thought we’re system administrators, not GUI clickers 😂 (Although, you’re right, it would be nice if it was exposed in the web UI, but at least it’s very easy to do)
@@Comeyd I'll still count it as hacking it. Using a network share as a datastore is not a supported configuration, and I've seen posts on their forums from developers that it is not recommended, which means they can make changes that cause it to fail (potentially silently) at any time and it's not their fault. Personally I don't want to rely on unsupported configurations with backup systems.
@@aaron57422 they support generic directories, it’s in their documentation as supported. When they say “unsupported” they mean that if the underlying file system is unreliable it’s not their fault if it breaks. As long as the file system works, you won’t have an issue…
Can you use this to migrate from a physical server to another ? And what is the best way to migrate proxmox server ? My setup is all lxcs made using the proxmox helper scripts
I run PBS in a LXC container on every node in my cluster to get around the "inception" problem. I just remote sync 2 from 1 and after an hour I remote sync 3 from 2, every night. If the primary pbs1 host or container goes down, I just adjust things so pbs2 (or pbs3) is the primary PBS target. It would take a total catastrophe to lose all 3 PBS backups. The LXC containers are allocated 512Mb ram, 4 cores and 4 GB storage (for PBS itself) with whatever dedicated storage device passed through.
Recommendation for a FOSS general backup solution w/ deduplication that suppots bare-metal restores and has agent for Windows machines? I was looking at Bacula but seems kinda difficult to setup (I am still considering giving it a try tho)
I would like to use it but... PBS needs storage. Plex needs storage. Torrents needs storage. And I need storage for my files. I can't just multiply physical machines in my room. They all must be on one machine. But no idea how to set up it the entire thing. Having a physical machine dedicated to PBS alone is not a viable option.
I have set up Proxmox Backup as a VM and use my NAS, which I have connected via NFS, as data storage. This works pretty well. I also have an small UnRAID server on which I mirror the backups. I have also set up Proxmox Backup as a VM unter UnRAID and mounted a directory on the server as data storage via the 9p protocol. If you don't have space for an additional server, you could simply use a USB disc as data storage and pass it through to a Proxmox backup VM. Not necessarily redundant but better than no backup.
i don't have enough lxc/vm to backup to considere using this PBS, my full backup take around 30min and is about 50Go then i copy it to another HDD on mt network then copy that copy to cloud storage i just keep only 14 day backup => it's about 700Go thx for the video, maybe will try it one day
Have a micro pc home lab with proxmox and just use the builtin backup. Would switch to PBS if it had host backup as thats the thing I'd use most. I dont have alot of VMs so dedupe or time or disk space saving are not critical. Would move though if I had host/config backup.
Instead of proxmox backup I use Veeam Backup. Application aware backup for applications and when restoring instead of only the complete vm you can choose vm, file or item. And ofcourse offsite backup to s3 immutable with an local performance tier as well.
@@fw_uke_ha Veeam Data Platform v12.2 ushers in a new era of data freedom, enterprise migration experience and secure streamlined operations! Now with Proxmox VE support, expanded Nutanix protection
I tested latest Veeam and I am disappointed. It fails to backup migrated Proxmox 7 to 8 /5.15 kernels to 6.10/ with disk error /still not resolved/. You need colosal infrastructure to run that elephant of a PostgreSQL mess. Even on SSD, 16GB RAM and 10GB networking its super slow backing up Proxmox VMs. It does not have VLAN support so its almost useless when you need keep it to spawn Veeamworker and Recover VM to Proxmox. Its first agentless backup for Proxmox, but Nakivo is already demoing same functionality for partners. I even used Active Backup from Synology with agents and its free and succesfuly recovered VM that way. Still PBC is fastest with Dedup.
hey tim yesterday i tried to install proxmox over a m2 drive , first i tried using ext4 and stuck at 3% then i read that someone suggest switch to zfs everything whent ok til i made an apt update and upgrade i restart and zpool error appear ? sugestións ? btw i just bougth a ssd so maybe i will use this drive for boot and the other one for storage
I done this before and works pretty good. The only problem is that like pve when I mount a smb share on my nas pve or pbs keeps my nas awake. So I have to work around and schedule a unmount and mount when needed.
What Proxmox PBS is missing, is support for ZFS snapshots (i.e. full or incremental backup from existing snapshots, or making snapshots itself and backing them up later)
It would be nice to have an interface like PBS, but you can use Sanoid/Syncoid to automate snapshots and remote backups for the zfs datasets you use for VM and CT volumes.
@@chestergregg8668 That's for zfs backups to zfs in another system pretty much, but I need to back up local zfs to tape... and Proxmox PBS is terrible for that
@@elminster8149 Sorry I missed that your VMs are on your storage. Most of us keep our VMs local and backup to a storage target. But even in your case, snapshots on TrueNas are not the same as doing a backup from proxmox. Proxmox backup from the host or using PBS stops, freezes or suspends the VM or container to ensure no data is being written while doing the backup. In your case you could potentially have corrupted files or missing data.
@@wojtek-33 Hmm, Truenas uses ZFS which is a copy on write file system. I don't see how I could have corrupted files during snapshot creation on that basis. Think I'll stick with what I have as PBS doesn't seem to offer anything I need.
@@elminster8149 Your VMs are constantly reading writing data in addition to what ever services they are performing. In addition to whatever is in RAM. TrueNas snapshots are meant for data at rest. This question and the issues snapshoting live data are well documented in the truenas forum.
I was using the standard PVE backup solution for my VMs and LXCs (8 in total). I ran them every weeknight and kept the last 5 versions of the backup. It was taking up over 1TB of storage on my Unraid server. I switched to PBS and I'm taking backups every 2 hours now for all VMs/containers, and I keep the last 120 versions. The backups are only taking up 750GB on my PBS server. That's pretty amazing considering I am now taking backups 12 times a day, instead of just once every weeknight, and keeping 10 days of backups instead of just 5.
I think proxmox developers should integrate PBS into the Proxmox VE for better and more efficient management. IMHO.
so if the host dies the backups are also lost? or am i missing your piont?
@@hawolex2341I usually don't backup to internal storage but external!
@@hawolex2341 It depends on how you integrate it. You could implement it as a kind of remote control so that you have separate hosts but direct access from one interface, similar to clusters.
The backup server is integrated as a data source. Currently you can schedule, start and restore backups from Proxmox VE. This is sufficient for most operations.
@@hawolex2341 I think they meant being able to manage them from the same interface, rather than a different interface.
Similar to how you can manage entire PVE clusters from the same interface.
@@hawolex2341host will be HA backing up remotely
I run PBS in a raspberry pi via docker. Works perfectly and it's a great use case for a raspi.
witch kind of pi ? rpi4 ? and you have connecter a usb drive to it ? please share more detail, thanks
how do you get significant amounts of storage on an Rpi?
@@Skullfurious on a pi 4, via USB. On a pi 5 you have more options because of the PCI slot.
I do! Dedicated little HP machine with a 10tb drive in it Works awesome !! Good Video Tim!
A couple of things that don't get mentioned much:
1. PBS requires more CPU than you might expect. I have it running on an old Microserver with the Dual Core AMD CPU and only getting 25MB/s or so. The CPU does a lot of hashing and compression to get incremental backups working properly
2. PBS can be easily run in an LXC container within another PVE, no need to have it on dedicated hardware. This allows you to pass through ZFS Mount points
With all that said it's a fantastic platform and it adds a lot of functionality. Live restores are so quick and easy and can really save your butt.
You do not need a CT, you can make a hybrid, this is a Debian machine. In the documentation you find how to install PBS on a PVE machine, check it out. One of my 3 PBS is a hybrid and after a year of testing I have had no problems.
as the first point, PBS doest not use a lot of CPU per se, but prefers very fast cores over lots of core.
As a MSP, we have over 40 different PBS installation, the biggest one on a 40 cores / 80 threads xeon server w/ 8 nvme drives in raid 10 and guess what? Another client with a SFF PC (Intel N100) and 2 sata SSD in mirror performs nearly 2.2x better while doing backup or restores over 25Gb LAN.
Yeah....I'm not really sure how CPU intensive the backup operations actually are, on the system that's hosting said PBS.
The verification tasks are CPU intensive, but as far as I can tell, the compression in preparation for the backup, appears to be running on my Proxmox server side rather than on the PBS side.
And I would think that would make sense because you want to compress the backup of your VM/CT that you're backing up, BEFORE sending it over the wire to the PBS, so that you actually end up sending less data over.
I don't think that it would really make sense for the PBS to be running the compression, over the wire, rather than having the Proxmox server do the compression, when you ask it to execute a backup task, and then sending the compressed result over to said PBS.
This has been my observation about how it works so far.
Just what I needed as I'm at that crossroads with my backup and I wanted to go the route of using a PBS or continue doing the backups from within VE on my NUC.
i was just thinking how i could reclaim some space from my truenas share lol. now i have a reason to start using PBS - thanks tim!
A note to PBS.
If you have a storage server like TrueNas, Unraid, OMV or the like, you can mount an NFS, SMB or another type, into PBS, and use that for the location for backups.
That also means you can have a PBS in each of your Proxmox servers connected to the same share, or whatever you like. So you could also put your backups in a TrueNAS cluster, so you have "cluster backup" which is AWESOME. Thanks for the video, even know I already knew all of this :)
Does PBS run in a container or vm if installing on the proxmox host? I've heard others recommend not to install pbs on the actual just but another machine.
@@kevinhilton8683
I'm running PBS in a LXC container running on a QNAP NAS right now.
Therefore; as such, you can absolutely install it on the same system as your Proxmox server, but if your Proxmox server dies on you, then you won't be able to access your Proxmox Backup Server Web UI, to be able to perform restores, which can cause problems for you, if that happens (and you want to try and restore/recover your Proxmox host and/or any VMs/CTs that was running on it).
How do you mount SMB into PBS?
In my lab I am running this and first got turned on to this backup solution from a great detailed tutorial from Jims Garage....My dedupe factor is currently 310, but I am doing some crazy stuff with taking snaps every 30min just so if I mess anything up, I just quickly revert to an old backup. Great for messing around in your lab without fear of losing anything really. Honestly no downsides. I also cannot wait for better clients in the future for better bare metal restores and backup other OS and hosts.
I do a nightly backup just via PVE,without retention, and then have a full backup routine with retention Job Prune and GC using PBS, all backing up on the NAS through a NFS share. The PBS is on a VM on the Synology (a meager DS918+ with 8gb of RAM). Setting up the PBS VM was a breeze in DSM. I used 2 cores and 2gb of ram for the initial setup, script install and updating of PBS , and then dialed it back to 1 core and 1gb for the working setup, works a charm. I'm also running a PVE VM on the Synology to run as always on as the third node in my Mini Lab 2x Mini PC cluster, that VM is set at 1 core and 2gb, no issues there so far either! Now if I could only get the Task Manager to actually run a schedule script to start and stop the PBS VM in DSM on a schedule, and an easy way to backup the PVE environments, it would be pretty damned near perfect for my modest needs.
Amazing timing. I was looking for another project and went back over your older videos and did a PBS install just two days ago. I shall watch this one to see if I've missed something important but it's working very well and I LOVE the ability to restore files instead of the whole damn thing!
I set it up over a year ago - it’s been so helpful a few times already.
I so agree with you. After just testing out the first generation of BPS I have been using it. Running 3 of them now to be able to test all of the features. One is a hybrid, a PVE and then installed on top the PBS, it’s Debian. Used the wiki/documentation for setting it up and after a year I have had no issues nor any problems.
PSA: the default vzdump backup is not a proper error checking backup, if let's say you have RAM corruption you would never know your backups are also corrupted. I learned the hard way you will not be able to restore those backups.
PBS on the other hand is perfect and does backups in a way they cannot get corrupt.
So when you can always go for PBS do not rely on the pve vzdumps.
Thanks for the heads up. I'm going to try and get a PBS setup tonight.
I love PBS. I’ve had it in operation since setting up my first PVE host and actually used this in favor of building a cluster because I didn’t have enough stuff running to justify a cluster. With that said I’m up to 3 PVE machines but still continue to use PBS on the regular and it’s saved me several times.
Good follow up Video, Thanks Tim. Was good to compare your settings against mine and test adjustments
I've been using it for some time and it's great and well integrated. Be aware it's much slower on HDD because of the way it works (lots of folders/small chunks). I used to backup on my SSD and synchronize with my HDD later.
Unlike VMs with block devices, LXCs rely on file system which make backup & restore slower with high IO delay... All files need to be read before the backup. You can even freeze your system carelessly. Hopefully they have some options to mitigate this.
I think ZFS is the way to go for LXCs.
Checkout the new metadata "change detection" mode that should be present in recent versions of PVE
@@nevermetme thanks! It looks promising from what I've read. I don't think my LXCs have high frequency data changes so I might benefit from it.
I would just set up Veeam as they recently introduced Proxmox backups. Currently trying to set it up, tho running into some issues still. But this way I only have 1 backup server for both Windows and Linux VMs, even if I need to use their agent due to PCI passthrough. With PBS it's "technically" possible but not ideal...
I’m a huge veeam fan, but the current implementation does yet provide application aware backup. Looking forward to this coming in a future release. This is a show stopper for me implementing this in production.
@@wizdudedoes the proxmox native solution have app aware backups?
@@90DaysOfDevOps @wizdude
For those that are uninitiated, can you explain what you mean by "app aware" backups?
Thanks.
So far, I've been able to back up multiple CTs which has the same GPU passed through to all of said CTs/between said CTs, without any issues.
@@90DaysOfDevOps no
Oh, thank you 🤩 it's time for me to use it again too
@techno Tim - I noticed you started signing (American Sign Language) THANK YOU at the end of your videos. Are you doing this on purpose? Is is directed at a particular viewer(s)? What's the deal?
Hi! I've done it for years. A large company I worked for always prioritized web and app accessibility and I learned a lot about how to make software more accessible. This also gave me many opportunities to work with people who had low/no vision and also those without hearing. This gave me such a new perspective on not only software but also life. Since then, I have always tried my best to make the software I write accessible as I can, and that has also been transferred over to my videos too 😄 I try to make the best subtitles I can and while I don't know much ASL, it's one way that I try to show my appreciation as well as teach it to those curious.
@@TechnoTim Being a fully abled person, when I first saw this I was enamoured by your consideration for the less so. Great work! Can't wait for Host backups - urrgh always feels like the weak spot in my DR
@@TechnoTim Awesome! I am a hearing person, but have a passion for the deaf community and ASL. Nice job!
I always enjoy your content. I took some time today to set up PBS to run as a VM on my Synology NAS. I'm looking forward to seeing the space-saving benefits!
@@JohnWalsh-uw3cj thank you! Please report back!
Sounds great. I will install it at home in my Proxmox Cluster as A VM with a NFS Share from my virtualized TrueNAS.
thanks for reminding me to at least throw this up to me proof my VMs, been putting it off while i try to get a NAS set up
Currently backing up OMV data to PBS with the PBS client. Works great ☺️
Just deployed it in a LXC container running on my QNAP NAS to see how well (or how poorly) this is going to work.
*edit*
I wished that you had covered the use of namespaces in this video so that you can create different namespaces for the different Proxmox systems that people might be backing up to said PBS, if they have more than one Proxmox server that they want to back up.
(You need to define the data store, and then you can add the namespace, and then on the Proxmox server that you want to backup, you can add the namespace to the backup task, and then back over on the PBS, you can see it by clicking on the datastore, and then clicking the namespace drop-down (which normally defaults to "root" to be able to see all of the VMs/CTs that have been backed up.)
Very useful feature, I think.
My favorite feature is LIVE restore of VM. I have my pfsense router as a vm in my proxmox. More then once I have screwed up a setting and could not figure out what I did, I had to restore from back up. While I can use a back up of a config, I would have to be smart enough to back up config before I mess around. I have backups every other day on PBS, so when I screw something up, I just live restore from yesterday or day before. Now my wife and kids don't come complain about no internet while backup is being done, because ITS LIVE. I'm sure I'll find other uses sooner then later.
Do a snapshot in proxmox of your pfsense before you tinker, and you can restore the snapshot, faster than you can restore the VM with PBS.
The three most important words in I.T. ....Backup, Backup, Backup.
Test if the restore works, thats the most what people forget
Oh! I thought the 3 most important words in IT were "more coffee please!" 😂
Actually it's Restore Restore Restore. 😜
Thanks so much Tim. Got my hands on a cheap mini pc (Lenovo ThinkCentre M710q) after watching this and it's working brilliantly.
Great to hear!
I have just implemented this myself, im backing up to a truenas NFS share mounted in the PBS vm, then backing those up to tape which is also passed through. Has saved me so much space!
How did you configure your True Nas NFS Share? I always get permission denied error :(
@@Blackincolorhad this as well. There is a solution for it but had to Google it alot. Something about how the nfs is mounted and the user, backup user from memory.
Reason why I can't remember totally as found nfs very very slow for some reason. Went back to just using host storage and using sync job between two pbs and all good so far.
@@detectiveinspekta Can you remember a guide or video where the setup in True NAS was shown? I've given up on the topic but I'm still interested
@@Blackincolor Forum post "NFS doesn't work on PBS". Basic idea was to go to the dataset in Truenas change the permission users and group to "backup". Edit /etc/exports on true nas and add in no_root_squash to the nfs share. When you mount the nfs share on pbs make sure when mounted it has the user backup and group as backup. Create the dataset. Creating the dataset took ages though which is why I just used the host storage instead. So far my duplication factor is 8.
Thanks for sharing Tim.
I run PBS directly on my proxmox homeserver, having a 18TB hdd on its own. SSDs are recommended in the documentation, but the price to backup would skyrocket, and i'm having acceptable performance for my needs. I went through a server rebuild after needing to changing drive config for zfs and it was easy as hell. install pve, add pbs repos, apt install pbs, mount hdd, point datastore to mount location, bam, vm and containers ready to restore. highly recommended
Using PBS since my new server which I got for 2 months now. I love it!
Simple solution for having this in a VM, simply install it, give it storage, then clone that VM to another node in the cluster. Then setup your backups such that the VM's aren't backing up their host nodes. Everything else can remain the same and it really works quite well.
Ah sweet, I was just trying to justify building a new system to replace one of my nodes, I mean I would be a fool if I didn't now!!! 😂
Definitely going to add PBS. I will continue to use urbackup for all my other machines and may look into getting it working for the PVEs themselves. The combo, if successful, will give me backups for everything.
As always Tim, good feedbacks ❤❤❤
Tk U for sharing, will give this a try
Moved to PBS a few months ago, and it's great. Saving about 2TB of disk space.
One thing I do is, backup the PBS VM, within proxmox to a network share. It's only about 2GB. This way if I must restore from scratch, I can just restore the PBS VM from proxmox, then restore all the VM's from PBS. Just saves time, as I don't have to install PBS and redo it's configuration.
Yes, adding PVE host backup would be the cherry on top. I look forward to the day they do this.
For those that might be reading this -- backing up the PBS VM is NOT enough to back up the actual chunks of data that constitutes the backup of your VMs and CTs.
For that, you'd have to back up the .chunks folder as well as the PBS VM/CT.
@@ewenchan1239 Correct, the chunks are on a smb share, this is just to backup the PBS itself to possibly save configuring it again.
@@mistakek
"Correct, the chunks are on a smb share, this is just to backup the PBS itself to possibly save configuring it again."
Agreed. :)
I use it. Very convenient especially when you need to restore a single file.
i use a old thin client to run PBS and it works great!
Same for me, HP Thin Client T610
I find PBS excellent for backing up my vm's, I don't really care to back up my proxmox hosts, I just do a fresh install of the host and restore the vm's, easy as pie. I'm looking forward to the day my local PBS can push a backup to a remote PBS in the cloud, so far it only works the other way with a cloud server pulling a copy of the backups on your local PBS.
As for pros and cons, incremental backups are really fast to do, but rather slow to restore.
PVE nodes easily run PBS for another PVE node in the cluster.
Backing up outside the PVE cluster to a dedicated PBS adds another level of resiliency.
Thanks, this is a great video. What are people using for physical host backups in a homelab?
I run my on gigabyte brix with N3350 cpu 8 g ram and 2TB hdd :)
There is one major limitation with PBS - it doesn’t support “application aware backups” like Veeam on VMware. This means I can’t use proxmox or PBS with windows based application servers such as SQL and Exchange. I was hoping Veeam for PVE would resolve this but it’s still not an available feature. If this functionality is added into PBS then the solution would be awesome. If you are a Linux shop only then you won’t be affected.
I wouldn’t call that a major limitation….
You can fix it in two steps
1) ensure guest agent is installed and enabled in Proxmox
2) setup a backup job inside of the VM that exports data to a directory inside VM regularly.
Step 1 ensures that any writes to disk are consistent, step 2 ensures that there is a definite known backup state available.
The deduplication of PBS will still works just fine as well!
Have terabytes of data in your SQL server? Any backup solution is going to be more difficult with that; however, using a backup solution inside of the VM itself isn’t really that difficult to do or manage if needed…
Being able to add windows and macOS options would be cool and even adding iOS and Android devices would make it an awesome backup solution.
Maybe someday!
I have been using PBS for a while. Works great, easy and noob friendly!
Do you mind sharing the link for mounting nfs shares for backup diretly on linux file systems? Or may I am blind, as I do not find him as mentioned!
Sorry! It's now included in the video notes!
@@TechnoTim Thx a lot!
You should mount the NFS drive using fstab on the host, then bind mount that to any VM or LXC .
Hi! Great video! Does file level restore work with windows VMs ?
It is important to have at least an zfs raid. The deduplication reuses chunks between the snapshots and if one chunk is corrupted, all snapshots referencing this chunk are broken. In the worst case all/most backups for one or multiple VMs may not restorable (e.g chunk with partiton table is identical in all snapshots and corrupted). This is the reason you should always use at least zfs raid1 which will detect and repair bitrod etc. Follow zfs best practice (pass-trough disks/storage controllers directly), because it will safe you when you need the backups the most. You may want to set up an secondary remote sync on a smaller box. Schedule regular GC and Verify (there is a setting to verify snapshots after x days) runs.
When the snapshots and chunk count grow over time you will notice that anything involving the dataset is getting slower each day. Then you can add an SSD for L2ARC and more RAM. Tis will speed up everything. While RAM is faster the L2ARC can be reused after a reboot (if configured to do). Also RAM is expensive.
I realize that I’m wasting tons of space on my NAS by avoiding this project. Those backups are my lifeline though. I’ve had to go to them in very hard situations multiple times and I just sleep well knowing that each backup is totally separate with no chance of snapshots corrupting one another. I’m running out of space though so I will be forced soon to pick up another mini optiplex or equivalent. Doesn’t need to be more than $200. It’s on my list.
I am using PBS as well virtualised on my second Proxmox VE. I backup my Proxmox VE with Rescuezilla. It does not take much time as I do not store any vm's on the Proxmox VE boot drive itself.
Currently i'm just using an old HDD (13y/o WD:Green) in my PVE server. when that drive dies, or i up the storage in my NAS. I'll probably going to move over to PBS virtualised in TrueNAS.
the de-dupe function is awesome
Great video as always!! Speaking of backups - Tim, how do you backup your regular ol workstations? I'm using Arch and trying to establish an incremental backup solution to my NAS, somewhat like Apple's timemachine. I've heard people use rsync but I am having some challenges with that.
Okay, so what do I do when I only have spare laptops, external HDDs? Mabye I could use a laptop for PBS with external HDD that will be syncing to my TrueNAS Core all the time?
I love PBS. It saved me many times !
cool. nice to see the dedup advantages but for homelab and proxmox running on an SSD wonder if the advantages of backup are useful for most. If only Truenas widely supported VM's and containers a hypervisor would not be necessary.
TrueNAS is a Hypervisor and NAS hybrid. Scale more so than Core. Way back my homelab was only my TrueNAS and some VirtualBox and KVM/QEMU on my PC and an old PC I got for free from work.
Using Proxmox backup server 1.1.1. Still works after 3 years. Luckily never needed it. Time to upgrade 👌
Good video Tim! Thanks for sharing it with us!💖👍😎JP
Nice video again Tim!
Don't forget you can also run it as a container in your NAS and call it a day.
DagNabbit Tim. Looks like i'll be exploring a PBS VM on my unraid server....
@TechnoTim, Is there a benefit to using PBS over, say, backing up to a Synology NAS?
I currently backup my VMs to an NFS share on my NAS, but am considering if it would be worth my while keeping it all "in-house" and backing up to PBS instead.
My existing VMs are only small - Debian 12 with mostly 16Gb of storage and at least 2Gb or at most 4Gb RAM and based on what I previously ran as physical Raspberry Pi SBCs.
I stopped using pbs cause it doesn't work well with containers. It sends the entire container over the network to the pbs server to do dedup there every time you backup. Since i have some large multi-TB volumes, it just takes far too long.
pve-zsync is much better if both server and backup use zfs.
Thanks for the video. What do you think about running this in a VM in PVE, with another VM running Unraid or TrueNAS? Would not be my only proxmox host node.
Great Vid, I am thinking of giving this a whirl on my unraid server. Hopefully it will be smooth sailing :) Thoughts before doing this >?
how do you backup your proxmox pve hosts? I personally didn't go further then just DD the entire disk nightly and store them gzipped on nfs.
Can you recommend good backup software for containers and volume docker ?
Great video, but reflection in your glases are killing me :) Why not setup lights at an angle ? :D
More Tape drive backup content 🙏 🙏
It's funny that the tape backup is "what year is this even?", but they don't support network share targets unless you hack them in.
My main frustration with it is needing to have another host running another VM to support it. It would be much better for me if at least some of it (mostly the dedup) was just part of PVE.
I assure you, tape is live and well in the enterprise space. Unbeatable in terms of $/tb for archival storage
I run LTO-8 tape backup in the basement of my home.
I think that the last time that I ran the numbers, at somewhere north of between 150-200 TB, it becomes cheaper (on a $/TB basis) to use tape than having HDDs, even if you've got two HDD servers, one for hot/live data, and one for cold storage of data (but still on HDDs).
What I've got to figure out now though, is how to share one tape drive between Xubuntu (which runs LTFS) and PBS, so that I would be able to back up host data AS WELL AS VMs/CTs via PBS.
Adding a line to /etc/fstab is “hacking it in” ?
I thought we’re system administrators, not GUI clickers 😂
(Although, you’re right, it would be nice if it was exposed in the web UI, but at least it’s very easy to do)
@@Comeyd I'll still count it as hacking it. Using a network share as a datastore is not a supported configuration, and I've seen posts on their forums from developers that it is not recommended, which means they can make changes that cause it to fail (potentially silently) at any time and it's not their fault. Personally I don't want to rely on unsupported configurations with backup systems.
@@aaron57422 they support generic directories, it’s in their documentation as supported.
When they say “unsupported” they mean that if the underlying file system is unreliable it’s not their fault if it breaks.
As long as the file system works, you won’t have an issue…
Proxmox backup server when backup the host?
Can you use this to migrate from a physical server to another ?
And what is the best way to migrate proxmox server ?
My setup is all lxcs made using the proxmox helper scripts
The promeage mention, lets go.
I run PBS in a LXC container on every node in my cluster to get around the "inception" problem. I just remote sync 2 from 1 and after an hour I remote sync 3 from 2, every night. If the primary pbs1 host or container goes down, I just adjust things so pbs2 (or pbs3) is the primary PBS target. It would take a total catastrophe to lose all 3 PBS backups. The LXC containers are allocated 512Mb ram, 4 cores and 4 GB storage (for PBS itself) with whatever dedicated storage device passed through.
If i make a VM in windows with PBS, could i backup my proxmox effectively?
I have a mini pc running PVE with lots of lxc's.
Where is the link in your description for how you used NFS for the backup? You said in the video you would post it.
Thank you for the reminder! I just finished up the video notes and it's live. Sorry! It's now in the description!
Recommendation for a FOSS general backup solution w/ deduplication that suppots bare-metal restores and has agent for Windows machines? I was looking at Bacula but seems kinda difficult to setup (I am still considering giving it a try tho)
I would like to use it but... PBS needs storage. Plex needs storage. Torrents needs storage. And I need storage for my files. I can't just multiply physical machines in my room. They all must be on one machine. But no idea how to set up it the entire thing. Having a physical machine dedicated to PBS alone is not a viable option.
So run it in a VM
I have set up Proxmox Backup as a VM and use my NAS, which I have connected via NFS, as data storage. This works pretty well.
I also have an small UnRAID server on which I mirror the backups. I have also set up Proxmox Backup as a VM unter UnRAID and mounted a directory on the server as data storage via the 9p protocol.
If you don't have space for an additional server, you could simply use a USB disc as data storage and pass it through to a Proxmox backup VM. Not necessarily redundant but better than no backup.
Can confirm, PBS is fabulous.
i don't have enough lxc/vm to backup to considere using this PBS,
my full backup take around 30min and is about 50Go
then i copy it to another HDD on mt network
then copy that copy to cloud storage
i just keep only 14 day backup => it's about 700Go
thx for the video, maybe will try it one day
sometimes when you restore a vm useing proxmox or proxmox backup they lose network capability :(
Have a micro pc home lab with proxmox and just use the builtin backup. Would switch to PBS if it had host backup as thats the thing I'd use most. I dont have alot of VMs so dedupe or time or disk space saving are not critical. Would move though if I had host/config backup.
So what do you use to back up the actual host?
i just finished my last proxmox project lol and everything is humming along back into it i guess lol
Instead of proxmox backup I use Veeam Backup. Application aware backup for applications and when restoring instead of only the complete vm you can choose vm, file or item. And ofcourse offsite backup to s3 immutable with an local performance tier as well.
How is the deduplication?
As far as I know, this is no open source, right? PS: I looked at there website and there is no proxmox solution out right now, still in development.
@@fw_uke_ha Veeam Data Platform v12.2 ushers in a new era of data freedom, enterprise migration experience and secure streamlined operations! Now with Proxmox VE support, expanded Nutanix protection
I tested latest Veeam and I am disappointed. It fails to backup migrated Proxmox 7 to 8 /5.15 kernels to 6.10/ with disk error /still not resolved/. You need colosal infrastructure to run that elephant of a PostgreSQL mess. Even on SSD, 16GB RAM and 10GB networking its super slow backing up Proxmox VMs. It does not have VLAN support so its almost useless when you need keep it to spawn Veeamworker and Recover VM to Proxmox. Its first agentless backup for Proxmox, but Nakivo is already demoing same functionality for partners. I even used Active Backup from Synology with agents and its free and succesfuly recovered VM that way. Still PBC is fastest with Dedup.
hey tim yesterday i tried to install proxmox over a m2 drive , first i tried using ext4 and stuck at 3% then i read that someone suggest switch to zfs everything whent ok til i made an apt update and upgrade i restart and zpool error appear ? sugestións ?
btw i just bougth a ssd so maybe i will use this drive for boot and the other one for storage
Seems the ir a bug in the installation when you are doing a update / upgrade.
I done this before and works pretty good. The only problem is that like pve when I mount a smb share on my nas pve or pbs keeps my nas awake. So I have to work around and schedule a unmount and mount when needed.
Could restore some old retrospect backup ?
how about the idea to install PBS on Truenas VM or another sort of?
Hey Tim, any chance you're thinking about doing a video series on Proxmox Mail Gateway?
@@JustinLeif sounds cool but I’d first have to host my own mail 😅
@@TechnoTim Thank you for the reply :) Keep up the great videos! Love'em, learn something new every time or a different way of doing it ;)
@@JustinLeif thank you!
Why not proxmox HA with ceph?
idea for a video: update your proxmox tutorial
What Proxmox PBS is missing, is support for ZFS snapshots (i.e. full or incremental backup from existing snapshots, or making snapshots itself and backing them up later)
It would be nice to have an interface like PBS, but you can use Sanoid/Syncoid to automate snapshots and remote backups for the zfs datasets you use for VM and CT volumes.
@@chestergregg8668 That's for zfs backups to zfs in another system pretty much, but I need to back up local zfs to tape... and Proxmox PBS is terrible for that
Good video but not really a tutorial and it was a bit rushed through. Nonetheless, it definitely can help increase awareness to the PBS project.
But isn't dedup built into zfs? So you just need to store all data on zfs?
That’s a different type of dedup
Is it possible to add google drive to pbs for offline backup ? Ever tried ?
You can use rclone to mount your google drive to a mount point.
@@michaelgleason4791 of course I know that. My question was did anyone try it or not.
How to get proxmox backup server a license? Looks like its costly for home users
It's free and open soource. You only have to pay if you want support and a subscription.
@@TechnoTim ah, thank you
How is this better than Truenas (my VM storage target) snapshots and snapshot replication to a backup NAS?
Space, time, network traffic as he explained. He showed that his backups on PBS used 1TB vs 27TB. Then you can still backup that 1TB to TrueNas.
@@wojtek-33 Snapshots perform the same way, so I don't see that as a difference.
@@elminster8149 Sorry I missed that your VMs are on your storage. Most of us keep our VMs local and backup to a storage target. But even in your case, snapshots on TrueNas are not the same as doing a backup from proxmox. Proxmox backup from the host or using PBS stops, freezes or suspends the VM or container to ensure no data is being written while doing the backup. In your case you could potentially have corrupted files or missing data.
@@wojtek-33 Hmm, Truenas uses ZFS which is a copy on write file system. I don't see how I could have corrupted files during snapshot creation on that basis.
Think I'll stick with what I have as PBS doesn't seem to offer anything I need.
@@elminster8149 Your VMs are constantly reading writing data in addition to what ever services they are performing. In addition to whatever is in RAM. TrueNas snapshots are meant for data at rest. This question and the issues snapshoting live data are well documented in the truenas forum.