Backing Up Home Assistant Container to Google Drive with Duplicati
ฝัง
- เผยแพร่เมื่อ 23 ต.ค. 2024
- How do you back up your Home Assistant, and other Docker Containers to Google Cloud when there is no Supervisor? I show you how to set up Duplicati to automatically back up all your Docker containers and configuration files to Google Could, Dropbox, OneDrive or Box on a daily basis. Restoring data is a breeze too!
Duplicati: www.duplicati....
Thank you for this summary video on Duplicati. It allowed me to discover the tool and to add it to my tooling. I use Borg but indeed Duplicati brings a "User frendly" side much more pleasant. On the other hand, as I configured my Stack to use mariaDB and influxDB databases, it becomes necessary to think of making a dump of the databases to avoid data corruptions. It is thus necessary to stop correctly a part of the stack. This can be automated by scripts, but it must be launched before the backup task. You will then have to go back to the state before the backup by restarting the stopped services. Duplicati is able to launch scripts before and after the execution of a backup task. I invite you to look at the --run-script-before and --run-script-after options.
Great video! As backup retention I suggest "smart backup retention". Sometimes it is good to have an "old" backup.
2 days after I saw this and set it up my SSD started acting up. This saved me SO MUCH WORK! Thank you much H.A.G.!
Loving duplicati, I switched my whole backup system to use it. So much easier to understand than restic
This is great. I've been running a custom script on bare metal to copy my config to dropbox. I've been looking for a better solution like this. It also illustrates, to me, the biggest advantage of using supervisor in that it makes it really easy to set things like this up. Looking for alternatives to some of these is not the easiest. I've never heard of Duplicati until today and I thank you for your research and the set up video. I can now expand my backups to include other config files from my other docker containers.
The supervisor is awesome, and I recommend it to 95% of people looking to use Home Assistant. Some of us just like to push the boundaries a bit more and be able to choose different options of software to use in our smart homes that don't currently come as Add-Ons.
Another great video that I followed to setup backups of my containers. Being new to containers and Home Assistant I am finding your videos a great help. The one problem I did have was that I didn't already have a \source directory so got stuck on the step trying to find the expandable 'source' directory. Sorted by creating the directory .
Glad you got it working!
Hi Alan,
first of all I have to thank you for your fantastic TH-cam videos. A week ago I only knew that Docker existed and had no idea what to do with it. My Home Assistant installation was running on an old Raspberry PI3+ with just 2GB of RAM. You can imagine how often I had to reboot the system. Today, everything runs on an old I3 Vaio notebook. I was able to integrate all the necessary components with Docker Compose. I have integrated all my Aquara components with zigbee2mqttt and the notebook switches itself on or off, depending on the state of charge of the battery.
Now I'll get to work on installing Duplicati. Maybe you have a tip for me on how to integrate the new Aquara cube T1 plus into Home Assistant. This device is not yet supported by default.
Man, that's really great what you're doing on your channel. Thanks again. All the best for you and continue to have fun and success
huge step ahead in my smarthome setup, thank you
You're welcome!
Great video! With Home assistant starting to use the database for more persistent storage (energy dashboard) I would love to understand how to properly backup the database too. Using Duplicati I corrupted my database a couple of times and I had to exclude it from the backup to avoid this from happening again... would love to hear a proper solution for including the db in the backup while avoiding the risk of corrupting it!
In general, there a 2 options. One is to stop the container gracefull BEFORE and start it afterwards, the other is to use the database build in backup tool (if there is one :-) and backup the backup.
I love to watch your video's. !
I am setting up your tutorials on an old laptop to see whats happening :-)
This is very nice, looks like my new backup solution. Thanks for making this nice guide 🙂
Thanks again Alan!
if you want to avoid file access problems you can set
environment:
# - PUID=1000
# - PGID=1000
- PUID=0
- PGID=0
in your docker-compose.yml, this will give to duplicati the root access.
To me it worked (I had a lot of warning on portainer), but I have not tried to restore to see what happens and also I am not sure if, give to a container the root privileges, can compromise the system security
Ciao
Roberto
thanks, worked for me too. i first tried chown ing the /opt to pi user but was stil getting some access warning, this worked like a charm
thanks you save me like a metric tone of research
Thank you for this.
don't use the root user!
Better use the command...
getent passwd | grep 1000
... on your host to check which user you will use with the id 1000.
If its not exist, create it with the command...
useradd -u 1000 YourUserName
... and give the user the needed access to your filesystem/directory !
learnt another nice tool to implement in my new HA container stack. thank you.
Hey mate! Firstly, great content. Been following your docker installation series and I was wondering whether you considered other options such as Restic, Kopia, Rclone, Borg before choosing Duplicati? Everywhere I look, I hear bad news about Duplicati so I'm a bit worried about this one
No, I didn't look at those other ones in any detail I'm afraid.
Okay, thanks! @@HomeAutomationGuy
Thanks for the informative video. I have noticed a problem using duplicati to backup home assistant: I find the home-assistant_v2.db database file is often corrupted during trhe baskcup process. I assume this is because the file is copied and written to simulataneously. I managed to restore an older database file but lost a few days of data (the energy data is what I like to keep and compare year to year). Have you noticed this or found a solution? I am stopping HA before the backup once a week but that is not an optimal solution.
Thanks great videos - I'm sort of following you HA builds. When I came to select the source directory there is no "source" at all !!!! What creates this the Duplicati yaml .....how can recreate it ? Or should just back up the /opt directory ?
Hi Alan, thanks for the video.
I'm not getting my folder duplication, is missing something?
volumes:
- /opt/duplicati/config:/config
- /opt:/source
And Im getting an error connecting with the drive :(
Thks
Hi, another amazing helpful video and very well put across, so easy to understand and follow. Don't know if it's helpful but I was going to do something similar but using Git, which would give something very similar to Duplicati in terms a way of recovery. It's a more geeky way to do it but also a bit more powerful in the way of version control. Anyway thanks again!
Hey, thank for the great video, first!
in the "Source data" step, I cannot find docker folders and specifically the HA one. what do I do? it can take me days to search it :\ my "source" folder has nothing under it.
Thanks!
Great tutorials. Thank you very much.👍
hi Alan, I want to use my rpi to backup my server. My backups will be my opt folder and a smb share with photos. Is it smart to install duplicati on my rpi to keep the workload of my server?
This will be the main purpose of running the rpi.
The rpi will be at another place and connected through wireguard to my home network.
Thx!
Hi! Great content! I would consider using "static" docker image version in my docker-compose file when using automatic backups. Doing so, when I need to restore a backup, including the docker-compose file, I know I'm running the files that matches the correct version. Then I could rely on a home-assistant automation to know if I run the latest version or not.
Good catch! Great suggestion
this video is great but doesn't address that you need to stop the containers before backing up, otherwise the backups are likely going to get corrupted and you will only find out when you actually need to restore them...
Thank you for this great video. I did't know duplicate until now. During my first testing I got a lot of file access error warnings. Is this because the containers are still running and accessing some of the files? For example some portainer file are in the list or my influxdb. Do you have also warnings during your backup?
I do, but I ignore most of them. I'm not as worried about backing up the database history of things, I'm more interested in backing up my configuration files etc. Ideally we would shut down all our containers before we back them up to prevent these warnings.
can I use a backup to install on a different house like my sister's house just delete my IP address and other stuff that links to my personal setup along with my devices that she doesn't have?
Thanks for tutorial. I am no expert just asking this question based on hearsay Would this method corrupt the files of active and running containers. I have heard if a file is backed up while it was being written/changed it can be damaged.
Yeh, that is a risk on some types of containers or applications. I personally haven't had this happen with any of the ones I am using.
fantastic! Thanks for a great video. As for backups, I am not clear where lovelace sores its config. I have to assume it's is the database. Is there a way to rebuild a homeassisant instance with just a handful of files? If so, which ones are needed?
I believe the lovelace config is stored in the .storage folder under your Home Assistant directory.
I have mapped my volumes as per Alan's example, however when I browse the 'Source Data' folder on the Duplicati page, I see nothing. I can only browse directories on the host I am using to view the Duplicati page. Any clues?
You have to look for the source folder under the computer tree, not the separate folder that shows below it.
@@GeoffAumaugher you are a saviour! I did the same thing as the guy above you! thanks!
Brilliant, thank you! 👍
Wait, i think im missing a video between you explaining the difference between explaining the difference between HAOS or the docker install and this video. You already installed docker and set up the VSC environement. Where is that video?
Check the playlist
Why do you create a docker compose file, start it up from the command line, then go into Portainer to check that it's running, when you manage all your compose scripts as Stacks in Portainer? Is it because it 'breaks' the good thing you've got going with VS Code?
I have Portainer running in a container, and it seemed a little dicey and counter intuitive to me to manage my containers from a container.... If that makes sense. It would probably be fine to do it all in Portainer, it just seemed scary to me 🤷🏻♂️🤣
I've been following your HA as a container video and have everything working great - thanks! backup was the missing link and I went ahead and rolled my own using borg directly from the host OS. I've been shutting down HA, MQQT and zigbee2mqqt pre backup to avoid corrupting their databases. Does your solution avoid having to do this?
It doesn't avoid doing this. I personally have never shut down my containers and have yet to corrupt any databases, but that doesn't mean it's a good idea to follow my lead!
@@HomeAutomationGuy I figured out it's possible to dump the db's with a little effort but it's so much easier to run a bash script on the host OS with cron that stops dockers, does that backup (borg for me), and restarts the docker. After the initial backup this only involves 60 seconds of downtime.
Will this solution do its backups to a local nas??? I really want everything possible within my own network... And should we script to stop all Docker containers that would be actively writing to databases, then restart them after writing backups???
Yep, it sure can. As long as the NAS is accessible from your Duplicati Docker Container it will be able to use it as a target. You can use SCP, FTP or Samba to write to it.
I am not sure on the best practises for stopping the docker containers. I personally don't do it, and I've not had problems yet, but I've heard some people having issues with sqlite databases.
Awesome. Super helpful videos.
I see there is the option to run a script before and after doing the back up. it would be great to stop HA container before doing the backup to avoid any DB corruption.
That's a good idea!
Thx heaps for this, I had to turn off "Prevent cross-site tracking" in Safari to get the to work otherwise it kept saying login failed, guess its something to do with cookies
Can i use it to do the back up to my NAS ?
Yep
thanks
Great video thanks a lot!
I am not able to make duplicati run, when I look at the logs I see the following:
s6-svscan: warning: unable to iopause: Operation not permitted
s6-svscan: warning: executing into .s6-svscan/crash
s6-svscan crashed. Killing everything and exiting.
s6-supervise s6-linux-init-shutdownd: fatal: unable to iopause: Operation not permitted
Also, the duplicati folder shows empty
I've solved this by setting:
image: duplicati/duplicati:latest
Nope. I get the same error when restoring. Yesterday it was with Debian 12 but today I setup an Ubuntu server and again get exactly the same error. Anyone know what I am doing wrong. I am trying to restore a single file with a timestamp to the original location
2024-03-28 13:50:52 +00 - [Warning-Duplicati.Library.Main.Operation.RestoreHandler-MetadataWriteFailed]: Failed to apply metadata to file: "/source/homeassistant/config/configuration.yaml", message: Access to the path '/source/homeassistant/config/configuration.yaml' is denied.
2024-03-28 13:50:52 +00 - [Warning-Duplicati.Library.Main.Operation.RestoreHandler-NoFilesRestored]: Restore completed without errors but no files were restored
Of course, panel don't work with ssl. That would be too ez...
Yeh, that's one drawback unfortunately.
@@HomeAutomationGuy Duplicati is awesome but Oauth seems to stop working from time to time. Another thing i couldn't integrate with docker is deluge. Impossible to get the autoadd plugin working...
As italian please I beg you to pronounce "duplicati" not "duplicadi"😅
I'm sorry, I meant no offence and will aim to do better! 😥🤣
@@HomeAutomationGuy ahaha i was joking, man love your channel keep it up 🎉
Again, a great series and thank you for doing them. Although I have gone a little out of order. Finally set up a space of my TrueNAS server to backup to using FTP with duplicati.
I installed duplicati and everything went without a hitch. There were warnings about access. So after I "chown -R /opt {myuser}" the /opt directory, I deleted the old backup config and the files. Next I recreated the backup config and run the backup to no errors or warnings. I done a restore the same as you, I didn't have any errors, but there was one warning that reads, "2022-06-23 16:08:06 -05 - [Warning-Duplicati.Library.Main.Operation.RestoreHandler-NoFilesRestored]: Restore completed without errors but no files were restored"
Any ideas on this?
[EDIT] as a test I renamed my docker-compose.yaml file to docker-compose.yaml.old and then done a restore of the docker-compose.yaml using the original location and overwrite option. The file _did_ restore. The option to use "save different versions with timestamp in file name" that it does not restore the file. Is this because there was no change in the file compared to the backup?
[EDIT 2] Another test. I added some line feeds to the live docker-config.yaml file, the tried another restore using the option with the time stamp, and this time it saved. I guess Duplicati is smart enough not to restore a file that has never changed? May be something good to pass along to new users like me.
Thanks again for all you fine videos! I appreciate them. I have never used home automation anything but your series is helping me to learn loads of stuff. Now that I have a couple of zigbee devices working under zigbee2mqtt, I think I will dig out my cheap Chinese IP camera and give your Frigate tuts a try.
Joe
Thanks for the extra information and glad you're testing it all!
Good luck with Frigate!
Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: /source/homeassistant/config/.storage/auth
any idea what this error is about?