New to HA ,and this channel has definitely been our #1 online resource: *** TIP *** not sure if this is known to all, but as a 'newbie' we found out the hard way: if you set up your 'super secure password' utilizing the character '@' in it (for example, to change the letter a,A with @, as in '@ppl3' instead of 'apple'), the database, logs, records, history will NOT START! Only after looking into the logs for an hour, we discovered the @ is used in the [>]recorder code as a designator for db/server location! So the only @ character in the [>]db_url line MUST BE the one between your 'password' and 'core-mariadb' FYI! :D
Just switched to MariaDB using this video after corruption with my SQLlite DB. Really appreciate the clear no-nonsense delivery of information. Surprised how easy it was to do. 5 minutes and all up and running.
Finally got round to adding an SSD to my install. These TTT videos are amazing. Every single one of them is so clear and concise. It's epic. Thanks again Lewis
@@EverythingSmartHome Hi! I have an update on the Maria DB and a possible issue. I've had to back out to my previous configuration. My hardware is an INTEL NUC Celeron 7CYJH with 8GB Ram and a 128GB SSD and I'm using NABU CASA. Basically Alexa Media TTS fails to work with Maria DB, at least it keeps requesting re-authentication and then the same again a few hours later. I backed out to the previous snapshot and all ok again. Very strange. I'll try installing again to double check, but has anyone else seen this issue?
Hello! That is strange, MariaDB doesn't have anything to do with the workings of HA, it only handles history and log book data, that is all. No integrations or anything is stored in the database. Did you check your logs before reverting? I've seen others complain that different things aren't working but it's because they've made an error in the mariadb setup and HA can't start properly, preventing it from loading components after the recorder. The HA logs should confirm this
@@EverythingSmartHome I will repeat the process to be 100% sure, but it was the only change I made. I have a policy of making one change at a time exactly for this reason, I'll feed back again. No doubt of the performance improvement on the NUC though. I was wondering if I could easily use a remote Maria DB as I run one on my Synology DiskStation for my Amateur Radio logbook.
Very important step missed - around 3:30 Restart HA, and wait and wait, make a cup of tea and wait ** Do not faff about with your config file think you have got your password wrong ....... patience is a virtue I don't have! Great Video again
I've only recently installed Home Assistant as a VM on my Desktop running Windows 10 (cause it's always on). I've watched some of your videos and find them very useful and easy to understand. Now I've got all my smart blubs, smart plugs, printer, router info, person tracking working. I'm adding MariaDB and Visual Studio Code after watching this video. Just wanted to say "Thank you". Keep it going :-)
Good info! I had recently set this up but now realized I’ve made some mistakes so will be going back and correcting them like going to an SSD card lol. One thing to note is that the history will have too much data, like all the entities so you can filter what you want in there by typing “include:” or “exclude:”. This is mentioned in the HA page
I know this video is quite old and a lot has changed with home assistant so Im wondering if things like MariaDB are still the way to go or would it be a good thing to go back to default? as always thanks for the videos looking forward to the new sensor when it comes
@@EverythingSmartHome makes sense, I did the MariaDB on a supervised Deb install and have let it run now a couple of days. I'll have a look into how the disk usage is now. Trying to read up on how I can controll how long data is kept and so on.
@@mrphyslaw i dont have nuc, they're somewhat expensive, i already have a rasp pi and a ssd is cheap as. i dont see a need for now with as little devices as i have.
@@EverythingSmartHome It can be done on a Raspberry Pi 3. Would have to be a USB to SATA Adapter with a SSD and would still need a SD Card. Just have to modify the System to use the SSD as the docker container storage. Could also be easily done with a existing system by using a directory link. Another option is a MicroSD to eMMC Adapter.
How about a video of a UPS for a Raspberry Pi? I'm living in an area with a lot of power outages and I am trying to find a solution to keep my Raspberry pi 4 running
Thanks! Tonight, I used this and your related video to setup both influxdb and mariadb with 372 day retention. I guess I'll have to use them to watch my free disk space, now ;) Tonight I also found the Plotly Graph Card (HACS) which I think I'll love. I've been using the apexcharts-card (HACS) and like it a lot.
I thoroughly appreciate your SD card warning. My strong preference is to not run Pi's from SD - I was so glad when the Pi 4's started shipping with built in support for booting from USB. I prefer to run them with m.2 SSDs. All that said, my HA now runs on a VM. I migrated a couple weeks ago and it's a LOT snappier than it was on the Pi4.
If you're seeing issues with recorder, logbook etc after switching over to MariaDB; try a shorter password and remove any symbols, restart after changes.
Another great tutorial.. Thank you.. Like the Cards format you show at the end.. Could you go into more detail for that final monitoring dashboard look?
Thank you! Sure I can look at that, they are made using "Apex" cards from HACS, there are even some example code in their Github. Hope that helps a little!
I'm surprised to hear that SQLite isn't performing well (or reliably) considering Home Assistant only has 10 days of data and this should be way, way below any threshold where performance is suffering. Of course i've never analyzed the queries (the achillieas heel of all SQL database apps) but was wondering if you had any metrics on how MariaDB had improved things.
Cheers ... sussed out my missing logbook and history. In MariaDB config I had changed the user name but hadn't updated the URL. Thanks for the kick in the arse. Been bugging me for months
Waiting to order my blue when they come back in stock (fingers crossed) before i do the mariadb switch over.. my rpi3 is busy enough :-) Another good video.... nice one
Love your video's just wondering why you would put you db_url in the configuration.yml instead of only a reference to the secrets.yml seems the better place like that
I actually do this in my configuration. the entire parameter is in the secrets.yaml file. then I reference it in the configuration.yaml like: db_url: !secret mariadb_url .... while secrets.yaml is not any more secure than the configuration.yaml file since they are colocated and not enciphered, it DOES allow me to do things like backup everything to github and not push the secrets file to github. It also allows me to easily share configuration samples with people without having to scrub them.
Great video! Using Home Assistant over a year now. Just switched to MariaDB. One question,- in the roundup you showed a couple of graphs. Which card did you used for the network network_throughput_in and network_throughput_out. Awesome if you can share this :) Kudos for the video. Keep up the good work.
HA newbie here. I'm on the 2024.2.1 version of HA and it looks like things have changed a bit with the UI since this tutorial was created. Would love an update!
@@EverythingSmartHome no problem, i've tried to watch a number of videos on the subject and they never quite seem to explain it clearly, you have a nice succint way of demonstrating it, definitely can tell you do prep and it shows
adapters that begin with en are cause its in a VM. If you are on bare metal such as a raspberry pi and using the built in ethernet,. the adapter is likely going to be "eth0" or if using built in wifi "wlan0"
Great video, really easy to follow. Any chance on doing a tutorial on energy cost monitoring. Particularly daily/weekly/monthly running cost using day/night tariffs
Great video, I've set up Maria DB a while back when I saw your video. Do you have any information/video on how to set up reporting or extract data from the events table? Keep up the great work!
Thanks for another great tutorial. It goes straight to favorites folder. Here's another potential video for us - at 0:48 it shows disk usage as full as 91%. My one is bit lower but I'm sure I'll catch you up soon enough. So what's way to clean the disk and remove old/obsolete files that are not needed in HA any longer. (and how to to delete something essential for sure). Thanks.
Yeah it would be entirely dependent on what you have running etc but are you storing any CCTV images etc? And yes on a VM, I could do with increasing the size at some point 😅
Top work sir, concise and bang on the money, I've been meaning to move off SQL-lite for a while now as I was getting table errors in the logs and the system started to feel a bit sluggish. Personally I use Netdata docker to monitor my systems though, as it's scalable and then centrally managed as I have a few low powered boxes running round the house that I need to keep an eye on. did I hear you say Node red ha ha ha ha
Despite the "speed increases" of the past 3 or 4 releases, I'm finding that with 2022.7 it takes much longer to access the HA web UI than it ever has. I'm running a VM with decent and dedicated resource allocation on a fast server and first access to HA feels like I'm on dialup.
Nice video, but this isnt working for me... meybe its because of newer verison of HA ? (2021.12.10 HA OS 7.2) but after MariaDB install whole HA is noticeably slower...
Great video.... now how about a step further by covering how to keep your database from getting out of hand using recorder to set what to record and also how to cap size of database by setting number of days... is there a simple way to wipe the database or compress it... I don’t know.... I just think there has to be a way to keep the data beast under control and from growing tooooo laaaarrrrrge.
Hey Richard, thanks! I think the default amount of data to retain is 10 days and after that it's purged, but you can lower that value if you want, as Miguel mentioned have a check of the recorder documentation
I tried but never got the feeling it was working as my database seem to constantly grow in size. Is there an way to report that the purge has occurred or... track on a panel the size of the database?
Great video. Any suggestions on manually entering some of the historical sensor data ? Example: I have about 12 utility meters that keep count of my solar gen, electric usage, grid usage and battery usage. I have 3 years of history now so it would be a sad day to lost it. I just have 4 sets of sensors for daily, monthly, yearly and all time. :-) any advice on how to add if I convert are welcome
It worked as you suggested, and I know you mentioned Sensor data will be lost, bu I was hoping I had heard you wrong. Of course, it is lost. Is there any way to get the sensor data back. Losing all the energy stats is a biggy. Is there a solution you are aware of?
Any idea how one might swap history recording to a Docker based MariaDB instance? I’m already using MariaDB for a couple of databases and I not sure of the impact of having two MariaDB instances - one in HA and one in my Docker container.
Have you tried using MariaDB with a socket connection? This would definitely be faster still. I have a feeling that socket may be obfuscated a bit because I think Maria DB is running inside a docker container and i'm not sure that HA and Maria have a shared volume where this socket can be reached.
Lewis thanks again for a great video. I followed along and all went well except I am missing a history tab. Is there something I need to add to get it to appear? I also did not see the old db file to delete?
Big problem when the supersecure password has the @ character included. The recorder handshake was not happening, and no data was accumulating. No documentation that I could find. Changed the db name, changed the core-mariadb with the IP address, and eventually I noticed that there was a @#$@ sequence in the yaml file that didn't look good at all. Beginner's mistake, I guess...
I wouldn't say that was a mistake on your part. No reason you wouldn't think @ was OK. It was only chance that I didn't think to include that character in mine. It (the HA interface) should make it clear in the settings if the password is restricted to a set of special characters.
Thanks for this video. I am also running HA virtualized using qemu/kvm and virtual manager however, I am unable to perform snapshots with the current HA qcow. I was wondering what were you using for snapshots? Thx.
Great video. I initially set up HA on an SD card and RPi 4 4G just to play around and see wat it could do. As I am planning on keeping it, is it possible to transfer HA to an SSD drive withing having to do a complete reinstall?
@EverythingSmartHome if you are running Home Assistant with Proxmox, would you continue to use the Add-On or move the database to a separate container or VM?
Great video. Hoping for some helpfull tips for more stability problems of HA. Almost daily HA is in a hung status. How can I see why this is happening ?
Hey, the main devs came out recently and said that most people should not be changing databases. any chances on updating the video so people know not to switch
I saw the same thing. Apparently Studio Code Server only shows files it can edit. Do you have Terminal installed? If so. you can do it by opening the terminal and entering the following commands: > cd /config > rm home-assistant_v2.db NOTE: instead of using the rm command (remove), I used the mv command (move - which is also used to rename) to rename it. I did this out of an abundance of caution (paranoia). That command is: > mv home-assistant_v2.db home-assistant_v2.db.bak After a few days I'll delete it with: > rm home-assistant_v2.db.bak
Hey, Lewis! What do you thing about a video thats help us switching our Raspberry Pi SD card to a SSD? Some adaptors and SSDs recommendations might be a good options for the video too!
very useful video. thanks for that. It did however leave me wondering if running the db in mardiab somehow will require me to take some steps to make sure that the database is also included in my auto backup to my nas. if anybody has any thoughts on this I would love to hear them!
Hi Lads, I understand Maria DB is sensitive to backups unless stopped prior to manual backups. Might I decide to use MariaDB in my forthcoming install , would this cause a problem with automatic routine backups with HA OS, generating corrupted files?
I need to check how my Docker got things set up, I have the "IOTstack" installer. That community is huge about InfluxDB and Grafana and then Mosquitto. Maybe this is overkill, however I wanted to ask if anyone watching this video had any experiences with InfluxDB
Excellent...thanks so much. One more question. On my test machine running HA in Docker on Synology, the temperature resource worked fine (which I was a bit surprised) but when I tried the same on my "NUC" (intel laptop running the nuc HA software install....so no VM), it did not. Any idea why?
Hi, thanks for the video... planning to move to Maria DB... just a quick question. I use RPi 3 and a SSD as external data disk. Home assistant boots from an SD card , but all data is stored on SSD connected via USB port. I heard you mention not to use it with an SD card and to move to SSD. Tried a lot, but can't get to boot from SSD on RPI 3. So only did I start using the data disk feature. So is it ok to use Maria DB on my existing setup? Or will i have to boot from SSD (possible on RPI 4's but hardly available in the market right now)
Hi, again great channel! I have semi-related question, you are suggesting not to base the HA on SD card while using mariaDB and InfluxDB. The new "Home Assistant Blue" is based on eMMC, is it different? Is it reliable enough? Or the SSD suggestion still stands? Thank you in advance!
Hello, I'm planning to move my installation to an ODROID with eMMC. Is it ok to move to MariaDB with eMMC? Or is it not recommended like an SD card install?
Im running home assistant on a VM in my Synology NAS. I also have a docker container with mariadb running for my kodi database. can i use this for home assistant too?
FYI: after installing MariaDB, MQTT communication with my Arduino was distributed and couldn’t get it working anymore. After rolling back a previous snapshot, everything worked again.
Hello Rob! That's a strange one, although it's not possible for the database to cause those errors, the database has nothing to do with the integrations or sensors, it's only stores history. You can run HA without a database. The only thing that it could have been is if there was a mistake made in the config for the database which made HA not startup properly, sometimes if one component fails to start up it can prevent those that load later than it from starting up too, like the MQTT integration. Were you able to see the history of other sensors after startup?
@@EverythingSmartHome odd thing was: I received my Arduino sensor data but could not switch my 24 relays on my Arduino anymore. Mqtt error: Client arduinoClient has exceeded timeout, disconnecting. After de-installing mariadb, all returned to normal. All other sensors were normal.
Question, I use influxDB integration for my historical data and graphs through Grafana. Moving to mariaDB, should the influx data just pick up where it left off?
Great info as always. Quick question, i got mariaDB set up as per your video and all seemed fine, then i noticed a couple of esphome devices had gone offline, any ideas? I went back to a backup i made on proxmox and they came back. Any thoughts would be appreciated, Allan
Thanks Allan, that is strange, the database doesnt contain any information about your sensors and integrations, it only contains sensor history so it's not possible for it to make your device go offline. There was potentially another issue simaltaneously that caused it to go offline? Did you check your HA logs?
@@EverythingSmartHome Hi thanks for your reply, no not yet, I have recovered to a backup just to get things going again. I will try again soon and let you know, busy decorating as well today, thanks, Allan
MariaDb can also get corrupted after power loss and it doesn't recover automatically! the workaround is to uninstall mariadb and install it again! its even worse than to rename the sqlite database. Do you have any advice?
Is it easy to migrate homeassistant from an SD card to SSD? And if I run from an SSD shouls I use the 64 bit version? And can you make a guide how to? I'm currently running on an pi4 4GB model. Thanks for the video's!
Thank you for the guide! But the processor_use entity is not created with a normal installation (no docker) on my rpi2 with a ssd. What could be the problem?
@@EverythingSmartHome yes, it is home assistant OS. I run it on an SSD by configuring it so it uses the SD card to start, then switches to the SSD. Using a RPI2 is not recommended but i have to say it runs very smoothly in my case www.creationfactory.co/2020/10/configuring-home-assistant-to-run-off.html
Hi. Great how to monitor lesson. All worked except the network monitor. Both In and Out return an unknown. I am running HA via a Hyper V VM. Is it possible that this will not work in a VM similar to your description on processor temperature? I have verified my network adapter name. Configuration check is "green" or good. Any thoughts are greatly appreciated.....
Hello and thanks! It should work in Hyper-V no problem, the only thing that won't work as mentioned is temperature. What was the error you got? If you need help feel free to hop into the discord server!
@@EverythingSmartHome I don't get an error. The state just shows "unknown" under developer tools and the history shows "0" for both in and out throughput lines. BTW, I love your channel. You really break things down into simple terms and procedures. I'l be sending you some coffee....
Thank you for explaining I had a Influxdb & Grafana Instalation running on my RP4/SSD before insatlling Mariadb - Mariadb is now running... but influx/Gafana not any longer :( - any idea?
Having moved over to MariaDb i have run into all sorts of issues with snapshots - specifically the fact that the MariaDb does not appear to function if restored from a snapshot meaning that when i run into issues upon major core releases (which i always do), the restore does not work for history/maria etc and i have to nuke it and rebuild from scratch.. Would you have any recommendations as to how to backup MariaDb? Other users in the forums have recommended stopping the service before taking a snapshot which is fine however it would be great if this could be automated (i am using the GDrive backup solution which is amazing! - apart from with this Db)
Did you ever figure this out? I was looking into moving to MariaDB, but I use the "Home Assistant Google Drive Backup" addon and I'm concerned that my restores won't work, or won't include the MariaDB.
@@davidhill8151 hi David, not as yet no however I would say that HA seems to much better of late in terms of stability that I haven’t needed to restore from backup. Also made some tweaks to my settings so Maria us used got short term only with Influx being used for long term storage, with this setup loosing the Maria would not be too much of a problem.
New to HA ,and this channel has definitely been our #1 online resource: *** TIP *** not sure if this is known to all, but as a 'newbie' we found out the hard way: if you set up your 'super secure password' utilizing the character '@' in it (for example, to change the letter a,A with @, as in '@ppl3' instead of 'apple'), the database, logs, records, history will NOT START! Only after looking into the logs for an hour, we discovered the @ is used in the [>]recorder code as a designator for db/server location! So the only @ character in the [>]db_url line MUST BE the one between your 'password' and 'core-mariadb' FYI! :D
Thank you, this really helped me out Jose
Just switched to MariaDB using this video after corruption with my SQLlite DB. Really appreciate the clear no-nonsense delivery of information. Surprised how easy it was to do. 5 minutes and all up and running.
Simplicity is the keyword for that channel. Man you make things look so simple. What a great explanation as usual. Keep the good work 💪
Appreciate it Nelson!
Finally got round to adding an SSD to my install. These TTT videos are amazing. Every single one of them is so clear and concise. It's epic. Thanks again Lewis
You are a hero sir! My home assistant owes its looks, functionality and general non broked ness to you! Thank you.
Thanks for the tip on the Maria DB, even though I have been working with HA since 0.4, I still gain a lot of ideas from your channel.
Thank you, your a true veteran of Home Assistant! Appreciate the support!
@@EverythingSmartHome Hi! I have an update on the Maria DB and a possible issue. I've had to back out to my previous configuration. My hardware is an INTEL NUC Celeron 7CYJH with 8GB Ram and a 128GB SSD and I'm using NABU CASA.
Basically Alexa Media TTS fails to work with Maria DB, at least it keeps requesting re-authentication and then the same again a few hours later. I backed out to the previous snapshot and all ok again. Very strange. I'll try installing again to double check, but has anyone else seen this issue?
Hello! That is strange, MariaDB doesn't have anything to do with the workings of HA, it only handles history and log book data, that is all. No integrations or anything is stored in the database.
Did you check your logs before reverting? I've seen others complain that different things aren't working but it's because they've made an error in the mariadb setup and HA can't start properly, preventing it from loading components after the recorder. The HA logs should confirm this
@@EverythingSmartHome I will repeat the process to be 100% sure, but it was the only change I made. I have a policy of making one change at a time exactly for this reason, I'll feed back again. No doubt of the performance improvement on the NUC though. I was wondering if I could easily use a remote Maria DB as I run one on my Synology DiskStation for my Amateur Radio logbook.
Definitely check the logs after migrating to see if there any issues, find us on discord if you need some help and we can try to get you sorted
This has fixed my biggest gripe i had with my Setup: Web UI being sluggish!! You are my Hero!! ;)
Very important step missed - around 3:30 Restart HA, and wait and wait, make a cup of tea and wait ** Do not faff about with your config file think you have got your password wrong ....... patience is a virtue I don't have!
Great Video again
Haha glad you got through it in the end 😂
Woo hoo! MariaDB up and running with all sensors spewing data! 😀
Your on a roll! 😅
I've only recently installed Home Assistant as a VM on my Desktop running Windows 10 (cause it's always on). I've watched some of your videos and find them very useful and easy to understand. Now I've got all my smart blubs, smart plugs, printer, router info, person tracking working. I'm adding MariaDB and Visual Studio Code after watching this video. Just wanted to say "Thank you". Keep it going :-)
That's awesome, thanks for the taking the time!
Good info! I had recently set this up but now realized I’ve made some mistakes so will be going back and correcting them like going to an SSD card lol. One thing to note is that the history will have too much data, like all the entities so you can filter what you want in there by typing “include:” or “exclude:”. This is mentioned in the HA page
Yes that's good information Miguel!
@@EverythingSmartHome what SSD do you recommend to buy?
I know this video is quite old and a lot has changed with home assistant so Im wondering if things like MariaDB are still the way to go or would it be a good thing to go back to default? as always thanks for the videos looking forward to the new sensor when it comes
You could also just use
Recorder:
commit_interval: 30
So the data is stored and will only be written on the sd card every 30 seconds
My old db was 120MB when i deleted it. That installation of HA did run for less than a week! Scary numbers there!
Depends how many entities I guess!
@@EverythingSmartHome makes sense, I did the MariaDB on a supervised Deb install and have let it run now a couple of days. I'll have a look into how the disk usage is now. Trying to read up on how I can controll how long data is kept and so on.
Another great video, added to the todo list for the weekend. Thank you
Thanks John! Let me know how it goes!
@@EverythingSmartHome All working as expected, thanks Lewis
Will be adding the monitoring and will be in touch soon!
Let me know how it goes!
Great video.... again!! You are awesome! Why? Because videos are high quality, quick, no-fluff, very informative. Keep up the good work!
Thanks my friend, really appreciate it! 🙏
Thanks again!
Speeded up the reaction of devices and also restarting!
Love your videos! Keep it up!
Thanks my friend!
Couldn't wait for tomorrow...or the weekend.....Job done! Big Thumbs Up.....Spot on as usual, thanks!
Awesome Paul, glad it worked!
Hey. Idea for a video: How to run home assistant on an SSD with a raspberry pi!
Just get a NUC or sff pc.
@@mrphyslaw i dont have nuc, they're somewhat expensive, i already have a rasp pi and a ssd is cheap as. i dont see a need for now with as little devices as i have.
Hey Finn, I don't have a Pi 4 to use but hopefully I can try and acquire one at some point!
@@EverythingSmartHome It can be done on a Raspberry Pi 3. Would have to be a USB to SATA Adapter with a SSD and would still need a SD Card. Just have to modify the System to use the SSD as the docker container storage. Could also be easily done with a existing system by using a directory link.
Another option is a MicroSD to eMMC Adapter.
try this video th-cam.com/video/Okf5WbJ_DAo/w-d-xo.html these steps worked for me
Brilliant, thanks for this, will get it moved over tomorrow.
Let me know how it goes Justin!
Hey, great video, but the adapter can also be named by "eth0" so it do not start with "en" :)
Good video, been meaning to setup MariaDB for a while watching this spurred me on. Now to see what difference it makes. Cheers!
Thanks Jon! Good luck!!
was hoping for this, thanks so much as always
Got you covered 👊
How about a video of a UPS for a Raspberry Pi? I'm living in an area with a lot of power outages and I am trying to find a solution to keep my Raspberry pi 4 running
Thanks! Tonight, I used this and your related video to setup both influxdb and mariadb with 372 day retention. I guess I'll have to use them to watch my free disk space, now ;) Tonight I also found the Plotly Graph Card (HACS) which I think I'll love. I've been using the apexcharts-card (HACS) and like it a lot.
Thank you. This worked great. I love your videos. Please keep them coming.
Thanks for the support Andy!
I thoroughly appreciate your SD card warning. My strong preference is to not run Pi's from SD - I was so glad when the Pi 4's started shipping with built in support for booting from USB. I prefer to run them with m.2 SSDs. All that said, my HA now runs on a VM. I migrated a couple weeks ago and it's a LOT snappier than it was on the Pi4.
Thanks for the great vids Lewis, all worked perfectly!
Excellent, glad it was useful!
great vid, everything went swimmingly.
Great job as always
Appreciate it Alejandro
If you're seeing issues with recorder, logbook etc after switching over to MariaDB; try a shorter password and remove any symbols, restart after changes.
Thank you Kevin for that info. That fixed my issue with the recorder, logbook, etc. = )
Another great tutorial.. Thank you.. Like the Cards format you show at the end.. Could you go into more detail for that final monitoring dashboard look?
Thank you! Sure I can look at that, they are made using "Apex" cards from HACS, there are even some example code in their Github. Hope that helps a little!
I'm surprised to hear that SQLite isn't performing well (or reliably) considering Home Assistant only has 10 days of data and this should be way, way below any threshold where performance is suffering. Of course i've never analyzed the queries (the achillieas heel of all SQL database apps) but was wondering if you had any metrics on how MariaDB had improved things.
Fantastic, keep up the good work!
Thanks buddy!
Cheers ... sussed out my missing logbook and history. In MariaDB config I had changed the user name but hadn't updated the URL. Thanks for the kick in the arse. Been bugging me for months
Glad you got it working!
Waiting to order my blue when they come back in stock (fingers crossed) before i do the mariadb switch over.. my rpi3 is busy enough :-) Another good video.... nice one
Nice, let me know how the blue goes! You've got lots of exciting projects now 😅
Thank you, it was helpfull !
I cannot hit Like fast enough! Lewis you are too awesome!
Thanks Mark!! Glad you enjoyed it!
Another great video - Thanks
Thanks Tony!
Great tutorial. Thanks.
Thanks for watching!
Love your video's just wondering why you would put you db_url in the configuration.yml instead of only a reference to the secrets.yml seems the better place like that
I actually do this in my configuration. the entire parameter is in the secrets.yaml file. then I reference it in the configuration.yaml like: db_url: !secret mariadb_url .... while secrets.yaml is not any more secure than the configuration.yaml file since they are colocated and not enciphered, it DOES allow me to do things like backup everything to github and not push the secrets file to github. It also allows me to easily share configuration samples with people without having to scrub them.
Great video! Using Home Assistant over a year now. Just switched to MariaDB. One question,- in the roundup you showed a couple of graphs. Which card did you used for the network network_throughput_in and network_throughput_out.
Awesome if you can share this :)
Kudos for the video. Keep up the good work.
HA newbie here. I'm on the 2024.2.1 version of HA and it looks like things have changed a bit with the UI since this tutorial was created. Would love an update!
Nice and concise, thanks. Idea for a video, MQQT, especially around naming conventions
Cheers Darren! Thanks for the suggestion!
@@EverythingSmartHome no problem, i've tried to watch a number of videos on the subject and they never quite seem to explain it clearly, you have a nice succint way of demonstrating it, definitely can tell you do prep and it shows
adapters that begin with en are cause its in a VM. If you are on bare metal such as a raspberry pi and using the built in ethernet,. the adapter is likely going to be "eth0" or if using built in wifi "wlan0"
Great video, really easy to follow. Any chance on doing a tutorial on energy cost monitoring. Particularly daily/weekly/monthly running cost using day/night tariffs
Thanks David! I'd loved too but unfortunately don't have anything I could use to demonstrate it on at the moment
Great video, I've set up Maria DB a while back when I saw your video. Do you have any information/video on how to set up reporting or extract data from the events table? Keep up the great work!
Thanks for another great tutorial. It goes straight to favorites folder. Here's another potential video for us - at 0:48 it shows disk usage as full as 91%. My one is bit lower but I'm sure I'll catch you up soon enough. So what's way to clean the disk and remove old/obsolete files that are not needed in HA any longer. (and how to to delete something essential for sure). Thanks.
He's on a VM mate, chances are he only allocated a few gigs to the install
Yeah it would be entirely dependent on what you have running etc but are you storing any CCTV images etc?
And yes on a VM, I could do with increasing the size at some point 😅
Yess!!! :D
Haha thanks for commenting Mateo!
Nice video!
Thanks Aswin! 🙏
@@EverythingSmartHome i got multiple vms running on my host machine how can i monitor they cpu usage?
What tool did you use to create the CPU, Memory, and Throughput graphs? The default Lovelace sensor graph doesn't provide that level of granularity.
Top work sir, concise and bang on the money, I've been meaning to move off SQL-lite for a while now as I was getting table errors in the logs and the system started to feel a bit sluggish. Personally I use Netdata docker to monitor my systems though, as it's scalable and then centrally managed as I have a few low powered boxes running round the house that I need to keep an eye on. did I hear you say Node red ha ha ha ha
Thanks Mark! Hopefully you can migrate soon! That's a good solution also, wouldnt go changing!
Despite the "speed increases" of the past 3 or 4 releases, I'm finding that with 2022.7 it takes much longer to access the HA web UI than it ever has. I'm running a VM with decent and dedicated resource allocation on a fast server and first access to HA feels like I'm on dialup.
Nice video, but this isnt working for me... meybe its because of newer verison of HA ? (2021.12.10 HA OS 7.2) but after MariaDB install whole HA is noticeably slower...
thanks
and another one done :p ,although not the system monitoring since i don't see the point for myself :)
Nice! 🙌 Yeah for your setup probably doesn't make an awful lot of sense!
Great video.... now how about a step further by covering how to keep your database from getting out of hand using recorder to set what to record and also how to cap size of database by setting number of days... is there a simple way to wipe the database or compress it... I don’t know.... I just think there has to be a way to keep the data beast under control and from growing tooooo laaaarrrrrge.
check out the recorder HA docs. They have what you need.
Hey Richard, thanks! I think the default amount of data to retain is 10 days and after that it's purged, but you can lower that value if you want, as Miguel mentioned have a check of the recorder documentation
I tried but never got the feeling it was working as my database seem to constantly grow in size. Is there an way to report that the purge has occurred or... track on a panel the size of the database?
Great video. Any suggestions on manually entering some of the historical sensor data ? Example: I have about 12 utility meters that keep count of my solar gen, electric usage, grid usage and battery usage. I have 3 years of history now so it would be a sad day to lost it. I just have 4 sets of sensors for daily, monthly, yearly and all time. :-) any advice on how to add if I convert are welcome
+1: I actually came to this video for that exact answer
It worked as you suggested, and I know you mentioned Sensor data will be lost, bu I was hoping I had heard you wrong. Of course, it is lost. Is there any way to get the sensor data back. Losing all the energy stats is a biggy. Is there a solution you are aware of?
Any idea how one might swap history recording to a Docker based MariaDB instance? I’m already using MariaDB for a couple of databases and I not sure of the impact of having two MariaDB instances - one in HA and one in my Docker container.
you can just create a database in your docker and use that one
Have you tried using MariaDB with a socket connection? This would definitely be faster still. I have a feeling that socket may be obfuscated a bit because I think Maria DB is running inside a docker container and i'm not sure that HA and Maria have a shared volume where this socket can be reached.
The next step up in data base is postgresQL
Lewis thanks again for a great video. I followed along and all went well except I am missing a history tab. Is there something I need to add to get it to appear? I also did not see the old db file to delete?
Big problem when the supersecure password has the @ character included. The recorder handshake was not happening, and no data was accumulating. No documentation that I could find. Changed the db name, changed the core-mariadb with the IP address, and eventually I noticed that there was a @#$@ sequence in the yaml file that didn't look good at all. Beginner's mistake, I guess...
I wouldn't say that was a mistake on your part. No reason you wouldn't think @ was OK. It was only chance that I didn't think to include that character in mine. It (the HA interface) should make it clear in the settings if the password is restricted to a set of special characters.
Thanks for this video. I am also running HA virtualized using qemu/kvm and virtual manager however, I am unable to perform snapshots with the current HA qcow. I was wondering what were you using for snapshots? Thx.
Great video. I initially set up HA on an SD card and RPi 4 4G just to play around and see wat it could do. As I am planning on keeping it, is it possible to transfer HA to an SSD drive withing having to do a complete reinstall?
Thanks! It would be possible yes, unfortunately I don't have a Pi 4 to be able to demonstrate it
@@EverythingSmartHome Have transferred HA from an SD card to SSD drive all went smoothly. Now going to install MariaDB as suggested.
@EverythingSmartHome if you are running Home Assistant with Proxmox, would you continue to use the Add-On or move the database to a separate container or VM?
Great video. Hoping for some helpfull tips for more stability problems of HA. Almost daily HA is in a hung status. How can I see why this is happening ?
Thanks! Check your logs to see what's going on!
Hey, the main devs came out recently and said that most people should not be changing databases. any chances on updating the video so people know not to switch
ever thought of adding a sensor to monitor the size of the mariadb database?
So I did everything but nowhere in VSCode can I find "home-assistant_v2.db" to delete?
I saw the same thing. Apparently Studio Code Server only shows files it can edit. Do you have Terminal installed? If so. you can do it by opening the terminal and entering the following commands:
> cd /config
> rm home-assistant_v2.db
NOTE: instead of using the rm command (remove), I used the mv command (move - which is also used to rename) to rename it. I did this out of an abundance of caution (paranoia). That command is:
> mv home-assistant_v2.db home-assistant_v2.db.bak
After a few days I'll delete it with:
> rm home-assistant_v2.db.bak
Hello, it's possible to use Grafana with MariaDB without installing InfluxDB?
Thanks for your inspire videos.
Hey, Lewis! What do you thing about a video thats help us switching our Raspberry Pi SD card to a SSD? Some adaptors and SSDs recommendations might be a good options for the video too!
Hey! I'd love to but I don't have any Pi 4s I can use, hopefully I can acquire one soon!
Looking forward for this!
About the SD warning.... how about if I use a SD Card on a IPbox with HA ?
very useful video. thanks for that.
It did however leave me wondering if running the db in mardiab somehow will require me to take some steps to make sure that the database is also included in my auto backup to my nas. if anybody has any thoughts on this I would love to hear them!
Hi Lads, I understand Maria DB is sensitive to backups unless stopped prior to manual backups. Might I decide to use MariaDB in my forthcoming install , would this cause a problem with automatic routine backups with HA OS, generating corrupted files?
I need to check how my Docker got things set up, I have the "IOTstack" installer. That community is huge about InfluxDB and Grafana and then Mosquitto. Maybe this is overkill, however I wanted to ask if anyone watching this video had any experiences with InfluxDB
Thanks for the video. Perhaps it's already been answered, but how did you do that Bandwidth card?
Hey Brian, this a custom card from HACS called "Apex"
Excellent...thanks so much. One more question. On my test machine running HA in Docker on Synology, the temperature resource worked fine (which I was a bit surprised) but when I tried the same on my "NUC" (intel laptop running the nuc HA software install....so no VM), it did not. Any idea why?
Hi, thanks for the video... planning to move to Maria DB... just a quick question. I use RPi 3 and a SSD as external data disk. Home assistant boots from an SD card , but all data is stored on SSD connected via USB port. I heard you mention not to use it with an SD card and to move to SSD. Tried a lot, but can't get to boot from SSD on RPI 3. So only did I start using the data disk feature. So is it ok to use Maria DB on my existing setup? Or will i have to boot from SSD (possible on RPI 4's but hardly available in the market right now)
Hi, again great channel!
I have semi-related question, you are suggesting not to base the HA on SD card while using mariaDB and InfluxDB.
The new "Home Assistant Blue" is based on eMMC, is it different? Is it reliable enough? Or the SSD suggestion still stands? Thank you in advance!
Thanks! eMMC generally has much more endurance and write cycles so you should be fine, this is drive dependant but yeah you will be good to go!
@@EverythingSmartHome thank you! You are amazing!
And you are awesome!! Thanks for the support
Where is the new MariaDB file database located (in case you might want to back it up seperately)?
Hello, I'm planning to move my installation to an ODROID with eMMC. Is it ok to move to MariaDB with eMMC? Or is it not recommended like an SD card install?
FYI I got a heap of errors on first boot and it turns out that you can't use certain symbols in your MariaDB password
Where is the new database located? I want to create a sensor out of it!
Im running home assistant on a VM in my Synology NAS. I also have a docker container with mariadb running for my kodi database. can i use this for home assistant too?
Anyway to do this while retaining all the sensor data?
Why sqlite less reliable ? Mariadb without high avail cluster is really more reliable ??
FYI: after installing MariaDB, MQTT communication with my Arduino was distributed and couldn’t get it working anymore. After rolling back a previous snapshot, everything worked again.
Hello Rob! That's a strange one, although it's not possible for the database to cause those errors, the database has nothing to do with the integrations or sensors, it's only stores history. You can run HA without a database.
The only thing that it could have been is if there was a mistake made in the config for the database which made HA not startup properly, sometimes if one component fails to start up it can prevent those that load later than it from starting up too, like the MQTT integration.
Were you able to see the history of other sensors after startup?
@@EverythingSmartHome odd thing was: I received my Arduino sensor data but could not switch my 24 relays on my Arduino anymore. Mqtt error: Client arduinoClient has exceeded timeout, disconnecting. After de-installing mariadb, all returned to normal. All other sensors were normal.
Is there a need for a strong password at all when it's being stored in plain text in the HA config?
Have these sensor settings changed now? I keep getting error messages when trying to type them in 🤔
How can i add configuration validation in server controls?
Question, I use influxDB integration for my historical data and graphs through Grafana. Moving to mariaDB, should the influx data just pick up where it left off?
There should be no issue since influxdb is a complete seperate standalone integration. Hope that helps!
Great info as always. Quick question, i got mariaDB set up as per your video and all seemed fine, then i noticed a couple of esphome devices had gone offline, any ideas? I went back to a backup i made on proxmox and they came back. Any thoughts would be appreciated, Allan
Thanks Allan, that is strange, the database doesnt contain any information about your sensors and integrations, it only contains sensor history so it's not possible for it to make your device go offline. There was potentially another issue simaltaneously that caused it to go offline? Did you check your HA logs?
@@EverythingSmartHome
Hi thanks for your reply, no not yet, I have recovered to a backup just to get things going again. I will try again soon and let you know, busy decorating as well today, thanks, Allan
No problem, if you need help please jump in the discord server and we can get you sorted!
@@EverythingSmartHome Cheers
MariaDb can also get corrupted after power loss and it doesn't recover automatically! the workaround is to uninstall mariadb and install it again! its even worse than to rename the sqlite database. Do you have any advice?
Is it easy to migrate homeassistant from an SD card to SSD? And if I run from an SSD shouls I use the 64 bit version? And can you make a guide how to? I'm currently running on an pi4 4GB model. Thanks for the video's!
Thanks for watching! Yes it's easy, I would just use the snapshot restore feature and do a fresh install. Hope that helps!
Thank you for the guide! But the processor_use entity is not created with a normal installation (no docker) on my rpi2 with a ssd. What could be the problem?
Your welcome! Your using Home Assistant OS? (If you are btw then this runs docker) also you have an SSD on a Pi 2?!
@@EverythingSmartHome yes, it is home assistant OS. I run it on an SSD by configuring it so it uses the SD card to start, then switches to the SSD. Using a RPI2 is not recommended but i have to say it runs very smoothly in my case
www.creationfactory.co/2020/10/configuring-home-assistant-to-run-off.html
Hi. Great how to monitor lesson. All worked except the network monitor. Both In and Out return an unknown. I am running HA via a Hyper V VM. Is it possible that this will not work in a VM similar to your description on processor temperature? I have verified my network adapter name. Configuration check is "green" or good. Any thoughts are greatly appreciated.....
Hello and thanks! It should work in Hyper-V no problem, the only thing that won't work as mentioned is temperature. What was the error you got? If you need help feel free to hop into the discord server!
@@EverythingSmartHome I don't get an error. The state just shows "unknown" under developer tools and the history shows "0" for both in and out throughput lines. BTW, I love your channel. You really break things down into simple terms and procedures. I'l be sending you some coffee....
Thank you for explaining
I had a Influxdb & Grafana Instalation running on my RP4/SSD before insatlling Mariadb - Mariadb is now running... but influx/Gafana not any longer :( - any idea?
There shouldn't be an issue, can you check your logs?
@@EverythingSmartHome THank you I found the issue ... an old non-updated IP-Adress
Having moved over to MariaDb i have run into all sorts of issues with snapshots - specifically the fact that the MariaDb does not appear to function if restored from a snapshot meaning that when i run into issues upon major core releases (which i always do), the restore does not work for history/maria etc and i have to nuke it and rebuild from scratch.. Would you have any recommendations as to how to backup MariaDb? Other users in the forums have recommended stopping the service before taking a snapshot which is fine however it would be great if this could be automated (i am using the GDrive backup solution which is amazing! - apart from with this Db)
Did you ever figure this out? I was looking into moving to MariaDB, but I use the "Home Assistant Google Drive Backup" addon and I'm concerned that my restores won't work, or won't include the MariaDB.
@@davidhill8151 hi David, not as yet no however I would say that HA seems to much better of late in terms of stability that I haven’t needed to restore from backup. Also made some tweaks to my settings so Maria us used got short term only with Influx being used for long term storage, with this setup loosing the Maria would not be too much of a problem.