Wow... thanks for the kind words. I did do a couple follow up videos on MQTT and Home Assistant... one that focuses a bit more on adding the devices on the Home Assistant side and another that focuses on adding Home Assistant MQTT discovery to your custom DIY devices. Thanks so much for watching and taking a few moments to leave a comment!
As a HA novice, I want to say thank you so much for this video and the other ones you've made - you are a great teacher, and I appreciate you making these topics so much more accessible!!
Undeniably a lot of effort put into making this video. It was very clear. I can't wait to use it in my DIY projects. I liked the part where you illustrated use cases for retention true/false. And the relatable examples with your own project. You covered everything I needed to know. You are awesome. Thank you for your time sharing this with us. What a blessing to have you on TH-cam.
Wow! Thank you so much for the kind words. I greatly appreciate the feedback on the particulars of what you found helpful. It is comments like yours that keep me motivated to make more content.
Never in my life would I have thought I'd not only be able to retain focus, but actually enjoy learning during a 30 minute video. Just for that you deserve a medal. I had a couple of projects lying around that I wanted to integrate and I figured using ESPhome was the way to go, until I found out having to upload their way of writing firmware was required. I like my C++ and wanted to be able to create projects without worrying if later I'd be able to integrate them should I wish to do so. Thank Google for letting me know MQTT existed and you for explaining it so well. The end part threw me for a bit of a loop but I figure that''ll be fine when I'm more versed in YAML. Thanks again.
Yeah... I do tend to make lengthy videos compared to the average . I try to always include the 'why' as well as the 'how' for beginners which result in long videos, but usually include chapter links for those that are a bit more advanced and want to skip certain parts. Glad this one didn't bore you too much! As I mentioned in the video, I struggled with the concepts of MQTT at first. But now it is my go-to for any of my projects that do not have a native integration. My C++ and Python starting frameworks both have MQTT stubs included... even if I don't initially intend to integrate them into Home Assistant. The MQTT is there if I ever want to integrate it.... or even control the device through another system as long as that system can publish an MQTT message. ESPHome and other firmware like Tasmota have their place. For ESPHome, if I find that I'm creating a lot of lambda raw C++ to accomplish what I need, then I figure I'm better off just writing in directly in C++ myself (I'm a somewhat late learner of C++, but I love it and prefer it over other options like Python in most cases). YAML will come to you. It's biggest issues (in my opinion) are trying to figure out if something is a dictionary without leading dashes or a list with leading dashes. And of course the very strict indentation rules. While I stick to YAML and don't really use the UI, the improvements is the capabilities with the UI means you can get away without even using YAML in Home Assistant these days, except for some really advanced or corner cases or you want complete control over where your code lives. Thanks for watching... and for taking the time to share such a nice comment. It is comments like yours that motivate me to continue making videos.
Nicely explained. I did look at using MQTT a couple of years ago, but it was above my understanding as a new user. Then I discovered ESPHome and never looked back. Finally now I'm moving into projects where I need to use C++, and MQTT seems the way to integrate them with HA. This video really helped me get my head around the nuts and bolts of it. Thankyou for the time and effort you put into helping others like myself.
ESPHome is fantastic... if you are using an ESP (or one of the handful of other related MCUs). But it is when you want or need to use something like a Raspberry Pi, Arduino or other platform/language where ESPHome cannot be used that MQTT becomes the easy and simple way to integrate into Home Assistant. I just have MQTT included in my standard Arduino/C++ template I use for any new projects so it is there to integrate into Home Assistant if or when I decide to do so. Thanks for watching and taking time to comment. Let me know if you run into any questions along the way.
Thank you for your great video explaining how to set sensors and switches. I looked throughout TH-cam 2 weeks ago for MQTT and HA videos; and not once your video was shown to me. Now, I come across it causally on their home tab; the proof the search engine is broken.
Well I'm glad you finally found me! I probably don't use enough of the right "keywords" in my titles or descriptions to 'optimize' my videos for ranking in the search results. But the TH-cam algorithm is a big secret mystery and I simply don't waste time trying to figure out the best way to get a video ranked or shown on the home page, when I could be using that time to make additional videos. Although this one had both MQTT and Home Assistant in the title... so who knows how it works? Regardless, I'm happy that you found the video helpful... and don't hesitate to reach out if you have any follow-up questions. Thanks for watching (after you finally found me)!
You are welcome! I was much the same way. I started using MQTT with Tasmota devices (well before there was a Home Assistant integration for Tasmota) and was really just copying and pasting the YAML needed without really understanding it either. It wasn't until I needed to add my own MQTT to a project that I really started to both understand it and see how useful it could be. Thanks for watching... and taking time to leave a comment. I appreciate it!
I have been using Home Assistant and MQTT. I develop ESP based devices. I love MQTT, but to retain or not has been one of those things where I play with it until it works, hopefully. However, even as you began to explain your thinking on when and how to use retain, I lightbulb went off! Thank you! That was an excellent concept and explanation. I have already fleshed out how I will implement my next device. If it works as well as I expect it to, I will go back and re-code my older devices and fix the little bits I now realize I CAN fix with this newer idea. I know I will change the implementation to suit my MQTT topic ideas, but the retain portion is great! And, again, Thank You!!
Glad it helped out! I too struggled to understand and use the retain flag properly when I first started implementing MQTT on my own ESP devices and it took a lot of troubleshooting of a device that I had that was exhibiting the "ghosting" behavior until I finally figured out what was going on and how the retain flag was causing that behavior. It's also why I now create separate state and command topics for my MQTT devices. Yes, you can get a way with using the same topic for both if handled carefully, but by having separate topics, one retained and one not, it is much easier to manage (at least for me) and avoids the issue of having a topic retained when it shouldn't be. Thanks for watching... and taking the time to leave a nice comment.
@@ResinChemTech I have used Shelly devices. I can see now that they use something similar, but it never clicked. Looking at documents never really conveys the concept, but pictures and words work WAY better. I am really ready to make a detection device for my front door and doorbell. Maybe even a light included. Thanks again! (I also subscribed. I will look over your earlier vids tomorrow. I found this one by accident. I know that is NOT what a TH-camr wants to hear, but at least it came up.)
Yeah, Shelly and Tasmota devices tend to use /tele (for telemetry) instead of /stat (for state). But both are basically serving the same purpose. I just selected /stat for my own devices as it keeps all my devices grouped together and separate from Shelly/Tasmota devices when I look at them using something like MQTT Explorer. But as you know, you can pretty much use any valid topic you like. Thanks for subscribing! I don't do a lot of social media promoting of my content, and really only use Discord with any regularity, which is why most people probably just stumble across my channel much like you did. But I'm glad you found me! Feel free to shoot me any questions or thoughts you might have on my other videos.
Unbelievably useful 101 on MQTT. You have a gift for breaking down concepts into fundamental bits and setting the base to build further understanding on. Brilliant! I'm new to HA and MQTT and am having trouble to find resources to explain the basics well in order to be able to use the vast amount of resources and examples that are already out there. This is by far the best explenation I've viewed or read. The HA team should hire you to improve the existing support documentation and tutorials.
Thanks for the kind words and I'm glad you found the video helpful. I do attempt to try to explain both the why as well as the how in most of my videos, as I find the "why" was lacking in a lot of the 'how-to' videos I watched when first starting out. It does tend to make my videos much longer, but I try to always include the chapter links so if there are parts of less interest, those sections can be skipped. I agree that the official Home Assistant documentation could be improved. Part of the issue is how rapidly Home Assistant changes. Often, I will make a video about Home Assistant and within six months that video is already out of date or contains incorrect info due to new changes. But I remember how confusing it all was when I first started, so if I can do just a little to help others just starting out, I feel like I've made at least a little bit of a contribution. Thanks for watching and taking the time to post such a kind comment. I appreciate it.
Your explanations are the best. Thanks for starting simple and going complex. Home Assistant is the most powerful tool out there and the most flexible too. Your explanation of a countdown timer let me power up my solar powered greenhouse watering system. For this project I am going to use a Pi Zero, with a POE hat, to read the gage on my house propane tank. THANKS.
You are very welcome... and I'm glad you found the video helpful. Most people (myself included) seem to struggle a bit with the concepts behind MQTT at first. I tried to break it down to something that folks might be able to relate to. But once you get the hang of it, it is one of the best ways to integrate custom devices into Home Assistant that don't have a native or add-on integration. I use timers a lot in my automations. They are very handy for a lot of different purposes. For me, timers and input_booleans are a couple of the most useful "helpers" when it comes to creating and managing automations. Thanks for watching and thanks so much for taking time to post your kind comment. I greatly appreciate it!
You are most welcome! I know a lot of folks have asked me about it... and it can be a bit confusing when first starting out. There are other alternatives, like ESPHome (as someone else mentioned), but depending on the platform and complexity of your project, ESPHome might not work or be a good fit. Even if not building your own code, other platforms like Tasmota or the newer Shelly devices either use or make MQTT available, so it's a good idea to at least have a basic understanding of how it works. Thanks for watching.. and taking time to comment!
As I mentioned, there are other ways of doing this... the MQTT device can also update the same topic as the command. But if not handled carefully, it can still lead to occasional ghosting. It takes a little bit more code up front to do it the way I am, with different stat/ and cmnd/ topics for states and 'command's, but it makes it very easy to know which messages should be retained and which shouldn't. But everyone has their own techniques or methods. A lot of people and devices, use tele/ instead of stat/. I use stat/ because when I look at my MQTT Explorer, everything under the stat/ branch is basically mine... then broken down by the device name... stat/brlamp... stat/garagelight, etc. It's just my own personal way of organizing all my MQTT topics. But it by no means the only way! Thanks for watching... and taking the time to leave a comment!
Also like the separation of command and status. This way you can see if the device got its message correctly. Thanks for showing more options how to use mqtt.
This is a really good video. The overhead with transmitting a topic say in LoRaWAN seems expensive on the bandwidth. Do you know if you can transmit a raw data field only, that may have proprietary information (no topic stuff), on the RF side and then have a custom HA integration (???) that formats this into JSON which then can be written to the MQTT Broker? If so what would that integration be? Thanks
Well, as I mentioned, MQTT was initially developed specifically for low power (often battery powered), low bandwidth applications. Do note that if you are developing your own devices/applications, the topics used are somewhat arbitrary... you can certainly create and use much shorter topics than I show as examples. Heck, you could use a topics like /a and /b if desired.. just as long as the publisher and subscriber are using the same topics. But to address your specific question, I haven't used LoRaWAN, so I can't say for sure. However, I did find this good forum thread about integrating LoRaWAN sensors into Home Assistant, including some discussion about how MQTT could be used with the process as well: community.home-assistant.io/t/integrating-lorawan-sensors-into-home-assistant/628322 Sorry I can't provide more info, but I really don't have any experience with LoRaWAN.
Speaking as a complete Home Assistant noob, my first overall impression of MQTT is "elegant, straightforward, complete". I have a couple of assumptions that I would like to get clarified. a) Is the communication between the publisher, broker, and client all TCP/IP? b) Is your recommendation (if you were to start over) to put the broker in a VM or a container versus in Home Assistant? c) What else from the Home Assistant world would you put into a VM rather than native inside Home Assistant if anything? Lastly, but most importantly I want to thank you for your videos and for sharing all your experience both positive and negative, when I watch any of your videos, I know I will always get honesty, so again, thank you!
Here are the brief answers to your questions: a) MQTT does use the TCP protocol for communication b) This is a bit too long to fully answer here, but I showed what I consider as the pros and cons of each method. Personally, I'd like to be able clone my entire Home Assistant production over to a test system. Having MQTT integrated into Home Assistant makes this more difficult. But maintaining a separate VM/Docker container, etc. also comes with its own additional overhead. As I said, the HA add-on works great, but for additional flexibility in my own use case, I'd put it in a separate install/container if I were to start over. c) For me... there isn't much of anything. If I were a NodeRed user, I would consider that in its own VM/Docker, etc. This is just because, similar to MQTT, if it was separate from Home Assistant, it would continue to run and function even if Home Assistant were to go down for some reason. Other than that, I don't really have anything else that would make sense to split out. Thanks for taking time to post a comment. If you want more details or to discuss further, you already know how to reach me!
c) You could have a look at 'IOTstack'. Some clever people have thought about this and done a load of work using Docker containers running: NodeRed, InfluxDB, Grafana, Zigbee2MQTT etc.
Good afternoon Sir..Your videos are very informative......I have a 4 relay installed in my bedroom with which I have used Nodemcu esp8266 mirco-controller. I programmed it with Arduino using Rc-switch library and Espalexa Library...it is working absolutely fine since a year.... But after seeing your video I diceded to give home assistant a try and with your guidelines I have installed home assistant on a raspberry Pi which is up and running perfectly.... I also have installed mqtt ad-on in home-assistant...and programmed Nodemcu with PUBSUB CLIENT Library I also have all my retain flag to true while publishing the relay status..... But still I experience random switching if trigger my relay with rf remote....
Well, it's a little hard to tell what is going on for sure based on the information above and without being able to see the full code, but it does sound like an MQTT issue. I would experiment with changing the retained status (and maybe use something like MQTT Explorer to monitor exactly what is being sent/retained) and see if the ghost switching goes away. If it does, then you know it is a retained flag issue. If the random switching continues, then it is due to something other then MQTT messaging. Let me know what you find.
I just wanted to let you know that TH-cam blocked your last comment. It will automatically block certain comments (those with links, certain words, etc.) But I was able to see the code. First, I see where pub3 and sub3 are defined, but your code uses other topics like pub1 and sub1 that aren't defined in the code you supplied. I'm guessing these are defined elsewhere and refer to the other relays? Again, I'd recommend that you try altering some of the publish statements and change the retained parameter from true to false and see if it resolves the ghost switching. At least that way, you can determine whether the random switching is a result of MQTT or caused by another issue. If all your MQTT topics are published with retain 'false' (at least temporarily for testing) and the random switching still occurs, then it is either not MQTT or something else (like a Home Assistant automation) is sending an MQTT topic... but you should be able to see if that occurs by watching the MQTT messages with MQTT Explorer (or other tool). But also verify with the multiple sub/pub topics that you aren't inadvertently publishing to the topic for one relay when you meant to publish for another (e.g. using the incorrect pubX topic). This could also appear to be 'random' switching. That's about the best I can offer at this point. TH-cam comments are somewhat limiting in terms of trying to assist in troubleshooting more advanced issues... especially those that involve large code blocks.
Thanks a lot for this video. I have one question. Is there a rule of building the topics? Could you reference a document? What is if I can use a sensor several times? Example "tempsensor/0/setmode" for sensor 0 and "tempsensor/1/setmode" for sensor 1. would that be correctly? To adjust the sensor address (here 0 or 1) there could be DIP switch on the sensor board for example.
There really isn't a 'rule' other than those imposed by the broker, but there are some best practices or recommendations... which of course vary widely. Some common themes however are to always use lower case letters, avoid spaces and special characters, don't start with a forward slash, etc. You can find a lot of different recommendations just by Googling "MQTT Topic best practices". Personally, I think it is more important that you just create your own standard and stick to it. Your example above is perfectly acceptable. If you don't have a utility like MQTT Explorer yet, you might install that and take a look at the topics that are currently being created by Home Assistant and any other MQTT devices already in use. For me, I already had a lot of Tasmota MQTT devices (pre-integration) which used base topics of tele/ stat/ and cmnd/ followed by a device name then the actual 'action' part of the topic (like power, brightness, etc.) so I more or less just adopted that as the standard I use. But that's no more valid than starting with a device name, followed by state or command, then the actual 'action'. Or you could group your topics by location, starting with livingroom/ bedroom/ kitchen/ and then the device and rest of the topic after that. It's really what works for you.
Thanks. mqtt seems genious. Not only for homeassistant stuff. Suddenly all of my devices can communicate via the homeassistant unit. And a great video b.t.w.
As long as you have a broker all those devices can communicate with each other. And if your broker is standalone, Home Assistant wouldn't even be required... or you could substitute another MQTT-compliant automation platform like NodeRed. While other options like APIs can have advantages, it's hard to beat the simplicity, speed and local lightweight nature of MQTT. Thanks for watching... and the kind words. Glad you found the video useful!
Hi, Thanks for creating this video. I have a custom control system for my outside lights and greenhouse temperature monitoring using (Arduino) C++ on ESP32 dev boards and NodeRed running on a Raspberry Pi Zero 2W. I have been considering using Home Assistant for a while but thought it would be difficult to integrate with what I’ve got now. You have shown me that I can use HA with what I already have and expand my system to use proprietary devices (which hopefully will be less flaky than the ones I’ve made and programmed myself). Self programmed devices are never finished, their programs can always be improved (For ‘improved’ substitute ‘broken’) when I learn something new about C++. For some of my devices a bought device would work out of the box without me having to reinvent the wheel. I already have a Raspberry Pi 4b waiting for to be used for ‘something’.
I actually just finished up another project today that uses Arduino/C++ on an ESP8266 (a D1 Mini) at this point. And of course it is integrated into Home Assistant. It has local push buttons for some of its features, but it is nice to have control of those features via Home Assistant and to be able to also use those in automations. I've gotten to the point, that my starting Arduino templates that I use for ESP8266 and ESP32 already have the MQTT code built-in.... even if I don't use it (although that is rare... I try to integrate everything into Home Assistant... even if it is nothing more than on/off or rebooting the controller remotely). And I hear you on the 'never finished'! I can't stop tweaking and adding new things to pretty much anything I build. I always think of some other feature or option that I could include in MQTT and control via Home Assistant. Thanks for watching and taking time to comment. Don't hesitate to let me know if I can help you out if or when you decide to move to Home Assistant.
First of all, thank you so much for this knowledge sharing. About the Mqtt Device, this is should be addressed to HA Mqtt broker ? Does is work by connecting in another Broker IP ? Thank you
I'm not sure I completely understand what you are asking. You normally only have one broker... this can be either the HA MQTT Add-on broker _or_ a standalone MQTT broker. You generally would not have both unless you have multiple networks or other special situations. Each device or application either publishes data TO the broker or subscribes to get data FROM the broker (or both). You connect to the broker for each device or application... and normally that connection is done by specifying the IP address of the broker, along with a User ID and password. I hope that answers the question you were asking.
In the interest of time, I could not go into all the details of the various MQTT options. But LWT and keep alive can be configured on the client side when establishing a connection to the broker (the exact format will vary slightly based on the language used). LWT is a message published by the broker on the client's behalf if the connection is lost. The keep alive time is how long to maintain an active connection between the client and broker. If you want to know more about these options (and many others), the following site is a really good source for MQTT information: www.hivemq.com/blog/mqtt-essentials-part-9-last-will-and-testament/
Some time ago, I added the Mosquitto broker as an add-on under Home Assistant, unaware of its purpose, but it was suggested in a TH-cam video. Despite never using it, I came across your video, which made it evident that the broker holds significant potential, yet its complexities were apparent. After my initial attempt to explore it, I found myself overwhelmed, prompting me to search the web for basic "how-to" information. Unfortunately, there wasn't much to be found, leading me to believe that mastering the broker requires more expertise. Presently, I feel like I would need guidance and support to make even a single device function with it. I appreciate the time you've taken to cover this intriguing topic. I will continue watching your videos, hoping to gather beginners' tips and tricks.
I agree that MQTT can be a bit confusing and overwhelming initially. I too struggled to grasp it at first. But after a little bit of study... and just playing with it by sending commands to MQTT devices and watching data come back, it started to make sense. And once you get the hang of it, you'll soon discover how great it is for tying different devices together and allowing them to "talk" to one another via the broker... even if they are speaking different base "languages". Stick with it... for me, it suddenly clicked into place once I started using it in one of my own projects.
Hi! Thx for the video! I'm struggling to find content about tasmota mqtt entitys without standard device class on HA (like MPU 6886 accelerometers/Gyros) and so, those entitys cant be shown on statistics with graphics. Could you give me some hint about how to solve it?
I'm not completely sure if I'm answering what you are asking, but do you have these entities integrated and showing in Home Assistant? In other words, can you see things like the accelerometer and gyro in the dev tools->states? If so, note that while a device class isn't necessary, to be able to plot a historical graph, the value has to be numeric. So check the actual values being returned by MQTT. If not numeric, then you may need to create a templated sensor based on the return value so that you can get a true numeric value. You should then be able to graph the history of this numeric templated sensor. Again, sorry if I am misunderstanding what you are asking.
I hope, in the next video, you can discuss thoroughly how to convert one of the Arduino projects on TH-cam so that it can be integrated into the home assistant, thank you.
Your video kicked me out of the gate. I’m new to Home Assistant’s YAML and have been struggling for some time to find anything that would show me how to setup MQTT sensors. THANKS! That being said, can you point me to any online documentation (perhaps how you figured it out) for sensor definitions? I’m old school and get frustrated meandering through the online documentation provided by Home Assistant. Specifically, I’ve learned that MQTT defined sensors cannot be modified through the UI. I’m ok with that, but would like to modify some of the parameters e.g. display precision, plus others. A document showing those options would be great. Enough Said - Thanks Again - fun to be up and running.
Glad my video was helpful! Like you, I'm "old school" and do everything in YAML in Home Assistant. Not only automations and scripts, but helpers, etc. I don't think I've used the UI for anything (when I show the UI in a video, it is just for the video and I don't actually save it). Part of the reason is that I had to learn YAML when I started out with Home Assistant (that was the _only_ way back then) and since I took the time to learn it, I'm going to use it! The other reason is that I like to keep all my automations, entities, etc. in Home Assistant packages. The UI throws all automations into a single great big automations.yaml file and you can't add your own comments to the code, control where the files are placed, etc. But for those new to Home Assistant, the UI is fantastic and I don't fault those that use it. I just prefer my YAML. As far as documentation... I hear ya' when it comes to the "official" Home Assistant documentation. Home Assistant updates so often that the documentation often doesn't keep up... or new features really aren't documented at all. I don't know if I can point you at a single source for MQTT sensors in Home Assistant as anything I learned from years ago is sure to be out of date. Learning to use templates is a huge help (but then you have to learn a bit of Jinja as well!). If you haven't watched any of Jeff's videos over at Slacker Labs ( www.youtube.com/@SlackerLabs ), he has a few tutorials on creating and using templated sensors that are pretty good. And of course I'm happy to try to help and answer any questions you might have as well. Thanks for taking time to leave a comment!
There's any video that explains how to do this integration with wled and ESP or Arduino or everything else? because it's nice to see how it works but there's no explanation how to do it actually. The video absolutely great but the screen that you show with all the options for example.. where I can find it how can I use it it is open source where I can download it it custome? It's ordered? It's in this app ? Will be happy to get more basic information.
This particular video was just meant to be a primer on MQTT and how it works. If you want to know how to actually integrate devices, then I have a couple of follow-up videos that are a bit more "how to" than this one: How to integrate devices with MQTT: th-cam.com/video/WYyPMHqMV-w/w-d-xo.html Using Home Assistant MQTT Discovery: th-cam.com/video/VHiCtZqllU8/w-d-xo.html Maybe one of those videos will provide what you are looking for.
I think I sort of covered that in the video when I talked about the retain flag. I write my code, and most well designed devices have a different topic for commands vs. states. So the command is sent on one topic (not retained) but the state is read on a different topic, which is retained. The command is sent to the device, which then returns the state... but the state change received does not cause a new command to be issued. This avoids the circular issue you are describing (if I understand what you are asking) and also prevents 'ghosting' issues when a device disconnects and reconnects to the broker and the state/command topics are the same and have been retained. If you didn't watch it, you might watch the section of the video on the use of the retained flag as it shows what I'm trying to describe.
Basic question... I must be missing something... I had everything working just as you described on a RP using HA 2021.6.5, but now moving to HA 2022.12.18 on a VM, I'm getting nothing! I manually inserted the "sensor" block with its topic in the configuration.yaml file, just as you described. Shouldn't there be a corresponding "entity" show up after the sensor's first message is received? Entity-wise I'm not seeing any indication that the new HA is aware of incoming messaging, so of course can't assign sensor entities to display cards or automations. It would seem that there's a config problem with the broker. Except that, when listening using Home Assistant's "MQTT Settings" configuration GUI, my manually published messages are reliably reported! Seems like I may have missed a "start" step somewhere?!
Well, jumping multiple versions of Home Assistant generally always results in things breaking. Between version 2021.6.5 and 2022.12.8 (I'm guessing .8 as there wasn't a .18), there is a year and a half of breaking changes to contend with! That is a LOT of breaking changes... and I know that there were multiple MQTT breaking changes in there as well. In fact, 2022.12 introduced a major change in the way MQTT entities are defined in YAML. I know you won't find this very helpful, but I always recommend that you don't fall more than a couple of versions before upgrading, or you will often find yourself in this situation. There are so many things it could possibly be, I can't even begin to tell you where to look. You can try using the Home Assistant logs as a source to see if you can find errors. I don't think it has anything to do with moving from a Pi to a VM. If you can't find the particular error, the only resort may be to drop back to the 2021.6.5 version, then apply upgrades in order until the problem occurs. And I was running the current 2022.12 when I made this video, so it is something between the 2021.6 and 2022.12 that has broken your install/MQTT. Also be sure that your MQTT add-on has been upgraded to match Home Assistant too. Don't feel too bad. I made this same mistake early in my Home Assistant journey of skipping multiple upgrades and then faced a daunting task of trying to repair Home Assistant. I've since learned and now upgrade each month so I can address any "breaking changes" as they occur and impact my install. Sorry I can't pinpoint the exact issue. I'm happy to try to help out further, but with so many version upgrades, I'm not sure where to tell you to start looking.
Hello, great video! have a question. How do you edit a Yaml file in the Home Assistant, Ul? In your example, you show a side by side by side window view in a code editor not sure which one. In my instance, I have Home Assistant running in a VM on a separate machine in proxmox I’m accessing Home Assistant through its IP address on a separate computer Chrome browser when I try to edit a default ESP home configuration for example a D1 Mini adding code to a standard binary sensor from say a get hub example or to say, create in download packages, I can’t copy and paste from say a git hub repository to add the code to the Home Assistant computer. Configurations, I’ve tried both the built-in, file editor also the built-in VS code. There’s no copy and paste. So my question is, In your set up What is your workflow for maintaining in managing your Home Assistant instance as far as configurations, working, and copying from GitHub, managing files etc.
Not completely sure what you are asking here... may be a bit different use of terminology for things. But my setup is basically the same as yours... I'm running Home Assistant on a Proxmox box and accessing it via the browser (Chrome) using Home Assistant's IP address. But I am able to cut/copy/paste code in both the File Editor and in the ESPHome editor. I'm using Windows, so the Ctrl-C, Ctrl-V for copy/paste works. I can copy text from really just about anywhere... Github, blog, another text editor, etc... and paste it into either the File Editor or the ESPHome editor within Home Assistant. You can also try a 'right-click' of the mouse to bring up the cut, copy, paste options... but for some reason, I do see cut/copy but not paste in the pop up menu in the ESPHome editor. Out of curiosity, if you copy something from say a Github example, can you paste that into a local text file (not in Home Assistant), even using something like Notepad? If you are asking something else please let me know, but I am able to copy text from outside files and paste those into either the File editor for YAML files or into the ESPHome node editor. One thought... are you logged into Home Assistant as the owner or an administrator? And do you have 'advanced' mode enabled? If either of those are not the case, it might prevent you from editing any files... and therefore, paste would not be available. Click on your icon in the lower left corner. In the profile, at the top see if the account is an owner or administrator. Then scroll down and see if the 'advanced' mode is toggled on. If you are unsure of accounts, go to the System -> People. This could likely be your issue.
@@ResinChemTech Hello and thanks for the help. Much appreciate it, so yes, advanced is selected and the keyboard shortcuts do work so everyone that’s copying pasting must be using the keyboard shortcuts, which I never have before. Mine is the same as yours. The copy is there, but not the paste with the mouse. Also, just curious how you got the side-by-side view and not sure some are using the built-in file editor and some are using the VS code not sure which is the best and some have mentioned that you need to be using Samba Share to edit the config. So I wasn’t sure what the issues were. Your videos are good. You’re very good at explaining things you must be a teacher.
Could you dive into the code for setting up an MQTT client? I am new to all this and trying to learn. Basically I am trying to control an LED Matrix running on a raspberry pi and want to control what it displays like you do on your led matrix.
Well, my matrix actually has dual controllers. If you are talking about the controller that shows patterns, etc. then that is the WLED controller and it doesn't use MQTT... it integrates into Home Assistant via an API and isn't even my code. If you are talking about the controller that shows a clock, scoreboard, countdown timer, etc. then that is custom Arduino code that I developed. You can see that complete code in the related Github repo: github.com/Resinchem/LED-Matrix-Clock_WS2812b If you look in the matrix_clock_main.ino file in the /arduino folder, you can see how I implemented MQTT. The sections that are probably of the most interest are the MQTT setup under the Setup() function (starting at line 371) and the callbacks (starting at line 581), although there are a lot of calls to publish MQTT values scattered throughout the rest of the code. If you are using a Raspberry Pi, then my guess is that you are also using Python. If so, then as I show in the video, the logic is basically the same between Arduino code and Python code... the syntax is just a bit different. So, you will have to take what I show in the Arduino code and modify it for Python. But the logic and processes are still the same. If you are new to the code and MQTT, this is probably still a bit overwhelming. It was to me at first as well. I recommend just trying to start out very simple. First, see if you can individually control a few pixels on your matrix. Once you have that, see if you can create a single MQTT command to call the procedure that turns on those few pixels. Then you just have to build out from there. It took me months to develop the code for my matrix, but I learned a ton in the process. So, hang in there, study online documentation and take it in small steps and you can get there! Good luck... if you run into any particular questions, let me know and I'll try to address them as best as I can (although there are limitations as to what I can do here in the TH-cam comments).
sorry, I'm a beginner at MQTT. I'm trying to understand this video, from a beginner's point of view, please help correct me, here are the steps that must be done: first step, install the code on Arduino, second step, setting it on the home assistant. My question, in my video look at the file location at /config/packages/basement_matrix.yaml. I can't find the /config/packages folder in my home assistant, did you create this folder, not the default one in the home assistant? Thank You
I have a /packages folder because I've opted to use Home Assistant packages for my configuration. This is not the default, which is to place everything within one big configuration.yaml file. The other alternative (which is what I used prior to packages) is to use a 'split' configuration, where you put things like lights, switch, sensors, etc. (or all your MQTT entities) in their own YAML files and then 'include:' the individual files in the main configuration.yaml file. I cover packages in another video: th-cam.com/video/VhHzBXYkVhw/w-d-xo.html (If you just want to know about packages, you can skip forward to about the 10 minute mark in this video). In addition, I'll have another video coming out this Saturday that focuses solely on the Home Assistant side of setting up MQTT entities. This will significantly expand on the short segment that talked about the Home Assistant side in this video. I think it should provide much more information and details on the Home Assistant side of setting up MQTT devices and entities.
Unless it's changed (I haven't installed a new Tasmota device in a while), Tasmota uses tele/ instead of stat/ for its "telemetry" values. You can use the same command topic for both (commands and states), but it can be tricky and certain situations can still lead to occasional ghosting. As I mentioned, there are other ways to do it, but using separate stat/ and cmnd/ topics is just the way I opt to handle retained vs. non-retained values. It leads to having to write a bit more code up front, but it's a bit easier (at least for me) to keep values that I want retained vs. ones that don't separate.
Noted. I do try to assure I'm using dark mode for most code windows (like I did for the Arduino/Python code), but the Home Assistant code editor is that blinding white. I'll be sure to try to use another method moving forward.
Thanks for this! Old hand at python/MQTT for my home alarms, but new to HA. Any chance you could post your yaml file? I would love to roll my whole system over to HA.
You are welcome! You can find the full YAML code for the dashboard, MQTT entities and automations for the project shown (matrix clock) in my Github repo: github.com/Resinchem/LED-Matrix-Clock_WS2812b If you want to see more examples of Home Assistant/MQTT YAML, then you might also take a look at my repo on the DIY Arylic amp project, as it also makes heavy use of MQTT... and also has all the Home Assistant YAML: github.com/Resinchem/Arylic-Amp-MQTT
There is really nothing special about the dashboard. It does use the custom button card, which I covered in a recent video. And you can find all the code, including the YAML code for the dashboard, in the Github repo for this project: github.com/Resinchem/LED-Matrix-Clock_WS2812b The dashboard code is published as lovelace.yaml in the /homeassistant folder of the repo. And there is some additional information in the wiki of that same Github repo.
You are most welcome. I'm glad you found it helpful. I know that MQTT was a bit confusing for me at first, but it is extremely handy for your own DIY projects once you get the hang of it.
I just discovered your channel. I'm trying to set up a USR-N540 4 port serial device server that uses MQTT. I managed to get the Mosquitto broker installed in HA but I'm going to need to find the right video on your channel to help with how to connect to such a device. I'm pretty clueless about this stuff but based on this video I suspect you have a good one on how to connect a device once you have the broker installed. Please direct me to your best video on this topic. Thanks, mfk.
I think this video is probably what you are looking for: th-cam.com/video/WYyPMHqMV-w/w-d-xo.html As long as you know the MQTT topics and types of payloads used by the device, this video shows you how to manually integrated and control that device from Home Assistant. Hope this helps you out!
Boy, that went over my head, I'm trying to get a ESP32 with a temp sensor to display in node red, esp32 is publishing, mqtt gives me following error:Client esp32-5834feab6224 disconnected, not authorised, any suggestion how to get it to authorize?
Well, it could be a number of things causing the disconnect. First, you didn't mention the broker you are using (e.g Home Assistant MQTT add-on, standalone, etc.), but whichever one it is, assure that you are updated to the latest version. Next, each client connecting to the broker must have a unique client ID. If multiple devices share the same ID, when one connects, MQTT will disconnect the other. I'm not a NodeRED user, so I'm not sure about the process that is used to connect to, and communicate with, MQTT. So I'm not much help on that front. But the 'not authorized' message generally means that you have the IP address, password or user name configured incorrectly when attempting to connect via the device. Be sure that you are using the local IP address of the broker and that you have a valid username/password for the broker. See this, which may help: community.home-assistant.io/t/mqtt-refusing-connection/437138
"MQTT add-on goes down with HA" - you mean when the whole host is rebooted? However, most of the time we only restart HA Core (the python app) which AFAIK doesn't affect the container with MQTT.
I could be incorrect, but when you do a full HA restart, it also restarts all the add-ons... including MQTT. I do know that when I restart HA, I can watch the status at the bottom and I see it specifically say "Starting MQTT, not everything will be available until it is finished...." So, obviously MQTT goes down when the whole machine and/or host goes down, but I believe the MQTT add-on, and other add-ons, are also restarted on a full system restart as well, meaning it is down.. at least briefly, which can lead to the entity errors I showed. Now, this is in the full Home Assistant install (used to be called HassOS) and not a containerized version. If you have some other type of of install, then it may well be that you can restart the core without impacting MQTT. But if using the add-on with full Home Assistant, I do believe that MQTT restarts as part of the process.
@@ResinChemTech I certainly don't know everything but what I think: "MQTT is starting..." you mentioned means that MQTT integration is starting, not the broker (which is in its dedicated container and is running uninterrupted throughout HA restart). "Entity errors" you see also because of the MQTT integration starting - so you should see the errors also if the MQTT broker runs on an independent machine. Finally, I think that HA restart itself doesn't restart any of the add-ons. Anyone can confirm or deny our polemic?
No disagreement here! I am far from an expert on the inner workings of HA! You may well be correct (and probably are!). In my particular case, I do have 3 times a week that I stop the VM to do a full backup... so for me in that case, MQTT goes down then because the entire VM is stopped and restarted. And now that I think about it, you are correct about the integration vs. the add-on... and I would see those same errors even if I had an external broker with non-retained values. Sorry for my own confusion... and lack of knowledge about the inner workings of HA! That's why I love the feedback... I often learn new things myself!
Thanks! MQTT is a terribly designed protocol, with often buggy implementations.... But it's usually the only thing we've got, so I guess we'd better learn about it!
In your explanation/example of the retain flag, IF during the time the device is disconnected, there was a CMND to the broker with ON, the STAT changes to ON, but then if the device reconnects, you now have an incongruence of state and reality since there is no CMND and the device doesn't subscribe to STAT. The light will remain off but the broker retains the state of ON. Yeah, the right solution is to get devices that subscribe; but how do you handle this situation when you don't control what the device supports?
I'm not sure I follow. If the device is disconnected when the CMND is published with an "on" command, the STAT won't be updated... since as in my example, the STAT is _only_ published by the end device... which is disconnected. When the device reconnects, it will publish its current state... which in your example is "off". In this particular example, the only thing that happens is that the CMND that was issued is simply ignored (and dropped, since it has no retain flag and the device is disconnected). If the state of the light was "off" before the disconnection... and it isn't programmed to turn on when powered up or booted... then when it reconnects, the published state will still be off. At no point in this particular example does the "state" topic become out of sync with the actual state of the light.... as it is only the light itself that is updating the retained state topic. I've seen this exact situation in Home Assistant where I have a switch tied to a custom-developed MQTT device. When I toggle the switch to 'on', the switch in Home Assistant briefly turns on... then turns right back off again on its own. This is because it toggles on briefly when the command is sent, but because the device is offline, the retained state is still off. Home Assistant quickly updates the state of the switch right back to off from the last retained STAT topic (which was off at disconnection). This generally occurs within 1-2 seconds, and is one way I know when a device is offline (or at least dropped the MQTT connection). For this particular video, it was about DIY devices that you do completely control. In my case, I am the one that wrote the firmware/code that runs on the end devices. If it is a retail device, or firmware that you didn't develop, then you are correct. You are somewhat at the mercy of what the device/firmware developer provides and if it both sends and receives on the same topic, then you can still have issues. Some devices/firmware are (in my opinion) better than others. I like Tasmota if I want to use MQTT with prebuilt firmware because it does use different topics for commands vs. states, so I can follow the same process to assure I don't get ghosting or states that are out of sync. Sorry if I am misunderstanding what you are saying... but in my experience with my own MQTT devices, using the techniques I described has never resulted in an 'out-of-sync' situation between what Home Assistant is showing for a state and the actual state of the device itself.
@@ResinChemTechYou understood and answered wonderfully. I missed a few points in the video that you clarified above. My fault. The thing I am trying to wrap my head around is how to support in the complete system, the ability to have an automated "prime" that will set state that even is one part is down, when the part comes back up, it will return to state it "should be in". I have run into some of things you mentioned. I think my solution is to follow your "scheme" but have this "prime" subscribe to the state (from the broker) and make some "decisions" based on said state. The wrinkle is managing temporary state caused by human interaction compared to the automated state in "this failure state of a device disappearing and when it reappears it may default to wrong state as to what the prime wants. (in other words, I have a more complex situation to solve). Example is outside driveway lights, tune on at sundown, turn off an hour later. But if a user turns them off stay off. If the car proximity triggers turn them on and return to the timed turn off time OR if after a short timer to turn off. Several similar scenarios of automated lights with possible user override but also trying to maintain robustness when a device gets disconnected (from reboot caused by poor code and a watchdog, bad wifi, battery power lost due to poor solar, etc).
Hey, no problem! Sometimes I tend to ramble a bit in my videos and don't always make my points as clearly as I should. For the more complex situations, I will often make use of things like input_booleans and/or timers within Home Assistant. My interior garage lights (controlled by a switch relay that uses MQTT) is sort of a more complex situation similar to what you described. The lights come on via door switches then turn off after a set time. But the wall switch can also be flipped manually by a user, regardless of door state. For this, I use an input_boolean helper (along with timers) that sort of acts like that "prime" that you mention. I can toggle or change the state of that input_boolean via various automations depending on what happens with the lights... and can return the light to a desired state based on this input_boolean. This may or may not work in your situation, but I use input_booleans a lot in my automations to help control devices and states that might also be altered by a user with a different physical device. Don't know if any of that might be helpful to you or not!
And ESPHome is great for many cases.... I use it for a lot of my own devices. But for more complex project, like the clock I show in this video, ESPHome is not an option. I also do not believe you can install ESPHome on devices like the Raspberry Pi... or other devices that might use Python or other non-Arduino based languages. I believe that he only supported platforms currently are ESP8266, ESP32 and RP2040. ESPHome is a quick and easy way to put code on custom devices, especially those that just need to connect sensors, switches, etc. But for devices that need to do more advanced local processing with highly customized code... or if you want to develop in another language like Java, then MQTT offers an easy alternative to integrating your devices into Home Assistant.
I can't even figure out how to make Home Assistant recognize the existence of the MQTT device. I have the MQTT integration in but that's it, can't find a way to tell it how to find the device, give it an IP address or anything. The device is an RS232 controller NB144. There are three downloads for configuring the device but the one that looks most important is in Chinese. Don't think there is much hope😞
You might try watching the follow up video to this one, if you haven't already, that focuses specifically on how to set up the Home Assistant side for MQTT devices. th-cam.com/video/WYyPMHqMV-w/w-d-xo.html This video was a bit more of an MQTT overview. The linked video above has much more info on the specifics of the Home Assistant side. I hope that will help a bit.
WOW---->Head exploded-->Ghosted Where does one go to start at the very basics? I looked and did not see an interface in my Home Assistant for MQTT? Sounds to me like I am on the border line for your intended audience.
Well, Home Assistant by default does not have an MQTT integration. But it's extremely easy to add it. It is a standard add-on that you can find in the add-on store under Settings. This is the add-on, so it works both as your MQTT broker and as the integration for Home Assistant. Lewis over at @EverythingSmartHome has a really good video on installing and setting up the MQTT add-on in Home Assistant: th-cam.com/video/dqTn-Gk4Qeo/w-d-xo.html Of course, you can create your own standalone MQTT broker on another device (like a Raspberry Pi, VM, old PC, etc.) and then install the MQTT integration via Home Assistant integrations. Both methods have pros and cons, but using the Home Assistant add-on broker is definitely the easiest way and doesn't require a different machine/VM, etc. Sorry to point you at other sources. Unfortunately I sometimes have to make a call as to what to show and what to skip over (and assume that folks already know... or point them elsewhere) in some of my videos simply due to the length. But I do appreciate you watching. And don't hesitate to follow up with any additional questions and I'll do my best to help you out.
I just watched it back on both the computer and TV and I'm not seeing any sync issues between the audio and video on my end. It may be a temporary TH-cam thing.
Thanks for the confirmation! It always worries me a bit when I get a report like this, but I generally check and watch back all videos after I upload them and I didn't see any issues then... or after watching again based on this comment. But it's good to know that others aren't seeing the same issue as the original comment. Thanks again for taking a moment to let me know it is OK on your end.
This is one of the best if not the BEST 101 MQTT Home Assistant video on the net! 👏 👏 thank you for making it!! 🙏
Wow... thanks for the kind words. I did do a couple follow up videos on MQTT and Home Assistant... one that focuses a bit more on adding the devices on the Home Assistant side and another that focuses on adding Home Assistant MQTT discovery to your custom DIY devices.
Thanks so much for watching and taking a few moments to leave a comment!
As a HA novice, I want to say thank you so much for this video and the other ones you've made - you are a great teacher, and I appreciate you making these topics so much more accessible!!
Wow! Thanks so much. It is comments like yours that keep me motivated to make more content. I really do appreciate it!
Undeniably a lot of effort put into making this video. It was very clear. I can't wait to use it in my DIY projects. I liked the part where you illustrated use cases for retention true/false. And the relatable examples with your own project. You covered everything I needed to know. You are awesome. Thank you for your time sharing this with us. What a blessing to have you on TH-cam.
Wow! Thank you so much for the kind words. I greatly appreciate the feedback on the particulars of what you found helpful. It is comments like yours that keep me motivated to make more content.
Never in my life would I have thought I'd not only be able to retain focus, but actually enjoy learning during a 30 minute video. Just for that you deserve a medal. I had a couple of projects lying around that I wanted to integrate and I figured using ESPhome was the way to go, until I found out having to upload their way of writing firmware was required. I like my C++ and wanted to be able to create projects without worrying if later I'd be able to integrate them should I wish to do so. Thank Google for letting me know MQTT existed and you for explaining it so well. The end part threw me for a bit of a loop but I figure that''ll be fine when I'm more versed in YAML. Thanks again.
Yeah... I do tend to make lengthy videos compared to the average . I try to always include the 'why' as well as the 'how' for beginners which result in long videos, but usually include chapter links for those that are a bit more advanced and want to skip certain parts. Glad this one didn't bore you too much!
As I mentioned in the video, I struggled with the concepts of MQTT at first. But now it is my go-to for any of my projects that do not have a native integration. My C++ and Python starting frameworks both have MQTT stubs included... even if I don't initially intend to integrate them into Home Assistant. The MQTT is there if I ever want to integrate it.... or even control the device through another system as long as that system can publish an MQTT message.
ESPHome and other firmware like Tasmota have their place. For ESPHome, if I find that I'm creating a lot of lambda raw C++ to accomplish what I need, then I figure I'm better off just writing in directly in C++ myself (I'm a somewhat late learner of C++, but I love it and prefer it over other options like Python in most cases).
YAML will come to you. It's biggest issues (in my opinion) are trying to figure out if something is a dictionary without leading dashes or a list with leading dashes. And of course the very strict indentation rules. While I stick to YAML and don't really use the UI, the improvements is the capabilities with the UI means you can get away without even using YAML in Home Assistant these days, except for some really advanced or corner cases or you want complete control over where your code lives.
Thanks for watching... and for taking the time to share such a nice comment. It is comments like yours that motivate me to continue making videos.
Nicely explained.
I did look at using MQTT a couple of years ago, but it was above my understanding as a new user. Then I discovered ESPHome and never looked back. Finally now I'm moving into projects where I need to use C++, and MQTT seems the way to integrate them with HA. This video really helped me get my head around the nuts and bolts of it.
Thankyou for the time and effort you put into helping others like myself.
ESPHome is fantastic... if you are using an ESP (or one of the handful of other related MCUs). But it is when you want or need to use something like a Raspberry Pi, Arduino or other platform/language where ESPHome cannot be used that MQTT becomes the easy and simple way to integrate into Home Assistant. I just have MQTT included in my standard Arduino/C++ template I use for any new projects so it is there to integrate into Home Assistant if or when I decide to do so.
Thanks for watching and taking time to comment. Let me know if you run into any questions along the way.
Thank you for your great video explaining how to set sensors and switches. I looked throughout TH-cam 2 weeks ago for MQTT and HA videos; and not once your video was shown to me. Now, I come across it causally on their home tab; the proof the search engine is broken.
Well I'm glad you finally found me! I probably don't use enough of the right "keywords" in my titles or descriptions to 'optimize' my videos for ranking in the search results. But the TH-cam algorithm is a big secret mystery and I simply don't waste time trying to figure out the best way to get a video ranked or shown on the home page, when I could be using that time to make additional videos. Although this one had both MQTT and Home Assistant in the title... so who knows how it works?
Regardless, I'm happy that you found the video helpful... and don't hesitate to reach out if you have any follow-up questions. Thanks for watching (after you finally found me)!
I've been using MQTT for about three years and didn't really understand it. Now I do. Thanks,
You are welcome! I was much the same way. I started using MQTT with Tasmota devices (well before there was a Home Assistant integration for Tasmota) and was really just copying and pasting the YAML needed without really understanding it either. It wasn't until I needed to add my own MQTT to a project that I really started to both understand it and see how useful it could be.
Thanks for watching... and taking time to leave a comment. I appreciate it!
I have been using Home Assistant and MQTT. I develop ESP based devices. I love MQTT, but to retain or not has been one of those things where I play with it until it works, hopefully. However, even as you began to explain your thinking on when and how to use retain, I lightbulb went off! Thank you! That was an excellent concept and explanation. I have already fleshed out how I will implement my next device. If it works as well as I expect it to, I will go back and re-code my older devices and fix the little bits I now realize I CAN fix with this newer idea. I know I will change the implementation to suit my MQTT topic ideas, but the retain portion is great! And, again, Thank You!!
Glad it helped out! I too struggled to understand and use the retain flag properly when I first started implementing MQTT on my own ESP devices and it took a lot of troubleshooting of a device that I had that was exhibiting the "ghosting" behavior until I finally figured out what was going on and how the retain flag was causing that behavior. It's also why I now create separate state and command topics for my MQTT devices. Yes, you can get a way with using the same topic for both if handled carefully, but by having separate topics, one retained and one not, it is much easier to manage (at least for me) and avoids the issue of having a topic retained when it shouldn't be.
Thanks for watching... and taking the time to leave a nice comment.
@@ResinChemTech I have used Shelly devices. I can see now that they use something similar, but it never clicked. Looking at documents never really conveys the concept, but pictures and words work WAY better. I am really ready to make a detection device for my front door and doorbell. Maybe even a light included. Thanks again! (I also subscribed. I will look over your earlier vids tomorrow. I found this one by accident. I know that is NOT what a TH-camr wants to hear, but at least it came up.)
Yeah, Shelly and Tasmota devices tend to use /tele (for telemetry) instead of /stat (for state). But both are basically serving the same purpose. I just selected /stat for my own devices as it keeps all my devices grouped together and separate from Shelly/Tasmota devices when I look at them using something like MQTT Explorer. But as you know, you can pretty much use any valid topic you like.
Thanks for subscribing! I don't do a lot of social media promoting of my content, and really only use Discord with any regularity, which is why most people probably just stumble across my channel much like you did. But I'm glad you found me! Feel free to shoot me any questions or thoughts you might have on my other videos.
Unbelievably useful 101 on MQTT. You have a gift for breaking down concepts into fundamental bits and setting the base to build further understanding on. Brilliant!
I'm new to HA and MQTT and am having trouble to find resources to explain the basics well in order to be able to use the vast amount of resources and examples that are already out there. This is by far the best explenation I've viewed or read.
The HA team should hire you to improve the existing support documentation and tutorials.
Thanks for the kind words and I'm glad you found the video helpful. I do attempt to try to explain both the why as well as the how in most of my videos, as I find the "why" was lacking in a lot of the 'how-to' videos I watched when first starting out. It does tend to make my videos much longer, but I try to always include the chapter links so if there are parts of less interest, those sections can be skipped.
I agree that the official Home Assistant documentation could be improved. Part of the issue is how rapidly Home Assistant changes. Often, I will make a video about Home Assistant and within six months that video is already out of date or contains incorrect info due to new changes. But I remember how confusing it all was when I first started, so if I can do just a little to help others just starting out, I feel like I've made at least a little bit of a contribution.
Thanks for watching and taking the time to post such a kind comment. I appreciate it.
Thank you very much, these examples helped me a lot to get the MQTT integration on HA running.
You are welcome. I'm glad my video was able to help you out.
Thanks for taking time to leave a comment.
Your explanations are the best. Thanks for starting simple and going complex. Home Assistant is the most powerful tool out there and the most flexible too. Your explanation of a countdown timer let me power up my solar powered greenhouse watering system. For this project I am going to use a Pi Zero, with a POE hat, to read the gage on my house propane tank. THANKS.
You are very welcome... and I'm glad you found the video helpful. Most people (myself included) seem to struggle a bit with the concepts behind MQTT at first. I tried to break it down to something that folks might be able to relate to. But once you get the hang of it, it is one of the best ways to integrate custom devices into Home Assistant that don't have a native or add-on integration.
I use timers a lot in my automations. They are very handy for a lot of different purposes. For me, timers and input_booleans are a couple of the most useful "helpers" when it comes to creating and managing automations.
Thanks for watching and thanks so much for taking time to post your kind comment. I greatly appreciate it!
You read my mind. Really needed this, thanks for the great explanation.
You are most welcome! I know a lot of folks have asked me about it... and it can be a bit confusing when first starting out. There are other alternatives, like ESPHome (as someone else mentioned), but depending on the platform and complexity of your project, ESPHome might not work or be a good fit.
Even if not building your own code, other platforms like Tasmota or the newer Shelly devices either use or make MQTT available, so it's a good idea to at least have a basic understanding of how it works.
Thanks for watching.. and taking time to comment!
Amazing video, thank you so much!
You are welcome. Glad you found it helpful. Thanks for taking a few minutes to leave a comment!
Excellent video as always. Interesting how you separated out the status and commands. I may redesign how I am using MQTT based on this. ~Brian
As I mentioned, there are other ways of doing this... the MQTT device can also update the same topic as the command. But if not handled carefully, it can still lead to occasional ghosting. It takes a little bit more code up front to do it the way I am, with different stat/ and cmnd/ topics for states and 'command's, but it makes it very easy to know which messages should be retained and which shouldn't. But everyone has their own techniques or methods. A lot of people and devices, use tele/ instead of stat/. I use stat/ because when I look at my MQTT Explorer, everything under the stat/ branch is basically mine... then broken down by the device name... stat/brlamp... stat/garagelight, etc. It's just my own personal way of organizing all my MQTT topics. But it by no means the only way!
Thanks for watching... and taking the time to leave a comment!
Also like the separation of command and status. This way you can see if the device got its message correctly. Thanks for showing more options how to use mqtt.
You are welcome! Glad you found something that might be helpful for you.
This is a really good video. The overhead with transmitting a topic say in LoRaWAN seems expensive on the bandwidth. Do you know if you can transmit a raw data field only, that may have proprietary information (no topic stuff), on the RF side and then have a custom HA integration (???) that formats this into JSON which then can be written to the MQTT Broker? If so what would that integration be?
Thanks
Well, as I mentioned, MQTT was initially developed specifically for low power (often battery powered), low bandwidth applications. Do note that if you are developing your own devices/applications, the topics used are somewhat arbitrary... you can certainly create and use much shorter topics than I show as examples. Heck, you could use a topics like /a and /b if desired.. just as long as the publisher and subscriber are using the same topics.
But to address your specific question, I haven't used LoRaWAN, so I can't say for sure. However, I did find this good forum thread about integrating LoRaWAN sensors into Home Assistant, including some discussion about how MQTT could be used with the process as well: community.home-assistant.io/t/integrating-lorawan-sensors-into-home-assistant/628322
Sorry I can't provide more info, but I really don't have any experience with LoRaWAN.
Speaking as a complete Home Assistant noob, my first overall impression of MQTT is "elegant, straightforward, complete". I have a couple of assumptions that I would like to get clarified.
a) Is the communication between the publisher, broker, and client all TCP/IP?
b) Is your recommendation (if you were to start over) to put the broker in a VM or a container versus in Home Assistant?
c) What else from the Home Assistant world would you put into a VM rather than native inside Home Assistant if anything?
Lastly, but most importantly I want to thank you for your videos and for sharing all your experience both positive and negative, when I watch any of your videos, I know I will always get honesty, so again, thank you!
Here are the brief answers to your questions:
a) MQTT does use the TCP protocol for communication
b) This is a bit too long to fully answer here, but I showed what I consider as the pros and cons of each method. Personally, I'd like to be able clone my entire Home Assistant production over to a test system. Having MQTT integrated into Home Assistant makes this more difficult. But maintaining a separate VM/Docker container, etc. also comes with its own additional overhead. As I said, the HA add-on works great, but for additional flexibility in my own use case, I'd put it in a separate install/container if I were to start over.
c) For me... there isn't much of anything. If I were a NodeRed user, I would consider that in its own VM/Docker, etc. This is just because, similar to MQTT, if it was separate from Home Assistant, it would continue to run and function even if Home Assistant were to go down for some reason. Other than that, I don't really have anything else that would make sense to split out.
Thanks for taking time to post a comment. If you want more details or to discuss further, you already know how to reach me!
c) You could have a look at 'IOTstack'. Some clever people have thought about this and done a load of work using Docker containers running: NodeRed, InfluxDB, Grafana, Zigbee2MQTT etc.
Good afternoon Sir..Your videos are very informative......I have a 4 relay installed in my bedroom with which I have used Nodemcu esp8266 mirco-controller. I programmed it with Arduino using Rc-switch library and Espalexa Library...it is working absolutely fine since a year.... But after seeing your video I diceded to give home assistant a try and with your guidelines I have installed home assistant on a raspberry Pi which is up and running perfectly.... I also have installed mqtt ad-on in home-assistant...and programmed Nodemcu with PUBSUB CLIENT Library
I also have all my retain flag to true while publishing the relay status..... But still I experience random switching if trigger my relay with rf remote....
Well, it's a little hard to tell what is going on for sure based on the information above and without being able to see the full code, but it does sound like an MQTT issue. I would experiment with changing the retained status (and maybe use something like MQTT Explorer to monitor exactly what is being sent/retained) and see if the ghost switching goes away. If it does, then you know it is a retained flag issue. If the random switching continues, then it is due to something other then MQTT messaging.
Let me know what you find.
I just wanted to let you know that TH-cam blocked your last comment. It will automatically block certain comments (those with links, certain words, etc.) But I was able to see the code.
First, I see where pub3 and sub3 are defined, but your code uses other topics like pub1 and sub1 that aren't defined in the code you supplied. I'm guessing these are defined elsewhere and refer to the other relays? Again, I'd recommend that you try altering some of the publish statements and change the retained parameter from true to false and see if it resolves the ghost switching. At least that way, you can determine whether the random switching is a result of MQTT or caused by another issue. If all your MQTT topics are published with retain 'false' (at least temporarily for testing) and the random switching still occurs, then it is either not MQTT or something else (like a Home Assistant automation) is sending an MQTT topic... but you should be able to see if that occurs by watching the MQTT messages with MQTT Explorer (or other tool). But also verify with the multiple sub/pub topics that you aren't inadvertently publishing to the topic for one relay when you meant to publish for another (e.g. using the incorrect pubX topic). This could also appear to be 'random' switching.
That's about the best I can offer at this point. TH-cam comments are somewhat limiting in terms of trying to assist in troubleshooting more advanced issues... especially those that involve large code blocks.
Great video!!! Thank you for posting!!
You are welcome. Thanks for taking time to post a comment!
Thanks a lot for this video. I have one question. Is there a rule of building the topics? Could you reference a document? What is if I can use a sensor several times? Example "tempsensor/0/setmode" for sensor 0 and "tempsensor/1/setmode" for sensor 1. would that be correctly? To adjust the sensor address (here 0 or 1) there could be DIP switch on the sensor board for example.
There really isn't a 'rule' other than those imposed by the broker, but there are some best practices or recommendations... which of course vary widely. Some common themes however are to always use lower case letters, avoid spaces and special characters, don't start with a forward slash, etc. You can find a lot of different recommendations just by Googling "MQTT Topic best practices".
Personally, I think it is more important that you just create your own standard and stick to it. Your example above is perfectly acceptable. If you don't have a utility like MQTT Explorer yet, you might install that and take a look at the topics that are currently being created by Home Assistant and any other MQTT devices already in use. For me, I already had a lot of Tasmota MQTT devices (pre-integration) which used base topics of tele/ stat/ and cmnd/ followed by a device name then the actual 'action' part of the topic (like power, brightness, etc.) so I more or less just adopted that as the standard I use. But that's no more valid than starting with a device name, followed by state or command, then the actual 'action'. Or you could group your topics by location, starting with livingroom/ bedroom/ kitchen/ and then the device and rest of the topic after that. It's really what works for you.
@@ResinChemTech Thanks for your feedback.
Thanks. mqtt seems genious. Not only for homeassistant stuff. Suddenly all of my devices can communicate via the homeassistant unit.
And a great video b.t.w.
As long as you have a broker all those devices can communicate with each other. And if your broker is standalone, Home Assistant wouldn't even be required... or you could substitute another MQTT-compliant automation platform like NodeRed.
While other options like APIs can have advantages, it's hard to beat the simplicity, speed and local lightweight nature of MQTT. Thanks for watching... and the kind words. Glad you found the video useful!
Hi, Thanks for creating this video.
I have a custom control system for my outside lights and greenhouse temperature monitoring using (Arduino) C++ on ESP32 dev boards and NodeRed running on a Raspberry Pi Zero 2W. I have been considering using Home Assistant for a while but thought it would be difficult to integrate with what I’ve got now. You have shown me that I can use HA with what I already have and expand my system to use proprietary devices (which hopefully will be less flaky than the ones I’ve made and programmed myself). Self programmed devices are never finished, their programs can always be improved (For ‘improved’ substitute ‘broken’) when I learn something new about C++. For some of my devices a bought device would work out of the box without me having to reinvent the wheel. I already have a Raspberry Pi 4b waiting for to be used for ‘something’.
I actually just finished up another project today that uses Arduino/C++ on an ESP8266 (a D1 Mini) at this point. And of course it is integrated into Home Assistant. It has local push buttons for some of its features, but it is nice to have control of those features via Home Assistant and to be able to also use those in automations.
I've gotten to the point, that my starting Arduino templates that I use for ESP8266 and ESP32 already have the MQTT code built-in.... even if I don't use it (although that is rare... I try to integrate everything into Home Assistant... even if it is nothing more than on/off or rebooting the controller remotely).
And I hear you on the 'never finished'! I can't stop tweaking and adding new things to pretty much anything I build. I always think of some other feature or option that I could include in MQTT and control via Home Assistant.
Thanks for watching and taking time to comment. Don't hesitate to let me know if I can help you out if or when you decide to move to Home Assistant.
First of all, thank you so much for this knowledge sharing. About the Mqtt Device, this is should be addressed to HA Mqtt broker ? Does is work by connecting in another Broker IP ? Thank you
I'm not sure I completely understand what you are asking. You normally only have one broker... this can be either the HA MQTT Add-on broker _or_ a standalone MQTT broker. You generally would not have both unless you have multiple networks or other special situations. Each device or application either publishes data TO the broker or subscribes to get data FROM the broker (or both). You connect to the broker for each device or application... and normally that connection is done by specifying the IP address of the broker, along with a User ID and password.
I hope that answers the question you were asking.
Nice, but how do you use the keep alive andlast will?
In the interest of time, I could not go into all the details of the various MQTT options. But LWT and keep alive can be configured on the client side when establishing a connection to the broker (the exact format will vary slightly based on the language used).
LWT is a message published by the broker on the client's behalf if the connection is lost. The keep alive time is how long to maintain an active connection between the client and broker. If you want to know more about these options (and many others), the following site is a really good source for MQTT information: www.hivemq.com/blog/mqtt-essentials-part-9-last-will-and-testament/
Some time ago, I added the Mosquitto broker as an add-on under Home Assistant, unaware of its purpose, but it was suggested in a TH-cam video. Despite never using it, I came across your video, which made it evident that the broker holds significant potential, yet its complexities were apparent.
After my initial attempt to explore it, I found myself overwhelmed, prompting me to search the web for basic "how-to" information. Unfortunately, there wasn't much to be found, leading me to believe that mastering the broker requires more expertise. Presently, I feel like I would need guidance and support to make even a single device function with it.
I appreciate the time you've taken to cover this intriguing topic. I will continue watching your videos, hoping to gather beginners' tips and tricks.
I agree that MQTT can be a bit confusing and overwhelming initially. I too struggled to grasp it at first. But after a little bit of study... and just playing with it by sending commands to MQTT devices and watching data come back, it started to make sense. And once you get the hang of it, you'll soon discover how great it is for tying different devices together and allowing them to "talk" to one another via the broker... even if they are speaking different base "languages".
Stick with it... for me, it suddenly clicked into place once I started using it in one of my own projects.
Hi! Thx for the video! I'm struggling to find content about tasmota mqtt entitys without standard device class on HA (like MPU 6886 accelerometers/Gyros) and so, those entitys cant be shown on statistics with graphics. Could you give me some hint about how to solve it?
I'm not completely sure if I'm answering what you are asking, but do you have these entities integrated and showing in Home Assistant? In other words, can you see things like the accelerometer and gyro in the dev tools->states? If so, note that while a device class isn't necessary, to be able to plot a historical graph, the value has to be numeric. So check the actual values being returned by MQTT. If not numeric, then you may need to create a templated sensor based on the return value so that you can get a true numeric value. You should then be able to graph the history of this numeric templated sensor.
Again, sorry if I am misunderstanding what you are asking.
I hope, in the next video, you can discuss thoroughly how to convert one of the Arduino projects on TH-cam so that it can be integrated into the home assistant, thank you.
Your video kicked me out of the gate. I’m new to Home Assistant’s YAML and have been struggling for some time to find anything that would show me how to setup MQTT sensors. THANKS!
That being said, can you point me to any online documentation (perhaps how you figured it out) for sensor definitions? I’m old school and get frustrated meandering through the online documentation provided by Home Assistant. Specifically, I’ve learned that MQTT defined sensors cannot be modified through the UI. I’m ok with that, but would like to modify some of the parameters e.g. display precision, plus others. A document showing those options would be great.
Enough Said - Thanks Again - fun to be up and running.
Glad my video was helpful! Like you, I'm "old school" and do everything in YAML in Home Assistant. Not only automations and scripts, but helpers, etc. I don't think I've used the UI for anything (when I show the UI in a video, it is just for the video and I don't actually save it). Part of the reason is that I had to learn YAML when I started out with Home Assistant (that was the _only_ way back then) and since I took the time to learn it, I'm going to use it! The other reason is that I like to keep all my automations, entities, etc. in Home Assistant packages. The UI throws all automations into a single great big automations.yaml file and you can't add your own comments to the code, control where the files are placed, etc. But for those new to Home Assistant, the UI is fantastic and I don't fault those that use it. I just prefer my YAML.
As far as documentation... I hear ya' when it comes to the "official" Home Assistant documentation. Home Assistant updates so often that the documentation often doesn't keep up... or new features really aren't documented at all. I don't know if I can point you at a single source for MQTT sensors in Home Assistant as anything I learned from years ago is sure to be out of date. Learning to use templates is a huge help (but then you have to learn a bit of Jinja as well!). If you haven't watched any of Jeff's videos over at Slacker Labs ( www.youtube.com/@SlackerLabs ), he has a few tutorials on creating and using templated sensors that are pretty good.
And of course I'm happy to try to help and answer any questions you might have as well. Thanks for taking time to leave a comment!
very nicely explained, thank you
You are most welcome. Thanks for taking time to leave a comment.
There's any video that explains how to do this integration with wled and ESP or Arduino or everything else? because it's nice to see how it works but there's no explanation how to do it actually. The video absolutely great but the screen that you show with all the options for example.. where I can find it how can I use it it is open source where I can download it it custome? It's ordered? It's in this app ? Will be happy to get more basic information.
This particular video was just meant to be a primer on MQTT and how it works. If you want to know how to actually integrate devices, then I have a couple of follow-up videos that are a bit more "how to" than this one:
How to integrate devices with MQTT: th-cam.com/video/WYyPMHqMV-w/w-d-xo.html
Using Home Assistant MQTT Discovery: th-cam.com/video/VHiCtZqllU8/w-d-xo.html
Maybe one of those videos will provide what you are looking for.
How do you avoid race conditions when commands can change state and states can alter buttons which issue commands?
I think I sort of covered that in the video when I talked about the retain flag. I write my code, and most well designed devices have a different topic for commands vs. states. So the command is sent on one topic (not retained) but the state is read on a different topic, which is retained. The command is sent to the device, which then returns the state... but the state change received does not cause a new command to be issued. This avoids the circular issue you are describing (if I understand what you are asking) and also prevents 'ghosting' issues when a device disconnects and reconnects to the broker and the state/command topics are the same and have been retained. If you didn't watch it, you might watch the section of the video on the use of the retained flag as it shows what I'm trying to describe.
Basic question... I must be missing something... I had everything working just as you described on a RP using HA 2021.6.5, but now moving to HA 2022.12.18 on a VM, I'm getting nothing!
I manually inserted the "sensor" block with its topic in the configuration.yaml file, just as you described. Shouldn't there be a corresponding "entity" show up after the sensor's first message is received? Entity-wise I'm not seeing any indication that the new HA is aware of incoming messaging, so of course can't assign sensor entities to display cards or automations.
It would seem that there's a config problem with the broker. Except that, when listening using Home Assistant's "MQTT Settings" configuration GUI, my manually published messages are reliably reported! Seems like I may have missed a "start" step somewhere?!
Well, jumping multiple versions of Home Assistant generally always results in things breaking. Between version 2021.6.5 and 2022.12.8 (I'm guessing .8 as there wasn't a .18), there is a year and a half of breaking changes to contend with! That is a LOT of breaking changes... and I know that there were multiple MQTT breaking changes in there as well. In fact, 2022.12 introduced a major change in the way MQTT entities are defined in YAML.
I know you won't find this very helpful, but I always recommend that you don't fall more than a couple of versions before upgrading, or you will often find yourself in this situation. There are so many things it could possibly be, I can't even begin to tell you where to look. You can try using the Home Assistant logs as a source to see if you can find errors. I don't think it has anything to do with moving from a Pi to a VM. If you can't find the particular error, the only resort may be to drop back to the 2021.6.5 version, then apply upgrades in order until the problem occurs. And I was running the current 2022.12 when I made this video, so it is something between the 2021.6 and 2022.12 that has broken your install/MQTT. Also be sure that your MQTT add-on has been upgraded to match Home Assistant too.
Don't feel too bad. I made this same mistake early in my Home Assistant journey of skipping multiple upgrades and then faced a daunting task of trying to repair Home Assistant. I've since learned and now upgrade each month so I can address any "breaking changes" as they occur and impact my install.
Sorry I can't pinpoint the exact issue. I'm happy to try to help out further, but with so many version upgrades, I'm not sure where to tell you to start looking.
Hello, great video! have a question. How do you edit a Yaml file in the Home Assistant, Ul? In your example, you show a side by side by side window view in a code editor not sure which one. In my instance, I have Home Assistant running in a VM on a separate machine in proxmox I’m accessing Home Assistant through its IP address on a separate computer Chrome browser when I try to edit a default ESP home configuration for example a D1 Mini adding code to a standard binary sensor from say a get hub example or to say, create in download packages, I can’t copy and paste from say a git hub repository to add the code to the Home Assistant computer. Configurations, I’ve tried both the built-in, file editor also the built-in VS code. There’s no copy and paste. So my question is, In your set up What is your workflow for maintaining in managing your Home Assistant instance as far as configurations, working, and copying from GitHub, managing files etc.
Not completely sure what you are asking here... may be a bit different use of terminology for things. But my setup is basically the same as yours... I'm running Home Assistant on a Proxmox box and accessing it via the browser (Chrome) using Home Assistant's IP address.
But I am able to cut/copy/paste code in both the File Editor and in the ESPHome editor. I'm using Windows, so the Ctrl-C, Ctrl-V for copy/paste works. I can copy text from really just about anywhere... Github, blog, another text editor, etc... and paste it into either the File Editor or the ESPHome editor within Home Assistant. You can also try a 'right-click' of the mouse to bring up the cut, copy, paste options... but for some reason, I do see cut/copy but not paste in the pop up menu in the ESPHome editor. Out of curiosity, if you copy something from say a Github example, can you paste that into a local text file (not in Home Assistant), even using something like Notepad? If you are asking something else please let me know, but I am able to copy text from outside files and paste those into either the File editor for YAML files or into the ESPHome node editor.
One thought... are you logged into Home Assistant as the owner or an administrator? And do you have 'advanced' mode enabled? If either of those are not the case, it might prevent you from editing any files... and therefore, paste would not be available. Click on your icon in the lower left corner. In the profile, at the top see if the account is an owner or administrator. Then scroll down and see if the 'advanced' mode is toggled on. If you are unsure of accounts, go to the System -> People. This could likely be your issue.
@@ResinChemTech Hello and thanks for the help. Much appreciate it, so yes, advanced is selected and the keyboard shortcuts do work so everyone that’s copying pasting must be using the keyboard shortcuts, which I never have before. Mine is the same as yours. The copy is there, but not the paste with the mouse. Also, just curious how you got the side-by-side view and not sure some are using the built-in file editor and some are using the VS code not sure which is the best and some have mentioned that you need to be using Samba Share to edit the config. So I wasn’t sure what the issues were. Your videos are good. You’re very good at explaining things you must be a teacher.
Could you dive into the code for setting up an MQTT client? I am new to all this and trying to learn. Basically I am trying to control an LED Matrix running on a raspberry pi and want to control what it displays like you do on your led matrix.
Well, my matrix actually has dual controllers. If you are talking about the controller that shows patterns, etc. then that is the WLED controller and it doesn't use MQTT... it integrates into Home Assistant via an API and isn't even my code. If you are talking about the controller that shows a clock, scoreboard, countdown timer, etc. then that is custom Arduino code that I developed. You can see that complete code in the related Github repo: github.com/Resinchem/LED-Matrix-Clock_WS2812b
If you look in the matrix_clock_main.ino file in the /arduino folder, you can see how I implemented MQTT. The sections that are probably of the most interest are the MQTT setup under the Setup() function (starting at line 371) and the callbacks (starting at line 581), although there are a lot of calls to publish MQTT values scattered throughout the rest of the code.
If you are using a Raspberry Pi, then my guess is that you are also using Python. If so, then as I show in the video, the logic is basically the same between Arduino code and Python code... the syntax is just a bit different. So, you will have to take what I show in the Arduino code and modify it for Python. But the logic and processes are still the same. If you are new to the code and MQTT, this is probably still a bit overwhelming. It was to me at first as well. I recommend just trying to start out very simple. First, see if you can individually control a few pixels on your matrix. Once you have that, see if you can create a single MQTT command to call the procedure that turns on those few pixels. Then you just have to build out from there. It took me months to develop the code for my matrix, but I learned a ton in the process. So, hang in there, study online documentation and take it in small steps and you can get there!
Good luck... if you run into any particular questions, let me know and I'll try to address them as best as I can (although there are limitations as to what I can do here in the TH-cam comments).
@@ResinChemTech thank you so much for the in depth reply! I am new and yes, it is overwhelming lol.
Thanks! nice explanation
I appreciate it... and hope it made some sense for those looking to use and better understand MQTT.
@@ResinChemTech Helped me- MQTT is not really the problem when you are starting it is understanding how to write JSON / Python/ etc. at least for me!
Cheers That was helpfull. Gone dig into your channel some more. :-)
Thanks! Glad you found it helpful. Just let me know if you run into any questions for this or any of my other videos.
sorry, I'm a beginner at MQTT. I'm trying to understand this video, from a beginner's point of view, please help correct me, here are the steps that must be done: first step, install the code on Arduino, second step, setting it on the home assistant. My question, in my video look at the file location at /config/packages/basement_matrix.yaml. I can't find the /config/packages folder in my home assistant, did you create this folder, not the default one in the home assistant? Thank You
I have a /packages folder because I've opted to use Home Assistant packages for my configuration. This is not the default, which is to place everything within one big configuration.yaml file. The other alternative (which is what I used prior to packages) is to use a 'split' configuration, where you put things like lights, switch, sensors, etc. (or all your MQTT entities) in their own YAML files and then 'include:' the individual files in the main configuration.yaml file.
I cover packages in another video: th-cam.com/video/VhHzBXYkVhw/w-d-xo.html (If you just want to know about packages, you can skip forward to about the 10 minute mark in this video).
In addition, I'll have another video coming out this Saturday that focuses solely on the Home Assistant side of setting up MQTT entities. This will significantly expand on the short segment that talked about the Home Assistant side in this video. I think it should provide much more information and details on the Home Assistant side of setting up MQTT devices and entities.
SOLID INFO thanks
You are welcome. And thanks for the comment!
has that cmnd/stat the same behaviour in tasmota, which i recall having both with same names?
Unless it's changed (I haven't installed a new Tasmota device in a while), Tasmota uses tele/ instead of stat/ for its "telemetry" values. You can use the same command topic for both (commands and states), but it can be tricky and certain situations can still lead to occasional ghosting. As I mentioned, there are other ways to do it, but using separate stat/ and cmnd/ topics is just the way I opt to handle retained vs. non-retained values. It leads to having to write a bit more code up front, but it's a bit easier (at least for me) to keep values that I want retained vs. ones that don't separate.
@@ResinChemTech thanks!
Fantastic.Thank you.
You are most welcome and I hope you found it helpful. And thanks for taking the time to leave your kind comment. I greatly appreciate it.
Thank you for doing this. Please consider using dark mode for HA screens... I couldn't read the bright screens...🤣
Noted. I do try to assure I'm using dark mode for most code windows (like I did for the Arduino/Python code), but the Home Assistant code editor is that blinding white. I'll be sure to try to use another method moving forward.
@@ResinChemTech Maybe code-editor/VS Code... Thanks, I have bad vision but everybody loves dark mode...👍
I'm taking a drink everytime you say MQTT.
I don't think I'll be able to finish the video. 😵💫
Thanks for this! Old hand at python/MQTT for my home alarms, but new to HA. Any chance you could post your yaml file? I would love to roll my whole system over to HA.
You are welcome! You can find the full YAML code for the dashboard, MQTT entities and automations for the project shown (matrix clock) in my Github repo: github.com/Resinchem/LED-Matrix-Clock_WS2812b
If you want to see more examples of Home Assistant/MQTT YAML, then you might also take a look at my repo on the DIY Arylic amp project, as it also makes heavy use of MQTT... and also has all the Home Assistant YAML: github.com/Resinchem/Arylic-Amp-MQTT
Sir I would also like to request u to make a video on dash board you are using for your mqtt clock
There is really nothing special about the dashboard. It does use the custom button card, which I covered in a recent video. And you can find all the code, including the YAML code for the dashboard, in the Github repo for this project: github.com/Resinchem/LED-Matrix-Clock_WS2812b
The dashboard code is published as lovelace.yaml in the /homeassistant folder of the repo. And there is some additional information in the wiki of that same Github repo.
thank you! This clarified so much!
You are most welcome. I'm glad you found it helpful. I know that MQTT was a bit confusing for me at first, but it is extremely handy for your own DIY projects once you get the hang of it.
I just discovered your channel. I'm trying to set up a USR-N540 4 port serial device server that uses MQTT. I managed to get the Mosquitto broker installed in HA but I'm going to need to find the right video on your channel to help with how to connect to such a device. I'm pretty clueless about this stuff but based on this video I suspect you have a good one on how to connect a device once you have the broker installed. Please direct me to your best video on this topic. Thanks, mfk.
I think this video is probably what you are looking for: th-cam.com/video/WYyPMHqMV-w/w-d-xo.html
As long as you know the MQTT topics and types of payloads used by the device, this video shows you how to manually integrated and control that device from Home Assistant. Hope this helps you out!
Boy, that went over my head, I'm trying to get a ESP32 with a temp sensor to display in node red, esp32 is publishing, mqtt gives me following error:Client esp32-5834feab6224 disconnected, not authorised, any suggestion how to get it to authorize?
Well, it could be a number of things causing the disconnect. First, you didn't mention the broker you are using (e.g Home Assistant MQTT add-on, standalone, etc.), but whichever one it is, assure that you are updated to the latest version.
Next, each client connecting to the broker must have a unique client ID. If multiple devices share the same ID, when one connects, MQTT will disconnect the other. I'm not a NodeRED user, so I'm not sure about the process that is used to connect to, and communicate with, MQTT. So I'm not much help on that front.
But the 'not authorized' message generally means that you have the IP address, password or user name configured incorrectly when attempting to connect via the device. Be sure that you are using the local IP address of the broker and that you have a valid username/password for the broker. See this, which may help: community.home-assistant.io/t/mqtt-refusing-connection/437138
"MQTT add-on goes down with HA" - you mean when the whole host is rebooted? However, most of the time we only restart HA Core (the python app) which AFAIK doesn't affect the container with MQTT.
I could be incorrect, but when you do a full HA restart, it also restarts all the add-ons... including MQTT. I do know that when I restart HA, I can watch the status at the bottom and I see it specifically say "Starting MQTT, not everything will be available until it is finished...."
So, obviously MQTT goes down when the whole machine and/or host goes down, but I believe the MQTT add-on, and other add-ons, are also restarted on a full system restart as well, meaning it is down.. at least briefly, which can lead to the entity errors I showed. Now, this is in the full Home Assistant install (used to be called HassOS) and not a containerized version. If you have some other type of of install, then it may well be that you can restart the core without impacting MQTT. But if using the add-on with full Home Assistant, I do believe that MQTT restarts as part of the process.
@@ResinChemTech I certainly don't know everything but what I think: "MQTT is starting..." you mentioned means that MQTT integration is starting, not the broker (which is in its dedicated container and is running uninterrupted throughout HA restart). "Entity errors" you see also because of the MQTT integration starting - so you should see the errors also if the MQTT broker runs on an independent machine. Finally, I think that HA restart itself doesn't restart any of the add-ons. Anyone can confirm or deny our polemic?
@@ResinChemTech I forgot one more thing: I think that HassOS is also containerized.
No disagreement here! I am far from an expert on the inner workings of HA! You may well be correct (and probably are!). In my particular case, I do have 3 times a week that I stop the VM to do a full backup... so for me in that case, MQTT goes down then because the entire VM is stopped and restarted. And now that I think about it, you are correct about the integration vs. the add-on... and I would see those same errors even if I had an external broker with non-retained values.
Sorry for my own confusion... and lack of knowledge about the inner workings of HA! That's why I love the feedback... I often learn new things myself!
@@ResinChemTech Absolutely no problem with that. I correct your mistakes, you correct mine - beneficial for both of us 😉
Thanks! MQTT is a terribly designed protocol, with often buggy implementations.... But it's usually the only thing we've got, so I guess we'd better learn about it!
In your explanation/example of the retain flag, IF during the time the device is disconnected, there was a CMND to the broker with ON, the STAT changes to ON, but then if the device reconnects, you now have an incongruence of state and reality since there is no CMND and the device doesn't subscribe to STAT. The light will remain off but the broker retains the state of ON.
Yeah, the right solution is to get devices that subscribe; but how do you handle this situation when you don't control what the device supports?
I'm not sure I follow. If the device is disconnected when the CMND is published with an "on" command, the STAT won't be updated... since as in my example, the STAT is _only_ published by the end device... which is disconnected. When the device reconnects, it will publish its current state... which in your example is "off". In this particular example, the only thing that happens is that the CMND that was issued is simply ignored (and dropped, since it has no retain flag and the device is disconnected). If the state of the light was "off" before the disconnection... and it isn't programmed to turn on when powered up or booted... then when it reconnects, the published state will still be off. At no point in this particular example does the "state" topic become out of sync with the actual state of the light.... as it is only the light itself that is updating the retained state topic.
I've seen this exact situation in Home Assistant where I have a switch tied to a custom-developed MQTT device. When I toggle the switch to 'on', the switch in Home Assistant briefly turns on... then turns right back off again on its own. This is because it toggles on briefly when the command is sent, but because the device is offline, the retained state is still off. Home Assistant quickly updates the state of the switch right back to off from the last retained STAT topic (which was off at disconnection). This generally occurs within 1-2 seconds, and is one way I know when a device is offline (or at least dropped the MQTT connection).
For this particular video, it was about DIY devices that you do completely control. In my case, I am the one that wrote the firmware/code that runs on the end devices. If it is a retail device, or firmware that you didn't develop, then you are correct. You are somewhat at the mercy of what the device/firmware developer provides and if it both sends and receives on the same topic, then you can still have issues. Some devices/firmware are (in my opinion) better than others. I like Tasmota if I want to use MQTT with prebuilt firmware because it does use different topics for commands vs. states, so I can follow the same process to assure I don't get ghosting or states that are out of sync.
Sorry if I am misunderstanding what you are saying... but in my experience with my own MQTT devices, using the techniques I described has never resulted in an 'out-of-sync' situation between what Home Assistant is showing for a state and the actual state of the device itself.
@@ResinChemTechYou understood and answered wonderfully. I missed a few points in the video that you clarified above. My fault.
The thing I am trying to wrap my head around is how to support in the complete system, the ability to have an automated "prime" that will set state that even is one part is down, when the part comes back up, it will return to state it "should be in".
I have run into some of things you mentioned. I think my solution is to follow your "scheme" but have this "prime" subscribe to the state (from the broker) and make some "decisions" based on said state. The wrinkle is managing temporary state caused by human interaction compared to the automated state in "this failure state of a device disappearing and when it reappears it may default to wrong state as to what the prime wants.
(in other words, I have a more complex situation to solve).
Example is outside driveway lights, tune on at sundown, turn off an hour later. But if a user turns them off stay off. If the car proximity triggers turn them on and return to the timed turn off time OR if after a short timer to turn off. Several similar scenarios of automated lights with possible user override but also trying to maintain robustness when a device gets disconnected (from reboot caused by poor code and a watchdog, bad wifi, battery power lost due to poor solar, etc).
Hey, no problem! Sometimes I tend to ramble a bit in my videos and don't always make my points as clearly as I should.
For the more complex situations, I will often make use of things like input_booleans and/or timers within Home Assistant. My interior garage lights (controlled by a switch relay that uses MQTT) is sort of a more complex situation similar to what you described. The lights come on via door switches then turn off after a set time. But the wall switch can also be flipped manually by a user, regardless of door state. For this, I use an input_boolean helper (along with timers) that sort of acts like that "prime" that you mention. I can toggle or change the state of that input_boolean via various automations depending on what happens with the lights... and can return the light to a desired state based on this input_boolean. This may or may not work in your situation, but I use input_booleans a lot in my automations to help control devices and states that might also be altered by a user with a different physical device.
Don't know if any of that might be helpful to you or not!
ur awesome!
All my stuff use Esphome.
And ESPHome is great for many cases.... I use it for a lot of my own devices. But for more complex project, like the clock I show in this video, ESPHome is not an option. I also do not believe you can install ESPHome on devices like the Raspberry Pi... or other devices that might use Python or other non-Arduino based languages. I believe that he only supported platforms currently are ESP8266, ESP32 and RP2040.
ESPHome is a quick and easy way to put code on custom devices, especially those that just need to connect sensors, switches, etc. But for devices that need to do more advanced local processing with highly customized code... or if you want to develop in another language like Java, then MQTT offers an easy alternative to integrating your devices into Home Assistant.
I can't even figure out how to make Home Assistant recognize the existence of the MQTT device. I have the MQTT integration in but that's it, can't find a way to tell it how to find the device, give it an IP address or anything. The device is an RS232 controller NB144. There are three downloads for configuring the device but the one that looks most important is in Chinese. Don't think there is much hope😞
You might try watching the follow up video to this one, if you haven't already, that focuses specifically on how to set up the Home Assistant side for MQTT devices. th-cam.com/video/WYyPMHqMV-w/w-d-xo.html
This video was a bit more of an MQTT overview. The linked video above has much more info on the specifics of the Home Assistant side. I hope that will help a bit.
WOW---->Head exploded-->Ghosted
Where does one go to start at the very basics? I looked and did not see an interface in my Home Assistant for MQTT? Sounds to me like I am on the border line for your intended audience.
Well, Home Assistant by default does not have an MQTT integration. But it's extremely easy to add it. It is a standard add-on that you can find in the add-on store under Settings. This is the add-on, so it works both as your MQTT broker and as the integration for Home Assistant. Lewis over at @EverythingSmartHome has a really good video on installing and setting up the MQTT add-on in Home Assistant: th-cam.com/video/dqTn-Gk4Qeo/w-d-xo.html
Of course, you can create your own standalone MQTT broker on another device (like a Raspberry Pi, VM, old PC, etc.) and then install the MQTT integration via Home Assistant integrations. Both methods have pros and cons, but using the Home Assistant add-on broker is definitely the easiest way and doesn't require a different machine/VM, etc.
Sorry to point you at other sources. Unfortunately I sometimes have to make a call as to what to show and what to skip over (and assume that folks already know... or point them elsewhere) in some of my videos simply due to the length. But I do appreciate you watching. And don't hesitate to follow up with any additional questions and I'll do my best to help you out.
sound is awefully out of sync with lip movement in video
I just watched it back on both the computer and TV and I'm not seeing any sync issues between the audio and video on my end. It may be a temporary TH-cam thing.
@@ResinChemTech, yep all good for me
Thanks for the confirmation! It always worries me a bit when I get a report like this, but I generally check and watch back all videos after I upload them and I didn't see any issues then... or after watching again based on this comment. But it's good to know that others aren't seeing the same issue as the original comment. Thanks again for taking a moment to let me know it is OK on your end.