Many thanks, this has been enormously helpful. Our system was thrown together years ago using a Pi to count flashes on import&solar meters & using Home Assistant + Grafana to give us real time displays on tablets. This helps the better half to decide whether to cook on induction vs the wood stove depending on solar generation & helps predict when to run the washing machine etc. Discovering the total energy counts (which Tasmota also does on Sonoff POW's) and using your subtraction technique has just given us decent accuracy & totals for the first time, so watching your video slowly whilst implementing your ideas on our system is a complete gamechanger. Huge thanks for creating this video :-) Rob
Thanks - that was very interesting and informative. I never realized that with the better resolution of a smart meter that the power company can get a lot of info about peoples lifestyle .
thanks a lot. Managed to get everything working as you did, except "Total for period", which returns very small values for couple of days (value below 1). Have to use spread() for this, but cannot be tied to selected time range.
Excellen tutorial, almost there with the monitoring. Read the comments below on the request on the dashboard Device Monitoring, hope you get around soon to help us get started.
Just remember that the Shelly EM actually sends 8 values plus the relay state, in spite of what their documentation says. If you collate 7 or 14 messages your data will end up mangled. It's possible they were only sending 7 at the time this video was made. However when I was collating only 7 items at a time on a recently installed Shelly EM I ended up with odd looking data in the database. This problem went away when I switched to collating 8 items, because the Shelly is also sending a pf value.
Interesting. May well be related to a Shelly update since I made this vid. As a younger person I tended to insist on installing updates immediately all the time. As a less young person (!) I resist all hardware updates unless I actively know what they do :)
Now I need a Shelly!! Any chance of a video or blog on the status dashboard? Look like just what I need to keep track of things. No more openHAB for you?
You mean my little Node-RED device monitor dashboard? I’ve been meaning to do that walkthrough next, but it does mean I need to tidy up and rewrite some code first hence putting it off! Will do that next. Yes I ditched openHAB in favour of Node-RED. I am still trying to find a reason to go back to openHAB or even migrate to Home Assistant, but I just cannot find one. I quickly learnt that Node-RED delivers everything I need, with the possible exception of pre built dashboard widgets etc. But I have started to build my own, so it’s no big deal!
@@MatSmithLondon yeah, the one around 3 mins in. I like the way it auto populates, sounds like a better solution than adding each device (manually I guess) in openHAB which I've been putting off doing. I've never really used Node-RED but I think a lot of people use a bit of both.
Mat thanks for you dashboard explanation. I'll found your video because I was looking for a special graph. Did you solved the graph of today, ex. the amount starting on a special hour. If it's of today it will be 4th of June on 00:00. When that is possible maybe starting at sunrise (up to sunset, and vise versa) is also possible so that I can measure the amount of energy needed from the grid without assistance of my PV installation and find out what's the best home battery capacity (when battery prices drops) and what's the peak capacity that's needed to level out the grid usage (from 1 July 22 we will be paying for the max quart hour simultaneously amount of energy in that month)
I was looking to do the same for Shelly 3m, and you just saved me a lot of time, thanks Man. I would be glad to send you 3M version to share it via your blog if you like. PS. kindly delete the space in front of "/"[3] in your Prep message function msg.key = msg.topic.split(" /")[3] + "_" + msg.topic.split("/")[4]
Thanks, my own flow stopped working the day I wrote that blog and it took ages to work out the problem was that accidental space that you noticed. Didn't realise the problem made it into the blog! I'll sort that out now, thanks for noticing and letting me know. Definitely share the 3M version and I will add it to the blog if that's okay with you.
Hi mat, I’m looking for a power data logger to determine maximum demand from the power grid over say a 48 hour period. Would the Shelly em be able provide this information? I’m just looking for the highest spike on a graph over 48 hour to calculate the max current drawn. Thanks Dan
Hi! Is there anything special needed to get MQTT working? I tried to get mine set up to talk to Mosquitto but I didn't get anywhere. I did test my Mosquitto installation and that seemed to be working. If it is working, how often does the Shelly send the data? I have a real-time use case so I may just continue with second-by-second polling.
Hello! As long as you have a working Mosquitto server (which you can test reasonably easily) and you've pointed the Shelly to that server (by IP or hostname), it should receive messages. Best thing is to hook up an MQTT listener. Depends how you've installed it (direclty on Windows / Linux, or as part of a package like Node-RED or HomeAssistant?) but either way you can use MQTT explorer or Node-RED or something like that. If you don't get messages from Shelly, make sure you are seeing messages at all, by sending a manual MQTT message e.g. from command line or within Node-RED or whatever. The Shelly is like similar devices e.g. Tasmota, in that it sends periodically (every 30 seconds if I recall correctly). I'm 99% sure this can be changed in the settings but can't recall and don't have it in front of me right now.
@@MatSmithLondon Thanks! That's a shame; I've been through all that and as far as I can tell, the Mosquitto side was working fine. It's installed on an Ubuntu server. But I'd need it to send a message every second anyway for it to be of use to me (I'm doing S2V/V2H controlling of an EVSE charger/discharger and it works pretty well with polling). It's always possible that I gaffed by putting in the wrong IP address; wouldn't be like me to make a mistake like that, but it's possible. Maybe I'll have another play. Either way, thanks for your informative video; I have influxdb running and it's helping me to monitor and fine-tune my application.
Thanks Mat for share this dashboard, Is working well after I changue some configuratios in influxdb and Grafana. What a pity that it does not include the cost depending on the hourly rate, I have different cost rates by houre.... Thanks and best regards
We used to have variable costs (cheaper at night) here in the UK but we don’t have that any more. My suggestion would be to process this calculation in Node RED and then export it with the sensor data. There may be a way to avoid this, but honestly I don’t know how in Grafana to change a number based on the time of day for the given data. It would be reasonably simple to do this in Node RED
When you say you'd like to have power logging in kW, what exactly do you want that to look like? The Shelly app can show you power usage and I believe Shelly cloud has graphs and historical data. If you want your data in a spreadsheet so you can do other things with it then you can listen to the live data from Shelly using MQTT or one of the other provided mechanisms. This video is about a different subject, namely visualising data that has been exported into a database, using Grafana..
Thanks sounds like i need to look into using MQTT but still im lost already! The shelly cloud shows live kW value but no way to log it which is what im after. I asked them and they said its not possible but i can request it. @@MatSmithLondon
Hello This is a great video thanks for the example... however I just can't get the Blue graph to work. I'm running Grafana 2.0 and my query definition is very different, are you using some kind of plugin which makes this easier ?
As a non-coder, I am overwhelmed. All i wanted is to monitor the energy and power difference respectivly between clamp 2 and clamp 1 in HA. That's my solar production. So that maybe I can dump load my boiler smartly using its contactor to enable other bigger contactor. Nope, you need databases, payloads, whatever.
With respect, this video is about monitoring *historic* energy usage, e.g. for spotting trends and seeing how much you used. Sounds like you are trying to do something completely different, i.e. monitor power in realtime and make decisions. Then of course, you don't need a database for this, you can just do it in Home Assistant. If you are using Shelly EM, this is very easy to do by getting the values over MQTT.
Quite helpful! One thing I missed is how to handle data from standard impulse output of various meters. Most of energy (but also gas or water) meters just send pulse every x unit (1 pulse per 1 W being the most popular), what's the best way to store such data? When using Tasmota, one can send its internal counter every x seconds (10s being the lowest delay). Just pass it to influx or rather convert it to something more meaningful in Node-RED? Also, your dashboard tab with info about Tasmota devices seems awesome. Willing to share a flow? Thanks! Great stuff!
Thanks. Certainly willing to share the tasmota dashboard. Will do at some point soon. Re the pulsed meter output: actually most meters measure kWh not W. So each kWh is a measure of how much power is used for one hour, hence that’s what the counter shows. My way here involves using a current clamp which measures W (not kWh) and it continuously measures and so it’s possible to determine kWh (the Shelly does it automatically, as it measures the time itself). It is arguably even better to measure the pulse because this will be more accurate. (Actually not more accurate, but more in-line with what the power company thinks you have used, so it relates exactly to what you are paying.) There is a TH-cam vid somewhere about measuring the meter’s numbers, using an ESP32 with camera. I like this idea so I have bought a few to test but haven’t set these up yet. The only thing about doing that is you cannot see how much certain circuits are using (eg upstairs sockets or lights), only the whole house.
@@MatSmithLondon you are right about 1 pulse every 1Wh, my mistske. Still not sure if just send that stright to influx or do some math in NR. As for checking certain circuits, you can add a separate meter per each section you would like to measure separately so no problem to have one global and a few for garage, upstairs, lights etc.
@@ziomalZparafii My advice: store an array of latest readings in Node RED, along with a timestamp for each one. Maybe you can store the last 10 readings using array.push. Each time a new reading arrives, you can compare the timestamp with the previous timestamp in seconds, in Node RED. You want to calculate W (power) from Wh (energy used over the period of time between pulses) so you take this difference in time and divide by 3600 seconds to give number of Watts. (That assumes the pulse is every 1Wh. If it's every 1kWh, then multiply final value by 1000). Instead of comparing the last two readings you could also average over 5 readings. Or you could average over a certain amount of time. Finally you could just do all of this within Grafana, then you only need to store each reading into Influx (nothing more!) But the benefit of doing this in Node RED is that you are monitoring the "current speed" as well as the "total distance", so you can use it to do other things like "notify when the washing machine is finished" or "send email when the power usage is very high". Also you can show a "instantaneous power" gauge, showing the current usage, in Dashboard. I think this approach is even better than the one I have shown you, because it actually correlates the number on the meter, which means you can *exactly* calculate the electricity bill, and check it is correct! I may do a video on this myself actually!
@@MatSmithLondon great, thanks! Yeah, I thought about "guessing" which devices are on based on current consumption. I've already written JS function (version 0.1 alpha ;) doing fuzzy logic that with input of consumption array (like [{device: "oven", power: 100}, {device: "washing machine", power: 150}, ...] ) returns ["oven"] so devices that summed up match total power consumption. We'll see how will it actually work. I'm installing meter this week.
Curious what is your influxDB retention setting for the values. I see the graphs have detailed precision displayed, yet I don't think it's worth storing this high detailed entries over longer periods. Still trying to get the scripts working to aggregate the data and downsample it in the same tables, perhaps you could have a video on that afterwards. Appreciate the way you presented it as a real project how someone would really use it and not like a starter sample.
Yes, retention is a subject I have not conquered. I posted about this on Stack Overflow (I am not the original post, just some comments): stackoverflow.com/questions/58555008/how-to-downsample-data-older-than-14-days-only-in-influxdb-and-keep-it-in-the-sa - basically you cannot resample an existing database / measurement. I find this really annoying. I actually have not managed to set up a CQ myself, so if you make progress please let me know.
@@MatSmithLondon Hi at work we stopped using CQ because they kill the influxdb aha. We managed to perform the same using telegraf starlak language and setting different retention policies in influx (one for the raw data and one for the downsampled data). IMHO you have more flexibility if telegraf handles the writing in your influxdb instead of trying to manipulate data directly in influxdb. Cheers!
Pretty new to influx and nodered. Using HomeAssistant OS so just using the GUI and all set up in HA itself (so on same box) I seem to be having trouble with creating the database. I have added a nodered dB to Influx, so should I be able to navigate via Web to it? Any help on that appreciated.
When I first installed InfluxDB I don't think it was possible to browse to the DB over a web interface, but the latest version (Influx2) has a web interface. I'm still running the old version without. Is it possible to get an SSH terminal from within the HA interface? If so you will need to see if the database is there by launching influx. Once you have managed that, there's a built-in help over the command line, e.g. just type help once in Influx. Hope that helps
@@MatSmithLondon thanks for the info but I don't think SSH is possible in this case. Its a shame there's not an easier way to establish if it exists. Appreciate help anyway.
Nicely presented and thank you for taking your time to show this to us. I have a question if you don't mind. You asked why would someone use the 50A clamp and said it would be used for small projects. However I see your dashboard gauge was half way through when it read 2.5kW, meaning even the maximum anticipated instant consumption you're expecting is about 5kW. My connection for the house is for 7.5kW so I went with just one 50A clamp for more precision and knowing that is going to be good for under 10kW. Why would the 100A be better, as I thought that lower ratings would have better accuracy?
For the voltage graph, I believe it's measured just from the L and N wires from the Shelly EM and that's why it will be the same value for both channels
Many thanks. Having read your comment I looked up the difference in precision between the two. Maybe you already understand the subject of split core CTs and accuracy ratings (I had no idea until just now!) So I don't think Shelly have published detailed properties of the current clamp sensor, but they use a general metric "precision = 1%" on the Shelly product pages for both 50A and 120A. I have found this online when researching: "CTs are available in a variety of accuracy classes ranging from 0.1% to 5% error. Typical CT's have an accuracy of 1% (referred to as Class 1.0). Accuracy will be expressed over a specific load range. In the case of a 1% rated CT, accuracy is expressed over a measurement range of 10 to 120% of the CT's nominal current rating. So a class 1.0 CT with a nominal rating of 100 amps will provide 1% accuracy from 10 to 120 amps of current. " and from the same page: "Most CTs start to become accurate at 5-10% of their rated capacity. At lower loads, some CTs will become highly inaccurate. Most CTs retain accuracy up to 120-130% of their rated capacity. Beyond the maximum rating, the CT will "saturate" and measurement accuracy will fall rapidly." -- this stuff is pretty interesting. Before reading about this, I wasn't sure if the accuracy figure applied to the measured current (in which case it wouldn't matter which one you used), or the rating of the CT, now I know it's about the rating of the CT. The reason I said it's better to get the 120A sensor is that it would be more flexible (e.g. if you moved house, or wanted to put it on a larger wire later...) but I agree now that 50A is definitely better for the reason of accuracy, as long as the load doesn't exceed the saturation current. Thanks!
Many thanks, this has been enormously helpful. Our system was thrown together years ago using a Pi to count flashes on import&solar meters & using Home Assistant + Grafana to give us real time displays on tablets. This helps the better half to decide whether to cook on induction vs the wood stove depending on solar generation & helps predict when to run the washing machine etc. Discovering the total energy counts (which Tasmota also does on Sonoff POW's) and using your subtraction technique has just given us decent accuracy & totals for the first time, so watching your video slowly whilst implementing your ideas on our system is a complete gamechanger. Huge thanks for creating this video :-) Rob
Thanks - that was very interesting and informative. I never realized that with the better resolution of a smart meter that the power company can get a lot of info about peoples lifestyle .
thanks a lot. Managed to get everything working as you did, except "Total for period", which returns very small values for couple of days (value below 1). Have to use spread() for this, but cannot be tied to selected time range.
Excellen tutorial, almost there with the monitoring. Read the comments below on the request on the dashboard Device Monitoring, hope you get around soon to help us get started.
Thanks Mat, just what I needed for my Shelly 3EM both for Node-Red and Grafana
Just remember that the Shelly EM actually sends 8 values plus the relay state, in spite of what their documentation says. If you collate 7 or 14 messages your data will end up mangled. It's possible they were only sending 7 at the time this video was made. However when I was collating only 7 items at a time on a recently installed Shelly EM I ended up with odd looking data in the database. This problem went away when I switched to collating 8 items, because the Shelly is also sending a pf value.
Interesting. May well be related to a Shelly update since I made this vid. As a younger person I tended to insist on installing updates immediately all the time. As a less young person (!) I resist all hardware updates unless I actively know what they do :)
Now I need a Shelly!! Any chance of a video or blog on the status dashboard? Look like just what I need to keep track of things. No more openHAB for you?
You mean my little Node-RED device monitor dashboard? I’ve been meaning to do that walkthrough next, but it does mean I need to tidy up and rewrite some code first hence putting it off! Will do that next. Yes I ditched openHAB in favour of Node-RED. I am still trying to find a reason to go back to openHAB or even migrate to Home Assistant, but I just cannot find one. I quickly learnt that Node-RED delivers everything I need, with the possible exception of pre built dashboard widgets etc. But I have started to build my own, so it’s no big deal!
@@MatSmithLondon yeah, the one around 3 mins in. I like the way it auto populates, sounds like a better solution than adding each device (manually I guess) in openHAB which I've been putting off doing. I've never really used Node-RED but I think a lot of people use a bit of both.
Mat thanks for you dashboard explanation. I'll found your video because I was looking for a special graph. Did you solved the graph of today, ex. the amount starting on a special hour. If it's of today it will be 4th of June on 00:00. When that is possible maybe starting at sunrise (up to sunset, and vise versa) is also possible so that I can measure the amount of energy needed from the grid without assistance of my PV installation and find out what's the best home battery capacity (when battery prices drops) and what's the peak capacity that's needed to level out the grid usage (from 1 July 22 we will be paying for the max quart hour simultaneously amount of energy in that month)
Hello Mat!
Wonderful piece of work!!!
May I ask you what hardware is supporting this scenario?
Thank you in advance!!!
Regards
Paolo
Awesome. How did you do the state green up indicators in node-red for your arduinos?
Sorry for the delay :) It’s Node Red dashboard LED. Can’t remember the exact addon name but something like that
Amazing videos please make videos on device monitoring to monitor all tasmota device. Or maybe share flows.
Great job, looks so nice and really helpfull. But one question, how can i change the Pounds to Euros? Thanks in advance
I was looking to do the same for Shelly 3m, and you just saved me a lot of time, thanks Man. I would be glad to send you 3M version to share it via your blog if you like.
PS.
kindly delete the space in front of "/"[3] in your Prep message function
msg.key = msg.topic.split(" /")[3] + "_" + msg.topic.split("/")[4]
Thanks, my own flow stopped working the day I wrote that blog and it took ages to work out the problem was that accidental space that you noticed. Didn't realise the problem made it into the blog! I'll sort that out now, thanks for noticing and letting me know. Definitely share the 3M version and I will add it to the blog if that's okay with you.
Hi mat, I’m looking for a power data logger to determine maximum demand from the power grid over say a 48 hour period. Would the Shelly em be able provide this information?
I’m just looking for the highest spike on a graph over 48 hour to calculate the max current drawn.
Thanks
Dan
Hi! Is there anything special needed to get MQTT working? I tried to get mine set up to talk to Mosquitto but I didn't get anywhere. I did test my Mosquitto installation and that seemed to be working.
If it is working, how often does the Shelly send the data? I have a real-time use case so I may just continue with second-by-second polling.
Hello! As long as you have a working Mosquitto server (which you can test reasonably easily) and you've pointed the Shelly to that server (by IP or hostname), it should receive messages. Best thing is to hook up an MQTT listener. Depends how you've installed it (direclty on Windows / Linux, or as part of a package like Node-RED or HomeAssistant?) but either way you can use MQTT explorer or Node-RED or something like that. If you don't get messages from Shelly, make sure you are seeing messages at all, by sending a manual MQTT message e.g. from command line or within Node-RED or whatever. The Shelly is like similar devices e.g. Tasmota, in that it sends periodically (every 30 seconds if I recall correctly). I'm 99% sure this can be changed in the settings but can't recall and don't have it in front of me right now.
@@MatSmithLondon Thanks! That's a shame; I've been through all that and as far as I can tell, the Mosquitto side was working fine. It's installed on an Ubuntu server. But I'd need it to send a message every second anyway for it to be of use to me (I'm doing S2V/V2H controlling of an EVSE charger/discharger and it works pretty well with polling).
It's always possible that I gaffed by putting in the wrong IP address; wouldn't be like me to make a mistake like that, but it's possible. Maybe I'll have another play. Either way, thanks for your informative video; I have influxdb running and it's helping me to monitor and fine-tune my application.
what a great video, exactly what i was looking for as i begin to monitor my energy usage thank you
Thanks Mat for share this dashboard,
Is working well after I changue some configuratios in influxdb and Grafana.
What a pity that it does not include the cost depending on the hourly rate, I have different cost rates by houre....
Thanks and best regards
We used to have variable costs (cheaper at night) here in the UK but we don’t have that any more. My suggestion would be to process this calculation in Node RED and then export it with the sensor data. There may be a way to avoid this, but honestly I don’t know how in Grafana to change a number based on the time of day for the given data. It would be reasonably simple to do this in Node RED
Awesome video and thanks for posting the json! Looks great!
Is there a simple way i can get this for my Shelly EM? All this is way over my head but i would like to have power logging in kW!
When you say you'd like to have power logging in kW, what exactly do you want that to look like? The Shelly app can show you power usage and I believe Shelly cloud has graphs and historical data. If you want your data in a spreadsheet so you can do other things with it then you can listen to the live data from Shelly using MQTT or one of the other provided mechanisms. This video is about a different subject, namely visualising data that has been exported into a database, using Grafana..
Thanks sounds like i need to look into using MQTT but still im lost already! The shelly cloud shows live kW value but no way to log it which is what im after. I asked them and they said its not possible but i can request it.
@@MatSmithLondon
Hello This is a great video thanks for the example... however I just can't get the Blue graph to work. I'm running Grafana 2.0 and my query definition is very different, are you using some kind of plugin which makes this easier ?
Very good. Mine now working. Any way of doing peak/off-peak?
What is software name and can I get ?
Do you know how to monitoring database activity ? like mongo db, clickhouse db, etc. I need information
Hi, yes in this video I cover monitoring a time series database. For other types of DB you may need to search separately?
As a non-coder, I am overwhelmed. All i wanted is to monitor the energy and power difference respectivly between clamp 2 and clamp 1 in HA. That's my solar production. So that maybe I can dump load my boiler smartly using its contactor to enable other bigger contactor. Nope, you need databases, payloads, whatever.
With respect, this video is about monitoring *historic* energy usage, e.g. for spotting trends and seeing how much you used. Sounds like you are trying to do something completely different, i.e. monitor power in realtime and make decisions. Then of course, you don't need a database for this, you can just do it in Home Assistant. If you are using Shelly EM, this is very easy to do by getting the values over MQTT.
great video, awesome energy dashboard
Quite helpful! One thing I missed is how to handle data from standard impulse output of various meters. Most of energy (but also gas or water) meters just send pulse every x unit (1 pulse per 1 W being the most popular), what's the best way to store such data? When using Tasmota, one can send its internal counter every x seconds (10s being the lowest delay). Just pass it to influx or rather convert it to something more meaningful in Node-RED?
Also, your dashboard tab with info about Tasmota devices seems awesome. Willing to share a flow? Thanks! Great stuff!
Thanks. Certainly willing to share the tasmota dashboard. Will do at some point soon. Re the pulsed meter output: actually most meters measure kWh not W. So each kWh is a measure of how much power is used for one hour, hence that’s what the counter shows. My way here involves using a current clamp which measures W (not kWh) and it continuously measures and so it’s possible to determine kWh (the Shelly does it automatically, as it measures the time itself). It is arguably even better to measure the pulse because this will be more accurate. (Actually not more accurate, but more in-line with what the power company thinks you have used, so it relates exactly to what you are paying.) There is a TH-cam vid somewhere about measuring the meter’s numbers, using an ESP32 with camera. I like this idea so I have bought a few to test but haven’t set these up yet. The only thing about doing that is you cannot see how much certain circuits are using (eg upstairs sockets or lights), only the whole house.
@@MatSmithLondon you are right about 1 pulse every 1Wh, my mistske. Still not sure if just send that stright to influx or do some math in NR.
As for checking certain circuits, you can add a separate meter per each section you would like to measure separately so no problem to have one global and a few for garage, upstairs, lights etc.
@@ziomalZparafii My advice: store an array of latest readings in Node RED, along with a timestamp for each one. Maybe you can store the last 10 readings using array.push. Each time a new reading arrives, you can compare the timestamp with the previous timestamp in seconds, in Node RED. You want to calculate W (power) from Wh (energy used over the period of time between pulses) so you take this difference in time and divide by 3600 seconds to give number of Watts. (That assumes the pulse is every 1Wh. If it's every 1kWh, then multiply final value by 1000). Instead of comparing the last two readings you could also average over 5 readings. Or you could average over a certain amount of time. Finally you could just do all of this within Grafana, then you only need to store each reading into Influx (nothing more!) But the benefit of doing this in Node RED is that you are monitoring the "current speed" as well as the "total distance", so you can use it to do other things like "notify when the washing machine is finished" or "send email when the power usage is very high". Also you can show a "instantaneous power" gauge, showing the current usage, in Dashboard. I think this approach is even better than the one I have shown you, because it actually correlates the number on the meter, which means you can *exactly* calculate the electricity bill, and check it is correct! I may do a video on this myself actually!
@@MatSmithLondon great, thanks! Yeah, I thought about "guessing" which devices are on based on current consumption. I've already written JS function (version 0.1 alpha ;) doing fuzzy logic that with input of consumption array (like [{device: "oven", power: 100}, {device: "washing machine", power: 150}, ...] ) returns ["oven"] so devices that summed up match total power consumption. We'll see how will it actually work. I'm installing meter this week.
Curious what is your influxDB retention setting for the values. I see the graphs have detailed precision displayed, yet I don't think it's worth storing this high detailed entries over longer periods.
Still trying to get the scripts working to aggregate the data and downsample it in the same tables, perhaps you could have a video on that afterwards.
Appreciate the way you presented it as a real project how someone would really use it and not like a starter sample.
Yes, retention is a subject I have not conquered. I posted about this on Stack Overflow (I am not the original post, just some comments): stackoverflow.com/questions/58555008/how-to-downsample-data-older-than-14-days-only-in-influxdb-and-keep-it-in-the-sa - basically you cannot resample an existing database / measurement. I find this really annoying. I actually have not managed to set up a CQ myself, so if you make progress please let me know.
@@MatSmithLondon Hi at work we stopped using CQ because they kill the influxdb aha. We managed to perform the same using telegraf starlak language and setting different retention policies in influx (one for the raw data and one for the downsampled data). IMHO you have more flexibility if telegraf handles the writing in your influxdb instead of trying to manipulate data directly in influxdb. Cheers!
Do you have a download / link to this setup? I’d like to use the same with a modbus meter
Check out hazymat.co.uk - I have posted the code on there!
Looks Impressive, what Device Monitoring Software did you use? or did you built it yourself?
Thanks! I built the device monitoring myself, using Node-RED which collects information and sends / receives messages via MQTT
Pretty new to influx and nodered. Using HomeAssistant OS so just using the GUI and all set up in HA itself (so on same box) I seem to be having trouble with creating the database. I have added a nodered dB to Influx, so should I be able to navigate via Web to it? Any help on that appreciated.
When I first installed InfluxDB I don't think it was possible to browse to the DB over a web interface, but the latest version (Influx2) has a web interface. I'm still running the old version without. Is it possible to get an SSH terminal from within the HA interface? If so you will need to see if the database is there by launching influx. Once you have managed that, there's a built-in help over the command line, e.g. just type help once in Influx. Hope that helps
@@MatSmithLondon thanks for the info but I don't think SSH is possible in this case. Its a shame there's not an easier way to establish if it exists. Appreciate help anyway.
Nicely presented and thank you for taking your time to show this to us. I have a question if you don't mind. You asked why would someone use the 50A clamp and said it would be used for small projects. However I see your dashboard gauge was half way through when it read 2.5kW, meaning even the maximum anticipated instant consumption you're expecting is about 5kW. My connection for the house is for 7.5kW so I went with just one 50A clamp for more precision and knowing that is going to be good for under 10kW.
Why would the 100A be better, as I thought that lower ratings would have better accuracy?
For the voltage graph, I believe it's measured just from the L and N wires from the Shelly EM and that's why it will be the same value for both channels
Many thanks. Having read your comment I looked up the difference in precision between the two. Maybe you already understand the subject of split core CTs and accuracy ratings (I had no idea until just now!) So I don't think Shelly have published detailed properties of the current clamp sensor, but they use a general metric "precision = 1%" on the Shelly product pages for both 50A and 120A. I have found this online when researching: "CTs are available in a variety of accuracy classes ranging from 0.1% to 5% error. Typical CT's have an accuracy of 1% (referred to as Class 1.0). Accuracy will be expressed over a specific load range. In the case of a 1% rated CT, accuracy is expressed over a measurement range of 10 to 120% of the CT's nominal current rating. So a class 1.0 CT with a nominal rating of 100 amps will provide 1% accuracy from 10 to 120 amps of current.
" and from the same page: "Most CTs start to become accurate at 5-10% of their rated capacity. At lower loads, some CTs will become highly inaccurate.
Most CTs retain accuracy up to 120-130% of their rated capacity. Beyond the maximum rating, the CT will "saturate" and measurement accuracy will fall rapidly." -- this stuff is pretty interesting. Before reading about this, I wasn't sure if the accuracy figure applied to the measured current (in which case it wouldn't matter which one you used), or the rating of the CT, now I know it's about the rating of the CT. The reason I said it's better to get the 120A sensor is that it would be more flexible (e.g. if you moved house, or wanted to put it on a larger wire later...) but I agree now that 50A is definitely better for the reason of accuracy, as long as the load doesn't exceed the saturation current. Thanks!
Indeed I found that interesting too and did not know this before seeing the DIY versions prior to the Shelly EM release.
Thanks for the video =)
Excelent!!! Thanks!
Nice 👍
Thanks!