only 1:30 in but I had the same thought about using chatgpt to truly automate home services before the gpt4 announcement of image processing was a thing. I was like, "why not just feed a frame from a webcam every 5-10 seconds through an image-to-text tool, then feed that text into chatGPT?" so I tried exactly that. I positioned a webcam up in my room corner so it could see outside through the window, it could see my bed, my reading chair, and my lamp. From there it was a matter of finding a good, free, GUI-based img2text tool, but after enough fiddling, I got a few to work reasonably well with chatgpt by manually copying text back and forth using a screen reader script. Ended up sitting in my chair with a book, chatGPT asked siri to turn on the bedroom light; I opened the curtains right next to the reading chair, chatgpt realized it was daytime outside and plenty of light was coming in through the window, and asked siri to turn the lights off. I didn't have to explicitly set rules to tell it to do this, it just... worked it out on its own, that it would make sense, as a home automation tool, to turn on and off lights based on ambient lighting, time of day, recognized activities, etc; I always dreamed of having something like that but dreaded the idea of trying to train a locally-run 'AI' to learn my activities and patterns. And now with GPT4's image processing capabilities I'm very excited to someday try and cut out the middleman/middlemen so it's all directly automated... Anyway, on with the video!
@@atomic14 Yeah! Also, as a relatively infant programmer (dabbled in toy robots and such, definitely would not call myself a 'developer' by any means) I'd consider myself to be somewhat allergic to APIs, so I appreciate the demonstration of how something basic like this can be done, and also using chatGPT to help is such a great tool! not guaranteed to be perfect but definitely a shot in the right direction.
External dependencies - never a good thing. Wife: Why can't we turn on the lights? Husband: Sorry, our internet is down - nothing I can do about that. Wife: I told you replacing the motion sensing switches with this AI stuff was a bad idea, but do you ever listen?
Just the tip of the iceberg. Everyone could have their own calendar and cell phone plug in. Then appointments and phone calls become history. Imagine asking chatGPT: Call my lawyer about my taxes and connect me when he is on the phone. No more listening to music on the phone Or asking Reschedule my dentists appointment till next week but let me know if there are any openings for tomorrow ChatGPT will become our next subscription service
Absolutely incredible! Now i just need to feed it camera feed of my surrounding so it can adjust lights, temperature and all the other stuff automatically for me
There's a ton I've learnt off yourself mate, I'd like to know how you ended up here developing your skill set over the years?!. I've found hands on fun experimentation has been the best learning aid for myself, if it's not fun you forget it ;)
It’s a good question - I only really got back into doing my own projects a few years back. One of the reasons I started the TH-cam channel was to force myself to document things more so that I wouldn’t forget! Not sure how well it’s working - every time I learn something new, it feels like something old gets pushed out…
I had an interesting conversation with my son last week about the future of AI within home automation. In my view simply detecting and reacting isn't enough, humans are too random for this to work. The AI assistant needs to either ask what we want, or learn our habits, but even then the decision process needs that two way negotiation of 'what do you want and how can I provide it for you' to cater for our strange human whims! For example, when you come into your house and AI should start by querying your plans for the evening and using that to tailor it's actions based on what it knows about you, but even then - it should be recommending actions not simply performing rules based on triggers... "Will you be watching TV later? I noticed there's that program you like at 8pm do you want it recorded or a reminder to watch it?" That said, this video was great (of course!) and it's going to be a fun evolution of the platform over the next few years!
I agree - there’s a weird symbiotic relationship that needs to be formed. I’m starting to think of it very much like how your see butlers or manservants in old films. There’s a predictive catering to needs in advance - but also a slight surrender of autonomy. It’s going to be very interesting.
"my" chatgpt only made "poor ascii arts" not good at drawing .. so where does your diagram came from ? a plugin ? thanks edit i have my reponse at 6:03 ok you must be in the dev list to be able to use plugin with chatgpt !
Yes, used a plugin. If you don’t have the plugin you can tell ChatGPT to generate a “mermaid” diagram - you can copy and paste that into an online viewer.
@@atomic14 tanks for your help i was a litle bit disapointed when i read your comment who talking about "mermaid" diagram translating the expression "mermaid diagram" in my mother tongue do not match ... but after taking time to browse i found this th-cam.com/video/C3uG583qbT0/w-d-xo.html and now the concept is clear ... but i was not be able to name this expression in my mother tongue so i gonna kept the English expression until. Thanks a lot for your help Atomic14 !
interesting but1 there is always a but. but whats the point, i ask? HA on the pie does all i need to just with timers. and sun rise and sun set offsets. don't need to have to talk to it. Until it can wash the bed linen, make the bed, select the meats to cook, then plate up i'll stick with the Mark1 Wife.
"I'm sorry Dave, I can't let you do that."
only 1:30 in but I had the same thought about using chatgpt to truly automate home services before the gpt4 announcement of image processing was a thing. I was like, "why not just feed a frame from a webcam every 5-10 seconds through an image-to-text tool, then feed that text into chatGPT?" so I tried exactly that. I positioned a webcam up in my room corner so it could see outside through the window, it could see my bed, my reading chair, and my lamp. From there it was a matter of finding a good, free, GUI-based img2text tool, but after enough fiddling, I got a few to work reasonably well with chatgpt by manually copying text back and forth using a screen reader script.
Ended up sitting in my chair with a book, chatGPT asked siri to turn on the bedroom light; I opened the curtains right next to the reading chair, chatgpt realized it was daytime outside and plenty of light was coming in through the window, and asked siri to turn the lights off.
I didn't have to explicitly set rules to tell it to do this, it just... worked it out on its own, that it would make sense, as a home automation tool, to turn on and off lights based on ambient lighting, time of day, recognized activities, etc; I always dreamed of having something like that but dreaded the idea of trying to train a locally-run 'AI' to learn my activities and patterns. And now with GPT4's image processing capabilities I'm very excited to someday try and cut out the middleman/middlemen so it's all directly automated...
Anyway, on with the video!
That's it exactly it! The always on assistant constantly monitoring the environment around you and making it suit what you are trying to achieve.
@@atomic14 Yeah! Also, as a relatively infant programmer (dabbled in toy robots and such, definitely would not call myself a 'developer' by any means) I'd consider myself to be somewhat allergic to APIs, so I appreciate the demonstration of how something basic like this can be done, and also using chatGPT to help is such a great tool! not guaranteed to be perfect but definitely a shot in the right direction.
Brilliant
External dependencies - never a good thing.
Wife: Why can't we turn on the lights?
Husband: Sorry, our internet is down - nothing I can do about that.
Wife: I told you replacing the motion sensing switches with this AI stuff was a bad idea, but do you ever listen?
Absolute gold
Awesome work! How would you integrate the snoring sensor? Is there some way to have your chatgpt session stay awake and "wait" for specific actions?
Just the tip of the iceberg. Everyone could have their own calendar and cell phone plug in. Then appointments and phone calls become history. Imagine asking chatGPT:
Call my lawyer about my taxes and connect me when he is on the phone.
No more listening to music on the phone
Or asking
Reschedule my dentists appointment till next week but let me know if there are any openings for tomorrow
ChatGPT will become our next subscription service
Absolutely incredible! Now i just need to feed it camera feed of my surrounding so it can adjust lights, temperature and all the other stuff automatically for me
Yep, imagine something that can take in all these inputs and synthesise a sensible response.
@@atomic14 reminds me of that black mirror episode where your cloned consciousness becomes the brains of your smarthome
"ChatGPT: let's play wargames"... I cannot help myself, but this comes to my mind again and again.
Very tempting to add a plugin to play tic tac toe - actually, that feels like a little project for tonight!
@@atomic14 I guess we will learn about your tonights jouney ...
So what is it doing that Hubitat or Home Assistant can’t do right now? I don’t want AI deciding to lock me out of my house:-)
There's a ton I've learnt off yourself mate, I'd like to know how you ended up here developing your skill set over the years?!. I've found hands on fun experimentation has been the best learning aid for myself, if it's not fun you forget it ;)
It’s a good question - I only really got back into doing my own projects a few years back. One of the reasons I started the TH-cam channel was to force myself to document things more so that I wouldn’t forget! Not sure how well it’s working - every time I learn something new, it feels like something old gets pushed out…
That's so fucking brilliant!
Great! Could you try to use ESP32 I2S mic to send voice commands to openia API to control the outputs of the ESP32? Thx in advance.
Definitely doable - would just be very slow at the moment.
@@atomic14 even for short 1 word commands? It is a project that I want to sponsor 🙂
I had an interesting conversation with my son last week about the future of AI within home automation.
In my view simply detecting and reacting isn't enough, humans are too random for this to work. The AI assistant needs to either ask what we want, or learn our habits, but even then the decision process needs that two way negotiation of 'what do you want and how can I provide it for you' to cater for our strange human whims!
For example, when you come into your house and AI should start by querying your plans for the evening and using that to tailor it's actions based on what it knows about you, but even then - it should be recommending actions not simply performing rules based on triggers... "Will you be watching TV later? I noticed there's that program you like at 8pm do you want it recorded or a reminder to watch it?"
That said, this video was great (of course!) and it's going to be a fun evolution of the platform over the next few years!
I agree - there’s a weird symbiotic relationship that needs to be formed. I’m starting to think of it very much like how your see butlers or manservants in old films. There’s a predictive catering to needs in advance - but also a slight surrender of autonomy. It’s going to be very interesting.
"my" chatgpt only made "poor ascii arts" not good at drawing .. so where does your diagram came from ? a plugin ? thanks edit i have my reponse at 6:03 ok you must be in the dev list to be able to use plugin with chatgpt !
Yes, used a plugin. If you don’t have the plugin you can tell ChatGPT to generate a “mermaid” diagram - you can copy and paste that into an online viewer.
@@atomic14 tanks for your help i was a litle bit disapointed when i read your comment who talking about "mermaid" diagram translating the expression "mermaid diagram" in my mother tongue do not match ... but after taking time to browse i found this th-cam.com/video/C3uG583qbT0/w-d-xo.html and now the concept is clear ... but i was not be able to name this expression in my mother tongue so i gonna kept the English expression until.
Thanks a lot for your help Atomic14 !
When it starts asking about Sarah Conner, it's time to run like hell.
Or if you’re wearing leather, own a motorbike and have a nice pair of shades. Just hide!
Very Intresting
Thanks!
Salad??? I stayed in the Waldorf Astoria apparently they invented the whole salad thing,. 🤣
made from rabbit food. .
Salad for breakfast feels very very wrong.
@@atomic14 Salad at anytime feel wrong 🤣🤣 👍👍
interesting but1 there is always a but. but whats the point, i ask?
HA on the pie does all i need to just with timers. and sun rise and sun set offsets. don't need to have to talk to it.
Until it can wash the bed linen, make the bed, select the meats to cook, then plate up i'll stick with the Mark1 Wife.
Just think of it as a special friend.
@@atomic14 how long till it starts to nag, can't be too long before that gets added. 🙂