subscribed before. I am going to watch this one, then start with chapter 1. Then assess if I want to use GPT. If so... I'll be hooked to this channel I guess (alongside a couple of other great HA channels)
Glad to have you aboard! Just FYI, some have reported issue with the instructions. I suspect if you follow it verbatim you may run into some issues too since this video is based on an older version of OpenAI however the concept is the same so you can adapt OpenAI’s newer API. To this example.
Love you channel and the way you explain things. I know enough about coding to understand the concept of what you are talking about, but you make it so simple.
I appreciate the comment! When I talk to my family about this stuff I can see their souls leaving so I try hard to make it understandable and engaging here on YT 😁
Hmmm unfortunately, since the technology is experimental you’ll have to do a bit of coding to get things working. I feel that over time once the tech becomes more mainstream we will start seeing easier implementation of Jarvis-like assistance.
These videos are outstanding. My only request is that further series have the episode number in the title or description. I went around out of order before finding the beginning. More please!
Thx for your videos ! Instead of asking for actions and entities to ChatGPT, in the case of a command, you could use the conversation service of HA, and ask to ChatGPT to simplify the command (because HA can't handle complexe commands). eg: "simplify the next sentence, without coma, with the format: action, entity, area" and then pass it to HA conversation service. If it's possible to get the result of a service call, it could be interesting to first, call the ha conversation service, and if it can't handle the command, run the chatGPT automation. The aim is to reduce the response time. Now I'm using Node-red because of you ! Thx ;)
Hey thanks for the feedback! I originally was using the conversation api but it stopped working after I updated a few weeks back. Additionally I also tried capturing whether the command failed but it kept returning wrong information so I had to pivot which led me down this interesting road. But Perhaps they fixed the conversation api by now🤷🏾♂️
18:54 to answer your challenge, I would imagine tag lining a string that can be looked at, and if it detected will do a certain action. I could ask the chatbot "please turn on the lights" and it could say "Sure, turning on lights... command.homeassistant("light1","turn_on")" and there, if the string is found. it could detect that and interpret it as what to do.
i followed your scripts, and it works really well, the ease is just great! i only have 2 questions, it seems to send across all entities, including disabled ones, and with those the list is so long, that openai seems to not work. Any idea how to filter deeper on states of entities, before sending the array across, secondly, how to deal with still long requests to openai and make sure it accepts the input and give a response? thanks for a great masterclass
Hey thanks! You’ll definitely have to adjust the templates to return relevant entities for your environment. The best way to to dig deeper is to use Templates section in the Developer Tools. That is what I used to test out how to query for the entities I wanted.
Cool. Video(s). I recreated all of your stuff and made little extensions: - All entites have the room name in the first position of the "friendly name", so GPT knows what entity is in which room - For all entities I want to expose to the KI I changed the name to .ki_, so I can filter in the "get entities" node for the regular expression "\w+\.ki_w+" to keep the result array small. - I extended the action list to "Action List: turn_off, turn_on, toggle, close_cover, open_cover, set_cover_position". After the "handle optimized response" node for commands I added a switch to choose between if its a "light" command or a "cover" command via reg. exp. and added two separate "call service" nodes (one for "light" commands, one for "cover" commands). So Chat gpt is now able to control lights and covers. At the moment I expose the main light(groups) of every room, the covers (in the moment only of the office for testing) and the temperature and illuminance sensors of every room. I am experimenting now. Funny: When I tell GPT "It's warm in the office" he closes the shutters :-). When I tell GPT "Make cinema atmosphere in office" he turns on the office lights... Keep up the great work. Looking forward for the next steps. Would be cool if - GPT would respect the states of the entities more (It turns lights off although they are already off) - GPT would use the outcome of the older calls ("Should I turn of the office lights?" "yes" does not work) - If GPT could also dim lights and change color etc. etc. 🙂 Regards, Daniel
Very exciting stuff! I do have something that could help with some of your wish list items but I’m trying to polish it up so it’s more user friendly. I’ll keep everyone posted when it’s done
Little question, maybe we could include the room in the datas we give to chatGPT, to help it answering more easily (like "turn on the kitchen", with the room, it don’t need to look for lights which have kitchen in their name, but use the room… (thinking about it because not all my lights have the room’s name in their name) As I already said in previous comments, love your content !
Great tutorials, man. Too bad I'm too limited to implement this. I spent a whole day banging my head against the wall, trying to get the bot to respond on Telegram. All it would say was "on my honor," and I couldn't figure out why. Additionally, I wanted the bot to retrieve text from the family group, but it only responded to direct messages. I checked the chat IDs and everything, but I think I'll hold off on delving deeper into this AI thing until Homeassistant comes up with a plug-and-play solution. :/ You're a genius!
There is a Telegram bot integration which may help a bit. Group chats are a bit trickier to setup but its possible though! I am working with another view who has an integration that allows you to use Home Assistant Assist's chat to communicate to your node-red environment. This way you wont need Telegram and you can use the home assistant App.
a simple question, in video 2 at 20:18 you told us in the next video we going to learn how to do the things you told at this moment of video but in video 4 you forgot to explain how... so I'm stuck and unable to continue the tutorial can we please have this video part please cause I think people like me who learn with you together can't go further cause this part is missing 🥺🥺🥺
I try to balance giving enough information with storytelling to ensure that the videos have a wide appeal and encourage enthusiast like yourself to explore for yourself. I mentioned the automations are “too-shelf” but a better way to put it is that they are complicated… The automations were very hard and many would require help that I don’t have the resources to give so I opted for a more storytelling format to inspire everyone to try. HOWEVER… I am working on something that should make creating automations with GPT easier. I will also creat a few videos to walk viewers through how to use this new tool so be on the lookout for the video and hopefully it should help make creating these automations easier. 🤞🏾
@@technithusiast i was thinking yesterday I watched a vidéo in French from underscore Chanel, they talking about create assistants ( gpt ) and inside you can put everything you want him to know inside like the home assistant guides or node red library ( idk how can we have accès on every pages of the node red library and home assistant library of guides but in theory you can put every thing inside and in theory someone can ask the assistant inside home assistant or node red how I can create this and this and I want that work with this ( it’s an exemple ) for those who are like me and know nothing and don’t know how to code, and il theory if in right he can answer us or even maybe do it by himself if we give him the rights actually it’s just an idea but event for your home you can give him eventually every entry’s etc every thing you want him to know about the home and it going to be more accurate because of his data where he can find every instructions. And gpt vision can actually understand images so eventually I’m going to try il juste taking your automatisations screen and give it to him trying understand a little bit better, it’s just an idea but maybe that can help you to have some ideas so if it’s can help you it’s good to know
@@vovas.1061 it sounds like you’re referring to the Assistant Api. It is powerful but it’s not meant for controlling smart homes. I created a video about it here: - th-cam.com/users/shortsHyY6BRVibc4?feature=share - th-cam.com/users/shortsOO4nxOEhOMM?feature=share
Hi, I had this working with the old code and saw that they have changed it but when I bring in the new one on your Chaperone site I get OpenAI is not a constructor error any ideas where I have messed up?
@@technithusiastThat worked a treat thank you did notice the the question check is coming back as True not true so was going to the action branch. Thank you for the assist.
I just found your channel and loved the video, I don't know if was only me, but when you change from talking to the camera to voice over the volume change a lot, anyways...nor here to complain, loved the content!
I need to sit down and dive in to this. I want Chat GPT to control my animatronics and haunt my household. *Faint whirring of servos outside the bathroom door at 2am*
Love the video! Just one thing of critique: if possible, try to keep the volume the same throughout the video. I'm assuming it is adjustable in the video editing software. I'm listening to this while I'm work so it a bit of a hassle turning the volume up/down. That a side, you've getting some crazy cool automations 😎
Thanks for the feedback. A lot of my earlier videos like this one had volume issues (I was working on crappy speakers) but a lot of the recent ones are better now
I'm struggling here - if I send the message - "Turn off the kitchen lights" it all flows correctly but, when I debug, the answer I get back from openAI is: gptAnswer: "To turn off the kitchen lights, locate the light switch, usually located near the doorway, and flip the switch to the off position." This response of course causes the "handle optimized response" node to fail!!!! Suggestions?
Love the series so far. Had to pay up to openai to get access to the API to try things out. Is voice control next step? Was comparing and testing against HA assist and voice with Google, but the language you can use when having it via gpt is something else. Also really like the format of the videos and how they are put together. Looking forward for more content!
Thanks and I’m glad u like it! Voice control is possible currently the options for it are a bit inconvenient or round about so I’m waiting for innovation in that department. The next step is… well I’ll wait till the next video drop. I think everyone will like it.😁
I keep getting "node: handle optimized response function : (error) "SyntaxError: Unexpected token ` in JSON at position 91"" With errors in various positions :(
I won’t take part in the contest, cause I’ll have to speak to chatGPT in French (and I’ll already have to try to translate your template so that chatGPT understands it 😅) But I hope someone will find something very smart to solve this problem ! Because of you, I’m starting to use node-red in HA… thank you !
subscribed before. I am going to watch this one, then start with chapter 1. Then assess if I want to use GPT. If so... I'll be hooked to this channel I guess (alongside a couple of other great HA channels)
Glad to have you aboard! Just FYI, some have reported issue with the instructions. I suspect if you follow it verbatim you may run into some issues too since this video is based on an older version of OpenAI however the concept is the same so you can adapt OpenAI’s newer API. To this example.
the quality of these videos is awesome. thanks so much for sharing these.
I appreciate it and I’m glad you enjoy the content!
Love you channel and the way you explain things. I know enough about coding to understand the concept of what you are talking about, but you make it so simple.
I appreciate the comment! When I talk to my family about this stuff I can see their souls leaving so I try hard to make it understandable and engaging here on YT 😁
I love the Garvis like home assistant, is there a way for a none brainiac coder to deploy at home?
And is it possible to do locally?
Hmmm unfortunately, since the technology is experimental you’ll have to do a bit of coding to get things working. I feel that over time once the tech becomes more mainstream we will start seeing easier implementation of Jarvis-like assistance.
Looking forward to the next update!! And sneak peaks, ir teasers of whats coming?
These videos are outstanding. My only request is that further series have the episode number in the title or description. I went around out of order before finding the beginning. More please!
Thanks for the suggestion and I’m glad you’re enjoying the content 😁😁
WOOHOO! Im excited!
🎉🎉🎉
Thx for your videos !
Instead of asking for actions and entities to ChatGPT, in the case of a command, you could use the conversation service of HA, and ask to ChatGPT to simplify the command (because HA can't handle complexe commands).
eg: "simplify the next sentence, without coma, with the format: action, entity, area" and then pass it to HA conversation service.
If it's possible to get the result of a service call, it could be interesting to first, call the ha conversation service, and if it can't handle the command, run the chatGPT automation.
The aim is to reduce the response time.
Now I'm using Node-red because of you ! Thx ;)
Hey thanks for the feedback! I originally was using the conversation api but it stopped working after I updated a few weeks back. Additionally I also tried capturing whether the command failed but it kept returning wrong information so I had to pivot which led me down this interesting road. But Perhaps they fixed the conversation api by now🤷🏾♂️
18:54 to answer your challenge, I would imagine tag lining a string that can be looked at, and if it detected will do a certain action.
I could ask the chatbot "please turn on the lights"
and it could say "Sure, turning on lights... command.homeassistant("light1","turn_on")"
and there, if the string is found. it could detect that and interpret it as what to do.
i followed your scripts, and it works really well, the ease is just great! i only have 2 questions, it seems to send across all entities, including disabled ones, and with those the list is so long, that openai seems to not work. Any idea how to filter deeper on states of entities, before sending the array across, secondly, how to deal with still long requests to openai and make sure it accepts the input and give a response?
thanks for a great masterclass
Hey thanks! You’ll definitely have to adjust the templates to return relevant entities for your environment. The best way to to dig deeper is to use Templates section in the Developer Tools. That is what I used to test out how to query for the entities I wanted.
Cool. Video(s). I recreated all of your stuff and made little extensions:
- All entites have the room name in the first position of the "friendly name", so GPT knows what entity is in which room
- For all entities I want to expose to the KI I changed the name to .ki_, so I can filter in the "get entities" node for the regular expression "\w+\.ki_w+" to keep the result array small.
- I extended the action list to "Action List: turn_off, turn_on, toggle, close_cover, open_cover, set_cover_position". After the "handle optimized response" node for commands I added a switch to choose between if its a "light" command or a "cover" command via reg. exp. and added two separate "call service" nodes (one for "light" commands, one for "cover" commands). So Chat gpt is now able to control lights and covers. At the moment I expose the main light(groups) of every room, the covers (in the moment only of the office for testing) and the temperature and illuminance sensors of every room.
I am experimenting now. Funny: When I tell GPT "It's warm in the office" he closes the shutters :-). When I tell GPT "Make cinema atmosphere in office" he turns on the office lights...
Keep up the great work. Looking forward for the next steps. Would be cool if
- GPT would respect the states of the entities more (It turns lights off although they are already off)
- GPT would use the outcome of the older calls ("Should I turn of the office lights?" "yes" does not work)
- If GPT could also dim lights and change color
etc. etc. 🙂
Regards, Daniel
OK, he now starts the roborock if i tell him "It's dirty" 🙂
Very exciting stuff! I do have something that could help with some of your wish list items but I’m trying to polish it up so it’s more user friendly. I’ll keep everyone posted when it’s done
Is there any free alternative to OpenAI? Can this method use a free local ai for example running on a linux machine?
Technically you can do something very similar using LocalAI. I'm currently testing it out and will let you all know what i find!
Little question, maybe we could include the room in the datas we give to chatGPT, to help it answering more easily (like "turn on the kitchen", with the room, it don’t need to look for lights which have kitchen in their name, but use the room… (thinking about it because not all my lights have the room’s name in their name)
As I already said in previous comments, love your content !
You’re absolutely right! We could add the area as part of the data to chatGPT. Over time I’ll refine this to be more robust.
Great tutorials, man. Too bad I'm too limited to implement this. I spent a whole day banging my head against the wall, trying to get the bot to respond on Telegram. All it would say was "on my honor," and I couldn't figure out why. Additionally, I wanted the bot to retrieve text from the family group, but it only responded to direct messages. I checked the chat IDs and everything, but I think I'll hold off on delving deeper into this AI thing until Homeassistant comes up with a plug-and-play solution. :/
You're a genius!
There is a Telegram bot integration which may help a bit. Group chats are a bit trickier to setup but its possible though! I am working with another view who has an integration that allows you to use Home Assistant Assist's chat to communicate to your node-red environment. This way you wont need Telegram and you can use the home assistant App.
had u found a solution here?! i have the same problem ..
@@BenjiBizz Personally I have "paused" that telegram project for now. But any tip will be much appreciated.
@@AlonsoVPR sad but i understand… this Videos here are unfortunately useless now .. really sad
a simple question, in video 2 at 20:18 you told us in the next video we going to learn how to do the things you told at this moment of video but in video 4 you forgot to explain how... so I'm stuck and unable to continue the tutorial can we please have this video part please cause I think people like me who learn with you together can't go further cause this part is missing 🥺🥺🥺
I try to balance giving enough information with storytelling to ensure that the videos have a wide appeal and encourage enthusiast like yourself to explore for yourself. I mentioned the automations are “too-shelf” but a better way to put it is that they are complicated… The automations were very hard and many would require help that I don’t have the resources to give so I opted for a more storytelling format to inspire everyone to try.
HOWEVER… I am working on something that should make creating automations with GPT easier. I will also creat a few videos to walk viewers through how to use this new tool so be on the lookout for the video and hopefully it should help make creating these automations easier. 🤞🏾
@@technithusiast oh thanks ! 🙏🏽 I will wait 🙌🏼
@@technithusiast i was thinking yesterday I watched a vidéo in French from underscore Chanel, they talking about create assistants ( gpt ) and inside you can put everything you want him to know inside like the home assistant guides or node red library ( idk how can we have accès on every pages of the node red library and home assistant library of guides but in theory you can put every thing inside and in theory someone can ask the assistant inside home assistant or node red how I can create this and this and I want that work with this ( it’s an exemple ) for those who are like me and know nothing and don’t know how to code, and il theory if in right he can answer us or even maybe do it by himself if we give him the rights actually it’s just an idea but event for your home you can give him eventually every entry’s etc every thing you want him to know about the home and it going to be more accurate because of his data where he can find every instructions. And gpt vision can actually understand images so eventually I’m going to try il juste taking your automatisations screen and give it to him trying understand a little bit better, it’s just an idea but maybe that can help you to have some ideas so if it’s can help you it’s good to know
@@vovas.1061 it sounds like you’re referring to the Assistant Api. It is powerful but it’s not meant for controlling smart homes. I created a video about it here:
- th-cam.com/users/shortsHyY6BRVibc4?feature=share
- th-cam.com/users/shortsOO4nxOEhOMM?feature=share
Hi, I had this working with the old code and saw that they have changed it but when I bring in the new one on your Chaperone site I get OpenAI is not a constructor error any ideas where I have messed up?
You may need to manually remove the openai version from your node modules folder. It could be using a cached deprecated version of OpenAI
@@technithusiast have removed the open ai folder and I have restrated the server but still the same error :(
@@aidanb8719 have you tried manually updating the openai version referenced in the package.json in the node-red folder?
@@technithusiastThat worked a treat thank you did notice the the question check is coming back as True not true so was going to the action branch. Thank you for the assist.
@@aidanb8719 👌🏾👌🏾
Question: is this also working with other languages? Or is it only working well with english language?
I haven’t tried GPT with other languages? I assume if OpenAI supports other languages then it should reasonably work within home assistant
I just found your channel and loved the video, I don't know if was only me, but when you change from talking to the camera to voice over the volume change a lot, anyways...nor here to complain, loved the content!
Thanks for the feedback and I’m glad you enjoyed the video!!
I really enjoy your videos and appreciate your content. Now if I could only enroll in ChatGPT - apparently they aren't taking any more testers...
I’m glad you’re enjoying the videos! You should still be able to register as a developer on open ai website: platform.openai.com/overview
@@technithusiast thanks! I thought I had to go through the general enrollment.
The "can you turn off..." command phrased as a question "bug" doesn't work for me. It recognizes it as a question and replies it did it.
Yeah… this is one of the downsides to using Ai. It’s essentially a black box. 🤷🏾♂️
@@technithusiast but funny enough, I think that what I came up with for your challenge solves that too 🤔
I need to sit down and dive in to this. I want Chat GPT to control my animatronics and haunt my household.
*Faint whirring of servos outside the bathroom door at 2am*
lol that sounds like fun 😆
Love the video! Just one thing of critique: if possible, try to keep the volume the same throughout the video. I'm assuming it is adjustable in the video editing software. I'm listening to this while I'm work so it a bit of a hassle turning the volume up/down. That a side, you've getting some crazy cool automations 😎
Thanks for the feedback. A lot of my earlier videos like this one had volume issues (I was working on crappy speakers) but a lot of the recent ones are better now
I'm struggling here - if I send the message - "Turn off the kitchen lights" it all flows correctly but, when I debug, the answer I get back from openAI is:
gptAnswer: "To turn off the kitchen lights, locate the light switch, usually located near the doorway, and flip the switch to the off position."
This response of course causes the "handle optimized response" node to fail!!!!
Suggestions?
My error - in the command section of the flow I was sending msg.prompt up to openAI instead of msg.template.
PS I'm loving your content
Glad u were able to figure it out!
I keep getting the same - "TypeError: Configuration is not a constructor" msg on the OpenAi node. cant wrap my head around it.
Chances are the api is different. Compare the code to the one OpenAI playground. Or if you don’t mind waiting a week or two a fix should be available.
I cant get the open ai to work in node red. Is something changed? Or do i need to pay for open ai to use this?
You’ll need an api key. Some people are reporting issues but it’s unclear if there’s a common denominator.
Love the series so far. Had to pay up to openai to get access to the API to try things out. Is voice control next step? Was comparing and testing against HA assist and voice with Google, but the language you can use when having it via gpt is something else.
Also really like the format of the videos and how they are put together. Looking forward for more content!
Thanks and I’m glad u like it! Voice control is possible currently the options for it are a bit inconvenient or round about so I’m waiting for innovation in that department. The next step is… well I’ll wait till the next video drop. I think everyone will like it.😁
I keep getting "node: handle optimized response function : (error) "SyntaxError: Unexpected token ` in JSON at position 91"" With errors in various positions :(
Have you tried this: th-cam.com/video/J0_mi7U0wCM/w-d-xo.html
Awesome
😁
Love your ideas. I sent you email of my approach.
Awesome thanks for the submission!
I won’t take part in the contest, cause I’ll have to speak to chatGPT in French (and I’ll already have to try to translate your template so that chatGPT understands it 😅)
But I hope someone will find something very smart to solve this problem !
Because of you, I’m starting to use node-red in HA… thank you !
No worries and it’s exciting to hear you’ve taken a liking to node-red! I’m not sure why more people don’t use it but I hope u enjoy the experience 😁
When my homeass setup is done i will intigrate chatgpt
👌🏾
GPT 3 sucks at this. GPT 4 does everything correct intuitively.
I managed to get gpt3 to work but it took a lot of effort. Now that I have access to gpt4 developing was definitely smoother