How to connect ESP8266 and Arduino with Google Assistant and Google Home App without IFTTT - Part 2
ฝัง
- เผยแพร่เมื่อ 17 มิ.ย. 2024
- This vide shows how to connect ESP8266 and Arduino devices with Google Home and Google Assistant without third party like IFTTT.
This video shows how to set up the a front-end application to create devices to be used with Google Assistant and Google Home app. Then you can connect through MQTT your IOT devices like ESP8266/Arduino/ESP32.
Instructions
Clone the github project smart-home-frontend :
git clone github.com/actions-on-google/...
- Make sure you have Node.js and npm installed, if not, install both.
- At the directory smart-home-frontend give the commands below:
- Obs: run cmds in a shell terminal in linux or gitbash in windows:
firebase login
- Use the same project-id as used for java backend application, part 1.
firebase use --add project-id
npm install
npm run create-firebase-config
npm run serve
Open the front-end application.
After adding devices, do not forget to add field errorCode with an empty string, as explained in the video.
ESP8266 code example on this video can be found here:
github.com/electrofun-smart/s...
git clone github.com/electrofun-smart/s...
Mqtt library used: www.arduinolibraries.info/lib... version 2.4.7
If you have any questions or suggestions, please add your comments down below and I will answer!
Thanks for watching! - วิทยาศาสตร์และเทคโนโลยี
Thank you so much for the videos - they really helped! Write a small client on my Pi, and now i can control the GPIO pins through my google nest speaker. Next up is a set of ESP32s devcices to control parts of my home brewery.
Thanks a million. Great work - couldn´t have done it without your videos :)
I have a question that I hope is not too off-topic: If I wish to have custom responses from my home-made devices, would that be possible in this scenario or do I need to use local fulfillment for that?
Olá, parabéns pelo vídeo, ficou muito bom e vou tentar implementar. Fiquei com uma dúvida. Após tudo configurado, o front-end em Node.js precisa ficar rodando para funcionar ou ele é necessário apenas para a configuração dos devices?
Obrigado.
Oi Rafael, obrigado! Ele é necessário só para configurar os devices. Como faz um certo tempo que fiz este video, pode ser que o Google atualizou o software e alguma informação do video esteja desatualizada, se vc encontrar algum problema me avisa aqui que tento ajudar. Abraço
Hi, thanks for the Video.
Before I'll start with this project, I would like to know whether there is a fee that needs to be paid for google account for using it or for the webpage itself. will it be free in the future? is there a limit on the number of devices connected?
Thanks,
Doron
There is a fee to host the application on Google Cloud, if you use it for yourself, the price is quite reasonable, less than 5 us dolars for my case. But if you want to add more users, which will increase the traffic in the application, then the price will also increase.
can you do an updated video please? would be greatly appreciate, thanks!
Hi! I have some questions to ask and I hope you'll answer them.
1. MQTT and smart speakers like google assistants can't connect directly because the assistant doesn't support MQTT. That's why we are making this whole setup.
2. Are you using firestore to switch/control the devices or just to save the device details?
3. How to connect AWS iot to google assistant?
Hi Vinay,
1. Google has also a local sdk that can connect directly with devices, but I'm not sure if it uses MQTT, you can check here:developers.google.com/assistant/smarthome/concepts/local
I never tried to use this Local SDK, so that is why I made and is needed this whole setup.
2. The firestore is just to save and keep the devices details. The switch/control is in one direction trigger by Google and on the other end by the device itself.
3. This AWS IOT I never used, could you give me more details?
@@ElectroFun_SmartHome Thanks for the quick reply😊
Few more questions.
1. Are you using firestore to also store the value of your IoT devices because you are sending the device state from firestore in QUERY function. shouldn't we fetch the device state from MQTT and why are you using firestore to store the device's state?
2. if you don't mind can you share your contact details. so, that we can chat quickly. Comments are not a very nice way to talk... you know. but if you want to increase your audience interaction then we'll go with this.
Hi.. GooD Work, I'm sorry if this question is too obivious, I'm kinda noob with server setup etc.
Can we add the option of different credentials (ID n pass) for different so that each person has different devices in google home. If yes, can u guide how to do this?
Hi Aamir, yes, it is possible, but it is something that I was also planning to investigate and implement. Basically you need to implement the Google Authentication mechanism to achieve that. There are many examples on how to implement that but it is an area that I did not explored yet. In case anyone here that is reading can help will be awesome!
@@ElectroFun_SmartHome thank u for quick response 👍, you are great.
Hi, i have just tried one simple home automation with blynk + node mcu + 4 channel relay. How to add this to google home app?
Thanks for the videos, I have found them extremely useful in trying to design some iot projects of my own. Thanks to your help I have been able to create a device and control the light! I just noticed that any device I create in the frontend app appears 'offline' in the google home app. Is this because of something to do with the SYNC intent in the backend? I see that in some of your other videos that the devices appear in an online state. I looked at the arduino code and couldn't see anything in there that is different from my approach other than that you have changed mqtt broker? Is this the way to resolve the issue? Thanks again for the helpful videos!
Hi Max, I'm indeed glad that it helps you!! Good question about the state online, I did not mentioned that in the videos, but it is something that needs to be improved in this project. Currently I changed the states-->online to true on the firebase device json definition. If you navigate on firebase Cloud Firestore then on Data and select your device, you can change the values manually, so do it for your device and sync them again.
What I think could be a permanent solution for it, is to have a kind of heartbeat from your device (like esp8266) updating by post request the states online to true from time to time. And then implement a kind of timer in the Java backend that monitors this update and in case it does not receive the heartbeat in the desired time then Java should update itself the device to "offline". I will try it soon and post a video about that. What I also have not shown in my videos is how to update the backend Java with data from the device, like temperature for instance. It is not by mqtt, but by REST post request to the endpoint /smarthome/update. I will also prepare a quick video about that. Cheers!
@@ElectroFun_SmartHome Thanks for the reply! I changed the devices->(my-device)->states->online = true in the firestore but I didn't see a difference straight away. I don't think that I know how to send the SYNC command, is there a simple way to do this? I tried to ask the assistant to reset but this had no effect. Another problem is that when I use voice command the action executes correctly but the assistent replies with 'there was a problem with '. Very confusing....
I wonder if the reason could be due to the 'report state' implementation in the java backend? I think this is google's preferred way to receive device state info but I'm not sure. I noticed that there were some errors in the appEngineDeploy gradle build script related to this? Let me know if you have any other ideas. Thanks again for the advice and videos!
@@hewhohatesall The sync you can say to G Assistant "Sync my devices" and it should work or try to sync. To check the erros you mentioned I used to check the Logs on Google Cloud console related to your project (console.cloud.google.com/logs/viewer?project=) it gives you some hints about what is going on.
@@ElectroFun_SmartHome Thanks for the advice I did as you said and I can see in the logs that the sync request was dealt with correctly by the java backend. However every time I try to send the OnOff intent google is sending a QUERY intent immediately afterwards. In the logs this query is not served correctly and I recieve error code "You must pass credentials in the app constructor". It is my suspicion that this is a problem with the backend mqtt java server? Have you encountered something like this before? Thanks again for your help!
@@hewhohatesall Yes, I got this issue in the past. The problem is on Java backend, check the class'es files that start with SmartHome... and check if there is a slash before the credentials file, like this: GoogleCredentials credentials =
GoogleCredentials.fromStream(getClass().getResourceAsStream("/smart-home-key.json"));
Hi . Good work . What if we use IFTT does it will be simple more than this way
Hi Nidal, thanks! IFTTT can be used, but the delay is higher than when using this system. Other point is that the commands in Google assistant is shorter when you integrate directly using a Google Application.
Also the advantage to integrate like I did is that you can use your devices in Google Home using the touch controls and take advantage to have your devices in Home Graph which allows you to control many devices at the same time. For instance if you have a room at your house and you are leaving you can say to assistant to turn off the room and all devices will be turned off. The same to turn on.
It's paid now
IFTTT also requires a subscription. You only get 3 applets if you keep the free account which is a huge hit to its customer base. A terrible move, imo. So videos like this are much needed.
how do you proceed from firebase login it is forcing me to use firebase init which brings up firebase console
Hello, thank you for this video. But the git smart-home-frontend has changed and the create-firebase-config and serve scripts no longer exist. Do you know how to work now or can you provide the smart-home-frontend.zip you used. Thank you
Yes, looks like there was a commit made in Nov 2020 that removed it. You can checkout the code from the previous commit from here.
github.com/actions-on-google/smart-home-frontend/tree/88e761920e95800a438f2e0f3ef040f689207579
@@melwynyou Thank you very much, I just have to try.
@@christianbricaud4577 did it worked? Have you done it?
@@camera_manu No, I didn't manage to make it work
@@christianbricaud4577 ok, thank you
hello, whats application you use to screen share android to pc?
I use this one in linux ubuntu github.com/Genymobile/scrcpy
The runable webpage do not work what should i do?
hey, i think the repo is not working as video this 2 commands are not in the code npm run create-firebase-config and
npm run serve
Yes, looks like there was a commit made in Nov 2020 that removed it. You can checkout the code from the previous commit from here.
github.com/actions-on-google/smart-home-frontend/tree/88e761920e95800a438f2e0f3ef040f689207579
Hello, how do you connect the google cloud with shiftr (or any broker mqtt)? thnks
Hi Uriel, I've adapted the Java Sample from Google to do it. I created a new class called MyMqtt.java, check it out here in github : github.com/electrofun-smart/my-smart-home-java-mqtt/blob/master/src/main/java/com/example/MyMqtt.java
You can use this code available on github instead of the one provided by Google.
Now I'm using cloudMqtt, but you can use any broker, just check the correct port that should be used, for instance now that I'm using cloud.mqtt I had to add its port in the hostname.
public MyMqtt() throws MqttException {
this(
"wss://postman.cloudmqtt.com:36157", // or "wss://broker.shiftr.io"
"SmartHomeJava",
false,
false,
"",
"");
}
@@ElectroFun_SmartHome thanks!!! I will try
@@drakjavier Great, any issues on your try let me know, I'll be glad to help. It is really nice when you get the cloud Java application up and running together with front-end application.
@@ElectroFun_SmartHome yes!!! I made it with nodejs, but yes... There are so many steps to control a device with Google home... It is a little frustrating. I want to know if Core IoT is usefull for this proposal. I think it could be used as broker ¿?
@@drakjavier I think the problem with Core IOT (pub/sup API from Google) is that it requires special security connection from IOT devices side with public key and also I could not create automatically new topics as I did with other brokers from Java application side, but I have not tried all the possibilities, with more investigation maybe it is also possible to be used.
Olá, tudo bem? Segui seus tutoriais e tenho a iluminação da minha sala funcionando perfeitamente, muito obrigado! Porém mesmo definindo o valor de online para true no firebase, no google home o dispositivo continua aparecendo como offline e com a mensagem de não respondendo. Alguma sugestão de como resolver?
Olá Andrey, mesmo você definindo no firebase o status - > online para true? Veja se a willReportState = true ou se há alguma mensagem de erro no Looger do seu projeto, fica neste endereço console.cloud.google.com/logs/viewer?project=
Hi, after "npm install" when I give "firebase login" I get this error " bash: firebase: command not found
"You can help me ?
Hi, try to follow the new instructions that Google posted here: github.com/actions-on-google/smart-home-nodejs
can i change shiftr broker to google firebase only? so we dont need server again.
Hi, any MQTT broker can be used, firebase as far as I know does not have a mqtt broker. You can also have your own mqtt broker like mosquitto, I installed it in one raspberry pi at my home and it is working fine with this Google Application.
@@ElectroFun_SmartHome can i using adafruit. Io? How to setting account info in mqqt project file?
Hello, thanks for your video! I'm not an expert and i don't understand very well all the things you do so i'm blocked at this step: i can't run this command in my linux console: "npm run create-firebase-config" the console say that: "npm ERR! missing script: create-firebase-config", maybe it's a basic error but i don't understand why it's not working...
Thank you guys.
Yes, looks like there was a commit made in Nov 2020 that removed it. You can checkout the code from the previous commit from here.
github.com/actions-on-google/smart-home-frontend/tree/88e761920e95800a438f2e0f3ef040f689207579
@@melwynyou I am facing the same problem. Could you explain what I should do?
As mentioned in the above link, I used npm run deploy and it is showing parse error in firestore rules.
Unexpected token 's' at 1:1
service cloud.firestore
@@jananivijayarajan3561 I have the same problem. This is a useless project.
Getting this error in the ESP-Code: mqtt_shiftr_smart_switch:15:13: error: no matching function for call to 'MQTTClient::MQTTClient()'
what mqtt library exactly were you using?
Hey, I'm using this one : www.arduinolibraries.info/libraries/mqtt version 2.4.7
@@ElectroFun_SmartHome thanks. can you show how to add a security system to google home?
Sir, your voice is too low. I want to do this project. I know basic coding. Can I do this project?
Hi, thanks for the feedback, I will try to fix the audio in the next videos. Yes you absolutely can do this project with basic coding knowledge, just follow the my first 2 videos and in case of any questions or clarifications I will be glad to help.
Everyone! Dont try this. Its outdated, i am using home assistant that controls the arduino, and google home can control the home assistant, so i can control arduino via google home. Its the best way to do this
Can u please tell me how can I control arduino/nodeMCU with home assistant?
@@gautamvarmak8207 sure, home assistant is a Operating system(witch you can self host, i host mine on my pc using a virtual machine) this home assistant uses a plugin called ESP HOME(or something) and it automatically creates a arduino code for your board(of course you need to specify in a code what do you want that arduino do. Like a smart switch, or a sensor, or both. Home assistant is something like google home UI, you can control things only via home assistant. But you can link google assistant to home assistant, to control the arduino, or in your case the nodeMCU. Hope you get it. I learned about it on the internet.
Edit: i have already done it and it works like a charm.
@@camera_manu Thanks a lot! I'll look into it. Can I reach out to u if I have any doubt? If yes, how?
My idea with this video was to show that is possible to have direct connection of Google Home with a ESP device, without HomeAssistant or other app and I agree that there are better ways to do it, like using HomeAssistant, specially when you have several devices to be connected. If you have just one or two devices, maybe sinric.pro is the best option or even my solution. Latter I discovered that if you have Alexa, then you can flash your ESP with Tasmota firmware and it can be connected directly with Alexa. So there are some ways to use and connect your ESP devices with voice assistant solutions. It depends of your project needs and how much money you can spend on it.
@@ElectroFun_SmartHome why the h€ll are you replying after 2 months(i know that youtube says 2 weeks, but its wrong, its been 2 months). you made me spend like 2 days of my life trying your stupi€d ideea
dosent work anymore video need to be deleted or title need to be changed
Please reply sir
Done