How I CREATED My Voice Assistant Device With WAKE Word Using ESP32 And Home Assistant
ฝัง
- เผยแพร่เมื่อ 20 พ.ค. 2024
- In this video, we will look at I built my own Voice Assistant device with ESP32 and Home Assistant
You can get the 3d model and the ESPHome code in the article here: smarthomecircle.com/created-v...
------------------
Support My Channel
___________
Buy me a coffee : www.buymeacoffee.com/amrutprabhu
------------------
Follow me on X : / smarthomecircle
-----------------------
-----------------------
Components Used In the Voice Assistant :
-----------------------
ESP32 WROOM : AliExpress: s.click.aliexpress.com/e/_DB4...
Amazon US: amzn.to/45vab4p
Amazon DE: amzn.to/3FbreOc
LED Strip WS2812B :
AliExpress: s.click.aliexpress.com/e/_DCY...
Amazon US: amzn.to/3R99biv
Amazon DE: amzn.to/40NLnE7
INMP441 Microphone :
AliExpress: s.click.aliexpress.com/e/_Dmn...
Amazon US: amzn.to/3GdxO7o
Amazon DE: amzn.to/47uScws
MAX98357A Audio Amplifier:
AliExpress: s.click.aliexpress.com/e/_Ddy...
Amazon US: amzn.to/46wKTmW
Amazon DE: amzn.to/47vEMjH
3-watt Speakers:
AliExpress: s.click.aliexpress.com/e/_DBD...
Amazon US: amzn.to/47mCC6g
Amazon DE: amzn.to/49BbAJR
-----------------------
Smart Home related board To Buy:
-----------------------
ESP32 WROOM : AliExpress: s.click.aliexpress.com/e/_DB4...
Amazon US: amzn.to/45vab4p
Amazon DE: amzn.to/3FbreOc
D1 Mini : AliExpress : s.click.aliexpress.com/e/_DEG...
Amazon US: amzn.to/3RPY5zH
Amazon DE: amzn.to/3PO5jSd
-----------------
Gear I use :
-----------------
Sony Alpha a6000 : Amazon India : amzn.to/3RM7QMJ
Germany : amzn.to/3G1Iw24
US : amzn.to/3cpKZ9E
UK : amzn.to/3J0g5Ry
Razer Microphone : Amazon India : amzn.to/3aQxI9C
Germany : amzn.to/3DRRrAh
US : amzn.to/3ITKu3Q
UK : amzn.to/3v3RHbU
--- Chapters --------
00:00 - Introduction
00:29 - 3D Model Description
01:37 - ESP32 For Voice Assistant
02:46 - MAX98357A Audio Amplifier
03:14 - INMP441 Microphone
03:40 - Voice Assistant With ESP32
04:03 - LED Light
05:45 - Microphone Position
07:22 - Connecting All the Components Together
08:44 - ESPHome YAML Code For Voice Assistant
13:05 - Installing ESPHome YAMl Code to ESP32
14:35 - Connecting ESP32 To Home Assistant
15:55 - Testing Voice Assistant With Home Assistant
16:22 - Final Piece Of the Device
17:25 - Final Voice Assistant Device Look
17:50 - What's coming Up Next?
Music Credits:-
Sappheiros - Promise
----------------------------
Social Media
TH-cam / sappheiros
Spotify goo.gl/hE9MDJ
Twitter SappheirosMusic
Instagram sappheirosmusic
Facebook SappheirosMusic
Discord discord.gg/Pk87yN9
/ promise
-------------------------------------------------- - วิทยาศาสตร์และเทคโนโลยี
Great tutorial! I've spent less than an hour to repeat your project on bread board and everything works! Thank you very much!
Glad it all worked 🙂
really cool project. Beautiful product good job.
After a full day of work and debugging, it is finally working! Thank you for this tutorial
Glad it helped 🙂
what did you have to do in your debugging to get it to work?
Fantastic work! Just subscribed.
Very well done! Great project! Thanks for the video! Keep up the good work!
Thanks a lot ☺️.
Thank you for all this great effort, eagerly awaiting the next video
Thanks a lot for the continued support.
Very cool! I remember writing this myself back in the day from scratch - this is much, much better!
Thanks a lot for such kind words 😄. Glad you liked it.
Thank you for your great contribution, I look forward to your next video.
Thank you 😄
Very cool
Super Job. Well done. I too look forward to your next video.
Thank you 😄
gracias!!!
👍👍👍 good Job 👍👍👍
this is a great home brew device well done..!! I'm working on something similar..👍
Thanks a lot. Hope yours also turn out to be great. Happy Building 🙂
Thanks!
well done.thanks 🙏
Thank you 🙂
Thanks for your works. Great tutorialt. Short question: can I use ESP32 S3 N8R2 for this setup?
I haven't tried it. But checkout my latest video wherein I used N8R2 with on device wake word detection. th-cam.com/video/rT_q0g8dtmY/w-d-xo.html
Can you please slice the STL to be 3 separate prints? Not one large one?
Thanks for the tutorial😊... Does this project use local wake-word, processing on the ESP32 rather than on Home Assistant? Sorry that I'm asking but I'm new to all of this, and I'm looking for a local esp32 controlled wake-word project.
No. This one is processing in Home Assistant.
But I am working on the local wake word project on an ESP32 .. video will be out soon.
I recently tested this (prior to watching your video). The ESP32 randomly stops working due to memory buffer limitations; it will recover but it is inconsistent. I am testing the Wyoming satellite project with a Raspberry pi 3 and same mic and amp, and it seems to be more reliable. Any raspberry pi supporting 64 bits and wifi should work.
do you mean the memory buffer while playing the media or the output of the voice assistant? then yes. I do face the same issue. I however dont care about the the output as long as the command is recognized and executed successfully.
I am still using the device as my daily driver and since my setup is completely local, I just love it. 😍
Not sure if you can help with this one, I built one into an old Echo device and implemented more pins on the ESP32 to use the buttons on the echo. I added a red LED to show if the mute mic button was pressed and code to turn off the wake word if its pressed. However, if I say Hey Jarvis, turn on the lights, it also turns on this LED
# Red LED
- platform: binary
name: "Mic Light"
id: mic_light
output: light_output
output:
- id: light_output
platform: gpio
pin: GPIO19
# Turn on off the mic
binary_sensor:
- platform: gpio
pin:
number: 18
mode:
input: true
pullup: true
id: "Mic_pin"
disabled_by_default: true
filters:
- delayed_off: 10ms
on_press:
- light.toggle: mic_light
- if:
condition:
light.is_on: mic_light
then:
- switch.turn_off: use_wake_word
else:
- switch.turn_on: use_wake_word
Your code seems logically correct.
May be you could add a delay after the light.toggle to allow the light state to be in the on state, so that the next if condition gets the light.is_on is true.
Let me know if it works.
How have you connected both the speakers.? The MAX98375A module has only one output? A great video again. I plan to make a wooden enclosure. Will share it when done.
I had connected both speakers to the same output. You can connect one speaker if you want to. Share it when you make one. 😃
Thanks. Will try it when the components arrive.👍
good project! can we use the speaker for multiroom?
If you specify a command with a room, it would work.
Hi, is it possible to have multiple microphones streaming to wake words?
With the new micro wake word, yes you can.
Check my other video on using micro wake word..
I do have an issue now, as its been mentioned somewhere else here, when you plug in you get the nasty sound coming out of the speaker. I now have that all the time, funny thing is, if I put my finger on one of the esp32 pins it stops, could there be some sort of grounding issue somewhere?
I was still doing some research somewhere. It could be the pins are not set in the right state and cause this problem.
I think it could that the amplifier pin is not set in at right state and causes this cracking issue. I am still working on a solution.
Once I find one, I will be making a video for it.
Could this also be done with a Raspberry Pi Pico W? I have the other parts, but no ESP32 yet. Thanks for the video!
ESPHome supports Pico. But I am not sure if it's supported or not. You can try it out and see.
For me I tried esp32s3 dev kit N16R8 and a Esp32 D1 mini, and they are very unstable. It crashed a lot, sometime it is because of brownout, sometime is because of the media_player component, which took too long to process and not reset the watchdog timer. What about yours?
For me I am using the normal ESP32 dev board. I started having problems with my batch of USB C based ESP32 related to wifi.. So I switched to the micro USB ESP32 dev board. Its pretty stable. Only in like 1 out 7 times the response from the command does not get played on the speaker.. Otherwise pretty stable..
Hey, is there higher wattage amplifier with I2S? I'd like to use more powerful speaker but i didn't find I2S amplifier with higher supported wattage.
As far as I know its not available.
@@SmartHomeCircleAlright, np, as it is just AMP with DAC, i split those two to individual components so i have DAC and AMP separated. With setup like that i'm able to power stronger speaker with those I2S components.
Ohh.. that's what you would have to do. There are DACs for I2S which I think would work. But ya separating them is the way to go.
Built this and it works great except I can't get any audio out. The ESP media player entity exists but no audio occurs. Tried two amps, same results.
Thanks for the project.
Does it absolutely not play anything?
meaning the output from the text to speech?
Thanks for getting back to me.
No output at all. I have built two with the same results. The voice input works really well, just no audio output. Tried different pipe lines and still nothing.
Speech output from other sources is ok in the HA setup.
Cheers,
Bill
Hello,
on_stt_end green light Doesn't work. If you change effect: "none" with "Scan Effect With Custom Values" working only 2 LED's are green after success stt end
this usually happens when the tts is fast or is cached. You might see the green color.
if you want the green led to glow up for a longer time, you can add a delay of 1 second for the green led to glow up.
Great work, got mine done today, had to wait for the container to be printed as I dont have a 3d printer. One issue I have, when a command is done I get the done message but the lights done turn green. However, if I click on the toggle in the interface they turn green. Any ideas?
This is correct.. the reason is the time is so short between text to speech, it doesn't show up. This is because after the first media file generation, the media file is cached. If you try to clear you media cache, and give the command again it will take a little time to generate the mp3 file and then you'll be able to see the green color.
If you really want that green color to show up every time, you can put a delay statement of 1-2 secs, after the green light turns on.
Also I am really glad I could inspire so many to create their own voice assistant 😄
@@SmartHomeCircle As I am not a yaml programmer I wouldnt know how to do that but thanks for the reply
@@SmartHomeCircle would this be right? on_stt_end:
- light.turn_on:
id: led_light
effect: "None"
red: 0%
green: 100%
blue: 0%
- delay: 2000ms
Yes.. that also works. Or If you want you can also add it here:
on_end:
- delay: 2s
- light.turn_off:
id: led_light
Woah, I didn't realize this could be done with the Arduino framework. So this can also be used as a media player? For example, I could send a text-to-speech message to it?
Yes you can. It had tested the setup using text to speech. And it works.
@@SmartHomeCircle thanks! That’s definitely worth a Super Like! (or whatever it’s called. 😀)
@@hi-ccowboy7983 Wow. Thanks a lot for this. That is such a huge appreciation. This keeps me going. 😍
Firstly, thank you for the project. My son and I are building our first local assistant.
We have an issue with static come through the speakers when wake word is enabled. Is anyone else had this issue?
This static sound is one of the issues that is still a problem. I am not able to figure it out yet.
The only way to get around it is to give a voice command and wait for the output from the speaker.. It then stops.
Awesome Video, just tried to recreate it but with slightly different hardware and without the speakers. I have everything installed and the LED's are working but it looks like I have trouble getting the mic to work. The ESP home logs just constantly repeat this: [12:29:24][D][light:036]: 'Voice_Control' Setting:
[12:29:24][D][light:085]: Transition length: 0.5s
[12:29:24][D][voice_assistant:200]: Requesting start...
[12:29:24][D][voice_assistant:412]: State changed from START_PIPELINE to STARTING_PIPELINE
[12:29:24][D][voice_assistant:433]: Client started, streaming microphone
[12:29:24][D][voice_assistant:412]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[12:29:24][D][voice_assistant:418]: Desired state set to STREAMING_MICROPHONE
[12:29:24][D][voice_assistant:519]: Event Type: 1
[12:29:24][D][voice_assistant:522]: Assist Pipeline running
[12:29:24][D][voice_assistant:519]: Event Type: 9
[12:29:29][D][voice_assistant:519]: Event Type: 0
[12:29:29][D][voice_assistant:519]: Event Type: 2
[12:29:29][D][voice_assistant:609]: Assist Pipeline ended
This log is perfectly fine.
Since the ESP32 does not support VAD(voice activity detector), the pipeline is invoked every 5 secs to try and detect the voice.
Now there is one way to see what is recorded from the microphone for every pipeline run.
You can set :-
assist_pipeline:
debug_recording_dir: /config/shared/assist_pipeline
This above yaml section in your configuration.yaml file and restart. All the recordings will be recorded at the location.
But please remove it after you debug the issue or else it will fill up the disk space in Home Assistant and crash it. So be very careful with this option.
@@SmartHomeCircle Thank you for your answer. I thought something was wrong with it because I didn't see your esp doing that and nothing happened when I say the wake word. Ok, at least I know the microphone should be working, I know the LED's work as well because I can turn them on and off from HA so need to figure out why nothing happens on wake word. I use an ESP32-C3 Supermini board, could that be the issue? I ordered some dev boards but they are still on the way.
@@martinzipfel7843 it could be. I did see someone complaining about c3 not working as a voice assistant. I think it was a comment on another video.
I had ordered a c3 super mini and was about to test it for a voice assistant because of its size. but I am currently in my home town.
@@SmartHomeCircle I'll send you an update if I get it to work with the dev board, once they arrive (ordered on Aliexpress)
Well I need the .stl file for 3d printing but it is not available in the blog. Hopefully you can add it.
I am waiting for it to be published. I'll update you here again.
I published the 3d model and the article is updated with a link to it.
oh cool...thanks heaps for your help and reply. Much appreciated.@@SmartHomeCircle
Sir good work ❤,
Sir have my task
I wand read firmware my Huawei wifi please help me how to read firmware without open ic
Sorry.. but I dont know that. 😄
please help me !!! can i use lm386 audio amplifier
I am not sure about that. Sorry :(
Is your system totally local? No subscriptions (nabu casa?) The reason I ask is that lots of people are having issues with a requirement of HTTPS to use the mic with Home Assistant ESPHome browser UI. This is a huge problem since the local URL for HA is HTTP. You seem to make it work without this requirement. My particular setup uses the M5 Atom, which is an integrated ESP32, mic, speaker and button. Never been able to make it work due to the HTTPS requirement of the browser UI to use a mic.
My setup is completely local running on docker images.
The HTTPS requirement is of the browser to use the microphone. In case you are using atom echo, this is not required. You can use the assist feature without HTTPS if you are using an external device and not using the browser and you dont even have to open the assist window. So you cannot use the microphone option on the assist window on the browser without HTTPS.
However if you have the mobile home assistant app, you can use the assist window microphone option.
If you want to know how to work with Atom Echo, check my video here : th-cam.com/video/P_f1wd3vxbA/w-d-xo.html
Thanks so much for your prompt reply. I will checkout your Atom Echo video and hopefully get a setup that works with the M5 Atom without use of HTTPS!@@SmartHomeCircle
@@SmartHomeCircle Wow it all worked with your video!!!! I have asked everywhere and you are the only one that provided a real reply that worked.
Thanks so much. I am going to get the word out on your channel!
Just a note....there was a recent update to ESPHome on HA. It now needs this to get the right OTA version 2. Get the update, go to ESPHome and UPDATE the M5. Works
Tolles Projekt läuft gut nur das Problem mit den lauten Geräuschen ist nicht toll ich habe das ganze in eine alte Bluetoothbox gepackt
Do you mean the cracking noise when the device starts? Ya. I am facing the same thing. There is no fix to it yet. But I am working on a solution.
Can this configuration also play notifications or other local media files or just tts?
Yes.. you get a media player entity in home assistant which can be used to play local media files.
@@SmartHomeCircleI saw the entity too, but the speaker won't play any media or tts. Any ideas why?
@@francisudeh1393 try playing a media from the devloper > services and then choose play media service. Also the media should be accessible over your wifi network to be played. Thats a pre requisite to play any media on a media player.
@@SmartHomeCircle thanks. I'll try it out in the morning
@@SmartHomeCircle this worked, thanks.
How many hours takes to print the 3d parts?
It depends on the printer and the print quality. I used the creality ender 3 and it took about 6 hrs for all..
Okay so I wired everything up but the microphone and the speakers do not seem to work. I tried using the wake word and even changing it, but there is no detection happening, the speakers do not play any sounds either. I used hot glue to secure all of the jumper cables into the components. Maybe I destroyed the microphone or amplifier? I'm not sure how to fix any of this.
everything else seems to work, the lights can be turned on and off, and the board is recognized and shows all the entities.
Can you play a media file using the amplifier?
If so then, then you are pretty close. Try to see if the connections are correct for the microphone and also check if you have the set the preferred assist pipeline in the ESP32 device page.
@@SmartHomeCircle I will have to try tomorrow as the cables I was using were not really appropriate for the hookups. I had to do a lot more splice than I would have liked and I did not have female to female dupont cables which would have made my work a lot easier. I will wait for the new cables and then I will try to solder the pins to each component and then use the female cables to connect everything, then I will try again. Thank you for all your help. The code installed without any issue to the esp32.
@@SmartHomeCircle I don't think it worked, but I cannot test it out on any other of my devices since the only other media players I have are actual alexas and they do not support TTS without the notify call service
@@b4iwakegaming just want to know what you tried. Did you use the tts service in the developer tools and selected the target as the media player from esp32?
2024.5 ESPhome update seems to have broken this. Works now only once after reboot and then stops responding. Anyone have an idea why?
Can you tell me what is broken?
Ahh, I see it stops working after the first command.
Ill try to find a fix for this.
I am currently using the on-device wake word detection which I felt is much more reliable.
Here is the link : th-cam.com/video/rT_q0g8dtmY/w-d-xo.html
So my instance is all connected correctly and works fine. Only issue is when first turned on I get a buzzing from the speakers that only stops after my first voice command and I get a voice response. Any ideas why this is?
Ohh yes.. its one of the problems that still exists. I mentioned it in the video and haven't got any work around for it. The first command solves the issue. And I am currently using it that way. 😄. If I get a solution, I'll make a video about it.
Thank you for taking the time to respond. Love what you’ve doing for the Home Assistant community. I’ve built mine into a Dr Who TARDIS toy.
@@nigelbye7965 ohh lovely. If you are on twitter, please share a photo. There would be many who would love to see what you created. 😃
@@SmartHomeCircle I’ll make a little video and upload it somewhere
@@nigelbye7965 Niceeeee...
I build this device.working good, but some times the Device dont „speak“ to me.
Do you mean sometimes it does not speak out on the speaker to the commands you have given?
@@SmartHomeCircle yes.
And this happens sometimes I guess.
Its a problem that exists. Sometimes the speaker buffer get too full to play the output. I also have the same problem in my current smart speaker.
Check my latest video related to on-device Wake word with ESP32 S3. As of now its working better. I will be creating a new device with this new setup.
Do I need a breadboard to do any of this?
If you have one, it would be helpful. But if you have jumper wires then you can connect them directly.
@@SmartHomeCircle okay thank you! Great work with this by the way, the unit looks really nice!
@@b4iwakegaming thanks a lot for such kind words 😄.
I love making things and want to share it with everyone so they can also make them. If you are interested you can make one by yourself. Everything is present in the article in the description.
Also working on a new V2 version of this one. So I will make a video about it also.
@@SmartHomeCircle I placed an order as soon as I saw your reply. The components wont be here for at least another week, maybe 2, but I plan on building this for myself (: Thank you again. I'm going to check out v2 now.
Code??
Check the description.