Thanks Jeff! I was just watching your video you published today. Everything you said in your video completely hit home with me between the comments you get and the unused tech all around. We should meet up at CES if you're going!
Basically it comes to this: when it comes to controlling critical stuff in your home (heating, lights, doors,...): it already works. When it comes to less critical stuff (playing media, getting stuff from the internet etc...) not there yet. BUT: this means that you can get your own personal assistent without internet for controlling lots of stuf in your home by voice commands without relying on google, amazon or others... thinking about buying one for use in my own office at home. Knowing the HA community it is just a matter of time and it becomes a competitor to the google/alexa stuff
HA has a fallback option that lets it go to an LLM (ChatGPT or a local Ollama for instance) should the built-in voice functionality be unable to grasp what you want, perhaps that could help with making voice interpretation more natural. Voice on a Raspberry Pi is kind of a fool's errand, although a Pi 5 has enough horsepower to at least keep delays down to a second or two; you really want beefier gear.
Funny cuz that particular echo didn’t have a display either. Not even the clock display. But… I don’t understand the whole starting up a conversation with you randomly. I am kind of torn. I think a smart home should be in the background just doing its thing. But it should be able to do what you want. Personally I wouldn’t want a smart home randomly suggesting to me what I want… or don’t want. Preferences… maybe. But “hey it’s bright should I shut the shades?” If you wanted them shut wouldn’t you just ask it?
The TTS and STT add-ons can be run on a separate device as Docker containers. I have mine set up to run on my Synology NAS in Synology's Container Manager application. That lets me run HA on dedicated hardware (HA Yellow) but also get more horsepower for the speech add-ons.
You should definitely try pairing it with an LLM. There are cloud and local options. It has built in support for ChatGPT (cloud) and Ollama (local). There is also HomeLLM addon that gives you a lot more flexibility in what endpoints to connect to (it supports OpenAI compatible endpoints, so TextGen WebUI, llama-cpp server and such all work). And it allows you to control prompting. You can make your own system prompt template using jinja for it. It sends all info about your house as context to the LLM, so then the LLM brains kick in and it is able to interpret many fumbly commands, even if TTS misinterprets some words, it can still understand what you meant. The downside is that it's an LLM so it may hallucinate. But that depends on what LLM you are running. Running all of this on an RPi is not a good idea, even STT and TTS are all neural networks so RPi cant realistically handle them fast enough. With a decent GPU you can get it down to 1-2 seconds between request and reply, including local LLM.
I've been playing with ESPHome the last few days with a Raspiaudio Muse Luxe speaker, and I agree that it is still quite dodgy. I'm certain that it will get loads better over time.
Could you use to announce alerts, ie. If you had a fridge sensor that said the fridge temp was above 4 degrees C could this device audibly alert you , play an alert tone?
So basically you're saying you need to straight up run it through your home hosted GTP so that it understands normal sentences, that's really not a big deal.
I’m pretty sure nothing leaves a Siri HomePod either. That’s a big reason why there’s such a huge variance of how smart each device is (HomePod Mini is dumber than an iPhone 15/16). Also a big reason why Siri seems so dumb, because they don’t have a huge mega server that’s getting smarter with every command.
Given your optimal use case.. could a stop gap video address updated information for actionable notifications in home assistant. As a new HA user, love the capabilities but struggling with outdated info and guides. Otherwise great video!!!
Honestly this is really cool and I love the fact that it has the option to integrate with things like ChatGPT since that actually means that it can process more complicated commands way easier and could even serve to have a lot more personalisation in the future especially if down the line it was able to implement functions like memories, I can also imagine that other LLMs that can be run locally will come out
Exposing entities is not unique to local voice control. You have to expose the entities to Google / Alexa also to use them. There is an option to automatically expose new entities if you wanted to do that. To be honest, I find plenty of times Google Home will not understand commands or get them wrong and need me to say them in a particular format. It's a little more refined than the Assistant in Home Assistant, but it's still far from perfect.
You can connect to Gemini for free and it fills in all the gaps for natural language interaction with the smart home. I realize this is not fully local, and actually goes against my philosophy, generally speaking. But in this case, I think it's a worthwhile trade-off until I have the hardware for a local LLM
I wonder how this compares to that local way of doing it with a raspberry pi. The only issue I had was that the speaker wasn't that great. So I was thinking if something like this works fine, I could rip open my echos and steal the speakers and maybe case and it could work the same way with great speakers.
It may be that average users would expect it to understand more than it does out of the box, but you can still make it recognize different commands. Give entities one or more aliases, and then it will know that you mean that one when you say it that way.
Is the function of the rotatable ring able to be changed? I'd much rather have physical controls for the dimming of a room light. Also, how are the microphones at picking you up from a distance?
I’d like to know what specs are considered “powerful hardware”, which is a very vague designation for what I might need to run it entirely locally. I have a Dell Wyse 5070 mini PC that has a pretty low base clock speed but can get some pretty decent boost speed. It’s been running HomeAssistant OS very reliably but I’m not sure how intensive the voice assistant add-on will actually be on it.
Heya! thanks for the review. Are there any other similar devices that work well with home assistant? I am looking to get rid of google home devices and place some microphone/speaker set-ups like this for commands only not listening to music or anything.
I’ve been working on my own (super basic) version of this thing recently after I realised that the WLED controller my Dad and I designed could work as one (since we recently added a mic and used the ESP32s3). And it totally works… except the speech to text I’m using SUCKS, so it basically never actually does what I want. But no worse than Siri these days I guess 🤷♂️
I'm super excited about this. Google Generative AI's free tier works amazingly well as my HA's Voice Assistant, but the missing step was how to physically get it into the rooms of my house so that I could get rid of my Google Home devices. This looks like it will complete that step. Also, for total offline voice control, you could consider the newly announced Jetson Nano Super, a $250 AI board that can run some of the small LLMs. I haven't looked into the specifics of how that would work yet, but it's a lot of compute for not that much money and worth investigating.
@Ballissle from what I've seen.. radically improved mics.. but the xmos chip inside it is the magic.. makes it *way* better experience... I'm a contributor to the s3-box community firmware... so I've managed to get a decent experience from it.. but this VPE will take HA's VA to a new level... if your primary use case is a desk touch screen though? sure.. the s3-box is also excellent for that.
I love that ending! That was a very fair and realistic review. I think, much as was intimated, this is exciting because it exists at all and can only get better. I hope it is designed such that it can eventually use ollama or an OpenAI API endpoint on a different machine. My homeassistant device is also a small arm7 hame assistant OS device which I'm happy with as it continues running happily when I'm messing aroung with the 'proper' computers. I currently have a Mac Studio that is my main work computer that I let sleep when I'm not using it. It would be ideal to let ollama running on it handle the LLM activities. I'm absolutely not interested in a cloud service.
I've been waiting for something like this for a long time! By the way, you are always the best when it comes to reviews! and everything in smart home subject🔥🔥🔥🔥🔥🔥
For STT and TTS, you can use CUDA accelerated whisper and piper and then use the home assistant Wyoming protocol to send the data around locally or over the internet. This means you can use your Nvidia GPU to do the STT and tts on a server in a Docker container and still run Hass on a pi, just setup Wyoming protocol to connect to the local IP of your server running those containers.
It's kind of surprising no one has created a one-size fits all solution to smart homes. Some platforms are getting close, but not EVERYTHING. First comment, but your chanenel has dramatically helped me with my home over a couple of years now. Appreciate what you do, sir!
Thanks for the nice comment and I'm glad the channel has been helpful! I agree it's kind of surprising one platform can't do it all yet. But at the rate Home Assistant is being developed I think they have the best chance.
Your timing is insane! I did already brought tons of Lenovo Smart Clocks cause my parents really were interested; since they run Android, hopefully there's a way I can transform them into Assist speakers easily 🙂
You can run Piper and whisper on a separate machine and have home assistant connect to it instead of having to upgrade the machine Home assistant is running on or use the cloud. Network. Chuck did a video about how to set that up recently.
I mean, Jarvis wasn't the problem there, there's several movies where Jarvis was very helpful. If you happen to be building a "suit of armor around the world" named Ultron, I would be more concerned.
Honestly, that new Nvidia Jetson 8GB development board could run Piper, Whisper and Llama 3.2. Any bigger model will make it choke though. Nvidia worked with Nabu/HA to port whisper, piper, OpenWakeWord and HA Core to the Jetson so they have custom docker packages. I would leave the GitHub link but I can't obviously.
I just ordered one, I currently have an S3 Box and a couple of M5 Stack atoms, both work 'ok' but have days when they just ignore me. I'm hoping this is a bit more reliable, and as you say its only going to get better. I'm really looking forward to Rhasspy speech next year. All I need now is someone to produce a SBC designed to run local LLM's and just sip power until it's actually required to do some work. I don't really want to have a PC with a GPU running 24/7 just for the occasions when I want to ask a question.
Thanks for bringing this to my attention. I've been try to build my own voice dohickey for home assistant. I do run a separate box for ollama host, with a 3090. There is a lower cost option to run ollama fairly fast on just a PI 5 with one of the AI Hats ($250) from Raspberry folks. I do use my ollama with GPU acceleration for ollama, for whisper, piper, and to even act as the AI attached to Home Assistant (and even have the voice from GladOs on it all). You really need to use the ollama as the AI for home assistant. th-cam.com/video/XvbVePuP7NY/w-d-xo.html cover this well.
Thanks for the review! Definitely agree on the limitations, especially on a RPi 4. Lots of good stuff is planned for next year, and we can only get better :)
Could we please perfect an over-the-air wave power source for these devices so that we can eliminate the wires. You'd think that technology would be here by now.
It's getting there... The Corporate world can't stop "We the People" if we stick together to help each other out on our own! This round is not for me ... YET, but getting closer all the time. I'm against cloud anywhere... That's why we have Home Assistant!
No general user, yes. People that want exposure to the latest technology to provide feedback and tinker with it at absolutely the perfect consumer for stuff like this.
@maxdeathlore if the software isn't finished then it's nowhere near final. You guys are hilarious. Bash every other company for releasing a half baked product, but HA does it and you're swooning ear to ear.
The few Amazon Echos I still have set up are terrified of this thing. You can get one here: home-assistant.io/voice-pe
They don't need to worry. This thing will be great in 2032.
Haha the whole video led up to that ending skit! Had me rolling.
Thanks Jeff! I was just watching your video you published today. Everything you said in your video completely hit home with me between the comments you get and the unused tech all around. We should meet up at CES if you're going!
Loved it!!
@@SmartHomeSolver Sadly, I'm skipping CES this year and heading to NAB with my Dad. But at some point, we shall meet again!
It's Jeff himself!
Basically it comes to this: when it comes to controlling critical stuff in your home (heating, lights, doors,...): it already works. When it comes to less critical stuff (playing media, getting stuff from the internet etc...) not there yet. BUT: this means that you can get your own personal assistent without internet for controlling lots of stuf in your home by voice commands without relying on google, amazon or others... thinking about buying one for use in my own office at home. Knowing the HA community it is just a matter of time and it becomes a competitor to the google/alexa stuff
Just ordered mine and stoked to test it out. I switched from a Pi 4 to a mini PC earlier this year, and everything is much faster!
Awesome! Oh and that's great you already upgraded to a mini PC. I need to do that.
I use a intel nuc and it’s supa fast
@SmartHomeSolver the upgrade to mini pc is very worth it. It also opens up the temptation to add heaps of random servers just because you can.
N100 minis came into my life during the pi shortage. Such great little devices.
They're 8 times more expensive than a pi....of course they're faster
Home Assistant, Home Assistant, HOME ASSISTANT!
So much for the summoning ritual. I'm curious how long it takes for Paul to show up.
He's basically beetle juice in smart home form
HA has a fallback option that lets it go to an LLM (ChatGPT or a local Ollama for instance) should the built-in voice functionality be unable to grasp what you want, perhaps that could help with making voice interpretation more natural. Voice on a Raspberry Pi is kind of a fool's errand, although a Pi 5 has enough horsepower to at least keep delays down to a second or two; you really want beefier gear.
Funny cuz that particular echo didn’t have a display either. Not even the clock display. But… I don’t understand the whole starting up a conversation with you randomly. I am kind of torn. I think a smart home should be in the background just doing its thing. But it should be able to do what you want. Personally I wouldn’t want a smart home randomly suggesting to me what I want… or don’t want. Preferences… maybe. But “hey it’s bright should I shut the shades?” If you wanted them shut wouldn’t you just ask it?
I just migrated from Raspberry Pi 3 to VirtualBox instance on my server PC and it's night and day. Highly recommend
Kinda bummed it doesn't have a POE option. Maybe in the future.
The TTS and STT add-ons can be run on a separate device as Docker containers. I have mine set up to run on my Synology NAS in Synology's Container Manager application. That lets me run HA on dedicated hardware (HA Yellow) but also get more horsepower for the speech add-ons.
damn that's fucking sick. I'm gonna look into this because a completely offline voice assistant is insane.
You should definitely try pairing it with an LLM. There are cloud and local options. It has built in support for ChatGPT (cloud) and Ollama (local). There is also HomeLLM addon that gives you a lot more flexibility in what endpoints to connect to (it supports OpenAI compatible endpoints, so TextGen WebUI, llama-cpp server and such all work). And it allows you to control prompting. You can make your own system prompt template using jinja for it.
It sends all info about your house as context to the LLM, so then the LLM brains kick in and it is able to interpret many fumbly commands, even if TTS misinterprets some words, it can still understand what you meant.
The downside is that it's an LLM so it may hallucinate. But that depends on what LLM you are running.
Running all of this on an RPi is not a good idea, even STT and TTS are all neural networks so RPi cant realistically handle them fast enough. With a decent GPU you can get it down to 1-2 seconds between request and reply, including local LLM.
I've been playing with ESPHome the last few days with a Raspiaudio Muse Luxe speaker, and I agree that it is still quite dodgy. I'm certain that it will get loads better over time.
Could you use to announce alerts, ie. If you had a fridge sensor that said the fridge temp was above 4 degrees C could this device audibly alert you , play an alert tone?
Yes, because of it being also a normal speaker. You could play audio and text-to-speech
So basically you're saying you need to straight up run it through your home hosted GTP so that it understands normal sentences, that's really not a big deal.
Would the Jetson nano super be good for home assistant voice?
I’m pretty sure nothing leaves a Siri HomePod either. That’s a big reason why there’s such a huge variance of how smart each device is (HomePod Mini is dumber than an iPhone 15/16). Also a big reason why Siri seems so dumb, because they don’t have a huge mega server that’s getting smarter with every command.
That last part was funny :D
This has got a lot of potential
Given your optimal use case.. could a stop gap video address updated information for actionable notifications in home assistant. As a new HA user, love the capabilities but struggling with outdated info and guides. Otherwise great video!!!
Honestly this is really cool and I love the fact that it has the option to integrate with things like ChatGPT since that actually means that it can process more complicated commands way easier and could even serve to have a lot more personalisation in the future especially if down the line it was able to implement functions like memories, I can also imagine that other LLMs that can be run locally will come out
Exposing entities is not unique to local voice control. You have to expose the entities to Google / Alexa also to use them. There is an option to automatically expose new entities if you wanted to do that.
To be honest, I find plenty of times Google Home will not understand commands or get them wrong and need me to say them in a particular format. It's a little more refined than the Assistant in Home Assistant, but it's still far from perfect.
You can connect to Gemini for free and it fills in all the gaps for natural language interaction with the smart home. I realize this is not fully local, and actually goes against my philosophy, generally speaking. But in this case, I think it's a worthwhile trade-off until I have the hardware for a local LLM
@1:43 Hilarious. Expect some weird guy to install this as a default response in the next software edition for the HA Voice Preview edition 🤣
The end of the video is FIRE! Loved it.
I wonder how this compares to that local way of doing it with a raspberry pi. The only issue I had was that the speaker wasn't that great. So I was thinking if something like this works fine, I could rip open my echos and steal the speakers and maybe case and it could work the same way with great speakers.
It may be that average users would expect it to understand more than it does out of the box, but you can still make it recognize different commands.
Give entities one or more aliases, and then it will know that you mean that one when you say it that way.
You can do full local if you want... Or you can add chatgpt as an assistant and blow Alexa out of the water.
Is the function of the rotatable ring able to be changed? I'd much rather have physical controls for the dimming of a room light. Also, how are the microphones at picking you up from a distance?
0:43 How did you do that in a video editor? That's crazy that your hand must be perfectly still and perfectly centered!!!
I’d like to know what specs are considered “powerful hardware”, which is a very vague designation for what I might need to run it entirely locally.
I have a Dell Wyse 5070 mini PC that has a pretty low base clock speed but can get some pretty decent boost speed. It’s been running HomeAssistant OS very reliably but I’m not sure how intensive the voice assistant add-on will actually be on it.
Can you run a docker of the llm locally on a more powerful machine? That would be useful given the home assistant yellow is low powered
Hahhahaha, hey babe definitely works sometimes. Hahahahahhahaha.
Heya! thanks for the review. Are there any other similar devices that work well with home assistant? I am looking to get rid of google home devices and place some microphone/speaker set-ups like this for commands only not listening to music or anything.
Do you get more feature support if you enable AI support?
Was that agent cooper from twin peaks i saw? Nice!👍
Now let’s see one for Hubitat.
can you do a video on using unify with home assistant, please
I’ve been working on my own (super basic) version of this thing recently after I realised that the WLED controller my Dad and I designed could work as one (since we recently added a mic and used the ESP32s3). And it totally works… except the speech to text I’m using SUCKS, so it basically never actually does what I want. But no worse than Siri these days I guess 🤷♂️
Home assistant should fire this to the pipeline so your effort is not in vain.
I'm super excited about this. Google Generative AI's free tier works amazingly well as my HA's Voice Assistant, but the missing step was how to physically get it into the rooms of my house so that I could get rid of my Google Home devices. This looks like it will complete that step. Also, for total offline voice control, you could consider the newly announced Jetson Nano Super, a $250 AI board that can run some of the small LLMs. I haven't looked into the specifics of how that would work yet, but it's a lot of compute for not that much money and worth investigating.
Does it work in languages other than English?
Looks like it does, according to their website, but this greatly varies depending on the Language. Mine is fine
This is cool, but 60 dollars is insane
Ordered mine within the first minute... looking forward to it replacing my s3-box.
I was considering the s3-box. How do they compare? I would have thought the s3-box would be better as it also can have a screen
@Ballissle from what I've seen.. radically improved mics.. but the xmos chip inside it is the magic.. makes it *way* better experience... I'm a contributor to the s3-box community firmware... so I've managed to get a decent experience from it.. but this VPE will take HA's VA to a new level... if your primary use case is a desk touch screen though? sure.. the s3-box is also excellent for that.
Hopefully in a year or so it can replace my Alexa devices for Home Assistant
I love that ending! That was a very fair and realistic review. I think, much as was intimated, this is exciting because it exists at all and can only get better. I hope it is designed such that it can eventually use ollama or an OpenAI API endpoint on a different machine. My homeassistant device is also a small arm7 hame assistant OS device which I'm happy with as it continues running happily when I'm messing aroung with the 'proper' computers. I currently have a Mac Studio that is my main work computer that I let sleep when I'm not using it. It would be ideal to let ollama running on it handle the LLM activities. I'm absolutely not interested in a cloud service.
If its anything like getting home assistant running then god help us all
I've been waiting for something like this for a long time! By the way, you are always the best when it comes to reviews! and everything in smart home subject🔥🔥🔥🔥🔥🔥
For STT and TTS, you can use CUDA accelerated whisper and piper and then use the home assistant Wyoming protocol to send the data around locally or over the internet. This means you can use your Nvidia GPU to do the STT and tts on a server in a Docker container and still run Hass on a pi, just setup Wyoming protocol to connect to the local IP of your server running those containers.
It's kind of surprising no one has created a one-size fits all solution to smart homes. Some platforms are getting close, but not EVERYTHING.
First comment, but your chanenel has dramatically helped me with my home over a couple of years now.
Appreciate what you do, sir!
Thanks for the nice comment and I'm glad the channel has been helpful!
I agree it's kind of surprising one platform can't do it all yet. But at the rate Home Assistant is being developed I think they have the best chance.
@@SmartHomeSolver Definetly. I just got a NAS so it was time for the switch from HomeKit. It can be a doozey, but worth it for sure!
Are you sure Siri is not in that thing?
Your timing is insane! I did already brought tons of Lenovo Smart Clocks cause my parents really were interested; since they run Android, hopefully there's a way I can transform them into Assist speakers easily 🙂
That would be sweet! Since all of this is open source it should be possible.
@@SmartHomeSolver Will definitely try doing so in the future!
Hahaha, I also got a bunch of Lenovo devices for the same sort of thing. Now I just need to find the time for some tinkering . . .
I just wanna be able to say hey Jarvis instead of hey google or hey Alexa
Does it work with SmartThings? Lol
Very interested to see where this leads with future revisions.
Great review! I've been eager to see this device come out. Thanks again!
You can run Piper and whisper on a separate machine and have home assistant connect to it instead of having to upgrade the machine Home assistant is running on or use the cloud. Network. Chuck did a video about how to set that up recently.
Open Source Master Race. OSMR, pleasing similar to ASMR
Great review! So excited for the open source voice assistant future!
1:49 the crossover I didn't know I needed
"Who doesn't want a Jarvis in their house?"
Anyone who has seen Age of Ultron?
I mean, Jarvis wasn't the problem there, there's several movies where Jarvis was very helpful. If you happen to be building a "suit of armor around the world" named Ultron, I would be more concerned.
Honestly, that new Nvidia Jetson 8GB development board could run Piper, Whisper and Llama 3.2. Any bigger model will make it choke though. Nvidia worked with Nabu/HA to port whisper, piper, OpenWakeWord and HA Core to the Jetson so they have custom docker packages. I would leave the GitHub link but I can't obviously.
@14:14 - HA should have responded with "No, you have a negative 25 billion dollars"
I just ordered one, I currently have an S3 Box and a couple of M5 Stack atoms, both work 'ok' but have days when they just ignore me.
I'm hoping this is a bit more reliable, and as you say its only going to get better. I'm really looking forward to Rhasspy speech next year.
All I need now is someone to produce a SBC designed to run local LLM's and just sip power until it's actually required to do some work.
I don't really want to have a PC with a GPU running 24/7 just for the occasions when I want to ask a question.
Sounds like another device released before it's ready. Although, I'm looking forward to see the future improvements.
"preview edition". It's almost like you didn't read....🙄🤔
signed and shared
Thanks for bringing this to my attention. I've been try to build my own voice dohickey for home assistant. I do run a separate box for ollama host, with a 3090. There is a lower cost option to run ollama fairly fast on just a PI 5 with one of the AI Hats ($250) from Raspberry folks. I do use my ollama with GPU acceleration for ollama, for whisper, piper, and to even act as the AI attached to Home Assistant (and even have the voice from GladOs on it all). You really need to use the ollama as the AI for home assistant. th-cam.com/video/XvbVePuP7NY/w-d-xo.html cover this well.
Thanks for the review! Definitely agree on the limitations, especially on a RPi 4. Lots of good stuff is planned for next year, and we can only get better :)
Does this also require SSL connection?
Sweet
Could we please perfect an over-the-air wave power source for these devices so that we can eliminate the wires. You'd think that technology would be here by now.
Talking about being CRITICAL!
if is not ready why make a video!
Try a language different than English and it's impossible to use. It simply doesn't understand what your saying.
Actually having it do text to speech to trigger an Amazon Echo sounds like a legitimate workaround to some things.
It's getting there... The Corporate world can't stop "We the People" if we stick together to help each other out on our own! This round is not for me ... YET, but getting closer all the time. I'm against cloud anywhere... That's why we have Home Assistant!
Firet
First
3rd !!!!
forth!!!
Home assistant again 🤮
No one should ever buy preview technology. It's a grift for companies to sell something this early in its design for money.
This is the for early adopters, hence the name.
No general user, yes. People that want exposure to the latest technology to provide feedback and tinker with it at absolutely the perfect consumer for stuff like this.
@sethkazzim731 hahaha, keep telling yourself that
You are paying the company, to do alpha testing, that's all.
In this case the hardware seems close to “final” and they’re not calling it a final product mainly because of the software.
@maxdeathlore if the software isn't finished then it's nowhere near final.
You guys are hilarious. Bash every other company for releasing a half baked product, but HA does it and you're swooning ear to ear.