00:00 AI and Microcontrollers 00:52 Welcome and Introduction 01:07 Rules and AI 01:31 AI: Rule Finding vs. Rule Creation 05:26 AI vs. Conventional Programming 06:30 The Future of Programming 08:43 AI-Supported Process Optimization 11:20 AI Agents and Hybrid Approaches 13:41 The Cat Water Fountain Project 18:00 Summary and Outlook These chapters are created entirely locally (with AI) on my Linux Debian PC (transcript with Whisper-Turbo & chapter with Ollama) Quality and accuracy may vary. Use in the video description is permitted.
Thank you! Preventing entering cats with mice is a really good improvement! Currently I am using AI to support programming in Python, JavaScript and XLS-tricks. Very handy.
So far, I did not test this one. I first want to get experiecne with the fountain. And indeed, because I sometimes need an Excel macro, I also ask ChatGPT. It is much faster than me...
As in your home, the most important task for an AI system would be a cat flap that blocks cats with their catch, like birds or mice. BTW, I heard a couple of times before from some Swiss guys that they had already invented this system. Never know that if it was offered to buy. Anyway, as a fish tank owner we didn't need a drinking well because all of our cats prefer self-service water with fish-taste ;-)
Such an add-on is offered for the cat flap. But the salesman did not want to sell us one because of bad experience. It is expensive and seems not to work. Maybe with a powerful engine like ChatGPT we could change that...
I solved the fountain problem with a 24Ghz LD2410C radar sensor. As they measure detection and distance. So the distance sensor is the only thing I need to turn it on or off. This also works offline.
Good point! I did not try it, but fear that our kitchen is so small that the distances would not be a sufficient discriminator. But for sure much better than the current "general purpose" PIR.
I was going to do a cat doorbell like that, but RF used too much power (couldn't wire the sensors in.) I put cheap BlueTooth trackers on the cats, and at the door put an IR sensor, which when triggered woke up the microcontroller (AtTiny) that started listening for BLE. If it got a match it sent a 433MHz signal to the base station that beeped & lit up as to what door the cat was at. Was going to use ESP32 but didn't have any, so random modules I had on hand was it. Detector boxes last about 3 months on a charge. Base station shows battery alert. Lipo battery, USB charged. Works well! Primary user is an 80 year-old woman, she thinks it's great. The cats do as well.
Very good ! Your title may scare some off, but this is very good summary of some interesting thoughts on LLM, in general. We do not agree about SLM though - they can be very good indeed
I think the newer ESP S3 cam has a built in model for recognizing cats. Anyways. I am experimenting with Frigate in Home assistant and amazed how can the model recognize so many objects, which are not there in reality. Especially Zebras and Giraffes 😂😂😂
Apologies if you've already done a video on this, but could the Raspberry Pi AI camera not do this locally? Just curious about your views on that device. Love your videos, Andreas!
That's a really nice idea and surprised me. However, how fast is this whole system? Is there a noticeable delay when the cat approaches the fountain? What if the camera is triggered by a human first but the cat comes a second later? I think there are a lot of potential problems with this.
1. It takes maybe 2 seconds to trigger. However, the PIR triggers before the cat is at the fountain. It only takes one picture during one session. If no cat there is no water fountain :-(
Thx, this kind of videos is helping a lot to estimate what AI can do and what not. I am using it for image creation. I made the experience that AI is not at all understanding content or context. But sometimes it is helpful to find an approach. In the end I always need a lot of my own modest natural intelligence.
Happy New Year, Andreaas, excellent approach, I am not too into AI in my hobby developments, but I have an ESP32 CAM and a Cat, I think is about time to give it a try, I recently attended a course at the local university about Python and Natural Language Interpretation, which I believe is very important to learn "prompting". Thank You.
You can build your own AI code with tensorflow in Phyton working the picture in some one board pc, that a nice project. I tried to do that in C direct in a microcontroller, I broke several fingers and a arm, perhaps next time, lol
Thanks. Hailo 8 would be good here - but of course the price and not microcontroller. But Espresiff have released an ESP OPencv for ESP - so can we do our detection locally and save a few cents? (You only mentioned TensorFlow). As things grow and we start analyzing if the wife is in the kitchen and wants a cup of tea to turn the kettle on, if its a bill (to be ignored) or a postcard (to be collected) in the mail box, the mother in law at the front door, or something unwanted in the garden and its sprinkler time - the AI requests to C-GPT will grow.
The idea is cool and innovative, but what about the environmental impact ? Prompting a huge multimodal AI running in huge data centers every so often, to try and alleviate the power consumption of a small pump... It would have been a lot more interesting to try and run a simple AI model trained at recognizing cats directly on the microcontroller (and would have been a lot more environmentally responsible in my opinion). Maybe a follow-up video can be made on edge computing ?
I agree that the whole project is not a good idea. I should train the cat to drink from still water ;-) But I will not try it with local AI because then, I have to run the ESP32-Cam all the time. And the experiments I did with such AI systems were less than encurraging :-( Not to mention the effort for training.
If the API call costs 10c and I spend an hour in the kitchen cooking Sunday lunch and trigger the PIR every time I move, does this not work out quite expensive for all the times the AI answers "no" cat?
With the resolution I use on the ESP32-CAM it costs much less than one cent. But I have no longtime experience. Still, AI hardware is quite expensive. I experimented with Frigate on my HA server and the fan ran all the time. CPU usage was quite high just for monitoring the camera.
@@AndreasSpiessIt was only a thought experiment, but thanks for the reply. Optimisation of PIR location, or alternative use of a RADAR sensor might also optimise the set up. I also wondered if AI could determine if the cat looked thirsty before turning on the fountain, but perhaps AI is not ready for that yet.....
Insightful video. What should someone starting out Embedded Systems as Intern, possibly should focus on as the "Experience" that is valuable and stands still for the future?
As shown in the video, writing specifications (prompting) will be very important in the future. However, it is not easy without understanding one layer below. So I recommend to search for a problem you want to solve and start with a project. I learn nearly everything with a project (as this video shows). BTW: Many things I learned were not immediately useful. But sometime in the future...
What about a simple YOLO v3 classification? Is a ESP32 not fast enough? I wouldn't like my kitty depends on Internet connection and chatGPT. Bless Zheng
My experiments with slamm scale ML was not encurraging. And I do not want to run the ESP 24/7. The easierst to solve your problem would be: If no anwer from ChatGPT means cat detected, BTW ;-)
Cool projet 👍 have you considered doing the inference directly in the microcontroller? ST has a new kit with camera and STM32N6 that should be perfect (with their NPU to accelerate the AI model) This way you don't need connectivity or external API: the fountain itself should recognize your cat 😊
My experience with such small scale AI was not good in the past. Plus I would have to run the ESP32 24/7. I have no problems with external APIs. My live depends in many situations on such services (like google maps while driving or credit card services for paying)...
simple rpi or rpi coral should work also, maybe there is some small ofline model on rpi, frigate also should work on one camera that do basically nothing all day
You are right. I would put it in the "expensive" department. Before this project, I tried the same with Frigate (without coral) and the fan of my HA server ran 100%. The CPU usage was very high. That is why I changed to the aproach presented here.
@@AndreasSpiess i was hoping that frigate would only work for recognition only after pir detect object not all the time as usual silly but in my mind even esp32 could takie cat ai task, everything is triggered by pir not in constant recognition mode like 30fps ;)
The ability to create rules not only requires freedom, it requires private property. The owner of a business can create whatever set of rules for his property and enforce them. He can demand visitors take their shoes off, for example. But he can't tell anyone to remove their shoes outside of his property. Law (i.e. objective, natural law) is universal and applies to all actors, but it is not created. It is deduced from observing reality and the human condition. It also requires private property, in order to avoid conflicts over the use of scarce resources. Then there is "positive law", a contradictory set of rules created by the state, which necessarily violates private property and natural law.
First ai replace the programmer and the manager is happy. Then the AI replaces the manager and the director is happy. Then the ai replaces the director and sam altman is happy. Would would be happier working all together but sam altman might be less happy.
I made a gpt that worked like a template for my lilygo t display S3. Tailoring it to already have all my details for the TFT, built in buttons etc. I call it AI wrangling. It's not the first answer but iterate .. evaluate.. repeat
@@AndreasSpiess it is usually written as: "why should I trust your definition, if you can't pronounce 'inference' properly. Let's assume the poster meant it as a joke (which it usually is), though missing the point of "the guy with the Swiss accent" :)
00:00 AI and Microcontrollers
00:52 Welcome and Introduction
01:07 Rules and AI
01:31 AI: Rule Finding vs. Rule Creation
05:26 AI vs. Conventional Programming
06:30 The Future of Programming
08:43 AI-Supported Process Optimization
11:20 AI Agents and Hybrid Approaches
13:41 The Cat Water Fountain Project
18:00 Summary and Outlook
These chapters are created entirely locally (with AI) on my Linux Debian PC (transcript with Whisper-Turbo & chapter with Ollama)
Quality and accuracy may vary. Use in the video description is permitted.
Thank you! Preventing entering cats with mice is a really good improvement! Currently I am using AI to support programming in Python, JavaScript and XLS-tricks. Very handy.
So far, I did not test this one. I first want to get experiecne with the fountain.
And indeed, because I sometimes need an Excel macro, I also ask ChatGPT. It is much faster than me...
As in your home, the most important task for an AI system would be a cat flap that blocks cats with their catch, like birds or mice. BTW, I heard a couple of times before from some Swiss guys that they had already invented this system. Never know that if it was offered to buy. Anyway, as a fish tank owner we didn't need a drinking well because all of our cats prefer self-service water with fish-taste ;-)
Such an add-on is offered for the cat flap. But the salesman did not want to sell us one because of bad experience. It is expensive and seems not to work. Maybe with a powerful engine like ChatGPT we could change that...
I solved the fountain problem with a 24Ghz LD2410C radar sensor. As they measure detection and distance. So the distance sensor is the only thing I need to turn it on or off. This also works offline.
Good point! I did not try it, but fear that our kitchen is so small that the distances would not be a sufficient discriminator. But for sure much better than the current "general purpose" PIR.
I was going to do a cat doorbell like that, but RF used too much power (couldn't wire the sensors in.)
I put cheap BlueTooth trackers on the cats, and at the door put an IR sensor, which when triggered woke up the microcontroller (AtTiny) that started listening for BLE. If it got a match it sent a 433MHz signal to the base station that beeped & lit up as to what door the cat was at.
Was going to use ESP32 but didn't have any, so random modules I had on hand was it. Detector boxes last about 3 months on a charge. Base station shows battery alert. Lipo battery, USB charged.
Works well! Primary user is an 80 year-old woman, she thinks it's great. The cats do as well.
Remarkable video. Awesome and inspiring...
Happy New Year btw, just in case I forgot it until now.
Very good ! Your title may scare some off, but this is very good summary of some interesting thoughts on LLM, in general. We do not agree about SLM though - they can be very good indeed
I think the newer ESP S3 cam has a built in model for recognizing cats.
Anyways. I am experimenting with Frigate in Home assistant and amazed how can the model recognize so many objects, which are not there in reality. Especially Zebras and Giraffes 😂😂😂
I like you, your projects and...your cats! Thank you for your effort for good presentations!
Thank you. You are welcome!
Apologies if you've already done a video on this, but could the Raspberry Pi AI camera not do this locally? Just curious about your views on that device. Love your videos, Andreas!
That's a really nice idea and surprised me. However, how fast is this whole system? Is there a noticeable delay when the cat approaches the fountain? What if the camera is triggered by a human first but the cat comes a second later? I think there are a lot of potential problems with this.
1. It takes maybe 2 seconds to trigger. However, the PIR triggers before the cat is at the fountain. It only takes one picture during one session. If no cat there is no water fountain :-(
Thx, this kind of videos is helping a lot to estimate what AI can do and what not. I am using it for image creation. I made the experience that AI is not at all understanding content or context. But sometimes it is helpful to find an approach. In the end I always need a lot of my own modest natural intelligence.
“If the only tool you have is a hammer, you tend to see every problem as a nail.”😉
What happens if your cat sleeps in front of the water fountain...? Would it keep calling the API frequently?
Happy New Year, Andreaas, excellent approach, I am not too into AI in my hobby developments, but I have an ESP32 CAM and a Cat, I think is about time to give it a try, I recently attended a course at the local university about Python and Natural Language Interpretation, which I believe is very important to learn "prompting". Thank You.
As you write, learning how to deal with AI is important. Even if we decide it is nothing for us.
You can build your own AI code with tensorflow in Phyton working the picture in some one board pc, that a nice project.
I tried to do that in C direct in a microcontroller, I broke several fingers and a arm, perhaps next time, lol
Thanks.
Hailo 8 would be good here - but of course the price and not microcontroller.
But Espresiff have released an ESP OPencv for ESP - so can we do our detection locally and save a few cents? (You only mentioned TensorFlow).
As things grow and we start analyzing if the wife is in the kitchen and wants a cup of tea to turn the kettle on, if its a bill (to be ignored) or a postcard (to be collected) in the mail box, the mother in law at the front door, or something unwanted in the garden and its sprinkler time - the AI requests to C-GPT will grow.
respect for getting cat a fountain
I am not sure if this is a positive or negative comment ;-)
@AndreasSpiess flowing water is generally less prone to slime growth than standing water
The idea is cool and innovative, but what about the environmental impact ?
Prompting a huge multimodal AI running in huge data centers every so often, to try and alleviate the power consumption of a small pump...
It would have been a lot more interesting to try and run a simple AI model trained at recognizing cats directly on the microcontroller (and would have been a lot more environmentally responsible in my opinion).
Maybe a follow-up video can be made on edge computing ?
I agree that the whole project is not a good idea. I should train the cat to drink from still water ;-)
But I will not try it with local AI because then, I have to run the ESP32-Cam all the time. And the experiments I did with such AI systems were less than encurraging :-( Not to mention the effort for training.
Brilliant as usual! Thank you!
You are welcome!
If the API call costs 10c and I spend an hour in the kitchen cooking Sunday lunch and trigger the PIR every time I move, does this not work out quite expensive for all the times the AI answers "no" cat?
With the resolution I use on the ESP32-CAM it costs much less than one cent. But I have no longtime experience. Still, AI hardware is quite expensive. I experimented with Frigate on my HA server and the fan ran all the time. CPU usage was quite high just for monitoring the camera.
@@AndreasSpiessIt was only a thought experiment, but thanks for the reply. Optimisation of PIR location, or alternative use of a RADAR sensor might also optimise the set up. I also wondered if AI could determine if the cat looked thirsty before turning on the fountain, but perhaps AI is not ready for that yet.....
Wonderful video, thanks Mr. Spiess
Glad you enjoyed it!
Is it legal to upload a picture of the mail person to chatgpt without their agreement?
As always: learned something, had much fun and "it was for the cat" 😂
:-))
You inspire me to learn more about the way of the electron. May the electromotive force be with you.
That is a good idea!
Insightful video. What should someone starting out Embedded Systems as Intern, possibly should focus on as the "Experience" that is valuable and stands still for the future?
As shown in the video, writing specifications (prompting) will be very important in the future. However, it is not easy without understanding one layer below. So I recommend to search for a problem you want to solve and start with a project. I learn nearly everything with a project (as this video shows).
BTW: Many things I learned were not immediately useful. But sometime in the future...
What about a simple YOLO v3 classification? Is a ESP32 not fast enough?
I wouldn't like my kitty depends on Internet connection and chatGPT.
Bless
Zheng
My experiments with slamm scale ML was not encurraging. And I do not want to run the ESP 24/7. The easierst to solve your problem would be: If no anwer from ChatGPT means cat detected, BTW ;-)
@@AndreasSpiessYes, simplest failsafe! ...but you don't need 7/24 if you wake up from deep sleep on pir. By the way: why not ESP 7/24?
Cool projet 👍
have you considered doing the inference directly in the microcontroller?
ST has a new kit with camera and STM32N6 that should be perfect (with their NPU to accelerate the AI model)
This way you don't need connectivity or external API: the fountain itself should recognize your cat 😊
My experience with such small scale AI was not good in the past. Plus I would have to run the ESP32 24/7. I have no problems with external APIs. My live depends in many situations on such services (like google maps while driving or credit card services for paying)...
Lol - AI suggests using AI.
That is how addiction works ;-)
A.S suggest A.I
Andreas S.
A Raspberry Pi with Camera would work as well! Only have it catch the Cat.
Maybe, but probably more expensive, even on the long run (consumes power 24/7).
@@AndreasSpiess - Understood!
simple rpi or rpi coral should work also, maybe there is some small ofline model on rpi, frigate also should work on one camera that do basically nothing all day
You are right. I would put it in the "expensive" department. Before this project, I tried the same with Frigate (without coral) and the fan of my HA server ran 100%. The CPU usage was very high. That is why I changed to the aproach presented here.
@@AndreasSpiess i was hoping that frigate would only work for recognition only after pir detect object not all the time as usual
silly but in my mind even esp32 could takie cat ai task, everything is triggered by pir not in constant recognition mode like 30fps ;)
Is Dishka (?) still there too?
No, she passed away last year. She was already old :-(
😢
Bizarre,. I was dreaming this morning about asking my 'phone' to convert my speech to morse code by flashing the camera light.
Morse should be very easy for an AI. However, I never tried.
The ability to create rules not only requires freedom, it requires private property. The owner of a business can create whatever set of rules for his property and enforce them. He can demand visitors take their shoes off, for example. But he can't tell anyone to remove their shoes outside of his property.
Law (i.e. objective, natural law) is universal and applies to all actors, but it is not created. It is deduced from observing reality and the human condition. It also requires private property, in order to avoid conflicts over the use of scarce resources.
Then there is "positive law", a contradictory set of rules created by the state, which necessarily violates private property and natural law.
I would've just used a button or a pressure mat.
AI is perfect for the average human as the average human is lazy and is constantly looking to find "short cuts" in order to use less effort.
if the average mean 90% of humanity, then your spot-on ;)
This seems to be "built-in" if we believe the theory presented in "Thinking, Fast and Slow" by Daniel Kahneman
"AI is not intelligent"
source: "trust me bro"
Lol. Für die katze
Too bad the ESP-CAM has garbage photo quality, we need a cheap solution for better quality photos
It seems to be sufficient for such an application. And I assume, ChatGPT consumes less tokens with smaller resolution (less data).
rpi with pi camera
great video
just small thing @14:23 : need a WIFE for crazy ideas
:-)) This time, it was mine. To get the fountain was heavily supported by her (Smokey knows how to deal with her)
First ai replace the programmer and the manager is happy. Then the AI replaces the manager and the director is happy. Then the ai replaces the director and sam altman is happy. Would would be happier working all together but sam altman might be less happy.
As I tried to show in the video, I do not fear that humans will be replaced. Just reskilled as happened already many times in the last 150 years.
Wow! 127 views in 2min. You must be doing something correct.
nothing special because my videos are timed to 9AM Swiss time...
❤❤❤❤❤❤🎉
😴😴😴
I made a gpt that worked like a template for my lilygo t display S3. Tailoring it to already have all my details for the TFT, built in buttons etc.
I call it AI wrangling. It's not the first answer but iterate .. evaluate.. repeat
Cool! Iteration is a very good way of learning.
You can't pronounce 'inference', but I should trust your definition of it?
I do not understand :-(
@@AndreasSpiess it is usually written as: "why should I trust your definition, if you can't pronounce 'inference' properly. Let's assume the poster meant it as a joke (which it usually is), though missing the point of "the guy with the Swiss accent" :)