Would this work on an ESP32 S3 chip board? or perhaps two of them togehter connected via uart? for tts/stt.. ? which version of the pi is the lowest one (cheapest) that would work for this.?
are there any other microphone modules that work with gpio pins with rp4 and other audio amplifiers that can communicate with the pins without having to attach audio cables
It is not easy to acquire audio input via GPIO pins. You probably need to purchase extra components for that. It is a little better for audio output, see here raspberrypi.stackexchange.com/questions/49600/how-to-output-audio-signals-through-gpio. The easiest way is to work with USB speaker and microphone.
@@TechMakerAI sir could you make full tutorial on Open cv because I need use on my project but still I don't know how to use on project and also make tutorial on how to install open cv on raspberry pi it's most important part sir please try to make as soon as possible I'm waiting sir
hey, thanks for the video. I just want to start project exactly the same with yours, mine is kind a companion robot. but the question is, is the service from google just like gemini ai or TTS are completely free? or there's additional cost for that?
@mukagedek4010: The Google Gemini API and Google TTS are currently free to use. The TTS will most likely be free in the future too. However, it is hard to say about the Google Gemini API. Given that the ChatGPT API is not free, Google may charge a fee in the future for Gemini or they may offer a limited version for free. Anyhow, I would say go ahead and build it. There will be some free options available. Even if Google charges a fee, I think it is still worth it. By the way, in this channel, we are also building companion robots. Please subscribe as you will see in future videos.
hey ! already thank you for sharing a file that really works 😀👍 I try to launch your python script at raspberry startup but impossible to start it with the virtual environment. I have an error message "import google.generativeai as genai" except if I enter the line manually everything works do you have any idea how to do it?
Hi, @pixyful7025, I am glad it is working. It is possible to automatically launch this Python program at the startup of RPi. I have a video on this topic coming up very soon. Please subscribe to my channel as you will be notified when it is out.
Yes, it will work with a Bluetooth speaker and without LEDs. I have used the Bluetooth speaker before, see this video th-cam.com/video/nHpJaE559r4/w-d-xo.html. But it requires me to run some commands to pair the Raspberry Pi with the Bluetooth speaker each time. So I switched to wired speaker after that.
I am working on a python jarvis project and i want it to work with a raspberry pi zero 2w because of the small formfactor do you think this is possible?
This is a limitation of the large language model. The dataset used to train the large language model always has a cut-off date and time. You can try to use a newer model. It will have a newer cut-off date and time.
I do not think you can get the live feed from Google Gemini, because the Gemini model was trained by using a data set that was collected before a cut-off date and time. So it does not have the live information we have today. However, you can get live time information from RPi and the weather information from internet. I will add those to a new version of the Python code.
That is a great question. The answer is Yes. Large language models (LLMs) like Gemini are trained on massive data. While you may not be able to train AI from scratch, you have the option to fine-tune the LLMs. You provide additional data (answers) tailored to your desired questions, and the LLMs adapt.
@@TechMakerAI This is my School Project. After A long search I found this video and Thank you that this works. Now Can you help me more please? I want to train the bot that when I ask the bot a question, it answer me such a format that I trained it. How to do that. please help me
Hello....I have a doubt...if you could help... what if i don't use audio amplifier and directly attach a usb speaker....will it work????.....also can this work on raspberry Pi 3B? Also, what is the wake word for it and the word to stop it from listening??
Yes, a USB speaker will work too. I am working on a version of the hardware and will have a video about this topic soon. RPi 3B may work but will be slower due to the small memory size. The wake word is "Jack". To end the conversation with AI, you just need to say "That is all". It will continue to listen but will not respond. Just like Siri, it is always checking to see if you are saying the wake word. If you do not like to see the printed output, just comment all the print() statements.
@@TechMakerAI can you make a video on usb speaker sonn cz I want to make this and present it in an exhibition after 2 weeks....I will be very grateful!
Script crashes from time to time, FYI. Something regarding the chat history and/or communication with the google gemini AI library. I looped the .py script to restart every 2 min as a workaround. Luckily, these sorts of errors are not too common. Also please tell me how to make it stop listening. I tell it "that is all" and it still picks up what I say.
Thank you for your feedback! I am aware of the communication issue with Google Gemini API. It can cause crash sometimes. I have seen less issues with the OpenAI ChatGPT API but that is not free. I made some changes to the code on line 259 in this version. When you say "That is all", it should stop interacting with Gemini API but it is still printing out what you are saying. You can comment out all the print() since you do not need it to print transcript when using RPi (It is there only for debugging purpose) or you can move lines 259 to 263 after line 270 (if "jack" in text.lower() and slang == "en-EN":). Note that this code was written so that users can interact with AI anytime they want. They can wake up AI and put it in sleeping mode. This voice assistant should be on all the time like Siri on your phone. If you stop it from listening, then how do you wake up it next time?
@@TechMakerAI Got it. I have follow up questions. If I do not say "That is all", how many minutes until it times out which will require the wake word? Which line can I modify this? Also, I already have an OpenAI API Key i paid for. How can I switch over to Open AI???
@xpectz: For your first question regarding how many times it takes for it to require the wake word, I think you can add an integer variable to count the times and then set the AI to sleeping mode after it reaches a threshold. For the OpenAI version, please see this video: th-cam.com/video/kIKgl3jDilY/w-d-xo.html
@@TechMakerAI How does it work now? When I stop talking to it, it goes to sleeping mode. What triggers it? I have not timed it. Will check out the video, thanks.
If you add an integer variable to count the times and then set the AI to sleeping mode after it reaches a threshold, then, it will be triggered by your integer. To wake it up, you just need to say the wake word again.
I am a total noob with a 2nd had pi4. i followed all your steps but encountered the following after pip install -q -U google-generativeai: WARNING: Retrying (Retry(total=4, connect=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', remoteDisconnected( 'Remote end closed connection without response'))': /simple/protobuf/ Does this mean the loading of the AI failed? How do i rectify this, please?
It means that you did not install the Google Generative AI package successfully. I would recommend that you try it again. If it fails, you can always format the micro SD card and start from the beginning.
Good video and explained well! Will try with my Pi 4 B.
I've watched almost all of your videos, great work! Please make more useful videos.
Thank you, will do!
Would this work on an ESP32 S3 chip board? or perhaps two of them togehter connected via uart? for tts/stt.. ? which version of the pi is the lowest one (cheapest) that would work for this.?
Any decent touch displays that can work with rp4 that communicate with gpio pins for animations?
are there any other microphone modules that work with gpio pins with rp4 and other audio amplifiers that can communicate with the pins without having to attach audio cables
It is not easy to acquire audio input via GPIO pins. You probably need to purchase extra components for that. It is a little better for audio output, see here raspberrypi.stackexchange.com/questions/49600/how-to-output-audio-signals-through-gpio. The easiest way is to work with USB speaker and microphone.
Sir please make more tutorials on raspberry pi and Arduino
Sure, I have new videos about Raspberry Pi, Arduino, and STM32 coming up soon.
@@TechMakerAI sir could you make full tutorial on Open cv because I need use on my project but still I don't know how to use on project and also make tutorial on how to install open cv on raspberry pi it's most important part sir please try to make as soon as possible I'm waiting sir
hey, thanks for the video. I just want to start project exactly the same with yours, mine is kind a companion robot. but the question is, is the service from google just like gemini ai or TTS are completely free? or there's additional cost for that?
@mukagedek4010: The Google Gemini API and Google TTS are currently free to use. The TTS will most likely be free in the future too. However, it is hard to say about the Google Gemini API. Given that the ChatGPT API is not free, Google may charge a fee in the future for Gemini or they may offer a limited version for free. Anyhow, I would say go ahead and build it. There will be some free options available. Even if Google charges a fee, I think it is still worth it. By the way, in this channel, we are also building companion robots. Please subscribe as you will see in future videos.
hey !
already thank you for sharing a file that really works 😀👍
I try to launch your python script at raspberry startup but impossible to start it with the virtual environment. I have an error message "import google.generativeai as genai" except if I enter the line manually everything works do you have any idea how to do it?
Hi, @pixyful7025, I am glad it is working. It is possible to automatically launch this Python program at the startup of RPi. I have a video on this topic coming up very soon. Please subscribe to my channel as you will be notified when it is out.
@@TechMakerAI I found a little thing that seems to work by tinkering. I wrote it in .bashrc and told the raspberry to log in automatically
@@TechMakerAI I will wait for your video to compare the most optimized 😉
can you also make a tutorial to work with a bluetooth speaker and without the leds.
or wouldnt it work that way?
Yes, it will work with a Bluetooth speaker and without LEDs. I have used the Bluetooth speaker before, see this video th-cam.com/video/nHpJaE559r4/w-d-xo.html. But it requires me to run some commands to pair the Raspberry Pi with the Bluetooth speaker each time. So I switched to wired speaker after that.
Is sound amplifier really needed?
It is needed if you use a passive speaker like in the video. However, if you use a USB speaker, then it is not needed.
I am working on a python jarvis project and i want it to work with a raspberry pi zero 2w because of the small formfactor do you think this is possible?
It may work, but you will experience a delay in the response
@@TechMakerAI yeah thats what i thought i will probably Switch to a better Raspberry pi
@@TechMakerAI thank you though
@Leith79: it will be a good idea if you switch to a better RPi. But if you manage to make it work on Pi zero 2 W, please let us know too.
Sir please guide me how to get the updated data after the cut off date and time.
This is a limitation of the large language model. The dataset used to train the large language model always has a cut-off date and time. You can try to use a newer model. It will have a newer cut-off date and time.
Hi sir ,
How to get live feed like time , weather etc in gemini ai
I do not think you can get the live feed from Google Gemini, because the Gemini model was trained by using a data set that was collected before a cut-off date and time. So it does not have the live information we have today. However, you can get live time information from RPi and the weather information from internet. I will add those to a new version of the Python code.
How many minute until it stops listening? Is there a way to adjust it? Or to tell it to stop listening?
Please see my reply to your newer question.
Hi sir , how to change raspberry pi wifi while ssh
Please check out this video: th-cam.com/video/KGtC7A8kONY/w-d-xo.html
Can I train the ai to produce me a particular types of answer as we can train chatgpt ?
That is a great question. The answer is Yes. Large language models (LLMs) like Gemini are trained on massive data. While you may not be able to train AI from scratch, you have the option to fine-tune the LLMs. You provide additional data (answers) tailored to your desired questions, and the LLMs adapt.
@@TechMakerAI This is my School Project. After A long search I found this video and Thank you that this works. Now Can you help me more please? I want to train the bot that when I ask the bot a question, it answer me such a format that I trained it. How to do that. please help me
It sounds like you want to fine turn the LLM. Please have a look at this article: www.datacamp.com/tutorial/fine-tuning-large-language-models
Hello....I have a doubt...if you could help... what if i don't use audio amplifier and directly attach a usb speaker....will it work????.....also can this work on raspberry Pi 3B?
Also, what is the wake word for it and the word to stop it from listening??
Yes, a USB speaker will work too. I am working on a version of the hardware and will have a video about this topic soon. RPi 3B may work but will be slower due to the small memory size. The wake word is "Jack". To end the conversation with AI, you just need to say "That is all". It will continue to listen but will not respond. Just like Siri, it is always checking to see if you are saying the wake word. If you do not like to see the printed output, just comment all the print() statements.
@@TechMakerAI can you help me change the wake work if you can?
@@TechMakerAI can you make a video on usb speaker sonn cz I want to make this and present it in an exhibition after 2 weeks....I will be very grateful!
Just do a search and replace of all the "jack" with any name your like.
@m9playz239: You can use the current version of the code with a USB speaker. It should work too.
Script crashes from time to time, FYI. Something regarding the chat history and/or communication with the google gemini AI library. I looped the .py script to restart every 2 min as a workaround. Luckily, these sorts of errors are not too common.
Also please tell me how to make it stop listening. I tell it "that is all" and it still picks up what I say.
Thank you for your feedback! I am aware of the communication issue with Google Gemini API. It can cause crash sometimes. I have seen less issues with the OpenAI ChatGPT API but that is not free. I made some changes to the code on line 259 in this version. When you say "That is all", it should stop interacting with Gemini API but it is still printing out what you are saying. You can comment out all the print() since you do not need it to print transcript when using RPi (It is there only for debugging purpose) or you can move lines 259 to 263 after line 270 (if "jack" in text.lower() and slang == "en-EN":). Note that this code was written so that users can interact with AI anytime they want. They can wake up AI and put it in sleeping mode. This voice assistant should be on all the time like Siri on your phone. If you stop it from listening, then how do you wake up it next time?
@@TechMakerAI Got it. I have follow up questions. If I do not say "That is all", how many minutes until it times out which will require the wake word? Which line can I modify this?
Also, I already have an OpenAI API Key i paid for. How can I switch over to Open AI???
@xpectz: For your first question regarding how many times it takes for it to require the wake word, I think you can add an integer variable to count the times and then set the AI to sleeping mode after it reaches a threshold. For the OpenAI version, please see this video: th-cam.com/video/kIKgl3jDilY/w-d-xo.html
@@TechMakerAI How does it work now? When I stop talking to it, it goes to sleeping mode. What triggers it? I have not timed it. Will check out the video, thanks.
If you add an integer variable to count the times and then set the AI to sleeping mode after it reaches a threshold, then, it will be triggered by your integer. To wake it up, you just need to say the wake word again.
I am a total noob with a 2nd had pi4. i followed all your steps but encountered the following after pip install -q -U google-generativeai:
WARNING: Retrying (Retry(total=4, connect=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', remoteDisconnected( 'Remote end closed connection without response'))': /simple/protobuf/
Does this mean the loading of the AI failed? How do i rectify this, please?
It means that you did not install the Google Generative AI package successfully. I would recommend that you try it again. If it fails, you can always format the micro SD card and start from the beginning.
@@TechMakerAI Thank you. I'll dp that and get the hardwares so I can do the voice test, like you did in your earlier video.
Yes, that is a good idea.