ESP32-CAM: Simple AI Robot (Object Detection | Object Tracking | Lane Tracking)
ฝัง
- เผยแพร่เมื่อ 22 พ.ค. 2020
- In this project, I would like to share with you how to make a simple AI Robot with ESP32-CAM module.
The robot can do the Object Detection, Object Tracking, Lane Autonomous Tracking and remote controlled by the Android Phone. The application would acquire the ESP32 address via a broadcast UDP package (255.255.255.255) , so we don't need to configure the address of the module, we just connect it to the same wifi network with the mobile.
ESP32-CAM firmware and Android source: github.com/longpth/ESP32CamAI
Android Application: play.google.com/store/apps/de...
More detail: www.hackster.io/dragonph/esp3...
The Android Application is tested and run smoothly on Android Phone with specifiation: Android 10, Snapdragon 730, Ram 6GB, 6.39 inches, 1080 x 2340 pixels.
So for best performace, please use the similar or higher specifiation.
Components:
1. ESP32-CAM module x 1
2. L298N Motor Driver module x 1
3. 1:120 Gear Motor x 2
4. wheel x 2
5. ball transfer unit x 1
6. Cardboard
7. Icream stick
8. Portable power bank
Other projects:
Raspberry Pi and Arduino: Logo Sorting System
• Raspberry Pi and Ardui...
ESP32-CAM: Remote Control Object Detection Camera
• ESP32-CAM: Remote Cont...
ESP32-CAM: Classification with tensorflow.js
• ESP32-CAM: Object clas...
Arduino and Android: Breath Alcohol Tester
• Alcohol Tester with Ar...
Arduino and Android: Object Tracking Camera
• Simple Object Tracking...
ESP8266 Arduino Project: Electric Lock and Unlocked by Android Fingerprint
• ESP8266 Arduino Projec...
#esp32camrobot #esp32computervision #esp32AI
Music and sound effect: TH-cam Audio Library.
****************************************************
If you like, don't forget to subscribe and share for more projects: / @roboticslearner
**************************************************** - แนวปฏิบัติและการใช้ชีวิต
![[Full Episode] MasterChef Junior Thailand มาสเตอร์เชฟ จูเนียร์ ประเทศไทย Season 3 Episode 3](http://i.ytimg.com/vi/A1XXYbV048k/mqdefault.jpg)
![เก่งแต่ลับหลัง (FINSTA) - ส้ม มารี (Zom Marie) Ft. Punch 4EVE [Official MV]](http://i.ytimg.com/vi/OOolBU2aC70/mqdefault.jpg)
I configured my board as Tools/Board/ESP32 Wrover Module, and define CAMERA_MODEL_AI_THINKER for the pins definition. Because there are some different kinds of ESP32-CAM modules, so please check your board and your pins definition respectively.
I clarified my module by trying to run the ESP32 example CameraWebserver first, then keep the setting for my other projects, but it maybe different from yours.
Regarding my code, to make sure the camera module is initialized fine, please enable DEBUG macro in ESP32CamAI ino file, then use Serial Monitor to check if this message is output: "camera init state 0".
Why when i try connect to ip, i cannot display the webcam? Respon please, because this is a really good project👍
@@elektroasik I have published the android source code, I think you should try debugging to figure out your issue, maybe there is some problem with the connection
@@roboticslearner Where you published the android source code? Please give me a URL
@@satriamuhammadazis6786 You can find it here: github.com/longpth/ESP32CamAI
@@roboticslearner what the software to make that android application? Android studi or else?
very very impressive .
seems easy to people , but object tracking takes a lot of time in training the machine to distinguish and come up with a sure results.
Wonderful job you've done there
This is a brilliant project. Thank you. It took me a while to figure it out, but when it worked it was beautiful. It's a complex code thanks for sharing it.
hi sir I am facing problem
esp is not connecting to phone 🥲
help me out pls
Thank you very much for sharing such a great project, it is magnificent. Congratulations.
Thanks for posting this. Keep up your amazing work 😁
Hello engineer, your project is great, thank you
I love this, Thankyou so much for sharing
Your projects are so much fun...
thank you. (Samsung galaxy tab A, works great!)
Great project!, which software did you use for the android app? I see the android source files in java in your github but I'd like to know where did you program it? Thank you in advance
Great project. In hackster tutorial you mentioned that Android phone can also control the ESP32-CAM GPIOs via UDP package. How to do this? simple code example will help a lot.
Thanks!
plzz make a video in which you will make the complete project step by step. I love this project and want to make it.
How to connect the app to eps32 can board?? the serial monitor showing "camera init state 0" but app showing "cannot connect to esp32 camera"
please help
Nice info, thanks for sharing it, keep it up :)
Do we have to put short pin on Enable A & B because my robot is not moving at all, rest all computer vision stuff is working well, what to do?? Please reply
Hello sir. The project you made was wonderful. Can you please somehow indicate on how would we process the source code? Like should all the source code you've given will be written or just a some of it? Thank you so much, sir!
Hi! Your project is awesome. But I have a question. Is it applicable to use L293D motor driver instead of L298N?
Hi Jayvelou, I think there is no problem.
sir can you please share me that which version of esp 32 you will use in these Project like esp32 Ai thinker or only esp32
i would like to know who is performing the "detection" operation . Is it the esp32 or esp32 is simply streaming the video to the phone and then phone is doing all the heavy lifting like the detection and recognition stuff .
Yes, esp32 is used for video streaming and actuator controlling, AI task is executed by phone
Hello there sir. Can you please explain how the code works and what all processes are going in the Esp32cam so that ot can detect objects?
Could the camera be used for an edge detection robot?
can i use the blue plastic servo instead the metal one
Can you please tell ke what should o do after doing all connections..and then what to do?
Hi there! I am interested in this kind of projects. I am planning to make an object tracker robot using Arduino UNO, ESP32 camera module and a bunch of servomotors for the movement of the robot. Can you help me for the code for the camera in order to detect and track the object? I would really appreciate if you can help. Thank you very much!
I always thought that it might be interesting to look into building a robot like this.
I see that your robot has a vision based line following ability and it can track objects and identify them but can it do anything else with that info?
Thanks for conment, I am thinking how to improve it
@@roboticslearner
It reminds me of the Cosmo and Vector robots.
Can it read bar codes and execute commands based on those barcodes?
Do you have any plans to add any other functions to this robot?
With a project like this have you reached your current range of knowledge in this are?
Do you have a continued interest in this area of exploration as a hobby?
What do you think of the Misty educational robot and other robots like the kuri robot and the Qbo original?
Have you ever played around with any off the shelf robots like a; Darwin, Nao, Cosmo, or a Vector?
How many years of programming do you have? Is it just hobby or college too?
What future robot projects do you have?
Do you find the current state of hobby robotics to be dwindling, stagnating, or plateauing and not advancing?
Wow, your questions encourage me a lots. Yeah sure I want to improve it more ... I am thinking of making it an easy learning stem robot too ...
Hi, I really like your job! Could you recomend some good Object Tracking Tutorial/Guide?
Hlw sir your project is great project.
Sir Whether to work with Direct App after uploading the program
Nice job, sir. Would it be possible to make the wheels turn right or left to follow the object? Also make the car go forward or backwards if the object gets too far or too close? Thank you.
Yeah, actually yes but because the number of esp32 cam gpios are limited
@@roboticslearner whats that mean?
Sir, if I want to replace it with my own target detection model, how should I modify the code?
The android app is not supported in Android 11..Can you please make it for Android 11 and higher..??
I am adding the zip file esp 32 cam ai in sketch but nothing is coming and same is happening when adding Arduino web socket zip file in sketch. Please help
I’m trying to build something similar to this to sort brass by caliber by using a camera to read a headstamp where can I go for more info to learn how to build what I’m wanting?
Try some ml model with python first
Can we connect with only browser because the app is lagging and app is difficult to connect
Is sd card needed for esp32cam in this project?
Its working dude thank you bro
'vector' in namespace 'std' does not name a template type
I got this error help me bro
esp32 not connected to phone help me bro
Redump the program bro
@@abhijithabhi6598 bro 😅
I have edited the code
I got an error
Vector in namespace std does not name a template type
So in yt comments only I got reply
To edit the code
Add #include
Add lib vector
I did that
Code complianced now not connecting to phone
Both ur mobile and esp32 connected to same wifi
I have downloaded the application from Playstore but it is not connecting to ESP32 cam module What should I do
Hello,friend. Great project. I have a question, Is the pin LEFT_CHANNEL and RIGHT_CHANNEL in the code connected to ENA and ENB of l298N ? Thanks!
Hello Ding, LEFT_CHANNEL and RIGHT_CHANNEL are pwm channels, they are not pin numbers
It seems all is OK, except that the speed of the playing video with Android phone is very slow. And I do not know where the bot is because I can not get the picture in time on the phone.
Please help me solve the problem.
Is this the Ai thinker's Esp32-S you were using? Pls let me know!
I don't think it is
@@roboticslearner then which model you were using?
Could you check my pinned comment, I bought it from a local shop and not sure exactly about the model, so I had to try many times to configure the IDE and pin definition and then figured out which as my pinned comment works
sorry, i try to implement some similar project in react native webview, how do you handle and ip default from a site, boradcasting to the app?
do you have some react native projects similar to this? using esp32 cam
I mean, how can you translate the ip defualt view to a webview in an application ?
Hi Edward-Hedz30,
I have not used react native before, but I had implemented a web browser remote application using some basic html and javascript, could you check it if it can be somehow a reference : github.com/ANM-P4F/ESP32-CAM-ROBOT
my ESP 32 cam takes 250mA with out the LED light so the bat drains quite fast.......or use a hughe bat
Can we track and follow a particular object with this ?
I have done everything then also esp32 cam is not connecting with app why plz help me through it
I'm sorry you haven't made more videos lately. I hope we get past this stupid pandemic thing soon.
Yeah, I'll try to come back soon :)
bro all connections are ok plese we need detailed video for code and libraries
I have a tutorial in the video description
where should i connect servo negative wire and we can give 7.4v to the car through motor driver plz answer
Please check the description below the video and you can find the hackster.io link for showing how to connect the electronic parts
Seeking help here,
Is there any possible if I supply 7.4v instead of 5 v??😅
In that case, you need a converter or an IC ex 7805 to convert from 7.4v to 5v for supplying the esp32 cam module, for the motors, I think you can supply them with 7.4v
sir plz help me.........
when i am trying to connect the cam app with the esp32 its witting that can't connect to esp 32 camera..............sir plz help me its argent
i am using oppo a31 phone.........
please give me the wiring diagram for esp32camera and lm298n. thank you
Hello! this is very nice project. I tried and it Works! I have som quetions may will improve the robot. for example if trying to turn to left should also right wheel turn in revers direction. The same as to turn to right should left Wheel turn in revers direction. And som time it reset the robot .button LED and object traking not showing in App?The second quetion is what software or program you used to design Android App software? Any advise would be great, thank you!
This one is a prototype I made when I have free time, so it still does not work well in some case I think, but I would improve it in the future, I used Android Studio to make the app.
Thanks for your reply , It will be great if you do a project for how to design App with Android studio for this project.
Yeah! That is a good idea, I am busy this moment, but will come back for more projects in couple of months, please dont forget to take a look at that time ;)
@@roboticslearner please update the code so it work more properly
can face recognition happen offline without laptop for example using just an 12C LCD screen to write names?
Good idea, not yet implemented
@@roboticslearner
I mean is it possible to just use offline esp32 to face recognizing?
😍👍
good work but, in your arduino code there is conflicts motor control pins. can you please open and clearly comment pinns for l298 motor driver. or provide a schematic please
Hi man! Fantastic project!!
I am having issues connecting ESP32CAM to smartphone. They are both connected to th same wifi...
bro is the code complied
I am getting an issue
'vector' in namespace 'std' does not name a template type
this is the issue i got
help me plz
same bro
your project is it working?
@@rahulchowdhary8063 just add this line of code at the header of your code #include
I want to ask, how do I connect this project, if I make my own application using the Inventor or Codular app? and can this project be opened using a browser like chrome or something else?
hello, if you want to use web browser, firstly you should try with this th-cam.com/video/xvkwcLCyFT0/w-d-xo.html
plzz help me with it my app is not working its argent plzz help me
Sir I have an error message will comes it will showing class WiFiClientSecure' has no member named 'setInsecure'
_client.ssl->setInsecure();
What can I do? Would you please give me a solution please
My god sir it iss thunder really
does this project uses arduino board?
May I know the reason y bot is hanging suddenly
Hai mate this is great!
I have an obstacle program that reads "camera_wrap. H: no such file or directory"
What is the solution?
Hello, could you follow the description below the video?
hello, the android app isnt working can you help please
He'llo sir in object tracking when we moving the box infront of espcam then the servo is working when we doing up and down but when we bring the box to left right why the car is not moving this system is only for servo only please say
Yeah, due to the limitation of the gpios so I have not implemented that function
@@roboticslearner can you try to add that also if it can 🙄?
hey buddy, I'm making a project where esp32 cam can identify that is there any person in the room or not and then according to that switch on/off the lights, using its camera module.
plz guide me in this, how to do it ?????
You should use object detection and filter person class only, then count the person if more than zero or not, object detection you could use yolo or ssd mobilenet, but esp32 is not capable of running these cnn itself, so I think you should use esp32 for streaming and control the switch only, ai task should be done eslewhere...
Hello cn u share the code only for the object detection ??
Hi, the app cannot connect to camera , but i can browse the ip with the browser , where the 2 motors work fine but not the servo. Do i have to install the AI firmware ? where?
Ip address on browser with port 86 shows : websocket server only ... and the app cannot connect
My mask is : 255.255.255.0
IT WORKED .. finally , it was a connection problem .. commands are significantly lagging .. GREAT
does it also work with an i pad
I 🔥 made this thank you 🌻
Is there any option to update app data base like different components objects
You can retrain the model
'vector' in namespace 'std' does not name a template type i got the error can you help me bro
How did you made it?is there any details paper or video?
hi while compiling i got
'vector' in namespace 'std' does not name a type error
please help how to resolve
same here....please help me
Hello please urgently help me
I am getting error "WStype_t" has not been declared 😢
Sir can you please help I am not able to stream Esp 32 AI cam App
Can be used without android application ? Can open the control from webserver ?
Yes, but in that case, you just can just do some manual remote control, or low rate ai task ...
Hello A&M I have errors : undefined reference to `grabImage(unsigned int&, unsigned char*) and undefined reference to `initCamera(). what can I do? Thanks
I have try "//cameraInitState = initCamera();" and "//grabImage(jpgLength, jpgBuff);" this two lines. It Done compiling but I believe that camera should not work properly.
You must keep these functions, they are important, it cannot work without these function, you should download the repository and then extract it, there should be ino and cpp, h files inside, then use arduino ide to load the ino file and flash to esp32-cam, then it should work
@@roboticslearner Thanks so much, It's worked!
'vector' in namespace 'std' does not name a template type
this is the error i got plz help me out
plz help me.same problem .new here.
Seeking Help!
I am having issue - the streaming is inconsistent and had to reopen app to restart video streaming. The response also lags..
Also want to know how to add and set own image range in it.
Basically doing it for my daughter as getting into AI is her hobby so much request your help to sort this issue.
Thanks in anticipation!
Hello 2my way, I am not sure about the inconsistent streaming, but there are maybe 2 reason: your network is not good or your esp32-cam hardware has problems, could you try with another wifi access point or another esp32-cam module. Regarding the image range, I am not sure what it is?
@2my_way, pls help me in connecting the module with the app! I am facing "Cannot connect to Esp32cam error" after launching the app even though I have successfully connected the module with the personal hotspot of my mobile phone!
Hello Sir I have to make a Robot which segregates dry and wet waste using machine learning so i have the trained and tested dataset ready but how to implement it on Esp32 is a big problem plzz help me with it sir.
Did you manage to do it?? I'm still struggling
Can we connect through browser ??
Bro, If I use the sg-90 servo motor instead MG-90S ? WILL IT WORK?
PLEASE HELP 😅
Yeah, I think it is fine
Great work. Do you know if theres is possible to lane tracking byself with the ESP32-cam instead use the android app?. I want to make a lane tracking stand alone with the ESP32-cam. Greetings.
Hi Juan Carlos,
You mean using computer vision? if not I think you can use IR sensors.
@@roboticslearner I mean to process the image and detect the lane byself the ESP32-cam, without needing to send the image to the app (smartphone).
@@JuanCarlos-ff6vp I think we can do it, you could try writing a image gray scale and threshold function, but I worry that ESP does these processing not fast enough, because you need to iterate through the image with size 640x480x3 pixels (I use the resolution 640x480), anyway, you can try it, for ex: stackoverflow.com/questions/54173733/convert-rgb-to-grayscale-in-bare-c. If you run these task on pc or Android, you can use opencv api.
@@roboticslearner I was thinking of applying Hough transform. I am building a robot that runs through the rows of vineyards. The idea is to keep the car in the center of the aisle that forms between the trees. Greetings and thanks for your answers.
Hi Juan Carlos,
I am interested in autonomous driving also, but for that kind of lane tracking, I think you could use segmentation CNN, but ESP seems not sufficient to do that task, if you want to use a same size but stronger controller, I think you can use openmv.io/products/openmv-cam-h7, or raspberry pi/jetson nano are good candidates also.
how do i connect to my mobile? pleas tell me
Seeking help,
Can I upload the code using Arduino nano instead of ftdi ttl module?
Please help 😅
I think it would be same :D
💘
I have question regarding control the robot using your app,how to connect app with the robot,?did you using WiFi?if yes ..so the robot Will have WiFi ssid?? Thanks...
Hi Made Wira, yes, you could check the description below the video, you need change ssid and pass corespondent to yours in the source code for esp32-cam module
@@roboticslearner thanks for your respond sir, i just check the description in your video,now i solve the problem,anyway thanks for code and great app.....
How will the camera move left and right ?
The camera module does not have enough gpio to control many thing, so if you want to control the camera to move left and right, you need to add a arduino board
bro ye aap robot app se control kar rahe hena
Please help : I using ESP32 Wrover Module and run ESP32 example CameraWebserver it smoothly. But when I try AI robot there are many errors such as: undefined reference to `WebSocketsServer::WebSocketsServer(unsigned short, String, String)', undefined reference to `WebSocketsServer::loop(), Multiple libraries were found for "WiFi.h". etc
Can you show me how to fix it?
Thanks
Hi K j, actually I had written in REAME in the github link, you should download github.com/Links2004/arduinoWebSockets as zip file and add this library to your Arduino Libraries by Sketch/Include Library/Add Zip... Regarding the "Multiple libraries were found for "WiFi.h"", there is a WiFi.h duplicated in the arduino ide/ libraries, you should delete it and use the library form esp.
@@roboticslearner Thanks for your reply . But what about camera_wrap.h
it's not found.
Hi K j, How to dowload my repository?, it should be in the same directory with ino file
ai possui a aplicação para download ou somente o apk?
Can it follow a human instead,someone please reply
Misleading title, the phone is the one doing the AI not the ESP32
It s formal or anual
hi,
dude how to connect the app with esp32 ???
I`VE everything but how to connect it to the app to control the robot
Hello, when you press "Stream" button, the app should automatically connect to the esp32-cam module, you should check if your esp32-cam connect to the same local network with the android phone or not...
@@roboticslearner ok i`ll try it, thank you so much buddy
objection is running in your mobile or in your hardware.
Hello, it runs on mobile phone
Hello Sir, your project is very interesting and I like it very much, so I cloned your code and made some changes for my project.At first it worked fine, but after a few days, for some unknown reason, object detection stopped working at all, and object tracking had problems as follows:If the program has not performed target detection before the object tracking mode, the object tracking function will still work. If the target detection button has been clicked before, the object tracking function will only draw an immovable rectangle, and the crosshair will no longer appear.I really don't understand, can you help me find the reason?Thank you so much!!These words are translated into English by software. Please forgive me if they are not smooth.I hope you can spare some time to help me, thanks again!!
I have done all the thing then also esp32 cam is not connecting with app why but esp32 is connected with hotspot
Plz help me
@@shweta6965 hi is your project worked
I got an error
'vector' in namespace 'std' does not name a template type
How can I slove it
@@rahulchowdhary8063 in .ino file add new header file #include vector
Can esp32 cam can detecting object like smart cane?
Actually, the detection is done by the smartphone app, so if you want to detect some custom object, you need to retrain the model and build the app again ...
I AM also wondering, can you connect directly to esp32 with a phone without using wifi??? would that be possible?? thanks :D
Yes, via wifi or bluetooth
@@roboticslearner i figured it out, I just changed the line of code from connecting to wifi to making its own server where I can connect the phone to.
hi bro did you did this project...?
@@rahulchowdhary8063 yeah, like I wrote
@@kobalov1 hi sir I am facing problem
I am not able to connect esp to phone 🥲
Help me plz
Hi, I made a robot, but when I press the forward key or any direction key, the app also works nonstop. I wonder what caused it. When I press forward or backward, the engines run nonstop for 2-3 seconds. the problem is exactly what. ESP32 stems
When I press back and forth or right and left, the motors start non-stop, but the servo motor moves up and down by itself.I changed the esp32cam, but unfortunately I have the same problem again
I don't have that problem but let me check for a while
@@roboticslearner Thank you so much for solving this problem. Would you share the object detection codes ? I would like to improve.
when I enable object detection, the application slows down a lot. What is the reason?
Hi, Object detection is running on a cnn model, currently I am using ssd mobilenetv2, it would take much cpu computing, so it would slow down the overall
I have did everything thoroughly, even after connecting the module with the mobile hotspot, the app is showing something like "Cannot connect to ESP32cam"... I guess there is some mistake in the code ! Even, I am unable to stream video with the IP address using your code so I guess something have missed in the code( The board is working fine with webcamserver example ) Please help me to fix the error !
I don't know why you get that error, the code run fine with mine, and confirmed by some other persons
You could check by connecting the module to pc via serial and check with serial monitor to view the debug infomation
@@roboticslearner what should be the debug information? 'Camera int value 0' is that it?
Yes, 0 means initialization fine
Nice one, but wire connection isn't clear
Dude, I'm getting this error while uploading the code.
ESP32CamAI_arduino:156:6: error: 'vector' in namespace 'std' does not name a template type
std::vector splitString(String data, String delimiter){. What to do now?
You should install arduino stl library in arduino ide
bro edit the code
#include
add lib of vector
error sloved
Brother how to uplod program?
can anyone guide me how to do the project from scratch....
Bro, I am experiencing an error "expected class-name before '{' token
class socketIOclient : protected WebSocketsClient {" while compiling the code in arduino.. Please help me!
How this you flash the code, you should download my code, keep same name with the repo name, remove "-master" in name I think, install some dependencies as README via arduino IDE, then flash it, I am not sure what happens to your procedure, but many other people confirmed with me they can run well
@@roboticslearner but u have said to add websocketsserver in arduino and to include it in ur code...
Hi, did you follow my Readme file?
@@roboticslearner yes bro, I did.. So there is no need to include "arduinowebsockets" into Esp32AIcam.ino ! In that case I am able to upload the code successful and IP address is generating fine in the serial monitor but whenever I try to connect with the module, in the app it is giving me an error "unable to connect with edp32cam" even It is connected to my hotspot! So, what can I do now? I want your support inorder to complete my project pls help me
The websocket is significantly important, without it, you cannot stream the images