Cambot! Servo controlled motion tracking camera.
ฝัง
- เผยแพร่เมื่อ 7 มิ.ย. 2024
- See how to use A PC running Open CV, an Arduino and servos to create a motion tracking camera.
Code, parts and technical details available here!
github.com/jgoergen/CamBot
This project is based on Hackerbox 24 from hackerboxes.com.
NOTE!
In the video, I initially mount the U shaped bracket incorrectly by placing the servo mount in the middle of the U. This piece should be mounted to one of the sides, which is how you will see it mounted through the rest of the video. Sorry for the mistake! - วิทยาศาสตร์และเทคโนโลยี
Love all the HackerBox stickers all over! Thanks for putting this together!
Hahaha, yeah!
Thank you for checking the video out! Hopefully the code / build saves everyone alittle time.
I have it build, But the code. I understand a little bit of it. But alot of the abbreviations you have I have to look up what they mean. I'll get it figured out. Thanks!
I would love to help clarify ( and edit my code / add comments / explain on the github ) if you want to bounce some things off of me!
The movements are a litte bit creepy. I think this is why that is such amazing and cool. Thank you 4 that!
Hahahaha, it's true. Thank you so much!!
THIS IS AMAZING! I shall adapt this to my own nefarious needs!
Awww, thank you! hahaha.
It was actually VERY easy to do, to be honest! Especially if you start with my code, you could whip this together in an afternoon.
Do itttt!
I gots to track down the electrics yo!
Gotchu, fam! ( I subscribe to these boxes, so this is literally what I used. )
hackerboxes.com/collections/frontpage/products/hackerbox-0024-vision-quest
Righteous!
Very very cool man, exactly what i wanted
Scorponch 457 thank you so much! Don’t forget to checkout the GitHub project in the description.
Hi, i loved your work, is it possible that i could use this for a custom image recognition ? i want to be able to develop something that distinguishes between objects and tracks a desired object.
Thank you so much!! You could certainly start with what I have here and add image recognition to it, but it would be challenging.
I would start with template matching ( docs.opencv.org/master/d4/dc6/tutorial_py_template_matching.html ), that would let you provide a static image of something you want the camera to track and then get it's location in an image ( the webcam feed. ) Once you have it's position in the feed you can change the camera movement to point towards the image match location instead of what used to be the movement in the webcam feed.
thank a lot! so i have a question what type of cam you used to control in this project is?
Thank you for checking the video out! I used a webcam taken out of an old laptop. Most of them are really just usb webcams so the trick is just figuring out what the pins are to hook them back up to a different usb port.
Nice .ty 4 share. Is there a way to get smoother movement of the tracker?
Thank you for watching!
Have you tried turning down the SERVO_MOVE_SPEED setting inside VideoSurveilance.cs ? That would result in the motors moving slower, which would make things smoother, although it'll make the camera track slower.
Thank you!
Thank YOU for checking it out! Hope it saves you some time!
Excellent - beautiful project :) Can't wait to try this at home! Did you consider doing this with a PI instead of a PC - would it be able to process the data in real time?
I haven't worked with Opencv on a Raspberry Pi, but I would expect it to run pretty slow, even on a Raspberry Pi 3B. This would make it really hard to track real time motion, like the windows code I provide does, but I would def think you could have it detect faces in frame at ~5 fps or something to that effect. It kind of kills the point of having the camera looking around as it wouldn't be able to keep up with that new stream of data as it did, but you could find some middle ground and have it light up, play sounds, notify your computer, etc.
Maybe I'll put a video together for something like that soon, I'd love to hear your ideas on what you'd do with the slower fps.
And thank you!! Thanks for checkin the video out!
@@ArtsyEngineering I tried opencv y'day on my laptop, and recognizing a face on a video was super simple and I got 30fps. So I would hope that a pi could do 5fps which I think should be more than enough for a servo-operated camera. Also, I didn't do any optimization (such as, for instance shrinking the image, hoping that it's faster but not too low res - to allow the algorithm to find out where the face is), so I am hopeful. I'll let you know when I get my pi cam.
@@plouf1969 That's awesome that you set that up! Raspberry Pi's are WAY slower then modern computers, so the only problem I'd be concerned about with a face following robot at 5fps would be you're sliding out of frame faster then it could follow you, so the image would just go from "frame with a person in it" to "frame with no one in it". For any kind of a follower you'd want enough frames per second analysis to be able to say "frame with a person in it" and then "frame with a person slightly to the left" and "frame with a person all the way to the left" so it knows you're moving off camera and can continue to try and recenter you ( which would appear as if it was 'following' your face. ) My code has this problem if you're too close to the camera and move away with any kind of speed, it will have no idea where you want and just assume there is no subject anymore.
Hey!
I was a bit confused on the github code that you provided in the description. Which code am I supposed to use and which resources am I supposed to implement? I am a beginner with this, and I am creating a pretty complicated project eventhough I am not supposed to.
The key to taking on complicated projects, is breaking them into small pieces that you can figure out as you go!
So in this case, the first thing I would suggest you do is concentrate on building and programming the Arduino to move the servo motors. You can find a list of parts needed on the Github code main page, and the actual Arduino code to get you started is in the "CamBotApp" folder in there.
Here's a quick list of small goals to get you moving!
1) Order your parts!
2) Install the Arduino IDE on a computer and program the nano with the blink sketch to test everything.
3) Install the Neopixel Arduino Library and hookup one Neopixel to the Nano. Program the Nano with a test sketch to test that the Neopixel is working.
4) Hookup the servos ( without the brackets ) to the Nano and write your own small test to make them move.
Once you've made it this far, you're really well on your way! Now have a look at the schematic I added to the Github page and wire everything up like that. And for your final goal, download my code from the "CamBotApp" folder and modify it to do interesting things with your servos and light automatically, instead of waiting for USB communications.
When you make it this far, you'll have accomplished and learned a lot already! Let me know and I'll help you with the rest of the project!
I forgot to mention, I added links to all the part's you'll need on the Github page as well, hope that helps!
Jeff Goergen not the original question asker, but wanted to thank you for the video and info!
My pleasure! Hope it helped!
would power source consisted of 4x AA battery on series sufficient to power up the servos ?
Assuming they were decent quality batteries that would be around 6v which should be just fine! Servos can pull alot of power depending on the amount of resist they are pushing around so it may not last very long.
Looking at this again, I wonder how it might resolve multiple targets in motion simultaneously ?
It's a tricky subject, but open cv ( the library I used to do this ) does return multiple regions of movement and I'm just filtering out ones that are too big and too small and then picking the largest of what's left to assume that must be something I care about. This gets really hairy if you actually want to track multiple objects and they can stand near eachother or can vary in size alot, even worse if your video feed is really noisy ( so opencv will pickup alot of 'movement' that's really just static. )
Computer vision seems to be 80% smoothing out a signal and filtering results and 20% clever application of what was found.
Of couse having a more defined idea of what you're looking for in the video feed will also help alot. For example, looking for colors or a face to qualify the object as something i care about would vastly reduce the amount of guesswork, but then you're cranking up the cpu load and hoping the person is always facing forward or there are no other green objects on screen. There are just SO MANY variables that need to work for it to appear as if it's working well.
Hey Jeff I have several questions
1)I did download visual studio2017 and once it got installed I clicked on additional data to be installed I assumed it was the EMGU libs and opencv , how do I confirm if I have installed everything correctly and how do I check if it is working perfectly without the actual hardware setup just to see if it's working?
2) I have a camera module OV7670 can I use it instead ? Do I need to download a camera library from Arduino libraries?
For your first question, if you just plug in any Arduino via USB, regardless of whether it is running any particular code, you should be able to run the app and see it draw boxes on your webcam feed indicating that it's seeing motion. If the project doesn't build you will get errors about what you're missing. If you have any issues with that, let me know!
As far as your second question, this project is made to use a USB webcam to feed video data to the computer for motion tracking. You would have to do a lot of work to use an Arduino based camera instead and it would be ALOT slower, so I would not recommend going that route. You can get USB webcams for super cheap now days off Alieexpress, Amazon, etc. Of if you don't want to order anything else, you could just use the webcam on your computer (if it already has one built in.) The robotic arm won't be pointing directly at the motion, but it will respond to your movement with it's own movement, so you could still make something kind of entertaining that way as well.
@@ArtsyEngineering thanks a lot !
@@ArtsyEngineering for the moment I have separated the servos from the camera although the servos just keep on rotating rather that rotating according to change in position , the app works and the camera is a webcam
@@bigbadwolf4075 OK, that's great news! So you can run the app, you installed everything correctly and you built the Arduino app to a microcontroller and wired all the parts up correctly. At this point I'd say you have 2 choices: 1, order a cheap USB Webcam off the internet and attach it to your servos so they can move it around. That would be using the project as it was originally intended, although you could certainly customize it to suite your needs, which is the second option.
Right now when the app sees motion it wants to keep that motion in the center of the video feed, which is why it's always trying to move your servos, because no matter how much it moves them, the movement isn't approaching the center. So you could change the code around so the desktop app really just wants to point the servos according to the area on the video feed that movement is found ( so if something moves in the top left, point the servos at the top left. ) You could do this by editing lines 172 - 174 of "VideoSurveilance.cs" which is where the desktop app gets the area of movement back and tells the robot to point towards it. You'd probably just want to try something like "if you're already pointing in the direction this is asking, then just stay where you are" and that would need some max distance before it points somewhere new. If you want to give this a try, let me know if you run into any snags. I can't help with actual code right now, but as soon as I get free time I can!
Hope this helped!
Awesome
You are
Great!
If i am gonna use Thermal Camera, will it work?
Thank you for checking it out! It depends on how high resolution your thermal camera is and how much thermal movement will be in the video naturally. If you had alot of movement in the video from something like steam it will confuse the movement recognition. Also if your thermal camera is too low resolution ( they are usually very low compared to a typical webcam ) it will limit the tracking quality as well. Let me know if it works, though! That'd be neat!
i can't understand why refresh rate of video is really low, i tried also to replace webcam with a 720P but is still slow, can you help me?
What kind of computer are you running this on? Image recognition is VERY cpu intense so you will need a fast computer to do it very quickly. Also the higher the resolution on your webcam feed, the slower it'll be ( the more pixels it has to process on every step, the slower it will go. ) You might want to start by reducing the webcam size down to something like 640 x 480.
AMAZING 😀😀😀😀
Aww, thank you! It was, surprisingly, not that hard!
Do you think it would be possible for me to use an Arduino code to make it motion track like this and then send a signal to some small 30mm mm EDF fans and servos to make some steer themselves toward a motion tracked location
Hahaha, don cheadle huh? Love your work!
I’m sure it would be, but to be more specific, you’re saying make a device that watches for something and points a fan at it? If that’s what you mean, it would be as easy as building this and attaching a fan to it, although you’d have some special considerations if you’re attaching a heavy fan, or a fan that will blow so hard it will destabilize the camera.
@@ArtsyEngineering well no I want the camera to identify the arm mostly at all times then the fans propel the attached object towards the motion tracked place
Don Cheadle I’m not sure I follow you. You want a fan to push something towards motion? Or you want a fan attached to an object to blow itself towards motion?
@@ArtsyEngineering basically if it sees a certain object such as like my face it will select that and then that will tell the fans what to propel themselves toward and steer to
@@ArtsyEngineering sorry I didn't read your last reply correctly that's actually exactly what I was looking to do get them to propel the object towards the motion tracked location
Will this work with the arduino uno, btw I subbed
miaowzerz aww, thank you so much for the sub!! The motor controller would work just fine on the uno! The Nano is essentially the same thing so you’d be good there. The computer is doing all the heavy lifting with the webcam reading and detection work so that side won’t have much of a preference for which microcontroller you use. Let me know if you hit any snags!
Ok
Hey man,i like your project,u dont have and the python code of this project,or u could try to make this project with with raspberry?
Thank you so much!!
this was done in c# and arduino c, which are both posted on the github page listed in the description. You could certainly do this with a raspberry pi 3b or similar, although I would expect the speed to be quite a bit slower ( both because of the slower processor and using something like python for it. )
Can this motion tracking work without connect to PC? Is it possible?
You will always need some kind of decently powerful system to read the camera frames, scan them for whatever content you're looking for ( in this case motion ) and send signals to the motor controller. So you will always need SOMETHING like a PC, but you could probably make this work ( although much slower ) using a raspberry pi! You could also change the webcam to something like an ESP32S Cam module and have it stream it's video signal to a pc somewhere else, and send it back data that it would then relay to the motor controller. That would atlest make your cambot wireless ( assuming you added a battery as well. )
@@ArtsyEngineering thank you for the answer, i want to make something a bit different, i found open cv bot on Playstore, when we use this app, camera can read motion by pointing the moving object with the red line, but i want my phone moves like this using servo. Emm.. any other way to make a motion tracking without camera? Its like putting motion sensor with servo, im sorry if my question is weird
Can different types of cameras work too
If you're using my code, you'll need something that acts as a webcam source on the computer so my code can read it. In general that's going to be the case.
Don't make the mistake of following this step by step. The first part of the video shows the wrong bracket to attach. Later in the video it magically shows the correct brackets mounted to the servos. Just a heads up bc I made the mistake and had to take it apart and do it right.
Jeff Goergen At about minute 2:15 the builder mounts the aluminum circle servo mount to the top of the tilt bracket. This is incorrect. Later in the video you will see that it is no longer mounted to the top, but rather correctly to the side of the same tilt mount.
Oh wow! I just realized what you mean! I'll try to find some way to make this mistake obvious, TH-cam doesn't allow annotations anymore unfortunately so I can't add text over the video anymore. Hopefully that wasn't a huge waste of your time, thank you for pointing it out!
No Problem. Luckily I was able to get the gear out of the disk. No worries. Just want to save time for others. Do you happen to have any info on the power supply? I'm new to this stuff and am not sure how to power both servos with one supply.
The details I've gleaned from different places seems to imply that anything between 4.8 and 7.2 volts should be fine as long as it is around 2amp. 6v 2.5 amp sounds like it would be ideal.
When I made this video I was using a 4.7 volt, 1 amp power source for it and it worked fine, but I guess eventually that will burn out. I have since changed it with a 6v 2amp supply and it seems to run fine.
Will 3 AMP be too much? I have a 6V @ 3 AMP power supply.
thx, awesome, can i put a color to track it?
Oscar DX hmmm, you mean make it specifically follow a color instead? I’m sure you could, although I don’t know open cv’s color tracking functionalities offhand. I may give it a try one of these days and reply back to you!
Yes thanks, or do you know how to only track one item? When i Run the program i have some ítems to track
@@Alexlml96 unfortunatley it's currently setup to just track all motion. You can filter that a bit by adjusting the settings to tell it to concentrate on larger items, that will cut down on the amount of 'noise' you get , but 2 large items will still both register. You'd def. need to have it look for something more specific, like a green ball. Ofcourse you could stil lhave a person with a green shirt wander in and confuse it, though!
The camera is not connected to the board, right??
Nope! The camera is USB and connects to a computer. The computer uses Open CV to read the camera, detect motion and instruct the microcontroller to move the servos in order to follow the motion it sees.
Thanks, I have made this! I have used your code but sometimes I believe it doesn't move at all. It also frequently mistakes background objects for faces. I am trying to improve the code. Can you provide any links to resources on that?? Thanks for the video..
Did you? Awesome! I'm glad I was able to help in some way!!
I wish I had some great articles to share, unfortunately I pretty much just winged this based on prior knowledge and alot of tinkering. Hopefully I can help you fine tune things with some more info.
It would be hard for me to diagnose what exactly your issues are. My copy does not have issues with "not moving at all" so I would wonder if you have an issue somewhere else in your setup that is making the hardware response intermittently?
And with the movement detection, it is VERY sensitive to noise in the video ( so it doesn't work well in dark rooms, and this gets worse and worse with extra cheap webcams. ) Also, your robot will need to be seated on a totally still surface. ANY movement in the video will render the motion detection useless. If you go into the c# code ( VideoSurveilance.cs ) you will see some settings towards the top of the file that you can play with to fine tune things alittle as well. I should add comments to the github for this, but I'll describe a few for you that might be usefull for this issue.
MS_PAUSE_DETECT_AFTER_MOVE is the number of milliseconds the software will wait before it moves towards ( what it thinks to be ) movement. The significance of this is the more frames of video Open CV gets before moving the camera the more the stationary objects will 'settle' out of the image. ( This is why the image will slowly go mostly black over time. Open CV is trying to remove things that are the same between frames leaving only the moving stuff. ) So if you turn this up, it will help remove stationary objects, but it will make your camera less responsive ( as it has to wait before it can move again. )
SUBSTRACTION_HISTORY works in tandem with the above setting. If you give the library more time to settle, you have to increase the number of frames it has in it's buffer to settle with. So if you increase MS_PAUSE_DETECT_AFTER_MOVE you should turn this up as well.
SUBTRACTION_THRESHOLD is something you will just have to play with, but the idea is the higher this number the less the library will assume a difference between frames is actually movement. Turning this up would probably help remove some webcam noise but if it's too high it will make the motion detection alot less usefull.
FRAME_BLUR_STRENGTH would help with webcam noise alot like SUBTRACTION_THRESHOLD. ( the higher the number, the more it smooths or blurs frames. ) Just be careful, this MUST be an odd number and the higher it gets, the less accurate the motion detection will be.
Hey man would it work when you don't programming the pc
All good until this turkey added cardboard...end of the line for me..
Ed Auzin Hahahah! Yeah I had no real need for this to be permanently together so I cut corners. The code and GitHub link are still useful if you hit a snag.
Hey man would it work when you don't programming the pc
Hmmm, you can run my code on your computer so you don't have to program anything. You would still have to install visual studio and get things setup to build the code, though, which would not be much fun if you're not trying to learn how this stuff works.
Hey man would it work when you don't programming the pc
Without the pc you'd only have a camera on a pan / tilt setup and an arduino that can move the camera around. The pc helps you figure out where to actually point the camera and tells the arduino to do it. You could use something like a raspberry pi to control it although image recognition is probably going to be VERY slow on that, but you could write your own software on a raspberry pi just to control the camera atleast. You could even modify my arduino code and add some buttons or something to the arduino to just have a webcam that can be manually controlled.
Hey man would it work when you don't programming the pc
The pc does all the visual recognition of motion, so without the pc you'd lose the ability to follow movement around.