Computer Vision Robot Arm - OpenCV Arduino | Karthik Ramprakash | VIT University Vellore

Karthik Ramprakash - B.Tech Specialization in Automotive Engineering
VIT University Vellore
SMEC - Robotics Lab
Arduino based Computer vision robotic arm
Executive Summary-
This paper presents the design and implementation of a computer vision-enabled robotic arm that is autonomous and can be operated without the need for a human interface application. As robots are becoming more common in industrial environments every day, there is a need for them to operate autonomously with minimal human operation and user inputs as it can be a very complex task to manage numerous human-guided robots in daily tasks. Such robots can be pre-programmed to do a set of repetitive tasks but more often, these configurations don't work well with real-world scenarios. This project aims to develop an autonomous vision system for industrial robotic arms that are present today in order to smartly manage factory line robots and enable them to do complex tasks on their own, adaptively. In this project, a simple & robust vision system for robotic arm control will be developed and tested in various scenarios in order to study and research the technology that aids further development in this technology. The robotic arm in this project will automatically identify an object placed on the canvas, determine its location, and will be picked and placed into the desired location. The image processing technique used is background extraction identification using the OpenCV python library and an inverse kinematic algorithm will be used to convert the camera coordinates to robot arm movement to pick up the object and return it to the initial position, everything is done automatically by the machine without human inputs.
Keywords: machine vision, 3dof robot vision system ;industrial robot; pick and place; inverse kinematics; robotics; image recognition; Arduino; python; opencv;

[@tallabrianmonde3021]

Very nice project can I realize this using esp32cam ?

[@kishorek5063]

bro, how did you convert the pixel readings from opencv to normal cartesian coordinates and how did you shift with respect to robot arm position?

[@catch22stuff29]

Very Cool Video Demo... What happens if you place 2 items on the paper ? - is the code open source ? - if someone wanted to expand on this project ?

[@karthikramprakash]

I have used Python - OpenCV image recognition which is open source code. The image processing technique is background image subtraction and I am finding the centroid of the object and using inverse kinematic I am programming the robot to reach the object’s X and Y coordinates with a predefined Z axis. If 2 items are placed I think it may pickup any one first then the other one next. Or if the centroid returns value of a point in between both objects then it won’t pick! This is a good question! I haven’t tried it out with 2 objects. There is so much scope to expand in this, starting with recognising the Z -axis too with another perpendicularly placed camera. Other refinements also can be done to avoid shadows, eliminate any errors in centroid detection, and more scope can be added by introducing more image processing techniques like Color based sorting technique or item detection, or size or shape based sorting… can use raspberry pie to avoid line by line control of arduino ….the scope is endless

[@RichardSanthosh]

Hi i am actually planning to do almost same project for my college i have some doubts about this how we can sent coordinates to Ardiuno

[@luckyalthaf2990]

Bro...which webcam you use in this robotic arm..?

[@karthikramprakash]

Normal webcam bro, I think it was a logitech

[@junaidmalik7140]

Can u provide the code

[@karthikramprakash]

This is the code for opencv image recognition is available here: github.com/dragonfist453/Centroid-detection-OpenCV
To send the coordinates to control the arm you need to configure yourself depending on the type of arm u used. Also have to use inverse kinematics to map the location where you want the arm to reach and put the object

[@junaidmalik7140]

@@karthikramprakash okay I get it thanks alot man

[@junaidmalik7140]

@@karthikramprakash can u give me the kinematic code also

[@lionnelakam4719]

@@junaidmalik7140 please i need it too

[@nmuniyappan931]

Send me