OpenCV Object Detection in Games Python Tutorial #1
ฝัง
- เผยแพร่เมื่อ 5 มิ.ย. 2024
- Learn how to use OpenCV for object detection in video games. This intro tutorial will show you how to install OpenCV for Python and get started with simple image template matching. This will serve as our foundation as we explore many different computer vision techniques in future videos.
Full tutorial playlist: • OpenCV Object Detectio...
GitHub Repo: github.com/learncodebygaming/...
OpenCV documentation: docs.opencv.org/4.2.0/
Official template matching tutorial: docs.opencv.org/4.2.0/d4/dc6/...
0:24 How to install OpenCV
1:45 How OpenCV relates to PyAutoGUI
2:52 How to navigate the OpenCV documentation
4:53 Comparison methods visualized
5:39 Writing code for cv.matchTemplate()
OpenCV is an open source computer vision library with hundreds of functions for processing and understanding images. In this tutorial, I'm going to show you how to get started with OpenCV in Python by using it to find an image inside another image. This simple form of object detection will be a good starting point before we move on to more advanced image recognition techniques.
The quickest way to get started with OpenCV is: pip install opencv-python
Once installed, you can use the library by importing cv2. Numpy is used extensively when working with OpenCV data, so the top of your Python files will look like this:
import cv2 as cv
import numpy as np
That's all there is for setup. Now let's grab an image we want to process. I'm going to be using this screenshot from Albion Online, but any screenshot will do.
What we're going to do is crop out a small section from our screenshot, save that as a separate image file, and then we're going to use OpenCV to find the position of the smaller image inside our entire screenshot. From my screenshot, I'll crop out one of the cabbages.
The OpenCV function we'll be focusing on is called matchTemplate(). In the documentation, we can see we're going to give this function an image to search over, an image to search for, and a method type for doing the comparison. And we'll end up with a result array. You'll want to experiment with the different comparison methods to see what works best for your use-case.
Alright, let's write some code. The first thing we want to do is load our image files.
haystack_img = cv.imread('albion_farm.jpg', cv.IMREAD_UNCHANGED)
needle_img = cv.imread('albion_cabbage.jpg', cv.IMREAD_UNCHANGED)
The "haystack" image is our screenshot, and we'll be search that for the "needle" image we cropped out. In imread() the first parameter is the image file path, and the second parameter is a flag that allows us to do some pre-processing when loading the images. In this case, we're loading them in unchanged.
Now that we have our images loaded, we can go ahead and call matchTemplate(). I've had good luck using the TM_CCOEFF_NORMED comparison algorithm.
result = cv.matchTemplate(haystack_img, needle_img, cv.TM_CCOEFF_NORMED)
We can quickly see the results from matchTemplate() by displaying that data with imshow().
cv.imshow('Result', result)
cv.waitKey()
In imshow(), the first parameter is the window name and the second is the image we want to show. I've also called waitKey() to pause our script while we review the image. Without this, our script would quickly close before we could see the image. Pressing any key on the keyboard will trigger waitKey() to stop waiting, thus ending our script.
In this result image, the bright white pixels represent the positions that best match the cropped image. The black pixels are the worst matches. Note that these best match positions correspond with the upper left corner of where you'd place the needle image.
Now that we've visualized the results of matchTemplate(), let's get those best match coordinates. We can do that using minMaxLoc().
The minMaxLoc() function returns four values. First are the confidence values for the worst and best matches, on a scale from 0 to 1. These are how black or how white the darkest/brightest pixels are in our result image, where 0 would be perfect black and 1 would be perfect white. The last two values minMaxLoc() returns are the positions of those worst/best match pixels in the form of an (X,Y) tuple.
For every needle image that we give matchTemplate(), we will always get back some values from minMaxLoc(), even if that cropped image appears nowhere in the haystack. We can tell when we didn't find a good match because the max confidence value will be low. How low is too low depends on the images you're working with and what you're trying to achieve.
Now that we've found a good match, let's outline where we found it in the haystack image. We can do that using OpenCV's rectangle() function.
Continue with the written tutorial here: learncodebygaming.com/blog/op...
Join me on Discord: / discord
I just found out your video and even as a senior python and opencv developer I enjoyed watching all of it. Learning image processing through games is an excellent idea. Your presentation is clear and precise, I disagree with comments which states it is a bit boring. Your explanation level is just right for the beginners, not overexplaining things and exploding beginner's minds :D
I wanted to take my time and display my support with a comment because you clearly dedicate your time and enthusiasm to create a helpful content for the community. You just earned a subscriber, keep up the good work my friend.
p.s. About the not so perfect matching score, it is probably due to using jpeg format's lossy compression. After you saved those images as jpeg, they will have very small differences invisible to eye but clearly visible to code as in this example. You might prefer using PNG for lossless image saving or even BMP to get faster image write/read times (high fps).
second this by far the best explanation i have came across, zoomed in code, well explained step by step
I went through this tutorial but deviated by using png files instead. ultimately this makes the tutorial not work as written because the data was no longer in the expected format of CV_8U or CV_32F for the matchTemplate function. Just a warning for those that like to experiment lol. My understanding is that you can easily convert to the proper structure, but me, being a noob, did not find success with this. I saved my files to .jpg and moved on with the tutorial instead of getting hung up and everything worked.
This was something I was wondering about,
Thank you. I don't think there are much interactive content like this out there. Glad to have found this channel.
Thank you once again.
Glad you like it!
I love your videos. they make it more fun to learn to code. thank you very much for taking the time to make these, especially into easily digestible chunks in a playlist. youre awesome!
I'm on part #3 of your series so far but wanted to revisit this to say - you've been helping me get out of tutorial hell. Thank you so much Ben! You have an amazing teaching style; The documentation and comments along the way in both video & blog posts provides more clarity than other tutorials. Your channel is one of the best out there, please don't stop. Peace.
Thanks man!
Honestly, the quality of your videos and tutorials is simply astounding. I'm constantly amazed that you don't have hundreds of thousands of subscribers....yet!
It took two years but I am very happy that I managed to find your channel! Thank you for the great tutorials and videos
I was JUST looking up info on this. Who knew it would be from the guy I subbed to from a few days ago. Keep up the good work your putting out unique and great content
Thanks means a lot!
Thanks alot for this video! Do you think you can cover how to find an object and also detect when it is gone in a video game?
Trueeee
@Carroll Horsely stop scamming people
Exact same here! What an underrated channel
I've just started getting into object detection in opencv along with tensorflow, yolov4 etc. Whilst I'm not using it for gaming, these tutorials have been great for real dynamic and practical applications and for that i want to say thank you :)
Your OpenCV videos inspired me so much, been learning constantly new things and doing some low profile botting too. Best channel I've stumbled upon in a long time :).
Here I was wondering how i was going to pull of object detection, then my man presents the answer to me just like that! Always appreciate you, never fail to expedite my thought process!
Nice to see you taking it to the next level
Just stumbled upon your channel! Love the style and format! Can't wait to check out more videos!
Hi there, this is the best channel about this content. I have watched your Videos about Python at least 20 times each! I am a beginner and was able to follow your explanation and even to costumize the code for my needs! Really amazing!
Whoever disliked this needs their head checking, this was one of the best programming tutorials I have seen yet.
I'm from Latin America, and I understand your tutorials better than the ones in my language. You're great, thanks
Brilliant channel. Enjoyed every bit of it. Love the short length. Subbed!
I have started learning computer vision not long ago and basically you are the best. Thank you very much for your tutorials
You did a great job at making this easy to understand, I followed it all the way through and it worked, Thank you
This channel is God sent. Keep going and you'll grow very fast
Wow dude been watching a lot of videos on open cv and yours is far the best I have seen. Just the way you teach is awesome.
God I love you and this series, the quality content we all need but do not deserve
Thanks for the video, I'm from Brazil and here we don't have many good videos about OpenCv and Python
, the simple and objective way you speak is perfect for me who doesn't is a native and thanks for not using so many slangs
Wicked video! thank you for the tutorial. After fumbling around with a few other guides, it was yours that really got me into it.
Awesome!
You're an EXCELLENT teacher. Thank you!
GOLD CONTENT
thanks , i just needed something like this to complete my project
you earned a new subscriber :D
Glad you like it! Good luck on your project.
Thank you, I love how you guide us through the documentation! This will help me start creating AI's to play games and automate tasks.
Thank you for this. Keep em coming :)
Thank you so much. Your explanations are so clear. I learn a lot from you. You are the best!
this channel is so much underrated.Very good content bro,keep it up
Underrated TH-camr. Great content!
This is my new favorite channel, thanks for good content ^_^
Thanks Ben :). I really like your teaching style.
Please do more if you’re available!
(Español)
Muy buen video, conciso pero detallado. Atiende a problemas pequeños de forma que permite aprender poco a poco y construir el código lentamente y de una forma muy natural. Mis felicitaciones sinceras, espero continúes con este trabajo de divulgación tan pulido. Solo me queda felicitarte de nuevo y seguir tu canal y aportes. Mucha suerte para el futuro.
(English by google translate)
Very good video, concise but detailed. It attends to small problems in a way that allows you to learn little by little and build the code slowly and in a very natural way. My sincere congratulations, I hope you continue with this polished disclosure work. I can only congratulate you again and follow your channel and contributions. Good luck for the future.
YES !! This is going to be very interesting ! =D
Thx, People like you make the world a better place
Thank you for your tutorials!Really appreciate it :)
You're welcome, glad you like them!
YOUR TUTORIALS ARE THE BEST IVE SEEN, THENK YOU SO MUUUCH!!!!!!!
why this guy don't have above 100k subs.. he explained it so good. +1
Bruh ty so much! You save so much time for as !
You're welcome!
absolutely amazing content !
Thanks for the video, I now learned what I needed. Cheers!
hey i JUST found this channel today, MapleStory is my childhood favorite game so this has motivated me to learn to code and use maplestory as my game, my younger self is screaming right now lol love your content
Yooooo Ben i hope you are still Alive and coding! I honestly cant thank you enough for all your content! It has finally gotten me started with programming and coding, im studying to become a python developer and ive truly found my passion and calling in life now. The way you teach and explain things is amazing, wish i would have gotten help like this sooner in life! Much love for you
Thanks for the video my man, very useful and well explained.
this guy should get more subscribers!
thank you for the great contents
Thanks!
Thank you for the great tutorial!
Thank u very much for this video. It was Informative and engaging. Makes me want to learn more about OpenCV!
This me first time watching your videos and loved it!
love you so much, I love this course, you just got a Chinese Subscriber!!!
You are great bruh I'd literally crown u as king.
Thank you so much for these videos, this is gold
Amazing videos!thankyou so much sir!! Much praise for your work
Hi there Ben!
I stumbled upon your channel because I wanna learn how to code.
I watched this video to see if it interests me though I don't have any knowledge yet. But boy! You explain things so well that I get the idea of what you are doing.
I am glad I've found your channel.
+1 sub!
Awesome, always happy to see new people giving code a try! Thanks for subscribing.
LOVED IT!
Please, please, please! I would love more tutorials like this!
Can we also make basic games like tower defense, basic shooter, or something similar?
Thank you for sharing your knowledge. Greetings from Brazil.
thanks for these 9 masterpiece videos
Thanks for the tutorials, I like all of them, it's much more fun to learn coding/new libraries if the examples are interesting topics too in this case, games! keep up the good work, looking forward to future tutorials, OpenCV looks awesome.
Yes, that's the idea of my channel! Thanks!
Your videos are super clear. Thankyou :)
great stuff. no idea why the views are not in millions. kudos
Thank you Ben , very interesting content !
thanks i will watch ur vids and learn this library with u
keep up the good work
Great content as always!
Thanks Shane!
Hey man just want to let you know that I'm so happy right now as a gamer and python programmer :) You just earned a new subscriber!
Glad other people like this stuff too!
hey this was insanely helpful, thank you very much
Amazing explanation!
Can't wait to see where this goes
Same! I need to slow down on coding and catch up with making the videos.
ความรู้ทั้งนั้นขอบคุณมากครับ
thanks for the clear explanation!
thank you so much Ben ! I viewed your video, red your web site, and i have done my new project in few minutes !
good video and a writeup! You are doing a great job!
Thank you for the video :)
That s very interesting
Keep making videos. This is awesome!
Thanks dude, I want to make a very simple bot to automate a PIA element for a really old game and this is a good excuse to learn Python a bit more
Open CV is great no denying but the library is just huge , it's full of functions to do all
sorts of thing , this is a great advantage(keeps ur code short and neat) and disadvantage
( I really find it difficult to remember these lol)
like think about it for a sec, there is function for like literally everything and parameters which makes no sense
for newbies (like me myself), but nevertheless great video mate , really liked how u walked through
the documentation (encouraging viewers to explore) and explained as much as possible while writing
the code and made the video as short as it possibly could have been u certainly deserve more subs
and recognition mate :)
Great playlist
print("Hello from Brazil")
Thx bro
Wonderful video!
Your a legend mate thank you so much!!!!!!!
🙏 Thank you sensei!
Nice video!
Good for starting programming a mmorpg bot!
Great tutorial, thanks!
I just did the sentdex opencv tutorial so this is perfect
Awesome hope it's helpful for you
@@LearnCodeByGaming Stardew valley is about to get more interesting.
Thank you for your videos, you are very good, I have received a lot of inspiration
Thank you!
ur the goat @code by gaming
just found good stuff on yo channel, Thanks!
Very good, thx for your videos
Great video!
awesome, thank you
Great Video!
Nice Need More Like This
good explanation, ty
Nice this verry useful for me 👍
Hi, love the vids! Hope you're doing well. Keep up the content. Cheers.
Thanks Ron!
So helpful
great explanation !!!
You're cool, thanks.
Very nice Video!
Amazing
great tutorial