This is what I needed, I was doing the method with many sample of the same resource in hope of finding it but due to light variation depending of the area it wasn't very reliable, now it's finally recognize 95% of the time the resources. Thanks you very much!
Hey Ben. I have watched my share of tutorials and followed dozen of online courses but you took it to the next level. Not only, you are very clear in your explanation but the format and the accuracy of your videos are mind-blowing. In a couple of hours, I've built an object detection tool combined with OCR features. Thank you so much for this great content. Keep up the great work.
I really enjoy your videos they are good quality and have good explanation and I’ve always enjoyed trying to automate games so this channel is just perfect
SendMessage is the best way I've seen. But it is not as reliable. Games that use DirectInput don't work. SendMessage(WindowHWND, WM_LBUTTONDOWN, MK_LBUTTON, MAKELPARAM(p.x, p.y)); Sleep(delay); SendMessage(WindowHWND, WM_LBUTTONUP, 0, MAKELPARAM(p.x, p.y));
even throwing it on a virtual machine is terrible, the performance is so bad that you can't get anything useable, unless I figure out how to add more gpu power to it. Otherwise I solved my problem by just buying a couple cheap machines and throwing my scripts on them.
Thanks so much for doing this series! With the game I'm testing out, I've noticed that the thresholding update actually makes my matching worse. I'm not sure if I need to find just the right Template Matching method and HSV settings, but it certainly is worse. I'm considering having a gnarly for loop for debugging which goes over all possible combinations, and prints out the combination which spits out the most high quality matches.
Keep the great work!, Do you think you could make a tutorial for implementing ocr? I'm having troubles trying to implement pytesseract because it runs in 1fps and it doesn't read correctly some areas
@@LearnCodeByGaming I used tesseract as ocr but the reability of this is sometime low... so I created and trained a model to read numbers in screen for me.
ok, mr wobble, very funny intro, i laughed so hard when u catually did, what you were speaking of :) just awsome edit4: everything works now, except for the recognition xD
Sadly windowcapture does not work for browsers and I don't know enough to fit something that does. Edit: This can work with browsers if you disable hardware acceleration!
Hi Ben! Your lessons are great!! Well structured and easy to follow your thoughts! Thank you very much for your efforts and best regards from Ukraine!!
Hello, Ben! Thank you for sharing your knowledge! Have you considered using SIFT? You would get rid of scaling and rotation problems. I think it can also beat the day and night cycle problem. For different shaped rocks, you can use multiple samples and loop them.
No I didn't know about SIFT, thanks for pointing it out! I'm going to try it out and if I can get it working well I'll make a video about it, or maybe put it in with the next video. Thanks again!
Thank you! I started by making websites on Geocities, and I just kept working on projects from there... like a Half-Life mod in C++, TI-83 calculator games, and I started building websites for people for like $20. I also took all the computer classes I could in school.
I would first like to thank you for the very well done tutorial. While watching your video I had a doubt, as in this game there is a day / night cycle, the hsv filter depending on the time of day will not be without working at some times due to the lighting of the game? otherwise, would it be worth doing a CNN to be able to recognize the image in black and white and predict whether it is a collectable resource or not? Congratulations again for the content!
Loving it! Ben, may i ask you to create something with game and machine learning? Dunno, maybe harvesting or an bot that improves any job that you gave to it. Thanks again for the videos and series.
Awesome, great video! what about using Tensorflow to help achieve our goals? would that work? Also, in the next video could you please show us how we can tell it to click on the object it finds and to loop it?
Thanks! Yeah, machine learning can work even better for this, but I'm gonna save that topic for another series. I'll definitely show how to click on objects before the end of this one.
That's a very good option, the issue is collecting the data, you'll probably need to take dozens, if not hundreds of screenshots of the objects you need to find (taking into account all the variations (shape, rotation, day/night, etc.), annotate the coordinates/shape/bounding box for each one, and feed that to the training algo.
OpenCV works best with static stuff, browsergames that are based around UIs with lots of buttons and so on and less frequent changes and diversity in images. In cases like Albion i would always resort to try to either hook into the client (against the TOS obviously) or dissect the client, get the de&encryption methods for the networkpackets and analyze the traffic to find objects in games or even send commands. It's way more reliable and efficient. With imagedetection you won't be happy on the long run if the aim should be to write game automation.
Thanks! If the health bars are always at the same position on the screen, you could just check the pixel color at certain places along the bar. Finding the type of health bars that float above characters heads would be more challenging, but it could be done with OpenCV.
@@LearnCodeByGaming Hello. Thanks a lot i came back reciently to make a bot for a game called Brawl stars. i successfully managed to detect everything on the map by the health bars and just making a rectangle around the player thats under the health bar. It detects everything and works at 90 fps which is insane. Time to find a way to make it play and detect walls! Thanks a lot for your videos they are insanely valuable!
You have to make an exe that integrates with the game by finding the automatic pid, exe then automatically creates the image folder where all the screenshots go...
I am confused, why do you need to add a separate adjustment for VAdd, SAdd, etc for addition and subtraction purposes? Cant you adjust by moving the trackbar that you created earlier?
Hi Ben, really love you tutorials! You helped me a lot in improving my python skills. However I'm a little stuck with object detection. Let's say you would not only want to find Limestone, but also two or more other stone types. I tried to import my pictures into a list and then repeat the "find" function with every object. But i think this is not the fastest way possible. What are your thoughts?
wassup dude i wanna ask u something, how can i make a ai or some stuff like that 2 run the map? like, idk if u played tibia, but on elfbot have an option called cavebot and u can set some checkpoints and ur character will run around them, andi'm trying 2 do something like this but i just failed... anw, nice vid dude
Thanks. I'm going to show how to integrate the character movement with these vision tactics in the last part of this series. Is there something in particular you're struggling with that I should cover?
I had a lot of errors and other stuff which i mange to fix on my onw till now when i get error "AttributeError: 'Vision' object has no attribute 'draw_rectangles'" and i have everything same as you :(
When I add the shift_channel part of the code the colors on the screen are all screwed up. Even the sliders for sAdd and so on work as expected, just that the base colors are already wrong. Without the shift_channel code the colors are normal. Wonder what I did wrong :(
Hey man, I know it's been a year but I just had the same problem and ended up fixing it. Double check you haven't put the wrong values inside of - lower = np.array([hsv_filter.hMin, hsv_filter.sMin, hsv_filter.vMin]) upper = np.array([hsv_filter.hMax, hsv_filter.sMax, hsv_filter.vMax]) I had upper = np.array([hsv_filter.hMax, hsv_filter.vMax, hsv_filter.vMax]) instead :) Hope this helps someone.
Hey, thank you for the tutorials first of all. I have a question, do you have any idea how I can make a bot to press the letters that appear on screen? The words appear in different colors each time. I don't really know a solution to this and I really want to make a bot for this game. I will leave a imgur link with some screenshots here so you can understand better what I mean. Also there are more colors that appear like red or violet, but I don't have screenshots with them. Thank you very much and good luck! Imgur: imgur.com/a/CDS9QdU
You'll want to look into Optical Character Recognition (OCR) to do this. I've researched Tesseract OCR a bit, but haven't used it in a project yet. I get this request a lot and I'll eventually make some videos about it. But yeah that's where you should start.
@@LearnCodeByGaming Hey thanks for coming back with a response! I tried 3 different ocr alghoritms but they all failed(about 20% accuracy). I took a trained tensorflow model and trained it with my data and i get about 90% accuracy with it
I tried to run this and sadly it is not working for me. Is it due to a newer version? I am getting "error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims()
try changing the flags when you read an image, COLOR_RGB2BGR... it depends on how the image was saved, if RGB or BGR. img = cv2.imread(path, cv2.COLOR_RGB2BGR)
It has been two years already, where is the continuation? Why did you abandon your channel? You explain better than all my teachers! You're the best! It has been 2 years already, perhaps you have improved object detection, bot performance, and their movement between points, or maybe there's something else interesting to see!
Nice, but obvious need a lot of experimentation.. And fact is this... google a lot. I don't know if it's possible to solve so much without others help :))
This is what I needed, I was doing the method with many sample of the same resource in hope of finding it but due to light variation depending of the area it wasn't very reliable, now it's finally recognize 95% of the time the resources.
Thanks you very much!
Hey Ben. I have watched my share of tutorials and followed dozen of online courses but you took it to the next level. Not only, you are very clear in your explanation but the format and the accuracy of your videos are mind-blowing. In a couple of hours, I've built an object detection tool combined with OCR features. Thank you so much for this great content. Keep up the great work.
And he I'm again. Nice video buddy!!! I'm using what I learned before with you in one of my "big" projects. And what I learned here will too
So cool, good luck with your project!
These videos are making my summer! I hope you cover multiprocessing/threading for some sort of interfacer that can run alongside the vision script!
Awesome man! Yeah I'm going to cover that in the final video.
Simply the best tutorial for learning from coding, big thx 😊
Hands down best tutorial series I have ever followed
Loving the great work, keep on going!
Thank you!
omg dude, you really are a genius. keep it going man.
The best programmer I have ever seen. Thank you so much!
I really enjoy your videos they are good quality and have good explanation and I’ve always enjoyed trying to automate games so this channel is just perfect
Glad you enjoy them, thanks for watching!
It has been two years already, where is the continuation? Why did you abandon your channel? You explain better than all my teachers! You're the best!
Your channel is really underated, great work man!
Amazing series!
could you do a tutorial on how to click in a minimized windows while you are using the computer doing other thins ?
This is probably the request I get the most. Once I figure out the best way to do it I will definitely make a video about it!
SendMessage is the best way I've seen. But it is not as reliable. Games that use DirectInput don't work.
SendMessage(WindowHWND, WM_LBUTTONDOWN, MK_LBUTTON, MAKELPARAM(p.x, p.y));
Sleep(delay);
SendMessage(WindowHWND, WM_LBUTTONUP, 0, MAKELPARAM(p.x, p.y));
even throwing it on a virtual machine is terrible, the performance is so bad that you can't get anything useable, unless I figure out how to add more gpu power to it. Otherwise I solved my problem by just buying a couple cheap machines and throwing my scripts on them.
Thank you for all the effort provided. You just earned a new subscriber!
Thanks so much for doing this series! With the game I'm testing out, I've noticed that the thresholding update actually makes my matching worse. I'm not sure if I need to find just the right Template Matching method and HSV settings, but it certainly is worse. I'm considering having a gnarly for loop for debugging which goes over all possible combinations, and prints out the combination which spits out the most high quality matches.
Automating that testing sounds like a great idea!
i guess Im kinda randomly asking but do anyone know a good place to stream new tv shows online?
@Makai Yosef Ehh I would suggest Flixportal. you can find it if you google=) -lian
@Lian Javier thanks, signed up and it seems like they got a lot of movies there =) Appreciate it!
@Makai Yosef glad I could help :)
The explanations are on another level! keep up the good work!
Can't wait for part #7. This series is awesome. Any plans to use YOLO in future projects for better object detection?
I'm not going to cover YOLO in this series, but it's on my radar for future videos. Glad you like these!
@@LearnCodeByGaming still waiting for the YOLO serie it would be awesome
Loving this series!
Keep the great work!, Do you think you could make a tutorial for implementing ocr? I'm having troubles trying to implement pytesseract because it runs in 1fps and it doesn't read correctly some areas
Thanks! I've got OCR on my list of topic ideas, and quite a few people have asked about it. I'll bump it up in priority.
@@LearnCodeByGaming Great ty! :D
@@LearnCodeByGaming I used tesseract as ocr but the reability of this is sometime low... so I created and trained a model to read numbers in screen for me.
@@OFadigas Yeah pytesseract is the easiest way to get started, but you can get way better results with machine learning.
@@LearnCodeByGaming If you know abou ML and want to teach, I will aprecciate a lot (to make my knowladge base stronger)
ok, mr wobble, very funny intro, i laughed so hard when u catually did, what you were speaking of :)
just awsome
edit4: everything works now, except for the recognition xD
Great video and good work with your editing.
Sadly windowcapture does not work for browsers and I don't know enough to fit something that does.
Edit: This can work with browsers if you disable hardware acceleration!
Hi, I can't find in windows how to disable this, how did you do it?
Great Work, Thank you !!
Are you still active on this channel Ben? Your content is next level...I really appreciate your contributions!
you're on another level
You sir, are a legend
Hi Ben! Your lessons are great!! Well structured and easy to follow your thoughts! Thank you very much for your efforts and best regards from Ukraine!!
Thank you!
Hello, Ben! Thank you for sharing your knowledge! Have you considered using SIFT?
You would get rid of scaling and rotation problems. I think it can also beat the day and night cycle problem. For different shaped rocks, you can use multiple samples and loop them.
No I didn't know about SIFT, thanks for pointing it out! I'm going to try it out and if I can get it working well I'll make a video about it, or maybe put it in with the next video. Thanks again!
@@LearnCodeByGaming Alright! If you need help with it (opencv tutorials need to be updated on it, along with ORB, AZAKZE and such), let me know!
Very nice work! But one question, what happens if we want to to detect multiple and different objects?
Great Video! How did you learn to program?
Thank you! I started by making websites on Geocities, and I just kept working on projects from there... like a Half-Life mod in C++, TI-83 calculator games, and I started building websites for people for like $20. I also took all the computer classes I could in school.
I would first like to thank you for the very well done tutorial. While watching your video I had a doubt, as in this game there is a day / night cycle, the hsv filter depending on the time of day will not be without working at some times due to the lighting of the game? otherwise, would it be worth doing a CNN to be able to recognize the image in black and white and predict whether it is a collectable resource or not?
Congratulations again for the content!
Loving it! Ben, may i ask you to create something with game and machine learning? Dunno, maybe harvesting or an bot that improves any job that you gave to it. Thanks again for the videos and series.
Epic stuff right here, What's the processing technique?
Thanks! We're going to look at canny edge detection in the next video.
Wooooo, waiting for more cool projects..
Awesome, great video! what about using Tensorflow to help achieve our goals? would that work? Also, in the next video could you please show us how we can tell it to click on the object it finds and to loop it?
Thanks! Yeah, machine learning can work even better for this, but I'm gonna save that topic for another series. I'll definitely show how to click on objects before the end of this one.
That's a very good option, the issue is collecting the data, you'll probably need to take dozens, if not hundreds of screenshots of the objects you need to find (taking into account all the variations (shape, rotation, day/night, etc.), annotate the coordinates/shape/bounding box for each one, and feed that to the training algo.
OpenCV works best with static stuff, browsergames that are based around UIs with lots of buttons and so on and less frequent changes and diversity in images.
In cases like Albion i would always resort to try to either hook into the client (against the TOS obviously) or dissect the client, get the de&encryption methods for the networkpackets and analyze the traffic to find objects in games or even send commands. It's way more reliable and efficient. With imagedetection you won't be happy on the long run if the aim should be to write game automation.
So i am doing this but with world of warcraft but it only shows a small black window don't know what to do can you help me ?
Love this. Thank you.
Hey, nice Video!! What vs code theme are you using?
Great video ! is there a way to detect health bars ?
Thanks! If the health bars are always at the same position on the screen, you could just check the pixel color at certain places along the bar. Finding the type of health bars that float above characters heads would be more challenging, but it could be done with OpenCV.
@@LearnCodeByGaming Hello. Thanks a lot i came back reciently to make a bot for a game called Brawl stars. i successfully managed to detect everything on the map by the health bars and just making a rectangle around the player thats under the health bar. It detects everything and works at 90 fps which is insane. Time to find a way to make it play and detect walls! Thanks a lot for your videos they are insanely valuable!
Great job 👍
You have to make an exe that integrates with the game by finding the automatic pid, exe then automatically creates the image folder where all the screenshots go...
I am confused, why do you need to add a separate adjustment for VAdd, SAdd, etc for addition and subtraction purposes? Cant you adjust by moving the trackbar that you created earlier?
hey ben ive been following your tutorial but the trackers don't show up and it says I need a configured python 2 sdk to render epydoc
Hi Ben,
really love you tutorials! You helped me a lot in improving my python skills.
However I'm a little stuck with object detection.
Let's say you would not only want to find Limestone, but also two or more other stone types.
I tried to import my pictures into a list and then repeat the "find" function with every object.
But i think this is not the fastest way possible. What are your thoughts?
What changed that increased the FPS from 6-7 to 25-26?
wassup dude
i wanna ask u something, how can i make a ai or some stuff like that 2 run the map? like, idk if u played tibia, but on elfbot have an option called cavebot and u can set some checkpoints and ur character will run around them, andi'm trying 2 do something like this but i just failed...
anw, nice vid dude
Thanks. I'm going to show how to integrate the character movement with these vision tactics in the last part of this series. Is there something in particular you're struggling with that I should cover?
@@LearnCodeByGaming Cant we like use a pathfinding algorithm and somehow fuse it with the map?
I had a lot of errors and other stuff which i mange to fix on my onw till now when i get error "AttributeError: 'Vision' object has no attribute 'draw_rectangles'" and i have everything same as you :(
Good job)
kool! thanks!
When I add the shift_channel part of the code the colors on the screen are all screwed up. Even the sliders for sAdd and so on work as expected, just that the base colors are already wrong. Without the shift_channel code the colors are normal. Wonder what I did wrong :(
Hey man, I know it's been a year but I just had the same problem and ended up fixing it.
Double check you haven't put the wrong values inside of -
lower = np.array([hsv_filter.hMin, hsv_filter.sMin, hsv_filter.vMin])
upper = np.array([hsv_filter.hMax, hsv_filter.sMax, hsv_filter.vMax])
I had upper = np.array([hsv_filter.hMax, hsv_filter.vMax, hsv_filter.vMax]) instead :)
Hope this helps someone.
Hey, thank you for the tutorials first of all. I have a question, do you have any idea how I can make a bot to press the letters that appear on screen? The words appear in different colors each time. I don't really know a solution to this and I really want to make a bot for this game. I will leave a imgur link with some screenshots here so you can understand better what I mean. Also there are more colors that appear like red or violet, but I don't have screenshots with them. Thank you very much and good luck!
Imgur: imgur.com/a/CDS9QdU
You'll want to look into Optical Character Recognition (OCR) to do this. I've researched Tesseract OCR a bit, but haven't used it in a project yet. I get this request a lot and I'll eventually make some videos about it. But yeah that's where you should start.
@@LearnCodeByGaming Hey thanks for coming back with a response! I tried 3 different ocr alghoritms but they all failed(about 20% accuracy). I took a trained tensorflow model and trained it with my data and i get about 90% accuracy with it
@@estebeste4045 Nice!
I tried to run this and sadly it is not working for me. Is it due to a newer version? I am getting "error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims()
try changing the flags when you read an image, COLOR_RGB2BGR... it depends on how the image was saved, if RGB or BGR.
img = cv2.imread(path, cv2.COLOR_RGB2BGR)
if you're using .png try refactoring it to .jpg
Do you got a Discord? I wold love to play a Game with u :D
Sorry I don't have much time to play right now, trying to get these videos out! What games are you playing?
@@LearnCodeByGaming if you don't have much time to play, we should set up a server to have our bots play each other. great videos by the way!
It has been two years already, where is the continuation? Why did you abandon your channel? You explain better than all my teachers! You're the best! It has been 2 years already, perhaps you have improved object detection, bot performance, and their movement between points, or maybe there's something else interesting to see!
Sir
Make a video on clash of clans
Nice, but obvious need a lot of experimentation..
And fact is this... google a lot. I don't know if it's possible to solve so much without others help :))
All thanks to Dr Alaho Olu on TH-cam for curing me and my boyfriend from HSV 2, I’m so much excited.
I am little bit offend is he blind?