Since this video has been getting a lot of views suddenly I figured I'd leave an update. Firstly, thank you to everyone for the overwhelmingly positive support on this video, I never expected it to get this many views. I really enjoy reading all of the comments and seeing everyone's suggestions / improvements that could be made to this project. The only thing I've done with this project in the past year is that I expanded it to work on the entire world and played some battle royale games with it. The results were interesting, I just never got around to making a video for it. I've wanted to set aside time to make more videos, and seeing the support on this video is definitely motivation to do so. Thanks to everyone who has subscribed in spite of the fact that I haven't uploaded in almost a year, I'm hoping to make more videos soon. Also, yeah I know the mic sucks in this video. I have a new one which should hopefully work better in the future.
"However, some of the best geoguesser players can guess the location with only a single image and a couple seconds. This makes me wonder-" me: this shit makes me wonder if they could doxx me just taking a picture of a gas station near by-
watch GeoWizard's series "guessing the exact locations of my fans using one single photo" his fans send him a photo of themselves and he tries to guess the exact location. it sometimes take 10 minutes, sometimes some hours, but he usually always gets it right.. but yeah, their photos are rarely a random house or something, so it helps. but still !
doing smaller countries could be interesting considering how a lot of the areas would look the same-ish it would be interesting to see how good the ai guesses the areas
@@lederp42 Given enough flags in the training set it would definitely be able to. Flags are basically rectangles with pastel colors, super easy to recognize for a computer ;)
Love this AI, well done! I may have a possible improvement. Averaging all cells to formulate your guess moves the average guess towards the middle, always. Your guess distribution will become guassian like with the center of the distribution being the middle. Example: the goal is washington, any contribution from any of the other states pulls the guess towards the center. The only way to guess an state on the edge of america correctly is when that is the only one having a probability. This is why your guesses are generally to far towards the center, not because of the pictures you've taken. I suspect that using the center of the max cell for a smaller grid has a better performance ceiling (if you would want to continue). Other than that great work and an inspiring video!
I was about to comment the exact same thing. You can also see that the prediction sometimes is in a box with a lower confidence score, even though there exists a box more to the edges of the map with a higher score. I assume this is caused by the same bias towards the middle of the map.
Another way to solve this is to include in learning set border parts of Canada and Mexico. When output is in the wrong country it will point to closest location in US, but probably it won't happen as you point out that it's always moving towards center. Problem is with states near ocean tho.
On top of that, this averaging also assumes that the distribution is a single gaussian. It's possible that an image depicts some part of the East Coast, but its features is similar to a region on the West Coast. The distribution is then disjoint, with the network predicting a high probability on both the East Coast and the West Coast, but averaging this distribution will incorrectly produce a guess at the center of the US.
@@zenith5180 well I think this argument is not as strong as one might first think, and I want to explain why: In such a case you suggest a good gest would be either east or west coast getting about a 50% chance to have either a good or a desasterous guess. The AI's guess would be bad in both cases but not desasterous, do you still have a 50% chance being better or worse than the AI's guess. Probably your strategy still is the better, but that is due to the reward function of Geoguessr, which I don't know anything about but probably decreases the reward points exponentially with distance from the target or sth. similar. Now imagine a different case, where you have this becomes even more trickier: you have 3 fields, each have the same likeliness of ⅓. These fields are disjoined and form a equilateral triangle. If you go full risk for one of these fields while the AI guesses for the center of the triangle, the AI has a better guess distance wise in ⅔ of the cases. What this means for the expected value of reward points depends even more on how Geoguessr's internal distance2target->points function. Therefore, I think the general idea of using not just the field with the highest likeliness but something, let's call it 'map awareness' is not wrong. However, a simple averaging is not the best idea, as explained in the previous comments. Personally, I think based on the AI's predicitions one could compute a final regularizes guess coordinate as the guess in a little bit more complicated way: one cross-relates the probability of a field with each of the other fields by considering also the distance between those fields. For example you have field A with a high likelihood and directly east of it field B, also with a high likelihood. This would be a clear cluster, and since the distance between the both fields is minimal the two high likelinesses would push each fields overall likeliness. If field A and B are from the east and west coast, the great distance between them would prevent, that they buff each others likelihood. And if field B has a not so high likelihood, it would buff the likelihood of field A just very weakly. Ofc, the likelihood values are not adding up to 1 anymore, that's why I'm not using the term probability here. However, the higher this value would be, the more likely is this region a good guess. I think that way, one would make improved guesses. Also, one can experiment if these values, likelihood and distance between different fields, should enter the problem just linearly or results improve, if the predicitions depend on these values in a squared/etc. fashion.
Hahaha. The "official" United States map on Geoguessr is notoriously bad and those are the usual locations you get in the middle of nowhere. The "United States (no blurry)" map is so much better.
Have you considered placing the guess exclusively within the box the AI is most sure about, and then determining where you put the guess (still within that one box) based on the weighted average? Meaning the guess would always be in the most likely box. How do you think that would affect the results?
Guessing from the video results, I also had the impression that this would be more accurate. It seems that the unsure boxes are skewing the guess away from the predicted location. More so when the prediction is close to the edges of the country and there are a lot of unsure boxes to the other sides.
My thought exactly. As soon as he announced his guess averaging technique I was skeptical of that being a good choice due to the nature of coastal areas looking similar...but being on opposite ends of the map.
Since Google's Street View pictures are taken in a road, an improvement would be done if the prediction were moved to the nearest road in the direction of the highest confidence rectangle:)
@@mst8819 The AI will not learn to do so given enough time, in it's current setting anyway. It outputs confidence values for each of the boxes. The output of each box is just that, it won't be able to adjust towards roads just like in the animal example it will not suddenly output what part of an animal is in the picture, e.g. "ear of fox", unless you trained it that way in the first place
Your newly 6th sub here. I'd love to see a new model perform on Geoguessr's earth map :D Btw how did you collect your image data from Google's streetview? What tool did you use? Also do you recommend any other books/courses on CNNs? Keep it up man!
Thanks for the sub! I collected all of the images using Google's Street View Static API developers.google.com/maps/documentation/streetview/overview and collected images from random latitudes and longitudes within the US. The resource I linked in the description developers.google.com/machine-learning/practica/image-classification is probably the best resource I found for understanding the fundamentals of CNNs and feature extraction. Its exercises are also pretty useful for seeing how to implement CNNs using TensorFlow. For this project in particular, the Keras Applications documentation keras.io/api/applications was useful for finding and using a pre-trained model for feature extraction.
For world mode, trying to perform an "average" over the international dateline may prove to be difficult. But it would be so cool to see it done anyway! perhaps choose the center of the max probability box? is that better or worse? if you choose to go further and get more accurate, there is the problem of training data. If you can somehow make fine enough grids, you might be able to produce more accurate results, assuming more data also, if you take multiple photos of the same area, you could possibly get a better idea of the place - like take a few photos, then get the average of the outputs or something. i love it !!!
There is some mathematics ionvolved in picking the avertage of a set of points on a sphere. In particular, it is not necessarily unique. Think for example of the mean of the north and south pole, every point at the equator is equally reasonable as a mean, anyway the average of two opposite points does not make that much sense. A way around this problem would be first picking the continent which is most likely and then taking the average over this landmass, islands would still be a problem. So a better solution is to just take a region around the box with the highest probability and consider only boxes lying in that region. Also the shape of the boxes is not square anymore, so we should multiply the weight by the size of the box. The problem of training data can be adressed by automatization. Google street view should provide a suitable interface that can be used to automatically choose a random spot and generate some pictures. Using multiple pictures would have a great potential for improvement.
That's difficult, because what we train here is an image classifier and therefore we need predefined classes (our boxes) before (!) we start training. Since we want to use the same AI for each location we need the same boxes everywhere.
@@jakobm.4183 for a sphere, you could take the coordinates of a sphere, then turn them into a vector from the center of earth and average that - but since not all grid spaces are going to be used (for example, oceans), then there's going to be an implicit bias. Your method is likely better.
im sorry if this is dumb but couldnt you just use a slightly incorrect mapping? there would be bias towards some areas but you could just use a flat map of the earth then relate that point to the same point on the sphere
The problem with the square method is that, for example, the square that contains death valley also contains a number of ski lodges that are in a completely different climate
I personally feel like the weight for the highest confidence square should have been higher for best results, maybe the conf level itself as a weight for all (or an equivalent system the result falls inside the top confidence squares)
Some suggestions: 1. About pushing to the center, you can tweak weighted average algorithm giving it weight in power (eg instead of 0.5, give it 0.5^2=0.25). It will keep resulted dot closer to darker areas 2. Write script that will automatically parse geogessr page, predict location and answer using script. This will make everything easier for you 3. Give AI whole 360 image instead of small part of it, it will give it more data that AI can work on 4. Play with size of boxes, it is possible that smaller squares could get better results And thanks for the video, it was interesting to watch!
Such a fascinating video, it would be really cool to see the updated version in action taking on the whole world! Take as much time as you need with your next video, me and many others are looking forwards to it 😅
it seems to me like using a weighted average of the various patches is maybe making a mistake? for an example of the sort of mistake that would encourage, imagine an image that is definitely inside a city, but you're not sure which city among a few candidates. averaging the longitude and latitude of each city quite possibly gives you some location in the middle of a field far away from anything urban. depending on the geogueser scoring system this is maybe still a score optimizing strategy, going for consistent but low score over a high risk but very high score if you actually get the city right? but it's just a question you've got me chewing on a bit, now, i suppose.
I had a similar thought, but for a different reason. Using averaging will negatively impact locations near the borders, since it can only average it away from the border. Would be interested to see how a "naive" AI (center of most probable square) would compare. Still a good video though!
I think the same. Average can't be used with terrain. Pictures of the coast would average to the middle of the country. Sides of a mountainous region would average to the mountainous region.
Using weighted average does make it statistically better than the average, though. Seeing as the lowest score possible would be to choose a point on the coast when the correct location is on the opposite coast, this possibility is eliminated when the AI will only choose a point closer to the center. Therefore, even when the AI is wrong, the way the point is calculated will still give it a better score on average. In conclusion, using a weighted average is a msitake because it's sorta like cheating, and makes it hard to see if the AI is actually doing well.
Very interesting conversation here! I think you all bring up reasonable points. Which method is preferred also depends on how the score in geoguessr is calculated. Take the opposite coast example: if the distance from the middle to one coast is too far to let geoguessr give you any points, you might as well make a guess between one of the two coasts and have a 50% chance for a lot it points, instead of 0% chance at any points when taking the average. What I keep thinking of is that fact that this question should actually be optimized *by the AI itself*. If you could train it based on the actual score it would get in a game, that would be awesome. Not sure how that would work technically, but from a theoretical view it would be really interesting!
This is really cool. You could make the AI much more accurate by somehow plugging it into GeoGuesser, and letting it navigate the environment until it found an image/view which it could match at a very high confidence. I'm not sure how easy that would be unless GeoGuesser has an API, though .
From what I'm seeing, the AI's most confident guess on where the location is is quite accurate, opposed to the weigthed average calculation of the AI's guesses that you are doing.
Oh wow, this was super interesting! I think the AI could definitely give GeoWizard a run for his money at his own no moving allowed 10-state challenge format. Maybe reach out to Tom and see if he's up for a head-to-head game?
Yess, finally im early to see a underrated channel that is probably gonna grow so fast before it has 10k subs. Todays date is 03/08/2021, time is 16:37 and this channel 6,37k subscribers. This video has 416k wiews and the channel has only 2 videos. Other video has 95k wiews. im sure this channel is gonna earn at least 140k subs in the next 2 weeks. if he keeps uploading videos, this channel probably would be a lot bigger. Love your videos! Please keep uploading :)
I think you can make the boxes smaller, use more training images. Then take 3 screenshots per location. Make AI predict per screenshot, and use the 3 predictions to triangulate.
It's pretty cool to see a video about some really complex AI. it's even cooler when the Geoguesser chooses a spot right in front of your old house at 12:43 and you can see what your house looked like 8 years ago. Those trees outside are long gone now. Very cool!
Really cool. You could create one for the new battle royale mode where you just need to guess the correct country. Could use road markings and road signs for the training data.
This is like one of the only videos I can find on someone creating an AI for GeoGuessr. I've had this similar idea ever since I first discovered GeoGuessr a few years back. Very cool stuff! Now I'm tempted to try doing this.
Great job my dude! I thought it was close to impossible to code an AI for GeoGuessr and yet you did it in a surprisingly straightforward fashion. I respect people who code AI and Neural Network stuff since I couldn't do that myself...
Great video! I think it would be interesting to retry this with a different output. You are doing multi-class classification and then weighting the outputs. Rather than making it a classification problem, you could make it a regression problem and directly predict latitude and longitude. Then make the loss the sum of the mean squared errors over both the lattitude and longitude. That would remove the “centralizing” bias you’re getting from weighting.
Nice simplification of the classification task! I think a simple improvement could be, that the final prediction must always be inside the highest probability cell, but the precise position is moved by the weighted average.
The fact that it didn’t even have access to the 360 images is impressive too, an AI that can accurately predict a location based on a still image is kinda spooky ngl
The fact you only chose very rural areas with almost no defining features is very tough, but cresting a more advance AI it may be able to learn building styles, road signs and street names for urban areas
Soooooo. Your AI guessed pretty much exactly where I used to live on the one where it landed in Idaho. The on-screen coordinates showing on the video for this round were still for the previous round. However, I used to live in Idaho and even from the short glimpse on the map, I thought to myself... REXBURG!!! Then I noticed the Texas Slough on the map next to the AI waypoint and I knew, without a shadow of a doubt, this is very close to my hometown (South Madison Country, Rexburg, ID). I zeroed in on your coords with similar methods that you described in the video and landed on the coordinates of 43.727127, -111.829871. I then used that to calculate the distance to my original home address. The AI "thought" you were within 5 miles of where I used to live! What's even crazier, is that I moved to WA State which just happens to be within about 200 miles of where the ACTUAL location is. Trippy huh?
I find it especially impressive considering the AI only has a single image to base its guess off of, while a human player can scroll along the road and turn the camera around
Cool vid! I think your square weighting method may be slightly flawed, because it means that the AI is very unlikely to guess correctly when the actual location is on the edge of the country. This is because the average of all squares without weighting would give a dot in the centre of the US, naturally drifting all guesses to the centre of the country.
It's not about the edge vs the center, it's in general that it cannot sensibly represent ideas such as Florida and California being likely and Texas unlikely. It assumes a uni-model distribution of confidence over the map.
If it's a linear relationship between distance and points, then the best way is weighted average. But that is not true (more something like 5000e-^(x/2000). Another point would increase the expected number of points, that point one can derive from the score to distance formula.
Ideas for improvement (maybe they are feasible): Implementing periodic boundary conditions for the average confidence of the position. Using the whole 360° Image instead of the usage of the image selection. I just found your channel. Interesting videos, keep it up!
you should try this with the entire world. i reckon the results will be better as the margin for error is slightly bigger. great video though i'll turn my notifications on for your next one/
Loved the video and really surprised by how well the AI did with so little information! Judging by the comments your video seems to be gaining some traction, that's well deserved ! Hoping more people see this !
This is great. I want to see the ai play a diverse world map, country streaks, and if there is there a way to make the ai more accurate. Thanks for the nice video!
Really interesting - have another new sub! A few random thoughts that occur to me on this. "Extremising" the probability outputs might help improve the fit on the edges of the map in particular. For example, square the probabilities (then normalise) to skew the prediction towards higher confidence levels. For an all world system you'll probably have to drop or at least modify the averaging approach, as others have said. Maybe it picks the continent first? How did you choose the size of your boxes? Obviously smaller boxes would allow a finer prediction but it's more work to train. Given that human players can move around, perhaps you could give the AI more than 1 photo to base its prediction on.
I think taking the second power of numbers to determine the location would make this more accurate because it would make an ai more confident when he gets a higher percentage of chance for it to be in a specific state.
An idea that may improve performance. When you train it, instead of comparing the distribution predicted by the the AI with a delta distribution on the square where the coordinates belong, you can form a distribution whose weighted average is the coordinate of the point.
I think a critical note here is that the comparison to the pro's is not entirely fair. Your AI is trying to optomise score across all games, where as a pro knows to take risks and only keep the 1 lucky group of guesses. It would be cool to see a best attempt of a group of tests where the AI goes with its highest confidence box being much more heavily weighted.
Really impressive stuff! Perhaps could you share more about your code? Like how long it took you, some snippets, and even share it! Loved the vid! Congrats on the great AI performance!
Github link? I work in the computer vision field and if you're wanting to make this better, look into siamese neural networks. The same idea is used for good facial recognition.
May have been interesting to run the same photos again but with you choosing a different view/ direction and see if the AI came back with similar predictions. You choosing a view from each location may have represented a biased input to the AI. Great video and concept, clever bastard 😎
Super cool project! I can't wait to see what you do with it next, should you decide to continue. Once you get the average bias problem sorted out, it looks like you'll have a really powerful tool. With more training, the program could be used for so many applications. Well done!
Use any non-linear editor (Premiere, Vegas Pro, Final Cut, Davinci Resolve etc) and animate position (typically using keyframes; i.e. google for "[name of your video editing software] position animation"). Group images somehow, so you wouldn't have to animate each image separately. Or simply create one big image in graphical editor. Add a white backdrop. Here's a random tutorial that should give you grisp of everything you need to make it in Davinci Resolve: th-cam.com/video/kYOTDnjuitc/w-d-xo.html
I wonder if the AI learning would improve in accuracy if it was able to see the entire panoramic view instead of the limited square window? Also, does the difference in camera quality over different locations/times have any effect?
Chances are very real it's using meta strategies. There's entire websites on how you can narrow down where you are in geogeussr based on some characteristics of the footage such as errors in the stitching of the 3D image or dirt on the lens etc. Entire areas are covered by the same car/team so it could be possible that the AI has learned to recognize those artifacts as patches. The AI can't differentiate between picture artifacts and the rest of the photo, so it might very well just be recognizing that, for example, every street view image in a certain state or county has a certain scratch on the lens or whatever
It looks like you could compensate for the bias it has against guessing around the edges of the map. If you could ball-park how much on average it was guessing more centrally, you could adjust each guess by that figure outwards.
This is really really cool, personally, think it would be funny to see if the AI can be trained to be able to read signage and pinpoint where it is from given distances to other cities. I'm a bit biased, but the Netherlands is a good country to do such a thing with, as there are signs indicating directions and distances nearly EVERYWHERE, mostly because these signs can also be found on many cycling routes.
What would happen if you tried to output the lat/long values (maybe scaled into a range of 0..1) instead of tiling the (game) world? Feels like tiling is an unnecessary restriction on the algorithm.
I assume the reason this doesn't work well is because you would need way too much training data to guess at that high a resolution. The thing about tiling the country is it already recognises that areas which are close to each other look similar, whereas if you give the AI completely free rein it has to learn that itself
Now make the AI say "Get in!" after an insane guess
hahaha yes!
Make another ai to confirm every 10 mins that he is recording
@@ryleypalmer Manke another AI that narrates from "the future" if the real AI isn't recording
LMAOO
Lmao this made my day
I think this would be interesting to see on country streak mode, since the guessing method in that mode is similar to how the AI works
Where do I know you from? Your name seems familiar
@@itzspiro6047 Do you watch Reid Captain?
@@NStripleseven yes sometimes
@@itzspiro6047 yea roaxial is a part of that community
@@NStripleseven I want to be a part of that community just because it looks gregarious, u guys know each other!
Since this video has been getting a lot of views suddenly I figured I'd leave an update.
Firstly, thank you to everyone for the overwhelmingly positive support on this video, I never expected it to get this many views. I really enjoy reading all of the comments and seeing everyone's suggestions / improvements that could be made to this project. The only thing I've done with this project in the past year is that I expanded it to work on the entire world and played some battle royale games with it. The results were interesting, I just never got around to making a video for it. I've wanted to set aside time to make more videos, and seeing the support on this video is definitely motivation to do so. Thanks to everyone who has subscribed in spite of the fact that I haven't uploaded in almost a year, I'm hoping to make more videos soon.
Also, yeah I know the mic sucks in this video. I have a new one which should hopefully work better in the future.
First
Any chance you'll make your code available? I assume this is a tensorflow build?
Is there a tutorial for how you did this?, i'd love to do it but with a worldwide map but am unsure how I would get started with this
i personally would love to see a continuation of this video!!!!
@@austeyen5628 YES PLEASE
"However, some of the best geoguesser players can guess the location with only a single image and a couple seconds. This makes me wonder-" me: this shit makes me wonder if they could doxx me just taking a picture of a gas station near by-
watch GeoWizard's series "guessing the exact locations of my fans using one single photo"
his fans send him a photo of themselves and he tries to guess the exact location. it sometimes take 10 minutes, sometimes some hours, but he usually always gets it right..
but yeah, their photos are rarely a random house or something, so it helps. but still !
Yep, most likely, it isn't actually that hard to do but requires some work.
We can. I've found people by the view from their window.
@@Standplaats_Krakow It depends though. Not all pictures are foundable.
@@imutig3451 True, but there's often way more info than people think.
TH-cam algorithm chose you, congrats!
I smell some nepotism.
@@Spoof10 what????? lmaooo do you know what nepotism means??? you think this guy has a friend at youtube who gave him views???
@@samcousins3204 You enjoy your time on the ground, while the joke is cruising at 10000 ft above you.
@@Spoof10 or the more virgin way, R/Whooosh
@@Spoof10 wooosh
doing smaller countries could be interesting considering how a lot of the areas would look the same-ish it would be interesting to see how good the ai guesses the areas
Ireland
Vatican City
malta
Tunisia
@@lederp42 Given enough flags in the training set it would definitely be able to. Flags are basically rectangles with pastel colors, super easy to recognize for a computer ;)
This vid deserves much more attention. Good job on this.
Love this AI, well done! I may have a possible improvement. Averaging all cells to formulate your guess moves the average guess towards the middle, always. Your guess distribution will become guassian like with the center of the distribution being the middle. Example: the goal is washington, any contribution from any of the other states pulls the guess towards the center. The only way to guess an state on the edge of america correctly is when that is the only one having a probability. This is why your guesses are generally to far towards the center, not because of the pictures you've taken. I suspect that using the center of the max cell for a smaller grid has a better performance ceiling (if you would want to continue). Other than that great work and an inspiring video!
I was about to comment the exact same thing. You can also see that the prediction sometimes is in a box with a lower confidence score, even though there exists a box more to the edges of the map with a higher score. I assume this is caused by the same bias towards the middle of the map.
Another way to solve this is to include in learning set border parts of Canada and Mexico. When output is in the wrong country it will point to closest location in US, but probably it won't happen as you point out that it's always moving towards center. Problem is with states near ocean tho.
On top of that, this averaging also assumes that the distribution is a single gaussian. It's possible that an image depicts some part of the East Coast, but its features is similar to a region on the West Coast. The distribution is then disjoint, with the network predicting a high probability on both the East Coast and the West Coast, but averaging this distribution will incorrectly produce a guess at the center of the US.
@@zenith5180 well I think this argument is not as strong as one might first think, and I want to explain why:
In such a case you suggest a good gest would be either east or west coast getting about a 50% chance to have either a good or a desasterous guess. The AI's guess would be bad in both cases but not desasterous, do you still have a 50% chance being better or worse than the AI's guess.
Probably your strategy still is the better, but that is due to the reward function of Geoguessr, which I don't know anything about but probably decreases the reward points exponentially with distance from the target or sth. similar.
Now imagine a different case, where you have this becomes even more trickier: you have 3 fields, each have the same likeliness of ⅓. These fields are disjoined and form a equilateral triangle. If you go full risk for one of these fields while the AI guesses for the center of the triangle, the AI has a better guess distance wise in ⅔ of the cases. What this means for the expected value of reward points depends even more on how Geoguessr's internal distance2target->points function.
Therefore, I think the general idea of using not just the field with the highest likeliness but something, let's call it 'map awareness' is not wrong. However, a simple averaging is not the best idea, as explained in the previous comments.
Personally, I think based on the AI's predicitions one could compute a final regularizes guess coordinate as the guess in a little bit more complicated way: one cross-relates the probability of a field with each of the other fields by considering also the distance between those fields. For example you have field A with a high likelihood and directly east of it field B, also with a high likelihood. This would be a clear cluster, and since the distance between the both fields is minimal the two high likelinesses would push each fields overall likeliness. If field A and B are from the east and west coast, the great distance between them would prevent, that they buff each others likelihood. And if field B has a not so high likelihood, it would buff the likelihood of field A just very weakly. Ofc, the likelihood values are not adding up to 1 anymore, that's why I'm not using the term probability here. However, the higher this value would be, the more likely is this region a good guess.
I think that way, one would make improved guesses. Also, one can experiment if these values, likelihood and distance between different fields, should enter the problem just linearly or results improve, if the predicitions depend on these values in a squared/etc. fashion.
+
So what I've learned is that 90% of America is dry, flat terrain with a road.
lmao
Similar to Poland with less greenery and more occasional mountains
Hahaha. The "official" United States map on Geoguessr is notoriously bad and those are the usual locations you get in the middle of nowhere. The "United States (no blurry)" map is so much better.
I love how this has just been shoved into the algorithm after nearly a year after publication
Have you considered placing the guess exclusively within the box the AI is most sure about, and then determining where you put the guess (still within that one box) based on the weighted average? Meaning the guess would always be in the most likely box. How do you think that would affect the results?
Guessing from the video results, I also had the impression that this would be more accurate. It seems that the unsure boxes are skewing the guess away from the predicted location. More so when the prediction is close to the edges of the country and there are a lot of unsure boxes to the other sides.
I'm also quite surprised that he didn't at least use a squared average or something
My thought exactly. As soon as he announced his guess averaging technique I was skeptical of that being a good choice due to the nature of coastal areas looking similar...but being on opposite ends of the map.
I wonder if you could train a model to take CLIP embeddings and predict lat+lon from that
Since Google's Street View pictures are taken in a road, an improvement would be done if the prediction were moved to the nearest road in the direction of the highest confidence rectangle:)
I think with more training data the AI should figure that out sometime
@@mst8819 The AI will not learn to do so given enough time, in it's current setting anyway. It outputs confidence values for each of the boxes. The output of each box is just that, it won't be able to adjust towards roads just like in the animal example it will not suddenly output what part of an animal is in the picture, e.g. "ear of fox", unless you trained it that way in the first place
Your newly 6th sub here. I'd love to see a new model perform on Geoguessr's earth map :D
Btw how did you collect your image data from Google's streetview? What tool did you use?
Also do you recommend any other books/courses on CNNs? Keep it up man!
Thanks for the sub!
I collected all of the images using Google's Street View Static API developers.google.com/maps/documentation/streetview/overview and collected images from random latitudes and longitudes within the US.
The resource I linked in the description developers.google.com/machine-learning/practica/image-classification is probably the best resource I found for understanding the fundamentals of CNNs and feature extraction. Its exercises are also pretty useful for seeing how to implement CNNs using TensorFlow. For this project in particular, the Keras Applications documentation keras.io/api/applications was useful for finding and using a pre-trained model for feature extraction.
Now this is wholesome
@@adumb_codes Thank you for sharing!
for the 7:30 one you should've used more of that mountain, probablly would result in a higher cofidence
Your video might just blow up. I randomly got it in my feed today, and I'm sure I'm not the only one.
yeah I predict it'll get 100k - 1m views, give it a few weeks or days
Yo dude, really pog video, Ignoring the mic, I would've thought it was from a 100k+ subs youtuber
Me too, then I saw he had under 100 subs
Just need to scream nonsense while failing an easy video game: 1M subs.
For world mode, trying to perform an "average" over the international dateline may prove to be difficult.
But it would be so cool to see it done anyway!
perhaps choose the center of the max probability box? is that better or worse?
if you choose to go further and get more accurate, there is the problem of training data.
If you can somehow make fine enough grids, you might be able to produce more accurate results, assuming more data
also, if you take multiple photos of the same area, you could possibly get a better idea of the place - like take a few photos, then get the average of the outputs or something.
i love it !!!
There is some mathematics ionvolved in picking the avertage of a set of points on a sphere. In particular, it is not necessarily unique. Think for example of the mean of the north and south pole, every point at the equator is equally reasonable as a mean, anyway the average of two opposite points does not make that much sense. A way around this problem would be first picking the continent which is most likely and then taking the average over this landmass, islands would still be a problem. So a better solution is to just take a region around the box with the highest probability and consider only boxes lying in that region. Also the shape of the boxes is not square anymore, so we should multiply the weight by the size of the box.
The problem of training data can be adressed by automatization. Google street view should provide a suitable interface that can be used to automatically choose a random spot and generate some pictures. Using multiple pictures would have a great potential for improvement.
maybe some kind of repeated grid patterns in finer detail with the center of the most confident box would work well
That's difficult, because what we train here is an image classifier and therefore we need predefined classes (our boxes) before (!) we start training. Since we want to use the same AI for each location we need the same boxes everywhere.
@@jakobm.4183 for a sphere, you could take the coordinates of a sphere, then turn them into a vector from the center of earth and average that - but since not all grid spaces are going to be used (for example, oceans), then there's going to be an implicit bias. Your method is likely better.
im sorry if this is dumb but couldnt you just use a slightly incorrect mapping? there would be bias towards some areas but you could just use a flat map of the earth then relate that point to the same point on the sphere
The problem with the square method is that, for example, the square that contains death valley also contains a number of ski lodges that are in a completely different climate
this is the most perfect video I will probably see this year
I personally feel like the weight for the highest confidence square should have been higher for best results, maybe the conf level itself as a weight for all (or an equivalent system the result falls inside the top confidence squares)
Some suggestions:
1. About pushing to the center, you can tweak weighted average algorithm giving it weight in power (eg instead of 0.5, give it 0.5^2=0.25). It will keep resulted dot closer to darker areas
2. Write script that will automatically parse geogessr page, predict location and answer using script. This will make everything easier for you
3. Give AI whole 360 image instead of small part of it, it will give it more data that AI can work on
4. Play with size of boxes, it is possible that smaller squares could get better results
And thanks for the video, it was interesting to watch!
Wow, this is a really well made video! You seriously need more attention, this is awesome!
Your final challenge: Create a geoguessr world bot that gets it down to the street more than 50% of the time.
Such a fascinating video, it would be really cool to see the updated version in action taking on the whole world! Take as much time as you need with your next video, me and many others are looking forwards to it 😅
it seems to me like using a weighted average of the various patches is maybe making a mistake? for an example of the sort of mistake that would encourage, imagine an image that is definitely inside a city, but you're not sure which city among a few candidates. averaging the longitude and latitude of each city quite possibly gives you some location in the middle of a field far away from anything urban. depending on the geogueser scoring system this is maybe still a score optimizing strategy, going for consistent but low score over a high risk but very high score if you actually get the city right? but it's just a question you've got me chewing on a bit, now, i suppose.
I had a similar thought, but for a different reason. Using averaging will negatively impact locations near the borders, since it can only average it away from the border. Would be interested to see how a "naive" AI (center of most probable square) would compare. Still a good video though!
I think the same. Average can't be used with terrain. Pictures of the coast would average to the middle of the country. Sides of a mountainous region would average to the mountainous region.
Using weighted average does make it statistically better than the average, though. Seeing as the lowest score possible would be to choose a point on the coast when the correct location is on the opposite coast, this possibility is eliminated when the AI will only choose a point closer to the center. Therefore, even when the AI is wrong, the way the point is calculated will still give it a better score on average.
In conclusion, using a weighted average is a msitake because it's sorta like cheating, and makes it hard to see if the AI is actually doing well.
And won't using this average method having a result always in the 8 middle squares ?
Very interesting conversation here! I think you all bring up reasonable points. Which method is preferred also depends on how the score in geoguessr is calculated.
Take the opposite coast example: if the distance from the middle to one coast is too far to let geoguessr give you any points, you might as well make a guess between one of the two coasts and have a 50% chance for a lot it points, instead of 0% chance at any points when taking the average.
What I keep thinking of is that fact that this question should actually be optimized *by the AI itself*. If you could train it based on the actual score it would get in a game, that would be awesome. Not sure how that would work technically, but from a theoretical view it would be really interesting!
don't worry those re-captchas will now never be click the sign questions. thank you adumb
the hero we didnt deserve
Unironically though: those recaptchas actually have grain added to them to confuse the AI, its some very specific grain too.
@@RubyPiec Good, good... as long as the humans are ahead of the AI then I am all for back to captchas.
This is really cool. You could make the AI much more accurate by somehow plugging it into GeoGuesser, and letting it navigate the environment until it found an image/view which it could match at a very high confidence. I'm not sure how easy that would be unless GeoGuesser has an API, though .
I'm guessing they don't have an API that can do it for that exact reason lol. You could use google maps API but afaik that costs money.
Just feed the ai like 50 images where ur navigating the environment, then u can create a weighted average of it's guesses for them.
@@matthewbadger8685 Well yeah but how many pictures would you need for that ._.
@@prav2568 50
@@chippyconqueror bruh i Don't know about Geosquare but as I assume it includes a whole lot of area , you would need more than that
From what I'm seeing, the AI's most confident guess on where the location is is quite accurate, opposed to the weigthed average calculation of the AI's guesses that you are doing.
Oh wow, this was super interesting! I think the AI could definitely give GeoWizard a run for his money at his own no moving allowed 10-state challenge format. Maybe reach out to Tom and see if he's up for a head-to-head game?
I saw the the answer was near to squares that were most dark, maybe use a power law distribution for the confidence
Yess, finally im early to see a underrated channel that is probably gonna grow so fast before it has 10k subs. Todays date is 03/08/2021, time is 16:37 and this channel 6,37k subscribers. This video has 416k wiews and the channel has only 2 videos. Other video has 95k wiews. im sure this channel is gonna earn at least 140k subs in the next 2 weeks. if he keeps uploading videos, this channel probably would be a lot bigger. Love your videos! Please keep uploading :)
I got this video recommended today.
I do not regret it
Fantastic content
Congrats on your video blowing up! Nicely done.
I think you can make the boxes smaller, use more training images. Then take 3 screenshots per location. Make AI predict per screenshot, and use the 3 predictions to triangulate.
It's pretty cool to see a video about some really complex AI. it's even cooler when the Geoguesser chooses a spot right in front of your old house at 12:43 and you can see what your house looked like 8 years ago. Those trees outside are long gone now. Very cool!
Great video CodeBullet, big fan since last 3 years
This seems really cool. You could do it again with more training data and maybe smaller squares.
1:39 annoying but I have to point it out: a 360x340 geometric shape is not a square but a rectangle lol
Really cool. You could create one for the new battle royale mode where you just need to guess the correct country. Could use road markings and road signs for the training data.
I love the idea of this video. Seeing the AI guess in the world map would be interesting. You got yourself another sub :)
Can you make deep dreams for each of the states? It would be cool to see how the network thinks each state looks.
This is like one of the only videos I can find on someone creating an AI for GeoGuessr. I've had this similar idea ever since I first discovered GeoGuessr a few years back. Very cool stuff! Now I'm tempted to try doing this.
Great job my dude! I thought it was close to impossible to code an AI for GeoGuessr and yet you did it in a surprisingly straightforward fashion. I respect people who code AI and Neural Network stuff since I couldn't do that myself...
Great video! I think it would be interesting to retry this with a different output. You are doing multi-class classification and then weighting the outputs. Rather than making it a classification problem, you could make it a regression problem and directly predict latitude and longitude. Then make the loss the sum of the mean squared errors over both the lattitude and longitude. That would remove the “centralizing” bias you’re getting from weighting.
Please make more videos on this, this is amazing. Im sure people like Geo, jack, etc would even be on board to help too, so cool!
Would love to see a video of some Battle Royale attempts!
Nice simplification of the classification task!
I think a simple improvement could be, that the final prediction must always be inside the highest probability cell, but the precise position is moved by the weighted average.
But will the AI say "Get in!" when it gets a particularly accurate guess?
The fact that it didn’t even have access to the 360 images is impressive too, an AI that can accurately predict a location based on a still image is kinda spooky ngl
It would be interesting to use some topology to allow guesses to be pushed to the edges
The fact you only chose very rural areas with almost no defining features is very tough, but cresting a more advance AI it may be able to learn building styles, road signs and street names for urban areas
Soooooo. Your AI guessed pretty much exactly where I used to live on the one where it landed in Idaho. The on-screen coordinates showing on the video for this round were still for the previous round. However, I used to live in Idaho and even from the short glimpse on the map, I thought to myself... REXBURG!!! Then I noticed the Texas Slough on the map next to the AI waypoint and I knew, without a shadow of a doubt, this is very close to my hometown (South Madison Country, Rexburg, ID). I zeroed in on your coords with similar methods that you described in the video and landed on the coordinates of 43.727127, -111.829871. I then used that to calculate the distance to my original home address. The AI "thought" you were within 5 miles of where I used to live! What's even crazier, is that I moved to WA State which just happens to be within about 200 miles of where the ACTUAL location is. Trippy huh?
I find it especially impressive considering the AI only has a single image to base its guess off of, while a human player can scroll along the road and turn the camera around
Cool vid! I think your square weighting method may be slightly flawed, because it means that the AI is very unlikely to guess correctly when the actual location is on the edge of the country. This is because the average of all squares without weighting would give a dot in the centre of the US, naturally drifting all guesses to the centre of the country.
It's not about the edge vs the center, it's in general that it cannot sensibly represent ideas such as Florida and California being likely and Texas unlikely. It assumes a uni-model distribution of confidence over the map.
If it's a linear relationship between distance and points, then the best way is weighted average. But that is not true (more something like 5000e-^(x/2000). Another point would increase the expected number of points, that point one can derive from the score to distance formula.
Great video. I would love to see it guess for the world map!!
Yes, moar geoguessr please!
Can we get the Dataset?
This was really well done, I hope you do more like this
TH-cam gods are not wrong again :) very nice presentation man and looking for the follow up video
i never realised your sub count untill the end of the video, good stuff keep it up
This is very interesting, glad that the algorithm has chosen you, so maybe we can see what happend in world map
That’s so cool! Would love to see more in that direction
This was an awesome video, keep up the work man
Now that Geoguessr has a U.S Streak mode, you should try it again :)
0:33 WTF That's literally the village in Switzerland where I live - I even recognize exactly that street 😱
Is that a village?
@@ALLINONETECH6 It's a road leading to this town (about 20k inhabitants), there's a garage and petrol station there that I pass weekly 🤫
@@chavem oh nice👍
It would be really interesting to see this done for the Urban World map, also Europe. Great vIdeo!
Ideas for improvement (maybe they are feasible):
Implementing periodic boundary conditions for the average confidence of the position.
Using the whole 360° Image instead of the usage of the image selection.
I just found your channel. Interesting videos, keep it up!
you should try this with the entire world. i reckon the results will be better as the margin for error is slightly bigger. great video though i'll turn my notifications on for your next one/
Finally, a channel like Codebullet!
Edit: you know, if you made that project open-source, you can improve the AI's image database as well...
1:29 I noticed that there were some area that might be there, but they are "unseen" by the AI, like the south part of Texas
Loved the video and really surprised by how well the AI did with so little information! Judging by the comments your video seems to be gaining some traction, that's well deserved ! Hoping more people see this !
10:41 anyone else notice that there's a Black Mesa?
This is great. I want to see the ai play a diverse world map, country streaks, and if there is there a way to make the ai more accurate. Thanks for the nice video!
I didn't realise that this is a 245 subscribers channel, it is over 1m subscriber quality! keep it up (I subbed :) )
i think there is a version with a GAP layer which can be used to visualize why the ai choose the class .
Really interesting - have another new sub! A few random thoughts that occur to me on this.
"Extremising" the probability outputs might help improve the fit on the edges of the map in particular. For example, square the probabilities (then normalise) to skew the prediction towards higher confidence levels.
For an all world system you'll probably have to drop or at least modify the averaging approach, as others have said. Maybe it picks the continent first?
How did you choose the size of your boxes? Obviously smaller boxes would allow a finer prediction but it's more work to train.
Given that human players can move around, perhaps you could give the AI more than 1 photo to base its prediction on.
The weighted average needs a lot more stock in the darker boxes. This is pretty good
Dang I thought there was no chance for the AI. Very interesting and well done video, good job!
Also - thanks YT algorithm
great video! with the algorithm blowing it up now, i think its time to try this again for a country streak or state streak game!
Making the dataset available would be super cool!
This video is phenomenal!
I think taking the second power of numbers to determine the location would make this more accurate because it would make an ai more confident when he gets a higher percentage of chance for it to be in a specific state.
An idea that may improve performance. When you train it, instead of comparing the distribution predicted by the the AI with a delta distribution on the square where the coordinates belong, you can form a distribution whose weighted average is the coordinate of the point.
I think a critical note here is that the comparison to the pro's is not entirely fair. Your AI is trying to optomise score across all games, where as a pro knows to take risks and only keep the 1 lucky group of guesses.
It would be cool to see a best attempt of a group of tests where the AI goes with its highest confidence box being much more heavily weighted.
Really impressive stuff! Perhaps could you share more about your code? Like how long it took you, some snippets, and even share it! Loved the vid! Congrats on the great AI performance!
Love this type of content, keep it up!
Github link? I work in the computer vision field and if you're wanting to make this better, look into siamese neural networks. The same idea is used for good facial recognition.
May have been interesting to run the same photos again but with you choosing a different view/ direction and see if the AI came back with similar predictions. You choosing a view from each location may have represented a biased input to the AI.
Great video and concept, clever bastard 😎
Super cool project! I can't wait to see what you do with it next, should you decide to continue. Once you get the average bias problem sorted out, it looks like you'll have a really powerful tool. With more training, the program could be used for so many applications. Well done!
2:39 how do you make this kind of animation?
Use any non-linear editor (Premiere, Vegas Pro, Final Cut, Davinci Resolve etc) and animate position (typically using keyframes; i.e. google for "[name of your video editing software] position animation").
Group images somehow, so you wouldn't have to animate each image separately. Or simply create one big image in graphical editor. Add a white backdrop.
Here's a random tutorial that should give you grisp of everything you need to make it in Davinci Resolve: th-cam.com/video/kYOTDnjuitc/w-d-xo.html
@@niter43 thanks a ton
I wonder if the AI learning would improve in accuracy if it was able to see the entire panoramic view instead of the limited square window? Also, does the difference in camera quality over different locations/times have any effect?
Chances are very real it's using meta strategies. There's entire websites on how you can narrow down where you are in geogeussr based on some characteristics of the footage such as errors in the stitching of the 3D image or dirt on the lens etc. Entire areas are covered by the same car/team so it could be possible that the AI has learned to recognize those artifacts as patches.
The AI can't differentiate between picture artifacts and the rest of the photo, so it might very well just be recognizing that, for example, every street view image in a certain state or county has a certain scratch on the lens or whatever
It looks like you could compensate for the bias it has against guessing around the edges of the map. If you could ball-park how much on average it was guessing more centrally, you could adjust each guess by that figure outwards.
This would be a really interesting RL problem. Allowing the AI to explore the area and all.
This is really really cool, personally, think it would be funny to see if the AI can be trained to be able to read signage and pinpoint where it is from given distances to other cities. I'm a bit biased, but the Netherlands is a good country to do such a thing with, as there are signs indicating directions and distances nearly EVERYWHERE, mostly because these signs can also be found on many cycling routes.
geowizzard; are u challengeing me?
interesting how the winter picture from colorado threw the bot off
especially since its one of the area it actually did good at
Cool project! I would be interested in how you acquired the data. I think you most definitely did in an automated fashion.
What would happen if you tried to output the lat/long values (maybe scaled into a range of 0..1) instead of tiling the (game) world? Feels like tiling is an unnecessary restriction on the algorithm.
I tried that as well, but the results were significantly better doing it this way.
I assume the reason this doesn't work well is because you would need way too much training data to guess at that high a resolution. The thing about tiling the country is it already recognises that areas which are close to each other look similar, whereas if you give the AI completely free rein it has to learn that itself