I created an AI to Play Poker
ฝัง
- เผยแพร่เมื่อ 28 มิ.ย. 2024
- I created a Poker AI and challenged my friends to a Heads-Up No-Limit Texas Hold'Em tournament. This involved solving for the Nash Equilibrium on an abstracted version of the Poker game.
My code: github.com/Gongsta/Poker-AI/
Note*: For the hand starting at 12:28, it's supposed to be a 6 of hearts in my hands (not a 6 of diamonds).
----------
SUPPORT ME
Not running a Patreon at the moment, but you can support me financially by donating directly through TH-cam, or buying me a coffee here ☕: ko-fi.com/stevengongg
----------
MY SOCIALS
🌐 Website: stevengong.co
✍️ Blog: blog.stevengong.co
📸 Instagram: / stevengongg
📱 Twitter: / stevengongg
----------
PRODUCTS I USE
The links below are Amazon affiliate links, they don't cost extra for you and it helps support the channel!
Lumix G7 Camera 📷: amzn.to/3JgNPLp
Blue Yeti Microphone 🎙️: amzn.to/43X4vRw
10$ Wired Lavalier Microphone 🎤: amzn.to/3Q2rBjZ
Wireless Lavalier Microphone 🎤: amzn.to/3VRQ9js
Softbox 💡: amzn.to/3xsGTYL
HyperX Alloys Origins Core Keyboard ⌨️: amzn.to/3xADGqh
Razer Deathadder Mouse 🖱️: amzn.to/3TYbCob
NVIDIA Jetson AGX Orin (for projects) 💻: amzn.to/3xF9N8o
----------
Subscribers at the time of uploading: 31,908
----------
ABOUT ME
I'm Steven, a university student currently studying Software Engineering at the University of Waterloo. I like to build cool things with code. I started this TH-cam channel to share my love for learning and engineering with the world. I do everything ranging from electronics, CAD, control theory, programming, 3D printing, physics, and design.
My ultimate goal is to be able to make anything that I want. I'm working towards this by building increasingly difficult engineering projects that push the limits of my knowledge. Consider subscribing if you would like to follow my journey and learn alongside me as a student of the universe!
#poker #ai #stevengong - บันเทิง
![[Full Episode] MasterChef Junior Thailand มาสเตอร์เชฟ จูเนียร์ ประเทศไทย Season 3 Episode 4](http://i.ytimg.com/vi/e7Oqqo90x_w/mqdefault.jpg)
There is the famous guy who won the AI only poker tournament.. His AI just always went all in, and the other AI's thought since its a large bet, they have better hand and just folded.
😂😂
Then those AIs were completely horrible, or the one that only went all-in got incredibly lucky.
This has been a project that I've been working on for a pretty long time... so I'm happy to finally release it! Let me know if you guys want a follow-up, since there are still quite a few things I can improve the AI on. I'd like to challenge an actual professional poker player next time. If you enjoyed watching the video, make sure to leave a like, it helps grow the channel :))
This is an incredible video. You combined two of my facorite things, AI and Poker. Cant believe this does not have more views! Such a high production quality also. As for me, I would definitely like a follow up to this video!
It sucks dude. I built a way better one. My poker bot would destroy yours
AI will take over eventually. Its not smart to be making videos like this. It will definitely ruin online poker one day if it hasn't already... Sad shame
for sure, this is very interesting, please continue
Maybe try training the AI on all the top poker tournaments (or just Dan Negreanu strategy) & also use facial recognition for player emotion/stress.
Or simply take into account the time to react factor.
Great project overall though! Kudos
Hahhaa the punt jam on AA2 followed by the even bigger punt call "I'm folding, don't get used to it" - I believe it lmao 😂
Confused why it calls the huge jam on AA2 with QT. Is it because pre-flop and flop are decoupled, somehow? Or is it not realising the difference between all in and small bet?
It could mainly be because this spot never comes up in nash equilibrium strategy. I think the bot is not playing GTO in general and has some flaws, but even a well made bot might struggle with this spot because 6x pot jam should never be played.
@@BarvGwydh Possibly. But watching the film I suspect his preflop model isn’t putting enough aces in this line because it’s decoupled from post flop action. Obviously with more aces in his range here, the need to call weaker hands reduces. To get the algorithm to compute a balanced preflop strategy requires CFR to be done across multiple streets, and he implies that he solved preflop and postflop separately. Basically if you just train a preflop model based on maximising equity before you see the flop, your AI won’t have enough board coverage when it’s deep, and it won’t slow play enough because it won’t see the value it gets from those things post-flop.
I went from a honest government ad warning me about AI, to this...
Great vid that one
"Honest government" is an oxymoron
@@authenticallysuperficial9874 yup,that's the point of it
@@authenticallysuperficial9874 It's a youtube channel that warns people for the bullshit during elections. It has pretty girls and plenty of swearing in australien. The Juice media.
why there is 2 cards 6 of diamonds on the deck? at 12:55
Nice catch
I have been wanting to see this for so long! Great job on the AI, it seemed to be making right plays most of the time!
Yeah but most of the time is bad enough to make you go broke...
It really isn’t. Seems easy to exploit.
The timing on the Helmuth rage clip was great
The Phil Helmuth reference is gold!
This needs a part 2
wow this channel is going to blow up, great idea and very fun execution!
this is so sick! i need to learn to code cus my god it seems so satisfying completing a project like this
Good job man, 👏 props to you regardless of the loss...Take failure, learn from it and embrace it to always make your self better..
I absolutely love your videos man, you inspired me alot, and now i will be an intern in the biggest it company in poland at 16 years old!
good job!! keep pushing
Love your videos!!! Keep pushing❤
Nice work Steven! Keep it up 👍🏻
This vid deserves 1000x more views that what it is at now holy moly
It would be interesting to see this how this would develop further if the AI takes into account the position it is sitting at the table to see how many players bet before or would have an opportunity to bet after. Its interesting to see how other players could interpret the AI's decisions, most poker players would tell you to always bet/raise/reraise the same amount preflop for hands you play, as changing the bet amount would only signal that you have a strong hand or were bluffing. 😃
The bot will use randomization so that sizes do not make it exploitable, it will also simply be balanced no matter what, it will have an unexploitable mix of value and bluffs. Nash equilibrium cannot be beaten, at best the opponent can break even if they also use a nash equilibrium strategy, like the rock paper scissors example where the NE strategy is to pick each action 1/3 of the time. If the bot used multiple sizes it will still balance them, or for example, if it uses multiple sizes preflop then it might use 2bb half the time and 2.5bb the other half of the time if it wanted to open raise a hand to 2 different sizes.
@@BarvGwydh That makes sense for the betting, but I still think it would make a difference based off the position you are sitting. If the bot calls/raises a lower equity hand at the bottom of its range, it has a higher chance of winning when it bets last at the table and there is only one other person in the pot then vs when its UTG and is betting the same hand into 6 people or calling the hand with 6 people in the pot. Nash equilibrium can be beaten in a game with more than two people if others have shifting strategies - it only ensures that a single player can't improve by deviating.
before watching the video I haven't decided whether you need to be taken out or not
Hey! Poker player here, first of all grat job, must not be easy at all!
I think you would have better success playing against your friends (and bad poker player in general) if you programme your AI not to play in equilibrium but to play unbalanced in order to exploit these weaker players' leaks!
A friend of mine coded a poker bot for texas hold 'em back around 2006. He set one instance running 1c/2c cash game tables and was returning around 10c/day profit playing the odds and position only. I haven't seen him for years to see how he went once he weighted it for variance and stake styles.
He's probably in prison now for theft on gambling sites.
@@spacebomb9126 I highly doubt it. The guy was a top level coder and wrote most of the original office polymorphic viruses for bug bounties with Microsoft. And worked a government job. More likely he knew his way around a VPN and a dummy account.
Worth the wait
You should’ve made a results part at the end, good video thooo 🎉🎉
we will demolish that AI
ChatGPT? More like ChatATM
You guys don't always seem to be playing all that great, but I think even you would be able to beat this so called GTO playing AI. I think every action was just horrible.
It didn't only make 1 mistake. After it calls with the first all-in with the QT, it is clearly that it is easy to exploit. You wait until you hit a hand, where the AI hasn't shown particular strength, and then you go all-in if you have hit. That it want's to call 1/3 of the time in the last hand, shows that this simple strategy will most likely win close to 100% of the time. There are other exploits, but you are far behind to win against any pro poker player. You have a lot more work to do, before it stand a chance against even semi-pros.
As an actual full-time professional kt was quite interesting to watch. But this ai at its current state would get absolutely destroyed by a real pro :).
Still a fun endeavour and for sure very hard to come up with making this system. Good job
Lets go Steven!!
my number 1 supporter ❤️
says he loves poker: Actively creates a program to help ruin it.
Exactly.
Idiots like these fools are the reason so many people are cheating online in poker.
The AI didn't seem to be playing very close to a gto solution of most spots
12:51 Heads up, I wouldn't exactly call this a semi-bluff as BB's range should be way weaker than the AI's, so it's a value raise. Very standard.
13:26 This hand is atrocious. I'm guessing you guys are playing 50/100 this hand, hard to tell. In which case we're ~30bb effective and this should just be a fold for the AI as even against the bottom of SB's range, it's not looking too hot. Plus, AI blocks the more natural semi-bluffs like Qxhh for a second nut flush draw. If we can really call a 7.25x shove a natural semi-bluff.
Why does the AI call? If I *had* to rationalize a call, I would say that this shove is either the nuts or air and because we block the Qh, we have more equity against most air hands, except Khx. This is a horrible justification for a call but, it's what it might be thinking.
Interesting note, at 20bb effective, solver calls QhTd at ~10% frequency.
15:02 It might seem insane but this hand is not so horrible from your AI's perspective. It's actually correct from a GTO perspective. Reason for the call is that you are beating 137/185 hands (74.5% equity) and getting ~1.53:1 pot odds (~40% equity required to call and break even). If we folded every time, we would be over folding and losing money long run given the price is just so good.
I think your AI plays kinda okay, considering it's only learnt over 1m hands. The QT hand is crazy, one other consideration is that it's not gonna adjust to the mistakes made by other players so it's gonna make some whack decisions like this one. I'd be interested to play it and also see how it plays against GTO Wizard.
Ive been getting crushed by these "AI`s" in online rooms for years
Just imagine: thousands of cheaters letting their PCs playing poker to farm money using this AI and AI vision. R.I.P. online
Yeah, these guys are losers and should be arrested for showing people how to cheat and steal money.
"Private" Cards.. I love it. I'm using this instead of hole cards from now on. Sir! These are my on cards, they are private!
so cool
Nice work !
So how is this different from all the existing solvers out there?
not that much different, I implemented it for my own intellectual curiosity :)
That was unreal
Good stuff
what a sick video!
Sick video!
8:12 LOL
pretty cool idea, imagine if you got glasses w/ a camera and an earpiece and took this AI to a casino
Sounds like a great way to get kicked out of a casino
There's an old saying "F*ck around and find out."
Try it and see what happens, kiddo.
0:30 Please fix this issue in your audio where you played both channels at once and they got garbled.
as a player whos won +250k in earnings and a programmer, super entertaining
Wow, surely noone ever did that before...
nice vid!!
very cool
I agree with the general reaction to the last hand I don't see how the math could be right, it should be a significantly lower EV to stay in (easy to calculate) especially if you consider the weight of the bet / pool size against the percentages you reported, in fact I think your opponent started to realize that issue/calculation bug in the previous all in bet, I haven't read any papers on this but even with imperfect information the bet/pool size should have a significant effect on the odds (even if some Nash optimization requires some additional randomness injected into that decision) . I would gamble a normal CNN with a lot of training should outperform this vastly (in its current form, you should be able to beat it if you can figure out the true Nash value of each state).
Either way its a cool project, but I think there are much more simpler approaches and code implementations that could perform better here. Thumbs up for a great project idea regardless.
It will be better if she can exploit from opposite player and applies it in to option percentages!
why dont you just use reinforcement learning? itspretty ez to code, just need a long time to train
Honestly it just played loose but still made good calls and good bets
................................................................................ That is how I felt watching this video
I agree.
They are teaching people how to scam people online in poker.
You need to rework your AI. Facing a river jam with bottom pair is a call at most maybe 5% not 30%.
Not if you have infinite chips or games i.e. you can afford consequences of losing few times.
@@OmateYayami I've been playing poker wrong all this time. Turns out I just need infinite money!
@@cpatterson365 you've been playing different poker. Not necessarily wrong. You manage risk differently depending on the relative bet, stack and pot sizes. The bot didn't. Look if you feel your chance of winning is 1 in 3 on a bluff, you won't go into it as likely with all-in as opposed to having 10 more games ahead.
If you have infinite chips or games there is no difference.
I'm just saying that calling vs folding being 5 or 30% is wrong. This percentage should change depending on the chips involved.
@@OmateYayami theoretically the Martingale system works on roulette. The reason it does this, is because it assumes you have infinite money, and infinite bet sizes. For these things to have any relevance, you need to look at non infinite. And funny enough, as soon as you look at non infinite at the Martingale system, it theoretically always fails.
Creating a system with an assumption that is not real, will make a system that is not relatable. Even then, this bot would not stand a chance against a pro, even with infinite money and time.
@@NSelsted i might have oversimplified but you do not have infinite bet size here. There is always a limit. Despite the name "no limit" the pot limit is the sum of all stacks which is always finite. You can't always place a bigger bet and reduce game to one win need.
I have oversimplified allowing to argue that infinite aggression will always win in an invite game, but that was not the point. The point was that finite aggression was too much and unstable. With invite games it would have been alleviated but that was too broad relaxation, which you properly exploited.
I am not sure if a pro could reliably win against a properly written bot. I have my doubts.
Great work. Loved the video. It seems like your friends worked out how to read the AI by the last round. When it checked after seeing each Ace and then the King, Brandon correctly guessed it had a bad hand. A human in the AI's position probably would have bluffed.
If the AI is working correctly, this is similar to saying that you picked up on the strategy of a nash equilibrium bot in rock paper scissors - where the bot's strategy is to randomly pick each action 1/3 of the time.
Id say that you programmed it in a mathematical way using equity. I dont know much about programming so please correct me if I am wrong but a better way would be the AI narrowing the opponents range down using the information that the player gives and use that to make accurate decisions
You could have all kinds of heuristics for a player if you were recording their decisions but that might contravene any kind of rules of a provider. But for personal use you could do all what you suggested and go even further to exploit the Mersenne Twist algorithm that all casinos use to shuffle their poker decks.
is it allowed to use AI in online poker tournaments?
No, its against terms of service for just about every platform. I bet there are tons of them out there though
Nope, and it could get you in prison in some cases if you try it.
Just tweak the algo. Tell it, when it faces elimination, it should tighten it's range and only call when it's 90% and above, or whatever .... why am I helping you? lol
I thought Greg Goes Allin uploaded video
What I'm missing in this is the AI trying to figure out the cards other players have based on their individual strategies, actions, and known data. For example if it figures out after a round or two that a specific player bets high and ends up winning, that's a pretty good indicator that when they bet high they have cards that would let them win. The AI knows the cards it has and the cards on the table, so by matching all of that together it could make a reasonable assumption about how likely each player is to have a winning hand.
It doesn't need to. In poker it's just extremely hard to not lose tremendously vs a nash equilibrium strategy and it's impossible to win. The bot can just play as if it's playing against an unexploitable opponent, and it will crush someone whose strategy has nothing to do with the unexploitable strategy, and it will comfortably beat a human who has spent a decade studying nash equilibrium strategies. If it adjusted to play against the poor strategy it would absolutely decimate them even moreso. However, in such a small sample, we will not discover someone's strategy. And trying to make significant adjustments based on a small sample just opens up the bot to being exploited itself. Sometimes human vs human you can find ways to exploit someone's strategy in a comically small sample like 100 hands, mostly because a lot of humans are terrible at the game, but a bot would really need like 10,000 hands just to get a vague idea of someone's strategy. Plus, if it adjusts before that, a smart player could intentionally make it adjust one way then overadjust their own strategy the other way to exploit the bot.
If anything, it might be better to datamine online heads-up games and take an aggregate strategy of all profitable humans, and the bot could maybe even play a strategy that's mostly GTO with some adjustments mixed in from this online data.
TLDR trying to make a bot adjust its strategy based on its opponent is probably a bad idea unless it's making very small careful adjustments over 10k+, 100k+ hands.
Actually I take back some of what I said. I think a bot could squeeze out a lot of EV by making adjustments over small samples as small as like 50 hands. But it could open the bot up to being exploited for sure. And in any case, the nash equilibrium strategy doesn't need to be improved, it's unbeatable.
It's just playing according to its preset understanding of optimal play. If it was an actual AI that was taking in other player's tendencies, it wouldn't bother making theoretically correct calls like the 54o hand as it'd know it's rarely good against Brandon. Against Olivier Busquet, though, I'd say it's not a horrible call.
Edit:
Think of solvers (like this AI) more as extremely complex and detailed flow charts.
Heads up all in calls need to be much more conservative. Really, overall, it needs to have less weight put on the opponent's bet if it's going to survive any real players at all.
Also understand that, poker solvers already exist, no need to reinvent the wheel there, all the AI needs to do is take a broader look at equity, not just a single momentary hand but the totality of hands, and what the pot values are because pot value has a whole lot to do with strategy. Think of a bet as buying a ticket to the lotto, and the cards in play determine the odds of winning. It appears to not be doing any of that.
Also understand that, data is available on how the professional players play certain hands, as is data on how people play online as well. That's a powerful tool that you don't have access to at the moment, but it's simple enough to get access, you just buy it. You could even have a way to have your system ingest that data, and a separate play mode for unknown players, such that you could eventually learn a given player and profile them.
Good luck.
This video was awesome! Don't really have anything else to say just commenting so this gets more views.
I'm missing the point here but so this is a 16minutes video on reinventing GTO Wizard / PIO Solver?
This bot isn’t even close to as sophisticated as those softwares
@@Gatorbait1869 I was kidding this is still a commendable endeavor.
GTO bots already exist. Most of these poker sites don’t do anything to stop them.
GTO doesn't know what to do when a donkey over plays marginal hands and doesnt follow standard range conventions.
The strategy assumes both players play very rationally/reasonably with specific ranges of hands. Once you have a non-conformist player the strategy starts to show gaps. For example, OMC only plays AA and KK, GTO doesn't factor that in and therefore will consistently be at a disadvantage assuming the player is playing a more reasonable range (i.e. ~50% from button, ~10% UTG). If you adjust your bot on the fly to adjust for this it will perform as intended, but you'd need to know and make that adjustment for every player. Also when you watch top players, they play close to GTO but break away from it on occasion to exploit their opponents.
This AI trys to adapt to the player which is interesting since it makes it closer to exploitative than GTO, which is more profitable if done correctly.
@@wesleykim1758 of course, Exploitative will always be more profitable and deviating is the correct thing to do. GTO will always be a net zero(or lose to rake) Vs other GTO bots. However an exploitative player will never beat a GTO bot(if it’s truly playing perfectly). Not to mention bots have no human emotions and will always be playing it’s A game.
With all that being said most bots aren’t playing GTO perfect as of yet and a good amount of them are losing players that only profit to rakeback/promos.
damn W
looks like this AI learned from viktor blom 😂😂😂😂
Making a bot to beat the market thru options would make boot loads, remove the human element and options is as close to poker then you would ever believe. Or in game sports betting especially basketball. A video on how to create said bot would be amazjng. I would kiss you on the lips 💋
I think it doesn't consider consequences enough. Fold Vs call or agressivness should probably be weighted against stack and pot size. You wouldn't call 5% chance hand because of a coin flip on an all-in, while you could on a small stake.
Bot went too aggressive on two occasions and lost.
Bot played just for a single game and didn't value in stack and pot sizes, did it?
It's due to the fact that I abstract away bet sizes, so it sometimes sees raising and going all-in as essentially the same, which can lead to sub-optimal decisions at times
@@stevengongg Thanks for a fast reply. By "abstract away", do you mean completely ignore or substitute with a simpler idea?
To me it's not only a raise/call being mixed with all in. It is about how many games you will have left if you lose. Maybe that will make the bot more exploitable, because it would play less aggressively when low on chips but on the other hand it would have more games to come back.
I think the optimal strategy changes depending on chip state. Bet vs pot vs stack.
To me it seems the bot lost because it got heavily punished for taking big risks at high stakes and getting unlucky.
Great work... but somehow a relief that a human won... for now!
You're AI need to understand potodds lmao
This video is really cool and it looks like you worked really hard, however how close do you think your AI is actually playing to an unexploitable strategy? Considering the best solvers currently on the market cannot solve things instantly like how you are doing in this video, I'm skeptical about the quality of your AI's play vs an actually unexploitable strategy.
I've always wanted to code a solver myself, never really considered involving AI in it but some of the newer tools have been implementing it somewhat so that is interesting. Nice job.
When did we normalize dudes wearing grandmas pearl necklaces?
i know fashion these days... it's getting out of hand
here before this blows up
That AI is dumb AF 😅😅😅😅😅😅
Well done. But slightly too educational on the “edutainment” meter. I would definitely do another video with a different spin on this later.
GPT 4o vision + 10m pre solved spots + online poker + document your results would slap. Play a bunch of $1 or $3 tournaments using GPT to tell you every single play, but essentially destroy online poker as we know it and cards like America’s cardroom would vanish. They are not very good people.
Cool video. I feel like this bot would do better in non-tournament setting. It’s ultimate loss highlighting the issue with sampling rather than using sampling to inform decision making.
A person would see bottom pair and know they still do have a chance but the cost of losing this hand is higher than usual because you don’t get to play another if you lose.
Sir do you earn 150k dollars a year? Plz reply. Thanks a lot.
Typical Indian like me.
gto only works against players who are good and making correct decisions
I see one problem with this ai. If folding is the better choice 67% of the time, then it should fold 100% of the time. I think it's pretty obvious why.
Not necessarily. If you always act the same way in the same position then it becomes easier for your opponent to predict your cards. Think of the scissors paper rock example, you can't always play the same move you have to switch it up
@@Danicker the same position basically doesnt exist. There will always be either a different hand, a different pot, a different opponents bet, a different balance, different cards shown. By your logic, the AI should in very very rare cases all-in with a 7-2off pre-flop because there are some cases where that would've been the best choice... Or in very rare cases it should call an all-in with a 8 high because sometimes that's the best choice. You will never see any poker pro making an irrational decision ever because "sometimes stupid shit ends up being the best thing you can do". No good player would ever play poorly on purpose to "switch things up" because that is at its core what you were suggesting. The ai should be allowed to make terrible choices to switch things up... That was the exact reason he lost the tournament.
@@k.a.8725 Yes, you're right, the AI shouldn't be using those percentages to make it's decision, however there is merit to having a non-deterministic strategy. There is a difference between switching things up for the heck of it and playing a balanced strategy that can't be exploited. The latter is what you want, but it's not easy to find. And I can almost guarantee that the Nash equilibrium is not deterministic just like how it is in scissors-paper-rock
@@Danicker i agree. I think the problem was within the learning process already. He should've found a way to train an AI for adapting to people's playstyles. What he was training the AI with was purely a statistical approach. That's like me telling someone to study the movement patterns of fish and then giving him an aquarium with a toy fish to look at in it. Instead of playing through random hands a million of times, the AI should ideally be trained on a variety of real poker players as that would also be the environment it is competing in. Being able to adapt to people's playstyles would also make you "unpredictable" to some extent as any attempt to adapt to the AI's way of playing would just be counteracted by the AI adapting to the changes you made in your playstyle. His current AI basically just knows the best play against random stuff, but not against actual players. Its basically the best player against idiots but loses to any good poker player.
The AI wants to be balanced and unexploitable, so it calls 33% of the time to protect itself from people over bluffing, calling with 8 high or shoving 72o are not the same thing as sometimes calling with a bluff catcher that mainly folds, since in poker you can’t outright know how often someone is bluffing or how often in a given situation.
How long until A.I leads to the depopulation of the poor?
If you suck, your AI is gonna suck as well
Nice vid! Thx!
😅
Early gang 🔥
How bout you stop?