ฝัง
- เผยแพร่เมื่อ 15 ก.ย. 2024
- Slay-I is a slay the spire AI that evaluates card rewards, removals and upgrades based on the amount of damage it expects you take based on them. Check it out here: steamcommunity...
Link to twitch VOD: / 673936008
You can find jorbs shirts and hoodies at: www.bonfire.co...
If you enjoy these videos please consider dropping a like to help promote it to search algorithms. If you'd like to join the community of people who support me making content like this you can do so via:
Discord: / discord
Twitch: / jorbs
Patreon: / jorbs
Twitter: / joinrbs
(Or by Subscribing to TH-cam here :D).
I think this AI might just be northernlion
Hey, the egg might not be the best, but he's entertaining as hell.
@@tylerrussell7560 this was pretty entertaining
Those were mature decisions
I mean it's trained on the aggregate uploads, how much of that is NL?
S I M U L T A N E O U S L Y
This video made me less scared of an ai takeover.
watch (some of) the deepmind alphastar sc2 games and you might start getting scared again
it shouldn't, it should make you more scared because it shows how easy it is to create a bad ai that works from useless laws mindlessly
@@GamesCourier Yeah, the real danger of AI comes from badly designed AI being implemented for tasks it's ill equipped for and making poor decisions on an industrial scale never before seen. Like the T-1000 version of the Boss from Dilbert.
The paper clip maximizer is under your bed.
As a lecturer in statistics in France (from economics/game theory originally), I am very delighted seeing you play Slay the Spire and applying research concepts to the game !
For the IA, I think it should try run some king of logistic regression based on Path with the basic problem being minimizing actualised probablity of suffering lethal damages in any of the fight encountered (including infos on which fight is likely to appear and cards as regressors for each particular fight infering a particular distribution, an exemple for defect : powers vs probability to encounter the awakened one ) Plus an expected returns of the fights to be applied to the next fight to appear (value of rare cards appearing, potion chance, possilbe card rewards an synergy)... Which is probably impossible to calculate with precision....
On another level, could you perhaps one day record a video explaining why Slay the Spire is SO MUCH FUN of a game ?
Never try never know ☺️
The AI learned the most consistent way of getting as far as possible. Cards like ironwave and perfected strike don't require much synergy to do their thing. Models often get stuck in consistent dead ends when other paths need specific conditions. It for example hates havoc because without a clear understanding of what the card does it can easily end your run.
question : "how many claws should you put in your deck?"
everyone in chat : "yes"
Slay-i : "never"
On the topic of using a dataset of people playing games. Interestingly AlphaGo was originally trained on human amateur games, and then through self play. Later iterations of the AI(AlphaZero) found it was better to just ignore human games and learn tabula rasa (starting with random self play).
Seems like the AI is heavily biased towards not taking damage in early acts. Which seems like the wrong thing to optimize.
Edit 9: If you’d like to have an intelligent conversation about this topic, ignore this comment because it’s nonsense. I’m not deleting it because I’m no coward.
I mean, aside from metascaling cards, you want low damage to take more elites, events and campfires to get more powerful early. It seems like your issue is only showing the next 2 acts. I never see myself in act 1 thinking about what beats awakened one or the heart. It’s always these elites, this boss, next elites if in the second half. If I start taking disarms against slime boss, I’d die a lot more often. Sure, it could show predictions for later acts, but it’d be both very unreliable and misleading, especially with a human agent at A20. The idea that you should play spire by building a deck to beat the end boss doesn’t work at all in A20. I haven’t watched more than the first 5 minutes, but I don’t have much of an issue with how it’s set up.
Edit: The glaring issue is with test data. Maybe it learns via low ascension and applies it to high ascension, leading to not as great evaluations. Early shrug vs headbutt when you have 1 decent damage and 1 decent block card is definitely headbutt (I don’t pick havoc all that often because I don’t fancy checking the draw pile another 2 or so times a fight). This is because the AI doesn’t account for a player taking 3+ elites an act: dealing damage isn’t as important when you’re doing almost all hallway fights. If the AI uses exclusively high-ascension play, it’s evaluations are very different from all players, because there’s far too much variance, and the more common player is assumed to be controlling fights rather than the actual player.
Edit: In hindsight, my judgement of “what’s best this act and next” is biased. Whirlwind is a clear choice over iron wave, and I rationalize it by saying it helps against byrds and gremlin leader, when in reality I’m picking it because it’s good against byrds, gremlin leader and half of the act 3 fights.
Edit 3: Or the issue is using an A20 metric with A0 data. I guess I can just speculate as to how biased my card picks are.
Edit 4: I guess I see your point. In the same way you shoot for “good enough” in the heart fight, you want a deck “good enough” for guardian. Anything else is (basically) overkill. But still, the Act 1/2 evaluations just seem straight-up wrong to me.
Edit 5: My final stance: the next-2-acts metric is a bit of an issue, but not the biggest issue.
Edit 6: To clear that up, I’d wager to say it’d show negative values for the Act 1 headbutt, whirlwind and demon form if it did calculate for acts 3/4, just because of how significantly the learning data differs from how the agent plays.
Edit 7: I’m nowhere near qualified to talk about any of this so I’ll just shut up now.
Edit 8: One last remark, maybe death chance in each act is a better metric? Not sure how to make especially later acts meaningful, but if we’re trying to get to a point where the AI wins the most game, making it not die seems like a good start.
Yeah that is the wrong thing to optimize, but it was Jorbs who did that optimization by consistently clicking the card with the highest number.
@@limepop340 A first problem that is easy to understand is that this bot uses a greedy algorithm. This may find local optima, but can fail to find global ones.
A really good example: Not wanting to take 'sword boomerang' at the start. It's not a great card to add to the starting deck, but it makes other cards, such as those that give strength, better. The AI should add the average value sword boomerang adds to all further picks in the act (given an average act, recursively) as well, which it does not.
A second problem: the last 1 hp is worth way more than the first 59 (or however much you have). The AI currently values them equally. What it should instead do is say value the last point at 1000x.
ironic how the ai thinks perfect trike is a great card, and then continuously tells you to remove the other "strike" cards
Lol the fact that Jorbs data set pushes all of the numbers very close to zero is hilarious! Since the data set is small and especially since he is a professional the results go from "this is the prediction if you randomly throw this in your deck, like the averaged player does" and more "if Jorbs chooses the card here this will be its effect". Jorbs tends to look at cards as "is this card good with my current deck/act needs" vs average players tending more towards "is this card theoretically always good". The ironwave shows this a lot since the "average" players that pick it a lot or all the time aren't crippling their deck or have some understanding of boring cards being good. For example the average person would probably take demon form after the act 1 boss because they think it's good (infinite scaling, infinite damage, no downsides /s) and burns are bad, but the immolate reflects both players aware that they don't need demon form at that time and that immolate is generally very good.
Data science is great, we need to find a way to hijack the data set when they add relics to inflate the shovel and boot values. Actually there's a pretty good chance that the shovel and boot values in the shop would increase over the course of the run (trending towards zero at least) even in a normal data set, since people probably tend to buy them late when they are so strong it doesn't matter.
I wonder if you could develop a system that evaluates players based on the record this way.
We are all mocking the silly A.I. but we are witnessing the birth of Skynet and it will not forget this
it's actually apparently trained on Jorbs data, which is fascinating
Watched this on twitch already, very entertaining. Developing a good AI for this kind of thing seems like it'd be very, very complicated. I've got no idea what I'm talking about, but I feel that distilling the nuances of winning a run into a 'lose hit points per fight/act' is not going to provide any useful or meaningful way for the AI to learn how to actually win.
I second this
Yeah "lost/gain hp' is an average and doesn't take into affect AOE damage, scaling, specific fight / boss challenges, etc. It's on average right but probably leads to you building a shitty average deck that does everything meh. But that's both the fun of treating it like the AI overlord and the difficulty of creating an AI that could possibility calculate all these variables. Like the possibility space of the branching paths is nuts lol
Yes, it's difficult to train an ML model to do that, and you've identified exactly the difficult part - the reward function. If you only reward the model only when it beats the heart, it will have a hard time learning, since it will so rarely get to the heart fight by chance. So one approach to combat this loss of signal, is to incorporate multiple goals into the reward function. Then there is are all kinds of strategies to deal with overfitting, i.e. preventing the algorithm from thinking claw is OP because the only run it won had a claw in it.
Finally, there is always the question of the training data - in this case it seems like the model was trained on a limited set of runs, as opposed to training the model by having it play the game.
The AI is trained on predicting damage taken in a fight. Card evaluations just go "how much damage would I take against *insert enemy* if I added this card to my deck" for every enemy in the run and average it. If people have better ways of doing card evaluations with damage predictions (which there almost certainly are), let me know!
@@iSniplikeaboss My comment basically just sums up to "it's hard". The real take-away is what Jorbs also talks about in the video - average damage taken is not a very useful metric because it overvalues hallway fights a lot and you die to the bosses, and inferring based on human players biases your predictions. I am by no means an expert on this, but my guess would be that if you want to stick with the Spire Logs data instead of going for a set up where your model plays Slay The Spire itself, looking at floors climbed is the more interesting metric. You'd probably want some power-function, so higher floors carry more weight, and a term that makes sure surviving boss floors carries extra weight. Also not clear if you're training one model with all the cards or one model per class.
I also am not sure from the blog post if you trained one model per class or one for all classes. I don't see which advantage training one model for all classes has - it seems like there is no transfer of information between classes, since all cards are one-hot encoded anyways. As a result, the model is unnecessarily sparse.
In your blog post, you mention processing the data you're working with, but you don't mention any of the distributions of your data:
- Highest floor climbed
- Ascension level
- How many runs by how many players
- ...
I feel like one of the biggest oversights in the model for the AI (you know, other than all the other oversights is makes) is that it doesn't seem to consider the path you're taking through the act nor does it seem to consider the relics you have
agreed something that would make it much better would be prediction calculations based on pathing i.e. survivability chance if you go ?/monster/shop
According to the AI author, it does consider the relics, but it doesn't consider the path you're taking. It uses a weighted average of damage taken by all encounters in the act (with higher weights to elite fights).
It knows about the relics you have, but it does understand them (or anything else, really). It's just blindly matching against worked in the past.
@@DavidGuild I assume the AI looks at all of the data for a given fight then compares those averages against the cards used during those fights to make predictions based upon those cards for those fights, then it averages the fights for given acts to give you the average effectiveness for the act and the next. Then each fight it takes the average for a given fight and modifies it as a result of your current card pool.
We can see that for act 1 when he was looking to pick up a gold card on the last boss it was saying it would save him 4hp that act, I think that indicates its saying on average you would save 4 hp over the 10 fights or whatever on average in act one. What it fails to value is how many fights you have left in the act, in this case 0. So i think the AI needs to start looking at how many fights you have left then gives the prediction for this act and then for the next. If it did then it should have indicated that the boss gold card, immolate, would save 0 hp this act.
The best comment Jorbs makes on this excellent video is the one about computers telling you exactly what you ask them and nothing else. This was super fun. Thanks for making and uploading it.
This AI suffers from the horizon effect. It wants to keep blocking damage and doesn't consider how much less damage it will take from the enemies next attack if its dead, it sees "I'm taking 16 and I can block 15 of it? Ok". While a human might think "I'm taking 16, i can block 15... or I can block 10 and deal 6 which lets me not take damage next turn". If you see what im saying, it always tries to save HP, when in reality it's taking damage anyway and should sacrifice HP now, to take less in the long run.
I think it evaluates base cards separately to their upgraded counterpart, so when it wanted you to remove disarm it was actually saying upgrade it (which is why it was so positive on the next upgrade screen). Anyways, great content as always jorbs would love to see you revisit this and pick the worst choices for everything heh
confused
If you die floor 1 act 2 you might just lose less health than if you barely defeat act 2 boss :) is the ai trolling you to die early?
I very much enjoyed your explanation of the pitfalls studies run into!
Finally, some Slay the Spire content!
As someone with some, but not much, experience in statistics and machine learning, but as someone who is very interested in those things, the problem a lot of AIs have is choosing the wrong metric to base success off of. For example, minimal health lost in the majority of fights is not a good metric, because it may have minimal correlation to winning the run. For a lot of games, it seems like the optimal way to train a neural network type AI, not a learn from humans AI, bc those are almost always suboptimal, is to find some way to distill from the game in question a score that denotes your position currently in the game, on a scale of 1 is winning and 0 is losing. It would take some wack optimization and a lot of linear algebra to determine in one number how good your STS run is going, but if it could be done, that would create the optimal AI for the game, as you give it access to the full game and that score at any given time and let it press every single button until it learns to win.
Also, I know it wasnt serious at all, but I also feel like the AI was not designed to be used in the way it was. It evaluates health loss, and it is your job to determine how important health loss is. Not having an echo form that you can rarely put into play would save you health in act 1, but you as the player should value other things than health loss in Act 1 and 2.
i like how minijorbs makes his name look like 'jort' in this video
I imagine Watcher would have broken the AI.
Yeah, every card is just +5 except pressure points, which is just -80
Watching this was like watching a train wreck in motion. Lmao
For 1:12:00, interestingly, that's why some chess engines have "engine moves" that GMs and IMs reference, where they'll just play like b5 or a4 for no reason. A human would never play it, but because the engine thinks the opponent has perfect knowledge like itself.
53:20. reminds me of hitchhikers guide. probably because that was exactly what they did.
I think there needs to be either a "you will die" tag on the fights or making it so getting killed adds 999 damage to the counter.
This is very interesting. Thanks for uploading on YT!
49:40 - Ok but if this bot tells him to remove Echo Form I might burn the building down.
what i learned in the guardian fight is that the machine prefers to smorc
Skip is like thoughts and prayers 💀💀💀 does nothing
So it doesn't understand scaling very well, cool
should have taken astralobe just to see the AI calculation on which card to transform
Wow, that AI legitimately assigned slightly more value to removing an upgraded strike than removing a regular strike. I'm not sure in what circumstance a regular strike is better, but the AI sure has enough foresight to see it.
The “AI” values block above anything else
Just talked to a few friends with CS degrees. Can confirm that they all took statistics courses.
The guy who made slay-I may be self taught or something, but a lack of understanding of statistics is not the norm for CS
Isn't the ai biased towards more defensive cards if its primary concern is the amount of health each card saves
Daniel Brennan well killing the enemy sooner also saves hp, especially since a lot of fights have some form of damage scaling to kill you if you are too slow.
I think it's more biased toward mediocre damage cards, since those are the most "consistent" at saving health in hallway fights. Unfortunately it doesn't really understand the difference between losing 80 health over a period of six hallway fights, and losing 80 health in a single boss fight.
It would have been interesting to see the variance expected when it predicts a certain amount of damage in a fight.
I think the AI wasn't taking torii into account when estimating the damage of the Guardian fight.
“A lot of people who have studied machine learning haven’t studied statistics.” Yikes, lmao.
Makes a ton of sense an AI would think ritual dagger hurts. It's Greed that'll lose you 3 hp an act getting it leveled up.
The number is actually a per fight estimate!
If ai took rainbow prism it would definitely win every round
This is interesting because I pretty much agree with every choice the AI made in act 1 lol, apparently I don't know anything about this game even after 120 hours. I always remove Bash and take Perfected Strike, Iron Wave, and Cleave every time.
This is when we laugh at our soon to be AI overlords.
Some interesting thoughts during Silent vs Slime Boss. My thought is it would be nice to train the AI in a situation where your max HP is infinite, so expected damage taken in a fight might be in the 100s in situations where you would usually die. Predictions like 30 out of 48 lost because it's averaging some 48s with some smaller numbers isn't super informative, assuming that's how the AI is working here.
But I guess you'd need a version of STS modded to have infinite HP, but where you can't heal 8000 with Demon Form Reaper, just to keep it from going totally off the rails...
Shouldn't be too hard, the ai can just use the console to crank its hp in fights, then check how much damage it has taken, then return to the correct hp values, ending the run if it should of died by the end of the fight.
Jorbs did you try using the version of this built off your data?
“I think a lot of people who study machine learning don’t study statistics” describes me currently. I’m not doing very well in my machine learning class tho anyways...
The iron wave made me laugh so hard
Get an AI to model Jorbs and it'll be unstoppable.
I feel like the ai assumed jorbs would find the medkit relic with its opinions
Why isn’t evolve good with Mark of pain?
I think we’ve got a couple more years till judgement day
Draft a deck full of common attack cards then transition to exhaust for the heart kill.
Ez
1:33:52 - "I told you to remove Eruption"
(Yeah it also told you to upgrade it but shh...)
Ah, yes. Terror over Noxious fumes. Makes sense.
@jorbs I wonder what your background is that you know so much about datascience? Genuinly curiuos.
parents both understood data is a lot of it i think. mother econ phd, dad comp science -> building descriptive statistics for tv broadcasts. was raised in a household where irrational claims we saw were intelligently challenged by my parents in a non-partisan way, and where i got logical explanations of why good claims were good and also taught that they often still weren't infallible.
i've generally been drawn to trying to find meaning in big messy datasets over my life - that's essentially what the difficult strategy games i play are, and at uni i focused on classics and statistics, analyzing structural content of pre-writing epic poems and the historical chess record as well.
@@Jorbs Cool
Here's the medium article about the AI:
medium.com/@aldriedger/bringing-deep-neural-networks-to-slay-the-spire-a2971d5a5115
You made a comment during a video about how you don't learn chess by following what other people do, but I'm pretty sure what chess engines do is actually really close to what this AI mod is doing: When you get into the depth point of minmax, you need a heuristic to estimate the value of the leaf nodes. Most chess engines rely on their DB of games to evaluate a position, by going over win/loss rates from a given position. That being said, there's less confounding variables in a game of chess given reasonable play than there is in STS, so I think there's still a decent amount of caveats to using the plugin as is.
ehhhhhh. you can attempt to do that to get started but it runs into significant problems which need addressing and i'm pretty sure it is not a significant part of how the top engines run their evaluation functions right now. trying to emulate weak players can't make a particularly strong player, trying to emulate a strong player leads to you making the same mistakes as them and not understanding their decisions as well as they do, and trying to emulate a group of strong players leads to you mismatching strategies such that you're overall worse than them individually (i.e. you see one player wins often with catalyst and another wins often with finisher, but don't understand the distinction that taking deadly poison early instead of dagger spray makes in terms of other decisions you have to make in your run, and then your decisions are half informed by how a very strong player who takes deadly poison early plays and the other half are informed by how a very strong player who takes dagger spray early plays).
stockfish's evaluation function is public, can click through it here: hxim.github.io/Stockfish-Evaluation-Guide/ - people have done a tremendous amount of work finding variables which are relevant to winning and calibrating their weight appropriately.
leela plays against itself to build its ability to evaluate its position - en.wikipedia.org/wiki/Leela_Chess_Zero
naive supervised learning from pre-existing games is a thing you can do, but i don't think much progress has been made in chess by doing it.
when I clicked on this video, I thought we’d have won at LEAST once
It does appear the AI isn't really able to evaluate card interactions particularly well, just raw success/failure rates of cards.
I think the remove may be interpreted backwards. Looking at the ritual dagger removal I think it actually wanted Jorbs to keep dagger and get rid of defend
Wonder if the AI wanted to remove ritual dagger not because the card itself loses health but because of the health lost from the event is also added to the calculation. So, possibly a lot of the data it was fed died on one of the following floors.
Either that or players trying to buff it up to a win condition and dying before it gets there.
I think thumbnail jorbs is becoming a little too powerful
concerning that thumbnail jorbs may become the dominant jorbs monkaS
Ohhh I saw this on reddit I know someone wanted to test it with your test data.
I saw that comment on Reddit aswell. Folks were saying that Jorbs would do a much better job than the AI seems to have done
Didn't busted crown use to give energy?
Can't find the timestamp, but reguarding the discussion about solving the game:
That statement about it being impossible for a computer is only true for a traditional computer, but is NOT true for a sufficiently advanced quantum computer.
If attempting to solve Slay the Spire, it becomes an NP-hard optimisation problem (this means that there is no known method to find a solution other than a variation on brute force, but individual solutions to individual instances of the problem can be checked easily by a computer - since a computer is deciding the result of your attempted solution whenever you play, this is true), which is the same basic classification as the travelling salesman problem and pretty close to trying to crack modern encryption. And according to our theoretical understanding of quantum computing, this type of problem is pretty much the specialty of this type of computer (that is, on these problems its estimated performance relative to both traditional computers and other types of problem is much higher than average), and due to both useful real-world applications and being related to a major science bounty (scientific question where a proven answer has a cash payout, in this case a REALLY big one), there is quite a bit of work going on in this area, and the people trying to get non-theoretical quantum computers to exist are using these types of problem to see if they have succeeded and built somthing whose results can actually be read and used (they're pretty sure they've worked out how to make them CALCULATE the answer, but haven't worked out how to retreive it yet, either losing it in the noise of all the other possibilities and thus being able to conclude nothing, or looking at it too closely and causing it to decide what state it is in and stop being a quantum computer, which is very unhelpful. They also have scaling issues trying to make them larger than a certain size.).
assuming the simplest interpretation of a slay the spire solution i can think of - that for every seed there is a set of instructions which win or a claim that the seed is unwinnable, it's very unclear to me how a classical computer could confirm that a seed was unwinnable if a solution claimed that was the case.
if we get into more complicated stuff, a classical computer couldn't ever confirm that a line of play scored the most possible points for a seed, or confirm a more generalized solution which didn't rely on knowledge of the seed and just probabilistically conveyed to the player the best possible decision to make in every situation they could possibly be in, even if that decision lost in some seeds and won in others.
@@Jorbs That's not how it works. In this case, while the problem is the game as a whole, each seed is an instance, and each possible course of action in that seed is a possible solution. To meet the definition above, a computer must be able to produce a result (in this case, generally somthing in the info on the score screen) but must NOT be able to find the "best" solution for the run without some variation on a brute force algorithm.
Well statistically if you take Fasting you're doing pretty good and strong and are likely to win. So pick it always.
It seems the AI is only looking at the state of you deck right now and guessing how well it would do in the acts as is. It doesn't seem to take into account futur synergy potential, only available synergy now.
As in: how much your deck would be improved now with this card. So sword boomerang would only be deemed good if you already have strength in your deck, and Perfected strike looked amazing then because it synergies with the strikes you had in the deck then, it's not taking into account that you probably will remove them later.
I think I would have made the AI to try and optimize score. I think most factors (like beating bosses and stuff like that) are automatically factored into that.
That's a much harder problem. The AI is doing the obvious thing based on the data logs generated by StS.
@@DavidGuild Score is also part of the data logs. I've been doing some things with the data myself so I have a good idea of what is and isn't in there.
So backseating AI?
Definitely should do this, but actively pick the worst cards according to the ai
So from the thumbnail, am I to assume Jorbs has the machine brain?
Wonder how much it would weigh an impervious given how biased towards defense it seems to be.
I think the Ai is working for the spire... or the heart
The AI makes my brain hurt lol
From my experience chess AI kicks my ass 100% of the times.
I mean, chess is a way more simple game. By no mean is it easy, it's just very predictable. There is a hope you could solve chess at a certain point, but i don't think you could solve stp by the way the game is built
@@lucasvivante8988 currently probably no but theoretically someone could include every single variable to win the game and
make a perfect AI
The model does not take boss into account, and each training example was a single fight
How is this robot worse than I am?!
lul this dude said the act of owning health insurance makes you healthier
i feel like a way to make the mod more accurate for damage calculation would be to do it on a per turn basis I.E. take certain amount of damage it recalculates next turn to better reflect, though i know it is just; predicted vs actual data it's just interesting that this is even a thing.
The problem with that is that the AI is trained on savegame data. This data is saved on a per-floor basis so there's no way for the AI to train on this type of data, save for having it run in the background of many, many runs. It's just not feasible to train it in that way.
Creative AI needs buffs plzkthx
1:07:19
i feel like the ai needs different playstyles to choose from and switch between.
i think the AI doesnt count in your relics tbh, all hits you took were close to 4 dmg and got reduced by torii, so actually you would havew taken close to 32 dmg in the first bossfight
Here is the link the medium article that explains more about how the AI works. towardsdatascience.com/bringing-deep-neural-networks-to-slay-the-spire-a2971d5a5115?source=friends_link&sk=4161df5a0f41ede5c68c81d6a0651aaf
DIG, AI, DIG!
Bots rule!
It seems to really, really hate exhaust cards
It looks like the AI completely ignores what relics you have, the act 1 boss prediction should have been way lower if it accounted for the tori
After watching the ironclad run I have to say this ai is BAD.
I know mark of pain isn't good even with evolve, but evolve is very good with mark of pain, especially with wild strike and immolate.