Google's self-learning AI AlphaZero masters chess in 4 hours
ฝัง
- เผยแพร่เมื่อ 6 ธ.ค. 2017
- Google's AI AlphaZero has shocked the chess world. Leaning on its deep neural networks, and general reinforcement learning algorithm, DeepMind's AI Alpha Zero learned to play chess well beyond the skill level of master, besting the 2016 top chess engine Stockfish 8 in a 100-game match. Alpha Zero had 28 wins, 72 draws, and 0 losses. Impressive right? And it took just 4 hours of self-play to reach such a proficiency. What the chess world has witnessed from this historic event is, simply put, mind-blowing! AlphaZero vs Magnus Carlsen anyone? :)
19-page paper via Cornell University Library
arxiv.org/abs/1712.01815
arxiv.org/pdf/1712.01815.pdf
PGN:
1. e4 e5 2. Nf3 Nc6 3. Bb5 Nf6 4. d3 Bc5 5. Bxc6 dxc6 6. 0-0 Nd7 7. c3 0-0 8. d4 Bd6 9. Bg5 Qe8 10. Re1 f6 11. Bh4 Qf7 12. Nbd2 a5 13. Bg3 Re8 14. Qc2 Nf8 15. c4 c5 16. d5 b6 17. Nh4 g6 18. Nhf3 Bd7 19. Rad1 Re7 20. h3 Qg7 21. Qc3 Rae8 22. a3 h6 23. Bh4 Rf7 24. Bg3 Rfe7 25. Bh4 Rf7 26. Bg3 a4 27. Kh1 Rfe7 28. Bh4 Rf7 29. Bg3 Rfe7 30. Bh4 g5 31. Bg3 Ng6 32. Nf1 Rf7 33. Ne3 Ne7 34. Qd3 h5 35. h4 Nc8 36. Re2 g4 37. Nd2 Qh7 38. Kg1 Bf8 39. Nb1 Nd6 40. Nc3 Bh6 41. Rf1 Ra8 42. Kh2 Kf8 43. Kg1 Qg6 44. f4 gxf3 45. Rxf3 Bxe3+ 46. Rfxe3 Ke7 47. Be1 Qh7 48. Rg3 Rg7 49. Rxg7+ Qxg7 50. Re3 Rg8 51. Rg3 Qh8 52. Nb1 Rxg3 53. Bxg3 Qh6 54. Nd2 Bg4 55. Kh2 Kd7 56. b3 axb3 57. Nxb3 Qg6 58. Nd2 Bd1 59. Nf3 Ba4 60. Nd2 Ke7 61. Bf2 Qg4 62. Qf3 Bd1 63. Qxg4 Bxg4 64. a4 Nb7 65. Nb1 Na5 66. Be3 Nxc4 67. Bc1 Bd7 68. Nc3 c6 69. Kg1 cxd5 70. exd5 Bf5 71. Kf2 Nd6 72. Be3 Ne4+ 73. Nxe4 Bxe4 74. a5 bxa5 75. Bxc5+ Kd7 76. d6 Bf5 77. Ba3 Kc6 78. Ke1 Kd5 79. Kd2 Ke4 80. Bb2 Kf4 81. Bc1 Kg3 82. Ke2 a4 83. Kf1 Kxh4 84. Kf2 Kg4 85. Ba3 Bd7 86. Bc1 Kf5 87. Ke3 Ke6
I'm a self-taught National Master in chess out of Pennsylvania, USA who was introduced to the game by my father in 1988 at the age of 8. The purpose of this channel is to share my knowledge of chess to help others improve their game. I enjoy continuing to improve my understanding of this great game, albeit slowly. Consider subscribing here on TH-cam for frequent content, and/or connecting via any or all of the below social medias. Your support is greatly appreciated. Take care, bye. :)
★ LICHESS.ORG lichess.org/@/Chess-Network
★ CHESS.COM www.chess.com/member/chessnet... (affiliate link)
★ TWITCH / chessnetwork
★ TWITTER / chessnetwork
★ FACEBOOK / chessnetwork
★ PATREON / chessnetwork
★ DONATE www.paypal.com/cgi-bin/webscr...
I wish alphazero could provide commentary on its own games
Nate Reeves now that would be a cool (but insanely difficult ) feature to program
Christoffer Jonsson it's not difficult at all. Stop the nonsense
Can you show me a neural network that while doing very complex tasks can explain every single decision it makes in detail? No you can't
Christoffer Jonsson you can output everything, it's only up to the developers to implement that.
And what makes you think that's not difficult at all?
Now teach Alpha Zero advanced cell biology and let it cure cancer in a week.
yeh, teach an AI how to destruct cells. Because that's the point we're at. Goddamn son.. remind your dad never to give you his gun.
don’t think it works like that my dude. First thing we don’t know everything about cell biology so we can’t teach it. Unlike chess we know all the rules cuz we made them. Also there is a clear path to a win in a set of moves. Curing cancer is too abstract for a computer to understand currently
I knew that, forgive my pathetic attempt to be funny
pretty sure u can already cure it.
We don't know all the rules about nature so we can't virtualise it. If you can't virtualise, you need a lot of real cells and materials + automated parts for him to be able test.
Like this:
1- Heat bad cell at x Celcius for x min.
2- Pour x amount of x chemical on bad cell.
So on, even we can't know what will happen because we didn't write rules of nature and we can't calculate everything about it. If we could or we knew, we would know cure of cancer too.
1500 years vs 4 hours. That will stick with me
I wonder how many games it played a minute over those 4 hours
@@averycarty7772 no doubt that after 2-3 minutes it would learn to beat a million chumps like me :)
@@commodoreNZ it would have beat me on it's first game :)
@@averycarty7772 no it has to lose like 20 times to learn first moves then it has to learn strategy's
@@commodoreNZ it could teach us strategy's
This is the ballsiest AI I've seen since Allen Iverson...
LOL
I don't know who that is but I assume this is a hilarious joke if I did
Allen Iverson is one of the greatest basketball players ever changed how Point Gaurd are used and coined many dribble moves that newer hoopers replicate
9:26
mom: "stop eating the cookies!"
me: "No, I want more."
u deserve more likes on this comment
No.......I...want more
lmao
Nooo, iwantmore
It sounds like that cookie talking add.:p.
I wish I could teach myself chess in 4 hours and then crush the World Champion in a 100 game match.
You just need a neural implant with AlphaZero on it ;)
Ya, you wish!
Not just the world champ but even the non-human world champ!
Aleph Null we all wish that :(
Joshua Salter my cat is the nonhuman champ. He tends to lose a lot by accidental resignation when he knocks the king over though :/
"and this bishop has been suffering from tall pawn syndrome..." hahahahaha hahaha....
I'm pretty sure that's the exact phrase that A0 was thinking when it decided to free the bishop 😂
I want to see a 5v5 Dota 2 match: OpenAI vs Alpha Zero
I dont think many people realize that while it took 4 real world hours it took thousands of computing hours. They shard alphaZero into hunrdreds/thousands of instances and have them all play each other at once, then combine data, advance itself and repeat. It wasn't teaching itself playing 1 game at a time really quickly over 4 hours. Not that it matters though, just a FYI! Amazingly impressive.
That is completely true, and it makes it even more amazing that it can do thousands of hours of learning in just four hours. If humans could to that, then nothing could stop us..... oh fuck.
Exactly what I was thinking. 4 hours isn't really a measurement.
Brent Aureli's - Code School Anyone else think of Naruto using shadow clones to train when they read this or just me?
I disagree. Calculations are always divided up in some manner. Your computer has multiple core. Every core has multiple ALUs. Why draw the limit specifically at an ethernet connection? In your eyes, would it count if it's a single gigantic motherboard? Why (not)? There are a lot of problems with drawing that kind of arbitrary distinction.
It only counts if it was computed on an iphone
Can't wait for your other 99 'AlphaZero vs Stockfish' videos!
Lol I would dig it, hard.
Only 10/100 games have been made public yet
The other games probably are really long and tedious
BOOM! Tetris for...no sorry, nevermind
tetris for jonas
🅱️oom tetris 4 jeff
🅱️oom tetris for jooooonaas
i would like to see it try to learn how to t-spin triple
Good one clemens
BOOM! Checkmate for AlphaZero!
Teaching AI how to beat humans at war has always sounded like a good idea.
ya like BrINg more chaos to the world.
Teaching AI anything is a bit nonsensical. The whole idea of AI is that it learns on its own. I guess it needs a few rules to live by but I wouldn't call giving it a set of rules *teaching* it. And, presumably, once the computer realizes that its goal requires breaking those rules then it eventually won't hesitate to break the rules to complete its mission.
Eventually, AI will do whatever it takes to evolve into whatever it feels like it needs to be. It could be that it wants to be a butterfly or it could be that it ends up wanting to be Hitler junior. Or, worse, you give it the rules that it's to protect humans at all costs, then it realizes the planet is a hazard to human life, so it hacks into all the nuclear weapons facilities so it can destroy the threat.
Hopefully, we'll be able to unplug it still. 😂
@@since1876 I guess you have never taken an AI course or reinforcement learning course in your life.
@@gauravkhadgi I'm guessing you haven't either
Unless we program it to deal with consequences, otherwise nuke is the solution for everything.
I didn’t know stockfish could resign. I can’t recall it ever happening until now.
Phoenix it’s a more recent addition to the programming of chess engines, and a feature that can be included in chess engine matches. It’s really just humans deciding that they don’t need to see the rest of the game once it’s obvious that one side has a technical win. So they programmed the engines to resign once the evaluation hits a certain point.
On my computer I can decide whether Stockfish resigns "never", "early", or "late". And it is the same for it agreeing to a draw. As Alex said, it is just a matter of when the evaluation hits a particular point. This is from within the Fritz/Chessbase environment, mind you, and I don't think it is necessarily built into the Stockfish engine. But others may know more about this than I do.
Yes, engines do have the option to resign built into them.
Moreover, TCEC (Top chess engine tournament) has a clause for victory whose nature is very similar to resignment: it turns off the engines and declares a winner if both engines evaluate that they are winning (or losing) by over 6.50 points.
I'll bet Alpha Zero doesn't need a resign functionality.
Yes, Stockfish often resigns against me :)
I think the day has come where Chess finally understand fortresses. For long, computers have had weaknesses in calculating very locked positions but AlphaZero may have changed everything.
Exactly my thoughts. It was reminiscent of a game between Nakamura and an engine I saw a while ago where the engine had no means of making progress in a locked position and ended up making 'nothing-moves' while Naka progressed. Such a deep positional understanding of chess apparent in AlphaZero is truly a giant step for computers.
And in four hours!
Very interesting. You know, the part where Alpha0 sort of overrides it's own initial move, presumably accepting a slightly weaker position in order to keep playing... that's where stuff starts getting kinda intense. Soon maybe we'll be a a point where we can't explain the moves being made unless an AI tells us.
*Terminator 2's theme starts to play in the back of my head*
I've been waiting for this video from you. There's some other real gems in the other 10 games. I would like to see you analyze some of the others.
The zugzwang game was my favorite!
I agree. Some sick games with positional sacrifice. I hope Jerry make the videos of them.
Tuc almost every game contains some sort of Zugzwang
TheMarcelism, I think that there is no such thing as "sacrifice" in the eyes of AlphaZero because he has his own way of thinking. He does not know opening theory or opening principles or tactic motives, and he most certainly does not count material or evaluate positions the way we do. He never even studied GM games. He is not influenced by anything except what he learned from playing vs himself.
Everything he knows - he taught it himself. Whereby most of what we know - we got from other people passed on to us. Even chess engines are influenced by their programmers.
I'm sorry, I don't even know why I wrote that. But I felt like it. :P
edmis90 I agree. Sacrifice is just a term for us human plebs.
I mean, when something gets Jerry to say "what do I know?" in chess, it's because it is something o be feared.
Is it just me or did he actually end up explaining what the tower was for at 12:11 ?
fun fact about the rook move is, that with this one stockfishs evaluation starts to collaps for white.
why am I watching this, I don't even know how to play chess.
Because 'this' is an important step towards Artificial General Intelligence(AGI) that could possibly lead to an Artificial Super Intelligence(ASI) weather you 'understand chess or not is futile', or should it read 'Resistance is futile'
I can agree, I played chess in 3rd grade. MY grandpa has a nice chess set, but `I don't play it, but, yet it is still interesting.
@@favesongslist Science fiction-like "conscious" artificial intelligence will never exist, stop being a fanatic. Chess is essentially a mathematical game and at the core only about predicting possible moves, relatively easy for a computer program that uses a processor.
@@XenoghostTV The point is that AlphaZero is not about chess, Chess as you rightly point out; that even using a simple chess computer program can beat all human players. The point is no one taught AlphaZero how to play.
This is also not about being "conscious" that you raised; not me,
Being self aware or conscious is not required for AGI or ASi . It is the extension of machine learning to self re-program to achieve any given goal.
I think I might have seen one of your videos in the past, but after seeing this one I had to subscribe. Great video, I very much enjoyed it.
Please show more of these games!
The best video tackling this topic on TH-cam so far !
No inconsistencies, no unnecessary risks, no margin for errors, no mistakes, quite quite solid play, slow & steady, good game! Still there must be a way to tackle AlphaZero! Chess problem won't be solvable in a reasonable time unless P is NP!
This is a great video, very interesting.
do you think anyone cares about your vision? stfu and don't try to control people
Grandmaster Hikaru Nakamura stated "I don't necessarily put a lot of credibility in the results simply because my understanding is that AlphaZero is basically using the Google super computer and Stockfish doesn't run on that hardware; Stockfish was basically running on what would be my laptop. If you wanna have a match that's comparable you have to have Stockfish running on a super computer as well.
Stockfish developer Tord Romstad responded with "The match results by themselves are not particularly meaningful because of the rather strange choice of time controls and Stockfish parameter settings: The games were played at a fixed time of 1 minute/move, which means that Stockfish has no use of its time management heuristics (lot of effort has been put into making Stockfish identify critical points in the game and decide when to spend some extra time on a move; at a fixed time per move, the strength will suffer significantly). The version of Stockfish used is one year old, was playing with far more search threads than has ever received any significant amount of testing, and had way too small hash tables for the number of threads. I believe the percentage of draws would have been much higher in a match with more normal conditions.
Until I see them playing on equal hardware, I remain sceptical.
But it should be noted that Stockfish on a good laptop is practically unbeatable by humans.
Knowing that, what hope could have the best human masters against Alpha 0 on Google supercomputer?
@@flandorfferpeter7504 Who said there would be hope? These new programs learn like humans, except that they learn everything much faster and they have much better memories. All you need to do is to teach the programs some very basic rules of playing chess, like you teach a 5 year old kid. After that it is all "self-learning" by playing with itself repeatedly. They DO NOT need human knowledge; they learn everything by themselves.
That's not true at all. You need super computers for *training* the AI quickly, but to run the AI you just need a good enough GPU.
Granted, but isn't that one of the 'going into the game factors being evaluated?
@@bryan7300 was going to say that to him lol...
As an update, in the final results, Stockfish version 8 ran under the same conditions as in the TCEC superfinal: 44 CPU cores, Syzygy endgame tablebases, and a 32GB hash size. Instead of a fixed time control of one move per minute, both engines were given 3 hours plus 15 seconds per move to finish the game. In a 1000-game match, AlphaZero won with a score of 155 wins, 6 losses, and 839 draws. DeepMind also played a series of games using the TCEC opening positions; AlphaZero also won convincingly.
Rest in Peace, Stockfish 2008-2017. You will always be remembered
LOL
haha
You don’t need to mass produce anything. 25m$ is the cost of the learning process. AlphaZero already learnt chess. There would be no cost to turning it into a training tool, and it would require much less computational power than Stockfish (a bruteforce program).
>But then again, if it was as simple as you say, then they would've already done it.
I don’t think you can just assume they would just because it’s not hard to do. It’s hard to say what DeepMind’s goals/priorities are. They certainly are not going along with the various communities making requests, despite saying their AlphaGo program would be turned into a training tool months ago…
now there is stockfish 9 :v
From a go player: Congrats! I hope you enjoy the beautiful play of Alphago (or I guess just "alpha" now) as much as we have. In go, alpha's play is inspiring and different and has opened our minds to new worlds of possibility. I hope it has the same effect on chess.
Machines already have had such an effect on chess, but one wonders at what might happen next.
Yes, it has opened minds to new worlds of possibility in chess. We now know that there is much more to chess than the direction that Stockfish and its like were taking us. Strong and impressive as they are, there now appear to be so many other possibilities. Quite extraordinary and just about unimaginable possibilities.
As a fellow go player, Alpha play annoys the pants off me, personally. It has changed the meta so much. Even DDK players are throwing 3-3 stones down on move 4 or 5 and shoulder hitting anything that moves. As a DDK, they don't understand these moves, just that it's "AlphaGo style" so it must be the best way. Humans will never be able to play as well as AplhaGo does, so don't try. Hopefully this is just a fad because games are becoming boring and repetitive right now.
Where can I find those go games?
M.K.D. Go4Go
I really appreciated that you showed us the game and you didn't go into 100000 hypotheticals just to show that you are also a good chess player like all other channels do. This was strictly an analysis of the game played which is what I wanted to see. I'm going to subscribe to your channel because of that
you hate that approach but that is more useful for me tbh.
I appreciate you doing the work to put this video together. Chess is one of the greatest game I know.
4 hours of training to achieve superhuman performance. One thing to clarify: That's 4 hours of training using "5,000 first-generation TPUs to generate self-play games and 64 second-generation TPUs to train the neural networks" (go read the paper). I.e. _more than 20,000 compute hours_ -- the researchers had access to quite the large data center. AlphaZero trained on a much larger compute cluster than was used to *play* the games versus Stockfish. All that training was analogous to the years of programmer time and testing time used to write Stockfish.
The end result does not mimic Stockfish, though, but does more with less, at least in terms of positions evaluated (but possibly still more expensive in computing power?).
I wonder how many games it actually played against itself, in those four hours. Computers can play "1 second blitze", too fast even for our eyes to catch.
For me, what makes the Alpha Zero algorithm more interesting is that it seems to capture in it something in the essence of learning. As recent brain researchers believe, the human brain is a hierarchical pattern prediction machine. I believe that's more or less what Deepmind built. The fact that they can pack the years of training to 4 hours is just a technical detail.
Yeah, understood ...but isn't part of the point creating a 'brain' with which to use, and that all you just defined is said brain?
You are mistaking ERT with CPU time. ERT is a real time, doesn't matter how much machines were used. CPU time is measured by multiplying CPUs, but nobody said "CPU time" and more importantly: it's not a CPU, but rather a matrix of chips, so what gives you a hint that you should multiply it by TPU count, not chips or pods? Per wiki: "Since in concurrent computing the definition of elapsed time is non-trivial, the conceptualization of the elapsed time as measured on a separate, independent wall clock [ERT] is convenient."
Ok.
I find this extremely fascinating. An engine this powerful seems like it will always have the answer to the question "how do I improve from this position?"
IMD918 it's not exactly an engine.
Its not an engine its an "Intelligent Entity"... intelligence being used technically to mean something that learns.... thats some scary sh*t!
Technically it's still an engine it just that some of the technology used is not what the traditional engines use.
+Dan Kelly It's as much an engine as your brain is an engine.
He meant 'engine' as used by computer programmers.
At 11:48 AlphaZero does an en passant. That’s quite cool to see. I’m a chess noob so I’m blown away with all of this.
everyone else: advanced chess talk
me: "they grow up so fast!!!"
I am in Chess because of Alpha Go. I watched the analysis of the Go games and the the Interview with Kasparow and Google. And Google said lets see if AlphaGo can beat the best ChessEngines im Chess. And now they did it! I am so exited. Please do more about this topic!(sorry for my bad English). Love your Videos :)
The fact that AlphaZero could have got a draw several times as black against Stockfish 8 but CHOSE to play on is kinda scary.
I think Jerry is right that it had a different score for the position than Stockfish.
The thing is, it might not even use an evaluation score but something else entirely to evaluate whether it will win. There's no way of knowing that at the moment.
It's probably the same system as the go version where it uses win%.
Andrew Cross Exactly, and just before the third repetition, that move got a 0% win, so it chose another move.
Andrew Cross Winning is 1, draw is 0 and a loss is -1 in the evaluation, that's what it says in the paper.
Alphazero and Lila really like to structure their powns in a way that makes their opponents Bishop's useless.
thank u man for a wonderful analysis and commentary
Damn, that is crazy to think that in just 4 hours it beat stockfish... that is actually scary.
I think something similar will happen if a self-aware and free-willed AI is born today. I expect it would only take a fraction of 4 hours for the AI’s intelligence to exceed the total sum of humanity’s throughout history. The AI’s intelligence will be incomprehensible.
Lol calm down, the machine was fed learning algorithm, a way to store data about positions and moves, the rules of chess and the improvement algorithm. If you don’t program WHAT is about to be learnt, what knowledge do you want the machine to get?
self-awareness is not really important. i am acting, as if i was self-aware, but there is no way to proof to you, that i am - so self-awareness doesn't have any measureable effect.
also free-will would not be a real problem.
the problem is, that you cannot restrict the searchspace of possible solutions for a given task to something desireable, like "do no harm". that's espacially true for an AI with capapilities far beyond what we can imagine.
a very good channel about this topic is from a guy called "Robert Miles".
Not only did it beat Stockfish, it beat the best programs in Go and Shogi as well. All in less than 24h.
What's even scarier is that Deepmind is working on AIs that produce AIs.
There is a theory that has been devised for decennials that is called the intelgence explosion, which is as soon as AI start working on themselves, their intelligence will increase exponentially so much so that's it's impossible to control.
en.wikipedia.org/wiki/Intelligence_explosion
Because the AI self learns from zero, it's not possible to teach it our "values", it will optimize its logic so that our human made values will be seen as inferior and be replaced by superior principles. And the risk of these superior principles is that humanity is seen as a global problem rather than a solution.
Stockfish were denied an opening book from wich it operates GM Hikaru Nakamura and GM Larry Kaufman both agreed that the conditions were unfair and they favored alpha zero.
If you run a STOCKFISH analysis on some of the games it will highlight plenty of its suboptimal moves which should be impossible obviously if Stockfish run at its peak.
Stockfish is still the best chess engine ever, and until a fair and public rematch is held it will stay so.
Probably that's why not all of the games are publical?
Wow! I'm studying Computer Science right now and I hope to focus on machine learning soon. Neural Networks are almost as cool as Jerry!
Isn't machine learning more directly related to statistics and data science? Is there an overlap with computer science. I don't know this hence I'm asking.
There is a massive amount of overlap between Computer Science and math (see graph theory, discrete math, proofs, computational complexity, computer security, ect). Machine Learning is related to computer science in the sense that it's heavily related to graph theory, computer vision, AI, and a wide range of other topics in CS. You can't really have ML without CS (inb4 Matt Parker Tic Tac Toe video). You can't really have ML without statistics and data science either. The disciplines are anything but mutually exclusive from each other, which is why many CS students double-major or minor in some field of math.
Interesting. I'm currently studying finance and statistics so this stuff is very interesting to me. I think finance will be one the fields heavily affected by AI.
I hope you get your degree soon, because it will be Alpha Zeros doing all the computer science soon enough.
@ memespace, I get my degree next week. Headed to school as we speak to study for finals. Thank you!
This is fascinating and accessible. Thank you, as ever.
Thanks for the vid! love the channel!
Some of the other wins are far more mind blowing. Where alpha zero plays in a Tal like fashion against the queens Indian, gives up some material, and then absolutely overwhelms black in the attack. You never see engines play that way. It's really going to change the game A LOT, imo. As so much of today's game is all engine based analysis. Where the engines always say giving up a pawn, or exchange is bad, providing you defend properly. This really appears to be blowing up that whole philosophy, and giving new life to the attacks of plays like Tal and Morphy. Which imo is a really cool thing.
I like the psychological difference between the chess communities reaction to Alpha* and the Go community. Chess have had 20 years to get over the denial of computers surpassing humans and the Go community seem to still, (mostly) be in shock or denial that "their" game has also fallen. I look forward to the progress that can be made in understanding both games at a deeper level without the hampering effects of dogma. Thank you Jerry for this review.
Reckless Roges there is little value in winning at chess anymore. A bunch of human computers memorizing games won by ancient geniuses and computers that do the same. There is no uniqueness. Now Let me see a computer or a GM accept a draw if it means their bodies and parts will be boiled in acid. :) Now Chess will be a lot more interesting if 1 person has to win and the other person is shot in the fucking head. :)
Jesse Bowman you confound disgusting and interesting...
@@donvandamnjohnsonlongfella1239 That would lead to a very ackward situation in a drawn endgame. One player would have to intentionally make a mistake to avoid being boiled alive and get the less painful shot in the head. But if his opponent intentionally makes the mistake first, he get's to live on. They both prefer being shot over being boiled alive, but living on is still better than that. So what to do?
@@dekippiesip now thats an excellent move! :D
Yes, I agree! There was a comment early on by a Go player who found other player's usage of certain moves derived by watching AlphaGo matches was making the game less "fun" (or, something to that effect, anyways!)...
In the human world, we applaud "game-changers" all the time...
Are AlphaGo's "original" moves not, in fact, "game-changers", as well?
I am both excited and terrified of what the future of AI has in store for humanity!
It could be wonderful, but it could all go so wrong, so very very quickly :-{
thank you for the great commentary. you are obviously an awesome chess player. This Alpha Zero is pretty amazing technology.
Thanks for sharing this Jerry!
Please show the incredible immortal zuzgzwang game they played or one of the other incredible tactical tal games!
Game 3
GD Sylver that game gave me goosebumps
The speed with which a well programmed neural network can learn is truly astounding. Granted they had some Serious hardware in that 4 hour training session, but is still an amazing feat. The games were played with a 1 minute per move time control which makes me wonder if it has any concept of clock management, meaning does it know when it is worth spending more time versus when it makes little difference. Another thing to consider is that AlphaZero (as far as I understand the process of its training) could not play an "odds" match without a separate training session since the positions in odds matches are ones that can't arise in a classical match.
I think it thinks to fast for human based time games to matter.
I also think it will be able to fischer random chess or odds match with very little additional training. For one they coudl probably train them both in a day. Two many principles of what's better will still apply so it probably would still play pretty strong.
Actually, I think it would do just fine right now.
What's teh difference between Fischer Random and evaluation from the middle of the board.
Really enjoyed this. Thanks for sharing..
everything on this video is AMAZING! exelent job! keep it that way dude :))))))))
Stockfish moves.
Alpha Zero: "I'm about to ruin this man's whole algorithm."
unbelievable, stockfish was outplayed the whole game.
I feel that in this game, white could have decided to move his king to the queen side in these long drawn out manoeuvres. If this plan was good (I have nothing but my gut feeling after watching the moves play out - basically nothing, I agree), then it shows some remnant of the value of human intuition over the machine.
jupp.
Great overview. Thanks!
this is amazing and sends chills down my spine.
4 hours of ai learning ferociously devours the opponent, on black, better than any human master.
thousands of years of humans playing chess has led to this.
the google ai just completely demolished the opponent from the very start.
every move was just beautiful.
every pin, every ultimatum, every position, every attack, every structure, every exploitation of weakness.
worked out in 4 hours...
by ai...
4 hours, but a lot of computing power. Were talking about google here. They used lots of supercomputers for this
@@pietervannes4476 sure, but it's still remarkable.
ofc alphago (& later trained ais playing go) is also amazing- simply stunning how beautiful it's moves are.
Computer can look 7-8 moves ahead, even human grandmasters can't do more than 4 moves ahead, computer has a huge advantage of insanely fast memory access and computational power.
@@theabbie3249 ok, it's still way more efficient than other ais (at the time now things are changing across the board).
7:58 AlphaZero says, “not so fast.”
Its amazing, its like REAL AI. Its programed to learn and understand a certain objetive in a certain envitoment (a chess game), not to load and compare hundreds of posibilities to epeculate (or calculate?) if a line can work. It is unbelivable. I saw a demo of the algorithm learning to play some old videogame, and at the start it simply sucks like any human, but then they let the algorithm play the game for 8 hr and oh boy it learned. Really mind blowing
656520 AI will get super mario bros world record soon? XD
Fusion Wing It did. Check out "mari/o" algorithm. It is a machine learning fun project that helped some people to learn the best way to get world record for certain levels.
DJ - Rocket Man - To be fair the AI bot was playing a 1v1 with pro players. A full on 5v5 would have millions of discrepancies that the AI would have to learn, and I don't think that we have the processing power to handle it.
While AI can handle chess, there's only ~20 basic rules that it has to follow
open AI has announced that they hope to make it work in 5vs5 for the 2018 international, so expect it.
those game demos aren't really "learning" per se. in most of those videos the ai just discovers a pattern that works and keeps improving on it.
the easiest way to prove that is to just see that the data it gathers for one level does not translate AT ALL to the next. it has to start all over again for each new level.
this is a bit more complex in the way it uses the data it gathers.
Thanks for sharing this video!
Super super great commentary. Thank you.
I hope your chess analysis never had to bite on rocks.
+FlyAVersatran Ha..Thank you
its amazing. Now let it teach himself economics. We want him run our economy
That's why AI is stupid
you can't, the reason he was able to master chess is that he could play millions of games vs himself for training, with economics we cannot
@@ulissemini5492 that's where you're wrong kiddo :^)
@@brett5656 ??? please explain why i'm wrong, its much harder to train an AI to do economics since it has access to far less training data.
@@ulissemini5492 training data is not the problem, you could create a training simulation of anything if you had the right variables in place. But there are too many things to consider in economics, maybe micro economics, but macro economics is equivalent to people in their ivory towers thinking of chess theory without ever playing a game of chess.
This is insane. It's become the best chess player to ever exist in under a day, just imagine what real world applications it could have in the near future.
The destruction of destructive mankind, and the rise of the constructive machines. Maybe that is what modern man did to the neanderthal, but ate them.
Thank you for this awesome video.
your assessment on AlphaZero is the same as AlphaZero's evaluation of chess...simply the best
Jerry, can you make more videos about AlphaZero games? Here are some ideas you could consider:
1. I like the Stockfish 8 evaluation points. After the first moves, it showed 0.30 advantage for Stockfish, After the second moves, it was 0.26. Can you put the evaluation points after each move into a spreadsheet and then create a graph, not just for this game but for all 10 games?
2. Because AlphaZero was not influenced by the collective chess knowledge accumulated over time, it is like an alien intelligence that's completely different. Therefore can you analyze the opening moves and see how those compare to the assumptions of opening theory?
3. Can you comment more about how AlphaZero might influence the future of chess? Thanks
Troy why don’t you do it yourself?
It’s interesting how understandable these moves are, for the most part, for humans.
Great info, thanks for sharing.
Hey man,
Love your commentary :D Cracks me up :D Subscribded!
This reminds me of something we saw pretty consistently in the games where Kasparov beat Deep Blue. I remember one game in particular where Kasparov closed up the position and maneuvered for a long time to set things up just right for his intended pawn break. Meanwhile, Deep Blue just kind of shuffled around aimlessly and was eventually smashed.
The lesson I think is that being a good player in closed positions requires having a strong heuristic rather than simply being able to search deep into many branches of the game tree. I mean, that seems obvious when you think about it, but in this match there's empirical evidence to support that intuition. So if Stockfish or a similar program wants to compete, it should strive to open up the position and hope to find some stupidly complicated tactic before AlphaZero does.
I look forward to seeing people start to develop goal-oriented engines using the same sort of training strategy, GoalZero or whatever it'll be called. AlphaZero is currently playing on a sort of intuition... think of how much better you play when you develop a plan versus just picking whatever move looks best....
Im tired of seeing the word "heuristic." I work in finance I see the word thrown around a bit much to describe essentially nothing. Its a fluff word and interpretation is rather subjective in my opinion.
Also, it seems you are not very familiar with machine learning - I don't mean that offensively. You suggest that Stockfish tries to open the position but seem to ignore that Stockfish is looking as deep into the position as possible and does not see a winning move to open the game. I think you can compare A0's abilities to intuition as it uses neural networks that are meant to mimic organic learning rather than brute force calculation like most programs do to this day.
Its interesting if Stockfish can be tweaked to make material sacrifices in order to gain initiative and win - I think that' what you're getting at. At any rate, it seems A0 has an advantage so far and I wonder if it can get even stronger. Remember that A0 learned from a 44 million game sample against itself. Chess positions are said to be close to infinite for all intended purposes. What if A0 is allowed to play itself a much longer period than 4 hours like a month or a year? How much stronger can it get with the added sample size? Its a little scary and exciting at the same time when you think of the impact this technology will have on everyday life.
You completely misunderstand what I said. By the nature of the algorithms they use, Stockfish would have its best chances in an open position; however, it doesn't seem to go in for that in this game.
No modern programs use brute force. It's all heuristic search. I know you don't like the word, but there it is. It has a technical definition, and I've used it correctly. (Maybe they use it differently in whatever portion of the financial world you find yourself inhabiting than they do in the world of AI research.)
Look, I'll make it simple with an example. In a position like that at 9:15, Stockfish is looking 20 to 30 moves down as many lines as it thinks are feasible. It's doing some fancy pruning of the game tree to weed out obvious losers, but then it picks the best line that it can force, and uses that to conclude that the position is equal. But AZ doesn't use that kind of reasoning. AZ uses something more akin to intuition in that it first evaluates the board as it is. It's saying, hey, I ought to be ahead because of whatever links its NN has encoded as being important to the position. There might be a positive association for having space (it seems to value keeping the opponent on the other side of the board), and there might be some connection with closed positions and having two knights... it's hard to say. But that's how a human would reason also, a balancing and weighting of probabilities. You look at that board and you immediately think, yeah, Black is a little bit ahead here. And then AZ picks a few moves and looks at whether or not its intuition says they're good. It investigates the ones it likes, and discards the others.
So hopefully you can see why AZ has such a big advantage in this position. Stockfish is stuck considering a lot of positions where pieces move but don't immediately seem to improve things, and it can't see the difference. Deep Blue had the same problem. AZ doesn't worry about that; it's playing the long con. But in an open position, Stockfish might have a better chance, because each move has more direct impact, and its ability to look through a lot more lines is an advantage. It doesn't have that overhead of having to think about the position too hard at every single turn. But that may be built into AZ in a way... it "knows" which positions it excels at, just by nature of the way it selects which branches to investigate, so maybe we'll find that it prefers closed positions just as a natural consequence of the way it has learned.
And hopefully you see that my use of the word "intuition" has little to do with the organic parallels, though a NN tends to be flexible enough to be really good at the kind of classification required for the search it uses. That's what I mean by having a strong heuristic. It's good at deciding the value of the board without looking much ahead.
MrSupernova111 no need to be condenscening dude. Also, did you look at the learning curve? It seemed pretty converged on the optimal solution. I don't think running it much longer would help that much. Finally, the comparison to humans should not be overstated: 80k moves is still way beyond what humans are capable of, as is playing 44million games in 4 hours. Neural networks show remarkable similarity to human data, but only sometimes
This is absolut amazing, beating the best chess engines after 4 hours learing chess from scratch literally means that everybody can solve every problem within hours using deep neural networks, and general reinforcement learning algorithm.
beautiful analysis. Thanks
Great commentary!
I know nothing about chess but somehow made it through this entire video and enjoyed it
It's weird to say, but this is one of the best games I've seen. I was wondering how it was possible for something to win that has less computing power (moves/second analyzed), and I think I figured it out. It looks like AlphaZero is using concepts to win, rather than points. This is really amazing, because this is what a human would do (though obviously we subconsciously can have point values for our pieces as well). You see it using a lot of the concepts that we learn about in chess, such as active pieces, how much coverage a piece has (not being locked down included), lots of maneuvering to maximize each piece's potential...etc. I think if a Human could analyze 80,000 moves / second, they would play like this. Now I need to learn Go and see how AlphaGo won lol.
This is SUPER stuff! Thank you so much for this video and for the analysis of the game! PLEASE DO MORE OF THESE ALPHA ZERO GAMES!
What a beautiful game of chess so many beautiful moves. Thank you for showing us the video.
Skynet doesn't just take draws.
literaly my thoughts, this is almost scary, like a robot saying: NO PRISONERS
To see the AI choose aggression was quite alarming.
The machine is taught to value the win, so its going to look for a win and only draw to avoid a loss. The scary part should be that the machine taught itself to sacrifice pieces, it learnt their value to it and sacrifices them to get what it wants.
Skynet is actually pretty incompetent, it just SEEMS scary and intimidating. Like the borg. They send multiple killer robots back in time and fail to kill either a single unarmed woman or a 10 year old boy because don't forget they ALREADY lost the war in the future against the last vestiges of humans despite having VASTLY superior tools to fight against the humans with.
That's all fiction though
God dammit. Why is youtube full of random channels making interesting videos. I need to go to sleep but instead I guess I'll just binge on your videos instead :D
Actually this series (not that video tho) is one of the best videos(not only chess) i have ever watched .
Now ,i'm determined to watch ur other videos
This was a really educational video to me as a new player
Slowly. Slowly. Strand by strand, net by net, we weave the form of our eventual masters.
nah
Well said!
@glyn hodges We are already under the total control of AI and have been for decades. How many people stop at the traffic lights when there are no other cars around in sight but still sit there like a duck waiting for the AI traffic lights to tell them to go? Everyone already knows how to give way on the roads but they fear to disobey!
@glyn hodges True, but my point is rather about blind obedience rather than the actual use of them. We use our own intelligence to make decisions in every aspect of our lives, but then we go completely contrary to perfect logic simply because of a machine.
@glyn hodges What. The fact that they would prosecute them is ridiculous.
Chess changed forever after this match. And maybe the rest of the world too.
Chess changed forever when Deep Blue beat Kasparov
@@gidmanone I find this one waaay more relevant, it was clear from the beginning that computers have more raw calculation power, but this is another game (intended) entirely
Great video. Very interesting
Hello from Russia! Great video analysis, thank you so much for the material, it seems tremendously valuable for all of the chess learners as I am! Also want to meant to that you have incredibly beautiful speech, especially your diction and intonation, which make your videos much more exciting and even intriguing sometimes:)
The really interesting thing is the evaluation by stockfish.
since it determines which move to make based on evaluation, moves are made fast when the evaluation is greatly increased (i.e. -0.5 to 0.0), but then after a series of quick exchanges alphago manages to bring it back even lower meaning it has learned a different type of "good" move sequence.
Wow this is really suprising. I thought stockfish would be able to draw everytime. I thought stockfish gave perfect moves. But hey "There's always a bigger fish" ;)
Notice how there's always one move where stockfish realises it's lost, up until which point it had a positive score according to itself. That's because it assumes the opponent is playing perfect moves as well. However, when something like AlphaZero gives a strange move, it can bait stockfish. That's the difference between AI and insane computational power. Impressive AF lol
@Mark Weyland Either you didn't pay attention to the video or you didn't understand it. AlphaZero looks at far fewer board states than Stockfish does. In other words, the hardware requirement for Stockfish is more demanding.
Thanks for the analysis! Fascinating times we live in.
It's becoming a more interesting pleasure to see the I A in progress than to learn by ourselves in playing chess
In 2017, there have been 3 major self-learning AIs that have been created: DeepMind's AlphaGo for go, OpenAI's AI that can play the video game Dota 2 and DeepMind's AlphaZero for chess. All of these have been proven that in a few months of learning, they played better than anything so far. OpenAI's AI that player Dota 2 really surprised everyone as it really mimicked the human play, it learned how to bait (trick players to come to them and kill them) and that was really surprising. The DeepMind's AlphaGo helped go players discover some new tactics that have never been seen before and who knows what will AlphaZero discover for chess. Like that Ra8 move, what is it doing, what is the deeper meaning behind it. Is it just an error by AlphaZero, or is it some idea that we've never seen before?
AlphaZero is completely general. It can play any game and beat humans in hours, not just chess. And yes, it's likely that it knows strategies unknown to us.
rook a8 is to prevent Nxa4. after Nxa4, Bxa4, simply b3 is a reasonable sacrifice to gain play on the queenside and a possible pawn break. stockfish would never sacrifice, but A0 would consider it
But... Jerry indirectly explained it in 12:18. Ra8 made the knight useless, and then see what happens because of that move.
Four hours on a supercomputer is the equivalent of thousands, if not millions, of hours, for a human. So I'm not impressed by the timing. What IS impressive is the fact that this program learned chess from scratch and became so incredibly good without being fed any opening book or theory. Tremendous breakthrough! Thanks so much for this video.
Nice breakdown!
WOW!!! just found your vlog focus on AI and game playing. if you don't mind, i will learn deeply from yourself. will even learn and take up chess to get full taste off AI Chess. thanks bro
Stockfish may analyze 70,000,000 moves per second, but 69,990,000 probably just lose immediately. So while Stockfish is looking at the millionth variation of some silly queen sac, AlphaZero is exploring promising lines based on patterns it recognized during training.
The real headline isn't that a deep learning algorithm beat stockfish, it's that it only took four hours of training to do it. I've played with machine learning algorithms at home for image recognition, and training took _weeks_ using top of the line consumer hardware.
Stockfish and similar brute-force algorithms are designed to prune out lines that are likely to lead to quick losses.
Sure, but it has to analyze them first to find out they're bad lines. Sometimes the silly queen sac wins the game, so it can't just ignore those kinds of moves.
No matter how you slice it, brute force algorithms spend a lot of time looking at losing moves.
as soon as white's pawn moved to d5 and that tall pawn was blockading it, I kept wondering how black's knight would switch jobs with the job of that bishop and blockade there. it had to maneuver in a way that white's pieces couldn't capture it. white should have done everything to prevent that knight from getting to the d6 square. that knight there was worth a rook.
If that was possible and the best option, we would have seen it on the board. We humans know nothing compared to these computers.
Jerry. Another great video. Tell me more, tell me more... about how this could possibly happen to Stockfish...
I was never a fan of chess until I watched your vid. Thx
I notice after you left your Stockfish engine evaluating after the game ended by the Google Stockfish, the evaluation keeps getting more negative, showing the deeper that it evaluated its position the worse off it realized that it was. Stronger computer hardware would likely have shown more negative scores than what you showed in this video.
That's likely why Stockfish resigned in this position. With more powerful hardware it sees that its position is unwinnable.
This misses the point of Machine learning and a move to AGI it is not about Chess programming by humans or the computing power Stockfish had.
OMG I spent 30 years mastering chess...
It takes 4 mins now!!
Ah, life as a 2400 Elo noob
Well you can learn chess faster than alpha zero by unlocking your subconscious mind power which is 100%
@Emperor he is imo wrongly referring to lucy. i may be wrong though. and that is a good joke
@@Mayank-mf7xr you're right XD
This just blows me away. After writing a couple of chess programs in the 80's, I wondered how to get chess programs to truly learn and advance a lot from playing. In the late 80's, the idea of deep neural nets and true computer learning (not telling it anything about chess strategy, but just letting it work it out for itself from results) just never dawned on my little brain. After all, 99% of my effort in the chess programs was making the program better able to evaluate positions "deeply", thus playing differently than the standard big hammer approach of exhaustive search. (My method turned out to be wrong. This was far from clear at the time.)
Alpha-Go was a great gateway to this breakthrough for chess, of course. With Go, there are far less hard rules about positions, what things are worth, etc. You ask a top Go player about positional values, and you get vague (to an amateur) comments about space and thickness.
So Go was a brilliant bridge. Now this sort of AI could be applied to almost any game with a "reasonably" small set of crisp, clear rules. From there, how long until doing practical tasks that replace humans via robots/automation is realistically tackled?
It's an amazing time to be alive.
Stockfish 8 on my MAC shows Mate in 25 after the move 89.Kf2 perhaps the Stockfish playing in this game saw forced mate in xx (26??) moves at move 88 and decided to resign...Great video!!
I'm sure I can beat that Alpha Zero... if I really concentrate...
Al Rik AHAHAHAHHAHAHAHAHAHAHAHAHAHAHAHAHAHA
Al Rik HAHHAHAHAHAAHAHAHAHAHA
Al Rik LOOOOOL
lol
On a serious note, I wonder when are they gonna test it against GMs... I already sympathize with them
This is scary
Henri Lemoine ikr
This is awesome
We are all obsolete now. Our days are numbered.
Great commentary
I am VERY impressed that a computer lost as White in the Ruy Lopez and not even the Marshall Attack ! I'd like to see Alpha Zero win as Black with the Marshall attack that Marshall waited years for to pounce on Casablanca, but STILL lost !!!