ALIGS - 28 October 2024 Clemens Possing, Stochastic Approximation in MARL

We've heard in our previous talk that stochastic approximation methods can be used to analyze a broad family of multi-agent learning scenarios. In this talk, we will delve a bit deeper into stochastic approximation theory and its applications to multi-agent learning. I'll try to give intuitions of the connection between long run / finite time behavior of algorithms and properties of an ODE. We'll do this via examples such as stochastic fictitious play, and its cousin Exp3. Time permitting, we'll also look at more recent applications, including actor-critic algorithms, learning to play Markov strategies in repeated games, Folk Theorems, etc.