Yuan Cao - Understanding Deep Learning Through Phenomena Discovery and Explanation

Abstract: Deep learning has achieved great success in many applications. However, the success of deep learning has not been well understood in theory. In this talk, I will discuss some recent efforts to bridge the gap between theory and practice through phenomenon discovery and explanation. In the first part of this talk, I will discuss the phenomenon of “benign overfitting” in deep learning, and present our recent results characterizing benign and harmful overfitting in training convolutional neural networks (CNNs). In the second part of the talk, I will discuss the recently discovered phenomenon on the generalization gap between Adam and stochastic gradient descent in image classification tasks. I will present an intuitive explanation for this generalization gap and provide a rigorous theoretical guarantee to support the explanation. Overall, this talk will provide insights into the “feature learning” procedure of neural networks, and how it is related to various interesting phenomena in deep learning.