Big Techday 24: Visualizing Transformers - Grant Sanderson (

Visualizing Transformers
The modern wave of AI, characterized by large language models and generative bots, is largely powered by a neural network architecture introduced in 2017, the Transformer. This talk aims to step through precisely what a transformer is, visualizing how data flows through it, with an emphasis on the attention mechanism.
About the speaker:
Grant Sanderson is the author of the mathematics TH-cam channel ‪@3blue1brown‬, with over 6 million subscribers, and more than 530 million views. The channel is characterized by a strong emphasis on visualizing topics in math, ranging from lessons in linear algebra, neural networks, calculus, topology, algorithms, problem-solving, and more. He also created the open-source mathematical animation software manim, which powers the graphics behind 3Blue1Brown. Sanderson studied mathematics and computer science at Stanford University and has worked at Khan Academy and MIT, producing online courses and lectures. He's contributed to numerous publications for mathematics outreach and education, including Quanta, Udacity, Manning, Numberphile, etc.