What is a graph, why Graph Neural Networks (GNNs), and what is the underlying math?
Highly recommended videos that I watched many times while making this:
Petar Veličković's GNN video → th-cam.com/video/8owQBFAHw7E/w-d-xo.html
Michael Bronstein's Geometric Deep Learning keynote speech (beautiful!) → th-cam.com/video/w6Pw4MOzMuo/w-d-xo.html
Xavier Bresson's Graph Convolutional Networks lecture → th-cam.com/video/Iiv9R6BjxH/w-d-xo.html
3Blue1Brown’s series on Neural Networks → th-cam.com/video/aircAruvnKk/w-d-xo.html
If you'd like to go further with GNNs, do get started with Petar's wonderfully compiled list of resources to continue → goo.gle/3cO7gvb
Here's also another awesome compilation, to go further with research → github.com/GRAND-Lab/Awesome-Graph-Neural-Networks
Also, the GNN literature is growing so quickly so subscribe to this Telegram channel by Sergey Ivanov to help you keep up → t.me/graphML
Reference blog posts about GNNs:
Michael Bronstein → towardsdatascience.com/geometric-foundations-of-deep-learning-94cdd45b451d (a must-read), towardsdatascience.com/do-we-need-deep-graph-neural-networks-be62d3ec5c59
Amal Menzli → neptune.ai/blog/graph-neural-network-and-some-of-gnn-applications
Eric J. Ma → ericmjl.github.io/essays-on-data-science/machine-learning/graph-nets/
Rishabh Anand → medium.com/dair-ai/an-illustrated-guide-to-graph-neural-networks-d5564a551783
(More recent) Distill → distill.pub/2021/gnn-intro/, distill.pub/2021/understanding-gnns/
Special thanks to:
Seb, Rish and Jet for reading drafts of this and giving such amazing feedback.
Serene for helping enhance production decisions like design, color, animation flow, time-management for my editing and recording (hahaha), and others.
Jay and Malcolm for being there and encouraging the decision to do this video.
Literature References:
Recommended survey → Wu et al. 2020
Convolutional GNN layers → Defferard et al. 2016; Kipf & Welling 2016
Attentional GNN layers → Monti et B 2017; Veličković et al. 2018
General Message Passing GNN layers → Gilmer et al.2017; Battaglia et al 2018; Wang et B 2018
Halicin → Stokes et al., Cell 2020
-----------------
Timeline:
0:00 - Graph Neural Networks and Halicin - graphs are everywhere
0:53 - Introduction example
1:43 - What is a graph?
2:34 - Why Graph Neural Networks?
3:44 - Convolutional Neural Network example
4:33 - Message passing
6:17 - Introducing node embeddings
7:20 - Learning and loss functions
8:04 - Link prediction example
9:08 - Other graph learning tasks
9:49 - Message passing details
12:10 - 3 'flavors' of GNN layers
12:57 - Notation and linear algebra
14:05 - Final words
------------------
Music by Vincent Rubinetti
Download the music on Bandcamp:
vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
Stream the music on Spotify:
open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u
------------------
Thanks for watching this, and I really hope it was helpful!
A quick introduction - I'm Alex from Singapore, a PhD student at NUS working on machine learning, computer vision and (I guess of course) GNNs for medical imaging and healthcare applications.
I've recently been thinking about doing explainer videos about machine learning or tech, and have always found great value in visual animations of math concepts.
So, thanks Grant Sanderson, James Schloss and the 3b1b team for organizing SoME1 which pushed me to pick up After Effects, research, script and put this together over the past month.
If you have questions or want to connect (please do!), you can:
Find me on Twitter → alexfoo_dw
Find me on LinkedIn → www.linkedin.com/in/alex-foo/