How I Understand Flow Matching

Flow matching is a new generative modeling method that combines the advantages of Continuous Normalising Flows (CNFs) and Diffusion Models (DMs).
In this tutorial, I share my understanding of the basics of flow matching and provide an overview of how these ideas evolve over time.
Check out the resources below to learn more about this topic.
===== Paper/blog survey =====
[Papamakarios et al. 2021] Normalizing flows for probabilistic modeling and inference arxiv.org/abs/...
[Kobyzev et al. 2020] Normalizing Flows: An Introduction and Review of Current Methods arxiv.org/abs/...
[Tor Fjelde et al. 2024] An Introduction to Flow Matching
mlg.eng.cam.ac...
[Jakub Tomczak] Flow Matching: Matching flows instead of scores
Blog: jmtomczak.gith...
Code example: github.com/jmt...
===== Research talks =====
[Yaron Lipman] Flow Matching: Simplifying and Generalizing Diffusion Models
• Flow Matching: Simplif...
[Michael S Albergo] Building Normalizing Flows with Stochastic Interpolants
• Building Normalizing F...
[Alex Tong] Conditional Flow Matching
• TransferLab Seminar: C...
Thumbnail background image credit: unsplash.com/p...

[@Neo-kx3fe]

@10:55 with that local outgoingness on the left, why there is one additional term p_t(x_t) inside the d/dx bracket? This term seems to disappear in @11:18. Thanks.

[@kvu207]

beautiful! May I ask how you made the animations for this video?

[@jbhuang0604]

Most of the animations are from the “morph transition” in PowerPoint slides. The rest are from Adobe premier pro.

[@manuellecha]

Thank you very much! You did a really nice job! The video is clear, visual, and informative. It is consistent with the timeline and evolution of the field, and it effectively conveys the information along with the motivation for the development of these models.

[@jbhuang0604]

Glad you liked it! It’s a lot of fun making this video!

[@pravin1390]

Thanks for the explanation. I feel it makes sense to use the notation p(x_t, t). Then, it is clear to prove ∂/∂t p(x_t, t) = -div(p(x_t, t) u(x_t, t)) is equivalent to
d/dt p(x_t, t) = -p(x_t, t) div(u(x_t,t)), which is indeed required here. Please let me know if my comments make sense. The first equation is partial derivative and second equation is total derivative is what I sense.

[@plcrodrigues]

This is gold. Congrats for doing such a fun, pedagogical, and informative on a topic that can often be quite dry in the literature.

[@jbhuang0604]

Thanks for your kind words!

[@AnujZore-pe9mg]

Thanks once again for easy-to-understand explanation! Gonna miss CSMC 733 lectures :(

[@jbhuang0604]

Glad you like them!

[@因幡の黒うさぎ-i1p]

Your lecture was truly inspiring and made complex concepts so easy to understand. Thank you for your incredible clarity and passion - I’m deeply grateful!

[@jbhuang0604]

You're very welcome! Glad that you like it!

[@ruoshiliu6024]

Amazing work Jia-Bin!!
P.S. you should create a bibtex for this video so it can be cited in literature :P

[@jbhuang0604]

Haha! Thanks! Too bad google scholar don’t include views of TH-cam videos.

[@jackshi7613]

excellent video!

[@jbhuang0604]

Thanks for watching!

[@JQXU-z3s]

Thank you for your excellent work! Absolutely clear and informative.

[@jbhuang0604]

Glad it was helpful!

[@r00t257]

Legend comeback 🙇! Your educational video is worth more than gold.💓🙏

[@jbhuang0604]

Thanks a lot! Glad you like it!

[@morrisfan2004]

I wondering if there are any urgent, potential applications of flow matching in industry?

[@jbhuang0604]

I think many of the recent text-to-image generation models are now trained with flow matching. There are also many other applications beyond image generations.

[@nathan_ca]

Thanks! This is amazing video to get students, like me, to re-engage with these topics that i haven't had a chance to explore more ❤

[@jbhuang0604]

Thanks! Glad that this is helpful.

[@catherineyang5199]

Thank you for the video! This is the most clear explanation of flow matching on the internet ❤

[@jbhuang0604]

Thank you so much for your kind words!

[@amirhosseinraffiee8270]

Thanks for the video. Great way to explain a complex concept

[@jbhuang0604]

Appreciate your comment! Thanks for watching the video. Hope you enjoyed it.

[@rexton123]

What are the advantage of flow matching compared to diffusion models?

[@jbhuang0604]

You can view it as a generalization of diffusion models. The training can converge faster and you won't have the difficulty where you cannot reach pure Gaussian distributions using finite diffusion steps.

[@adrienforbu5165]

nice visuals, good job

[@jbhuang0604]

Thanks a lot!

[@sayakpaul3152]

Do you have the next video already? It's so good!

[@jbhuang0604]

Thanks! I am working on it. :-)

[@yeon6761]

perfect video! Thank you for your works.

[@jbhuang0604]

You're very welcome!

[@thienthuoan1081]

感觉有点像李宏毅老师的风格哈哈哈

[@jbhuang0604]

李宏毅 is THE BEST!

[@julienblanchon6082]

This is brilliant !

[@jbhuang0604]

Glad that you enjoyed the video!

[@DimitrivonRutte]

Awesome to see easy-to-understand explanations of current research topics, keep up the great work!

[@jbhuang0604]

Glad you liked it!

[@rexton123]

Amazing

[@jbhuang0604]

Thanks!