LLM Foundations (LLM Bootcamp)
ฝัง
- เผยแพร่เมื่อ 24 ก.ค. 2024
- New course announcement ✨
We're teaching an in-person LLM bootcamp in the SF Bay Area on November 14, 2023. Come join us if you want to see the most up-to-date materials building LLM-powered products and learn in a hands-on environment.
www.scale.bythebay.io/llm-wor...
Hope to see some of you there!
--------------------------------------------------------------------------------------------- In this video, Sergey covers the foundational ideas for large language models: core ML, the Transformer architecture, notable LLMs, and pretraining dataset composition.
Download slides from the bootcamp website here: fullstackdeeplearning.com/llm...
Intro and outro music made with Riffusion: github.com/riffusion/riffusion
Watch the rest of the LLM Bootcamp videos here: • LLM Bootcamp - Spring ...
00:00 Intro
00:47 Foundations of Machine Learning
12:11 The Transformer Architecture
12:57 Transformer Decoder Overview
14:27 Inputs
15:29 Input Embedding
16:51 Masked Multi-Head Attention
24:26 Positional Encoding
25:32 Skip Connections and Layer Norm
27:05 Feed-forward Layer
27:43 Transformer hyperparameters and Why they work so well
31:06 Notable LLM: BERT
32:28 Notable LLM: T5
34:29 Notable LLM: GPT
38:18 Notable LLM: Chinchilla and Scaling Laws
40:23 Notable LLM: LLaMA
41:18 Why include code in LLM training data?
42:07 Instruction Tuning
46:34 Notable LLM: RETRO - วิทยาศาสตร์และเทคโนโลยี
Great presentation
So far this has been the simplest brief explanation of transformers I have seen. Not complete though but goos starting point.
As a data scientist prepping to retrain a transformer for a classification task at work, this is so helpful in refreshing what I learned in school!
Great presentation! Thanks!
Amazing explanation of Transformers! And everything else... LOVED the talk! Thank you so much, Sergey!
This is the best explanation so far for Language Model and its parameters. The speaker though didn't attributed Google for their Attention is all you need white paper.
People from Google not Google
He attributed the researchers. That's all that matters.
really cool content. the part about training the models on code enhancing non-code capabilities is mind blowing. congrats!
Good stuff, thank you!
Nice attempt! 😊
This was amazing! As a "neophyte" to the field of AI and LLMs, highly recommend this for foundational knowledge and perspective.
amazing content!
excellent
Sound. Very small
Subtitle. Will help
Whats the presentation date please ?
April 21, 2023
Pls use AI to enhance this audio tnx. Either way, thanks for the vid!