Mapping to geometry is pro. I have thought since my education about 40 years ago that current mathematics is taught incorrectly. Here is a pro example of how math should be taught!
A cristal clear explanation of Transformers. Papers in many cases are very difficult to follow. Pointing out the important omited details which are critical for the model, even if not explained, is very useful. Many out there try to explain transformers without having a clue of what it is. Clearly, this is not the case. Thanks in its deepest tokenized meaning for sharing your knowledge. BTW, the last programming tip is really helpful. A small hands on demo of using BERT(or any flavor of BERT) with a classifier for a particular application would be amazing for another video.
Great Video. The first Transformer explanation that (correctly) does not use the Encoder/Decoder diagram from the Transformer paper, well done! Additionally talking about the exact outputs (using only one output for predictions) was very helpful.
After a year returning to that video finally I fully (or at least saw the entire video in a row) understand what is going on!! Maybe one more time to fix and go to the next part!! Thanxx
Great overview and explanation of the Transformer network. I am just starting my exploration into NLP and this talk has saved me lots of time. I now know that this where I need to be focussing my attention. Thank you 👍🙏😍
Hands down the best explanation, this after watching so many videos, terrific, Looking forward to some videos on understanding on BARD and its fine tuning
She's so good! I've watched a few videos attempting to explain these self-attention version of transformers and this one is by far the best in so many aspects with actual deep understanding of the architecture at the top followed closely by coherently communicating concepts, good script, presentation and graphics! I hope she narrates more videos like this... I'm about to search and find out lol! 🧐🤞 🤓
I expected some wishy-washy feel-good "explanation", but I'm pleasantly surprised. So far the best explanation. Goes after the relevant distinguishing key features of the transformers without getting bogged down in unnecessary details.
Julia, Your presentation has triggered a Eureka moment in me . What makes a great training video? Can AI help answer that. Here is a suggestion. Get a collection of videos and rank them by review comments. Using a large language model, find patterns and features and see whether there are correlations between the features and the views and review rankings. The model should be unsupervised. Some of the features can be extracted from comments
This is a fairly good presentation. There are some areas where it summarizes to the point where it becomes almost misleading, and at least very questionable: 1. Several other sources that I read claim that the Bert layers will have to be frozen during fine tuning, so I think it is still open for debate what the right thing to do is there? 2. This presentation glosses over the outputs of the pretraining phase. I think the output corresponding to the CLS token is pretrained with the “next sentence prediction task”. So, is this output layer dropped entirely in the fine tuning task? Otherwise I don’t see how the CLS token output would be a good input for sentiment classification. 3. The presentation suggest that the initial non contextual token step is also trainable and fine tunable. Isn’t it just fixed byte pair encodings? I know that these depend on frequencies of letters in the language but can these be trained in process with Bert? 4. This presentation equals transformers very silently to transformer encoders, and thus drops the fact that transformers can also be decoders. I think all initial transformers were trained on sequence to sequence transformation, and then the decoders were trained on next token prediction giving rise to things like GPT, whereas the encoders were trained on a combination of masked token prediction and next sentence prediction giving rise to the BERT like models.
at minute 35 the video describes transfer learning, and it is said that during the fine tuning phase ALL the parameters are adjusted, not only the classifier parameters. Is that right? In contrast, when using a pre-trained deep network for a specific image calssification, I froze all parameters belonging to the CNN and just allowed the classifier parameters to vary
okay what you are saying is completely vague . like for the query matrice you mentioned ( some other representation [why do we need another representation at all ?])
Sorry to say, but this was not very good. Key information is missing mostly the WHYs ? why is there a need for Query and Key Matrices? what is the main function of these matrices? How does the Attention function alter the Feedforward NNs?
I always find the face of presenter distracting when it is on the slides … can you just talk over slides instead of covering them with presenter’s face??
This was one of the best videos I’ve seen explaining transformers and NL models….well done and look forward to the other videos in the series!🍻
This is the best video on youtube that introduces transformer models
The comparison between attentions heads and CNN filters made so much sense!
I believe this video provides the most comprehensive explanation of transformers.
After reading about language models, word embeddings, transformers, etc. for a month, this video put everything in order for me. Thanks!
I didn't understand a single word 😕
Please @Tensorflow Team continue with this lecture series ML Tech series
best youtube NLP walk through without cutting corners. best delivery as well.
One of the best explanation i have come across on transformers. Thanks
Mapping to geometry is pro. I have thought since my education about 40 years ago that current mathematics is taught incorrectly. Here is a pro example of how math should be taught!
A cristal clear explanation of Transformers. Papers in many cases are very difficult to follow. Pointing out the important omited details which are critical for the model, even if not explained, is very useful. Many out there try to explain transformers without having a clue of what it is. Clearly, this is not the case. Thanks in its deepest tokenized meaning for sharing your knowledge. BTW, the last programming tip is really helpful. A small hands on demo of using BERT(or any flavor of BERT) with a classifier for a particular application would be amazing for another video.
Great Video. The first Transformer explanation that (correctly) does not use the Encoder/Decoder diagram from the Transformer paper, well done!
Additionally talking about the exact outputs (using only one output for predictions) was very helpful.
So clear. This is one of the best videos explaining transformer architecture.
One of the best explanations of transformers that I've seen!
Best Video on Transfer Learning. So much clarity
After a year returning to that video finally I fully (or at least saw the entire video in a row) understand what is going on!! Maybe one more time to fix and go to the next part!! Thanxx
Great overview and explanation of the Transformer network. I am just starting my exploration into NLP and this talk has saved me lots of time. I now know that this where I need to be focussing my attention. Thank you 👍🙏😍
Thank you for the awesome talk on all the main NLP models, in particular, the great explanation of the Transformer model!
Hands down the best explanation, this after watching so many videos, terrific, Looking forward to some videos on understanding on BARD and its fine tuning
She has done an incredible job.
She's so good! I've watched a few videos attempting to explain these self-attention version of transformers and this one is by far the best in so many aspects with actual deep understanding of the architecture at the top followed closely by coherently communicating concepts, good script, presentation and graphics! I hope she narrates more videos like this... I'm about to search and find out lol! 🧐🤞 🤓
Wow that explanation actually dissipated many of my questions. Thanks a lot Julia!
awesome; the very BEST explanation on self-attention and trasformers
My favorite explanation so far. Great job.
Very well explained! Thank you very much. Especially loved the comparison between CV kernels and multiple QKV parameters.
Agreed with all. This person should take the lead for other Google educational videos.
This is very beautifully explained!
Best explanations so far of the attention or QKV concept... I was searching for a good way to visualize it.. Thanks a ton!!
Thank you very much ! Great video and very well explained. Yes a video about sentiment analysisfine tuning would be Amazing !
Great presentation! Really easy to understand exaplanations of some hard topics, thank you.
I expected some wishy-washy feel-good "explanation", but I'm pleasantly surprised. So far the best explanation. Goes after the relevant distinguishing key features of the transformers without getting bogged down in unnecessary details.
44:40: Thanks for your attention 😁
Caught that too
Very straightforward.
Thank you so much
Great presentation. Really well structured.
That was super interesting. Very clear explanation
Great explanations, thank you so much for this video!
Glad it was helpful!
The explanation is so clear, thank you.
Excellent presentation of complex NLP topic.
super professional explanation of the topic! Excellent work!
It's very important library in nlp great work
Julia,
Your presentation has triggered a Eureka moment in me . What makes a great training video? Can AI help answer that. Here is a suggestion. Get a collection of videos and rank them by review comments. Using a large language model, find patterns and features and see whether there are correlations between the features and the views and review rankings. The model should be unsupervised. Some of the features can be extracted from comments
Very clear explanation. Thank youuu
Very nice topic discussion! Thank you 🙂
This is a positive comment. TH-cam should let it past it’s sentiment filter.
Great Presentation
Great Explanation
very good presentation!
I came here from tutorials sections of tensorflow official webpage, but i get caught by her beauty
Great video, it would be nice to have a video of reinforcement learning in future ml tech talks.
well done video!
Simply awesome
Crystal clear .. Tnx
Great talk, really clear, thanks!
Also I see what you did "Thanks for your attention" 🤣
Excellent teaching !
Thanks a ton for the explantion! Just wanted to ask how do we arrive at the values for matrices K, V and Q?
Awesome.. great explanation. Thanks.
This is a fairly good presentation. There are some areas where it summarizes to the point where it becomes almost misleading, and at least very questionable: 1. Several other sources that I read claim that the Bert layers will have to be frozen during fine tuning, so I think it is still open for debate what the right thing to do is there? 2. This presentation glosses over the outputs of the pretraining phase. I think the output corresponding to the CLS token is pretrained with the “next sentence prediction task”. So, is this output layer dropped entirely in the fine tuning task? Otherwise I don’t see how the CLS token output would be a good input for sentiment classification. 3. The presentation suggest that the initial non contextual token step is also trainable and fine tunable. Isn’t it just fixed byte pair encodings? I know that these depend on frequencies of letters in the language but can these be trained in process with Bert? 4. This presentation equals transformers very silently to transformer encoders, and thus drops the fact that transformers can also be decoders. I think all initial transformers were trained on sequence to sequence transformation, and then the decoders were trained on next token prediction giving rise to things like GPT, whereas the encoders were trained on a combination of masked token prediction and next sentence prediction giving rise to the BERT like models.
Amazing talk! very informative. Thank you :)
Thank you
amazing video !
Nice, thank you ❤
at minute 35 the video describes transfer learning, and it is said that during the fine tuning phase ALL the parameters are adjusted, not only the classifier parameters. Is that right? In contrast, when using a pre-trained deep network for a specific image calssification, I froze all parameters belonging to the CNN and just allowed the classifier parameters to vary
very nice
Simply great! 👏👏👏
Where can I get the slides for this talk? Great talk
Great slides.
real help, thanks.
Thank you for shraing !
تحياتي الخالصة شكرا جزيلا
Great presentation! Are the slides available for download? This would be fantastic. Thank you.
nice video
Immaculate!
i love it
Thanks for your ATTENTION 🤗🤗.. Pun intended!44:39
Interesting
Where can I get the GitHub file
Can someone explain the inputs dict shown in the code at 42:15.
Please provide the slides
Thanks for not calling Sentiment Classification as Sentiment Analysis!
okay what you are saying is completely vague . like for the query matrice you mentioned ( some other representation [why do we need another representation at all ?])
Sorry to say, but this was not very good. Key information is missing mostly the WHYs ? why is there a need for Query and Key Matrices? what is the main function of these matrices? How does the Attention function alter the Feedforward NNs?
👍🏻👍🏻👌
I always find the face of presenter distracting when it is on the slides … can you just talk over slides instead of covering them with presenter’s face??