Transformers for beginners | What are they and how do they work
ฝัง
- เผยแพร่เมื่อ 29 มิ.ย. 2023
- Over the past five years, Transformers, a neural network architecture, have completely transformed state-of-the-art natural language processing.
*************************************************************************
For queries: You can comment in comment section or you can mail me at aarohisingla1987@gmail.com
*************************************************************************
The encoder takes the input sentence and converts it into a series of numbers called vectors, which represent the meaning of the words. These vectors are then passed to the decoder, which generates the translated sentence.
Now, the magic of the transformer network lies in how it handles attention. Instead of looking at each word one by one, it considers the entire sentence at once. It calculates a similarity score between each word in the input sentence and every other word, giving higher scores to the words that are more important for translation.
To do this, the transformer network uses a mechanism called self-attention. Self-attention allows the model to weigh the importance of each word in the sentence based on its relevance to other words. By doing this, the model can focus more on the important parts of the sentence and less on the irrelevant ones.
In addition to self-attention, transformer networks also use something called positional encoding. Since the model treats words as individual entities, it doesn't have any inherent understanding of word order. Positional encoding helps the model to understand the sequence of words in a sentence by adding information about their position.
Once the encoder has calculated the attention scores and combined them with positional encoding, the resulting vectors are passed to the decoder. The decoder uses a similar attention mechanism to generate the translated sentence, one word at a time.
Transformers are the model behind GPT, BERT, and T5
#transformers #naturallanguageprocessing #nlp - วิทยาศาสตร์และเทคโนโลยี
![[TH] VCT Masters Shanghai Playoffs DAY 2 // PRX vs G2 | EDG vs TH](http://i.ytimg.com/vi/T5j2XVcDFs0/mqdefault.jpg)
This is the only video around that REALLY EXPLAINS the transformer! I immensely appreciate your step by step approach and the use of the example. Thank you so much 🙏🙏🙏
Glad it was helpful!
exactly
Its great. I have only one query as whats the input of the masked multi-head attention as its not clear to me kindly guide me about it?
Hello and Thank you so much. 1 question: I don't realize where the numbers in word embedding and positional encoding come from?
Very nice high level description of Transformer
Glad you think so!
Great Content
Thanks!
Great Explanation, Thanks
Glad it was helpful!
I had watched 3 or 4 videos about transformers before this tutorial. Finally, this tutorial made me understand the concept of transformers. Thanks for your complete and clear explanations and your illustrative example. Specially, your description about query, key and value was really helpful.
You're very welcome!
Very well explained ! I can instantly grab the concept ! Thank you Miss !
Glad it was helpful!
Accidentally I came across this video, very well explained. You are doing an excellent job .
Glad it was helpful!
Well explained. before watching this video i was very confused in understanding how transformers works but your video helped me alot
Glad my video is helpful!
Great explanation! Keep uploading such nice informative content.
Thank you, I will
Just amazing explanation 👌
Thanks a lot 😊
Wow.. you are amazing. Thank you for the clear explanation
You're very welcome!
Very well explained
Thanks for liking
Great Explanation mam
Glad you liked it
excellent explanation
Glad you liked it!
This is a fantastic, Very Good explanation.
Thank you so much for good explanation
Glad it was helpful!
thank you very much for explaining and breaking it down 😀 comparatively so far, your explanation is easy to understand compared to other channels thank you very much for making this video and sharing to everyone❤
Glad it was helpful!
Very well explained, even with such niche viewer base, keep making more of these please
Thank you, I will
excellent explanation madam... thank you so much
Thanks and welcome
Best video ever explaining the concepts in really lucid way maam,thanks a lot,pls keep posting,i subscribed 😊🎉
Thanks and welcome
best explanation i saw multiple video but this provide the clear concept keep it up
Glad to hear that
Nice explanation Ma'am.
Thank you! 🙂
Can you please let us know I/p for mask multi head attention. You just said decoder. Can you please explain. Thanks
can you please upload the presentation
Thanks for making such an informative video. Please could you make a video on the transformer for image classification or image segmentation applications.
Will cover that soon
Thanks Aaroh i😇
Glad it helped!
Ma'am, we are eagerly hoping for a comprehensive Machine Learning and Computer Vision playlist. Your teaching style is unmatched, and I truly wish your channel reaches 100 million subscribers! 🌟
Thank you so much for your incredibly kind words and support!🙂 Creating a comprehensive Machine Learning and Computer Vision playlist is an excellent idea, and I'll definitely consider it for future content.
Hello Ma’am
Your AI and Data Science content is consistently impressive! Thanks for making complex concepts so accessible. Keep up the great work! 🚀 #ArtificialIntelligence #DataScience #ImpressiveContent 👏👍
Thank you!
Thanks. Concept explained very well. Could you please add one custom example (e.g finding similarity questions)using Transformers?
Will try
your video is good, explanation is excellent , only negative I felt was the bg noise. pls use a better mic with noise cancellation. thankyou once again for this video
Noted! I will take care of the noise :)
Thank you so much
Welcome!
Really very nice explanation ma'am!
Glad my video is helpful!
Very Good Video Ma'am, Love from Gujarat, Keep it up
Thanks a lot
The best explanation of transformer that I have got on the internet , can you please make a detailed long video on transformers with theory , mathematics and more examples. I am not clear about linear and softmax layer and what is done after that , how training happens and how transformers work on the test data , can you please make a detailed video on this?
I will try to make it after finishing the pipelined work.
@@CodeWithAarohi Thanks will wait for the detailed transformer video :)
Question about query, key, value dimensionality
Given that
query is a word that is looking for other words to pay attention to
key is a word that is being looked at by other words
shouldn't query and word be a vector of size the same as number of input tokens? so that when there is a dot product between query and key the word that is querying can be correctly (positionally) dot product'd with key and get the self attention value for the word?
The dimensionality of query, key, and value vectors in transformers is a hyperparameter, not directly tied to the number of input tokens. The dot product operation between query and key vectors allows the model to capture relationships and dependencies between tokens, while positional information is often handled separately through positional embeddings.
Thank you. The concept has been explained very well. Could you please also explain how these query, key and value vectors are calculated?
Sure, Will cover that in a separate video.
Can you also talk about the purpose of the 'feed forward' layer. looks like its only there to add non-linearity. is that right?
Yes you can say that..but mayb also for make key, quarry and value trainable
thank you mam
Most welcome 😊
Can you please make a detailed video explaining the Attention is all you need research paper line by line, thanks in advance :)
Noted!
Didn't understand what is the input to the masked multi head self attention layer in the decoder, Can you please explain me?
In the Transformer decoder, the masked multi-head self-attention layer takes three inputs: Queries(Q), Keys(K) and Values(V)
Queries (Q): These are vectors representing the current positions in the sequence. They are used to determine how much attention each position should give to other positions.
Keys (K): These are vectors representing all positions in the sequence. They are used to calculate the attention scores between the current position (represented by the query) and all other positions.
Values (V): These are vectors containing information from all positions in the sequence. The values are combined based on the attention scores to produce the output for the current position.
The masking in the self-attention mechanism ensures that during training, a position cannot attend to future positions, preventing information leakage from the future.
In short, the masked multi-head self-attention layer helps the decoder focus on relevant parts of the input sequence while generating the output sequence, and the masking ensures it doesn't cheat by looking at future information during training.
maam can you please make one video of classification using multi-head attention with custom dataset
Will try
can you please explain 22:07 onward
hi, Good explanation but at the end, when you explained what would be the input to the decoder's masked multi-head attention, you fumbled and didn't explain clearly. But the rest of the video was very good.
Thank you for the feedback!
Could you make a video on image classification for vision transformer, madam ?
Sure, soon
Great Video ma'am could you please clarify what you said at 22:20 once again... I think there was a bit confusion there.
same here
hello maa is this transform concept same for transformers in NLP?
The concept of "transform" in computer vision and "transformers" in natural language processing (NLP) are related but not quite the same.
Can you please make a video on bert?
I will try!
how to get pdfs mam
Could you explain with python code which would be more practical. Thanks for sharing your knowledge
Sure, will cover that soon.
I thought it's transformers in CV. all explanations were in NLP
I recommend you to understand this video first and then check this video: th-cam.com/video/tkZMj1VKD9s/w-d-xo.html After watching these 2 videos, you will understand properly the concept of transformers used in computer vision. Transformers in CV are based on the idea of transformers in NLP. SO its better for understanding if you learn the way I told you.
Gonna tell my kids this was optimus prime.
Haha, I love it! Optimus Prime has some serious competition now :)
Use mic, background noise irritate
Noted! Thanks for the feedback.
Speaking in Hindi would be more better
Sorry for inconvenience