Vision Transformer and its Applications

Vision transformer is a recent breakthrough in the area of computer vision. While transformer-based models have dominated the field of natural language processing since 2017, CNN-based models are still demonstrating state-of-the-art performances in vision problems. Last year, a group of researchers from Google figured out how to make a transformer work on recognition. They called it "vision transformer". The follow-up works by the community demonstrated superior performance of vision transformers not only in recognition but also in other downstream tasks such as detection, segmentation, multi-modal learning and scene text recognition to mention a few.
In this talk, Rowel Atienza will go into a deeper understanding of the model architecture of vision transformers. Most importantly, Rowel will focus on the concept of self-attention and its role in vision. Then, he will present different model implementations utilizing the vision transformer as the main backbone.
Since self-attention can be applied beyond transformers, Rowel Atienza will also discuss a promising direction in building general-purpose model architectures. In particular, networks that can process a variety of data formats such as text, audio, image and video.
→ To watch more videos like this, visit aiplus.training ​←
Do You Like This Video? Share Your Thoughts in Comments Below
Also, You can visit our website and choose the nearest ODSC Event to attend and experience all our Trainings and Workshops:
odsc.com/calif...
odsc.com/apac/
Sign up for the newsletter to stay up to date with the latest trends in data science: opendatascienc...
Follow Us Online!
• Facebook: / opendatasci
• Instagram: / odsc
• Blog: opendatascienc...
• LinkedIn: / open-data-science
• Twitter: / odsc

[@jhjbm1959]

This video provides a clear step by step explanation how to get from images to input features for Transformer encoders, which has proven hard to find anywhere else.
Thank you.

[@SarangBanakhede]

10:58
Scale Equivariance:
Definition: A function is scale equivariant if a scaling (resizing) of the input results in a corresponding scaling of the output.
Convolution in CNNs: Standard convolutions are not scale equivariant. This means that if you resize an object in an image (e.g., making it larger or smaller), the CNN may not recognize it as the same object. Convolutional filters have fixed sizes, so they may fail to detect features that are significantly larger or smaller than the size of the filter.
Example: If a CNN is trained to detect a small object using a specific filter size, it might struggle to detect the same object when it appears much larger in the image because the filter is not capable of adjusting to different scales.
Why is Convolution Not Scale Equivariant?
The filters in a CNN have a fixed receptive field, meaning they look for patterns of a specific size. If the size of the pattern changes (e.g., due to scaling), the fixed-size filters may no longer detect the pattern effectively.

[@crapadopalese]

10:46 - this is a mistake; the convolution is not equivariant to scaling - if the bird is scaled, the output of the convolution will not be simply a scaling of the original output. That would only be true if you also rescale the filters.

[@PrestonRahim]

Super helpful. Was very lost on the process from image patch to embedded vector until I watched this.

[@DrAIScience]

Very very very nice explanation!!! I like learning the foundation/origin of the concepts where models are derived..

[@ailinhasanpour]

thanks for sharing , it was extremely helpful 💯

[@OpenDataScienceCon]

Thank you!

[@xXMaDGaMeR]

amazing lecture, thank you sir!

[@sahil-vz8or]

you said 196 patches in imagenet data. No of matches will depend on the input image size and the patch size. For eg: if the input image is of 400X400 and patch size of 8X8, then no of patches will be (400X400/8X8) = 50X50 =2500.

[@rikki146]

20:17 I think the encoder blocks are stacked in parallel fashion rather than sequential?

[@DrAIScience]

Do you have a video about beit or dino?

[@scottkorman4953]

What exactly is happening in the self-attention and MLP blocks of the encoder module? Could you describe it in a simplistic way?

[@PRASHANTKUMAR-ze6mj]

thanks for sharing

[@mohammedrakib3736]

Fantastic Video! Really loved the detailed explanation step-by-step.

[@DrAIScience]

Are you the channel owner??

[@user-co6pu8zv3v]

Thank you, sir

[@anirudhgangadhar6158]

Great resource!

[@muhammadshahzaibiqbal7658]

Thanks for sharing.

[@liangcheng9856]

awesome

[@hoangtrung.aiengineer]

Thank you for making such a great video

[@capocianni1043]

Thank you for this genuine knowledge.

[@saimasideeq7254]

thankyou much clearer

[@improvement_developer8995]

Tax evader 🤮

[@improvement_developer8995]

🤮