What are Transformers (Machine Learning Model)?
ฝัง
- เผยแพร่เมื่อ 27 ก.ย. 2024
- Learn more about Transformers → ibm.biz/ML-Tran...
Learn more about AI → ibm.biz/more-ab...
Check out IBM Watson → ibm.biz/more-ab...
Transformers? In this case, we're talking about a machine learning model, and in this video Martin Keen explains what transformers are, what they're good for, and maybe ... what they're not so good at for.
Download a free AI ebook → ibm.biz/ai-eboo...
Read about the Journey to AI → ibm.biz/ai-jour...
Get started for free on IBM Cloud → ibm.biz/Bdf7QA
Subscribe to see more videos like this in the future → ibm.biz/subscri...
#AI #Software #ITModernization
In our graduation years we used to learn about something called codec, as in coder-decoder (something like modem for modulation-demodulation or balun for balanced-unbalanced in the domain of communication technology. So as I can understand from the video is that the transformers are just a fancy and advanced name for a codec, which functions at much bigger capitalistic scale.
Nailed it!
Love the roast :D
While I'm at a remedial level, I said "this is ADVANCED CODING"🤔☺
Banana joke GPT-4:
Sure, here's a banana joke for you:
Why did the banana go to the doctor?
Because it wasn't peeling very well!
Good transformer 🤣
I see some improvement
is the fact that he is able to write text mirrored incredible or is there a simple trick here?
There is a trick. Hint: he's not left handed.
its flipped and rotated, done through editing
@@IBMTechnology yeah, I though he can't be left handed.
@@vaibhavthalanki6317 it's not a glass, it's a mirror I think.
@@IBMTechnology thought he was left handed
Great video! I'm stumped on how you made this. Did you really write backwards? Can you reveal your magic trick?
You write it naturally and then flip the video when editing.
Exactly what's gng on in my mind lmao
I think he write on the glass normally and the camera got it backword so they montage it to be flipped so the written words can be shown notmally.
Perhaps the AI made the banana joke as a subtle way to tell us humans that we are a cruel species that mash anything we come across. The AI finds it funny because the banana would rather cross the road and take on the high likelihood of being mashed violently by a vehicle to avoid the certain mashing by humans. Perhaps the AI identified with the banana 🤔
Next level empathy: thinking about a banana's perception of reality 🧠
Q
You guys are overthinking it. 😁
that's how I interpreted it too - like yeah, the AI knows the banana doesn't want to be mashed by a car, neither do I
No, you’re definitely overthinking it. The AI doesn’t think anything because it is incapable of such context like “we are a cruel species that mash anything we come across”. Unless you specifically input that in the prompt, it has no mechanism to even conceive of the phrase.
claude sonnet 3.5:
Why don't bananas ever get lonely?
Because they hang around in bunches!
The Transformer technology is the reason why you see AI everywhere.
How does the transformer take something of variable length (like a sentence) and shove it into a neural network (which requires a fixed number of inputs)?
Generic NNs take only fixed inputs but this is one of the specialities of these types of models! RNNs (the older model used for NLP) were created back in the 80s addressing mainly this issue, along with memory being important for sequences. LSTMs n now transformers came in to solve the issues with RNNs
Does this mean they can fix my adhd?
I don't quite know why but all this transformer tech helps me understand my own glitched mind better
Dr. Ashish Vaswani is a pioneer and nobody is talking about him. He is a scientist from Google Brain and the first author of the paper that introduced TANSFORMERS, and that is the backbone of all other recent models.
Agreed
He should be documenting his work like our guy, and make interesting vids.
Hope it happens.
“Before too long, they might even be able to come up with jokes that are actually funny.”
2 years later, here’s the banana joke ChatGPT 4 (already 1y old) came up with for me.
> Why did the banana go to the doctor?> Because it wasn't peeling well!
I think we can call that a win.
Things are judged by their appearance. And this video looks way way better than it actually is. That explains the views.
like to see the energy which you put on to it, Thanks for this.
Lol, i find the banane joke funny :)
So is it like ... a layered, parallelized autoencoder?
why did the banana cross the road?
GPT 4o: To prove it wasn't a chicken! 🍌🐔
How are you able to write a mirror image of the words so effortlessly? :O
Why did the attention mechanism NOT cross the road? Because it was paralyzed!😜😁
BTW did I hear that part correctly near the end of the video?
How do you get loads of loss on on a neural network in given ways for analytics
I didnt get it
Thank you!
how do i use transformers on a new pair of language?
Transformers: More than meets the eye...
it doesn't "come up" with a thing, it regurgitates what it's learned. It's nothing but a copy machine and being made out to be much more than it really is by all the AI hype machine artists.
Very true, but this is exactly what we're trying to build as a first step toward AGI. If you look at it, aren't humans living the same life day after day? Aren't people simply trying to imitate what they see, hear, and feel?
Hi, what does an autoregressive language model mean?
IBM: Next-Level Tech explained.
Chat: How does he write backwards on that invisible board?
See ibm.biz/write-backwards
I do searches for Transferormer in Machine learning.and in my mind same those transformer there and video starts with the same.
Are encoders and decoders both RNN? Plz clear my doubt.
Is he writing on reverse so we can see it correctly?
Correct me if I'm wrong but it seems that translating a document would require a human doing Quality Control right before publishing. Transformers are impressive in how close they come to mimicking humans but they seem to be The Great Pretenders. Now, how does that QC step get implemented in real time?
Sir Will u give me a research topic in transformer
this dude can write reversely. so awsome
ha. it looks the right way around to him. The final image is inverted in the video we see. Fun trick.
the banana … skidded …
it wanted to split
Now it can indeed write funny banana jokes!!
Where can I summon autobot?
not very descriptive.. it is for those who already are studying deeply about sequencing, encoder decoder etc.
may be i am not smart enough to understand..
Are you guys open to Guest Speakers
Latest update on banana humor of AI
Why did the banana go to the doctor?
Because it wasn't peeling well! - GPT 3.5 11th January 2024, 23:06 IST
I don't like people ripping Me off, whether IBM or Google.
They've got better jokes now. 😂
can we use transformers over spacy for NER?
ok but how about a more detailed explanation ?
❤❤
I came here to understand how on earth he writes backwards or what camera trickery I am obviously missing, LOL.
See ibm.biz/write-backwards
@@IBMTechnology LOL thanks!!! I suppose it shouldn't surprise me there is a video about that. Very cool and elegant technique.
Instead of the content I started thinking how this guy writing in opposite direction 😭😂😂 Is this some AI trick or fr?!
See ibm.biz/write-backwards
Well, jokes are hard.
Kids take several years to learn how to be funny.
KIDS ARE OBSOLETE, AI IS BETTER
are you writing backwards in real time? because if so..... 🤯
See ibm.biz/write-backwards
@@IBMTechnology one of the few times in my life I wish to be lied to 😂
Why do they always translate English sentence to French in every video that explains Transformers :D
🤣😂
ask it why did the potato cross the road
this is the worst explanation video I've ever seen
You didn't really explain anything.
The explanation was unsatisfactory.
Boring 🤷🏽♂️
I can't concentrate I can't understand how he manages to write backwards
Hi Martin from the Homebrew Challenge! ML and beer clearly go hand in hand!
didn't find it helpful to conceptually understand transformers
I dont think the joke was that bad. Picture meatwad from AquaTeen Hunger Force, but very pale beige.
This was prophetic. I wonder whether at that time you realized that Transformer would revolutionize the world.
Hey its the guy from the beer channel...
Indian SME's might be able to create this and be a unicorn. Easily.
I really love your videos as they are really friendly to understand. Really graceful for the high quality of the synthesis of key messages on AI/ML/DL. I am a medical doctor and biomedical researcher. I can see the great potential of using the different technics to further develop a bunch of areas, for example: economic evaluations based on modeling (using a combination of approaches in the sensitivity analysis to find out the internal consistence of the predictions…to gain internal validity as a cornerstone to have external validity). So, looking forward to learn more through your channel.
Thank you, again for sharing good quality knowledge.
L.
Congratulations to all the team work!, I will keep learning more. Thank you all, Leslie.
the joke was just too deep for your puny mind to get
😂
how can text algorithm (transformer) work in image domain like vision transformer over CNN
Transformers are being used in many ways. For example you could take a bunch of vectors (representing image features extracted from Convolutions) and feed them into Transformers to decode as text. This gives you a lot of power combining the NLP and Computer Vision Domain
@@ChocolateMilkCultLeader
Generic features or specific?
@@strongsyedaa7378 what do you mean?
What a great way to introduce the topic. First 4 seconds made me laugh out loud. Well done (and the rest of the video as well)
Did I miss something? This didn't seem to give any clue as to how transformers actually work.
Aged like a wine!
wait, does this guy write backwards?
See ibm.biz/write-backwards
Thanks for this video - a simple and concise introduction to transformers.
Do large language models really possess reasoning capabilities? Or, the way they operate makes it seem so.
The joke would’ve worked if it was a potato. Pretty close though.
You do realize the joke about the chicken crossing the road is a suicide joke right? He wanted to get to the other side?
And with this simple idea the civilization ends. No, kidding, the AI will be so smart, it will leave us alone as we will be like bugs to it.
I just have to say it
TRANSFORMERS MORE THEN MEETS THE EYES!
Actually, I'm interested in the hand writing presentation style. How is it made ?
Thanks you for your enthusiasm and explanation , by far the best
The banana joke is an instance of an “anti-joke”… just like the chicken joke.
Need Explanation for GRU , BERT , LSTM
You are mirror writing, Great skill🤩
so starting about 4:10 when he explains the difference between classical agrothims verses a Generalzed pre trained Transformor model using an attention mechinism- Coould this be described as a typical PC processor compared to a quantium computer, I understand super positioninig on the quantium side and both are a set of one verse many calculations ? Its immateing thinking in the Ai model where the quantium PC is, well, I dont think we know except it goes and comesback?
No, they are not analogous.
Thank your video,your video really easy understand
I have been more then blown away by the unfathomable exponential growth from just increasing transformers an a few weights lol
Your skills in writing backwards were really distracting.
See ibm.biz/write-backwards for how it's done
This is the pinnacle performance of training.
This was greate!!!
But how does the human write backwards?
See ibm.biz/write-backwards
Can we use this method to detect outliers in time series data
While you can use Transformers for Time Series, I'm not sure why you'd want some network architecture to look for outliers instead of regularizing it and let the network learn to ignore those during optimization.
Transformers are a bit overkill for anomaly detection. A lot of time more traditional methods might perform better faster (especially if the resources for training the models are constrained like not having dedicated chips or an insufficient amount of training data)
In reference to the summary of an article example, How does that work? How does the program know to summarize the article and not continue it?
Also, how do you go from language processing to playing chess or other games or functions?
I'm not a machine learning expert so I can't verify the validity of this answer, but from my POV I think these questions about "how the program... instead of..." is generally dependent on
1. The actual architecture of the model (in this case, a transformer)
2. The input data it's based upon (text vs maybe piece type and board position labels for a chessboard)
3. The output data it's trying to predict (predict a summary text vs predict the next words in an article)
Because such supervised/semi-supervised learning models learn off labelled data, (to a certain extent, for semi-supervised learning), all the model is really doing is mapping an input to an output. Think of it like a maths graph (which is actually exactly what it is); given a dataset with many points, you'd want to find a "best fit" line that models the rough trend accurately without over or underfitting. Machine models do this but on many axes (due to the use of vectors, some with just an insane number of dimensions).
Of course there are many other things like hyperparameters, activation functions, loss functions, and nuanced variables to each model architectures, but hopefully this gives you a good understanding of ML in general.
A summary is a continuation of the text in that case. Consider a webpage on the internet which has an article and then at the bottom of the page it says, "here is a summary of the key points we learned above" and it goes on to summarise. This is an example of the kind of content the AI is trained on. So as long as you do some Prompt Engineering then you can ask your question in such a way that the answer comes from completing the text! It's like magic! 🙂
@@xerxel69 Yeah, articles do often contain a summary section at the end. Or parts of an essay say, "To summarise so far". Not sure if it can learn this totally unsupervised. Mu guess is summaries are a popular feature - so they will train it specifically to look for them and learn from them i na focused way. Not sure though.
Do I get this right, that a transformer is a special case of a state machine, which is designed to learn on, or update it's weights on demand, and is still general enough to cover most data?. Wouldn't an FPGA be optimal to implement such a state machine in flip flop, so that you can generate with 100mhz.
It really all boils down to performing matrix multiplications. GPUs are best at that. An FPGA can be a GPU if it wants to (:
To reach the other bunch. chat gpt3.5