yes, good question, there is nothing like a peak. To be honest, the only peak is the human imagination to create new incredibles models and maths. Remember before transformers, there was a peak, and before diffusion models the peak was Gans. Now yes we are stucked in transformers and diffusion and everybody adopt them, so we have to wait for someone working on others concepts.
I may be missing something but in the second quiz why would full fine tuning increase the number of trainable model parameters by 100%? Wouldn't it just act further on 100% of the original trainable model parameters?
Yea maybe I could have phrased better. You are still training the same parameters. But the amount of parameters to store would be 100% for each task. So for a model that has 1000 parameters that we fine tune on 15 tasks, we would need to store 15 * 1000 = 15k parameters. I was trying to go for “number of trainable model parameters to be stored per task”
Your question whether AI has peaked is not well formed. What AI are we talking about? If it's general AI, then no, clearly AI models today don't get close to do everything humans are capable of. If it's ML and language models, well there seems to be some kind of plateau and there are no clear advantages of one solution over the next one. Maybe the salvation will come from a new learning algorithm but they clearly need a change of paradygm.
yes, good question, there is nothing like a peak. To be honest, the only peak is the human imagination to create new incredibles models and maths. Remember before transformers, there was a peak, and before diffusion models the peak was Gans. Now yes we are stucked in transformers and diffusion and everybody adopt them, so we have to wait for someone working on others concepts.
what are the chances that I was searching for exact same content, and got notification at the same time
One of the best PEFT explanations till date. Should deserve more subscribers
Thank you for watching and for the compliments!
Loved it
I may be missing something but in the second quiz why would full fine tuning increase the number of trainable model parameters by 100%? Wouldn't it just act further on 100% of the original trainable model parameters?
Yea maybe I could have phrased better. You are still training the same parameters. But the amount of parameters to store would be 100% for each task. So for a model that has 1000 parameters that we fine tune on 15 tasks, we would need to store 15 * 1000 = 15k parameters. I was trying to go for “number of trainable model parameters to be stored per task”
@@CodeEmporium OK that makes sense. Thanks for all your excellent videos on the subject.
We haven't peaked. Now, technology or hardware will have to get stronger, better, faster.
Your question whether AI has peaked is not well formed. What AI are we talking about?
If it's general AI, then no, clearly AI models today don't get close to do everything humans are capable of.
If it's ML and language models, well there seems to be some kind of plateau and there are no clear advantages of one solution over the next one. Maybe the salvation will come from a new learning algorithm but they clearly need a change of paradygm.