Sure, implementing research papers from scratch is great and all, but why stop there? Using PyTorch? Nah, you’re still relying on a framework. To really understand it, you should rewrite PyTorch from scratch. But wait, PyTorch is written in Python-so, obviously, you should just rewrite Python, right? Actually, why trust Python at all? Better dive deep and rewrite C while you're at it. But hold on… C is just an abstraction too. Time to rebuild assembly from scratch! 😂
I think writing the model using C++ would be a great advanced exercise, especially since some models, like 3D reconstruction, are implemented in C++. Just to clarify, I haven't implemented a single paper before, but I think it would be a good idea and if i had time i will try doing it.😂
00:05 Focus on these two AI/ML projects for long term success 00:27 Implementing new research papers monthly boosts your AI/ML projects. 00:49 Coding AI/ML projects from scratch is essential for ML enthusiasts 01:10 PyTorch is beneficial for learning AI/ML concepts 01:32 Using PyTorch to master machine learning concepts in just five lines of code 01:57 AI/ML projects provide unfair advantage 02:19 Fine-tuning AI models for better performance. 02:44 Implement models from scratch to excel as a scientist or engineer.
To get started: run repos available with papers (easy to find on paperswithcode) in the cloud or locally. Try them on different datasets and try tweaking them. Understand the main parts of the code. Then replicate these papers before moving onto similar ones.
Aside from implementing the models from scratch, learning the math behind the Machine Learning algorithms will also definitely make you a better scientists and engineers. That's what I'm planning to do.
32 k people watched it. Half of them will forget about the video after 2 minutes. Other half will think about it but will move on without doing anything. Very few will actually follow what the video said. The competition is actually really low if you are ready to put in serious effort
heavily depends on your own experience and what you are starting from, if you dont know any programming its going to take a lot longer than someone who knows programming and basic machine learning concepts
Yup, definitely depends on experience and the complexity of the paper. If I focus 100%, I am usually able to implement a paper within a day, but it definitely took me a lot longer when I was starting out
Maybe in 2020 this gave you an unfair advantage, in 2024 without a Masters + software engineer experience or PhD forget getting into the field if you’re in the USA, but cases do exist where you could get lucky or get a company that doesn’t know what AI actually is, giving you a lower barrier of entry
@@BusinessDuck69 Thanks for sharing your perspective! From what I've seen, it seems very dependent on the company, i.e. big tech and F500 companies have different recruiting filters.
I first recommend learning concepts like Gradient Descent, and becoming familiar with PyTorch before implementing papers! Gradient Descent:th-cam.com/video/bbYdqd6wemI/w-d-xo.html PyTorch Intro: th-cam.com/video/SuqPZfwbAvA/w-d-xo.html If you're already familiar with those concepts, check out this video on my Top 5 AI/ML Papers: th-cam.com/video/zmmWjEDZn6g/w-d-xo.html
hey! can you give me more insights about your work with finetuning open source models with maths related dataset. I am working on something very very similar and I think it would help me alot! Thank you
Can someone please explain why people prefer PyTorch over tensor flow nowadays. I am learning tensorflow for a while now creating NN models . Now there is a trend of PyTorch everywhere.
Okay totally noob question: Implementing a paper? What does that even mean? Do what the researchers did? I maybe am kinda doing option 2 right now, but never even heard of Option 1
I usually recommend that beginners learn PyTorch over TensorFlow if they don't know either, since there are more PyTorch implementations these days. But if you already know TensorFlow, that's great man! You can definitely implement papers in TensorFlow.
That will depend on how much of ML you already know. If you have complete understanding of how a traditional neural net works and it's mechanisms feel completely natural/intuitive to you, then you should read on RNNs and try to understand a paper like *Sequence to Sequence Learning with RNNs by Surskever et. al.* where they described a multi layered LSTM encoder-decoder model. That was a novel idea at the time and has lots of material out there going over it including colab notebooks with in depth explanations. After that try finding out the flaws or things that could've been explored in more detail than what you saw on the paper. Tweak the model, try adding dropout, teacher enforcing etc. Experiment. After that search for other areas that might interest you or go deeper within NMT with something like *Neural Machine Translation by Jointly Learning to Translate and Align* by K. Cho (It's the seminal paper for the attention mechanism) as it's a natural follow up to the previous one.
@@Koan__ Seq2Seq Learning With RNNs is a classic! Thanks for sharing. This is wonderful advice. I also really agree on the benefit of implementing NMT papers even for those not as interested in NMT as they are in other NLP use cases.
These AI/ML Projects Give You an Unfair Advantage: Explore innovative and cutting-edge projects that can set you apart in the AI and machine learning landscape. Gain practical experience and showcase your skills with high-impact solutions that make a real difference.
Multiple hiring managers have said that these two projects (re-implementations and also fine tuning) are strong additions to a portfolio! So I would say yes.
More Resources!
Review of math concepts: th-cam.com/video/OYJhBjnLp0I/w-d-xo.html
Gradient Descent: th-cam.com/video/QLlPzNvwYSM/w-d-xo.html
Linear Regression: th-cam.com/video/2vE3DqWdEXo/w-d-xo.html
PyTorch and Neural Networks: th-cam.com/video/SuqPZfwbAvA/w-d-xo.html
First-Principles Framework (Learn Fundamentals): www.gptlearninghub.ai/first-principles-framework
Beginner's Blueprint (Build Projects): www.gptlearninghub.ai/beginner-blueprint
Sure, implementing research papers from scratch is great and all, but why stop there? Using PyTorch? Nah, you’re still relying on a framework. To really understand it, you should rewrite PyTorch from scratch. But wait, PyTorch is written in Python-so, obviously, you should just rewrite Python, right? Actually, why trust Python at all? Better dive deep and rewrite C while you're at it. But hold on… C is just an abstraction too. Time to rebuild assembly from scratch! 😂
Waiting for Karpathy to drop llm.asm haha
Wait why write in asm just code in binary 🗿
Binary needs cpu lets build it.😂😂😂
I think writing the model using C++ would be a great advanced exercise, especially since some models, like 3D reconstruction, are implemented in C++. Just to clarify, I haven't implemented a single paper before, but I think it would be a good idea and if i had time i will try doing it.😂
@@zenitsu4139 llm.bin on the way
First project: Implementing a paper from scratch
Second project: Fine-tuning an Open Source LLM
00:05 Focus on these two AI/ML projects for long term success
00:27 Implementing new research papers monthly boosts your AI/ML projects.
00:49 Coding AI/ML projects from scratch is essential for ML enthusiasts
01:10 PyTorch is beneficial for learning AI/ML concepts
01:32 Using PyTorch to master machine learning concepts in just five lines of code
01:57 AI/ML projects provide unfair advantage
02:19 Fine-tuning AI models for better performance.
02:44 Implement models from scratch to excel as a scientist or engineer.
To get started: run repos available with papers (easy to find on paperswithcode) in the cloud or locally. Try them on different datasets and try tweaking them.
Understand the main parts of the code. Then replicate these papers before moving onto similar ones.
That's a really good suggestion, especially understanding the main parts of the code & attempting to replicate them!
Aside from implementing the models from scratch, learning the math behind the Machine Learning algorithms will also definitely make you a better scientists and engineers. That's what I'm planning to do.
Now 26k people have unfair advantage 😮💨, there's compitition even if you have unfair advantage
Bonus points for watching the full video though
32 k people watched it. Half of them will forget about the video after 2 minutes. Other half will think about it but will move on without doing anything. Very few will actually follow what the video said. The competition is actually really low if you are ready to put in serious effort
I am learning web dev rn so some of us are *just* watching
Now 50k, competition is increasing 😂
62k now
I unironically skip all the way to fine-tuning Mamba for better text classification
Yeah forget BERT that's for rookies
How many hours would it take to implement a paper? Like a very rough estimate?
heavily depends on your own experience and what you are starting from, if you dont know any programming its going to take a lot longer than someone who knows programming and basic machine learning concepts
If you don't use chatgpt for some reason - it would take eternity
Yup, definitely depends on experience and the complexity of the paper. If I focus 100%, I am usually able to implement a paper within a day, but it definitely took me a lot longer when I was starting out
Are there any particular papers you recommend to implement?
Video coming soon on that!
Maybe in 2020 this gave you an unfair advantage, in 2024 without a Masters + software engineer experience or PhD forget getting into the field if you’re in the USA, but cases do exist where you could get lucky or get a company that doesn’t know what AI actually is, giving you a lower barrier of entry
I agree that ML Research roles are difficult to crack without a Master's or PhD. But I've seen many crack ML Engineering with no PhD!
@@gptLearningHub what about without masters? Very few
This is the truth (coming from someone in the field with SWE experience and masters).
@@BusinessDuck69 Thanks for sharing your perspective! From what I've seen, it seems very dependent on the company, i.e. big tech and F500 companies have different recruiting filters.
You can still do MLOps
Hey, Where to find the begineer friendly research papers for ML as well as where to find the code for those papers?
I first recommend learning concepts like Gradient Descent, and becoming familiar with PyTorch before implementing papers!
Gradient Descent:th-cam.com/video/bbYdqd6wemI/w-d-xo.html
PyTorch Intro: th-cam.com/video/SuqPZfwbAvA/w-d-xo.html
If you're already familiar with those concepts, check out this video on my Top 5 AI/ML Papers: th-cam.com/video/zmmWjEDZn6g/w-d-xo.html
hey! can you give me more insights about your work with finetuning open source models with maths related dataset. I am working on something very very similar and I think it would help me alot! Thank you
Can someone please explain why people prefer PyTorch over tensor flow nowadays. I am learning tensorflow for a while now creating NN models . Now there is a trend of PyTorch everywhere.
I believe pytorch is used more in research setting, while tf is used more in commercial setting
Researcher use pytorch more since it has simpler syntax so you are more likely to find paper with implemented code
Okay totally noob question: Implementing a paper? What does that even mean? Do what the researchers did? I maybe am kinda doing option 2 right now, but never even heard of Option 1
I know tensorflow, I am certificate, is that ok to not go on pytorch?
I usually recommend that beginners learn PyTorch over TensorFlow if they don't know either, since there are more PyTorch implementations these days.
But if you already know TensorFlow, that's great man! You can definitely implement papers in TensorFlow.
Please recommend some papers for NLP, beginner starting out
word2vec is a great option! I'll have a video on this topic coming soon too.
That will depend on how much of ML you already know. If you have complete understanding of how a traditional neural net works and it's mechanisms feel completely natural/intuitive to you, then you should read on RNNs and try to understand a paper like *Sequence to Sequence Learning with RNNs by Surskever et. al.* where they described a multi layered LSTM encoder-decoder model. That was a novel idea at the time and has lots of material out there going over it including colab notebooks with in depth explanations.
After that try finding out the flaws or things that could've been explored in more detail than what you saw on the paper. Tweak the model, try adding dropout, teacher enforcing etc. Experiment.
After that search for other areas that might interest you or go deeper within NMT with something like *Neural Machine Translation by Jointly Learning to Translate and Align* by K. Cho (It's the seminal paper for the attention mechanism) as it's a natural follow up to the previous one.
@@Koan__ Seq2Seq Learning With RNNs is a classic! Thanks for sharing. This is wonderful advice. I also really agree on the benefit of implementing NMT papers even for those not as interested in NMT as they are in other NLP use cases.
Also NNFS is a great book for absolute beginners.
For sure, from-scratch implementations are the best
What is NNFS?
@@hemantthawani8719 neural network from scratch
Neural network's from scratch
Sentdex is a legend
Thank you
TOP CONTENT
Much appreciated!
These AI/ML Projects Give You an Unfair Advantage: Explore innovative and cutting-edge projects that can set you apart in the AI and machine learning landscape. Gain practical experience and showcase your skills with high-impact solutions that make a real difference.
Will this look good on my resume?
Multiple hiring managers have said that these two projects (re-implementations and also fine tuning) are strong additions to a portfolio! So I would say yes.
Lol CS humour