You are so under- rated, this video needs to reach everyone out there who are struggling with the basics, as videos like this is super rare to find online. Keep up the good work :-)
I love finally finding a series that explains the absolute bare basics. So tired of tutorials that are like "Ok, so start by importing torch, then use the prebuilt DataLoader class to load a built-in dataset, then run it through a built-in model and print the result. Congrats, you did AI"
After watching one video, I subscribed to your channel and planing to finish PyTorch tutorial in 2 days. It is very helpful in understanding all the basics. Highly appreciate your efforts in making this course, keep up the good work:-)
Holy crap, thank you for making this series. I've been struggling with this stuff for months now and was finding it nearly impossible to make headway towards really grasping it. The basics seemed impossible to get a firm hold of and apply. These videos and how you broke everything down to the most basic levels made it so much clearer. It feels like I've made more real headway in the last couple hours than in the month before that.
7:56 here, output_size = n_features is just because n_features is 1. I think you should simply write output_size = 1 to avoid confusing. For regression problem, in general, output_size is 1.
Thanks very much for this tutorial! It really helps me a lot. Even though I am doing deep learning, I never went through the basics systematically step by step, and your videos make me understand things better. Keep up your amazing work!
I really love your series! Beside of that, I think I've spotted a minor mistake or rather "unpleasant" thing and just wanted to quickly mention it here for the sake of completeness. At 4:12, you've called "loss(Y, y_pred)". Technically it should be "loss(y_pred, Y)" as the usual notation of the loss functions of Pytorch is always "loss(y_prediction, y_target)". In this case, as mean squared error was used, it doesn't make any difference (because it is commutative with respect to Y and y_pred). But if you would've used another loss function, this leads to unexpected behavior or errors.
@@patloeber Sir, I copied your code from Github and managed to run on Jupyter, but the compiler showed up an invalid syntax from your print function File "", line 45 print(f'Prediction before training: f(5) = {model(X_test).item():.3f}') ^ SyntaxError: invalid syntax Can you fix it for me? Thanks
Thank you very much for your excellent course. Just because I was unfamiliar with the syntax of classes but had taken a machine learning course at university, I needed to pause the videos and search for some concepts. (such as super() , __init__ and so on)
was trying to play around with the params and inputs. initially x_train values were [1,2,3,4]. I tried to put a larger value in it. like x_train = [100,2,3,4]. After that the gradients started exploding and the loss became nan after few epochs. Any idea why this is happening.
We are implementing a linear regression model here, which has single parameter = w. The .item() can be only applied on tensors which have 1*1 dimensions - meaning - essentially - they are scalars. Hence, doing w.item() is also okay here. But, when we will have complex models - which won't have single parameter w or may have parameters like w1, w2, .. etc. captured by the tensor w (model.parameters() unpacked), then, in that case w.item() won't be possible. Hence, probably as a good practise, the tutor used w[0][0].item() instead of w.item()
Thanks for this series. I am moving from tensorflow to pytorch and it's always good to start from scratch. I appreciate that you are starting from math to numpy to pytorch. If there was another step to move to pytorch lightning that would be great ! (I think I can figure that out myself for now). Thanks again ! :)
Thank you again for your excellent videos. To make this experiment more similar to the previous video, I think that the bias shouldn't be optimized: model = nn.Linear(input_size, 1, bias = False)
@@patloeber And to make it even more similar, we can initialize the (in this case, only) weight to zero with: "model.weight.data = torch.zeros(model.weight.shape)"
Hi, thanks for the videos. Quick question: at 8:00, we say that the output_size is equal to the input_size. This is generally the case or only in this particular example? Thanks in advance.
hi I don't get it when you define model=nn.linear(inputsize,outputsize) which has 2 arguments but in epoch you give model 1 argument(in line: y=model(x) in 9:49)!!!!! lets put aside that x and inputsize,outputsize are not the same thing
Very nice tutorial. Is there a mistake here saying "in the last tutorial, we implemented logistic regression from scratch?" but it is about linear regression?
Could you please explain how this class extension is done: like why does the class LinearRegression, need to implement the forward method in the class? why does the lin(x) take only one parameter? I always get confused about extending classes and if you could explain/make a video for it. what would be a good resource to learn about extending classes and/or extending classes for custom layers?
Mostly, how do you pick the starting learning rate? Afterwards, I think you tend to use a scheduler? (question should have been posted in the comments of the lr_scheduler video...sorry!)
in manual optimization (e.g. in SGD) we need and use the gradient directly in the update formula. it seems there should be a direct relation between loss.backward() and optimizer.step(), (i.e. optimizer.step() should get an input from nn.backward() ) unless under the hood the optimizer uses the grad calculated by loss.backward() if somebody could explain what happens ...
optimizer.step performs a parameter update based on the current gradient (stored in .grad attribute of a parameter) and the update rule. See here: discuss.pytorch.org/t/how-are-optimizer-step-and-loss-backward-related/7350
This is the syntax to call the super method of the nn.Module base class: See here: stackoverflow.com/questions/576169/understanding-python-super-with-init-methods
Thank you for this video,but i have a question about training model with pytorch, I'm working on prediction model with pytorch and i need that the process of preparing data abd training run in parallel ,is there any way of doing that?
Thanks great work again! One question related to the topic you might have an idea on. When we only have one output like "good" or "bad" I have seen people using softmax with a Dense layer and 2 neurons at the end and people using sigmoid using a Dense layer with 1 output. What do you think is the better approach?
Good question. I prefer one output for binary classifiers. It needs fewer parameters (less prone to overfitting and errors), and therefore fewer calculations, so slightly faster updates. On the other hand 2 layers can be more flexible...
Great tutorial buddy!! Can u explain "model(X_test)" is coming as -0.449 . I want to know the process like what is happening "model = nn.Linear(input_size, output_size)"
calling model(X) will calculate the forward pass and give us the predicted values. Usually we have to define the forward pass in our model class, but since we are only using one nn.Linear layer, the forward pass is already available
Dear Python Engineer, When you hover over a method or a class, you get a really nice quick documentation for pytorch methods. Did you do any additional operation such as adding some external documentation to pycharm, or it is the default behavior of your pycharm program? Btw, great content. We look for more customized models! Best, Ibrahim
can someone please explain why Adam is considered adaptive and how it operates in conjuncture with zero_grad? if it is adaptive, using zero_grad will only allow it to be adaptive within the batch that you supply it at every step; am I correct? it adapts within the batch or is something wrong with this interpretation and with using zero_grad before each step?
how to write the python code in oop? I know the basics ...but I still dont know how to go about the flow.. like you wrote..like writing an essay or a story
I actually love this video and the tutorial 7. I am a newbie in the ML/AI. This video helped me understand how to code from scratch to train a prior model. I'd love to see a video on how to modify this code to do reinforcement learning. I am working on a task in which I can train a prior model but now I need to apply reinforcement learning to this model. I know my reward function (score from 0 to 1) but I am not sure what exactly I should change to the code and especially how to modify the loss function. It is the nn.CrossEntropyLoss loss function. I know that my new loss function should be a function of both the original loss function and reward. (correct me if I am wrong). Is there anyone can give me some guidance? thanks
Hi. Really glad you like it! hmm Reinforcement Learning can be complex. I am planning a tutorial for this, but it may take a few weeks...Maybe this tutorial is helpful for you in the meanwhile: pytorch.org/tutorials/intermediate/reinforcement_q_learning.html
When I copy the custom model and run, "TypeError: LinearRegression.__init__() takes 1 positional argument but 3 were given" occurs. I check the code line by line, I promise there's no typing mistakes. Is there anybody can help me out?
@@ВикторКуркучеков Yes in Python 3 it's ok to just use super(). The arguments are just for backwards compatibility, and because PyTorch uses it like this in their documentation as well. But you can omit them.
I was executing the code, and I encountered this error: w -= learning_rate * w.grad (your code is right) w = w - learning_rate * w.grad (mine causes error) Can anybody explain this to me? Aren't these two lines the same?
![เปิดคำทำนาย นอสตราดามุส - บาบา วานก้า ปี 2025 คนทั่วโลกต้องเจออะไรบ้าง? | แฉ 11 ธ.ค. 67 [2/3] |GMM25](http://i.ytimg.com/vi/xmTIKWJUMPM/mqdefault.jpg)
You are so under- rated, this video needs to reach everyone out there who are struggling with the basics,
as videos like this is super rare to find online. Keep up the good work :-)
thank you so much! I'm happy that it is helpful :)
I love finally finding a series that explains the absolute bare basics. So tired of tutorials that are like "Ok, so start by importing torch, then use the prebuilt DataLoader class to load a built-in dataset, then run it through a built-in model and print the result. Congrats, you did AI"
Thank you ,so much. Surviving university because of guys like you.
Much appreciated!!
After watching one video, I subscribed to your channel and planing to finish PyTorch tutorial in 2 days. It is very helpful in understanding all the basics. Highly appreciate your efforts in making this course, keep up the good work:-)
Thank you! I'm happy that it's helpful :)
I did the same
Dude, this is fucking better than any tutorials I've ever seen
Hehe thank you so much :)
Thanks for this awesome series on Pytorch. I would definetely love to see more!
Thank you very much!
This is the good tutorial ever I have seen for PyTorch
Thank you!
Holy crap, thank you for making this series. I've been struggling with this stuff for months now and was finding it nearly impossible to make headway towards really grasping it. The basics seemed impossible to get a firm hold of and apply. These videos and how you broke everything down to the most basic levels made it so much clearer. It feels like I've made more real headway in the last couple hours than in the month before that.
Really happy to hear that :)
7:56 here, output_size = n_features is just because n_features is 1. I think you should simply write output_size = 1 to avoid confusing. For regression problem, in general, output_size is 1.
I liked every single one of your video I watched so far. Thanks so much for the tutorial!
Thank you for the feedback! I’m happy you like it :)
Really Awesome Series of Videos with no words to express my Gratitude Patrick.!!!!!!!!
Just what I needed and have been looking for.
glad to hear that
Great stuff, helped me break through some of the basics!
glad to hear that :)
Thanks very much for this tutorial! It really helps me a lot. Even though I am doing deep learning, I never went through the basics systematically step by step, and your videos make me understand things better. Keep up your amazing work!
Great to hear! That's why I did this series :)
Brother, you are doing wonderful work.
Keep it up!!
Take love!!
Thanks! Glad you like it :)
Damn.. This was the bestt video i had ever seen of pytorch.. Thank you so much for the amazing amazing amazing content!
Wow, man!! You are awesome. Thanks for uploading this playlist. Much Love from India
glad you like it :)
You are really amazing in explaining! Solid examples of every step.
thank you!
I really love your series!
Beside of that, I think I've spotted a minor mistake or rather "unpleasant" thing and just wanted to quickly mention it here for the sake of completeness.
At 4:12, you've called "loss(Y, y_pred)". Technically it should be "loss(y_pred, Y)" as the usual notation of the loss functions of Pytorch is always "loss(y_prediction, y_target)".
In this case, as mean squared error was used, it doesn't make any difference (because it is commutative with respect to Y and y_pred). But if you would've used another loss function, this leads to unexpected behavior or errors.
yes, thanks for pointing this out :)
I just wanted to thank you for your awesome codes, man I learned a lot here. please don't stop and keep going man
Thank you very much! I am really happy that this is helpful :)
Very interesting, awesome and step by step explanation. Thank you so much
You deserve 1.000.000 subscribers
Thank you 😊
@@patloeber Sir, I copied your code from Github and managed to run on Jupyter, but the compiler showed up an invalid syntax from your print function
File "", line 45
print(f'Prediction before training: f(5) = {model(X_test).item():.3f}')
^
SyntaxError: invalid syntax
Can you fix it for me? Thanks
Hey, Thank you so much. Your videos are helping me with my research.
Happy to help!
Great vid, super clear!
Your channel is very helpful my friend ❣️
Thanks for the meticulous tutorial ✨ after plenty of try I find the all the fundamentas here🔥
Thanks for watching :)
Thank you so much! This playlist has helped me a lot!
glad to hear this!
Thank you very much for your excellent course. Just because I was unfamiliar with the syntax of classes but had taken a machine learning course at university, I needed to pause the videos and search for some concepts. (such as super() , __init__ and so on)
was trying to play around with the params and inputs. initially x_train values were [1,2,3,4]. I tried to put a larger value in it. like x_train = [100,2,3,4]. After that the gradients started exploding and the loss became nan after few epochs. Any idea why this is happening.
why don't we use only w instead of w[0][0] in printing statement (timing 10:40)
We are implementing a linear regression model here, which has single parameter = w. The .item() can be only applied on tensors which have 1*1 dimensions - meaning - essentially - they are scalars. Hence, doing w.item() is also okay here. But, when we will have complex models - which won't have single parameter w or may have parameters like w1, w2, .. etc. captured by the tensor w (model.parameters() unpacked), then, in that case w.item() won't be possible. Hence, probably as a good practise, the tutor used w[0][0].item() instead of w.item()
Thanks for this series. I am moving from tensorflow to pytorch and it's always good to start from scratch. I appreciate that you are starting from math to numpy to pytorch. If there was another step to move to pytorch lightning that would be great ! (I think I can figure that out myself for now). Thanks again ! :)
Glad you like it!
Thank you again for your excellent videos. To make this experiment more similar to the previous video, I think that the bias shouldn't be optimized: model = nn.Linear(input_size, 1, bias = False)
You are correct! I see you are very familiar with PyTorch :)
@@patloeber And to make it even more similar, we can initialize the (in this case, only) weight to zero with: "model.weight.data = torch.zeros(model.weight.shape)"
thank you very much，very nice tutorial
You teach better than my professor 😩
Thanks! Glad you like my teaching style :)
Which college r u from?
this is so good!! thank you
Hi, thanks for the videos.
Quick question: at 8:00, we say that the output_size is equal to the input_size. This is generally the case or only in this particular example?
Thanks in advance.
only in this particular example
@@patloeber great, thanks :) i saw most of the series btw and it rocks! Keep up the good work.
Can we also apply this in the matrix similarly (the gradient)?
hi I don't get it when you define model=nn.linear(inputsize,outputsize) which has 2 arguments but in epoch you give model 1 argument(in line: y=model(x) in 9:49)!!!!!
lets put aside that x and inputsize,outputsize are not the same thing
Thanks for this Class. It's helping me a lot.
I'm glad it is helpful :)
Very nice tutorial. Is there a mistake here saying "in the last tutorial, we implemented logistic regression from scratch?" but it is about linear regression?
Could you please explain how this class extension is done: like why does the class LinearRegression, need to implement the forward method in the class? why does the lin(x) take only one parameter? I always get confused about extending classes and if you could explain/make a video for it. what would be a good resource to learn about extending classes and/or extending classes for custom layers?
How do you tune hyperparameters like learning rate? Do you use a lr_finder? Or what's your pipeline for improving metrics?
Mostly, how do you pick the starting learning rate? Afterwards, I think you tend to use a scheduler? (question should have been posted in the comments of the lr_scheduler video...sorry!)
in manual optimization (e.g. in SGD) we need and use the gradient directly in the update formula. it seems there should be a direct relation between loss.backward() and optimizer.step(), (i.e. optimizer.step() should get an input from nn.backward() ) unless under the hood the optimizer uses the grad calculated by loss.backward()
if somebody could explain what happens ...
optimizer.step performs a parameter update based on the current gradient (stored in .grad attribute of a parameter) and the update rule. See here: discuss.pytorch.org/t/how-are-optimizer-step-and-loss-backward-related/7350
Hi.. why do you use the super() method for the same class?
This is the syntax to call the super method of the nn.Module base class: See here: stackoverflow.com/questions/576169/understanding-python-super-with-init-methods
Thank you for this video,but i have a question about training model with pytorch,
I'm working on prediction model with pytorch and i need that the process of preparing data abd training run in parallel ,is there any way of doing that?
Does the super initialization just initiate the NN.module constructor (that we cant see), as well as the super LinearRegession class constructor?
the NN.module with all its magic behind the scenes...
How do I modify the code if the inputs are multiple linear instead of 1,2,3,4 I have like [1,1], [2,2,], [3,3], [4,4] and is there a way plot it out.
Helpful ❤
pls explain or give sources or make a vid about what is optimizer and what are its benefits?
optimizer will make your life easier ;) it solves the update calculations for you
Thanks great work again! One question related to the topic you might have an idea on. When we only have one output like "good" or "bad" I have seen people using softmax with a Dense layer and 2 neurons at the end and people using sigmoid using a Dense layer with 1 output. What do you think is the better approach?
Good question. I prefer one output for binary classifiers. It needs fewer parameters (less prone to overfitting and errors), and therefore fewer calculations, so slightly faster updates. On the other hand 2 layers can be more flexible...
Great video!
Do you consider making videos with reinforcement learning models with PyTorch? That would be great!
thanks! I already have one with RL and the snake game
@@patloeber great! Found it! Thanks!
Can you please tell which IDE you are using, is it sublime text? because for me sublime editor is not working with PyTorch
VS Code
thats awesome!
Sooooo helpful :-)
Great tutorial buddy!! Can u explain "model(X_test)" is coming as -0.449 . I want to know the process like what is happening "model = nn.Linear(input_size, output_size)"
calling model(X) will calculate the forward pass and give us the predicted values. Usually we have to define the forward pass in our model class, but since we are only using one nn.Linear layer, the forward pass is already available
Dear Python Engineer,
When you hover over a method or a class, you get a really nice quick documentation for pytorch methods. Did you do any additional operation such as adding some external documentation to pycharm, or it is the default behavior of your pycharm program?
Btw, great content. We look for more customized models!
Best,
Ibrahim
Thanks! My editor is VS Code. I just have the Python extension installed, this comes with auto suggestions.
can someone please explain why Adam is considered adaptive and how it operates in conjuncture with zero_grad? if it is adaptive, using zero_grad will only allow it to be adaptive within the batch that you supply it at every step; am I correct? it adapts within the batch or is something wrong with this interpretation and with using zero_grad before each step?
Thank you
glad you like it!
how to write the python code in oop?
I know the basics ...but I still dont know how to go about the flow..
like you wrote..like writing an essay or a story
Aren't the number of samples and dimension of input different things?
perfect!!!!!
Why on earth this series has so less views.. not justified
I actually love this video and the tutorial 7. I am a newbie in the ML/AI. This video helped me understand how to code from scratch to train a prior model. I'd love to see a video on how to modify this code to do reinforcement learning. I am working on a task in which I can train a prior model but now I need to apply reinforcement learning to this model. I know my reward function (score from 0 to 1) but I am not sure what exactly I should change to the code and especially how to modify the loss function. It is the nn.CrossEntropyLoss loss function. I know that my new loss function should be a function of both the original loss function and reward. (correct me if I am wrong). Is there anyone can give me some guidance? thanks
Hi. Really glad you like it! hmm Reinforcement Learning can be complex. I am planning a tutorial for this, but it may take a few weeks...Maybe this tutorial is helpful for you in the meanwhile:
pytorch.org/tutorials/intermediate/reinforcement_q_learning.html
When I copy the custom model and run, "TypeError: LinearRegression.__init__() takes 1 positional argument but 3 were given" occurs. I check the code line by line, I promise there's no typing mistakes. Is there anybody can help me out?
Excellant
I am still not clear about using super(), if we are in the same class then why are we using it?
We need it to initialize the super class (nn.Module) correctly so we can use its functionality
@@patloeber Thank you, I loved your tutorial. One of the best tutorials on pytorch I have ever seen.
@@patloeber Maybe I am wrong, but is super(LinearRegression,self) the same to super()? Then why we are use arguments?
@@pulkitverma8923 thanks :)
@@ВикторКуркучеков Yes in Python 3 it's ok to just use super(). The arguments are just for backwards compatibility, and because PyTorch uses it like this in their documentation as well. But you can omit them.
loss function should be loss(y_pred, Y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
I guess the order of those matters right??
Looks like pytorch requires more tedious work compared to keras
a little bit but it gives you more flexibility
@@patloeber which one do you use more often in your production systems?
I was executing the code, and I encountered this error:
w -= learning_rate * w.grad (your code is right)
w = w - learning_rate * w.grad (mine causes error)
Can anybody explain this to me? Aren't these two lines the same?