In my opinion, what makes this a great video is its (quick) review of the prerequisite knowledge, and including simple examples to that we can compute by hand and then verify that we actually get the same thing when using PyTorch. Thanks a lot!
14:50 x.grad contains the valuesd of partial{y} / partial{x} 17:50 x.grad.zero_() 25:00 gradient for multiple inputs -> multiple outputs. Since the Jacobian is a matrix, we need to input a 1-d tensor to get a valid vector-output. => But our loss function has been a scalar, so this is why I am not accustomed to this form. 34:10 explaining .detach(). => treat those as constants, not a variable that we differentiate w.r.t.
24:10 y.backward(torch.ones(3)) the input to y.backward is upstream gradient and the size of the upstream gradient should be same as the size of y and in previous cases we dont need insert upstream gradient cause y,backward() same as y.backward(torch.tensor([1]))
thanks man i was finding it extremely difficult to understand the maths behind backward and detach (although i have doen it in my high school) because no one was explaining them in this depth.........love you 😍😍
First of all, I want to express my gratitude to you for the work you have done. There is one thing i want you to ask: why do we write the partial derivatives of the scalar function in the form of column? Whereas, following the logic of Jacobian matrix, it should be a row. Thanks in advance!
The derivative of any variable. let's say x, wrt itself, i.e., x, is equal to 1. That is why we use a vector of v of all ones. If for some reason, you want to exclude one variable from the gradient vector, you can then replace the corresponding entry of v with 0.
@@DrDataScience that does make sense thanks! But why do we multiply with v or ones what does the dot product of jacobian and vector v represent? Why do we multiply derivative of self with jacobian?
Please Do cover a playlist on Graph Neural Networks (at least discuss all the basics and methods of GNNs). The internet world lacks quality contents on this topic
The way you define the y is different from what i am thinking. In first example, I thought we should define y as x**2 or in second one y as x. But if i define the y like this, I will get an error which say that it needs just one number not sequence of numbers.
This is what I was searching for months. I m so thankful Dr. Science for breaking down this concept for us
I'm glad you find it helpful!
In my opinion, what makes this a great video is its (quick) review of the prerequisite knowledge, and including simple examples to that we can compute by hand and then verify that we actually get the same thing when using PyTorch.
Thanks a lot!
Thank you for the nice comment! Glad this video was helpful.
This is what a proper tutorial should be. Thanks a lot. Subscribed
Thanks!
This video deserves more view.
Thanks a lot!
Excellent videos and the textbook, deeply admire your contributions
Truly thankful to you. To the point without confusion. Thank you once again
14:50 x.grad contains the valuesd of partial{y} / partial{x}
17:50 x.grad.zero_()
25:00 gradient for multiple inputs -> multiple outputs. Since the Jacobian is a matrix, we need to input a 1-d tensor to get a valid vector-output. => But our loss function has been a scalar, so this is why I am not accustomed to this form.
34:10 explaining .detach(). => treat those as constants, not a variable that we differentiate w.r.t.
24:10
y.backward(torch.ones(3))
the input to y.backward is upstream gradient
and the size of the upstream gradient should be same as the size of y
and in previous cases we dont need insert upstream gradient cause
y,backward() same as y.backward(torch.tensor([1]))
Thanks for the video, it really cleared up Pytorch autograd, now I will be making notes on this gold nugget
Thanks!
Amazing as always!! Very helpful and valuable video!
Nice explanation. Found it from youtube recomendations
I am glad you found it helpful!
why do you need to multiply v vector to the jacobian matrix? and what is v vector exactly?
thanks man i was finding it extremely difficult to understand the maths behind backward and detach (although i have doen it in my high school) because no one was explaining them in this depth.........love you 😍😍
great explanation! I hope it will get the likes it deserves
Thanks a lot!
Thank you sooo much! This finally clicked for me.
I am glad you found it helpful!
Awesome video! Thank you!
Thanks!
First of all, I want to express my gratitude to you for the work you have done. There is one thing i want you to ask: why do we write the partial derivatives of the scalar function in the form of column? Whereas, following the logic of Jacobian matrix, it should be a row. Thanks in advance!
thank you very much for uploading this video! very helpful!
way better than my prof...
Thank you!
Loser.
29:49 - As far as I understand in 'a.grad' should turn out [12., 18.]
Thanks a lot.
Thank you
Thank you!
hi !! thanks for the wonderfull video :) can you please explain why vector v is 1?? and what is derivative w.r.t to self??
The derivative of any variable. let's say x, wrt itself, i.e., x, is equal to 1. That is why we use a vector of v of all ones. If for some reason, you want to exclude one variable from the gradient vector, you can then replace the corresponding entry of v with 0.
@@DrDataScience that does make sense thanks! But why do we multiply with v or ones what does the dot product of jacobian and vector v represent? Why do we multiply derivative of self with jacobian?
thnks myfriend - in last section - whats is the z.sum ? what is the the (SUM) function for ? why yyou put the sum ?
Please Do cover a playlist on Graph Neural Networks (at least discuss all the basics and methods of GNNs). The internet world lacks quality contents on this topic
Great suggestion! I'll work on it.
The way you define the y is different from what i am thinking.
In first example, I thought we should define y as x**2 or in second one y as x. But if i define the y like this, I will get an error which say that it needs just one number not sequence of numbers.
Oh I got it. But i did not get what v vector is?
Fixed your broken web page that forces us to bend to your flaws.
No!