This is not Backprop through time, this is just normal backprop. It does not work on LSTMs or even RNNs as the differential of loss on input weight does not only depend on hidden state, but as each hidden state in time t depends on state in t-1 which in turn depends on input weights again. the differential itself propagates backwards through all time steps. This was the whole point of the paper "Backpropagation through time".
great lecture Sir. I was just wondering at 20:00 the derivation of Ct has its first component as (Yhat - Y) but then a few seconds later you refer to the same derivation as (Yhat - Yhat). is this a mistake? if not, does this simply meen subracting each element of Yhat by itself?
Looks like the backward propagation here , is only doing it for current time step , and not going back in time similar to RNN. Ex: Since both g2 and h1 have Wg in their formulas indirectly , shouldn't we add Dg2h1 to Dg2Wg , as was case in RNN. Could anybody help clarify ?
Yeah, i think you are correct, these formulas look easier than for RNN, it should be the other way around. I think the real formulas are more complex. But according to this paper formulas are correct arxiv.org/pdf/1610.02583.pdf . I am confused...
LSTM architecture is more complex than RNN architecture, but formulas for gradient look much easier than for RNN. I think your formulas are incorrect. For example: dL / dWi = .....dit/dwi, and you write that dit/dwi = ht-1, but ht-1 is also depends on wi, because ht-1 = Ot-1*tanh(ft-1 * ct-2 + it-1 * gt-1), where it-1 also pedends on on wi
The only lecture series I've found that actually goes into the needed details, great job!
Thank you so much! This was the most in-depth video for LSTMs.
You deserve more likes sir...I loved it
Congratulations sir, a very clear explanation, not only here but in your other videos as well. Very helpful!!
A huge amount of work done by you. Thanks teacher!!!
Great lecture. Could you please upload your next video soon. Wanted to know on why vanishing gradient problem is not happening in LSTM.
This is not Backprop through time, this is just normal backprop. It does not work on LSTMs or even RNNs as the differential of loss on input weight does not only depend on hidden state, but as each hidden state in time t depends on state in t-1 which in turn depends on input weights again. the differential itself propagates backwards through all time steps. This was the whole point of the paper "Backpropagation through time".
Sir your lectures are really awesome ✨
A small request sir,if you can make videos on machine learning algorithms, it'll be very helpful ✨
Thanks Sir! Nice explained backpropagation!
why Ht-1 dimension is HxH, shouldn't be BxH?
great lecture Sir. I was just wondering at 20:00 the derivation of Ct has its first component as (Yhat - Y) but then a few seconds later you refer to the same derivation as (Yhat - Yhat). is this a mistake? if not, does this simply meen subracting each element of Yhat by itself?
this is gold.
awesome explanation. One question, shouldn't we calculate the derivative of the Loss with respect to all biases as well?
yes, bias is also a tunable parameter. he just forgot about it
Looks like the backward propagation here , is only doing it for current time step , and not going back in time similar to RNN. Ex: Since both g2 and h1 have Wg in their formulas indirectly , shouldn't we add Dg2h1 to Dg2Wg , as was case in RNN.
Could anybody help clarify ?
Yeah, i think you are correct, these formulas look easier than for RNN, it should be the other way around. I think the real formulas are more complex.
But according to this paper formulas are correct arxiv.org/pdf/1610.02583.pdf . I am confused...
clear cut explanation thank u sir
Nice and thanks 👍👍
Finally, subtitle were added.
Deep Leaning lecture 68 and 69 are not available yet.
Sir, thanks a lot.
Thank you!
Hi.. Is vanishing gradient video available for LSTM?
LSTM architecture is more complex than RNN architecture, but formulas for gradient look much easier than for RNN. I think your formulas are incorrect. For example: dL / dWi = .....dit/dwi, and you write that dit/dwi = ht-1, but ht-1 is also depends on wi, because ht-1 = Ot-1*tanh(ft-1 * ct-2 + it-1 * gt-1), where it-1 also pedends on on wi
goodddd jobbbb my man, you are the freaking only person who goes into the details, i fucking love you please marry me.
果然是印度的英语😥