Tutorial 31- Back Propagation In Recurrent Neural Network
ฝัง
- เผยแพร่เมื่อ 11 ก.ย. 2024
- Please join as a member in my channel to get additional benefits like materials in Data Science, live streaming for Members and many more
/ @krishnaik06
Please do subscribe my other channel too
/ @krishnaikhindi
Take the best online course on Data Science
www.appliedaic...
Connect with me here:
Twitter: / krishnaik06
facebook: / krishnaik06
Instagram: / krishnaik06
I have to admit, I was stuck on Recurrent networks for the longest time until I found your forward and backward propagation videos. Everything seems to have finally clicked. Thank you!!
if you are watching this in lockdown you are one of the rare species on the earth . many students are wasting their time on facebook, youtube, twitter, netflix, watching movies playing pubg, but you are working hard to achieve something . ALL the best ...nitj student here
Yo ! Thanks for sharing your Director with us ;-).
@@Virtualexist Director?
@@charukant100 Yes. 😂😂 It's an inside college joke.
I defintely recommand your tutorials to data science learners who ever asks me ....before entering into this feild.
When i think to give up all.... just i go through the video then i think how easy this made ..you motivates a me lot
Thanks helping Sir.
Only thanking you is not enough. Lots of dua for you Sir.
Danke from Germany!
video is good,
if you want take this as a suggestion,
try to write the generalized equation at the end.
thankyou for a good video
Krish, "agr aap na hote to kya hota" , i am thinking :) , amazing explanation. could you please make a video on what is loss function.
We are computing the gradients at every timestep right. What is the use of all these timestep gradients, as we update the weights only after 1 iteration right. Moreover the weights are the same throughout the iteration at every time step.
Great content Krish! just a suggestion, try using different color markers to explain such contents where you have existing forward propagation and explaining back propagation on top of it!!
How does weights get updated during reverse time? As he mentioned that the diagram is the same and he has drawn it only to show how it works in time frames. So is it something like for every time frame we use different set of weight? or didn't I understand the concept properly?
th-cam.com/video/Xeb6OjnVn8g/w-d-xo.html
He has explained it in a simple way
back propagation is in different direction than normal ann ie through time patterns i guess
Bought your book. Liked it so far!
Great video Krish. Keep it up.Thanks
Great content 🙏
Hello sir. The videos and the entire deep learning playlist is very informative. Learnt alot from you. Thank you so much.
I had a doubt.
Dont we apply learning rate while updating the weights in RNN?
Yes learning rate also applied.
Yes
Sir please make a video that how the backpropagation equations are derived in RNN. Thanks Sir.......
You work hard bro
watching the first 10 lectures will make your process faster than ever!
bro i have an exam tomorrow and i guess i am doomed because all this is too much for my small brain. thanks for explaining.. idk what u did but it sounds correct.. keep it up man
hi I have a doubt regarding the topic pls see though it
1st . when we find the weight w then we have to find it only once ?? (since weight is shared )
2nd can u pls tell how we can calculate/update the weight w'
pls reply thanks
Nicely done. The notation could have been slightly different to make it slightly less confusing.
5:30 while doing back prop after finding Loss, do we update same w 4 times, cuz it's a same w in all time steps ?.
Is y^ is only dependent on w'' not on O4? why do we neglect O4 and w'' while updating the weights in back propagation? waiting eagerly for the answer...
Where you found negate o4 and w" in backs propagation? Look at the derivative dL/dw
Is input weight at layer t-1 and output weight at layer t-1 are the same or different?
if different what is the relation?
Happy new year , u r doin so much efforts yaar. God bless
Happy new year 2020
Sir,
@ 3:41 I think that, It will
W`` = W`` - [ (learning rate) * (dL/dw``) ]
But in the video, it is written that
W``= W`` - (dL/dw``) =================> learning rate is missing
Am I right sir?
Yes it is.. sorry missed the learning rate
Great explanation sir! Just one doubt, since the weights are same for inputs, do we need to propagate back through all the time sequences?
Good question bro. Yes each input (word here in NLP case) has its own features and affecting the output in that way. So that affect factor or weight for each input must be different. To find that optimal value we have to change each weight.
@@ansariarbaz3374 oh I get it now! Thanks a lot!
Krish you are best as always
So for each process of weight updating, the RNN updates T times the length of the sequence?
thank you for the effort you are making
So much for Happy new year 2020😀😁
you mentioned that w and w' is the same for all inputs and outputs in the beginning but during back propagation you state that each w and w' will change? Arent they all the same?
The goal of back propagation is to determine how much the weights need to change to minimize error. While this is a single layer network, we treat it as a t*layer network (with t being timesteps), but all using the same neuron. Thus, the weights are all the same at each layer. If I understood correctly, we determine how much the weights change each layer (as if this were a normal t*layer network), sum the changes, and update w and w'. TLDR they're the same during calculating, then changed at the end.
Happy new year 2020 (seriously) [Just kidding ] love your work and appreciate your hard work.
I am learning on "Happy New Year 2024". 😀
So in this situation, we are not going to add the learning rate in the formula to compute the new weights?
Like we were doing in the ANN back Propogation?
Is each hidden layer giving an output y^ or in final we are getting a single output y^ ?
because in the beginning of this video he told that this is single hidden layer based on different time intervals.
what is w ' ' .... is w' and w' ' are same ?
please anyone explain.
yes I guess
So basically this is a many to one RNN , so multiple inputs single output . Example can predicting the last word of a sentence from previous words . So each hidden layer does not give an output Y^ , it gives the current information as output which is transferred to next layer and so on . w' and w'' are different , one is for each hidden layer for multiplying the input and the w'' is for multiplying the information / output being transferred from one layer to next
I have a question: Why does input of softmax has a different weight w' ' and not w ' ?
in the previous video, you said that weight assigned with input features that is x1*w or x2*w is the same which is represented by w (as in this video as well) and while explaining weight update at 4:48 you said that various equations will be considerd while updating weights, if weights are same then why is it needed to update weights independently ??an quite confused with this video.
Because the output of each layer is diff so while backpropogating w for each time stamp will change
did you get the actual reason ?
Is Elman Recurrent NN or Simple Recurrent NN anonyms to each other or they r different in theory?
small confusion sir.. please clarify.. dl/dw = (dl/dyp x dyp/d04 x d04/dw)
sir, did you missed to mention about the learning rate to be multiplied while updating the weights?
Thanks 😊 sir
In this video, where is the output 'Y' defined ? is it part of the dataset where there is a collection of sentences and thier sentiments are represented as "Y" output column?
What about the learning rate that needs to be multiplied to the partial derivative when calculating the new weights while backpropogating?
He forgot about it or ignored for simplicity I think
hi just one thing its not global minima it's a local one.......in any case
You have said you will reach the global minima…. How can you prove that RNN will reach global minima….any reference?
Sir I think this video is incomplete which makes it kinda wrong. Can you please reupload an updated and correct version.
Thanks Krish!
Any one answer please. So an input of 10 words are passed to RNN. After the first word is processed, we get 'y1' and also 'o1' . Now the output we get is not final. But will 'y1' this be passed through the activation layer and intermediate result or the activation happens only when the 10th word is trained?
input weights and output weights are same ? I know input weights are same and output weights are same .
is the hidden state and hidden layer is the same also this hidden layer is same as we have the in ANN so how the dimension of this calculated
Hlw bro...
Which laptop is best for machine learning (for beginners) under 40000?
Can I use it for long time (After becoming an intermediate)? Or should I upgrade?
Any laptop with minimum 8GB of RAM. You can use Google colab once you get into Deep Learning or you can update your RAM storage. It's up to you! Eventhough 4GB is enough for beginners, you can't depend on 4GB for some algorithms like 'Random Forest'. So I recommend you to buy any laptop with minimum of 8GB RAM.
You can use Kaggle or Google Colab for training,No need for a excellent computing computer
but in RCNN the backprpgation is done in time steps as opposed to other neural networks
you are the best
Thanks Krish
short and precise :)
the gradient dL/dw=(dL/dO4) (dO4/dw)
I have a doubt so while forward propagation the same weights are used for each time step right, but while back propagation we are talking about different weights for taking derivative and updating them, I am confused now...
did you got the conclusion
How do we use RNN for multivariate time series models?
🔥🔥🔥🔥
What is the difference between w' and w1?
thanku sirrr
So no learning rate in RNN?
I think learning rate also should be there in that formula written.
yes
Please make a seperate video on how actually the chain rule working in back propagation and please someone help me that what will be the chain rule equation for w of x11 and w' of O1.
yeah happy 2020 :(:(:(
But in Andrew NG's video from the deep learning course of coursera, there were outputs (y^) counted for each feature seperately. But you calculated one final output only. Will these two methods generate different results?
That's a many to many architecture, here we're assuming a many to one architecture. That's why we have many input units but one output.
@@midun2977 yes I got that part later as I progressed through the coursera course. Thank you.
I am planning to buy a laptop for machine learning ....but confused between Mac book air and Lenovo legion y540...which will be better?
Try buying a laptop with good gpu so that you can complete training faster
YOU LOOK TIRED ! TAKE REST AND BOUNCE BACK
Isnt there a bias in RNN?
To understand properly we can take bias as zero.
yes,you have to add bias in rnn
how will i find you w` any suggestion?
can anyone clear my doubt? My doubt is " Number of words is equal to no of hidden layers?"
Thus we can conclude that the only difference in weight updation b/w RNN and ANN is the learning rate absence coz it seems like rest all process is same. Please tell!
krish sir, forgot to include learning rate
when it knows to end i mean it has reached global minima
Hi everyone,
can anyone tell me how does the chain rule applies on the w' weights, and how we update them?
Actually you need to add all errors for calculating dl/dw, like
dl/dw=dl1/dw+dl2/dw+dl3/dlw+..,
(dl1/dw is for time stamp 1, dl2/dl3 for time stamp 2,...).
Then, you can update w=w-dl/dw, as w is same in all layers loss..
if we change the weights of neural network "w" will that weight "w" will change in all neural networks Plz help!!!
th-cam.com/video/Xeb6OjnVn8g/w-d-xo.html
He has explained in a simple way
only last updated value of W will be in record ?
I have the same question. How the fuck can we find derivatives wrt weights of all prev. Times
Watching this on 1st jan 2023 .. happy new year 😂😂
LSTM please
Now it 2022
bhai aap irritate bahut krte ho
Great video Krish. Keep it up.Thanks
Thank you sir