L-6 Optimizer | Learning Rate | Weight Updation

Explained: What is Optimizer and why we use Optimizer?
What is Learning rate and how to choose right value for learning rate?
You can ask me your queries in comment section. I will try to answer all your queries.
or You can email me at [e-mail]
What is an optimizer?
Optimizers are algorithms or methods used to minimize an error function(loss function)or to maximize the efficiency of production. Optimizers are mathematical functions which are dependent on model’s learnable parameters i.e Weights & Biases. Optimizers help to know how to change weights and learning rate of neural network to reduce the losses.
This video will walk you through the optimizers and some popular approaches.
Optimizers are algorithms or methods used to change the attributes of your neural network such as weights and learning rate in order to reduce the losses.
How you should change your weights or learning rates of your neural network to reduce the losses is defined by the optimizers you use. Optimization algorithms or strategies are responsible for reducing the losses and to provide the most accurate results possible.
We’ll learn about different types of optimizers and their advantages in this video.
Gradient Descent
Gradient Descent is the most basic but most used optimization algorithm. It’s used heavily in linear regression and classification algorithms. Backpropagation in neural networks also uses a gradient descent algorithm.
Stochastic Gradient Descent
It’s a variant of Gradient Descent. It tries to update the model’s parameters more frequently. In this, the model parameters are altered after computation of loss on each training example. So, if the dataset contains 1000 rows SGD will update the model parameters 1000 times in one cycle of dataset instead of one time as in Gradient Descent.
Learning Rate
How big/small the steps are gradient descent takes into the direction of the local minimum are determined by the learning rate, which figures out how fast or slow we will move towards the optimal weights.

[@indrajitnaskar6851]

The most experienced teacher ever

[@vasugakher4657]

the way you explaining each topic is really good

[@CodeWithAarohi]

Glad my videos are helpful

[@nazmakhan4794]

Nice video

[@CodeWithAarohi]

Thanks

[@pifordtechnologiespvtltd5698]

👏👏

[@hamidraza1584]

Also launch videos on large language models , llama , gork.. .and rag pipe line.love from Lahore Pakistan

[@CodeWithAarohi]

Sure, Soon!

[@mohamed-rayanelakehal1324]

what's the relation between Learning rate and number of epoches, since we use both of them to update weights ?, THANK YOU

[@CodeWithAarohi]

The relationship between the learning rate and the number of epochs depends on the specific problem and the characteristics of the data. In general, a higher learning rate may require fewer epochs to converge, while a lower learning rate may require more epochs to converge.

[@bsuresh1406]

super explain madam and please make nlp videos advanced madam it will help full to me

[@CodeWithAarohi]

Will do soon

[@nasimthander1296]

Hi Arohi, you are doing great job. Can you able to share your ppts please?