Overfitting, Underfitting, and Model Capacity

Can a machine learning model predict a lottery? Let's find out!
Deep Learning Crash Course Playlist: • Deep Learning Crash Co...
Further reading:
Deep Learning by Ian Goodfellow:
www.deeplearningbook.org/
Introduction to Machine Learning by Ethem Alpaydin
www.cmpe.boun.edu.tr/~ethem/i...
CS231n: Convolutional Neural Networks for Visual Recognition by Andrej Karpathy
cs231n.github.io/

[@mashiach_a]

You're literally answering all of my questions regarding Machine Learning and NNs in every minute in these videos. Thanks a lot!

[@billy818]

I just finished an undergrad datascience unit and this feels 20x better than the little ann lectures we had

[@MUKESHSINGH-ou1ou]

thank you for this video .You have explained underfitting and overfitting in easy to grasp way.

[@chengjoseph9459]

Thank you Leo. I've gone through alot of explanations on underfitting and overfitting + difference between validation and test set and this one of the best explainations so far :)

[@Patiencelad]

Excellent explanation Leo! Thank you.

[@zablonwafula7976]

Great explation. Start showing them practically in R

[@Nedjmeddine16]

one of the best series on Deep Learning I ever seen in the whole web !
keep it up :)
PS: I watch yours videos every morning before going to work.

[@leoisikdogan]

Thanks, happy to hear that!

[@abhishek.bansal05]

To the point content. Easy to digest and understandable. This is the best course on deep learning. Thanks👍🏻

[@giampaoloflace8887]

Best explanation ever! Thanks

[@stay_fr0sty]

Thanks, awesome explaination

[@shreyanshjain4250]

Great Work! Best so far. Please do a detailed series on CNN please.

[@fns3122]

very good course, please also upload a course on the KERAS ?

[@aryansathish]

Thank you

[@studiespotatoe5757]

clap !!! Excellent !!!

[@jorgeamaral8537]

Leo, Thanks for the excellent course! I have a question about the use of the train, validation and test set. I saw an implementation where they implemented a deep learning network by first using gridsearch CV to find the best configuration of the hyperparameters such as learning rate, optimizer, etc and then they use a validation set to perform the early stopping with the validation set. Do you have another recommendation on how to use the validation set to perform the hyperpararameter tunning?

[@leoisikdogan]

Random hyperparameter search works better than grid search given a limited number of experiments you can run. One caveat is that if you tune so many hyperparameters to maximize validation set performance there is a risk of overfitting the hyperparameters to the validation set. To avoid that you can either use two separate validation sets or use early stopping during the search. I usually do neither though. So far my strategy in choosing the hyperparameters is to use heuristics such as fast.ai's automatic learning rate finder rather than doing an exhaustive search. I'm actually planning to make a video about hyperparameter and architecture search whenever I get to have some extra free time.

[@cahitskttaramal3152]

Hocam emeğine sağlık.

[@saeidbagheri7407]

Great video!

[@leoisikdogan]

Saeid Bagheri Thanks!

[@saeidbagheri7407]

You mention regularization techniques for reducing the models capacity. What are some other techniques to reduce the capacity?

[@leoisikdogan]

That's what I will be talking about in the next video. I will cover the techniques that limit the effective capacity of models, such as early stopping, data augmentation, weight decay, etc.

[@saanvisharma2081]

Very informative and clear.
Btw, can you please make some video's on time series analysis.
Ounce again thank you very much.

[@leoisikdogan]

Thanks! I briefly covered time series in this video about recurrent neural nets: th-cam.com/video/k97Jrg_4tFA/w-d-xo.html

[@MrJaleelJamal]

7:13 so there is a difference between validation set and test set. But is there a difference between validation loss and test loss? Because I would have thought that in the table test loss is instead of validation loss.

[@ramapatipatra4814]

awsome

[@thedashh3675]

thanks , love from india

[@haoranxi3319]

Can someone explain, What does "the model has low variance but high bias" mean? thanks

[@ankitjain2398]

Model having high bias means high training error and low varience means model is not overfitted i.e model is best fitted

[@akankshabali5859]

please can you make video on adam optimizer !! i didnot understant it if you have this video please share with me

[@leoisikdogan]

I briefly explained Adam and other popular optimizers in this video: th-cam.com/video/kK8-jCCR4is/w-d-xo.html

[@sureshothwarang]

Is this also called model convergence?

[@sureshothwarang]

Please confirm this thanks

[@leoisikdogan]

Model convergence usually refers to when the loss becomes stable and stops improving.

[@sureshothwarang]

Leo Isikdogan how is the model fitness measured? Is this through MSE?

[@niallmcauley5325]

Your voice is horrific

[@niallmcauley5325]

Very good video though, explained the concept well.

[@kemchobhenchod]

His voice is one of the most interesting and endearing parts of the presentation. Don't be a dick.

[@mattbiss6414]

If you dont have anything nice to say dont say it at all! Great video, very well explained thank you so much :)