Breaking Linear Regression

What happens if we break the assumptions of linear regression for machine learning?
RESOURCES
[1] What do linear regression parameters mean: datascience.st...
[2] This has some good intuition that distinguishes applications of linear regression: stats.stackexc...
[3] Why should residuals be normally distributed? qr.ae/pvl91W
[4] Penn State's notes on the assumptions of linear regression: online.stat.ps...
[5]What do normal residuals tell you about your data: stats.stackexc...
[6] Wikipedia on linear regression (it's actually pretty good): en.wikipedia.o...
[7] Better understanding of "Linear in parameters": datascience.st...
MATH COURSES (7 day free trial)
📕 Mathematics for Machine Learning: imp.i384100.ne...
📕 Calculus: imp.i384100.ne...
📕 Statistics for Data Science: imp.i384100.ne...
📕 Bayesian Statistics: imp.i384100.ne...
📕 Linear Algebra: imp.i384100.ne...
📕 Probability: imp.i384100.ne...
OTHER RELATED COURSES (7 day free trial)
📕 ⭐ Deep Learning Specialization: imp.i384100.ne...
📕 Python for Everybody: imp.i384100.ne...
📕 MLOps Course: imp.i384100.ne...
📕 Natural Language Processing (NLP): imp.i384100.ne...
📕 Machine Learning in Production: imp.i384100.ne...
📕 Data Science Specialization: imp.i384100.ne...
📕 Tensorflow: imp.i384100.ne...

[@Brickkzz]

This channel is like an alternative universe where Raj Siraj is actually humble, smart and not a scammer

[@CodeEmporium]

Many thanks for the compliments:D

[@NicholasRenotte]

😂 preach

[@AI_BotBuilder]

Funny seeing Nicholas and Code emporium, two of my favs and the one I hate discussed all in one comment space

[@kalpeshmore9499]

THANK YOU, MATH GIVES A CLEAR IDEA BEHIND THE ALGORITHM

[@theepicturtleman]

Thank you so much for the video, super easy to understand and cleared up a lot of misconceptions I’ve had about the assumptions of linear regression

[@EuforynaUderzeniowa]

7:23 The peak of blue line should be above the intersection of green and dashed line.

[@chirchan]

Love your work ❤️. Keep posting more stuff

[@CodeEmporium]

Thanks so much! I shall :) (new video in 2 days)

[@blairnicolle2218]

Tremendous!

[@Raj-gc2rc]

Please do math for deep learning and explain why it's mostly trial an error ... how can math guide in achieving state of the art models if it's possible ...
And how much would theory of statistics be helpful in actually coming up with good models .. like right now I dont know how it can be applied, ex) knowing that stochastic gradiant descent is an unbiased estimator of the gradient ... what does knowing this fact tells us

[@harryfeng4199]

Much appreciated. Thank u

[@CodeEmporium]

You are very welcome :)

[@NeoZondix]

This is awesome, thanks

[@CodeEmporium]

You are very welcome :)

[@RAHUDAS]

Can we do same MLE for logistic regression??? Can you make a tutorial on that??

[@CodeEmporium]

Yep!

[@RAHUDAS]

@@CodeEmporium Sorry I forgot thanks you for the content, it was really great, I am learning a lot from your tutorial

[@shivamkaushik6637]

Damn! So MSE I was using was just Maximum log likelihood this hole time.

[@CodeEmporium]

Yep!

[@javiergonzalezarmas8250]

This is beautiful

[@CodeEmporium]

Thanks so much!

[@user-wr4yl7tx3w]

Why is it sometimes called ordinary least squares?

[@CodeEmporium]

Ordinary Least Squares and Maximum Likelihood estimation are 2 different techniques to find the parameters of a linear regression. The former tries to minimize the sum of squared errors while the other latter will maximize the probability of seeing data. But for the linear regression case, MLE will effectively converge to the OLS equation (just doing a lil math shown in the end of the video)

[@chirchan]

@@CodeEmporium thank you for the explanation

[@n_128]

Why is theta^2 a problem?, if it is just a constant, the model can learn that constant. Let's say you have a k^2 = theta and we find theta. The only problem I understand is if that theta in theta^2 changes (like x does), but that's not the case.

[@CodeEmporium]

Good question and it gets me thinking. The true / best value of theta is a constant (as you correctly point out). But at the time of training, we don’t know it‘s value. In the eyes of the model during training (in particular, the cost function), theta is now a variable. And so we can take derivatives with respect to these theta values to find the minima / maxima values and get an estimation for the true cost.
However as I pointed out in the video, if you take the derivative with respect to the theta squared terms, you’ll end up with equations that are not linear (in terms of theta) and hence can’t be solved with linear algebra. This is why we can’t write a definite statement of “The best value for theta is X”. Instead, we would need to resort to estimation techniques like gradient descent to get an “estimate” of the value for theta.
Hope this clears something’s up

[@adityaj9984]

If I frame this problem again in form of least sum of squares, can I incorporate probability into it? If yes then how? I asked this question because MLE is purely based on probability where it incorporates knowledge of Normal Distribution in it. But in case of least sum of squares, I didn't see it happening.

[@user-wr4yl7tx3w]

An idea, can you consider a video on generalized method of moments.

[@CodeEmporium]

Potentially yes. :) I'll put some more thought into this

[@louisvuittondonvg9040]

Like your thumbnails

[@CodeEmporium]

Thanks so much! I’m trying out new ideas :)

[@Septumsempra8818]

Have you guys tried the EconML library by Microsoft for causal machine learning?

[@CodeEmporium]

I have not. But sounds interesting :)