How to implement Logistic Regression from scratch with Python
ฝัง
- เผยแพร่เมื่อ 12 ก.ย. 2024
- In the third lesson of the Machine Learning from Scratch course, we will learn how to implement the Logistic Regression algorithm. It is quite similar to the Linear Regression implementation, just with an extra twist at the end.
You can find the code here: github.com/Ass...
Previous lesson: • How to implement Linea...
Next lesson: • How to implement Decis...
Welcome to the Machine Learning from Scratch course by AssemblyAI.
Thanks to libraries like Scikit-learn we can use most ML algorithms with a couple of lines of code. But knowing how these algorithms work inside is very important. Implementing them hands-on is a great way to achieve this.
And mostly, they are easier than you’d think to implement.
In this course, we will learn how to implement these 10 algorithms.
We will quickly go through how the algorithms work and then implement them in Python using the help of NumPy.
▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬
🖥️ Website: www.assemblyai...
🐦 Twitter: / assemblyai
🦾 Discord: / discord
▶️ Subscribe: www.youtube.co...
🔥 We're hiring! Check our open roles: www.assemblyai...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#MachineLearning #DeepLearning
Best concise video on logistic regression I have seen so far
That's great to hear, thanks Josiah!
went through when I first started video editing, now it's taking a whole new switch and learning soft will only boost my courage for the
Great video, but my only dubt comes when J'() is calculated as the derivate of MSE and not as the derivate of the Cross Entropy, which is the loss function that we are using
Found the answer yet? Please let us know if you do!
That's what I noticed too.
Derivation of Cross Entropy looks like derivation of MSE but y^ is calculated differently.
log loss
We should start with Maximum Likelihood. Take the log. Then take derivative with respect to B (or W). Not sure how she took derivative. I guess she used MSE for classification problem instead of using Binary Cross Entropy. 🤔. Log L(B) = Sigma_{i=1}{N}((y_i*B*X_i - log(1 + exp(B*X_)).
def sigmoid(x):
x = np.clip(x, -500, 500)
return (1/(1+np.exp(-x)))
To avoid overflow runtime error as the return statement can reach large values
This was a great video, will there be one in the future that covers how to do this for multiple classes?
how is the derivative of loss function w.r.t weights same for cross entropy loss and MSE loss ?
need more algorithms , you are the best
Thank you!
Great work from @AssemblyAI 👍✨thank you from India.
superb video! I am saying that because coding from scratch is important for me.
AWESOME EXPLANATION THANKS A LOT !!
You're very welcome!
Wow , what a great video, very helpful
Glad it was helpful!
Studying CSE in GUB from Bangladesh , Love the way you teach the explanation & everything ; )
Thanks for sharing this, I am doing something similar in JavaScript. The part about calculating the gradients for backpropagation is very helpful!
We should maximize likelihood or minimize minus likelihood, I think the cost function is missing a minus , Am i right ?
7:51 Why you didn't multiplied by 2 the derivatives?
Technically you would have to but in the end it does not make a difference, you only effectively half your learning rate if you don't multiply by two.
Same question arises for me too why it wasnt multiplied by 2 and for bias it took sum instead of mean ?
Very good! Thanks for your videos!
I have seen other videos where people use a ReLU function instead of sigmoid. Would this logistic regression algorithm be an appropriate place to use ReLU instead of Sigmoid? If not, why not?
Logistic regression typically uses the sigmoid activation function because it provides a probability interpretation and is mathematically suited for binary classification. ReLU is more commonly used in deep neural networks with hidden layers, not in logistic regression.
Great work! Thank you :)
love your code 👍
Thank you!
Amazing video, I'm liking so much this free course, I'm learning a lot, thanks! 😁💯😊🤗
Happy to hear that!
Love it. Keep up the good work
Excellent video. Thanks.
The same problem of missing the 2 multiplication in calculating dw, db in the .fit() method:
dw = (1/n_samples) * np.dot(X.T, (y_pred-y)) * 2
db = (1/n_samples) * np.sum(y_pred-y) * 2
It does not affect too much, but we follow the slide to not be confusing
I noticed this as well. But adding in those *2 reduced the accuracy of my predictor from 92.98% to 88.59%
But in bias why we are not using mean coz the formula says summation then it has 1/N too ?
with the partial derivatives, where did the multiple 2 go?
wow you are an amazing teacher thanks alot god l love youtube !!!!!
Awesome work, also, great English !
When you write the dw and db, shouldn't be (2 / n_samples)? There is the 2 in the derivative that you can take outside of the summation
That's what I am thinking from previous video
Hello @AssemblyAI, where can I find the slides?
why you have not used summation for dw for calculating error
how can find presentation file.
I have a question, why can't you include accuracy function in class module?
Why doesn't this algorithm work on the Diabetes dataset? I'm getting an accuracy of 0
Please I want to build a personality assessment through cv analysis using this model, could you please help me?
7:09 The gradient should not have the coefficient 2 . 7:43 linear_pred there should be a minus sign before np.dot.
The above mentioned issue does not have much effect on the predicting result though.
Does this use regularization?
Thanks 🙏 ❤
What if you have categorical data? Like if different scale
Is there a way to visualize this?
Amazing video!
Can you add the plot of it?
Super 👏👏
Hi, does an increase in sample size increase the prediction accuracy?
Look for statquest's Logistic regression playlist.
Amazing
Relevant feature selection not shown
I didn't know exactly why you imported Matplot Library
Can we make this work for multiclass classification?
yes you can use logistic regression for problems like digit recognition as well
You are superb
My accuracy came to be around 40, even after tweaking with n_iters, lr couple of times, is that okay ?
same
I think we made the same stupid mistake where we did not loop through for n_iters. for _ in range(self.n_iters):. silly me
But why are we using numpy here it's supposed to be from scratch
Can it be used to predict house prices?
Why not!
wish one day i will be in this level of coding
Where is the backward propagation? In conclusion logistic regression is also a neural network
In predict(), is it X.T or just X for the linear_pred dot product?
I haven't used enough Python yet to accept the soul-crushing inevitability that there's going to be "self." everywhere. I guess you could call it "self." hatred. Maybe ligatures could come to the rescue, replacing every instance with a small symbol. While we're at it, put in ligatures for double underscores and "numpy." (or in the case of this video, "np.").
Yes, it's an aesthetic rant that is ultimately not a big deal, but gradient descent is a beautifully simple concept. The presenter does a great job of matching that simplicity with clean, easy to follow code. Maybe it's not such a bad thing to be irritated at the parts of her code which are inelegant only because the language doesn't give her better options.
make for multiclass also
Like it most
Awesome!
A M A Z I N G
تو عالی هستی❤
What is this witchcraft? I thought ML was supposed to be too hard to know wth is going on!
NICEEE ♥💙💙💚💚
-1/N
♥♥♥♥♥
Amazing video but you're going over too fast
Thanks for the feedback Lucian!
i got 0.94 accuracy with below code:
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn import datasets
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression
bc = datasets.load_breast_cancer()
X, y = bc.data, bc.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1234)
clf = LogisticRegression(penalty='l2',max_iter=10000)
clf.fit(X_train,y_train)
y_pred = clf.predict(X_test)
def accuracy(y_pred, y_test):
return np.sum(y_pred==y_test)/len(y_test)
acc = accuracy(y_pred, y_test)
print(acc)