Why did not I stumble upon this playlist earlier? The way you have clearly explained each algorithm from scratch is amazing! Thanks a lot! Keep up the good work
What a beautiful ML video, i regret not finding this when i started machine learning back a year ago, gotta go back to basics and watch this whole series.
U are Incredible, your teaching style is exceptional. U are doing such an amazing job for Tech Geekies; Keep Going with the full course. May God Bless U.
The Euclidean distance is wrong, it should be something like that: np.sqrt(np.sum(np.square(x2 - x1))) which gives a different answer an example would be: X_test = [7.1 3. 5.9 2.1], X_train =[4.8 3.1 1.6 0.2] ((4.8-7.1)**2+(3.1-3)**2+(1.6-5.9)**2+(0.2-2.1)**2)**(1/2) = 0.2236...
Is it just because of simplicity that you choose to implement the majority vote in this way, as it "arbitrary" chooses the most common label if there is more than one most common label?
Loved this video! I really enjoyed the way you described everything you were doing. It was really helpful as I was able to take notes as well! Great lesson, Thank you!
beginnerquestion here. in this line: euclidean_distance(x, x_train) for x_train in self.X_train....... where do you get x_train from? i understand self.X_train is the original iris-matrix that we will test against. i understand the x is every x-value in the test-samples. but how does python know that x_train is every x value in self.X_train?
I like your video. I want to ask about KNN method, I have a problem. How if the target is dependent value in string type with KNN method. So I have case, I want to classify customer in few segmentations like A, B, C and D but using KNN method. I will appreciate your help, thank you so much
Planning to start this series after watching the Pytorch. is it just possible to learn deep learning and not worry about ML? I understand it's all related, but wondering if it's possible to go in the Deep Learning/Neural Network route. Any feedback will be appreciated. thanks. In short, is it worth watching this series for getting into ML route.
It might be possible. fast.ai has a free course that directly jumps into DL. But it is always helpful to understand the basics, and I would recommend starting with ML basics. My ML From Scratch series helps a lot of beginner to really see the concepts behind those algorithms. Have a look if it is helpful for you, too...
that's kind of like saying you want to be a rocket scientist without learning fundamental physics. You really need the ML foundations to understand how neural networks came about and how new things are discovered. Additionally, neural networks aren't the best thing for all problems. In plenty of cases you can save hours of fine tuning a deep learning model by using a classical ML algorithm. I'd highly recommend Andrew Ng's machine learning course on coursera along with this series of videos. You can quickly look through the algorithms, understand how they work and move on to the exciting deep learning things! Good luck!
@@sidesplitter9497 I agree with you in that learning about the fancy stuff should be done after learning the basics and your suggestion of Andrew Ngs course is right on in my opinion. I find it interesting that just in a couple of years the framing has changed from can neural networks do X to can X be done with anything other than a neural net Let's say for argument sake though that someone is interested in the subject of image classification do you still think they should learn about the basics of machine learning then, or learn about CNNs directly? This kinda brings the discussion to a top-bottom vs bottom-up approach to learning
@@AladdinPersson It may be possible for others but I can't imagine diving straight into CNNs without a good knowledge of ANNs as a whole. I also can't imagine learning about the neural network model without a decent grasp on logistic regression and before that, linear regression. If you knew that what you wanted to do was going to involve purely CNN image classification then I do understand the argument against learning some classical machine learning algorithms (naive bayes, SVMs, and such) but I struggle to believe that someone could jump straight into fairly complicated deep learning models without some classical background However, I do accept that some are much more apt than me!
Instead of class KNN:, I used function knn() so clf.fit(X_train,Y_train) gives me AttributeError: 'NoneType' object has no attribute 'fit'. How to resolve this error? Please help.
If a give an input list for the KNN algorithm to predict the classes of each element, How can I print out the list of inputs only belonging to a particular class?
This is very helpful. How could this be changed to a weighted k-nearest neighbor? Assume we would switch out the counter feature for the vote with a weight calculation.
Hello. You can use the inverse distance function to calculate a weighted sum for class 0 and class 1. So w0 = 0, w0+= (1 / distance) for every nearest class 0 point, and same with class 1. And then decide the class based on: w0 > w1
Hey man I have a huge question, I've been comparing the accuracy of this model vs the one that's implemented by Scikitlearn and apparently there's a huge difference, this one seems to perform better, I'm suspicious that it might be because it uses most_common method, which is pretty much the mode while the scikit one says it works with mean. Any comments or suggestions on this? Thanks in advance!
scikit-learn uses more complex and optimized implementations that perform differently (but in most cases it performs better than my simple example implemenation)
Hi bro, I check all of your videos. it was very helpful. but please make a proper video that how to deliver a project to a client. mean a proper complete project.
Yes there is no training in the fit() implementation, but we have to store the training features and labels in the fitting step. Then in the predict step we compare the new data to the stored features and look at the closest neighbors
hello, thank you for making this course, but I can't follow every of your code, can you give me a suggest of which course should I study first, I already studied basics python
You need to know the basics of linear algebra. (Calculus and functions mainly tbh), there's a series on neural networks by 3blue1brown(I think it was only 4 videos). I suggest you watch that!
Hi , I know the concepts of K-NN algortihm very well and theory behind it .But when it comes to implement the same in python prog I am struggling hard as I am new to coding world. How to overcome this problem ? I love to implement ml algorithms from scratch but once I open python console I struck and don't know how to start and write code for algorithms :-(
Probably you should learn python first until you feel confident enough. Then move on to learn the basics of numpy. And only then try to implement ML algorithms. Don’t feel overwhelmed. Follow some tutorials and then Do your own little projects step by step and then slowly increase the difficulty
@@patloeber that would be a superb idea🔥....I downloaded all your code from GitHub, and has been working on them, You are helping a lot of enthusiastic newbies, please keep doing what you are doing ❤️🌟
Hi +Python Engineer +UCbXgNpp0jedKWcQiULLbDTA I recently started learning python, it is one of my courses in graduation. I am wondering how would we know a certain function exists in python modules. In this case counter method. And how do you know while your learning? Please help me. Since there is n number of functions and modules and most of the people don't know all of them. Please help beginners like me. I love this video and thanks for the video.
don't worry! you don't have to know all the built-in functions. just practice and code as much as you can and then you'll pick up the functions when you need them. when you feel confident enough with beginner stuff, then i have a playlist with advanced topics. in episode 6 i will explain the counter method th-cam.com/video/UdcPhnNjSEw/w-d-xo.html
These are 2 musical (jazz) solos generated using K Nearest Neighbors classifier: th-cam.com/video/zt3oZ1U5ADo/w-d-xo.html th-cam.com/video/Shetz_3KWks/w-d-xo.html
everyone just using sktlearn and pass but this king explaining the algroithm and writing without any library. You are the best man!
thank you!
This and Corey schafer's playlist are true goldmines. Keep up the good work :)
thanks :)
Why did not I stumble upon this playlist earlier? The way you have clearly explained each algorithm from scratch is amazing! Thanks a lot! Keep up the good work
Thanks 😊 I’m glad that it’s helpful
What a beautiful ML video, i regret not finding this when i started machine learning back a year ago, gotta go back to basics and watch this whole series.
Glad you like it :)
mindblown !!!!!!!! Took the word scratch too seriously : )
Way too underrated . Keep up the good work bro .
Thanks a lot :)
@@patloeber Hi, Can you please make a video on XGBoost from Scratch ?
Better than any other ML courses video from famous Universities. Every concept is crystal clear by following the workshop once.
This is really called how to make ml algos from scratch.
Highly underrated video! Such a great way to learn the algorithm and python.
Thank you very much ! Glad you like it :)
Discovered it now. This is absolutely amazing! Thx.
this is the definition of from scratch :D
Hehe thanks
@@patloeber thank you for such good tutorial!
Very elegant coding which makes the learning experience a pleasure, congratulations and very many thanks!!!
Thank you :)
U are Incredible, your teaching style is exceptional. U are doing such an amazing job for Tech Geekies; Keep Going with the full course. May God Bless U.
Thank you very much!!
Thanks a lot. Now i see the usefullness of OOP even in such small code.
Great! I plan a short OOP series in a few weeks :)
Fantastic. Thank you for taking the time to make and share the video. Simply beautiful.
can i start this playlist directly? or should i have a background of some other things? advise me plz!
The real machine learning course what I wanted to hear.
glad you like it!
Thank you，clear and easy to understand
This's really helpfull for me to understan the algorithm, thank you so much!
Thank you. Your implementation is easy to understand and follow. Keep it up.
Thanks!
Thank you! Thank you! Thank you! (Came here from your reddit post)
Cool :)
16:08 you coulda used mode from scipy stats
The Euclidean distance is wrong, it should be something like that:
np.sqrt(np.sum(np.square(x2 - x1)))
which gives a different answer
an example would be:
X_test = [7.1 3. 5.9 2.1], X_train =[4.8 3.1 1.6 0.2]
((4.8-7.1)**2+(3.1-3)**2+(1.6-5.9)**2+(0.2-2.1)**2)**(1/2) = 0.2236...
Is it just because of simplicity that you choose to implement the majority vote in this way, as it "arbitrary" chooses the most common label if there is more than one most common label?
Loved this video! I really enjoyed the way you described everything you were doing. It was really helpful as I was able to take notes as well! Great lesson, Thank you!
Thank you for the feedback :) I’m glad that you like it!
Mind Blowing! Sir what about other models please. Are you gonna prepare it for us on this?
Thanks! Right now there are 14 algorithms in the ML from scratch series. I will add some more in the future
@@patloeber Thanks a lot sir. Thanks for your reply
beginnerquestion here. in this line: euclidean_distance(x, x_train) for x_train in self.X_train....... where do you get x_train from? i understand self.X_train is the original iris-matrix that we will test against. i understand the x is every x-value in the test-samples. but how does python know that x_train is every x value in self.X_train?
Thank you for teaching
Wow, man your videos are amazing. I am almost new in coding but carrying good background knowledge of math and ML. They are like a liquid gold
can you make al lthe videos same as the PCA video where you have given a theory (very crisp) and then implementation? Thanks! awesome video thought!
Thanks for such an elegant easy to follow implementations.
thanks you!
Could you use a heap to make the sorting more efficient?
You are amazing... Fabulous coding... Keep it up buddy 💕
Thank you :)
6:43 Sir, I want to ask, if the label is of type string, should it be converted to an integer?
Yup
@@tobiemmanuel8306 thank bro
Nice one! Keep going!
Thank you very much! More videos are coming soon...
Which notebook you are using
Jupyter notebook or something else ?
Getting the error: unsupported operand type(s) for -: 'str' and 'str', while calculating Euclidean Distance. How should it be solved?
Me, too here: unsupported operand type(s) for ** or pow(): 'tuple' and 'int'. Did you find any solutions?
@@dioutoroo you are trying to calculate distance between tuple and int. both have to be int. you ca access the value within ur tuple
@@AKSHATABHATABCE that is what I found, finally. Thanks, mate!
Great session! if possible can we have video abut implementation KNN-weighted algorithm? Thank you .
Thanks for this good tutorial
Hi
Why didn’t use Jupiter notebook and sikitlearn instead of vscode ? Your answer will be very helpful
Very informative and easy to understand explanation, thank you sir!
glad you like it!
I like your video. I want to ask about KNN method, I have a problem. How if the target is dependent value in string type with KNN method. So I have case, I want to classify customer in few segmentations like A, B, C and D but using KNN method. I will appreciate your help, thank you so much
I have started following you
Thank you:)
Planning to start this series after watching the Pytorch. is it just possible to learn deep learning and not worry about ML? I understand it's all related, but wondering if it's possible to go in the Deep Learning/Neural Network route. Any feedback will be appreciated. thanks. In short, is it worth watching this series for getting into ML route.
It might be possible. fast.ai has a free course that directly jumps into DL. But it is always helpful to understand the basics, and I would recommend starting with ML basics. My ML From Scratch series helps a lot of beginner to really see the concepts behind those algorithms. Have a look if it is helpful for you, too...
that's kind of like saying you want to be a rocket scientist without learning fundamental physics. You really need the ML foundations to understand how neural networks came about and how new things are discovered. Additionally, neural networks aren't the best thing for all problems. In plenty of cases you can save hours of fine tuning a deep learning model by using a classical ML algorithm. I'd highly recommend Andrew Ng's machine learning course on coursera along with this series of videos. You can quickly look through the algorithms, understand how they work and move on to the exciting deep learning things! Good luck!
@@sidesplitter9497 I agree with you in that learning about the fancy stuff should be done after learning the basics and your suggestion of Andrew Ngs course is right on in my opinion. I find it interesting that just in a couple of years the framing has changed from can neural networks do X to can X be done with anything other than a neural net
Let's say for argument sake though that someone is interested in the subject of image classification do you still think they should learn about the basics of machine learning then, or learn about CNNs directly? This kinda brings the discussion to a top-bottom vs bottom-up approach to learning
@@AladdinPersson It may be possible for others but I can't imagine diving straight into CNNs without a good knowledge of ANNs as a whole. I also can't imagine learning about the neural network model without a decent grasp on logistic regression and before that, linear regression.
If you knew that what you wanted to do was going to involve purely CNN image classification then I do understand the argument against learning some classical machine learning algorithms (naive bayes, SVMs, and such) but I struggle to believe that someone could jump straight into fairly complicated deep learning models without some classical background
However, I do accept that some are much more apt than me!
Wish I found this Channel earlier!
now you found it :)
Where is part 2? Thank you
this is gold, thanks 🤠👍
Instead of class KNN:, I used function knn() so clf.fit(X_train,Y_train) gives me AttributeError: 'NoneType' object has no attribute 'fit'. How to resolve this error? Please help.
Excellent video ... Thank you
Thanks for watching :)
If a give an input list for the KNN algorithm to predict the classes of each element, How can I print out the list of inputs only belonging to a particular class?
so, how do we use it for predict some new dataset?
This is very helpful. How could this be changed to a weighted k-nearest neighbor? Assume we would switch out the counter feature for the vote with a weight calculation.
Hello. You can use the inverse distance function to calculate a weighted sum for class 0 and class 1. So w0 = 0, w0+= (1 / distance) for every nearest class 0 point, and same with class 1. And then decide the class based on: w0 > w1
Hey man I have a huge question, I've been comparing the accuracy of this model vs the one that's implemented by Scikitlearn and apparently there's a huge difference, this one seems to perform better, I'm suspicious that it might be because it uses most_common method, which is pretty much the mode while the scikit one says it works with mean. Any comments or suggestions on this? Thanks in advance!
scikit-learn uses more complex and optimized implementations that perform differently (but in most cases it performs better than my simple example implemenation)
thank you so much
why there is sum in Euclidean distance bcoz those are just single values .. later on x and xtrain we are doin list and other calc etc.
what are you using to write this?
Hello sir, what is the difference between machine learning and machine learning forecast?
i just tried your code but then comes the error "unhashable type: 'numpy.ndarray' " . Am i missing something?
Thank for this video, I have a question : why did you use predict() and _predict() method separately?
to maintain clean code
Awesome keep going!
This is excellent
Hi bro, I check all of your videos. it was very helpful. but please make a proper video that how to deliver a project to a client. mean a proper complete project.
good point, I will add this to my list!
@@patloeber thank you. Because lot of people don't know it. included me.
can this course replace Andrew Gn's course on coursera ?
I have a question, if there r no parameters to optimize in KNN, then what are we fitting for?
Yes there is no training in the fit() implementation, but we have to store the training features and labels in the fitting step. Then in the predict step we compare the new data to the stored features and look at the closest neighbors
hello, thank you for making this course, but I can't follow every of your code, can you give me a suggest of which course should I study first, I already studied basics python
have a look at this study guide: th-cam.com/video/dYvt3vSJaQA/w-d-xo.html
Can you make knn with cosine distance??
Can i use this for the Titanic dataset problem if I don't want to use scikit learn
Yes you can!
what's the best place to download ML research papers? is there any one good site? also looking for beginner friendly :)
have a look here: paperswithcode.com/ and here: arxiv.org/list/stat.ML/recent
Daaammnn! Loved it
Great :)
What are the requirements to follow this serie ?, I'm intermediate in python programming and i forgot mathematics
You need to know the basics of linear algebra. (Calculus and functions mainly tbh), there's a series on neural networks by 3blue1brown(I think it was only 4 videos). I suggest you watch that!
@@raunak1147 thanks a lot, i'll check this series thanks. ❤🌹
Hi ,
I know the concepts of K-NN algortihm very well and theory behind it .But when it comes to implement the same in python prog I am struggling hard as I am new to coding world. How to overcome this problem ? I love to implement ml algorithms from scratch but once I open python console I struck and don't know how to start and write code for algorithms :-(
Probably you should learn python first until you feel confident enough. Then move on to learn the basics of numpy. And only then try to implement ML algorithms. Don’t feel overwhelmed. Follow some tutorials and then Do your own little projects step by step and then slowly increase the difficulty
please add video about mlticlassfication
Not sure what you mean. KNN is able to do multi class classification. The example in the video is actually multi class
can you do a tutorial about t-sne?
Not yet. Maybe in the future
@@patloeber looking forward to it
Can you add Greek subtitles to your video?
Can you add on to this video by adding cross validation?
Good suggestion! I'll add it to my list
wow, perfect video !
thank you!
how to get the data graph.?
Hi! any idea on how to solve this? ValueError: Expected n_neighbors
more samples or less neighbors ;)
Almost there, but you missed some key details.
Hi there,
I am new to Python, I am used to using R.
Simple question. Which editor do you use?
VS Code
@@patloeber thanks very much. your lessons are awsome.
@@songyouk9522 thanks :)
so fucking good!
I tried it but I get "no module named knn". What do I have to do to fix this?
Try from .knn import KNN (with the dot). Import paths can be a little bit different on Windows and Mac...
@@patloeber Thank you.
how to import iris dataset in pycharm? any tutorials?
The same testing code I am using at the end should work independent of the editor, so it should work with PyCharm too
can someone explain why the first prediction function returns np.array and why not a simple list?
because we work the whole time with numpy arrays. it's much faster than a list
@@patloeber
yes but why distances returned a list insteand numpy array
can u help me how to do that tutorial on specific datasets(microarray data sets)
Hey, for specific questions you can contact me on Twitter or email.
why would the default be 3?
usually you want an odd number, and 3 is the smallest one after 1. but you can use others as default too
Please use .csv file data. I tried and not working.
Can I know what is iris.data and iris.target
iris.data is your x (feature vector), iris.target is your y (the class labels)
Does anyone know of any books that does this (code the Algorithms from scratch)?
I don't know of any books that do this. Maybe I should write one :D
@@patloeber that would be a superb idea🔥....I downloaded all your code from GitHub, and has been working on them, You are helping a lot of enthusiastic newbies, please keep doing what you are doing ❤️🌟
great! Thanks for the support :)
I love your accent ...where are you from?
Germany ;)
Python Engineer wundarbar! 🌭
How about you ?
Python Engineer Australia 🇦🇺
I like the Australian accent too :)
i love you video 😍😍
Hi +Python Engineer
+UCbXgNpp0jedKWcQiULLbDTA
I recently started learning python, it is one of my courses in graduation. I am wondering how would we know a certain function exists in python modules. In this case counter method. And how do you know while your learning? Please help me. Since there is n number of functions and modules and most of the people don't know all of them. Please help beginners like me. I love this video and thanks for the video.
don't worry! you don't have to know all the built-in functions. just practice and code as much as you can and then you'll pick up the functions when you need them. when you feel confident enough with beginner stuff, then i have a playlist with advanced topics. in episode 6 i will explain the counter method th-cam.com/video/UdcPhnNjSEw/w-d-xo.html
These are 2 musical (jazz) solos generated using K Nearest Neighbors classifier:
th-cam.com/video/zt3oZ1U5ADo/w-d-xo.html
th-cam.com/video/Shetz_3KWks/w-d-xo.html
Google Colab won't take __init__ T^T
clap clap clap
for theoretical clarification of concept:
th-cam.com/video/CJjSPCslxqQ/w-d-xo.html