Stacking Classifier | Ensemble Classifiers | Machine Learning

Stacking is an ensemble learning technique to combine multiple classification models via a meta-classifier. The individual classification models are trained based on the complete training set; then, the meta-classifier is fitted based on the outputs -- meta-features -- of the individual classification models in the ensemble. The meta-classifier can either be trained on the predicted class labels or probabilities from the ensemble.
Let's first understand how a stacking classifier works and create a simple stacking classifier in Python.
If you do have any questions with what we covered in this video then feel free to ask in the comment section below & I'll do my best to answer those.
If you enjoy these tutorials & would like to support them then the easiest way is to simply like the video & give it a thumbs up & also it's a huge help to share these videos with anyone who you think would find them useful.
Be sure to subscribe for future videos & thank you all for watching.
You can find me on:
GitHub - github.com/bha...
Medium - / bhattbhavesh91
#stackingclassifier #ensemble #metaclassifier

[@bhattbhavesh91]

Code Link : github.com/bhattbhavesh91/GA_Sessions/blob/master/ga_dsmp_5jan2019/20_Stacking_Classifier.ipynb

[@malharjajoo7393]

Just a quick point: Would have been good if you had kept the dimension of the training set and the number of classifiers different. (here both are 3)
Can avoid confusion.

[@magelauditore333]

Awesome explanation. A very underrated channel

[@bhattbhavesh91]

Glad you think so!

[@deepakkapoor5427]

Hi I have few doubts-
1. So for the meta classifier we will not take any x features, we will only use probability from previous stage model?
2. We used logistic as base and meta classifier here. Wont there probability have high correlation?
3. Should we choose model as meta with maximum accuracy?
4. How can we extend this to multiple stage meta classifier than just two.
Thanks

[@VIGNESHPRAJAPATI]

Nicely Explained. Thanks Bhavesh.

[@mahadahmed6193]

amazingly well explained, thanks

[@bipinkapri9986]

What is the difference between stacking and voting? Difference in terms of performance and usage? When should we go for voting and when should we go for stacking?

[@bhattbhavesh91]

I have created a playlist on Voting Classifier, have a look -
th-cam.com/play/PLreVlKwe2Z0TYh4aCLNw91q9FjRftMSc9.html
There is no clear distinction when you should use voting or stacking! It all depends on what gives you better performance.

[@emmanuelameyaw6806]

The stacking classifier didn't improve the classification when I tried to reproduce your results. It had the same results as KNN and Random Forest. Not sure, maybe colab works differently for everyone...
F1 Scores: 0.9532 (+/- 0.0099)[KNN]
F1 Scores: 0.9532 (+/- 0.0099)[Random Forest]
F1 Scores: 0.9130 (+/- 0.0199)[Naive Bayes]
F1 Scores: 0.9532 (+/- 0.0099)[StackingClassifier]

[@bhargavpotluri5147]

Thanks for this video, it is helpful. One doubt, So if stacking classifier (which has both linear & non linear models) is giving better accuracy or result. What shall we conclude? Data is explained better with linear & non linear methods? which is hard to understand in normal terms

[@aneeshbose6331]

Since the meta classifier is learning features of each of the classifiers in the stacking classifier array, it seems to be trying to fit the data according to multiple classifiers, hence the tendency for overfitting or underfitting on the data is lesser and might be leading to a more accurate representation of the dataset to test on.

[@kunalmakwana4451]

Why we train meta classifier on the cv dataset why not on the training dataset?

[@LetsCOLORup]

Hi Bhavesh , thank you for explanation ... Would this fit for categorical data set and when feeding data into it should there be any pre processing done ?
Looking forward to your response and thank you .

[@malharjajoo7393]

Explanation is unclear:
1) How many classes are there ?
2) What are p1, p2 and p3 ?
If there are k-classes, each model will output k-probabilities.
Where are these k-probabilities in your explanation ? I can only see p1,p2,p3.

[@rajkumarbatchu4763]

Why do u take score.mean()

[@niveditadwivedi6466]

Hi, can we stack unsupervised algo on supervised?

[@moderncollectionkalyan3280]

can you explain stacking on regression model

[@zshan101992]

yea waiting for the same. @Bhavesh Bhat

[@azamtekno]

Hi... Thank you for the video it helps me to know more about stacking classifier, however i still confused regarding what makes stacking classifier result not increase or even worst than base claassifier ?
Currently I have tried to use stacking with naive bayes and decission tree as base classifier and (bagging naive bayes ) as meta classifier... But most of the results show stacking not increase the accuracy and even there is some which is worst.
Regarding diversity how can we analyse it ?
Thank you very much for your help.

[@RajaSekharaReddyKaluri]

where is train, validation and test size as part of code excersize??

[@kaviya2]

I'm also having the same doubt

[@anujsharma4011]

Great video.

[@magelauditore333]

Waiting for Ensemble on neural network

[@truthsacrament369]

Subscribed

[@bhattbhavesh91]

Thank you :)

[@billpeng5441]

How can I have your coding file on this video?

[@tomsmith6109]

is stacking classifier is good for binary classification?

[@bhattbhavesh91]

Hi Tom, It an ensemble classifier so typically it works better when compared to standalone classifiers.

[@karantekwani4503]

Sir what is the difference between voting and stacking classifier?

[@bhattbhavesh91]

Go through the video again, you will understand the difference! There is something called as a meta classifier in Stacking which is to be trained which is not present in Voting! Hope it helps :)

[@karantekwani4503]

@@bhattbhavesh91 but ultimately goal of both the algorithm is same.
Does both the classifier give same results?

[@bhattbhavesh91]

Every algorithms goal is to create something accurate! How it does it is different! No, both of them will give different results as both of them are differently trained!

[@karantekwani4503]

@@bhattbhavesh91 main doubt in stacking classifier is that, suppose model1 give prediction 1 , model 2 give prediction 2 , model 3 give prediction 3 , so how meta classifier give prediction
Means what input does meta classifier take to give final prediction?

[@karantekwani4503]

@@bhattbhavesh91 also what is difference between sklearn stacking classifier and mlxtend staking classifier?
Which one is faster?

[@MAK335]

why dont you forward the code here?

[@bhattbhavesh91]

github.com/bhattbhavesh91/GA_Sessions/blob/master/ga_dsmp_5jan2019/20_Stacking_Classifier.ipynb

[@MAK335]

2:27 500 or 5000?

[@bhattbhavesh91]

5000! Its a one take video, so some mistakes are bound to enter.