Decision Tree Pruning explained (Pre-Pruning and Post-Pruning)

In this video, we are going to cover how decision tree pruning works. Hereby, we are first going to answer the question why we even need to prune trees. Then, we will go over two pre-pruning techniques. And finally, we will see how post-pruning works.
Links:
- Corresponding blog post: www.sebastian-mantey.com/theo...
- Post-Pruning from Scratch: • Post-Pruning from Scra...
- Decision Tree Algorithm explained: Intuition • What is Machine Learni...
- Decision Tree Algorithm explained: Entropy • What is Machine Learni...
- Decision Tree Algorithm explained: Regression • Coding a Decision Tree...
Timestamps:
00:00 - Why do we need to prune trees?
01:57 - Overfitting example
04:44 - Pre-Pruning: Min-Samples approach
07:08 - Pre-Pruning: Max-Depth approach
10:18 - Post-Pruning

[@AndrewCodeDev]

Thank you very much for making these tutorials. Your visual presentation and general descriptions are great. I'll be watching out for future content!

[@SebastianMantey]

Glad you like them!

[@mansoorbaig9232]

Great blog post Sebastian. I am glad I figured this.

[@izb1275]

Thanks for the tutorial finally I understand pre-pruning and post-pruning!

[@wennie2939]

The BEST explanation and code EVER! Thank you so much, Sebastian!

[@SebastianMantey]

Glad it was helpful!

[@redforestx7371]

SUPERB EXPLANATION! THANK YOU!

[@gautamdawar5067]

Great tutorial and so structured! Amazing!

[@SebastianMantey]

Thanks so much!

[@minakshimathpal8698]

thank you for this video

[@eladiomendez8226]

Great explanation!

[@AA-yk8zi]

Thank you!!

[@rohan_b_ra]

nice explanation ..thanks!!!

[@maximebelloable]

Thank you so much! Great tutorial, it really helped me out for an exam

[@SebastianMantey]

Great to hear!

[@xiujiesong3095]

brilliant!

[@madhurir9646]

Very Informative video. Thank you for sharing it helped to solve my machine learning assignment. Waiting for more conceptual videos.

[@SebastianMantey]

Glad it was helpful!

[@syasyazuhaimi]

may i know the difference between testing data and validation data?

[@AkshayRaman97]

Great explanation! Earned a sub

[@SebastianMantey]

Thanks! I appreciate it!

[@juanete69]

Do you do it with cross-validation?
How? What happens if at each k-fold you get a different model?

[@SebastianMantey]

I’m assuming that you are referring to post-pruning:
In this video, I just focus on the most basic use case of post-pruning where you build the tree with the training data, prune it with the validation data and then test it with the testing data.
K-fold cross-validation is another technique on its own. It doesn’t really have something specifically to do with post-pruning. However, I think, you could also use it with post-pruning if you wanted to.

[@hopelesssuprem1867]

could you pls explain what type of pruning is it i.e. is it cost complexity pruning like in CART or something another and why did you decide to use this method?

[@SebastianMantey]

I am assuming you are referring to post-pruning: As I mention at 14:44, the process is called “Reduced Error Pruning”. And I used it simply because that’s the process that was described in the book I was using, namely “Fundamentals of Machine Learning for predictive data analytics”.

[@hopelesssuprem1867]

@@SebastianMantey oo, thanks. Now I've understood everything.