In some literature infor gain is given by formula 1- weighted entropy . Whereas in some literature h(y) - weighted entropy. However for finding best split , both formulas work the same. As it is just finding the maximum. But is there any justification of formulas ? Please share your knowledge.
@@LearningMonkey I saw it on analyticsvidya.com May b it is approximation of parent entropy - weighted child entropy . As parent entropy is mostly high , they took it as 1
No suraj. When we find entropy. We get maximum value 1. In information gain we are taking entropy of a data subtracted with weighted entropy of a column split. Instead of weighted entropy. If you take sum of all entropies. There is a chance of going the sum value greater than 1. So that we take weighted entropy which is average of the entropies.
Sir , Clearly explained and understood the concept. Thank you So much Sir....
Thank you have a great learning
Very clear in explaining!
Sir how someone can explain it in this much easy way 😮
Have a great learning in CSE
Nicely explained, better explained than other tutorials
Thankyou
In some literature infor gain is given by formula 1- weighted entropy . Whereas in some literature h(y) - weighted entropy. However for finding best split , both formulas work the same. As it is just finding the maximum. But is there any justification of formulas ? Please share your knowledge.
Hi
Sklearn decision tree classifier class uses h(Y) - weighted entropy formulae.
Where u see the equation 1-we.
Share the link.
@@LearningMonkey I saw it on analyticsvidya.com
May b it is approximation of parent entropy - weighted child entropy .
As parent entropy is mostly high , they took it as 1
Yes may be.
Tq for sharing ur knowledge.
Have a great learning
@@LearningMonkey just a beginner...raw knowledge
@@shahnaz1981fat yes, as beginners we needed that only. perhaps you should see videos of nasa people.
Thank you
Have a great learning
Why we calculate weighted entropy
Can't we simply add the entropies???
Weighted entropy gives average.
If u simply add. The summation may be greater than 1
@@LearningMonkey Summation great than 1 is not good for best split??
No suraj.
When we find entropy.
We get maximum value 1.
In information gain we are taking entropy of a data subtracted with weighted entropy of a column split.
Instead of weighted entropy. If you take sum of all entropies.
There is a chance of going the sum value greater than 1.
So that we take weighted entropy which is average of the entropies.
@@LearningMonkey ok i understood
Thank you 😍