Code & Dataset for practice: github.com/pik1989/MLProject-ChurnPrediction Also watch, Churn Analysis dashboard creation using Power BI: Part 1: th-cam.com/video/Dryfm9VaEw4/w-d-xo.html Part 2: th-cam.com/video/mAPcW_YDUKQ/w-d-xo.html
This channel is one of the best I've come across for ML, Data Analysis and Python. The level of content uploaded here is so remarkable. Thank you so much Sir. ❤
I am at half the lecture and can not contain my gratitude: Totally awesome lecture and explanations! Please do other lectures related to ML- you have rare gift to impart understanding. Million Thanks! God Bless!
One of the best End to End ML project explanation and implementation I have seen till now.... Thank you @Satyajit for taking out your time and preparing such a wonderful video for us...
Hello, I wanted to thank you. I got the job as data analyst. In the technical interview, I was asked to do any dummy projects and walk through the process .I used this data and topic of churn analysis and did similar project like yours with the help of your code. I used Power for data visualization and used python for data modeling for the same data and was able to impress the interviewers . I also had the theoretical knowledge on what I was doing but I am not good at technical stuff because all I learnt in the school was to calculate eigen-value, PCA, manually, definition of p-value and regression analysis😂😂. I can understand what the code is doing but if I have to code,I will spent all day on youtube. Its my second day on job but I don't have proper understanding of relational database. They are using Powerbi and SSMS together for the daily reports and all. I need to sustain on the job😂😁 Please advise me with the link of videos if you have made on SQL,powerbi, and python on how they are working together.
Thank you for providing such a detailed and clear explanation for this data science project. I am learning data science using Gemini and your videos, and I am so happy to have found you as a resource.
Hi Satyajit, I believe performing resampling on entire data might be responsible for data leakage since some of the resampled examples will be the part of both training and testing datasets now contributing to a high F1 score.
@@mohitpandey5190 But my performance metrics dropped significantly after this, Came down to around 80% in almost all metrics. Do you have suggestions over other techniques that could work much better.
Finding telecom domain projects on youtube are very difficult. Thank you for sharing..Please share more end to end projects focused on telecom domain..
Please bring projects which can be showcased on CV. Like..customer segmentation, customer retention, customer call anlysis.It would be very helpful. Thanks for ur work!
Amazing work..clear explanation of EDA and explaining insights by analysing...Keep doing more end to end ML projects for us in different domains.. thank you
Yeah me too...could u please help..In Webpage i'm getting the error as internal server error and in anaconda prompt Value error as: X has 51 features,but RandomForestClassifier is expecting 50 features as input...
@@gauravmore8578 No..I'm not getting the output everytime...for most of the cases I'm getting Value error: X has 51 features, but randomforestclassifier expecting 50 has input...
Hey there love you teaching skills... I have a question that the EDA steps used over here can we use it as the base for different DS projects or there is something different
Thanks for the amazing End To End Explanation...... Sir can you please make a video playlist on Speech To Text Machine learning Model building from Scratch?
Hi Satyajit. .In addition to predicting whether a customer churned out or not, we can also predict when the customer is going to churn out for e.g. 6 months /9 months etc. How can we accomplish that?
There's a column called as tenure, so if you are testing the model on customer X and he's in the system for 24 months, just pass tenure as 30 for that customer and u will get the results for that customer if he's going to be churned in next 6 months or not.
I’m working on a project titled "Predictive Analytics for Customer Churn in E-commerce Using Machine Learning and Sentiment Analysis on Customer Reviews" for my master’s. I’ve explored many datasets, but I’m struggling to find one that has both transactional data and customer reviews from the same source. Most datasets either have one or the other. Could you please help me find a suitable dataset or guide me on how to approach this? Since my project integrates both traditional analysis and sentiment analysis, having data from a single source is crucial. Thanks in advance for your help!
Thanks a lot for this session Satyajit .... I have a doubt that how data scientists typically finalise a machine learning model for a given problem since there are many machine learning models available?
The model that predicts better on test data is fine tuned and pushed into production to get tested on live data, further the results are evaluated by sales and various teams, they give feedbacks whether the model is actually doing good or not, and then further remodelling is done, it's a long process
The error message is quite clear, you must be passing some value that is not present in original data, hence once you are doing encoding a new feature is getting generated hence you got 2 extra features
Hi Satyajit, Thank you for your detailed steps to implement End-to-End ML with deployment. I have few questions, can you please clarify? Please explain the usage of the following lines df_1=pd.read_csv("first_telc.csv") # how did you get this file "first_telc.csv? df_2 = pd.concat([df_1, new_df], ignore_index = True) single = model.predict(new_df__dummies.tail(1)) # as per this line, you are considering only the input which we enter from html so, df_1, df_2 steps may not be required. If I just process the input from html using pd.get_dummies and then do model.predict , this will be enough for prediction. Am I correct? Any how, Thank you once again for your contribution
Hi Satyajit, I found the reason for using 'first_telc.csv' and the above clarifications which I raised. I have figured it out, you have to use some sample data and then add your input data because, then only all combination of categorical values are covered and it will be converted into equal number of columns as compared with Analysis file using pd.get_dummies. To match 51 columns, you are loading that csv file. However to predict, you are using only the last row (tail) which is entered by user in UI. Thanks a lot.
You have label encoded the data before splitting the data into test and train that is why you are getting such high precision and recall. There is data leakage here.
Hi actually I am having doubt how to handle the imbalanced data set like having only two col x and y and y is having both Pos and negative values in it how to approach with it I tried transforming x to log x and many more but didn't worked
thank you for the tutorial, i have a question, when im running all your code, ValueError: X has 64 features, but DecisionTreeClassifier is expecting 45 features as input. can you explain it? thank you
Shouldn't be, can u check again, many people are using this, And it works fine. If u can't figure out, upload your code on GitHub, share me the link and tell me which line is it failing..
Project explanation is great, just one problem, when u applied smoteenn before splitting train and test data, u r getting 92% accuracy through wrong approach, on correcting this , accuracy comes down below 80%, if u could fix this issue.
having problem while running it, could you please help me out. while running app.py error occur in browser.... Internal Server Error The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
Very good explanantion but I have a doubt as in if we choose decision tree in this case is there a way to find out the root node? any code available to do so?
@@SatyajitPattnaik Thank you very much but I got it very small nodes I used fig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (5,5), dpi=300) tree.plot_tree(model_dt_smote,filled=True) and I am getting nodes that are not clearly visible any way to increase size of nodes and make it look clear? That would be great help!!
Hi, Every industry have predicting wheather a person churn or not churn how about making this project more realistic with prescriptive analytics by converting churn customer into non churn and if person is already a churn by providing him to good premium to convert churn customer into non churn customer..it will be helpfull to entire community..please make video on this
Good suggestion, but the part you explained is really difficult to implement as it's need cost analysis and ROI calculation, and it needs sales team involvement too, which is really difficult to show.
@@SatyajitPattnaik thanks for ur reply please can u make video on reinforcement learning on banking or insurance data it will helpfull for aspiring data scientist .hope that video soon 🎉
@@SatyajitPattnaik I have idea in customer retention in financial industry but i don't know how to implement reinforcement learning although it requires huge data ...my idea is to implement prescriptive analytics in churn prediction ..it needs normal any ensemble algorithm to predict churn or non churn customers then taking churn customer data we implement some prescriptive analytics by before customer going to churn we will recommend some premiums to them then using LSTM models and reinforcement learning and time series ..LSTM needs because we need to focus on customer past behaviour with the company and time series needs because it's customers travel how loyal they are with company and finally reinforcement learning needed because we need to train our model cantinously with the data then model learn it self for future data
I am getting error while implementing SMOTENN....when I am running it is giving me error 'nonetype' object has no attribute 'split'. Kindly help me to solve this error I am stucked
For deployment can you please explain what is in df_1 ? and why you concat with new_df to create df_2. I am working on a similar project with this data set.
Sir As i am from Pharmaceutical sales background,is it possible make a video on end to end Business Analytics project on pharmaceutical company used cases like ur very nice Telecom project,it would be very helpful for us.
A lot of things can be done in this model, this is not the most optimized model, like you said CV can be performed, other feature engineering steps plus hyperparameters Optimization can also be done.
@Satyajit Pattnaik Thanks a lot for helping out and providing the vedio for college project..but I'm getting an error at the last...after submitting the user values in webpage..I'm getting the error as internal server error and the same in Anaconda prompt..I'm getting the Value error : X has 51 features,but RandomForestClassifier is expecting 50 features as input...But I'm not getting the error always but mostly...for 1st input given in CSV file I'm getting the output but for 2nd input I'm getting the output...I followed the same code given in github...could u please help..
hi sir, i am anu, recently i joined in data analyst course, your video's are very help full for me i am facing issue on after installing masked rcnn then i fail to importing "from mrcnn import model as modellib " kindly resolve the issue, thank you
@@SatyajitPattnaik thanks for reply AttributeError Traceback (most recent call last) in () 1 from mrcnn.config import Config 2 from mrcnn import utils ----> 3 import mrcnn.model as modellib 4 from mrcnn import visualize 5 from mrcnn.model import log /usr/local/lib/python3.10/dist-packages/mrcnn/model.py in 253 254 --> 255 class ProposalLayer(KE.Layer): 256 """Receives anchor scores and selects a subset to pass as proposals 257 to the second stage. Filtering is done based on anchor scores and AttributeError: module 'keras.engine' has no attribute 'Layer' i search on stack overflow it tells about all instances keras application KE is changes to KL but i don't understand how to fix it
Hi Satyajit, thank you well-done project. I had a data science interview this week can I take this end-to-end project to present in front of the interviewer?
Very detailed explanation. Thanks a lot for the efforts in creating this. I have a query regarding selection of training dataset for a subscription based model. If I select historical data with churners in the last 3months, for example, would I need to restrict the subscription start date in the training dataset for creating the base data?
@@SatyajitPattnaik Thanks for replying, but my question was should I filter out the data by subscription start date too like only taking subscriptions for a particular time period? I am taking churned customers for last 3months.
You can read more about eager and lazy learning, the algo used here in this use case is probably DecisionTree if I recall correctly, and DT is basically a eager learner. So how the learning happens is something that is internal to the algorithm, to know more about it, you have to dig in to the model's base codes, here we are just implementing the algorithm and performing our predictions. I hope you got my point, for example, if u want to know why Decision Trees are called eager Learners, you will have to check how a decision tree is implemented and what's the code within the algos. A link which can help you: towardsdatascience.com/machine-learning-classifiers-a5cc4e1b0623 In case you still have the doubts, we can probably connect over LinkedIn.
After training the model , you have shown the accuracy on test data i.e 93% but how we can get churn/Not churn against that in test dataset, not through API but in the python only
Yes, we can get it, just run the entire test data in a loop, and append the prediction score in a new column, for ex: df["Prediction"] = model.predict_proba(X_test) Just giving you ideas..
@@SatyajitPattnaik Thanks for the clarification. I will try that . One more ques . - We have done the correlation part, so where we used that? those variables which were not affecting the target variable can be deleted before model building.. Please let me know if i m thinking in right way ? Or correct me if i m wrong. would be really helpful.
Thanks for the clarification. I will try that . Two more ques . - 1.We have done the correlation part, so where we used that? those variables which were not affecting the target variable can be deleted before model building.. Please let me know if i m thinking in right way ? Or correct me if i m wrong. would be really helpful. 2.max_depth=6, min_samples_leaf=8 , how we have decided this in this e.g. ?
Awesome! i enjoyed practicing the project along with listening your insights thanks a lot .I am statistician by virtue of my profession wish to connect with you how can i reach you!
@@sandeepthukral3018 yes, i have deployed similar application in 2 of my previous companies, however the complexity in real production level projects is immense
I noticed one thing. You applied SMOTEENN function on the train and test data. Applying SMOTEENN function on the test set and later calculating f1 score, recall etc from predictions made from the test set does not give the correct results. When I didnt apply SMOTEENN to the test set my results were actually very similar to the decision tree classifier score calculated on data without SMOTEENN function. What makes sense is applying SMOTEENN only to the train data and not test data.
hey man create an recommendations engine to the people not get churn recommended something so they got customers back N already have they get more interested
@@SatyajitPattnaik But I like your EDA approach. Kudos. I am a very new player in ML. Doing data science masters from Great learning. There are so much to learn from you. Will keep watching more case studies shared by you. Thank you for your amazing work.
In the telecom churn dataset, "Monthly Charges" refers to the total amount charged to the customer for their telecom services each month , while "Total Charges" refers to the total amount charged to the customer over the entire duration of their service with the telecom company .
Yes, I'm experience some difficulties with understanding the pronunciation. You would be so kind if allow subtitles in this video. Thank you, again for this great content!
Can use the model to predict data in PBI right? Then, create 1 table to have churn values customers in PBI. I don't use file Excel but use data from a database to test. Can you help me ?
Predicting on PBI isn’t possible, but predictions can be integrated with PBI to showcase the results. In PBI you can do time series forecasting but no ML predictions
@@SatyajitPattnaik I have finished building the customer churn prediction model and saved it in the pickle library. Now how can I use that model to predict data on PBI (can use integration or some other way) and show convenient results for calculating customer churn? I don't know what to do :( Please help me. Please. Thank you very much.
@@SatyajitPattnaik I can't get data from PBI to predict outside because data in database db2 :(. How to connect VS code with database db2 to create churn customer in data on the database? Then, I can use the data it use in PBI, right ?
@@VyĐặngThịTường-i3m use chatGPT on how to connect db2 from your python notebook, once u r able to create connections, start working on the predictive model , and then u can connect with power bi and show predictions on pbi
Your time related parameters will play a vital role here, let say, you want to test your existing customers, and latest tenure period of that customer is 27 months, if u want to test whether he stays after 6 months or not, u need to pass the tenure as 33 for that customer with other details..
Read this: glemaitre.github.io/imbalanced-learn/generated/imblearn.combine.SMOTEENN.html I will try to make a separate video explaining about SMOTEENN in depth..
@@akshajagarwal3519 well if u r using exact similar values then theres no way it Will fail, its tested and used by many, pls debug it, or else modify the input fields to be drop down values
sir i have a problem in model deployment in spyder. can you help me("UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte") this is the error anyways thanks for great and amazing video
After executing the below code X_resampled1, y_resampled1 = sm.fit_sample(x,y) getting this error ------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in ----> 1 X_resampled, y_resampled = sm.fit_sample(x,y) AttributeError: 'SMOTEENN' object has no attribute 'fit_sample' But if i am executing X_resampled1, y_resampled1 = sm.fit_resample(x,y) it is running without error in this case. changed Sample to resample
Code & Dataset for practice: github.com/pik1989/MLProject-ChurnPrediction
Also watch, Churn Analysis dashboard creation using Power BI:
Part 1: th-cam.com/video/Dryfm9VaEw4/w-d-xo.html
Part 2: th-cam.com/video/mAPcW_YDUKQ/w-d-xo.html
can i get ppt of this ?? or can you tell how w can make such a wonderfull ppt on telecom customer churn like this ??
Thanks a lot 🎉🎉
@@WOW-vn5jd Canva 😁
Didn't get the insights that you find from describe() method like 75% clients have 55 months tenure and 25% clients pay some USD
@@jahangagan5396 i thnk 25 is wrongly typed it should be 75
This channel is one of the best I've come across for ML, Data Analysis and Python. The level of content uploaded here is so remarkable. Thank you so much Sir. ❤
😊
The best end-to-end machine learning project. I have learnt a lot from this video. Thank you, sir.
Thanks Rishab
I am at half the lecture and can not contain my gratitude: Totally awesome lecture and explanations! Please do other lectures related to ML- you have rare gift to impart understanding. Million Thanks! God Bless!
One of the best End to End ML project explanation and implementation I have seen till now....
Thank you @Satyajit for taking out your time and preparing such a wonderful video for us...
Thanks, that keeps me motivated.
Highest way of teaching.. U covered each and every minute details in just one project.... 🙏thanks
Thanks a lot..
Wow, you really have put in great effort to make such a detailed video. Thanks a ton! Keep sharing your knowledge 🙂
Hello, I wanted to thank you. I got the job as data analyst. In the technical interview, I was asked to do any dummy projects and walk through the process .I used this data and topic of churn analysis and did similar project like yours with the help of your code.
I used Power for data visualization and used python for data modeling for the same data and was able to impress the interviewers . I also had the theoretical knowledge on what I was doing but I am not good at technical stuff because all I learnt in the school was to calculate eigen-value, PCA, manually, definition of p-value and regression analysis😂😂. I can understand what the code is doing but if I have to code,I will spent all day on youtube.
Its my second day on job but I don't have proper understanding of relational database. They are using Powerbi and SSMS together for the daily reports and all. I need to sustain on the job😂😁 Please advise me with the link of videos if you have made on SQL,powerbi, and python on how they are working together.
Check my playlists, there are videos on power bi and sql too (end to end)
@srijanakhatri7854 can you please guide me how to go on as i am someone who is starting from scratch
it felt so real life...like i am a da in an company and my manager is giving us walkthrough of the project...kudos
Hi, is this industry level project for data scientist?
Thank you for providing such a detailed and clear explanation for this data science project. I am learning data science using Gemini and your videos, and I am so happy to have found you as a resource.
Thank you so much Satyajit for this wonderful end-to-end full tutorial. That was really helpful.
Thanks Anirban
Wonderful explanation. Love how you get into detail wrt business understanding and coding
Really appreciate your effort best project explanation so far for me ! Thank you !
X_resampled, y_resampled = sm.fit_sample(x,y) become X_resampled, y_resampled = sm.fit_resample(x,y) thanks for the video
Hi Satyajit,
I believe performing resampling on entire data might be responsible for data leakage since some of the resampled examples will be the part of both training and testing datasets now contributing to a high F1 score.
Yes, that's a mistake I did here, you can do resampling on training data instead of the full data, I already acknowledged it in some other comment 👍
Noted @@SatyajitPattnaik, Make sense. Amazing work on churn modeling 🙌🏼
@@mohitpandey5190 But my performance metrics dropped significantly after this, Came down to around 80% in almost all metrics. Do you have suggestions over other techniques that could work much better.
Very great project and explanation! Thank you! ❤
Finding telecom domain projects on youtube are very difficult. Thank you for sharing..Please share more end to end projects focused on telecom domain..
Definitely
Please bring projects which can be showcased on CV. Like..customer segmentation, customer retention, customer call anlysis.It would be very helpful. Thanks for ur work!
@@riyatiwari4767 noted, pls wait for 2 weeks, you would see new Telecom projects 🔥🔥
Thank you Satyajit. The sessions was very informative and helpful!!
Thanks Sainath
What a Great project, thanks Sir
thank you for the tutorial you are really amazing and really have a gift to teach. Keep the good work up
Thanks for your kind words 👍
Big fan for ur efforts and explanation. I feel motivated from u
Thank you for this, I really learnt a lot
Great to hear this 😁
thank you so much for this video
what an amazing video..thanks for it
Amazing work..clear explanation of EDA and explaining insights by analysing...Keep doing more end to end ML projects for us in different domains.. thank you
Thanks Srikanth.
@@SatyajitPattnaik thanks sir and please do one more end to end ml project like this in a different domain.
@@depeshkumarmohanty9522 Already done, pls search "end to end satyajit" you will get more projects
Awesome video!
Very very helpful!!
Hi, i would want to know the procedure of what tweak in code for next 3 month churn prediction you were talking about
Hey, I'm getting this error
What am I doing wrong?
ValueError: X has 65 features, but RandomForestClassifier is expecting 50 features as input.
Yeah me too...could u please help..In Webpage i'm getting the error as internal server error and in anaconda prompt Value error as: X has 51 features,but RandomForestClassifier is expecting 50 features as input...
I'm not getting the error always but 3 in 5 times..for the 2nd input i'm getting the output but for 1st i'm getting the error
@@ittaboyinaharshithayadav7759 Did you manage to get output ?
@@gauravmore8578 were you able ro solve it..?
@@gauravmore8578 No..I'm not getting the output everytime...for most of the cases I'm getting Value error: X has 51 features, but randomforestclassifier expecting 50 has input...
This is a great project and I got to learn a lot from it. I wanted to ask will this be a good project for my final year dissertation?
Yes surely
'SMOTEENN' object has no attribute 'fit_sample'
and
name 'X_resampled' is not defined
can you explain the error and correct answer sir please
Sir please reply
its changed to "fit_resample", please use chatGPT for faster resolution :)
02:24:00 : pinned
Hey there love you teaching skills... I have a question that the EDA steps used over here can we use it as the base for different DS projects or there is something different
Same 😀
Thanks for the amazing End To End Explanation...... Sir can you please make a video playlist on Speech To Text Machine learning Model building from Scratch?
Sure, will do 👍
Hi Satyajit. .In addition to predicting whether a customer churned out or not, we can also predict when the customer is going to churn out for e.g. 6 months /9 months etc. How can we accomplish that?
There's a column called as tenure, so if you are testing the model on customer X and he's in the system for 24 months, just pass tenure as 30 for that customer and u will get the results for that customer if he's going to be churned in next 6 months or not.
I’m working on a project titled "Predictive Analytics for Customer Churn in E-commerce Using Machine Learning and Sentiment Analysis on Customer Reviews" for my master’s. I’ve explored many datasets, but I’m struggling to find one that has both transactional data and customer reviews from the same source. Most datasets either have one or the other.
Could you please help me find a suitable dataset or guide me on how to approach this? Since my project integrates both traditional analysis and sentiment analysis, having data from a single source is crucial.
Thanks in advance for your help!
Thanks a lot for this session Satyajit .... I have a doubt that how data scientists typically finalise a machine learning model for a given problem since there are many machine learning models available?
The model that predicts better on test data is fine tuned and pushed into production to get tested on live data, further the results are evaluated by sales and various teams, they give feedbacks whether the model is actually doing good or not, and then further remodelling is done, it's a long process
ValueError: X has 52 features, but DecisionTreeClassifier is expecting 50 features as input. How to Fix this problem , please?
The error message is quite clear, you must be passing some value that is not present in original data, hence once you are doing encoding a new feature is getting generated hence you got 2 extra features
Hi Satyajit, Thank you for your detailed steps to implement End-to-End ML with deployment. I have few questions, can you please clarify? Please explain the usage of the following lines
df_1=pd.read_csv("first_telc.csv") # how did you get this file "first_telc.csv?
df_2 = pd.concat([df_1, new_df], ignore_index = True)
single = model.predict(new_df__dummies.tail(1)) # as per this line, you are considering only the input which we enter from html so, df_1, df_2 steps may not be required. If I just process the input from html using pd.get_dummies and then do model.predict , this will be enough for prediction. Am I correct?
Any how, Thank you once again for your contribution
Hi Satyajit, I found the reason for using 'first_telc.csv' and the above clarifications which I raised. I have figured it out, you have to use some sample data and then add your input data because, then only all combination of categorical values are covered and it will be converted into equal number of columns as compared with Analysis file using pd.get_dummies. To match 51 columns, you are loading that csv file. However to predict, you are using only the last row (tail) which is entered by user in UI. Thanks a lot.
Okay a
@@DilipKumar-ww5si very helpfull, Thanks
You have label encoded the data before splitting the data into test and train that is why you are getting such high precision and recall. There is data leakage here.
But isn't that normal to do
Hi actually I am having doubt how to handle the imbalanced data set like having only two col x and y and y is having both Pos and negative values in it how to approach with it I tried transforming x to log x and many more but didn't worked
thank you for the tutorial, i have a question, when im running all your code, ValueError: X has 64 features, but DecisionTreeClassifier is expecting 45 features as input. can you explain it? thank you
Shouldn't be, can u check again, many people are using this, And it works fine.
If u can't figure out, upload your code on GitHub, share me the link and tell me which line is it failing..
Ok I will check it again, I hope running well like others, thank you very much
@@SatyajitPattnaik Your code is awsome, it's really work finally, thank you very much
@@iqballatifable thanks Iqbal, do share my channel with your friends & colleagues 🔥🔥
@@iqballatifable I too got same error how did you resolve it
Hi Satyajit. Nice End-to-End project. Could you let us know when we need to do Hyper-parameter tuning, and why is it required.
Once you build your prototype model, and want to take a further step in enhancing the model, you might have to do hyperparameters Optimization....
Hyper parameter tuning is required to reduce the variance.
With introducing small amount of bias we are able to reduce variance to a large extent.
@@sarfrazjaved330 Spot on 👍👍
Project explanation is great, just one problem, when u applied smoteenn before splitting train and test data, u r getting 92% accuracy through wrong approach, on correcting this , accuracy comes down below 80%, if u could fix this issue.
Lovely project..
having problem while running it, could you please help me out. while running app.py error occur in browser.... Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
Pls go through comments, many had this issue because of the input that they pass
Hi how tenure_group hot coded into dummies? and how it generated 0's and 1's
Very good explanantion but I have a doubt as in if we choose decision tree in this case is there a way to find out the root node? any code available to do so?
You can plot a tree, sklearn.tree.plot_tree
@@SatyajitPattnaik Thank you very much but I got it very small nodes I used
fig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (5,5), dpi=300)
tree.plot_tree(model_dt_smote,filled=True) and I am getting nodes that are not clearly visible any way to increase size of nodes and make it look clear? That would be great help!!
Hi,
Every industry have predicting wheather a person churn or not churn how about making this project more realistic with prescriptive analytics by converting churn customer into non churn and if person is already a churn by providing him to good premium to convert churn customer into non churn customer..it will be helpfull to entire community..please make video on this
Good suggestion, but the part you explained is really difficult to implement as it's need cost analysis and ROI calculation, and it needs sales team involvement too, which is really difficult to show.
@@SatyajitPattnaik thanks for ur reply please can u make video on reinforcement learning on banking or insurance data it will helpfull for aspiring data scientist .hope that video soon 🎉
@@VenuGopal-dr8ln Reinforcement learning is a huge topic, let me know a specific topic that you want, and I can make a video on it.
@@SatyajitPattnaik I have idea in customer retention in financial industry but i don't know how to implement reinforcement learning although it requires huge data ...my idea is to implement prescriptive analytics in churn prediction ..it needs normal any ensemble algorithm to predict churn or non churn customers then taking churn customer data we implement some prescriptive analytics by before customer going to churn we will recommend some premiums to them then using LSTM models and reinforcement learning and time series ..LSTM needs because we need to focus on customer past behaviour with the company and time series needs because it's customers travel how loyal they are with company and finally reinforcement learning needed because we need to train our model cantinously with the data then model learn it self for future data
I am getting error while implementing SMOTENN....when I am running it is giving me error 'nonetype' object has no attribute 'split'. Kindly help me to solve this error I am stucked
For deployment can you please explain what is in df_1 ? and why you concat with new_df to create df_2. I am working on a similar project with this data set.
When I check that df_1 'first_telc.csv'. It has 19 columns and 75 rows ? can you please explain where this comes from
and why you concat this with the data coming from the user ?
Sir As i am from Pharmaceutical sales background,is it possible make a video on end to end Business Analytics project on pharmaceutical company used cases like ur very nice Telecom project,it would be very helpful for us.
there are some healthcare case studies on my channel
why is cross validation not performed while building the model. Is there any criteria that you have considered to not use CV.
A lot of things can be done in this model, this is not the most optimized model, like you said CV can be performed, other feature engineering steps plus hyperparameters Optimization can also be done.
thank you so so much sir............
Welcome 🔥🔥
Sir, can you please tell what is the use of concatenating df1 with the input queries ?
To test the model on a new data, I am just capturing the input values in a new dataframe and then calling model predict on that dataframe
Thank you brother, You saved me from failing my assignment. You are really a Savior sent by god.
Thanks Jaideep ☺️😍
@Satyajit Pattnaik Thanks a lot for helping out and providing the vedio for college project..but I'm getting an error at the last...after submitting the user values in webpage..I'm getting the error as internal server error and the same in Anaconda prompt..I'm getting the Value error : X has 51 features,but RandomForestClassifier is expecting 50 features as input...But I'm not getting the error always but mostly...for 1st input given in CSV file I'm getting the output but for 2nd input I'm getting the output...I followed the same code given in github...could u please help..
Sorry,for 2nd input in CSV file I'm getting output,but for 1st input I'm getting Value Error and internal server error
@@harshithayadavittaboyina7390 Hey, did you find the solution as i am facing same kind of issue..
thank you sir.
welcome sir
hi sir, i am anu,
recently i joined in data analyst course, your video's are very help full for me
i am facing issue on
after installing masked rcnn then i fail to importing "from mrcnn import model as modellib "
kindly resolve the issue,
thank you
Whats the error?
@@SatyajitPattnaik thanks for reply
AttributeError Traceback (most recent call last)
in ()
1 from mrcnn.config import Config
2 from mrcnn import utils
----> 3 import mrcnn.model as modellib
4 from mrcnn import visualize
5 from mrcnn.model import log
/usr/local/lib/python3.10/dist-packages/mrcnn/model.py in
253
254
--> 255 class ProposalLayer(KE.Layer):
256 """Receives anchor scores and selects a subset to pass as proposals
257 to the second stage. Filtering is done based on anchor scores and
AttributeError: module 'keras.engine' has no attribute 'Layer'
i search on stack overflow it tells about all instances keras application KE is changes to KL
but i don't understand how to fix it
Hi Satyajit, thank you well-done project. I had a data science interview this week can I take this end-to-end project to present in front of the interviewer?
Ofcourse you can..
@@SatyajitPattnaik are you from odisha??
@@susantakumarsahoo1982 yes
Even am planning for the same. Thank you for the indepth explanation.
Very detailed explanation. Thanks a lot for the efforts in creating this. I have a query regarding selection of training dataset for a subscription based model. If I select historical data with churners in the last 3months, for example, would I need to restrict the subscription start date in the training dataset for creating the base data?
You can create tenure as a derived metrics based on expiration date - subscription date
@@SatyajitPattnaik Thanks for replying, but my question was should I filter out the data by subscription start date too like only taking subscriptions for a particular time period? I am taking churned customers for last 3months.
dear sir, what is first_telc.csv file which was in app.py , please explain
It's not used anywhere, pls ignore.
yes i also have the same doubt
How can we apply Artifical Intelligence , eager and lazy learning in it, finding common patterns in it
You can read more about eager and lazy learning, the algo used here in this use case is probably DecisionTree if I recall correctly, and DT is basically a eager learner.
So how the learning happens is something that is internal to the algorithm, to know more about it, you have to dig in to the model's base codes, here we are just implementing the algorithm and performing our predictions.
I hope you got my point, for example, if u want to know why Decision Trees are called eager Learners, you will have to check how a decision tree is implemented and what's the code within the algos.
A link which can help you: towardsdatascience.com/machine-learning-classifiers-a5cc4e1b0623
In case you still have the doubts, we can probably connect over LinkedIn.
After training the model , you have shown the accuracy on test data i.e 93% but how we can get churn/Not churn against that in test dataset, not through API but in the python only
Yes, we can get it, just run the entire test data in a loop, and append the prediction score in a new column, for ex: df["Prediction"] = model.predict_proba(X_test)
Just giving you ideas..
@@SatyajitPattnaik Thanks for the clarification. I will try that .
One more ques . - We have done the correlation part, so where we used that? those variables which were not affecting the target variable can be deleted before model building.. Please let me know if i m thinking in right way ? Or correct me if i m wrong. would be really helpful.
Thanks for the clarification. I will try that .
Two more ques . -
1.We have done the correlation part, so where we used that? those variables which were not affecting the target variable can be deleted before model building.. Please let me know if i m thinking in right way ? Or correct me if i m wrong. would be really helpful.
2.max_depth=6, min_samples_leaf=8 , how we have decided this in this e.g. ?
Awesome! i enjoyed practicing the project along with listening your insights thanks a lot .I am statistician by virtue of my profession wish to connect with you how can i reach you!
All Social media links are provided in my video descriptions, you can reach me via those links :)
oooo okay
Awesome content
Hi, is this industry level project for data scientist?
@@sandeepthukral3018 yes, i have deployed similar application in 2 of my previous companies, however the complexity in real production level projects is immense
I noticed one thing. You applied SMOTEENN function on the train and test data. Applying SMOTEENN function on the test set and later calculating f1 score, recall etc from predictions made from the test set does not give the correct results. When I didnt apply SMOTEENN to the test set my results were actually very similar to the decision tree classifier score calculated on data without SMOTEENN function. What makes sense is applying SMOTEENN only to the train data and not test data.
@@traeht yes thats a good observation, i did that mistake, balancing out should be done for training data only
hey man create an recommendations engine to the people not get churn recommended something so they got customers back N already have they get more interested
Sure, will build one
Hello sir,
Can I know the ML model of which you are using like linear regression, logistic regression, random forest
Sir, Can we get the PPT you have used to explain the model.
I don't share ppts for free, but you can create the ppts by yourself by taking ideas from mine.
Hi Satyajit, May I ask you to share the link for python code used in today’s session!!
I will upload it tomorrow first half for sure, it's a busy day for me today, thanks for understanding!!
github.com/pik1989/MLProject-ChurnPrediction
In my understanding, SMOTE should be applied on training data set...not on entire dataset.
@@saurabhagrawal3691 yes I already mentioned that in multiple separate comments, it was a mistake that i did
@@SatyajitPattnaik But I like your EDA approach. Kudos. I am a very new player in ML. Doing data science masters from Great learning. There are so much to learn from you. Will keep watching more case studies shared by you. Thank you for your amazing work.
@@SatyajitPattnaik Could you please share any detailed work of yours which include saving model and deployment using pickling and Flask. Many thanks.
can you give exact pre reqrusite for this projects ? so i learn and directly do this project
Python, Stats, Machine Learning is what you need to learn.
Hello sir can you tell me the source from where did you get the Dataset from?
It's a public dataset easily available on Kaggle and various other websites.
@@SatyajitPattnaik thankyou sir
Can any of you please tell me what is the difference between monthly charges and total charges? Thanks in advance.
In the telecom churn dataset, "Monthly Charges" refers to the total amount charged to the customer for their telecom services each month , while "Total Charges" refers to the total amount charged to the customer over the entire duration of their service with the telecom company .
@@SatyajitPattnaik Thanks a lot.
Hi..Satyajit ,can you make a video how anyone get fresher datascientist job in industry....and another question is,Are you from Odisha ?????
Sure i can make a video, and yes i am from Odisha
Many thanks
Can I know what are pre-requisites of this project?
Python and basics of ML should be enough
may I know what ML algorithm is used for this churn prediction sir?
Classification algorithms
does this come under predictive analytics sir?
@@tamizhmalar9843 yes
Asking if u. Have any playlist for the very first things of excel, Powerpoint or ms word. If u have, than pls provide me with the link.
At first, thank you a lot for this video! At second )))) - can you provide a subtitles for it?
Well, it's in English already, let me know if you're facing any difficulties
Yes, I'm experience some difficulties with understanding the pronunciation. You would be so kind if allow subtitles in this video. Thank you, again for this great content!
Hi how can we compare two float numbers in python with greater than operation I am getting error saying >= is not a function in numpy?
1:53:00 where is it
Could you please explain --telco_data.TotalCharges--...AS we dont know which columns contains the null data , then how should we direct write it ?
Can you point the time where you have this doubt
Can use the model to predict data in PBI right? Then, create 1 table to have churn values customers in PBI. I don't use file Excel but use data from a database to test. Can you help me ?
Predicting on PBI isn’t possible, but predictions can be integrated with PBI to showcase the results. In PBI you can do time series forecasting but no ML predictions
@@SatyajitPattnaik I have finished building the customer churn prediction model and saved it in the pickle library. Now how can I use that model to predict data on PBI (can use integration or some other way) and show convenient results for calculating customer churn? I don't know what to do :( Please help me. Please. Thank you very much.
@@VyĐặngThịTường-i3m Predictions can be done outside Power BI, Power BI is just a visualisation tool
@@SatyajitPattnaik I can't get data from PBI to predict outside because data in database db2 :(. How to connect VS code with database db2 to create churn customer in data on the database? Then, I can use the data it use in PBI, right ?
@@VyĐặngThịTường-i3m use chatGPT on how to connect db2 from your python notebook, once u r able to create connections, start working on the predictive model , and then u can connect with power bi and show predictions on pbi
Bro how to download your dataset bro
GitHub repository link is provided, pls fork the project on GitHub and use the data..
Hi ,How to create the churn data in retail industry.. ?
You have to read some papers about it.
Is it mandatory to be good enough to write code on our own, If we are looking for data engineer jobs? .........from a final year student
Not mandatory
how do we incorporate the 6 month or 3 month prediction into this?
Your time related parameters will play a vital role here, let say, you want to test your existing customers, and latest tenure period of that customer is 27 months, if u want to test whether he stays after 6 months or not, u need to pass the tenure as 33 for that customer with other details..
there are other columns that would change as well, like total charges, age etc etc..
I run project and i gave input to the fields,it does not showing any output
did you solve the issue?
smoothen function is showing error
Hi sir..can i use this in data analyst role resume?
Can, as Predictive Analytics is a part of Analytics :)
Upsampling + ENN
what does ENN stands for ?
Read this: glemaitre.github.io/imbalanced-learn/generated/imblearn.combine.SMOTEENN.html
I will try to make a separate video explaining about SMOTEENN in depth..
Can i add this project in my resume?
Sure, you can!!
Have u got placed ?
@@003_abhishekgupta9 Yes got placed, didn't added this in my resume, but learned a lot from this.
Sir,
If we use KNN, Random Forest, SVM, Decision Trees, Linear regression. Then which one would be the most accurate model?
Its very difficult to say, we should always try all the models and compare them
@@SatyajitPattnaik Okay sir. Thank you so much for the quick reply!
I am getting Internal server error when i click on submit on the webpage , kindly help ASAP
So your values arent matching with the expected values
@@SatyajitPattnaik I am inputting the values directly from the dataset itself , can you provide with the values to be entered.
@@akshajagarwal3519 well if u r using exact similar values then theres no way it Will fail, its tested and used by many, pls debug it, or else modify the input fields to be drop down values
@@SatyajitPattnaik Thank You
How did you fix this error??
how to get the customer chrun dataset
Checj video description
sir i have a problem in model deployment in spyder.
can you help me("UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte") this is the error
anyways thanks for great and amazing video
Its an encoding error, spot the error in the exact line, it must be failing while you are reading the file
After executing the below code
X_resampled1, y_resampled1 = sm.fit_sample(x,y)
getting this error
-------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
in
----> 1 X_resampled, y_resampled = sm.fit_sample(x,y)
AttributeError: 'SMOTEENN' object has no attribute 'fit_sample'
But if i am executing
X_resampled1, y_resampled1 = sm.fit_resample(x,y)
it is running without error in this case. changed Sample to resample
I will have to check that, thanks for pointing out..
Yeah i am also getting the same problem. My accuracy is also not coming good . After SMOTEEN its coming 80%