Hi Sir, It's a really good video. I have a suggestion: instead of loading data into Excel and then using it in Jupyter Notebook, we can directly connect to the database and use that view for our ML algorithms.
Hi, Thanks for a great suggestion. I did consider this earlier and used the pyodbc library to fetch data from server. However this method sometimes gives errors due to accessibility or drivers. So I didn't want the new learners to get discouraged by this, hence decided to keep it simple. Let me know if you have used any other library which makes the process simpler. Will be really happy to learn & use it in my next project. Thank you so much for watching! :)
@pivotalstats Sir, I have used 'from sqlalchemy import create_engine', but you are right; initially, during connection, it gives a really hard time due to drivers.
It is a great end to end project, from getting a understanding about how data is converted from a raw data to visually appealing dashboard. Thanks alot.👍🏻
kia baat ha sir g, you are amazing. thanks for this all. I believe you are the only one on TH-cam who provides detailed lectures with full access to data and lecture notes. You did this very professionally. can't be better than this. Just waiting for more and more. Could you also please make a video on how to develop websites and how to upload all the projects onto it, as you did for yourself. thanks for this amazing work.
Thank you so much :) Reading such nice comments really motivates me to bring more quality content. I don't know if web development related content will be correct for this channel, but I will surely plan a video on creating your portfolio page on github. Cheers!
Hello sir, how are you? We don't talk or chat, but we've learned a lot from your videos. We're very grateful for teaching us, sir. Hope you're doing well and staying healthy
Hi, The main reason is to improve performance because in a reference table we can remove duplicates and reduce the row count. Also in future if I decide to add more mapping columns then its easier to manage in a separate table.
Glad that it helped! Sure, I'm not very active on linkedin but I'm planning to post content there as well. You can follow my page below: www.linkedin.com/company/pivotalstats/
Sir, thank you for the video. However, I don’t understand the creation of that production table at all. Similarly, what is the downside of removing null values at the time of importing the table instead of the rigorous activity later?
Glad you liked the video! 1. Creating a prod table is considered as a best practice because it gives you the opportunity to revert back to the original data in case you make any errors without running the entire data load process again. 2. We can always do the data cleaning part in power bi, but the problem with that approach is that in case you are working with a team, and all of them are working on the same data. Then each of you would have to do the same cleaning process on each of your files. Also lets assume that after sometime you add a new cleaning step to your file then that steps will have to be implemented on all files again. Hope this helps!
@pivotalstats Thank you so much for responding promptly. That's awesome. Pardon me if am being slow here, however, regarding the first question, how that production table was created was fast so I didn't get it even though I understand the reasons for the creation considering the fact that am more used to PowerBi than SQL. Regarding the 2nd part, I meant the initial time you were importing the file into SQL you didn't ignore the null values but you later tried to remove them during the query. Is there any special reason to it?
Hi, Since this project covers almost all major aspects of Data Analytics, you can write SQL Database Management, Power BI Data Transformation & Visualization, Data Analysis & Random Forest Modelling.
Hi, Thanks for the wonderful learning video. I'm able to follow and re-create all the charts, I'm just encountering one error in the Churn Reason sheet is showing "The Total Churn" against each of the rows (1732) and also the total is showing as (1732). Please guide.
Hi, Check the table from which you are dragging the Churn Reason column into the visual. Most probably you are using the Prod_Services table. Instead fetch it from Prod_Churn, that will solve the issue. Hope this helps!
Thanks. I have kept it very light in this video but still since sql is an essential skill for anyone working with data, you can watch below sql bootcamp video th-cam.com/video/QDz8be0L2PQ/w-d-xo.html
I am getting the correct output but when I am going to my prediction data.xlsx file to see my result it is telling me Excel cannot open the file predictiondata.xlsx because the file or format is not valid
The error suggests that the Customer_Status column which is our Y variable (target variable), contains missing values. But in the original data there were no missing values in this column. Could you check if your vw_ChurnData contains null values? If it does then check the ETL process & Original Excel raw data file once more. Hope this helps!
hi, check out below link. It should help! community.fabric.microsoft.com/t5/Power-Query/Trying-to-connect-to-MY-SQL-from-Power-BI/td-p/195060#:~:text=Before%20you%20can%20connect%20to,MySQL%20in%20Power%20BI%20Desktop.
Sir , provide data file not working on SQL server , error coming. When 1 error is solved, then another error is coming. It is better for us if you provide all 4 data files link.
Hi, May be I did not understand the question, but the data link I provided contains only 1 data file (not 4) i.e. Customer_Data. If you are facing errors even after following all the import process I showed in the video. May I suggest you to import that data directly to Power BI and do the transformations like null removal in the Power Query editor, so that you dont have to waste time on SQL server troubleshooting. Hope this helps!
@@pivotalstats Sir, the data you provide ,i find multiple errors, 1. Data not loading in SQL server 2. I changed the data structure ,then it loaded same things happened.so i loaded it in my SQL. 3.But multiple things can't be done in my SQL, due data structure. So , sir if you provide Main data file , with working data file , 2 view files , then it will be good for me . Thank you for replying & Understanding the issue
@@SSMB-dw2zr Here you go, this link contains a excel file which has stg_Churn, prod_Churn, vw_Churn & vw_Join data. Cheers! e3da6ab4-ff6e-4f55-bfa1-a8fb6979d99b.usrfiles.com/archives/e3da6a_ab9ca4f21efb4e6a9c16a94583f064d9.zip
Sir that project is awesome mainly the machine learning portion could you boost yourself to make a more advanced project right now I am finding jobs and they want the best projects because I don't have any work experience. and also sir could you provide me your LinkedIn profile I will tag you with this project.
Hi Sir, It's a really good video. I have a suggestion: instead of loading data into Excel and then using it in Jupyter Notebook, we can directly connect to the database and use that view for our ML algorithms.
Hi, Thanks for a great suggestion. I did consider this earlier and used the pyodbc library to fetch data from server. However this method sometimes gives errors due to accessibility or drivers. So I didn't want the new learners to get discouraged by this, hence decided to keep it simple. Let me know if you have used any other library which makes the process simpler. Will be really happy to learn & use it in my next project. Thank you so much for watching! :)
@pivotalstats Sir, I have used 'from sqlalchemy import create_engine', but you are right; initially, during connection, it gives a really hard time due to drivers.
It is a great end to end project, from getting a understanding about how data is converted from a raw data to visually appealing dashboard.
Thanks alot.👍🏻
Welcome :)
One more nice project and concept of implement SQL + power bi as wel machine learning . thank you for nice video
You are welcome :)
Great. It is really very interesting project. I really learnt a lot from this.. Keep doing good work. Good luck..
Thanks a lot :)
kia baat ha sir g, you are amazing. thanks for this all. I believe you are the only one on TH-cam who provides detailed lectures with full access to data and lecture notes. You did this very professionally. can't be better than this. Just waiting for more and more.
Could you also please make a video on how to develop websites and how to upload all the projects onto it, as you did for yourself. thanks for this amazing work.
Thank you so much :) Reading such nice comments really motivates me to bring more quality content. I don't know if web development related content will be correct for this channel, but I will surely plan a video on creating your portfolio page on github. Cheers!
Really loved the video. Good way to bring everything under on roof. Looking forward to more such content.
Thanks a ton!
your channel is under rated sir you will achieve great success in future😊😊
Thank you so much 😀
Respect to the world of India
Cheers!
Hello sir, how are you? We don't talk or chat, but we've learned a lot from your videos. We're very grateful for teaching us, sir. Hope you're doing well and staying healthy
Hello, I'm doing very well. Thanks for asking 🙂. Really glad that my videos are helping. Cheers!
Wow... Just Wow
Thank you soo much for this wonderful project.....this means a lot
Thanks for watching, cheers!
This is a nice project. Well done.
Thank you very much!
Bro ure a legend!
Thanks for watching buddy 👍
Loved it like a adventure movie ❤
Thanks for watching :)
New to it great work 👍👍
Thanks !
Thank you sir ❤❤❤
Most welcome 🙂
This is awesome!
Thanks 🙂
Thanks Bro. one question, why did we create new reference table for Age group instead of creating new column in existing table?
Hi, The main reason is to improve performance because in a reference table we can remove duplicates and reduce the row count. Also in future if I decide to add more mapping columns then its easier to manage in a separate table.
@@pivotalstats great, thanks for the explanation
This is a perfect video and helpful thank you very much!!!! can i connect with you in linkedin sir ? thank you
Glad that it helped!
Sure, I'm not very active on linkedin but I'm planning to post content there as well. You can follow my page below:
www.linkedin.com/company/pivotalstats/
Sir, thank you for the video. However, I don’t understand the creation of that production table at all.
Similarly, what is the downside of removing null values at the time of importing the table instead of the rigorous activity later?
Glad you liked the video!
1. Creating a prod table is considered as a best practice because it gives you the opportunity to revert back to the original data in case you make any errors without running the entire data load process again.
2. We can always do the data cleaning part in power bi, but the problem with that approach is that in case you are working with a team, and all of them are working on the same data. Then each of you would have to do the same cleaning process on each of your files. Also lets assume that after sometime you add a new cleaning step to your file then that steps will have to be implemented on all files again. Hope this helps!
@pivotalstats Thank you so much for responding promptly. That's awesome. Pardon me if am being slow here, however, regarding the first question, how that production table was created was fast so I didn't get it even though I understand the reasons for the creation considering the fact that am more used to PowerBi than SQL.
Regarding the 2nd part, I meant the initial time you were importing the file into SQL you didn't ignore the null values but you later tried to remove them during the query. Is there any special reason to it?
What key points should i use to describe this project on my resume
Hi, Since this project covers almost all major aspects of Data Analytics, you can write SQL Database Management, Power BI Data Transformation & Visualization, Data Analysis & Random Forest Modelling.
Hi sir can u please make video on end to end project with large data set using direct query in power bi??
Sure ill plan something for future 👍
Hi, Thanks for the wonderful learning video. I'm able to follow and re-create all the charts, I'm just encountering one error in the Churn Reason sheet is showing "The Total Churn" against each of the rows (1732) and also the total is showing as (1732). Please guide.
Hi, Check the table from which you are dragging the Churn Reason column into the visual. Most probably you are using the Prod_Services table. Instead fetch it from Prod_Churn, that will solve the issue. Hope this helps!
Yes, that worked. Also when I changed the relationship between the tables to "both" it worked as well. Thanks for you quick response.
you could have easily created age group column in Calculations table rather than creating refrence table
Hi, Yes you are absolutely correct, we can create a calculated table as well using DAX here. Thanks for the feedback!
Awesome sir but i don't know much about SQL
Thanks. I have kept it very light in this video but still since sql is an essential skill for anyone working with data, you can watch below sql bootcamp video
th-cam.com/video/QDz8be0L2PQ/w-d-xo.html
I am not able to open the output of random forest algorithm into my excel file
Hi, Please check the file path that you have given. Also are you receiving any errors?
I am getting the correct output but when I am going to my prediction data.xlsx file to see my result it is telling me
Excel cannot open the file predictiondata.xlsx because the file or format is not valid
when i am trying to Train Random Forest Model getting error-ValueError: Input y contains NaN.
,how do i resolve this
The error suggests that the Customer_Status column which is our Y variable (target variable), contains missing values. But in the original data there were no missing values in this column. Could you check if your vw_ChurnData contains null values? If it does then check the ETL process & Original Excel raw data file once more. Hope this helps!
can we use mysql also? as ms sql is not there for mac
hi, check out below link. It should help!
community.fabric.microsoft.com/t5/Power-Query/Trying-to-connect-to-MY-SQL-from-Power-BI/td-p/195060#:~:text=Before%20you%20can%20connect%20to,MySQL%20in%20Power%20BI%20Desktop.
Sir , provide data file not working on SQL server , error coming. When 1 error is solved, then another error is coming. It is better for us if you provide all 4 data files link.
Hi, May be I did not understand the question, but the data link I provided contains only 1 data file (not 4) i.e. Customer_Data. If you are facing errors even after following all the import process I showed in the video. May I suggest you to import that data directly to Power BI and do the transformations like null removal in the Power Query editor, so that you dont have to waste time on SQL server troubleshooting. Hope this helps!
@@pivotalstats
Sir, the data you provide ,i find multiple errors,
1. Data not loading in SQL server
2. I changed the data structure ,then it loaded same things happened.so i loaded it in my SQL.
3.But multiple things can't be done in my SQL, due data structure.
So , sir if you provide
Main data file , with working data file , 2 view files , then it will be good for me .
Thank you for replying & Understanding the issue
@@SSMB-dw2zr Here you go, this link contains a excel file which has stg_Churn, prod_Churn, vw_Churn & vw_Join data. Cheers!
e3da6ab4-ff6e-4f55-bfa1-a8fb6979d99b.usrfiles.com/archives/e3da6a_ab9ca4f21efb4e6a9c16a94583f064d9.zip
@@pivotalstats thankyou sir
Sir that project is awesome mainly the machine learning portion could you boost yourself to make a more advanced project right now I am finding jobs and they want the best projects because I don't have any work experience. and also sir could you provide me your LinkedIn profile I will tag you with this project.
Hi, Sure I will definitely try to bring another project soon. And my linkedin profile is sajit-simon.
Sir your project is very awesome