1. Programming Language (Python, Java, Scala) 2. SQL 3. Basic Linux Commands 4. Data Modelling 5. Big Data 6. Spark and/or Kafka 7. Data Orchestration (Airflow) 8. Cloud
Thank you Darshil only a self-taught Data Engineer can understand the struggle of finding suitable sources for learning. There is so much information out there so many roadmaps to follow not saying that those roadmaps are bad, but I have experienced that it is very easy for a fresher to get lost in this loop of roadmaps. I will definitely follow this roadmap because first of all, it is free (LOL) and second it is from you a self-taught successful freelance Data Engineer. Cheers Mate :)
Wonderful Darshil.....This is great stuff actually. I think you are making someone's life so easy with your readysetgo document. God bless you and your family.
Darshil please make videos of python interview questions for data engineer. Like sql questions that you make earlier. I believe that will really help full for all.
Roadmap: 1. Research on what is DE. - Videos recommended by darshil. 2. Basic fundamentals, playlist by Harvard, 07 videos. 3. Programming Language: Python. 4. SQL, most important skill. - Hacker Rank for practice. - Ankit Bansal for Interview on SQL 5. Linux. - Basic commands. 6. Projects. - Data model, very important. 1-2 months. 7. Big data. - Pipelines - File formats. - OLAP Vs OLDP. Etc. 8. Simplilearn, course. Theory as of now. 9. Data warehouses. 10. Practical. - Batch processing and stream processing. (SPARK and KAFKA) 11. Data orchestration - Airflow. Tweepy and Pandas, libraries. 12. Cloud Computing. - AWS or Azure (Same fundamentals) - ETL Pipelines in AWS - Other projects.
Bro I am bsc cs graduated student but an average student will data engineering will be easy for me or should I complete first msc cs and then go for it what will be better pls reply 🤞
I really want to learn data engineer so badly and i have no background in program so this video really help me I will start to learn for one hour everday and after 5 days i will make 2 hour and so on Thank you very much 💜
Hello brother, I am from Infosys Limited. I am working on a BI/DMDW project as an ETL developer. We are using information cloud (IICS) in our project. In my free time , we have a learning platform where i upskill myself in the domain of data engineering. Like doing Microsoft Certified Data Engineer Certification. Learning real time data processing etc. And i will complete two personal projects after my learning will complete. Is it fine to crack another company interview?. Having 2.5 years of experience in Infosys in the domain of ETL and switch into Data Engineer
Hey, Currently am responsible for maintaining and designing the Data pipeline for end to end data flow. Am pretty much interested to learn and grow in the same field ie., Data engineering. So doing Masters in Data engineering is a good option..??
Can u please make a detailed video like this for the roadmap of data scientist skills choosing the free courses on yt That would be of really grt help 🙏 and even answer that can I become directly a data scientist as a fresher
I'm in love with this content. I recently read a similar book, and I was completely in love with it. "Mastering AWS: A Software Engineers Guide" by Nathan Vale
Thanks a lot for this informative roadmap for data engg. Would you pls come up with some information regarding how to get clients/projects in this segment as a freelancer.. (as I have no intention to land-up to any job, so it is important to know, even before I start learning, if it is possible to grab freelancer projects in data engineering). Thanks in advance for your help...
Hi, I have 3.5 years of manual testing experience and now I want to switch for good career path which will be in demand and useful for me. Please give your suggestions, I am skeptical at the moment what should I learn. Whether Data science would be good career path or not.
Basically your Data Engineering skillset looks like this: RDBMS Database + NOSQL + Datawarehousing/Data Lakes + Traditional ETL/Custom ETL (You can learn both) + Python/Scala/Java + Distributed Computing + Visualization Tools Tools to learn (One of your choice): RDBMS Database - Oracle, MYSQL, Postgre SQL, MS SQL etc NOSQL - MongoDB, Dynamo DB etc Datawareshousing/Data Lakes - Includes concepts about data architecture. Snowflake, AWS Redshift, Bigquery, Synapse Analytics are some examples. Traditional ETL - Informatica, AWS Glue, Matillion, Talend, Azure Data Factory etc., Custom ETL - This is combination of a data orchestration tool and your python data transformation libraries. Here you write custom code for transformations using python. Airflow for orchestration, Python and Pandas for Transformations. Distributed Computing - Spark and Hadoop (Not complete alternatives of each other) Visualization Tools - Tableau, Power BI, Looker etc., Keep adding these skills with the tools of your choice. If you ask me RDBMS + Datawarehousing + ETL are the must have and should be started with.
bhaiya data engineer m fresher k liye itni kaam job kyu hoti h or isse career start krna jyd shi h y phle frontend s suru krkw bad m isme shift hona ??
if you are from india, i would suggest you to consider CSE, because in most colleges that provide these specialized courses mostly teach subjects of CSE.
Darshil I am a food technologist and having 4 years in food industry but now want to shift in data field should I learn on my own or else to take any course as I am having doubt after learning skills will I get shortlist with food tech background
@@dkboyz actually I am also want to start but There is so many things to become data engineer. I dont know whether i will get any interview call or not
Great video sir... Sir I'm from a bsc physics background and i want to switch my career to data engineering... is it possible for me to become a data engineer without prior knowledge in all data engineering field?
They both have lot of overlap skills but have similar path, in data science you might have to learn extra tools and skills such as Machine Learning and modelling I have Data Science roadmap th-cam.com/video/kU58_NaEjCk/w-d-xo.html and th-cam.com/video/fzyv5wxAeG8/w-d-xo.html
These things can be learned when you need them, these are no mandatory at the start! If you get a job and there let's say you need to hit some APIs then you can learn then and there, you don't need to master API at this moment
Web 3 and Data Engineering are different! Data Engineering or in general data market won't go for atleast 10-15 years or more W3 will take time to adapt, people are still researching and working on it's applications
![🔴 ฟุตบอลแชมป์กีฬา 7HD แชมเปียน คัพ 2024 [รอบรองชนะเลิศ]](http://i.ytimg.com/vi/zPm8ClLVhoE/mqdefault.jpg)
1000 Likes and I’ll upload next project video soon❤️
DAMN! I wish I could give you millions of like ❤️❤️❤️
You have covered querys of every beginner who know nothing about DE. LOVE YOU DUDE ❤️✨
i wish i could like more than once!
1. Programming Language (Python, Java, Scala)
2. SQL
3. Basic Linux Commands
4. Data Modelling
5. Big Data
6. Spark and/or Kafka
7. Data Orchestration (Airflow)
8. Cloud
Thank you for posting this
Thank you Darshil only a self-taught Data Engineer can understand the struggle of finding suitable sources for learning. There is so much information out there so many roadmaps to follow not saying that those roadmaps are bad, but I have experienced that it is very easy for a fresher to get lost in this loop of roadmaps. I will definitely follow this roadmap because first of all, it is free (LOL) and second it is from you a self-taught successful freelance Data Engineer.
Cheers Mate :)
Dude thank you sooooooo much you have no idea how your channel helped me, i don't know what/how else I can do to thank you enough. 😭
Wonderful Darshil.....This is great stuff actually. I think you are making someone's life so easy with your readysetgo document. God bless you and your family.
Sending your way a warehouse full of gratitude. Comprehensive.
Thank you Saurabh
Tons of valuable information in this video. Thank you Darshil ❤
Completely ❤️ it. Thanks 🙏 for creating such a great Roadmap.
You are welcome
Thank you so much. Darshil
for sharing Road Map and free resources to learn Data Engineer
Darshil please make videos of python interview questions for data engineer. Like sql questions that you make earlier.
I believe that will really help full for all.
I’ll try my best Tejas
@@DarshilParmar thank you
Yeah darshil It will be more helpful if you make a video on this
bro , i need help. i know pyhton , sql ,dsa
now what to do next in data engineering roadmap
Just found your channel!
Crazy content
Keep going!🎉
Best Video that I found
That you youtube for recommending this amazing channel
Roadmap:
1. Research on what is DE.
- Videos recommended by darshil.
2. Basic fundamentals, playlist by Harvard, 07 videos.
3. Programming Language: Python.
4. SQL, most important skill.
- Hacker Rank for practice.
- Ankit Bansal for Interview on SQL
5. Linux.
- Basic commands.
6. Projects.
- Data model, very important.
1-2 months.
7. Big data.
- Pipelines
- File formats.
- OLAP Vs OLDP.
Etc.
8. Simplilearn, course.
Theory as of now.
9. Data warehouses.
10. Practical.
- Batch processing and stream processing.
(SPARK and KAFKA)
11. Data orchestration
- Airflow.
Tweepy and Pandas, libraries.
12. Cloud Computing.
- AWS or Azure
(Same fundamentals)
- ETL Pipelines in AWS
- Other projects.
Bro I am bsc cs graduated student but an average student will data engineering will be easy for me or should I complete first msc cs and then go for it what will be better pls reply 🤞
Hey darshil, you are a genuine guy. Just that.
I really want to learn data engineer so badly and i have no background in program so this video really help me
I will start to learn for one hour everday and after 5 days i will make 2 hour and so on
Thank you very much 💜
Awsome Content,Amazing Road map==> for DE++ Free On You Tube.....Thanks a ton brother....Keep up the spirit....Respect from Bangalore....
Thanks thanks and a ton of thanks,this one is a much needed one.
My goodness gracious. This video was great! Only proble was to find out how much more I still need to study if I want to become a data engineer 😂
Have you finished it?
thank you, very valuable infirmation
fabulous information. i appreciate your efforts bro
Very Helpful Content, thanks!
Lol, The Seattle Guy recommends you and you recommend him. 😅🎉
Heartily thanks Darshil✨
This ll helps us alot..
Hi darshil , I am also from maharastra ❤️
Kay चाललं भावा मस्त मजेत ना .. 😀
Aurangabad ❤️.
👍👍
Majet bhava
@@DarshilParmar where do you live ?
Thanks a lot, Darshil😇
Thank you for this video can you make videos on GCP Data Engineering projects
One of the best video for Data engineers
Great! This video.
Thanks a lot 🙂🙂 for the roadmap.
Thank you for your instructions
Best content ever ❤
Motivated!!✌
This is GOLD.
I like timestamp 0:03 😀
thanks a lot!
This is what I need. Done like your video
You are welcome
is it i5 8GB ram is enough for HADOOP,MAPREDUCE,HIVE,HBASE,OOZIE,SQOOP,JAVA?
Thank you ❤️
Thanks a lot sir ♥
Thank you brother
Welcome
thanks for making this video... really helps a lot for noob data engineers like me... 👍
You are welcome!
Video is very helpful bhaiyya
i see datavidhya is no more available, can you help to provide valid links, thanks
Hello brother,
I am from Infosys Limited. I am working on a BI/DMDW project as an ETL developer. We are using information cloud (IICS) in our project. In my free time , we have a learning platform where i upskill myself in the domain of data engineering. Like doing Microsoft Certified Data Engineer Certification. Learning real time data processing etc. And i will complete two personal projects after my learning will complete. Is it fine to crack another company interview?. Having 2.5 years of experience in Infosys in the domain of ETL and switch into Data Engineer
Yes it’s completely fine to crack other companies while you are learning new skills
@@DarshilParmar Thanks a lot bro. and thanks for giving reply.
Hi bro can contact u ...I need some guidance..it's about how to do project on my own
When we're learn data engineer and also need to learn pyspark, java... Etc
Can you tell which certifications really helpful to give more weightage to the profile?
Do it for data science please 🙏
Hey, Currently am responsible for maintaining and designing the Data pipeline for end to end data flow. Am pretty much interested to learn and grow in the same field ie., Data engineering. So doing Masters in Data engineering is a good option..??
Thanks
Darshil can you make for data science road map too..
Can u please make a detailed video like this for the roadmap of data scientist skills choosing the free courses on yt
That would be of really grt help 🙏 and even answer that can I become directly a data scientist as a fresher
I already have two videos on this
th-cam.com/video/kU58_NaEjCk/w-d-xo.html
and
th-cam.com/video/fzyv5wxAeG8/w-d-xo.html
Loved The Video
Thank you!!!
I'm in love with this content. I recently read a similar book, and I was completely in love with it. "Mastering AWS: A Software Engineers Guide" by Nathan Vale
Hi Darshil please make the same in Hindi language so it easy to understand ...so please start making in Hindi language as well..
Thanks a lot for this informative roadmap for data engg. Would you pls come up with some information regarding how to get clients/projects in this segment as a freelancer.. (as I have no intention to land-up to any job, so it is important to know, even before I start learning, if it is possible to grab freelancer projects in data engineering). Thanks in advance for your help...
hey, was wondering if you got successful after an year ..
Do we need this many learning’s , how about start data engineering career from learn Databricks and ADF
Where you provided link for all that spark and Kafka
Hi, I have 3.5 years of manual testing experience and now I want to switch for good career path which will be in demand and useful for me.
Please give your suggestions, I am skeptical at the moment what should I learn.
Whether Data science would be good career path or not.
what u chose?
Thank you so much bhai....
You are welcome bhai
Basically your Data Engineering skillset looks like this:
RDBMS Database + NOSQL + Datawarehousing/Data Lakes + Traditional ETL/Custom ETL (You can learn both) + Python/Scala/Java + Distributed Computing + Visualization Tools
Tools to learn (One of your choice):
RDBMS Database - Oracle, MYSQL, Postgre SQL, MS SQL etc
NOSQL - MongoDB, Dynamo DB etc
Datawareshousing/Data Lakes - Includes concepts about data architecture. Snowflake, AWS Redshift, Bigquery, Synapse Analytics are some examples.
Traditional ETL - Informatica, AWS Glue, Matillion, Talend, Azure Data Factory etc.,
Custom ETL - This is combination of a data orchestration tool and your python data transformation libraries. Here you write custom code for transformations using python. Airflow for orchestration, Python and Pandas for Transformations.
Distributed Computing - Spark and Hadoop (Not complete alternatives of each other)
Visualization Tools - Tableau, Power BI, Looker etc.,
Keep adding these skills with the tools of your choice. If you ask me RDBMS + Datawarehousing + ETL are the must have and should be started with.
Hello Darshil when you have youre live session..?
Soon…
Thanks bro
You are welcome Pratik
Will ChatGPT 4 replace data engineers, is it worthwhile to learn that? Thanks.
I have video on this please watch it
bhaiya data engineer m fresher k liye itni kaam job kyu hoti h or isse career start krna jyd shi h y phle frontend s suru krkw bad m isme shift hona ??
Apki new video he resources ki paid
Ki un dono me se konsi better he
Gcp data engineering exam ko clear karne ke liye best resource bata de bahii
Ya azure
Kya pick Karu gcp vs azure
Hi darsheel......i m a data scientist .......n want to be a permanent freelancer ......plz help....
@Darshil Parmar Thanks for this amazing content. It's really helpful and please keep providing this type of contents. It helps a lot. 🙏🙏
I’ll try my best
Just be good in logical . Rest is easy
Hi can you please make video on best courses from udemy for data engineer
I already have it - th-cam.com/video/sGPWVYriqW4/w-d-xo.html
Hi I am sql support can i move to data engineering?
whet degree do you consider is better for a data Engineer position , data science Bachellor degree or a software Engineer degree ?
if you are from india, i would suggest you to consider CSE, because in most colleges that provide these specialized courses mostly teach subjects of CSE.
Darshil I am a food technologist and having 4 years in food industry but now want to shift in data field should I learn on my own or else to take any course as I am having doubt after learning skills will I get shortlist with food tech background
have u got data job?
@@Somnath-je9nd Nai bhai I am still in learning phase
@@dkboyz bhai have u given any interview till date?
@@dkboyz actually I am also want to start but There is so many things to become data engineer. I dont know whether i will get any interview call or not
Great video sir...
Sir I'm from a bsc physics background and i want to switch my career to data engineering... is it possible for me to become a data engineer without prior knowledge in all data engineering field?
Yes, you can switch to Data Engineering just by following this roadmap
@@DarshilParmar thanku 😄
@@nakulmohan7263 have u switch into data?
Q) Does data analytics / Data scientist have same roadmap ?? There are alots of roadmap uploaded on various platform can anyone suggest ??
They both have lot of overlap skills but have similar path, in data science you might have to learn extra tools and skills such as Machine Learning and modelling
I have Data Science roadmap
th-cam.com/video/kU58_NaEjCk/w-d-xo.html
and
th-cam.com/video/fzyv5wxAeG8/w-d-xo.html
@@DarshilParmar thank you 🙏
♥️♥️♥️♥️
Is Dsa also necessary for data engineer role?
Yes
Hi Darshil could you please share the sheet link for reference Thank You In Adavance.
heyy is api not included in this path? or it is not important?
These things can be learned when you need them, these are no mandatory at the start!
If you get a job and there let's say you need to hit some APIs then you can learn then and there, you don't need to master API at this moment
hello Darshil ! i am from pune :D
is there any way to get in touch with you ?
I am soon going to open 1-1 session for this
where are the links that you described in the video
Description
@@DarshilParmar can you also do similar vedio of data science course step.by step
@@MrTechGuy9999 I already have it
th-cam.com/video/fzyv5wxAeG8/w-d-xo.html
Could you tell what will be the job of data engineering in web 3.0
Web 3 and Data Engineering are different!
Data Engineering or in general data market won't go for atleast 10-15 years or more
W3 will take time to adapt, people are still researching and working on it's applications
Is this sufficient bro 😊 can u say im trusting like footprints as fresher ❤ can u sah
Yes 100%
Tku
What about etl tool bro can u say it is important like informatica
drive pdf link??
Extract transform load?
After so long.
Shivansh thanks for support from the start
@@DarshilParmar ❤️🙌
True that even paid service also waste of money and time
Hey much needed video. I have fallen for you for selflessly sharing things , after watching all your video.Will you marry me 🙃!!!!
link does not work
It is working
Python ops imp?
you can enroll free CS50 course in edX
Yes it is but this series I mentioned is latest, not sure if it’s available on Edx
You are a youtuber and not a data engineer. Period.
I can be both! On weekday I work on my projects and on weekends I do my TH-cam stuffs to share knowledge and experience to other people
No se le entiende un pingo!
bhai agar apka course khreed lun 5000 wala usme sb milega????????????😝😝😝😝
Thanks a lot...!
You are welcome