If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k
I like how you mentioned Docker and Kubernetes. By the way, I've heard that Resubot can help streamline the resume creation process, which is pretty useful for anyone looking to transition into a new role.
I've just done the same thing. I've spent years being a solution designer / data analyst and I've just switched into data engineering as a senior technical specialist with no experience in actual hands on data engineering but I've got experience in designing data warehouses and ETL solutions.
I transition from a cook to a Integration Developer 2 years ago and now I am taking a Master in Computer Science while working, hoping that 3 years from now to get into Data Engineering,
If this isnt proof that my phone is always listening. Last night my husband was telling me how he (manager of data analytics) is being moved under the programming department and being made data engineer. This popped up in my suggested today lol
So lucky to have found you. Doing my masters in data science after being a data analyst for about a year. Didn’t even start the vid I can tell this guy has real wisdom
I just got hired as a Cx (Customer Experience) Quality Analyst as my first job. With this video, you helped me a lot on how to start my path toward a Data Engineer role, thanks.
Hi, im not there yet, but i recently got promoted to a Sr. Continuous Improvement Analyst position. Im very happy on where i am, but im still pushing myself to my goal.
Great video. I've been in my first "Data Analyst" contracted role as an Enterprise Analytics Analyst & Data Product Owner for 15 months now and while the salary of $94k is decent I'd love to transition to the more rewarding DE career path. I currently delegate work to 2 engineers with different specialties and am hoping to shadow them more often to see some of the DE processes carried out in a real business scenario. Thanks for the tips!
@@abhisekgupta5381 A full-stack developer knows how to get the code to interact with the database and it's data. They know how to write the SQL queries needed to get the data the applications need. And if it's a new application, in many instances they also help design and build most of the database objects as well. Basically, by being "full stack" it implies detailed knowledge of the database tier of the system.
on my journey from DA to DE. Taking any opportunity to do ETL's, models, automatizations that I can find. Unfortunately, in my department our tech stack is so poor, that I realized that I'm never going to transit to DE internally and the DE work I'm doing not valued as much as of good DA or BA.
I'm taking the IBM Data Engineering certificate. I'm blowing through the intro courses because I already have those skills down. It's the warehousing, Spark, and so on that I'm taking that course for.
@Hamza Qahoush I'm also taking this certification since it's free through my university. So far I have only taken the first few classes on Python, SQL, and relational databases. The material is not very thorough or in-depth. It introduces a topic, guides you through a few simple exercises, and moves on to another topic. Unless you can get it for free, I would recommend skipping it and looking for free resources, then building your own projects.
I've noticed that having a solid resume is key for landing interviews in this field. It seems like an AI-driven approach might help refine the process.
Amazing video. I’m still in my undergrad majoring in CS but I on following this exact path. It’s my dream job to be a freelance data engineer/analyst for small businesses.
Hi Ben, You mentioned in the video that you linked the interview test prep? I don't think I see it in the drop down. Would be very interested in taking a look at it - if available? thanks!
The image is here res.cloudinary.com/practicaldev/image/fetch/s--cnXIakxd--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/github.com/datastacktv/data-engineer-roadmap/raw/master/img/roadmap.png
Thank you for taking such a positive, imposter-syndrome-fighting stance on becoming a data engineer. Sometimes the bar to land a tech role feels so high!!
Have you checked out the data engineering zoomcamp and if so, is it worth the time? Seems pretty popular and results in a personal project with decent resources to pull from. Currently in a hybrid ETL and Support role where I am trying to work in the DW/DE and hoping I can get some more hands on experience.
Great video. I'm currently in the process of becoming a data analyst but my end goal is to become a data engineer. Do you recommend any certifications to have that will help gain knowledge and experience I was looking at some of the Microsoft certifications for data engineers.
Hey, I have gained some data engineering skills and have done end-to-end ELT. I am looking for some open-source projects to contribute to but to my shock I can't find any, could you suggest some maybe even well funded startups.
Thanks for the very helpful video! Do you have an online community for your channel on Discord or something, or any other Data Engineering community you would recommend? Also the Data Modelling link in the description is broken for me, any other data modelling resources?
Hey Ben. Just pass my az900 dp900 and dp203(data engineer). There are many job's however no luck for me so far. What advice you would give to get my first job? Thanks
In terms of breaking into DE? You can do it, it can be harder. But you should attempt to get some projects under your belt and perhaps if you already have a corporate job, see if you can move into a DA or DE position at your current role
@seattledataguy Thanks for videos. Trying to find where I can get Sr Data Engineer prep courses which Includes system design and other important things
My English is not good because im learning bbut im gonna explain my situation 😅 I was hesitant to choose Data analytics or a front-end developer because before coming to the USA I worked as an accounting assistant and when I got here I dedicated working in a restaurant cooking. Three months ago I was just learning html and css on my own, it's giving a bit of difficulty js. I heard other people that i would have chosen data analytics because it would be a good combination with my accounting degree that I did in my native country.. So now I am lost.. 😅 Any advice please 🙏
Who is exaggerating pay? Skills, I always try to be honest where I am and always trying to improve myself! The DE world never lets you rest in terms of learning
If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k
I like how you mentioned Docker and Kubernetes. By the way, I've heard that Resubot can help streamline the resume creation process, which is pretty useful for anyone looking to transition into a new role.
I appreciate how you broke down the steps. It’s easy to get overwhelmed with all the tools and technologies out there.
I've just done the same thing. I've spent years being a solution designer / data analyst and I've just switched into data engineering as a senior technical specialist with no experience in actual hands on data engineering but I've got experience in designing data warehouses and ETL solutions.
Nice! how is it going!
Data pipelines are definitely an area I need to focus on. The links you provided make it easier to start learning.
Great insights on transitioning from data analyst to data engineer. The resources you shared are super helpful for anyone looking to make that leap.
The emphasis on data pipelines is super relevant. I think understanding this will really set you apart in the job market.
This video really breaks down the skills needed for data engineering. I appreciate the focus on Python and SQL, they are essential.
I transition from a cook to a Integration Developer 2 years ago and now I am taking a Master in Computer Science while working, hoping that 3 years from now to get into Data Engineering,
How did you become integration developer from cook/chef? 🤔
Please elaborate your path. I come from hospitality background & looking for inspiration.
@vivekjoshi3769 I have studied for about 6 months to get AWS - Solutions architect certification and SQL. That was enough to get my first IT job
@@andreinitescu1768what is the title of the job you applied for ?
If this isnt proof that my phone is always listening. Last night my husband was telling me how he (manager of data analytics) is being moved under the programming department and being made data engineer. This popped up in my suggested today lol
Happens all the time. I could talk about something with a friend over a phone call and the next day an ad about it pops up. Lol
Let's normalize this now. Let's show no surprise to these things. It has been long time now 😂
It isn't necessarily proof. It's likely just the Baader-Meinhof phenomenon/frequency illusion.
Already onboarding to DE work at Amazon :) thanks for the inspiration!
Congrats!
Great resources shared here, especially about data modeling. I'll definitely check out the links provided.
Learning about data modeling seems crucial. It really feels like the backbone of data engineering.
So lucky to have found you. Doing my masters in data science after being a data analyst for about a year. Didn’t even start the vid I can tell this guy has real wisdom
you're too kind!
where are you doing the masters? i.e. which program? I'm thinking about it too
@@abiygirma1925 Rutgers University! New Jerseys very own!
I just got hired as a Cx (Customer Experience) Quality Analyst as my first job. With this video, you helped me a lot on how to start my path toward a Data Engineer role, thanks.
Good luck!!!
you're welcome! how has it been going?
Hi, im not there yet, but i recently got promoted to a Sr. Continuous Improvement Analyst position. Im very happy on where i am, but im still pushing myself to my goal.
Great video. I've been in my first "Data Analyst" contracted role as an Enterprise Analytics Analyst & Data Product Owner for 15 months now and while the salary of $94k is decent I'd love to transition to the more rewarding DE career path. I currently delegate work to 2 engineers with different specialties and am hoping to shadow them more often to see some of the DE processes carried out in a real business scenario.
Thanks for the tips!
I've pivoted to a full-stack developer route which can also easily translate into a data engineer role at the 3-5 year career point.
Yeah 100% any easy switch
Can you please help me in understanding how full stack and data engineering is related?
@@abhisekgupta5381 A full-stack developer knows how to get the code to interact with the database and it's data. They know how to write the SQL queries needed to get the data the applications need. And if it's a new application, in many instances they also help design and build most of the database objects as well. Basically, by being "full stack" it implies detailed knowledge of the database tier of the system.
on my journey from DA to DE. Taking any opportunity to do ETL's, models, automatizations that I can find. Unfortunately, in my department our tech stack is so poor, that I realized that I'm never going to transit to DE internally and the DE work I'm doing not valued as much as of good DA or BA.
I'm taking the IBM Data Engineering certificate. I'm blowing through the intro courses because I already have those skills down. It's the warehousing, Spark, and so on that I'm taking that course for.
Me too, I took it specifically for Kafka and Spark
@@panic_seller, I couldn't find any Kafka or Spark tutorials on TH-cam that worked for me, which is why I decided to enroll in the course. 😊
@@theinquisitor18 im interested now !
Hey!
I think to take a course in Data engineering .
Do you recommend it IBM? What do you think about it ?
@Hamza Qahoush I'm also taking this certification since it's free through my university. So far I have only taken the first few classes on Python, SQL, and relational databases. The material is not very thorough or in-depth. It introduces a topic, guides you through a few simple exercises, and moves on to another topic. Unless you can get it for free, I would recommend skipping it and looking for free resources, then building your own projects.
I've noticed that having a solid resume is key for landing interviews in this field. It seems like an AI-driven approach might help refine the process.
Great video man !
Appreciate it!
Amazing video. I’m still in my undergrad majoring in CS but I on following this exact path. It’s my dream job to be a freelance data engineer/analyst for small businesses.
No dude, go away. You have a CS degree. Go be a developer so us lesser MIS-er’s can have a chance 😂
What an awesome explanation! Very usefull, thanks!!
You're very welcome!
Hi Ben,
You mentioned in the video that you linked the interview test prep? I don't think I see it in the drop down. Would be very interested in taking a look at it - if available? thanks!
found it in the description of his other vid - title: "The Last Data Science/Data Engineer Interview Prep Video You Will Ever Need To Watch"
Camera upgrade ? :D Finally ;-)
I bought a new phone hahahah
So great 👍🏿
Glad you liked it!
Trying to head there at the moment
Good luck!!!
3:10 Hi Ben, where can I find the diagram you displayed here? I really like it! I tried the links in the description but couldn't find.
The image is here res.cloudinary.com/practicaldev/image/fetch/s--cnXIakxd--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/github.com/datastacktv/data-engineer-roadmap/raw/master/img/roadmap.png
@@SeattleDataGuy Thx sir! You da man!
Love this video! Could you make one for SE -> DE?
Oh! thats a great idea!
Cool idea. I am a SE and currently working toward DE. I would like to see a video that you suggested, too.
You should considered seriously to launch an online course related to the Modern Data Stack: Fivetran, DBT and Snowflake.
maybe! for now i have launched a course on consulting because it felt like thats what people were asking for on linkedin
@@SeattleDataGuy really? could you share the link pls?
@@WarrenCandiasis sure!, you can find it here - courses.technicalfreelanceracademy.com/courses/starting-6-7-figure-consulting
Thank you for taking such a positive, imposter-syndrome-fighting stance on becoming a data engineer. Sometimes the bar to land a tech role feels so high!!
Have you checked out the data engineering zoomcamp and if so, is it worth the time? Seems pretty popular and results in a personal project with decent resources to pull from. Currently in a hybrid ETL and Support role where I am trying to work in the DW/DE and hoping I can get some more hands on experience.
Great video. I'm currently in the process of becoming a data analyst but my end goal is to become a data engineer. Do you recommend any certifications to have that will help gain knowledge and experience I was looking at some of the Microsoft certifications for data engineers.
Amazing thank you Ben! How much time would it require to complete the switch in your opinion?
Hey, I have gained some data engineering skills and have done end-to-end ELT. I am looking for some open-source projects to contribute to but to my shock I can't find any, could you suggest some maybe even well funded startups.
Nice to see that it's not just my cat that sees my work chair as its arch nemesis.
Yeah, that chair is gone now hahah...
Hi. when you say baseline skills what do you mean by it? Specifically for data scientist.
Thanks for the very helpful video! Do you have an online community for your channel on Discord or something, or any other Data Engineering community you would recommend? Also the Data Modelling link in the description is broken for me, any other data modelling resources?
if you would completely start from 0 will you choose data again
Can you share please the learning path diagram from minute 3:05 ?
Would you recommend taking a similar path, starting in Data analytics then transition to engineering?
A lot of people have done it! so it works great
Will it be considered a downgrade if one move from Data Engineering to Data Analytics?
I don't think so. A great data analyst drives a ton of value
Hey I'm in the same shoe. So many videos/topics on DA to DE but I don't see any DA to DE.
Hey Ben. Just pass my az900 dp900 and dp203(data engineer). There are many job's however no luck for me so far. What advice you would give to get my first job? Thanks
where can i find the interview guide broski
Here is the Data Engineering Study Guide!
docs.google.com/spreadsheets/d/1GOO4s1NcxCR8a44F0XnsErz5rYDxNbHAHznu4pJMRkw/edit#gid=0
What would you recommend for someone with no experience in Data Analytics
In terms of breaking into DE? You can do it, it can be harder. But you should attempt to get some projects under your belt and perhaps if you already have a corporate job, see if you can move into a DA or DE position at your current role
Step 1 - Get a B.S. in computer science.
@seattledataguy Thanks for videos. Trying to find where I can get Sr Data Engineer prep courses which Includes system design and other important things
Some of the links don't work. Where can I find the Data Modeling resource? Thanks!
I have learned about SQL, Python, Spark and Azure Databricks , should I start applying for Data Engineering jobs right away ?
does his mean one has to start in another role before pivoting into data egineering?
You don't have to, but it does make it easier
Can you please share link to interview prep?
How about mathematics? how much math should you know?
not too much, more solutions thinking and logic
Hi where can I find the interview prep link?
Yeah wanted to try out those SQL problems
Hey can you link the interview prep that contains SQL problems and videos
^^
I am having more skills & confidence in being a data engineer
While as a data analyst i am not confident enough
What should i do then?
My English is not good because im learning bbut im gonna explain my situation 😅
I was hesitant to choose Data analytics or a front-end developer because before coming to the USA I worked as an accounting assistant and when I got here I dedicated working in a restaurant cooking. Three months ago I was just learning html and css on my own, it's giving a bit of difficulty js.
I heard other people that i would have chosen data analytics because it would be a good combination with my accounting degree that I did in my native country.. So now I am lost.. 😅
Any advice please 🙏
You couldn't switch to data analyst or financial analysts instead of front end dev. It entirely far from account packages compared with data Analyst
Data engineer vs Data Scientist??
Hello! Thanks for your comment. What are you aiming for salary? work life balance? problems?
Why would one want to stop being a data analyst... 🥺
😜jk
....💸💸💸
Can you link the interview prep sheet?
found it in the description of his other vid - title: "The Last Data Science/Data Engineer Interview Prep Video You Will Ever Need To Watch"
0:45 what is my cat doing there
All good, but I am pretty bummed about how everyone exaggerates their pays or skills, I guess it is a part of marketing culture here.
Who is exaggerating pay? Skills, I always try to be honest where I am and always trying to improve myself! The DE world never lets you rest in terms of learning
I’m 16 but I want to learn how to use python and other programming, I want yo be trained if anyone is free I need someone to explain everything
Which best for data engineering , Mac or Windows?
This guy is knowledgeable not to mention his handsome face
thi video sounds demeaning Data Analyst Roles. Although Data Analyst or Dat Eng. Or Data Scientist. all have same MAX salary.
I'm a data engineer but why people are doing brain wash to switch to DevOps!!!
Does this even wortth it now in the pace of AI developing?
I think so :) we have a while until ai really catches up
Bro what’s that tumbnail? 😂
Whats wrong with it hahahaha