thanks for the video, benjamin! I love doing ML, but my official role is DS, but again I am mostly doing DE/ETL work. At startups all these are kinda mixed, haha
Yes, MLOps is still a newish term. I believe it was coined about 2-3 years back officially. But prior to that I believe the push for the concept was due to a 2015 google paper( proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf ) that pointed out how hard it was to deploy ML models. So I do think ML engineers have been doing this kind of work for a while but I think there has been some specialization over the last few years.
@@SeattleDataGuy all these terms are very loosely defined and in each company it could mean something completely different -- but very generally speaking i think the term MLE was coined to differentiate it from more "pure" data-science work that does not concern itself with operations or deployments. i.e. a "research" team, or maybe you could think of them as data analysts on steroids. So an MLE by contrast would basically be a DS who has more coding skills and more responsibilities of building viable production-grade components. That would make an MLOps engineer someone who takes care of the "path" between these models or components all the way to production (+ monitoring, pipelining, continuous training, etc.) but does not actually do any data science - and - finally, a DataOps engineer or a Data Engineer - responsible for the infrastructure , pipelines and processing that occurs UP to the ML components In smaller companies (like mine) - the role of DE could in fact combine both MLOps with DataOps and be in charge of the entire data-flow from the source, through ML, all the way to production (and beyond). Should also mention the "Big Data Engineer" which is basically the same as a DE but with a specialization of working with Big Data tools and technologies.
Excellent breakdown of each role. As a graduate that has recently entered the work force, I have wanted to switch careers and move into the data space. Out of the 4 roles mentioned in your video, which one has high demand and low supply? I want to know which one has a lower threshold in terms of accepting at entry level. Thank you !
My background is accounting which is pretty strong and solid, Im opting for data analyst role, initially was business analyst but I became obsessed with python. Do I need calculus 1 and 2 or linear algebra in case of transitioning to data engineering role? I feel as if maths and statistics skills needed are way more soft for business analyst/data analyst roles than data science roles, data engineering, mlops engineer...
@@georgejetson9801 Hmm, I don't have anything in particular there. Usually, if I start posting more often on linkedin...i suddenly get tons of amazon recruiters hitting me up.
1. 2:34 Mistake-o Numero Uno: You want a PPT displayed overhead. You don't want to be hunched explaining over graphs on Letter printouts. 2. You don't know SQL? The word "Data" should not be anywhere on your title. Shame on you.
Let me know if you can tell where I accidentally gave my editor the wrong information and I had to make a quick edit...🙃
5:18 “Data Lake-house”, that’s a great new term :)
Yeah, I believe databricks has really started to own the term as they are trying to focus not only on BI needs but also DS and ML needs.
Thanks for this video, as always great data-related content!
Thank you for the support!
Dude, great perspective provided here. I feel there is such confusion over these roles and I feel this video helps
Thank you! I am glad it was helpful. There are so many other data roles I didn't even cover. BI, Analytics engineer, etc...
Thanks for sharing this overview!
Thank you for the comment! Let me know if you have any questions.
thanks for the video, benjamin!
I love doing ML, but my official role is DS, but again I am mostly doing DE/ETL work.
At startups all these are kinda mixed, haha
Yeah at start-ups everyone does everything!
I am BI and I am doing DE/ETL work too bcs the data isn't gonna run itself XD
Really great content.
Thank you!
There is the Machine Learning Operations Engineer role too (MLOps), but I think it's included in the ML Engineer role that you described.
Yes, MLOps is still a newish term. I believe it was coined about 2-3 years back officially. But prior to that I believe the push for the concept was due to a 2015 google paper( proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf ) that pointed out how hard it was to deploy ML models. So I do think ML engineers have been doing this kind of work for a while but I think there has been some specialization over the last few years.
@@SeattleDataGuy all these terms are very loosely defined and in each company it could mean something completely different --
but very generally speaking i think the term MLE was coined to differentiate it from more "pure" data-science work that does not concern itself with operations or deployments. i.e. a "research" team, or maybe you could think of them as data analysts on steroids. So an MLE by contrast would basically be a DS who has more coding skills and more responsibilities of building viable production-grade components.
That would make an MLOps engineer someone who takes care of the "path" between these models or components all the way to production (+ monitoring, pipelining, continuous training, etc.) but does not actually do any data science -
and - finally, a DataOps engineer or a Data Engineer - responsible for the infrastructure , pipelines and processing that occurs UP to the ML components
In smaller companies (like mine) - the role of DE could in fact combine both MLOps with DataOps and be in charge of the entire data-flow from the source, through ML, all the way to production (and beyond).
Should also mention the "Big Data Engineer" which is basically the same as a DE but with a specialization of working with Big Data tools and technologies.
As a data analyst, most of my job feels like data accounting
Is it because of all the data reconciliation you have to do 😅
Thank you for sharing this! I think Data Engineer is for me. 🤞
Do it!
Me too
@@JoyFay i love that the DE community is growing!!
Another great video. I need to stop comparing myself and salary to top 10% and even top 5%.
Yeah, jealously is the thief of all joy.
Ben, do you have some indication for staffing company in seattle?
What do you mean? Like do I have connections with a specific staffing company in seattle?
Excellent breakdown of each role. As a graduate that has recently entered the work force, I have wanted to switch careers and move into the data space. Out of the 4 roles mentioned in your video, which one has high demand and low supply? I want to know which one has a lower threshold in terms of accepting at entry level. Thank you !
I believe DEs and DAs tend to be the highest in supply based on a few surveys i have seen
My background is accounting which is pretty strong and solid, Im opting for data analyst role, initially was business analyst but I became obsessed with python.
Do I need calculus 1 and 2 or linear algebra in case of transitioning to data engineering role? I feel as if maths and statistics skills needed are way more soft for business analyst/data analyst roles than data science roles, data engineering, mlops engineer...
You probably won't need linear algebra for data engineering work. If you went the ML route sure, but not so much for DE.
Worked my ass off just to realize that I am the oompa loompa of the data scene.
I am not even sure what you mean. But I am dying.
That's a huge microphone
Great info. How's the job market there in Seattle right now? Are there contracting roles? WLB?
There are contracting roles. Plenty of work up here! I think we were talking on another thread about this right?
@@SeattleDataGuy Hi. Yes, on another thread. Are there any good recruiters/agencies you'd recommend?
@@georgejetson9801 Hmm, I don't have anything in particular there. Usually, if I start posting more often on linkedin...i suddenly get tons of amazon recruiters hitting me up.
Am I supposed to know cloud computing to be data engineer ?
You should know how to work with cloud components
Lul, what about BI Devs/Engineers - I can see that they ofteb have simillar responsibilities are simillar
There are a lot of similarities. Do you work on the big data side too or more the analytical side?
9:44 Where he got a job at H U G E
I didn't even see that
1. 2:34 Mistake-o Numero Uno: You want a PPT displayed overhead. You don't want to be hunched explaining over graphs on Letter printouts.
2. You don't know SQL? The word "Data" should not be anywhere on your title. Shame on you.