Data Scientists vs Data Engineers: Which one is for you?
ฝัง
- เผยแพร่เมื่อ 18 ธ.ค. 2019
- HOW DO I GET A TECH JOB?
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
📚 Video courses from JomaClass:
🎓 New to programming? Learn Python here: bit.ly/joma_python
🎓 Learn SQL for data science and data analytics: bit.ly/joma_sql
🎓 Data Structures and Algorithms: bit.ly/joma_dsa
💼 Resume Template and Cover letter I used for applying to software internships and full-time jobs:
resume.joma.io
💼 Interviewing for jobs now? Get access to interview question database, courses, coaching, and peer community today:
www.tryexponent.com/?ref=joma
📱 SOCIAL MEDIA
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
/ jomakaze
/ jomakaze
/ jomakaze
Some of the links in this description are affiliate links that I get a kickback from
![[UNCUT] รัฐสั่งหยุดปล่อยคลื่น! แต่หยุดพลังปะทุ “น้องหญิง” ไม่ได้ I คนดังนั่งเคลียร์](http://i.ytimg.com/vi/RmQh47eIiV0/mqdefault.jpg)
I am a Data Engineer. I hate it.
:( I’m sorry
Do you get much face time with the customer or are you behind the scenes?
Why do u hate it?
@@malcorub Yeah, I get to face the clients and business vendors and we need to understand the same business problems that Data Scientists, Insight Analysts and Project managers need to understand but our job is to provide them the correct data, make Data Pipelines and refresh the datamart for the dashboards etc. But I feel like a data tech support for the rest of the project team. Also involved in working with legacy system in Teradata and modernizing it with the latest Big Data tools. But I am slowly learning the Data Analyst / Scientist work as well. That is where the "sexy" part of the work revolves around. Data Engineering is not a sexy job but it is one of the most important positions in a company.
@@sanjogh777 As long as you like working with data there is hope. I am a data analyst/report developer, I rely on my D.E's (who get paid more than I do) heavily to keep the data infrastructure going, for data pipelines and building complex data models and without you guys, my job would be so much harder or dare I say, IMPOSSIBLE. Like Jona said at the end of the video and as you mentioned in your comment, don't limit yourself. Dabble in the data analyst, PM and data scientists side. It's much more satisfying when you understand and can derive meaning from the data you are working with which many times, at least in my world, the DE's don't or choose not too.
This guy looks like Yagami Light from Death Note. He needs a shinigami.
He has a millionaire one
@@psyberpirate LOL
I've tried Data Analyst in the past and it wasn't my thing. You have to have that "analytical thinking" to translate data to words. Soon after, I switched to become a Data Engineer and have been loving it so far because as the title says for itself, it's more of an engineering role rather than analytical. There comes a task where you need to get some data in a specific format and that's it! You don't need to waste time shift through data to understand anything and that's what I love about being a Data Engineer. Data Scientists however, is a role that I have not tried yet as it requires a certain knowledge of mathematics and analytics which to be honest I have no intention to be mastered at but imho DS is an upgraded version of DA with more complexity of machine learning models and mathematics.
Therefore I would say it depends on people's mindset to see which one is for them. If you think more analytical, then go with DA or DS but if you prefer less analytical and more engineering, then DE is your chance!
dude u make no sense DS is no upgrade of DE its not even the same job nor the same skills , a data engineer is a software engineer while the data scientist is not an engineer im sure u re not a data engineer cause i ve never seen someone move from data analytics to data engineering
@@abdessamadbaahmed8930 Lol okay, well I am a DE and I did it! And I never said DS is an upgrade of DE, idk where did you infer that from.
@@faridakhavan1996 but imho DS is an upgraded version of DA with more complexity of machine learning models and mathematics.
@@abdessamadbaahmed8930 I said DA not DE! Data Analyst not Engineer!
@@faridakhavan1996 ooops my bad man
> Data science analytics is super rewarding
> Leaves his data analyst job at Facebook to become a software engineer because he's tired of doing bitch work
Love the ending. Thank you for the advice. 👍
As a poet who wants a data scientist/engineer job to just make money temporarily, "not intellectually challenging" is exactly what I'm looking for.
Hit me up for more details ☝️☝️
Are you me?
@roshankr3938 bhai (i assume) can you update your email in settings so i might be able to reach out to you?
Great advice on not restricting yourself! I landed a Data Engineer internship for the summer - will definitely keep this in mind and try to be as proactive as I can. Thank you Joma :)
Wow this is the exact opposite of my experience. In NYC the market is saturated with math / physics grads willing to work as data scientists, but who are less familiar with coding. As such wages are suppressed and it pays around $90k - $110k here. On the other hand, data engineering sits between devops and software engineering, and pays between $140k and $180k, although I have acquaintances making north of $400k as data engineers. As a data scientist you mostly write SQL queries and make Tableau dashboards on top of the ETL pipeline that data engineers built out. It's less challenging and honestly kind of mundane. Given a question, you usually know which SQL query to write to answer it. The few data scientists where I work are trying to shift into data engineering by learning more about kubernetes / spark / airflow / clojure.
That's interesting. In what field is this?
From my little experience I agree with you. I always considered DE > DS. Especially since they are the ones building a platform that all of the analysis/ML is done on. Meaning that company pay big bucks for very good DEs.
@@sohum6326 Yup, Without the DE the DS can't do their job. Luckily I'm kinda a hybrid of DE/SE/PM and a little DS but that just comes from years of experience.
Right! Same is true here in Italy. I think Joma said it best when he pointed out that these colossal SaaS companies usually have all their main processes already mapped out and not much challenging/stimulating work is left... thus the job market shift
Interesting perspective. My experience is that data science is "sexier". Anyway, I'm a data engineer, I actually have a video about it on my channel! :)
I felt this is because you are in a big company like Facebook. If you were in a startup or medium size company, data engineers have chance to do a lot of exciting things including algorithm, data pipeline, or even machine learning model. Nevertheless, thanks for sharing the relations between these three roles in big companies! Hope to see more video like this!
I am a data analyst/report developer, we do not have any data scientists at my place of work. ... truth be told, the data engineers at my place of work get paid more than me and my peers and they have alot less face time with the business units and costumer... but because they are so behind the scenes, the analysts are seen in a more positive light by the customers, business units and leadership than the data engineers so we get more accolades and bonuses by leadership.
like your video as always!
I will give my two cents as a data engineer. Pay really depends on region and company. In Toronto, data engineers typically get paid higher than data scientists and software engineers. But data engineers in Toronto typically work on hadoop/spark/distributed system problem, which is different from large tech companies, like facebook.
“Technical part is the easiest part of the job at a large company”
I used to work at a relatively large company in Tokyo, still it felt exactly the same!!! Thank you for uploading new video!!
Boy, thanks for this video, helped me a ton!
absolutely love your brutal honesty!
Great video Joma.
Congrats!
Data scientist is a just an illusion of a cool title for what is really a data analyst or just simply working in analytics. It's all just names and semantics. Data engineer can be limited in scope but it depends on what you're doing like how you're developing and maintaining a framework or pipeline. A good infrastructure is always needed and requires having a good knowledge all roundz that is, develops for building and deploying, sound testing techniques for validating the data, framework for expanding how data is added and so on. A data scientist typically don't have that background. Like i said most of who you call data scientist are really data analyst. Nowadays machine Learning engineers are the real data scientist but they also have a very solid data Engineer background for understanding algorithms and setting up platforms.
This comment put into words exactly what I feel about the data science space. An machine learning engineer is just an advanced software engineer with more knowledge about data science.
Thank you for the truth about modern day glorified data analysts lol. This is partly why I rather have a data engineering background before embarking to a role more involved in machine learning etc.
yeaah i feel u, idk why they are called scientists but then again, most ppl who hold these positions are either master or phd grads in math
Agree 100%. Most often the Data engineer has to know the data better than the Scientist. This is like being a report developer and you get an ad hoc request from a dept manager asking for everything. A good report developer will know to ask 'what are you really looking for? I'm not just dumping a whole dataset on you for you to slice/dice in excel'. Same applies to DE/DS. But Data Scientist is sexy term and front facing. If you have good mgmt they know the DEs provide the data.
Really awesome video man. Keep up the good work!
Joma rox! Thank you for all pointers :)
thanks for this video Joma! Gave me a new perspective on the DE aspect of things.
Just wanna say bro, love your videos. Keep up the great work!
I'm a data engineer , I use a variety of tools to do my job including Informatica, SQL Server, Oracle etc. As part of my role I am also learning about data governance and will soon go on training courses how to use Governance and Data Quality tools.
A good data engineer would also have good working knowledge of the corporate data. I have to disagree with the notion that DE is not intellectually stimulating... but it might be different over at Facebook where you used to work.
Hey man, I'm currently doing my current job as an ETL Developer. But sometimes I feels like "am I a Data Engineer or an ETL Developer?"
I uses IBM DataStage ,Pentaho, Bash, Oracle, Postgresql, and Mysql.
So Am I a Data Engineer or ETL Developer? Sometimes I'm confused.
@@andrewmaxwell9399aren’t ETL Devs a subset of DE’s?
Can you do software engineer vs data scientist? About their pay, working hours, and which is better. Since you have experience in bothof these fields, you must be knowing a lot.
DE is akin to being a construction worker. And honestly, I love it. Versus being a home architect=DS, interior designer=front end dev etc. It's a technical way to be a builder. Yes, less creative freedom and intellectually less challenging. But if you like semi mundane & repetitive tasks that still make a difference than DE is the way to go! It's more of a personality fit, I'd say.
Security room outside your entrance that's where cybersecurity comes into the picture
@@garimagupta7227 Lol or the guesthouse. Not sure which analogy fits best. Haha
@@ajtam05 In you Astral Chart, in one of them ( Sun, Mercury or Mars ), might have either Capricorn or Taurus.
Good to see you are still on TH-cam
Can we get an update on how the SWE job is going and how your life is now as a SWE compared to a DS?
Oh yea, that’s a good idea
lol next is devops vs ios engineer vs sysadmins vs embed systems engineer .... ooops infinite loop
please !
Oh yup, this sounds interesting!
Thanks for the insight! Totes new to this area but defs keen to learn more! Hello from Australia
Hit me up for more details ☝️☝️
As a data engineer, it was the toughest part is constructing data quality because there are so much stress to fix things under a short period of time. The requirements constantly change so you are supposed to be flexible.
What do you mean by “requirements” ?
@Marc B. Requirements are based on the clients needs. Sometimes there are errors found and you have to address it to the clients to see how they want the dashboards to look or how the data is structured. Requirements are based on business logic needs. Some business have tables they don’t even use anymore but that is communicated last min sometimes
@crystal12w Do you enjoy your job as a data engineer?
@@88andreaw I do like it but I also need creative outlet because I like photography and videography as well. It’s best to have hobbies too.
Agree. And what’s tough for DEs is that the moment a project starts, immediately we are the dependency, the pressure is always on us 😅
This defenetely made me want to get a job as a data scientist, I hace knowledge of it already, and the part about thinking about the product, its future and supporting It with conclussions from data totally defines what I like. And so does when you talk about a data scientist with PM knowledge. Thanks a lot now I am more determined about my future.
that last bit was inspiring. Thanks for that
"Take the opportunity so that you can invest in yourself so that you can be a best AT EVERYTHING!" 인생의 교훈.
Great career tips man :)
I think you can transition into ML from DS/DE because a large portion of ML is cleaning, normalizing and legitimizing the incoming data plus you work with a lot of graphs?
Great vid! I’m an ex PM of a bluechip, turned into a ‘everything’ guy in a data analytics startup. Company culture plays a big part in how you gain value out of your role. I’m in a much happier place because I am now learning new skills again, and feel like I’m adding tangible value. Sometimes you need to drop the roles, forget about pay for a second and just follow the pathway that interests you / gets you out of bed in the morning.
why did you turn from PM role to data analytics field?
Wow. This video really hits. Blunt honesty. Currently a Data science masters student at Pace university
Great video! I watch you from MX, YUC.
In a smaller market this has been the opposite of my experience: when you have to build stuff from scratch the order of priority and value comes from data engineers then data scientists. Sometimes data scientists do both though because of a shortage of money and/or talent. Facebook is the opposite probably because as he says most of the work has already been done and so they call typical BI analysts Data Scientists and DevOps people Data Engineers and the really hard stuff (from a technical standpoint) is left to people who are way more senior than he was or have PhDs.
Finally, Joma make good contet, yeaah
We need more of joma startup...
Machine Learning Engineer vs Software Engineer?
Sometimes, I am so confused about how to distinguish DE vs DS and how can i involve to either. You clarified clearly what I curious about and much appreciated! Thank you.
I cant leave without comment. :)))
Good to see you man
Zein El Mokhtar thanks man u 2
wearing patagonia... seems like someone watched the latest techlead video :D
More videos like this please!
Joma makes a lot of good points here, and I just want to rant a little as a DE myself.
DE vs DS is not a good comparison, because one is an engineering role while the other is an analytical/quantitative role. Very different career paths and skills.
I think it's better to compare data engineering with software engineering. The difference is that software engineers build the tools, while data engineers use the tools. For example, software engineers build Spark, while data engineers use Spark. Joma is right here about data engineering not being as intellectual challenging. Think about it, would you rather be the one that build the distributed systems underlying Spark, or do you just want to import pyspark and use APIs? However, for most engineers, they would never get the opportunity to build cool distributed systems like Spark. Most engineers work at product driven companies that just need to build a good product with good UI/UX. If you work at such a company, I think DE is a bit more intellectually challenging than frontend web development. If you want to build cool distributed systems, then you gotta go work of companies like Google and Databricks.
It might be better to compare DE to backend engineering. I actually think a backend engineer with solid data modelling skills and database knowledge would be able to transition to a DE role very easily. However, a DE who only works with pyspark and cloud technologies would find it a bit more difficult to transfer to backend engineer role.
In conclusion, if an "engineer" wants to stay relevant and a "beast" like Joma, then they should build the tools, rather than use them.
👏👏👏❤️
Great insights
Thank you for this video. I heard there are different types of data engineer, like data engineer analitycs and architect etc. So I think in this video you told of the first.
So that you can be a beast at everything. Amazing advice.
Well said Joma
I watch this video again, I do have a bit confusion that why you decided to be a SE ? and how DS skill support you as SE ?
"Take the opportunity to invest in yourself so that you can be a beast at everything."
Well said, thank you Joma. I agree that too many people limit themselves and miss out on opportunities for growth and career advancement, and even just for personal pleasure.
This hits hard. As a data science analyst turned PM, I miss the technical stuff but am highly aware that going back into a technical role is equal to destroying my career.
What's a PM?
product manager -> product owner deluxe
"going back into a technical role is equal to destroying my career"
WUT
Scorpion in our company, technical roles are well paid but are limited in perspective. For instance the maximum a data scientist gets paid is around 6.200 euro. For other roles like (product) manager the maximum is above 7.800 euro.
Astarte Rap, I can’t fully disclose. Large financial services company with strong focus on data (and more and more on data science).
Loved it ❣️
Hi Joma, I'm currently struggling to choose between those 2 (also thinking about going into masters degree for ML, dunno). I was at first very convinced to go for Data Science, but my main difficulty is on the analysis and cases of the interviews. I don't know if this is something of a natural "talent" I lack, or something I can work and practice to learn. I find no difficulty in digging into large data and organizing it or creating a dashboard, but I struggle to actually create something meaningful with it and finally generating the "insight" I'm looking for. I don't know if you can understand what I mean, but could you give me some advice?
As a DS I can tell you it’s not a lack of talent - without an understanding of what questions need to be answered it can be very overwhelming to pore through a large dataset with tons of variables to try to find a needle in a haystack that you know may not even exist
When is the startup series coming out, Jonathan ?
Except based on what sort of data asks he's referring to, most of those would be done by an analyst...not a data scientist. In my experience, data scientists are brought in to work on very specific features or upgrades (and are then let go once done), while data engineers are kept around to make sure it all keeps working.
I have to say, your words are so true about data stuff = =
That last statement... 🔥🔥🔥🔥🔥🔥🔥🔥
Nice patagucci fleece, I have the same one!
Thanks for the insight. What about data architects? Is this position still alive and relevant? Or has it merged in one of DS, DAnalyst, DEng?
Make a video about Research or Practical path in Ai/ML !!!
I am a DataNinja. I work in a tangential field as a Data Engineer, Scientist and PM. There are 700 of us at Syniti who do not allow ourselves to be boxed in. While the bifurcation here is natural and aligns to our consulting levels, we are still all hybrids. Is this crazy to this group?
"Data Science Analytics" just sounds like a fresh name for business intelligence work, not actual science.
You may hear it as a "Data Analyst" role too.
Nice channel...nicee space work
I was exactly searching for this video. What about entering Data Engineering as a bridge to switch later to Data Science (as a software engineer) ?
I’ve seen a few do that at old company. For people who enjoys more analytics
it would be awesome if you make a podcast about information technology job..
Data Engineer in Silicon Valley: SQL, Python, R, Java, C#, Scala, AWS/Azure/GCP, PySpark, Databricks, Tableau, Azure Data Factory... works along with Data Scientists and ML Engineers.
Data Engineers from the rest of the world: SQL, SSRS, SSAS, SSIS, Qlikview, Cognos, Crystal Reports, advanced Excel skills creating macros and VBA... works alongside BI Developers.
Do companies typically look for Data Analysts that come from a math background? I did 3 years of my undergrad in Finance and I switched to business analytics and a minor in data science. I did my first internship as a DS consultant for a startup over the summer but now I'm having trouble landing my second internship. Most of the listings I am seeing on my school's job board are looking for students with a degree in Software Engineering or CS, I apply anyways but because my major is Commerce I have a feeling my application is just being ignored. Feeling pretty discouraged.
Try Facebook, your background is pretty common for DS analytics there
Your channel is getting better, I'm resubscribing :)
Great video
Currently taking on a data scientist role in my grad capstone project, and while I like it, I prefer aspects from DE more. Very jealous of my colleague who is taking on the data engineering role with the project I'm on haha.
Honestly DE sounds awesome. I am working on my Master degree but I don't mind the mundane work.
Beast is coming. :D
fucking genious, thats a nice end out conclution. fucking love your channel, i'm your fucking biggest fan in Argentina, and also i'm fucking love meditating... HAHAHA
Where did you get that poster from?
Hi Joma, I am a SWE going my masters in Big Data Analytics
thank you is all I can say
i love this video a lot
Is there a position that combines Data analyst & Data engineering? I want to analyze but also program.
IE (with information systems specialty) can get something like this??
Machine learning engineering
Normally smaller companies do not have the manpower to fully separate roles, it's common to have just 1 person responsible for all "Data" stuff so you'll have to be a one man orchestra.
You should rename the title to Data Analyst vs Junior Data Engineer
yeah Data Analyst is basically business job.. like BS Data Science is a mixture of Business Management and Computer Science. And a Data Analyst/Scientist can work in both Tech & Business.
I have the same Patagonia jacket Joma oppa 😍
Everyone in tech does!
I work as a Database Engineer in a small team and I get to fully create cloud-based data solutions by doing DE, DS, Architecture, and DevOps Management plus I get paid more than Project Managers. That said, I have to still deal with Project Managers, and sometimes it's an absolute pain in my ass.
Well, that's their purpose, right? :D
@@jenniferbyrne2980 It is, but if they're unfamiliar with the jobs they are managing, the expectations of their timeline can become extremely unrealistic. Additionally, if you are more forward-thinking, some can't keep up.
@@ChronicSurfer you are so right! I've experienced both
i love that screen thanks bro
Did you have any Data scientist collegues with an Industrial Engineering bachelors?
The two sound really similar minus the programming part.
I work in digital Marketing SEO and what to make a career shift to either data science or data engineering, which would be easier most rewarding?
9:09 bro Awesome 🔥🔥🔥
I am landing a job as DE. At the beginning I did image that I would work about distributed system such as data, task and solve these relational problem. But in short time I realized that this is belong to your company size ( Joma has video talk about this), and it like Joma said, DE just use tools and build some kind of pipelines, monitor pipelines from raw to usable data. Moreover, the tools that I use that is developed by another team with stuffs back-end services. For now, I install Hadoop/ Spark on local machine and learn by self. So I really appreciate Joma because you help me to understand more behind the since :D .
Joma made a vid
Yeah man, I'm a Data Engineer!
Do you have a CS degree?
So to chance the wording a bit, the PM is the CEO of the project, and the DS is the "CTO"?
Very insightful stuff @Joma Tech. Wanted to ask you, what book(s) could you say made an impact on your ability to maneuver around social situations (politics as you mentioned)?
i'm breaking my back learning stuff, taking online courses and learning in school a lot of tthe technical part and it worried me a lot when in 5:55 he said it is the easy part
Im sorry to say but he is right.I am a data engineer and the most important aspect for me is the ability to convey concepts to non-tech people as well as the ability to persuade the team to do things in more effective/productive ways.The learning is the easy part. Once you've learnt something its in your noggin forever.
usually there's only few involved in executive decision, and a lot do the bitch work.
from this, do you think that DE will emerge more in the future than the DS, because the company needs more people to do the bitch works??
I don’t think it’s necessary true that data engineers makes less than software engineers. Of course a lot of it varies for company, but imo de make the same if not more than swe. A DE can be anything from writing a lot SQL, to managing dbs, to apis, doing some DS things, to etl.
If I would like to work as a DS in the end in the healthcare. (Self-taught in IT/ medical degree from another country). Could you recommend me where I do begin?? . I’ve been learning Data Analysis (excel, Python), Data engineering tools and a little bit Software engineering tools (JS, react). 🙏
What is the outro music named?
I have a question(s) that may seal my fate for my future career to come. I am a recent apprentice graduate in Electrical and Electronic Engineering, with around 6 years of industry expericance within civil infrastructure. For my dissertation (final year project) I some what created a machine learning program using computer vision for a facial recognition application, I was hooked. Recently, there has been a decline within my sector in the industry I work in (I specialise within Traffic signal engineering and modelling ). I had numerous conversations with a colleague of my at work about the situation of the industry and how it may it effect our career in the future. He mentioned about getting in the world of data science /engineering. That has got me thinking, with my background in EEE and a real interest in machine learning which path should I considered, data science or engineering?
Hit me up for more details ☝️☝️
Pls What courses or languages do I need to do study to be a data scientis
Create a course for the basic start of DS/DE/DA, for those who can't pay like me but can be a master by learning from an extraordinary educator as you.
without masters is ther will be any good opportunity in data science field as a comp.sc. bach. degree??(specially in US)
Hey guys, I came here to learn before taking my first job in this area, and... what is a pm? (that Joma and some comments talk so much about?)
redbull, into workable software!