Machine Learning vs Data Engineering - What's the difference?
ฝัง
- เผยแพร่เมื่อ 5 ก.ย. 2024
- My FREE guide on how to become a data engineer:
karolinasowins...
This video answers a question that I was getting a lot: "What's the difference between Machine Learning and Data Engineering?
To make a good comparison between Machine Learning vs Data Engineering, we first need to clarify what we mean by these two areas.
Machine Learning can be actually split into two separate "camps":
-scientists (data scientists, machine learning researchers, machine learning scientists)
-engineers (machne learning engineers).
And data engineers are always engineers :)
So in this video I compare scientists, machine learning engineers and data engineers :) I hope this will help you to decide on a career path!
As always, any questions, let me know in the comments.
Find me on instagram:
@karo_sowinska
You can warm my heart with a cup of coffee as a thanks!... :)
ko-fi.com/karo...
I've done some research, and I recommend these courses for data engineering:
1. Learn Python
I recommend following the Python for Everybody specialization course on Coursera, which is one of the most popular courses there:
click.linksynergy.com/link?id=kOQH55A53NA&offerid=759505.1560499266&type=2&murl=https%3A%2F%2Fwww.coursera.org%2Fspecializations%2Fpython
2. Learn SQL
SQL is still the lingua franca of data. I recommend going with Learn SQL Basics for Data Science course, because it contains some chapters which are very releavant to data engineering in partcular, e.g. distributed computing with Spark
click.linksynergy.com/link?id=kOQH55A53NA&offerid=759505.15198708466&type=2&murl=https%3A%2F%2Fwww.coursera.org%2Fspecializations%2Flearn-sql-basics-data-science
3. Learn Bash scripting/Linux
I wouldn't take a full course on it, but at least read a good article.
if you do prefer to take a course/guided project, I think this one is short and good:
www.coursera.org/projects/command-line-linux
4. Learn how to develop on the cloud, e.g. on AWS
There are a few good courses around there, but I think the Coursera one is the most comprehensive
click.linksynergy.com/link?id=kOQH55A53NA&offerid=759505.18533796022&type=2&murl=https%3A%2F%2Fwww.coursera.org%2Fspecializations%2Faws-fundamentals
Disclosure: They may contain affiliate links
Hey Karolina! I recently came across your channel. Data Science community is already cramped up with so many people coming up with their own channels and vlogs. There are few people like you who aren't running the rat race of making good vlogs! People like you truly deserve to talk, thankyou so much, your videos have given me some good insights of what a data engineer is and what are the skills required. I am about to start my internship as a data engineer soon !
Oh thank you so much, I'm really happy that my videos are useful to you! :) Good luck with the internship!
@@karolinasowinska Thank-you !
I really like the way you deliver your contents! I am about to switch role from data scientist to data engineer soon. Both roles are great, but I want to see the data from lower layers. So it would be great if you create more contents related to data engineering😆. Looking forward for future contents, keep up the good work!
Oh, that cool stuff! I actually did the same - started off in data science, and slowly moved down the "data layers" :) Thanks and enjoy!
U r absolutely awesome, thanks for brief us.
The worst part is when u work with a small company which they told you "u r ur boss" and ur department is only u, in that particular situation u will should act like every single thing that u mentioned in this video and the result it will be a resignation letter after few months😅
Haha I completely feel the situation that you've described!
You and your videos are such a blessing! Whenever I’m having a doubt , the very next day you post a video about it!😂❣️
Thanks!🌸
I'm so glad to be able to somehow read your mind!! ;)
Same here 😅
Thousand views and subscribers on your way! Brilliant brilliant content. Keep it up and share more!
Thank you! Will do! :)
Thank you❤️
Can you please make a more detailed video about the skills or responsibilities required for both of them?!
Noted, might do soon! :) In the meantime, I already have a video where I talk purely about data engineering :)
I've watched several videos to understand the difference between Data Scientist and Machine Learning Engineer - this is by far the most practical video, because there is no better way to put it. Data Scientists build the model relying on scientific techniques, Machine Learning Engineers scale the model relying on core computer science/machine efficiency knowledge. Thank you so much for this. I think the title should be renamed to Machine Learning Engineer vs Data Scientist vs Data Engineer
This is an incredibly insightful video, thank you very much for doing it.
Thanks, I'm glad you found it useful :)
This is dynamite information :0) thank you so much. Data/Machine Learning. Engineering positions are the ones for me! And having these actual job titles to search makes a world of difference. I can't thank you enough
My pleasure! And always feel free to shout if you need more info ;)
@@karolinasowinska definitley will! Thank you so much
Veery clear explanation again! Thanks a lot!
Excellent explanation of terms for beginners.
Glad you think so! ;)
architecture, engineering, science, operations, strategy, domain, product...
it all kinda becomes just a yes, is fun because it's challenging, but is challenging none the less
1:24 as a former gamedev technical artist I should say the red color isn't trouble in post-processing, it can be changed to anykind you want: hsla and some cryptomatte magic!
Hahha there we go, and all I did was coming up with a lazy apology! :D
Beautiful explanation!
Machine learning and data science are same ,
If Artificial intelligence is a circle,ML is a sub circle deep learning is a sub circle in ML, Ds is a circle combined all of these
NB: Deep learning is not related with python
Thanks it's very clear now
As the old saying goes... you learn something new every day. I am glad you clarified the differences. Too often we confuse one of these titles; I know I am guilty of that. Also, I think bright colors, do suit you.
Definitely! I used to confuse them too! And thanks... I might get myself convinced that red is not the least flattering colour ever!
Karolina
Whenever I watch you're Episode I wish my Major was Data Engineering.
So,thanks.
Luckily you don't need a major in data engineering to become a data engineer! ;) Good luck!
Hi, I will take course on data science in the uk next year : )
Best of luck! :)
When you say quantitive background do you include computer science?
What area would a computer science degree be best suited for?
Thanks in advance I’m so confused about what to do in my life aaaaaaaa 😖😖😖
Lol same so many different titles
Hello Karolina,
I hope you are doing well and everything :) I have just discovered your channel and it's awesome.
My name is Amine and besides Music, I am an electromechanical engineer with more than 5 years of experience in the manufacturing industry as a Quality Engineer. However, I have always been drawn to theory and Mathematics, and I am at a point in my life where I am thinking of switching my career completely to Machine learning. My question is: do I need to get a different Master degree in AI/Machine learning in order to apply for jobs? or do I need just to learn the required skills (Via a certification course online and work by myself) and then apply directly, and maybe my current master of engineering is enough.
Thank you so much for your content Karolina it is very helpful + you are beautiful ^.^
Cheers from a fan :)
MA is not necessary, it's necessary to know the maths and stats under the roof. I'm deciding to leave the ML career because I'm conscious to have to study too much to be familiar with the statistics under the neural networks and other ML algorithms. In my opinion, your background in Quality Control gives you a big advantages in ML
Hi 👋
I'm currently working as database developer in which I work on data modelling ,creating stored procs as well as batch jobs which sends everyday data to data warehouse for analytics on batch fashion but I don't clearly understand how the streaming architecture of data lake works. Can you do a video on this topic
Hello! I'll put it on my backlog, thanks for the suggestion! :)
brilliant!
Please make a Vedio on data engineer salaries and demand in 10 years as you mentioned in one of your Vedio lots of people moving into it then so increase in average salary decreases from 2019 2021
Hello, Karolina! Thank you for the video, it is very helpful to understand the difference between ML and DE! I have a question, how many months have you learned the python and sql after economic background, and is it hard to find a job for this specialization, I mean is it hard to get interview, and something like that?
Hey, that's a great question, I think I will make a separate video with an elaborate answer! But roughly, if you focus only on learning and nothing else, then I think you can make yourself ready for interviews in something like 3-6 months. I did it slightly differently, but I'll tell the whole story in the video!
@@karolinasowinska Okay, waiting for the next video!
Hello Karolina,
I've got an MSc in engineering and am thinking of becoming a machine learning professional. Zero background in it. My question is do one needs to be a data analyst and take a data analytics training to be able to ladder to machine learning engineering?
Thanks,
Behnam
No, you can go straight for ML :)
Thanks for an useful video. Can you put the subtitle choice on the video because I'm not really good at English. :)
Hey! I think TH-cam removed the auto-generated English subtitles :( I will need to write my own ones - I'll try to do that soon! :)
thank you for the information!!! If a person is introducing to this fascinating world of data Would you recommend focusing on one aspect? or learn general topics of data and then in a job look for what position is right for the person?
I would suggest first deciding whether you'd like to go down the scientist or engineer route by exploring various areas. Then if you decide broadly which one's more for you, e.g. engineering, you might learn some general skills that will be useful for whatever engineering job you get, e.g. object oriented language, CS fundamentals like databases, networking etc :) I think as you go, it will become clearer what you like most, and what you want to focus on :)
3:35-
4:42-
6:10-
ML on new m1 chip ? Are you planning any video on it ?
I'm not planning to do it. My 16inch is good enough for me!
Data engineer vs backend engineer???? Thanks.
I've just been offered a Data Engineering position after applying for a software engineering position. Do you think skills learned in Data engineering transfer to Software Engineering and if so should I accept it if I'm sure I want to be a software engineer after graduating?
I would even say that data engineering is a type of software engineering. Definitely lots of transferable tech skills to learn (OO programming, databases, bash scripting, networking...). I'd say - go for it ;) Besides, it's very "hot" at the moment, and it will only be getting "hotter"!
@@karolinasowinska Btw, why are DE so 'hot' and well paid? It seems like just a kind of a software engineering role that does require any outstanding skillset and does not have high entry barrier.
I am a Sophomore ( Computer Science ) studying in India .I know how to build ETL Pipelines ,AWS,Machine Learning Algo's Implementation in TensorFlow and Keras,A Solid Knowledge in SQL(Post,Maria,Oracle,MySQL),Scripting (bash,PHP,Python) ,React & some backend development using NodeJs,PHP. But I am confused what to choose ML engineer or a Data Engineer.And Is it possible to get a direct Job in UK after my bachelor's in DE or ML Engineer ?(It would be a Great Help)
I think you have a lot of very releavnt skills/knowledge for both of these roles! Pick which one you prefer :) Both are in high demand in the UK, and I do think it is possible to get a job there after graduation. I can't advise on the visa process, because I've never gone through that. All the best! ;)
@@karolinasowinska Thanks A Lot!!!
It does suit you 😉
Wow, I've missed your comment for a long time haha! Thanks :)
Hi very nice explanation. But is it a good time to switch now to machine learning or am I too late?
Good question - personally I don't think it's too late. But be prepared that you'll have to constantly learn new things! :)
Should I go to University for a data engineering degree??
Sister, which one has more scope in future AI and ML or AI and datascience ,past few days I am in extreme dilemma kindly help me please ..
Hi karo👋🏻
Hello there! ;)
I am From Pakistan ,Here it is extremely difficult to find internship especially when you are not completed bachelors , i am currently doing A levels ,I know python ,sql , python libaries , please guide me how could i got intership in datascience . i am learning datascience and terminologies for last 1.5 years when i was in O levels but havenot get any value out of it , so dissapointed i am that i am seriously considering to switch to webdeveliopmnt tasks as thwere are grets op[portunites especially for fresher or inecperience younsters like me , do you think it would b e a good chioce
Don't get discouraged by the initial setbacks! Keep looking for those opportunities. Perhaps if not in your country, then abroad? Unfortunately I don't know the Pakistani job market, so I can't really advise you on that :(
Do you go for data science job in future ?
Who knows where the future will take me!
@0:01You look good no matter what side to the camera ;)
The 2020 award for the bestest and nicest cousin goes to...Robi! ;D
@@karolinasowinska Awwwww
The beautiful girl is back. Let us welcome her.
How nice of you! :)
Thank you for welcoming me. Lolol Jk I’m a guy
@@karolinasowinska 😍
I hope im as smart as you one day :)
Oh, knowing some programming languages doesn't make a person ultra smart! Btw, I'm super happy to see you in the creator's hat, I'm going to check out your video in a sec! ;)
@@karolinasowinska thanks for subbing :) I didn't even know I had comments off.. its tough for an infp out here.. 😂 you sure are sweet. Thank you.
Engineering seems less masochistic than scientists. Too much math IMO
True, the math struggle is real in science
Beep boop
Beep boop!
thank you, cs gigi hadid