What Does a Data Engineer Do? Explained Simply
ฝัง
- เผยแพร่เมื่อ 18 พ.ค. 2024
- Combo Package Python + SQL + Data warehouse (Snowflake) + Apache Spark: com.rpy.club/pdp/yYnEMzLOX?pl...
USE CODE: COMBO50 for 50% off
Learn more about courses here - datavidhya.com/courses
The Data Engineering market is going to cross the $100 Billion mark by 2028, this is the best time to learn about data engineering and start your career, in this video we will understand what a data engineer really do and skills required for data engineering
👦🏻 My Linkedin - / darshil-parmar
📷 Instagram - / datawithdarshil
🎯Twitter - / parmardarshil07
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
3 Books You Should Read
📈Principles: Life and Work: amzn.to/3HQJDyP
👀Deep Work: amzn.to/3IParkk
💼Rework: amzn.to/3HW981O
Tech I use every day
💻MacBook Pro M1: amzn.to/3CiFVwC
📺LG 22 Inch Monitor: amzn.to/3zk0Dts
🎥Sony ZV1: amzn.to/3hRpSMJ
🎙Maono AU-A04: amzn.to/3Bnu53n
⽴Tripod Stand: amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: amzn.to/3IRpiL2
👉Data Engineering Complete Roadmap: • Data Engineer Complete...
👉Data Engineering Project Series: • Data Engineering Proje...
👉Become Full-Time Freelancer: • Best Freelancer Series...
👉Data With Darshil Podcast: • Podcast Series - Data ...
✨ Tags ✨
✨ Hashtags ✨
Hi Darshil, Please make a series on AWS cloud server less API and datalake development using python. Where glue jobs should be developed using pyapark, source data must be fetched from msk connectors, upload data in s3, create Iam users, using AWS boto3, unit testing using moto3.
Maintain db and tables in lakeformation using lf-tags.
Athena automation for visualization.
Please make series complete based on programming automation
Some complex project on Microsoft Fabric ...
is this course availabe on web OR app ? Is course can be accessible anywhere ?
Wow you explained my job in just 10 to 11 mins in a smoother way. Much appreciated Darshil❤
Glad to hear that
aaj sabse pehle is channel ki main main saari videos dekhunga....
Hey bro books vs online tutorial/udemy course which one is best for beginners to intermediate ❤
Hii darshil may i know please
Whas is the duration of apache spark +databricks course??
How to collect data generated from survey websites and derive insights??
Can you recommend books on python, sql and linux?
I've got aws dva certificate 1 year back, but I didn't worked/practiced on aws. And am planning to switch product based company. Please guide me where to start my learning.
Hello Darshil Sir,
Is it possible to switch career from one domain to another domain.
Currently I am in Accenture but want to change my career to another domain...could you please guide me
Hi Darshil,
I am from a non technical background having BCom(Hons) degree and right now I am having almost 3 years of experience as Data Analyst with skill set of SQL, Power BI and Python (still learning). I have recently left my job and looking for new opportunities for Data Analyst. Also am aspiring for a Data Engineer role in future. I am thinking to get the job first in Data Analyst role and then I'll start learning for Data Engineer. Is this a good way to follow or shall I start preparing immediately for Data Engineer?
this is a good video
I am targeting data engineer role. I have aws solutions architect associate. Lack good grasp over python and sql.
Shall I focus on python, sql first or go for aws data engineer associate and databricks cert?
Love you Brother ❤️
Thank you
Hi great information shared as always. Btw did you start taking protein shake and working out? Just curious since you are looking more buffed
Haha
I am working out from last 3 years but most of the time I wear long sleeves this tshirt is tight so it makes me look buff
Great video but next time decrease background music or remove it
1view love from Lucknow brother ❤
Thank you!
great
hey darshil is there any way I can get a discount on your 4 course combo as I am a student so it is difficult for me to pay the full fee.Thanks
Please email me at info@datavidhya.com
hit that LIKE button and let me know what do you want to see next?
Hii darshil may i know please
Whas is the duration of apache spark +databricks course??
I am curious what is the difference between a data engineer and a business intelligence developer?
www.linkedin.com/posts/darshil-parmar_dataengineer-datascience-dataanalyst-activity-7196031026966495232-gzE3?
Hey bro; for data scraping selenium is a good option or not
No, lot of website have bot protection now, not worth doing
Then what is the good suggestion for web scraping
@@subratparida5647 No web scraping go something else like data analyst/engineer or SDE
@@DarshilParmar Thank you for your suggestion 😀
Hey Darshil I have been in Data engineering field for 1.5 yrs, Is there any research opportunity/ scope in data engineering?
How you got your 1st job as fresher.?
Hey can you please tell me how you got started with this field.. I'm currently in college and been studying ML/DL mostly but now I'd like to expand my domain.. Is it even worth it and if so then how to I go about it.. Will really appreciate if you take some time and give an in depth response
@@sunilKumar-ci9vn hi it was campus placement only. But the role in company was data engg
@@Rhly001 see if u r already doing Ai ml for a long time, i suggest not to spent time in data engg. But if u want to then sql (all joins and when to use them, then group by , combination of group by and joins, self join, once done with these then windowing functions) - practice from leetcode 50 basic sqls... Python only upto lists.. shell scripting to automate..
Here , mostly u will work on sql only. Python+shell scripting to automate tasks.
@@Rhly001 bhai youtube is pathetic it's deleting my reply
Sir what is the duration of your zero to data engineering course ?
Recorded content around 50hours for 4 courses with lifetime access, more will be released soon
Hi sir i have applied for many companies by watching your videos and doing your projects and making changes but donk know where I am lacking behind
drive.google.com/file/d/1hVVoqvitd2wCSFv44f2aVWew2Cbgpg6-/view?usp=drivesdk
Check resume and portfolio projects they are very important
@@DarshilParmar drive.google.com/file/d/1hVVoqvitd2wCSFv44f2aVWew2Cbgpg6-/view?usp=drivesdk
@@DarshilParmar here's my resume sir drive.google.com/file/d/1hVVoqvitd2wCSFv44f2aVWew2Cbgpg6-/view?usp=drivesdk
drive.google.com/file/d/1hVVoqvitd2wCSFv44f2aVWew2Cbgpg6-/view?usp=drivesdk
Bro... I got all the skills which are you mention in your thumbnail... I will finish my final semester exam in first week of june... And still unplaced... Entry level job pe bhi 3-4years ka experience maang rhe h🙃😭
Try Data Analyst position or get internship
R uh stay in Mumbai ?
@@prathamgupta9256 same here mumbai
Who edits your videos
I hired new editor, what happend?
They are good, i mean how they use exact flow diagrams , are they teach persons? Or you have to collaborate with them to make it happen?
Hey there, am new to your channel & your study info on Data Engineer and the motivation you give is really appreciated⭐
One clarity/query plz which one is suggestible to set as a career is it Data Analyst or Data Engineer (Azure) plz suggest...
Starting with analyst is much easier than engineering
Bro do you know dbt ?
Yes
@@DarshilParmar thank bro, I am runing the dbt models with databricks and I want to know the running time(I am able to see the running time in VS code) but I want see the running time and bytes consumption in databricks for each models. Is this possible bro?
I want to connect with you for discount offer? is it possible?
Please email me at info@datavidhya.com
Data engineering is not what I think
But you are exactly what I thought
You keep milking the topic data engineering rather than telling something else on data engineering
Bro, I have 8 projects on the channel, 130+ videos
My goal is to target just beginners and fresh to get started, there is no point in going on advanced level stuff because that won't work in this case
10min series to explain Spark, Airflow, Snowflake
Videos on new trends
End To End Projects
Tutorials
there are many things, I need to keep balance between different target audience so that everyone can understand different parts