The Top AWS Services A Data Engineer Should Know In 2024

In this video we take a look at top AWS services you should know as a data engineer. We cover a use case from ingestion through to analytics looking at the best ways to orchestrate our pipelines.
SUPPORT THE CHANNEL:
ℹ️ Udemy Practice Exams: www.udemy.com/course/practice...
☕ Buy Me A Coffee: www.buymeacoffee.com/johnnych...
🖥️ My VPN: go.nordvpn.net/aff_c?offer_id...
▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬
00:43 - Ingest
02:22 - Storage
03:28 - Analytics
04:28 - Orchestration
05:29 - Monitoring & Discoverability
06:08 - AI/ML
06:49 - Outro
The video covers realtime ingestion using Amazon Kinesis as well as batch ingestion in with AWS Lambda, AWS Glue and Amazon EMR. We look at how we can store this data in Amazon S3, Amazon DynamoDB and Amazon DynamoDB before using Amazon Quicksight to build dashboards.
😎 About me
I have spent the last decade being immersed in the world of big data working as a consultant for some the globe's biggest companies.My journey into the world of data was not the most conventional. I started my career working as performance analyst in professional sport at the top level's of both rugby and football. I then transitioned into a career in data and computing. This journey culminated in the study of a Masters degree in Software

[@SharequeSRQ]

Dude, your videos are so helpful, I got a Data Engineer job after practicing with your videos and they are still helpful.. More power to you man, I hope you get more success.

[@ccc_ccc789]

Thanks!

[@LoveisHell85]

Love your vids. Could you maybe do a vid on airflow hosting on fargate + simple pipeline? Something practical

[@cuatrofour4]

Thanks for the video!
What do you think about using ECS/EKS to run your python ETLs inside docker containers? so you can execute your tasks/pods from MWAA after. In case you don't need spark, could be an alternative to EMR and cheaper than Glue.