Data pipeline design - Top 10 most viewed products
ฝัง
- เผยแพร่เมื่อ 20 ส.ค. 2024
- In this video, we will dive deep into use case of designing batch data data pipeline. We will go in detail on how to approach designing a pipeline in multiple ways given input requirements.
Useful Resources:
AWS SNS:aws.amazon.com...
AWS SQS:aws.amazon.com...
AWS Lambda:aws.amazon.com...
AWS Glue:aws.amazon.com...
AWS Kinesis:aws.amazon.com...
AWS OpenSearch:aws.amazon.com...
AWS Quicksight: aws.amazon.com...
Whether you're preparing for a job interview at Big Tech companies or simply looking to enhance your data engineering skills, this video will be helpful.
Don't forget to like this video, subscribe to Data Guy and hit the notification bell so you never miss our latest content. Let's start learning and growing together!
It would be better if you also include a design with spark , EMR , S3 , Hive/Delta lakehouse architectures so that more people can relate.
Definitely, will try to comeup with videos covering the same architecture using different services for people to relate. Thanks for the comment.
the video of system design framework, Realtime pipeline design and this video is very good and upto the mark. BUT there are something that you can improve and will be helpful.
1. font size is very small please zoom in while recording.
2. increase the voice
3. the video quality gets better on 720p and using 360p feels like 144p sometimes. kindly address this
4. you are designing pipeline with AWS, kindly give reference of other cloud platforms too that if we use AZURE or GCP how this pipeline will change. this helps lot to understand.
As I'm azure data engineer some where i find it difficult to connect
Thanks for the feedback - agreed, recording quality isn't up to the mark. Will definitely improve it and yes - will make videos covering all 3 cloud platforms.