AWS Services used for Data Engineering | AWS Services that every Data Engineer must know

แชร์
ฝัง
  • เผยแพร่เมื่อ 30 ก.ย. 2024
  • In this video , i discuss the important AWS Services which are used in Data Engineering projects.
    The following services are discussed :
    1. Data Ingestion / Streaming
    AWS Kinesis Data Streams
    AWS Kinesis Firehose
    AWS Database Migration Service
    AWS MSK ( Manages Service for Kafka)
    2. Data Processing / ETL
    AWS Glue
    AWS Lambda
    AWS EMR (Elastic MapReduce)
    AWS Batch
    3. Data Storage / Database
    AWS S3
    AWS RDS
    AWS DynamoDB
    AWS AuroraDB
    4. Data Warehouse / Analytics
    AWS Redshift
    Athena
    AWS Redshift Spectrum
    5. Orchestration and Workflows
    AWS Step functions
    AWS MWAA (Managed workflows for Apache Airflow)
    AWS Data Pipeline
    6. Version Control and CI/CD
    AWS CodeCommit
    AWS CodePipeline
    #aws #dataengineering #dataengineer

ความคิดเห็น • 2

  • @kanisetti1
    @kanisetti1 หลายเดือนก่อน

    It would be great if you can do a video with an example touching one component in each section, TIA

  • @universal4334
    @universal4334 3 หลายเดือนก่อน

    Please do videos on redshift and redshift spectrum. What is it and how does it works. Especially redshift spectrum, how does it query data without actually loading data and etc.