Processing Delta Lake Tables on AWS Using AWS Glue, Amazon Athena, and Amazon Redshift

Delta Lake is an open source project that helps implement modern data lake architectures commonly built on cloud storages. With Delta Lake, you can achieve ACID transactions, time travel queries, CDC, and other common use cases on the cloud.
There are a lot of use cases of Delta tables on AWS. AWS has invested a lot in this technology, and now Delta Lake is available with multiple AWS services, such as AWS Glue Spark jobs, Amazon EMR, Amazon Athena, and Amazon Redshift Spectrum. AWS Glue is a serverless, scalable data integration service that makes it easier to discover, prepare, move, and integrate data from multiple sources. With AWS Glue, you can easily ingest data from multiple data sources such as on-prem databases, Amazon RDS, DynamoDB, MongoDB into Delta Lake on Amazon S3 even without expertise in coding.
This session will demonstrate how to get started with processing Delta Lake tables on Amazon S3 using AWS Glue, and querying from Amazon Athena, and Amazon Redshift. The session also covers recent AWS service updates related to Delta Lake.
Talk by: Noritaka Sekiyama and Akira Ajisaka
Here’s more to explore:
Why the Data Lakehouse Is Your next Data Warehouse: dbricks.co/3Pt...
Lakehouse Fundamentals Training: dbricks.co/44a...
Connect with us: Website: databricks.com
Twitter: / databricks
LinkedIn: / databricks
Instagram: / databricksinc
Facebook: / databricksinc

[@illiakaltovich]

Hi, guys, a big thank you for the detailed demo.
However, some aspects are not clear to me: why were no glue partitions created using "Create native tables" compared to "Create Symlink tables," and why does AWS represent the classification as parquet in the second case (but previously at the crawler level we explicitly set that we expect to see Delta)?

[@la_da_kid]

Extremely based