Announcing Delta Live Tables with Demo | Michael Armbrust | Keynote Data + AI Summit NA 2021

Distinguished Engineer Michael Armbrust announces Delta Live Tables, making it possible to do production-quality ETL using only SQL queries. Live Tables runtime takes care of operational, governance and quality concerns, allowing you to spend more time getting value from the data. Can even mix Python with SQL to do advanced analytics and AI. Learn more at databricks.com/.
Register for free to see the rest of the keynotes and exciting announcements live, plus over 200+ sessions. Learn from the creators and top contributors of technologies like PyTorch, TensorFlow, MLflow, Delta Lake, Apache Spark, Hugging Face, DBT and more.
databricks.com...
Connect with us:
Website: databricks.com
Facebook: / databricksinc
Twitter: / databricks
LinkedIn: / databricks
Instagram: / databricksinc Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. databricks.com...

[@chelseaip3120]

This is gold tier

[@maraboinaravi3183]

Hi Michael Armbrust, This is really great, but how it is different from existing structure streaming with cloud files? Also how to create delta live tables from other data sources like Kakfa.

[@VibhakarSinha]

very helpful

[@gustavemuhoza4212]

How is this different from what AWS Glue Crawlers do when you set them to update schemas or add colums and stuff like that?

[@volodymyrb7380]

Great idea, thanks! One more question - is it possible to submit live-tables related code from local machine with Databricks connect? Suppose I have egg/wheel package and don't want to bound my code to notebooks, while still want smooth development process (w/o deployments on every change)

[@berkerkozan3659]

When we are reading json files incrementally, does it do it in an efficient way or just list all the files and pick the unprocessed ones like glue!?

[@Databricks]

yes, you can set up DLT to use SNS/SQS notifications to efficiently identify new files rather than file listing.

[@cchalc-db]

So does this only work on the delta format?

[@MrInk113]

"Injested"......😞 Ingested bro.