Utilizing Spark as the Analytical Core to an Open Source HTAP Relational Database: John Leach

Utilizing Spark as the Analytical Core to an Open Source HTAP Relational Database on HBase
Splice Machine utilizes Spark on Yarn as the analytical execution architecture for our open source HTAP relational database. This talk will walk through how a dual engine architecture can exist where Spark supports analytical queries and large database maintenance operations (HBase Compactions, Index Maintenance, imports) while an operational relational architecture mirrors the analytical architecture. The presenters will demonstrate how to maintain ACID properties during both real-time and spark analytical read/write operations.
The talk will explain how advanced RDBMS topics like constraints, foreign keys, triggers, and secondary indexes can still occur using Spark’s dataset interface and what use cases are enabled by fusing ACID properties with large scale elastic compute.

[@RushyNova]

Hey John, awesome presentation. You're extremely knowledgeable and provide clear, concise explanations. Are there any plans to incorporate the cost optimiser and sketching within the Apache Spark project?