Very interesting stuff. Is there any support planned for synchronising back AWS Glue Catalog when using Snowflake-managed? That is not to try and make the AWS catalog "authoritative" - and so hopefully avoiding the problem you describe at the end - but rather just to keep that metadata up to date such that the Snowflake-managed Iceberg table can be queried by other AWS services. Or does that kind of synchronisation simply not work?
In today's news, Apache Iceberg Tables are now in public preview! Check out the details here: www.snowflake.com/blog/build-open-data-lakehouse-iceberg-tables/
20:24 o…k… so adding the HIVE API (which other vendor have done) to your catalog is a bad solution, so cannot for instance plug Athena to your catalog to read your iceberg managed tables, and on top of that you cannot move switch a managed table to unmanaged tables! Please let me know how this is addressing your customers feedback? Are do you address customers concerns about vendor locking with you closed solution which is only make believes of openness? So I can build a Lakehouse with Snowflake but that will not be an open lakebouse like for instance with Dremio!
A Hive API is fine, but an old solution and often a crutch vendors use when they want to lock you into a catalog and also the Spark ecosystem. We have no intention on locking people into the catalog, so just slapping an (old) Hive API on our catalog is not the best approach.
Interesting features but again poor execution! Each time some data are added or each time you need to read from the external iceberg table, you have to do an alter table to get the new set of manifest files sync up so you can access the latest data! Yeah that a great demo to showcase a poor execution which doesn’t consider data team experience what so ever! Feel you ask for it, we’re giving it to you and we’ll figure out later how to fix this additions work you now have to do! That doesn’t solve the data onboarding feedback from your customers!
![Apache Iceberg on AWS with S3 and Athena [FULL COURSE IN 30MIN]](http://i.ytimg.com/vi/iGvj1gjbwl0/mqdefault.jpg)
Very interesting stuff. Is there any support planned for synchronising back AWS Glue Catalog when using Snowflake-managed? That is not to try and make the AWS catalog "authoritative" - and so hopefully avoiding the problem you describe at the end - but rather just to keep that metadata up to date such that the Snowflake-managed Iceberg table can be queried by other AWS services. Or does that kind of synchronisation simply not work?
Very informative talk, thank you!
Learnt how Snowflake and iceberg integration works through AWS glue /Athena
Glad it was helpful!
When is it GA?
In today's news, Apache Iceberg Tables are now in public preview! Check out the details here: www.snowflake.com/blog/build-open-data-lakehouse-iceberg-tables/
when is it going to be GA
While I cannot give a specific answer, we are working on making it PuPr and then GA ASAP.
21:44 seriously!
20:24 o…k… so adding the HIVE API (which other vendor have done) to your catalog is a bad solution, so cannot for instance plug Athena to your catalog to read your iceberg managed tables, and on top of that you cannot move switch a managed table to unmanaged tables!
Please let me know how this is addressing your customers feedback?
Are do you address customers concerns about vendor locking with you closed solution which is only make believes of openness?
So I can build a Lakehouse with Snowflake but that will not be an open lakebouse like for instance with Dremio!
A Hive API is fine, but an old solution and often a crutch vendors use when they want to lock you into a catalog and also the Spark ecosystem. We have no intention on locking people into the catalog, so just slapping an (old) Hive API on our catalog is not the best approach.
Interesting features but again poor execution!
Each time some data are added or each time you need to read from the external iceberg table, you have to do an alter table to get the new set of manifest files sync up so you can access the latest data!
Yeah that a great demo to showcase a poor execution which doesn’t consider data team experience what so ever!
Feel you ask for it, we’re giving it to you and we’ll figure out later how to fix this additions work you now have to do! That doesn’t solve the data onboarding feedback from your customers!
? That is actually not the case - that is specifically why there is a catalog integration. :)