Good video. It's worth having a peek at Delta Lakes Liquid Clustering offering, and by default it does clean up the version history. I don't know if that's a good or a bad thing. I think it's bad. Anyway great work! Might be outdated a bit though. Both techs seem to be arms racing against each other.
@johnnychivers Good video, but often you are not interacting with this layer directly. In databricks for example, I can write delta tables using SQL syntax etl to the same effect. I get this is a native review, but few will be interfacing directly.
Hi Rory, It really depends on the organisation. I have experience working with many businesses that do operate this layer themselves, and do not use the likes of Databricks. These businesses are running TBs of data through these self built data lakes on a daily basis. On the flip, there are other businesses which do use the likes of Databricks. If this is where you are more familiar it is still good to have an understanding of how the storage layer works under the hood.
![Apache Iceberg on AWS with S3 and Athena [FULL COURSE IN 30MIN]](http://i.ytimg.com/vi/iGvj1gjbwl0/mqdefault.jpg)
![Apache Iceberg on AWS with S3 and Athena [FULL COURSE IN 30MIN]](/img/tr.png)
Good video. It's worth having a peek at Delta Lakes Liquid Clustering offering, and by default it does clean up the version history. I don't know if that's a good or a bad thing. I think it's bad.
Anyway great work! Might be outdated a bit though. Both techs seem to be arms racing against each other.
In the company I work we use Hudi and I can say that's a pain in the ass to make that shit working properly
Another good video from the Chiverse.. :)
@johnnychivers Good video, but often you are not interacting with this layer directly. In databricks for example, I can write delta tables using SQL syntax etl to the same effect.
I get this is a native review, but few will be interfacing directly.
Hi Rory,
It really depends on the organisation.
I have experience working with many businesses that do operate this layer themselves, and do not use the likes of Databricks. These businesses are running TBs of data through these self built data lakes on a daily basis.
On the flip, there are other businesses which do use the likes of Databricks. If this is where you are more familiar it is still good to have an understanding of how the storage layer works under the hood.
Good one. Thank you.
Thank you too!
Is this guy Scottish or Jamaican? Never heard an accent like this before it’s wild
Iceberg > delta lake > hudi
Good one..Thank you