Really interesting video! It definitely relates to what I am working on currently. I work in IT recruiting. I'm always looking for a data engineers for my jobs and this video helps me understand a little more about what data engineers have to face in their field. Thank you!
It is such a fantastic talk! It summarizes all the good practices for modern data management and puts them into the right perspective so that "old school" BI/DW people can understand.
i dont mean to be so off topic but does anyone know of a trick to log back into an Instagram account?? I stupidly lost the login password. I would appreciate any tips you can offer me!
Overall not a bad presentation. Although, I do not think using Presto with 1 million per day snapshot on dimensions is viable (specifically out of box). Presto likes to put all joins onto the main fact into memory, modelling this way and using Presto isn't really in line with how it was meant to be used if your joining all the records each day. But in general, I do like the shift of keeping everything in dimensions if the technology allows for it. Edit: The Q&A section talked more to this, he adapted his answer to "apply common sense"
A persistent staging area might only be effective when there is one place where your raw data resides. For example if you stage your data on AWS S3 and then copy the data again to snowflake, you will end up having two places of storage with the same raw data.
Talked a bout a lot of interesting stuff but I had a hard time really figuring out what is meant by functional engineering. Each individual topic was very clear for me, but not the red thread linking it all together.
Thanks a lot for the video. I have a question. For modern datawarehouse solutions are there any other data models emerged apart from starschema, snowflake ?
Thanks for sharing your great knowledge with us again. One question about snapshotting let's say daily. What if multiple changes in dim_supplier happen, then you wouldn't catch these once. Would that just be a small tradeoff you would accept or would you have something else in mind to track that?
If keeping track of intra-day changes is important I'd recommend denormalizing the dimension attribute into the fact table. Conceptually if some dimensional attributes are "flickering", meaning going back and forth and changing fairly often, it's pointing towards that attribute being logically attached to the fact and not to the dimension.
Interesting talk. Nice to see a tech video where they display the slides as well as the speaker.
Interesting the approach to the slowing change dimension. Data storage nowadays are cheap, the time and price of engineering is a lot more expensive.
Really interesting video! It definitely relates to what I am working on currently. I work in IT recruiting. I'm always looking for a data engineers for my jobs and this video helps me understand a little more about what data engineers have to face in their field. Thank you!
It is such a fantastic talk! It summarizes all the good practices for modern data management and puts them into the right perspective so that "old school" BI/DW people can understand.
i dont mean to be so off topic but does anyone know of a trick to log back into an Instagram account??
I stupidly lost the login password. I would appreciate any tips you can offer me!
wow, I started to understand those concepts so much better now
Overall not a bad presentation. Although, I do not think using Presto with 1 million per day snapshot on dimensions is viable (specifically out of box). Presto likes to put all joins onto the main fact into memory, modelling this way and using Presto isn't really in line with how it was meant to be used if your joining all the records each day. But in general, I do like the shift of keeping everything in dimensions if the technology allows for it.
Edit: The Q&A section talked more to this, he adapted his answer to "apply common sense"
Really great talk! Help me to understand many new concepts
A persistent staging area might only be effective when there is one place where your raw data resides. For example if you stage your data on AWS S3 and then copy the data again to snowflake, you will end up having two places of storage with the same raw data.
Talked a bout a lot of interesting stuff but I had a hard time really figuring out what is meant by functional engineering. Each individual topic was very clear for me, but not the red thread linking it all together.
Great efforts thanks @maxime
Great talk. And quite the resemblance to the singer from 311.
Awesome talk! Thanks for taking the time to share some best practices from top the talent!
Great talk!
Thanks a lot for the video. I have a question. For modern datawarehouse solutions are there any other data models emerged apart from starschema, snowflake ?
data vault and incremental approach are kind of emerging as norms in some quarters
Thanks for sharing your great knowledge with us again. One question about snapshotting let's say daily. What if multiple changes in dim_supplier happen, then you wouldn't catch these once. Would that just be a small tradeoff you would accept or would you have something else in mind to track that?
If keeping track of intra-day changes is important I'd recommend denormalizing the dimension attribute into the fact table. Conceptually if some dimensional attributes are "flickering", meaning going back and forth and changing fairly often, it's pointing towards that attribute being logically attached to the fact and not to the dimension.
I think you use changing history to catch this changing, then as Maxime Beauchemin, this changing history should be in fact table.
🤯
Great talk!
Maha Guru!
honestly did not get much from this talk - pretty technical