I don't disagree with bronze/silver/gold in the file path. The suggestion to associate with a project or domain within the lakehouse file path makes sense. I will note that bronze/silver/gold can have unintended meanings so using raw/processed/enriched may be more clear.
Jeremy, I agree and would probably name it differently if I started over. From my perspective there is quite a bit of freedom to organize different steps of a project based upon organizational norms. Naming conventions, User access per Security requirements, Organizing Multiple Data Lakes, hierarchical organization patterns in Data Lakes, CICD patterns, etc can all be different. One piece of advice I'd hold firm is to standardize on an architectural pattern that can scale across collaborative teams and projects to prevent a data lake from becoming a data swamp.
At the time it was a simple way to add a Date table to the Gold layer. It also shows that you can strategically unite data from different sources with a Lakehouse architecture. I'd probably do it differently today with Fabric.
Awesome!!
41:57 You placed the output file name in wrong Data flow, it must to be in BronzetoSilverGeo
Thank you! I'll take a look and see if I can fix it.
I don't disagree with bronze/silver/gold in the file path. The suggestion to associate with a project or domain within the lakehouse file path makes sense. I will note that bronze/silver/gold can have unintended meanings so using raw/processed/enriched may be more clear.
Jeremy, I agree and would probably name it differently if I started over. From my perspective there is quite a bit of freedom to organize different steps of a project based upon organizational norms. Naming conventions, User access per Security requirements, Organizing Multiple Data Lakes, hierarchical organization patterns in Data Lakes, CICD patterns, etc can all be different. One piece of advice I'd hold firm is to standardize on an architectural pattern that can scale across collaborative teams and projects to prevent a data lake from becoming a data swamp.
Why did you use date.csv file in this ?
At the time it was a simple way to add a Date table to the Gold layer. It also shows that you can strategically unite data from different sources with a Lakehouse architecture. I'd probably do it differently today with Fabric.