Organize a Fabric Lakehouse using Medallion Architecture Design

Let’s explore the potential of the medallion architecture design in Microsoft Fabric. Organize and transform your data across Bronze, Silver, and Gold layers of a Lakehouse for optimized analytics.
The video is based on Microsoft Lab which supports "Organize a Fabric Lakehouse using medallion architecture design" module in Microsoft Learn (see links below).
In this video, you'll learn how to:
00:00 Intro
00:57 Create a Lakehouse
01:50 Upload data to the bronze layer
03:38 Transform data and load to silver Delta table
13:30 Explore data
32:13 Create a semantic model
35:22 Summary
** Useful links: **
Whole learning path: bit.ly/microsoft-fabric-learn...
Module: bit.ly/fabric-medallion
Lab03b: bit.ly/fabric-lab03b
** Socials: **
- 📓 GitHub: github.com/NowinskiK/
- 📢 Twitter: / nowinskik
- 📢 Twitter: / azure_player
- 👥 Facebook: / theazureplayer
- 🏙️ LinkedIn: / kamilnowinski
- 🖼️ Instagram: / kamilnowgeek
- ⭐️ Microsoft MVP Profile: mvp.microsoft.com/en-us/Publi...
** Hungry more? Learn with me **
- 👨‍🎓 Courses: learn.azureplayer.net/
- 🌏 Blog: AzurePlayer.net
- ✉️ Newsletter: AzurePlayer.net/newsletter/
If you're reading this, please leave a like and comment for the algorithm.
#MedallionArchitecture #ApacheSpark #DeltaLakeTable #DeltaLake #ELT #MicrosoftFabric
#BronzeLayer #SilverLayer #GoldLayer #DataLayers #notebooks #pyspark

[@manojshetty87]

Beautifully explained and demonstrated. I had a question, If we use plain SQL by selecting %SQL in the Notebook, can we do all the sql operations like Merge, add new columns which you did in video Or are there any limitations if we use SQL?

[@matheusduarte8022]

Great Video! Thanks. You have a great knowledge in coding.

[@KamilNowinski]

Thanks! I like to think coding is my second language, right after emoji 😉

[@pavara.maddumage]

Great video! Thank you for your time and effort. In the traditional warehouse, we use separate bronze, silver and gold schemas. Can't we do the same in Fabric?

[@KamilNowinski]

Thanks. Medallion architecture (AKA data design pattern) is rather applicable to Lakehouses. Traditional warehouses have/had layers like Staging and main DWH, but nothing should stop you to use B/S/G layers if you have justification for them. However, it doesn't make bigger sense, as you won't store unstructured data (e.g. files) in "bronze" layer in Warehouse. So, it's more about naming each stage (layer) and describe precisely their role in the whole data platform.