Microsoft Fabric - Incremental ETL

Welcome to today's episode of the Microsoft Fabric MVP video series!
Amit Chandak joins Ryan to talk about incremental ETL in Fabric.
Meet Amit:
Twitter: / amitchandak78
TH-cam: ‪@AmitChandak‬
Fbaric TH-cam Series: • Microsoft Fabric: A Br...

[@82ndAirborneTrooper]

FABRIC...utterly awesome - unbelievable ecosystem integrations with on the fly build-in/code-in logic functionality. Thank you, Microsoft Team.

[@Milhouse77BS]

Maybe add the "incremental ETL" subject to the video title?

[@bheatrizurias8427]

KINGOOO

[@thepakcolapcar]

hello @amit
When I follow the steps, in power BI I see the "Trial" option as disabled adn by default it has selected my "Pro" licence.
However on top it shows me "PPU trial: 59 days left". Is that how it is uspposed to be?
Further as I proceed, and try to create Lakehouse, it gives me a message asking to upgrade to free Fabric tiral capacity.

[@Babayagaom]

can anyone tell me in this case what is Medallion Architecture DW will be gold layer, lake will be bronze and which one will be silver layer? also is it necessary to create medallion arch everytime ?

[@marimuthukalyanasundram3151]

Kudos Amit, Its excellent explanation about Incremental load in Fabric how data engineers can use in real time project. More exciting to know about SCD Type-2 implementation in Fabric without using the store procedure, needs to be handle in Dataflow activities. I Compared with Synapse Data flow activities but SCD Type-2 options are not available.

[@AmitChandak]

I have used the stored procedure to implement SCD, but I am looking forward to the built-in capability in Fabric: th-cam.com/video/2zEJ62W_rIQ/w-d-xo.html
I was able to append data without using a stored procedure, but SCD without a stored procedure is more interesting-
How to use Append data option of Data Flow Gen 2 and Data Pipeline| Incremental- th-cam.com/video/E5axX6auBN4/w-d-xo.html

[@VivekDevalkar]

How do you do incremental ETL based on modified date i.e. I want updated source records to be updated in the destination and new records to be inserted. Would that basically need to be handled at a stored procedure level using perhaps a merge statement or is there is better/different way to do that within Fabric?

[@spacespectale]

u did not tell from where did you get server name for postgre sql in fabric

[@ramusubbu3778]

It’s not the purpose.

[@spacespectale]

@@ramusubbu3778 then how can we know if we want to practice