Hello, thanks for sharing walkthrough. I have a question on the same. How do we copy multiple files from SPO to data lake? If multiple files are different type ex. excel , csv...then what is the option...Thanks in advance.
This is a great intro, but how would we extend this to download multiple files from a SharePoint document library. Specifically, if we have a document library with a mixture of file types and names, and we only want retrieve all .CSV with name format of YYYY-MM-DD.csv.
What if we have many nested folders inside document drive and we want to copy all files to our azure data lake storage using ADF and maintaining the same hierarchy there also.
Hi . Thanks for the tutorial. But when I am trying by using Power Automate it is giving me the below error : This request is not authorized to perform this operation using this permission. clientRequestId: 18961521-a002-49dc-8cfd-bec1d0dd25ff
One idea for incremental loads is using Azure Data Factory or the Pipelines in Azure Synapse (basically the same ETL tool) and using something like a date column with update date\time in the SharePoint file to determine what has changed. There are templates in Azure Data Factory to do this type of thing.
@@SQLTalk Thanks . 0:46, when Clicked 3 dots in front of folder, > , Automate > then it shows Create a rule or manage rule but no options to Power Automate. I checked being a GAdmin i can access Power automate portal but how to get power automate there now ?
Hey Kirby. Can you please take us through the metadata ingestion task. If you are able to do this, please reply to this comment and we can get in contact privately. Thanks in advance.
Thank you! Super helpful!
Hello, thanks for sharing walkthrough. I have a question on the same. How do we copy multiple files from SPO to data lake? If multiple files are different type ex. excel , csv...then what is the option...Thanks in advance.
This is a great intro, but how would we extend this to download multiple files from a SharePoint document library. Specifically, if we have a document library with a mixture of file types and names, and we only want retrieve all .CSV with name format of YYYY-MM-DD.csv.
0:45 i am wondering how you added Power Automate @kirby's Sql talk
Hey, This is great but I am wondering how do I achieve other way around ( ADLS/Blob Storage to Sharepoint )
What if we have many nested folders inside document drive and we want to copy all files to our azure data lake storage using ADF and maintaining the same hierarchy there also.
Thank you!
Can We auto trigger using ADF, whenever the file is updated ?
Hi I am interested in metadata copy activity as well. Can you explain that as well
Hi how to move all files inside a folder in SharePoint?how to create relative URL?
Can you please explain for data ingestion from SharePoint list to Adls ?
Hi . Thanks for the tutorial. But when I am trying by using Power Automate it is giving me the below error :
This request is not authorized to perform this operation using this permission.
clientRequestId: 18961521-a002-49dc-8cfd-bec1d0dd25ff
I too got the same error 403 while browsing folder id in sharepoint.
@@shameershaik4754 you need access to the sharepoint site, not just the document you are trying to copy
@@jean2901 I ran into the same issue using Power Automate following the steps in the video, and received the same error at the 2nd step.
Hi. Thanks for the tutorial. Can we do incremental load into SQL warehouse if data in sharepoint has new rows?
One idea for incremental loads is using Azure Data Factory or the Pipelines in Azure Synapse (basically the same ETL tool) and using something like a date column with update date\time in the SharePoint file to determine what has changed. There are templates in Azure Data Factory to do this type of thing.
@@SQLTalk Thanks . 0:46, when Clicked 3 dots in front of folder, > , Automate > then it shows Create a rule or manage rule but no options to Power Automate. I checked being a GAdmin i can access Power automate portal but how to get power automate there now ?
@@SQLTalk Can you create detail video on steps before ADF as its not clear.
Hey Kirby. Can you please take us through the metadata ingestion task. If you are able to do this, please reply to this comment and we can get in contact privately. Thanks in advance.
Thank you!