Glad to hear that you have learned a lot from my videos! :) For many files I would recommend using wildcards (e.g., *.csv) when copying them because copying thousands of files one by one can become very costly very fast.
I love your content but learn better interactive....is there any training you would recommend that's instructor led? I'm a software manager and i want to learn adf for my job. Been trying to follow but learn best from doing. Any suggestions?
Thanks for your feedback and question! I don't have any particular course in mind but I think udemy has many courses on ADF to choose from. I don't know any courses that would have a live instructor that would be ready to help with exercises and questions. Almost all of my ADF videos are tutorials at the same time that you can do along with the video and after that I would recommend to modify the pipeline a bit to try out what happens if x in the pipeline is changed. In my opinion the best way to learn ADF is to do pipelines that are going to actual use and have a mentor who would help and do some quality control.
Hi aleksi, may i ask some question, if i want to use variable and call the variable in data copy task, its possible ? For example in ssis i have variable ‘var,, just declare the variable, and then in data flow task i just using query like this ‘select * from mytable where descriptions =‘var’
This depends on your source. If you have SQL database type source you can use query when reading data. Then you would use variables in dynamic expression like this: @concat('select * from dbo.table1 where col1=',variables('var1')). Hope this helps you :)
Great video, but Im trying to copy data from parquet files into an Azure SQL DB from blob, which has a storage event trigger. How will the schemas map if Im using a generic parameter and not an actual file or table schema?
Thanks for your question! Hmm I don't know if I understood the situation correctly but you don't have to specify the schema when copying data to SQL DB. In that case it uses the columns in the file for the table structure. Also, event trigger itself has some properties that you can use in your logic and pass them as parameters for the pipeline if that is something that you are after here.
@@AleksiPartanenTech The copy activity wont ccopy my actual data to the SQL table, it copies the schema alone but my table is empty. I check the file itself and it has data.
Hi Aleski nice explanation. Please can we create one practical session on parameters separately showing difference how it is diferent @piplinelevel,@linkedservicelevel@datasets level so it would be more helpful..😇😇😇and which level situations w e need to use which one?
Thanks for you comment and feedback! I will consider if I do a follow up video where I dive deeper into parameters. Sorry for a late reply since I have been on a holiday for few weeks.
You deserve much more views! Please keep on creating practical and straight to the point videos like this.
Thanks! I will keep making more videos :)
@jarker3107 I agree.
@AleksiPartanenTech Erinomaisia opastuksia, kiitos!
@@janisdata7874 kiitos!
Please continue making these
Thanks, I will! :)
Tutorial is great. However during the practical pls change your background to white for better visibility
Thanks! I have also noticed the same issue and in my new videos I try to remember it
I'm a newbie learn a lot from your videos.Thank you so much for the videos. But How can you loop through thousands of files?
Glad to hear that you have learned a lot from my videos! :)
For many files I would recommend using wildcards (e.g., *.csv) when copying them because copying thousands of files one by one can become very costly very fast.
I love your content but learn better interactive....is there any training you would recommend that's instructor led? I'm a software manager and i want to learn adf for my job. Been trying to follow but learn best from doing.
Any suggestions?
Thanks for your feedback and question!
I don't have any particular course in mind but I think udemy has many courses on ADF to choose from. I don't know any courses that would have a live instructor that would be ready to help with exercises and questions.
Almost all of my ADF videos are tutorials at the same time that you can do along with the video and after that I would recommend to modify the pipeline a bit to try out what happens if x in the pipeline is changed.
In my opinion the best way to learn ADF is to do pipelines that are going to actual use and have a mentor who would help and do some quality control.
Hi aleksi, may i ask some question, if i want to use variable and call the variable in data copy task, its possible ? For example in ssis i have variable ‘var,, just declare the variable, and then in data flow task i just using query like this ‘select * from mytable where descriptions =‘var’
This depends on your source. If you have SQL database type source you can use query when reading data. Then you would use variables in dynamic expression like this: @concat('select * from dbo.table1 where col1=',variables('var1')). Hope this helps you :)
@@AleksiPartanenTech thanks somuch about your answer very helping me 😁
Great video, but Im trying to copy data from parquet files into an Azure SQL DB from blob, which has a storage event trigger. How will the schemas map if Im using a generic parameter and not an actual file or table schema?
Thanks for your question! Hmm I don't know if I understood the situation correctly but you don't have to specify the schema when copying data to SQL DB. In that case it uses the columns in the file for the table structure. Also, event trigger itself has some properties that you can use in your logic and pass them as parameters for the pipeline if that is something that you are after here.
@@AleksiPartanenTech Do you have a linkedin or some other social media, the error I continue having is a lot to explain in the comment section
@@AleksiPartanenTech The copy activity wont ccopy my actual data to the SQL table, it copies the schema alone but my table is empty. I check the file itself and it has data.
Hi Aleski nice explanation. Please can we create one practical session on parameters separately showing difference how it is diferent @piplinelevel,@linkedservicelevel@datasets level so it would be more helpful..😇😇😇and which level situations w e need to use which one?
Thanks for you comment and feedback! I will consider if I do a follow up video where I dive deeper into parameters. Sorry for a late reply since I have been on a holiday for few weeks.