⚠⚠⚠ Attention ⚠⚠⚠ When considering user assigned managed identity please note that currently this scenario is not yet supported. Services using user assigned identities are not considered trusted service. Thanks Andrzej J. for bringing this to my attention! 😊💪💪 More info: docs.microsoft.com/en-us/azure/storage/common/storage-network-security#trusted-microsoft-services
Very helpful. The UI is slightly outdated... the concept is current and very helpful. Using Event grid is much more efficient compared to When blob is added or modified. Thanks.
Great content again!!.. i have one question.. after assigning the Managed Identity to the Logic app and given permission to Managed identity on storage account , Can't we use the GetBlob content action in logic app? Do we need to use the Http call only?
Unfortunately GetBlob action uses Storage Account keys to connect, hence it's not considered a trusted connection. Once MS updates the connector to leverage Managed Identity it will work :) Thank you!
Adam, this is an awesome video. I just have a question. What happens if we need to monitor only a folder within the container and not the entire container? I addee /(nameoffolder) but that didn't fix the issue Thanks again
Please check this out. It shows example for the appropriate filter settings docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-event-overview thanks for watching :)
Hi Adam option 1 when a blob is modified , I am using this mechanism in my current env. but I dont know how can I use Managed identity in option 1 approach. As Option 2 is having HTTP trigger which by default gives a option to choose authentication method .
Can you suggest a workaround to make some changes(modify, update, delete) in the storage account using logic apps but without public access and also without ISE?
@@AdamMarczakYT Firewall is enabled for the storage account and Microsoft trusted apps are also allowed. Also assigned the role in IAM for the logic app, Still unable to access the container. I want to create blob or update container contents with the firewall enabled. How to achieve that?
This video shows exactly how to do it. Maybe just delete logic app and try again. Sometimes, it worked for me to edit logic app > remove authentication from HTTP request > save > edit > add managed identity on http request back again > run.
@@AdamMarczakYT I understand what you are trying to say, even after successfully running the trigger we can't see any updates/changes in the blob storage. That is my aim, modifying/updating/deleting the files in blob storage.
Hi Adam, I have been searching the internet for a video or an article to use logic app to convert xlsx files on an SFTP directory to csv or text but i cant find any leads , can you please give suggestions or do a video on that , thank you so much
Hey Zoe, while logic apps have excel connector for onedrive/sharepoint I'm not a big fan of it. The best flow that I personally found more enterprise grade for data processing is to copy data from SFTP to blob in 1:1 fashion. Just plain copy no extraction yet and then run databricks notebook which can work on excel files when you add excel libraries. Then it's simple data transformations which is what databricks excels at. Best of luck!
Hi Adam, when I run trigger on logic app with firewall after uploading the Json file. It shows error as below: "Failed to start a run of logic app logicapps-with-firewall. The template language expression evaluation failed: 'The execution of template trigger 'When_a_resource_event_occurs' failed: the result of the evaluation of 'splitOn' expression '@triggerBody()' is of type 'Null'. The result must be a valid array.'.." I demo as your steps to the default design on Logic Apps with Event Grid Can you help me :)
You must have made a slight mistake somewhere. Usually this error happens when you click 'run' button. You should not use run button for event based programming :)
@@AdamMarczakYT - Even i am facing the same error here. I have already registered Microsoft.Eventgrid provider in my subscription. Still the event is not getting triggered when i create or delete blob so tried to trigger it manually thereby facing this error. But when i set SplitON to OFF also automatic event triggering is not taking place but when i hit run it is succeeding. Please lemme know if i missed something here.
Thank you. I haven't seen any good tutorials, I'm not also SAP expert but you can check connector reference here docs.microsoft.com/en-us/azure/logic-apps/logic-apps-using-sap-connector it seems to support BAPI functions
Hi Adam, these are great videos and helped me a lot. Can you tell me how to setup Azure Event Grid Subscription when my LogicApp resides within an ISE and or In WorkflowSettings of LogicApp we have configured "Only other LogicApp" or "Specific IP ranges". Will be great if you can help me. Thanks
@@AdamMarczakYT I tried creating an Azure Event Grid ISE and configuring it but it doesn't seem to work. Will be great if you can add a video on this as well.
@@AdamMarczakYT Seems like even after deploying Azure EventGrid as Managed connector inside ISE doesn't work for this usecase. It would be really helpful if you can just send me a working doc from your end. I have already sent you a mail on LinkedIn. Thanks for your help again.
@Adam - thanks for the great bit of info & tutorial. It's a shame that the OOB API Connector (Storage) only has the "Access Key" authorisation option. In my case, we have a 3rd Party Vendor building us a Logic-App solution that needs to read/write into one of our BLOB containers. I really do not like having to give ANYONE, much less external vendors, our Access Key which is essentially the 'key to everything' in the account. Besides the technique you demonstrated, is there not any other means of allowing a Logic-App access to our Storage Account (which is not Firewall protected) that won't requires so much extra HTTP request involvement, and no Event-Hub involvement either? :-)
Yea, Microsoft is updating API Connectors constantly, so probably soon enough we will get this option. Also, in my opion... NEVER share Access Key. In many cases we even take away permissions using custom roles so that people can't see keys in the portal. Unfortunately there isn't better option with logic apps now. But data factory can use managed identities.
By default, Split on is 'ON' while creating the Event Grid and it throws error. so, Go to settings on the block of Event Grid and change that Split on setting to 'OFF'. While trying and using Event Grid, i faced this one, so added as comment. The error that i got is - "The template language expression evaluation failed: 'The execution of template trigger 'When_a_resource_event_occurs' failed: the result of the evaluation of 'splitOn' expression '@triggerBody()' is of type 'Null'. The result must be a valid array.'. ". In event grid's run history --> body part is not showing for me, how to bring that ? (10:30 timeclip)
It must have been some different issue because SplitOn should be set to "ON". if you set it to OFF then it means you want to process file changes in batches. In that case event grid will return list (array) with objects rather then singular object. In which case you will need to handle items with for-each loops. Thanks for watching. :)
I've watched a couple of your videos, finally subscribed. You do a great job explaining the how and the why. Thank you.
Awesome, thank you!
⚠⚠⚠ Attention ⚠⚠⚠
When considering user assigned managed identity please note that currently this scenario is not yet supported. Services using user assigned identities are not considered trusted
service. Thanks Andrzej J. for bringing this to my attention! 😊💪💪
More info: docs.microsoft.com/en-us/azure/storage/common/storage-network-security#trusted-microsoft-services
If this changes in future I'll make sure to update this comment!
Thank you Adam, this is first video of you i have seen, and its really cool.
Awesome! Come for one, stay for more I hope! :)
This is too good, You have explained many concepts in one post. Hats off
Thank you! You are too kind!
Very helpful. The UI is slightly outdated... the concept is current and very helpful. Using Event grid is much more efficient compared to When blob is added or modified. Thanks.
Thanks Adam for this video. It's really helpful. Looking forward to have more videos on Azure Integration Services clubbed together
More to come! Thanks for stopping by!
Really good training. Good job Adam!
Glad you enjoyed it
Thanks Adam. Great explanation
My pleasure!
Thank you Adam. It's nice
Thanks for wonderful Tutorial.
My pleasure 😊
Thank you so much Adam for this tutorial. Its much helpful for me.
Wonderful. Found this information very useful.
Cheers! More videos are coming every week :)
Nice demo, Thanks
Yay!! Superlikes.... wonderful presentation... AAA (Adam Azure Awesome!)
Thank you! Cheers! This is surely a very cool comment to read, I appreciate it.
Great content again!!.. i have one question.. after assigning the Managed Identity to the Logic app and given permission to Managed identity on storage account , Can't we use the GetBlob content action in logic app?
Do we need to use the Http call only?
Unfortunately GetBlob action uses Storage Account keys to connect, hence it's not considered a trusted connection. Once MS updates the connector to leverage Managed Identity it will work :) Thank you!
Thank you Adam!!..I dont see about this in Microsoft documentation.. if you have any document link please share.
Connector reference is what you are looking for docs.microsoft.com/en-us/connectors/azureblob/?WT.mc_id=AZ-MVP-5003556#default-connection
Adam, this is an awesome video. I just have a question. What happens if we need to monitor only a folder within the container and not the entire container? I addee /(nameoffolder) but that didn't fix the issue
Thanks again
Please check this out. It shows example for the appropriate filter settings docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-event-overview thanks for watching :)
@@AdamMarczakYT Thank you Adam
It's a very detailed and informative tutorial as always... Do we have connectivity to ADLS Gen 2 from Event grid or Logic apps..
Thanks in advance
Heya yes! this will work with ADLSg2 in the exactly same way :).
@@AdamMarczakYT Thanks for the reply Adam
Thank you
Thanks!
Hi Adam option 1 when a blob is modified , I am using this mechanism in my current env. but I dont know how can I use Managed identity in option 1 approach. As Option 2 is having HTTP trigger which by default gives a option to choose authentication method .
You can't use Managed Identity in option 1 unfortunately. That's why I shown option 2. Otherwise I would stick with option 1 myself.
@@AdamMarczakYT ok got you point.
How about write/create file into BLOB container using managed identity!! DO you have any demo on that? please let me know thanks
Can you suggest a workaround to make some changes(modify, update, delete) in the storage account using logic apps but without public access and also without ISE?
Do you mean with firewall enabled but with no exception for trusted apps? or blob storage with private link enabled?
@@AdamMarczakYT Firewall is enabled for the storage account and Microsoft trusted apps are also allowed. Also assigned the role in IAM for the logic app, Still unable to access the container. I want to create blob or update container contents with the firewall enabled. How to achieve that?
This video shows exactly how to do it. Maybe just delete logic app and try again. Sometimes, it worked for me to edit logic app > remove authentication from HTTP request > save > edit > add managed identity on http request back again > run.
@@AdamMarczakYT I understand what you are trying to say, even after successfully running the trigger we can't see any updates/changes in the blob storage. That is my aim, modifying/updating/deleting the files in blob storage.
Hi Adam,
I have been searching the internet for a video or an article to use logic app to convert xlsx files on an SFTP directory to csv or text but i cant find any leads , can you please give suggestions or do a video on that , thank you so much
Hey Zoe, while logic apps have excel connector for onedrive/sharepoint I'm not a big fan of it. The best flow that I personally found more enterprise grade for data processing is to copy data from SFTP to blob in 1:1 fashion. Just plain copy no extraction yet and then run databricks notebook which can work on excel files when you add excel libraries. Then it's simple data transformations which is what databricks excels at. Best of luck!
Hi Adam, when I run trigger on logic app with firewall after uploading the Json file. It shows error as below:
"Failed to start a run of logic app logicapps-with-firewall. The template language expression evaluation failed: 'The execution of template trigger 'When_a_resource_event_occurs' failed: the result of the evaluation of 'splitOn' expression '@triggerBody()' is of type 'Null'. The result must be a valid array.'.."
I demo as your steps to the default design on Logic Apps with Event Grid
Can you help me :)
You must have made a slight mistake somewhere. Usually this error happens when you click 'run' button. You should not use run button for event based programming :)
@@AdamMarczakYT - Even i am facing the same error here. I have already registered Microsoft.Eventgrid provider in my subscription. Still the event is not getting triggered when i create or delete blob so tried to trigger it manually thereby facing this error. But when i set SplitON to OFF also automatic event triggering is not taking place but when i hit run it is succeeding. Please lemme know if i missed something here.
have they changed the UI again? I don't get these options when using the Blob Storage connector in Logic Apps
Can you provide a demo with ISE
Adam, thanks for your video. do you have any reference to SAP BAPI connector? I'm looking for a tutorial to pass the input parameter.
Thank you. I haven't seen any good tutorials, I'm not also SAP expert but you can check connector reference here docs.microsoft.com/en-us/azure/logic-apps/logic-apps-using-sap-connector it seems to support BAPI functions
Hi there,
Im getting a "Issuer validation failed. Issuer did not match." error when I run the logic app. Do you know what that could be caused by?
Hi Adam, these are great videos and helped me a lot. Can you tell me how to setup Azure Event Grid Subscription when my LogicApp resides within an ISE and or In WorkflowSettings of LogicApp we have configured "Only other LogicApp" or "Specific IP ranges". Will be great if you can help me.
Thanks
Use ISE connectors specifically designed for this use case
docs.microsoft.com/en-us/azure/connectors/managed?WT.mc_id=AZ-MVP-5003556#ise-connectors
@@AdamMarczakYT I tried creating an Azure Event Grid ISE and configuring it but it doesn't seem to work. Will be great if you can add a video on this as well.
@@AdamMarczakYT Seems like even after deploying Azure EventGrid as Managed connector inside ISE doesn't work for this usecase. It would be really helpful if you can just send me a working doc from your end. I have already sent you a mail on LinkedIn. Thanks for your help again.
Does this also work for consumption app??
Hi, how do I get like ur t shirt with bits
@Adam - thanks for the great bit of info & tutorial. It's a shame that the OOB API Connector (Storage) only has the "Access Key" authorisation option. In my case, we have a 3rd Party Vendor building us a Logic-App solution that needs to read/write into one of our BLOB containers. I really do not like having to give ANYONE, much less external vendors, our Access Key which is essentially the 'key to everything' in the account. Besides the technique you demonstrated, is there not any other means of allowing a Logic-App access to our Storage Account (which is not Firewall protected) that won't requires so much extra HTTP request involvement, and no Event-Hub involvement either? :-)
Yea, Microsoft is updating API Connectors constantly, so probably soon enough we will get this option. Also, in my opion... NEVER share Access Key. In many cases we even take away permissions using custom roles so that people can't see keys in the portal. Unfortunately there isn't better option with logic apps now. But data factory can use managed identities.
By default, Split on is 'ON' while creating the Event Grid and it throws error. so, Go to settings on the block of Event Grid and change that Split on setting to 'OFF'. While trying and using Event Grid, i faced this one, so added as comment. The error that i got is - "The template language expression evaluation failed: 'The execution of template trigger 'When_a_resource_event_occurs' failed: the result of the evaluation of 'splitOn' expression '@triggerBody()' is of type 'Null'. The result must be a valid array.'.
". In event grid's run history --> body part is not showing for me, how to bring that ? (10:30 timeclip)
It must have been some different issue because SplitOn should be set to "ON". if you set it to OFF then it means you want to process file changes in batches. In that case event grid will return list (array) with objects rather then singular object. In which case you will need to handle items with for-each loops. Thanks for watching. :)
In my opinion run the demo again from scratch, maybe you missed something. Also remember to register event grid provider on entire subscription.
@@AdamMarczakYT , Thanks , I have registered now by seeing your 'Azure Event Grid Tutorial'- video clip at 7:20