@@KnowledgeAmplifier1 Highly Appreciatee the efforts, this sns email notification isn't explained by anyone but you on TH-cam, and I bet many teams would be using this architecture if they don't have airflow like my team. A small request: can you make a series 2-3 big and complex projects focusing on glue/pyspark and snowflake, I viewed your earlier videos in projects but few more and large would highly help
Thank you venkata ramana! If you are integrested to learn Snowflake in-depth , then don't miss this doc --share-docs.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510 Happy Learning
Great work! Thank you for this. It would be nice if you include the architectural diagram that is part of the thumbnail in the actual start/end of the video for a summary of what we're accomplishing
Thank you for the feedback Jack Daoud! I will make sure the architecture I am showing somewhere in between the explanation. For this video the architecture used in the thumbnail is available in this link , if needed you can check -- github.com/SatadruMukherjee/Data-Preprocessing-Models/blob/main/Error%20Notifications%20for%20Snowflake%20Tasks%20using%20SNS.png Happy Learning
Hello suseendar ramadoss , from SNS , you can trigger Lambda and using another SNS Topic or SES you can send mail after formatting it properly as per your requirement 😊 or if you want to send all task failure notification together with some time intervals , you can refer the below video where the data flow is : One monitor task (run every x mins interval)will capture all the failed tasks within last x minutes and then unload the data in s3 , from s3 using Lambda , SES , send the report as alert -- th-cam.com/video/PScK6OyL5RU/w-d-xo.html Hope this will be helpful! Happy Learning
Hi, how can i send a message into a sns topic but when a task finished ok? this error_integration in the tasks sends a message only when the task failed or when finish ok also?? If not, create another child task after the one i want to validate with a python or javascript code to send a message would be the best solution? thanks
Hello Muktiar Ahamed, I have already uploaded some end-to-end projects , you can refer the "Snowflake End-to-End Projects" section of the document -- doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510 Will upload more projects in future soon .. Till then stay tuned 😊 Happy Learning
bro you are making us learn indusstry level stuff, my team recently implemented this kind of mechanism, and you did it 2 years back!!!
I try to make videos keeping industry-usage in mind @mickyman753 & for more such videos stay tuned ! Happy Learning
@@KnowledgeAmplifier1 Highly Appreciatee the efforts, this sns email notification isn't explained by anyone but you on TH-cam, and I bet many teams would be using this architecture if they don't have airflow like my team.
A small request: can you make a series 2-3 big and complex projects focusing on glue/pyspark and snowflake, I viewed your earlier videos in projects but few more and large would highly help
Your effort is appreciable
Thank you venkata ramana! If you are integrested to learn Snowflake in-depth , then don't miss this doc --share-docs.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510
Happy Learning
Great work! Thank you for this. It would be nice if you include the architectural diagram that is part of the thumbnail in the actual start/end of the video for a summary of what we're accomplishing
Thank you for the feedback Jack Daoud! I will make sure the architecture I am showing somewhere in between the explanation. For this video the architecture used in the thumbnail is available in this link , if needed you can check -- github.com/SatadruMukherjee/Data-Preprocessing-Models/blob/main/Error%20Notifications%20for%20Snowflake%20Tasks%20using%20SNS.png Happy Learning
than u so much for sharing
No Problem Muktiar Ahamed! Happy Learning
How can i format the output before sending it as a email
I tried using lambda function. But it aint integrating with the sns
Hello suseendar ramadoss , from SNS , you can trigger Lambda and using another SNS Topic or SES you can send mail after formatting it properly as per your requirement 😊 or if you want to send all task failure notification together with some time intervals , you can refer the below video where the data flow is : One monitor task (run every x mins interval)will capture all the failed tasks within last x minutes and then unload the data in s3 , from s3 using Lambda , SES , send the report as alert --
th-cam.com/video/PScK6OyL5RU/w-d-xo.html
Hope this will be helpful! Happy Learning
How we manipulate error_message in mail . I mean any exception message from snowflake proced.
Can we set up one for SUCCESS , notifying task has been executed successfully ?
You can refer this AJJESHH , the SYSTEM$SEND_EMAIL feature can be used -- docs.snowflake.com/en/user-guide/email-stored-procedures
Hi, how can i send a message into a sns topic but when a task finished ok? this error_integration in the tasks sends a message only when the task failed or when finish ok also?? If not, create another child task after the one i want to validate with a python or javascript code to send a message would be the best solution? thanks
sir could u please make a video on snowflake real time project end to end implementation
Hello Muktiar Ahamed, I have already uploaded some end-to-end projects , you can refer the "Snowflake End-to-End Projects" section of the document --
doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510
Will upload more projects in future soon .. Till then stay tuned 😊
Happy Learning
thank u so much for sharing link
project a lot because I've been working on other stuff (and being lazy lol). Also, I had been facing a recurring problem of content dropouts in
I have not got it James sorry , if you can tell in other way with some more detail context..