dude! thank you so much for this, now i can probably stop my brain thinking about the pipeline over and over again. The sns via topic worked like a charm for me!🤓 i was trying with sqs but the pipeline never got established even though all the necessary configs were done as per snowflake documentation, still could'nt find out what went wrong. I would also like to see the same content via sqs if possible. thank you once again! keep up the good work✌
@@kushalappabe1993 For snowpipe working , Python is not needed , I added Python to perform some transformation activity in s3 before leading the data in Snowflake using Snowpipe ...
Hello Prashant , please refer this video -- Snowflake Snowpipe - Email Alert Mechanism th-cam.com/video/7X52Jv2uflo/w-d-xo.html Hope this will be helpful! Happy Learning :-)
Hello balaji ts , first of all, Glad to know the video is helpful to you 😊! Now coming your requirement , hope from this video you understood clearly how to configure Snowpipe using SNS , now all you need to know , how to use Snowpipe to load Json data , for that , you can refer this video -- th-cam.com/video/KCoC9ju-BfE/w-d-xo.html , so all you have to do is combine the 2 concepts , that's all. Hope this will be helpful! Happy Learning
@@KnowledgeAmplifier1 Thanks for your response, the twist is need to trigger the SNS using lamda function. Which means i already created one sns to trigger snowpipe once JSON file arrives on S3 I have another bucket once files arrives there need to run lambda function to call the same SNS..how we can achieve it
Sorry for the inconvenience Vikram , actually some part of the topic is similar like snowpipe with SQS setup which I already covered in my previous video , so went bit fast so that it won't appear reiteration of previous video and feel boring 😅 You can decrease the playback speed in setting as per your convenience. To explore more on Snowflake with AWS , you can have a look in this doc --doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510 Happy Learning :-)
dude! thank you so much for this, now i can probably stop my brain thinking about the pipeline over and over again.
The sns via topic worked like a charm for me!🤓 i was trying with sqs but the pipeline never got established even though
all the necessary configs were done as per snowflake documentation, still could'nt find out what went wrong. I would also like to see the same content via sqs if possible. thank you once again! keep up the good work✌
Hello Kushalappa BE , you can refer this video , if you want to implement Snowpipe with SQS -- th-cam.com/video/9hsdnLrx4iE/w-d-xo.html Happy Learning
@@KnowledgeAmplifier1 but according to snowflake documentation there is no need of python, anyways i will refer your link , ✌
@@kushalappabe1993 For snowpipe working , Python is not needed , I added Python to perform some transformation activity in s3 before leading the data in Snowflake using Snowpipe ...
very lucid and concise ..thanks 👍
You are welcome Varun Jain! Happy Learning
When I added a new record to the file and upload it again to the bucket do i need to run the pipe again or running the select statement is enough?
Can you make a video on SNS vs SQS, when should we use what service?
Edit: I got the answer from your other video about SQS. Thanks !! :)
Glad to hear that Ravi K R! Happy Learning
Hi ... How to get email notifications or error message for snowpipe data load to snowflake tables
Hello Prashant , please refer this video --
Snowflake Snowpipe - Email Alert Mechanism
th-cam.com/video/7X52Jv2uflo/w-d-xo.html
Hope this will be helpful!
Happy Learning :-)
Hello your contents are helpful, I need to load the JSON file data to snowflake db using SNS. Please help me on this
Hello balaji ts , first of all, Glad to know the video is helpful to you 😊! Now coming your requirement , hope from this video you understood clearly how to configure Snowpipe using SNS , now all you need to know , how to use Snowpipe to load Json data , for that , you can refer this video -- th-cam.com/video/KCoC9ju-BfE/w-d-xo.html ,
so all you have to do is combine the 2 concepts , that's all. Hope this will be helpful! Happy Learning
@@KnowledgeAmplifier1 Thanks for your response, the twist is need to trigger the SNS using lamda function. Which means i already created one sns to trigger snowpipe once JSON file arrives on S3 I have another bucket once files arrives there need to run lambda function to call the same SNS..how we can achieve it
@@balajits9081 from the another s3 bucket also , you can send the events to same SNS directly na? do you really need Lambda there?
@@KnowledgeAmplifier1 yes I need lambda there that is the requirements
@@KnowledgeAmplifier1 Hi any luck? how we can call SNS using lambda
Nice one but its very fast here
Sorry for the inconvenience Vikram , actually some part of the topic is similar like snowpipe with SQS setup which I already covered in my previous video , so went bit fast so that it won't appear reiteration of previous video and feel boring 😅
You can decrease the playback speed in setting as per your convenience. To explore more on Snowflake with AWS , you can have a look in this doc --doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510
Happy Learning :-)