Great intro tutorial. Can I filter by a filename instead of a prefix / suffix? We get multiple files dropped in the same bucket so filtering on a name is what would make EventBridge attractive. Thanks
the s3 stuff has now been updated. you do not need to manually copy it now, it auto updates bucket name under event pattern but without the key and prefix. ill check if it works well though
I am trying to do the same thing but the file which i am trying to track is uploaded into a dynamically named folder with name as current date . Do you habe any suggestions how can i track its upload or creation. Also the filename have a predefined prefix . Lets assume xyz_ how to configure it in an event trigger. if you can help me out thay will be great
Hi Osama, I would check cloud watch insights to see if there was a failed invocation, most likely there is a permission missing somewhere which may be the cause of the issue of nothing happening.
How can we get alert if the bucket size reaches certain bytes Ex: If my bucket size reaches or exceeds 5TB then i should get an alert. Can you please help me in this.
Hi Sai, sorry I don't know the answer to this. I figure you could look into cloudwatch events or eventbridge to set up some notification event with your specific criteria.
Fantastic tutorial. Thank you. Great introduction. Subscribed. Edit: this worked. However, the SNS sent the notification to another email. I have an email separate for AWS testing, which is what I added to the setup for this tutorial, but it sent it to my personal email. There must be some configuration in my account that allowed that to happen.
Subscribed. Keep continue on aws and data engineering hands on tutorials.
Thanks for subscribing Sany, Yes! A ton more aws and data engineering tutorials are in the pipeline!
@@DataEngUncomplicated sounds awesome 😎
Top quality content. Keep up the great work! This is exactly what i was looking for. Your channel will blow up soon with new subscribers.
Thanks for your kind words! I really appreciate it!
Thanks a lot for the video... Really appreciate your efforts!!!
Thanks Vamsi!
Keep up the great hand's on example. Quality content
Thank you for the feedback!
very practical content! thanks a ton!
You're welcome Hoijong! Thank for leaving me a comment, I appreciate the feedback.
Thanks dude this led me to achieve what I needed to do
I'm glad it helped!
Great intro tutorial. Can I filter by a filename instead of a prefix / suffix? We get multiple files dropped in the same bucket so filtering on a name is what would make EventBridge attractive. Thanks
the s3 stuff has now been updated. you do not need to manually copy it now, it auto updates bucket name under event pattern but without the key and prefix. ill check if it works well though
Thanks for leaving a comment about this update! The process is always changing.
@@DataEngUncomplicated it sure it! Can you do one with Sqs dead letter queue and event bridge triggered by a lambda function?
I am trying to do the same thing but the file which i am trying to track is uploaded into a dynamically named folder with name as current date . Do you habe any suggestions how can i track its upload or creation. Also the filename have a predefined prefix . Lets assume xyz_ how to configure it in an event trigger. if you can help me out thay will be great
do you have terraform scripts to perform this same operation? How to perform this using Terraform script ? Please help
Unfortunately I don't, I have been using terraform recently and was thinking of making some aws terraform videos in the future.
Hello. Where did the "key" and "prefix" come from when editing the Event pattern?
Hello, it comes from your aws s3 bucket for the directory/object you are interested in being the trigger
@@DataEngUncomplicated Thank you. If you wanted the trigger on the whole bucket would the prefix be the bucket name itself?
I see. No prefix if using the bucket itself.
its not working when I choose step functions as a target. Nothing happens , I can not see any invocation on the eventbridge rule side as well
Hi Osama, I would check cloud watch insights to see if there was a failed invocation, most likely there is a permission missing somewhere which may be the cause of the issue of nothing happening.
How can we get alert if the bucket size reaches certain bytes
Ex: If my bucket size reaches or exceeds 5TB then i should get an alert.
Can you please help me in this.
Hi Sai, sorry I don't know the answer to this. I figure you could look into cloudwatch events or eventbridge to set up some notification event with your specific criteria.
@@DataEngUncomplicated Yes in event bridge we don't have any specific option that tracks/ collects the size of the bucket.
Maybe this is a good question for the aws community: repost.aws/
Fantastic tutorial. Thank you. Great introduction. Subscribed.
Edit: this worked. However, the SNS sent the notification to another email. I have an email separate for AWS testing, which is what I added to the setup for this tutorial, but it sent it to my personal email. There must be some configuration in my account that allowed that to happen.
Thanks for subscribing!
You must have subscribed to the topic on your personal account. That's the only way it would know to email that address.