When EventBridge pipes were released during re:invent I couldn't understand the point why I need them if I can do this with Lambdas. After watching this video I realise that I dont need to write those "glue" Lambdas anymore. Will try pipes. Thanks!
Does pipe guarantee exactly once delivery ? Say I am using fifo sqs and want to trigger my step function as a pipe target and the step function needs to run a 2 hour job to pass down the data to next stage. I cannot afford the step function triggered twice with the exactly same message. I guess I need to do some deduplication logic using dynamodb lock in the enrichment phase?
I have an aws pipe which is getting json input from sqs. And the target for the pipe is lambda function. If i remove the filter from the pipe then target lambda is getting invoked. But if i use the filter, nothing works even i cant see anything in monitoring. Please help me on this problem. Note: i have already tested the json event with the filter pattern, and it works. i have tried removing the filter step and it works but i need to filter my json based on filter pattern.
Currently Pipes supports event sources that are polled, so it doesn't process events directly from EventBridge event bus. You could connect the two via SQS, though.
![AWS re:Invent 2022 - [NEW] EventBridge Pipes simplifies connecting event-driven services (API206)](http://i.ytimg.com/vi/xXGXCOc3cBs/mqdefault.jpg)
When EventBridge pipes were released during re:invent I couldn't understand the point why I need them if I can do this with Lambdas. After watching this video I realise that I dont need to write those "glue" Lambdas anymore. Will try pipes. Thanks!
Great to hear!
Does pipe guarantee exactly once delivery ? Say I am using fifo sqs and want to trigger my step function as a pipe target and the step function needs to run a 2 hour job to pass down the data to next stage. I cannot afford the step function triggered twice with the exactly same message. I guess I need to do some deduplication logic using dynamodb lock in the enrichment phase?
I have an aws pipe which is getting json input from sqs. And the target for the pipe is lambda function.
If i remove the filter from the pipe then target lambda is getting invoked. But if i use the filter, nothing works even i cant see anything in monitoring. Please help me on this problem. Note: i have already tested the json event with the filter pattern, and it works.
i have tried removing the filter step and it works but i need to filter my json based on filter pattern.
Can the source be events from an event bridge rule?
Currently Pipes supports event sources that are polled, so it doesn't process events directly from EventBridge event bus. You could connect the two via SQS, though.