Useful session, abundant insights to glean indeed ! At t=24:00, it was mentioned that adding a DLQ and Lambda would add unnecessary complexity in case failures, and can be replaced using EventBridge API destinations. But, even in that case, for failed events we would need a DLQ and some way to push those messages again ?? Basically, I'm failing to understand how is EventBridge API Destinations helping to reduce the complexity (for failed events) ? Appreciate if anyone can help clarify my confusion.
Thanks for your feedback. What I am trying to convey is about the automatic retry available with API Destinations and how it can eliminate a measure we otherwise need to put in place to store failed requests to the target (in a DLQ) and resubmit when the target becomes available. This is to cover the failure between API Destination and the external HTTP endpoint target. In this case, it assumes that EventBridge has successfully delivered the event to its rule target, which is the API Destination. You may still add a DLQ here to catch any failed event delivery to API Destination by EventBridge. I hope this clarifies.
To grasp the distinctions between Kafka and Amazon EventBridge, it's essential to understand the fundamentals of each platform. Starting with Kafka: Essentially, Kafka functions as a commit log and introduces the concept of topics, which serve as specific channels for events. Each topic comprises one or more partitions, which are immutable, ordered streams of events. Multiple consumers can subscribe to one or more topics to access the events published to them. Consequently, consumers are tied to the topics they subscribe to, necessitating knowledge of the specific topics to consume. It's then the responsibility of the consumer to filter out irrelevant events. Turning our attention to Amazon EventBridge, this service operates as a serverless, managed event bus. An event bus facilitates the publication of events from multiple sources or publishers without concern for the eventual destination; publishers simply emit events to the event bus's default endpoint. On the other side, consumers or targets of these events connect to EventBridge or define how events should be forwarded to them. Unlike Kafka, consumers don't need to select specific event topics. Instead, targets establish routing and filtering rules that can be as detailed as required. For instance, a target may instruct EventBridge to forward all events related to order placements or apply more specific payload value filters to scrutinize the event details. By doing so, EventBridge enhances service decoupling by abstracting the collaboration between applications. For instance, in systems I've designed, bounded contexts (domain services) utilize Kafka topics for event egress between systems. In some scenarios, we've employed Amazon EventBridge to aggregate events from multiple Kafka topics, applying rules to filter events and deliver them to another system interested in multiple topics under certain conditions (e.g., events related to specific activities). Utilizing EventBridge for filtering and routing from multiple topics allows for adaptability to business requirements changes-such as the need to consume additional topics as more systems are developed over time-without necessitating updates to the consumer's code. Instead, we adjust EventBridge's rules and filters for the target as a configuration task. If you want to know more about Kafka navigate to my channel :)
Excellent information on Event Bridge and related services.
Incrible apresentation. Thanks very much
Excellent presentation and content!
We're glad the explanation was helpful. 😍
Great presentation!!!
Useful session, abundant insights to glean indeed !
At t=24:00, it was mentioned that adding a DLQ and Lambda would add unnecessary complexity in case failures, and can be replaced using EventBridge API destinations. But, even in that case, for failed events we would need a DLQ and some way to push those messages again ?? Basically, I'm failing to understand how is EventBridge API Destinations helping to reduce the complexity (for failed events) ?
Appreciate if anyone can help clarify my confusion.
Thank you for your in-depth insight. 💡 I've gone ahead and shared your question internally for review. ✅ ^AR
Thanks for your feedback. What I am trying to convey is about the automatic retry available with API Destinations and how it can eliminate a measure we otherwise need to put in place to store failed requests to the target (in a DLQ) and resubmit when the target becomes available.
This is to cover the failure between API Destination and the external HTTP endpoint target. In this case, it assumes that EventBridge has successfully delivered the event to its rule target, which is the API Destination. You may still add a DLQ here to catch any failed event delivery to API Destination by EventBridge. I hope this clarifies.
How would you compare event bridge vs kafka ?
To grasp the distinctions between Kafka and Amazon EventBridge, it's essential to understand the fundamentals of each platform.
Starting with Kafka: Essentially, Kafka functions as a commit log and introduces the concept of topics, which serve as specific channels for events. Each topic comprises one or more partitions, which are immutable, ordered streams of events. Multiple consumers can subscribe to one or more topics to access the events published to them. Consequently, consumers are tied to the topics they subscribe to, necessitating knowledge of the specific topics to consume. It's then the responsibility of the consumer to filter out irrelevant events.
Turning our attention to Amazon EventBridge, this service operates as a serverless, managed event bus. An event bus facilitates the publication of events from multiple sources or publishers without concern for the eventual destination; publishers simply emit events to the event bus's default endpoint. On the other side, consumers or targets of these events connect to EventBridge or define how events should be forwarded to them. Unlike Kafka, consumers don't need to select specific event topics. Instead, targets establish routing and filtering rules that can be as detailed as required. For instance, a target may instruct EventBridge to forward all events related to order placements or apply more specific payload value filters to scrutinize the event details. By doing so, EventBridge enhances service decoupling by abstracting the collaboration between applications.
For instance, in systems I've designed, bounded contexts (domain services) utilize Kafka topics for event egress between systems. In some scenarios, we've employed Amazon EventBridge to aggregate events from multiple Kafka topics, applying rules to filter events and deliver them to another system interested in multiple topics under certain conditions (e.g., events related to specific activities). Utilizing EventBridge for filtering and routing from multiple topics allows for adaptability to business requirements changes-such as the need to consume additional topics as more systems are developed over time-without necessitating updates to the consumer's code. Instead, we adjust EventBridge's rules and filters for the target as a configuration task.
If you want to know more about Kafka navigate to my channel :)
API Destinations will eliminate a lot of very simple Lambdas