Event Driven Architecture EXPLAINED in 15 Minutes
ฝัง
- เผยแพร่เมื่อ 9 ก.ค. 2024
- What is Event Driven Architecture and why is it so popular these days? Event Driven Architecture is the most loosely coupled, most scalable approach to building large, complex and high performance systems, it may be implemented via collections of microservices or not, and represents a fundamentally different, and often better, way to organise the behaviour of our systems. But it can seem strange and different at first, until you realise that most people who have ever written any UI code are already experienced at it.
In this episode Dave Farley, author of best selling books "Continuous Delivery" and "Modern Software Engineering" and also a long term innovator in the use of Event Driven Systems, describes how Event Driven Architecture is an evolution from Event Driven Programming. What the benefits of this style of design and programming are and offers tips on how to best design them.
-
⭐ PATREON:
Join the Continuous Delivery community and access extra perks & content! ➡️ bit.ly/ContinuousDeliveryPatreon
🎥 Join Us On TikTok ➡️ / modern.s.engineering
-
👕 T-SHIRTS:
A fan of the T-shirts I wear in my videos? Grab your own, at reduced prices EXCLUSIVE TO CONTINUOUS DELIVERY FOLLOWERS! Get money off the already reasonably priced t-shirts!
🔗 Check out their collection HERE: ➡️ bit.ly/3Uby9iA
🚨 DON'T FORGET TO USE THIS DISCOUNT CODE: ContinuousDelivery
-
BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is available as paperback, or kindle here ➡️ amzn.to/3DwdwT3
and NOW as an AUDIOBOOK available on iTunes, Amazon and Audible.
📖 The original, award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ amzn.to/2WxRYmx
📖 "Continuous Delivery Pipelines" by Dave Farley
Paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
📖 "Domain Driven Design", by Eric Evans ➡️ amzn.to/2WXJ94m
📖 "Reactive Design Patterns", by Roland Kuhn & Jamie Allen ➡️ amzn.to/3uCqNph
NOTE: If you click on one of the Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
-
🔗 LINKS:
🔗 "Event Driven Architecture", Wikipedia ➡️ en.wikipedia.org/wiki/Event-d...
🔗 "Event Driven Architecture", AWS ➡️ aws.amazon.com/event-driven-a...
🔗 "Event Storming", Wikipedia ➡️ en.wikipedia.org/wiki/Event_s...
🔗 "Event Storming", Alberto Brandolini • Event Storming - Alber...
🔗 "CQRS", Wikipedia en.wikipedia.org/wiki/Command...
🔗 "CQRS", Martin Folwer martinfowler.com/bliki/CQRS.html
-
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
TransFICC provides low-latency connectivity, automated trading workflows and e-trading systems for Fixed Income and Derivatives. TransFICC resolves the issue of market fragmentation by providing banks and asset managers with a unified low-latency, robust and scalable API, which provides connectivity to multiple trading venues while supporting numerous complex workflows across asset classes such as Rates and Credit Bonds, Repos, Mortgage-Backed Securities and Interest Rate Swaps ➡️ transficc.com
Semaphore is a CI/CD platform that allows you to confidently and quickly ship quality code. Trusted by leading global engineering teams at Confluent, BetterUp, and Indeed, Semaphore sets new benchmarks in technological productivity and excellence. Find out more ➡️ bit.ly/CDSemaphore
#softwareengineer #developer - วิทยาศาสตร์และเทคโนโลยี
Excelent video. I've recently designed, implemented, and deployed to production systems based on EDA, Event Sourcing, and CQRS. It radically changed the way I design and program systems, with incredible results: more cohesion, better separation of concerns, cleaner code code that is focused on the problem domain, with true object orientation, using DDD, highly and easily testable. Excellent response times, excellent maintainability. New feature were so easy to implement and discuss. Now it's hard for me to listen to a problema and imagine a solution in a different way.
About how to design it, I'd highly suggest a "step further" than event storming: event modelling. It starts with the same brainstorm concept, but the way it proceeds is a bit different, and I felt it makes it much easier to identify what are the business domains, and their bounded contexts in term of the business flow of the timeline.
Also, I must add: in the mentioned projects, I used a lot of the way of thinking that this chanel presents. I must thank you, Dave, as you confirmed a lot of things I've discovered ony my own, and taught me a lot more, and that played a crucial role in the success of these projects.
@@Ashton666 this GPT right here 😂
@@philipfisher8853 what do you mean?
I've seen EDA be a total disaster. Everything is over complicated. Boundaries between domains are more complicated and harder to test. Lots of inconsistent and complex code necessary to source the events. Lots of bugs in the event integrations. After a few years the org started trending back towards old fashioned REST integrations and things started to get simpler and faster. Been working on deprecating all this garbage.
Disclaimer: I haven't watched the video.
@@smithersj90 REST is not "old fashioned", it's just another approach to solve another problem. The codebase you're working seems to be a mess not because of EDA itself, but because the lack of skill of those implementing it, or the business needs not being compatible with such an architecture.
@@Ashton666 your initial comment is way too positive
Event driven architecture has been around for ages though. It just took forever for people to pay attention to it.
I think it's just that a bigger mainstream need for that kind of design arose with the widespread of microservices and distributed systems.
Exactly what I was thinking. It was popularised on the desktop by GUIs in the mid to late 80s, but before then used (and still is used extensively) in embedded and other low level programming aka hardware interrupts. Client-server, and later messaging & orchestration systems harnessed these ideas in the 90s for distributed systems. Even the X Window System was based on RPC and thus was event driven.
Any sort of RPC is fundamentally an event driven system, but it's become hidden from the programmer over the years thanks to the ever harder push for abstraction: kinda ironic we're now building and exposing an abstraction on top of abstraction that was trying to hide the original abstraction.
I'm sure there'll be new fancy pattern names from the CS dogma doctors.
@@DodaGarcia It never went away, it was just hidden by abstraction, and now we seem to be building an abstraction on top of the abstraction that was built to hide it in the first place!
9:08 ...yeah... unless you want to add a step between order processing and all the other activities (e.g. fraud detection, validation...) because now you have to know each service using the event and change it. And obviously, you need compensation events in case of errors (if debit fails, you don't add it to history). Now add 15 more activities and have fun understanding, what was actually happening with a failed order (and fixing it) because even with distributed tracing, it gets really hard to understand the order of events and which events trigger what when. Implementing, documenting and changing a business critical complex process with a choreography approach looks clean in a demo, but is a nightmare in many situations. I'm not against it, but it is a really challenging approach for most enterprises with their processes.
Yeah, I kept waiting for Dave to address these obvious concerns in the video, but he never got there. :-?
Yeah debugging EDA is pretty much impossible without a log of events and listeners, but that log is simply NEVER implemented.
can't talk in general, but the validation/fraud detection seems a separate step on which everyone else depends, so should be inserted in the process before "order placed" (only valid orders should be placed). Or generate an "order placed" event on which the validator listens, and let the validator, on success, emit a "valid order placed" on which everyone else listens. Otherwise I agree it can get hairy, and for that probably there are some other youtubes - I see one "complexity is the gotcha of event-driven architecture" (didn't watch it)
After over 20 years of professional programming, event sourcing with CQRS is the closest to a silver bullet I've seen. Contrary to what I was expecting, it also scales down really well to really simple systems.
The biggest issue standing in the way of adoption right now in my opinion is that there aren't standard free software solutions to implement it based on common databases. That means you have to write a few slightly tricky parts yourself to get event handling to work. However, once I have implemented these parts in my stack, I can reuse this code again and again for new projects.
Of course as with anything there are cases where this architecture is not a good fit, like for example if eventual consistency can not be tolerated. But when there are no fundamental issues, I'd go for ES+CQRS ten times out of ten.
@@dedalusow if you're on Java, there's the axon framework. You can use for free without the axon server, and using databases as an event bus is an option - I used that in my previous role, for a bank, and it worked wonders.
Yes, I agree entirely. I am convinced that a simplified programming model based on these ideas would represent a significant step forward in the adoption of this approach for lots more things. There are a few frameworks like Akka or Axon I think that they overcomplicate the programming model a bit, but it I too am sold - this is good for nearly every type of software.
We use it repeatedly for every project. Anything else would make things more complicated. In September we'll be starting our 10th year of that.
@@ContinuousDelivery Call me a cynic, but this is reinventing the wheel. As mentioned, we've been doing this for many decades already, but it's been hidden by the drive for more and more layers of abstraction. Almost everything in a distributed or loosely coupled environment is event driven under the hood. Even inside the OS, thread scheduling works around event driven processes, such as when waiting for I/O, rather than tying up a core in continuous polling.
I'm from a data engineer background, currently assigned to event-driven project. This video dropped just in time for me!
These types of videos are highly valuable across the roles of an SDLC. I share your videos with everyone in my org.
This is the single best video explaining EDA and related concepts I have ever watched. Not a second wasted and presented clearly, neutrally and insightfully. Bravo!
I appreciated the format, breaking down the pattern. I've only ever experienced "micro services" done badly, but Event Driven Architecture is the underpinnings of ms and this just makes so much sense as the method for developing systems
While event-driven programming is the natural approach to solving a number of problems, it's not as easy as this video makes it out to be once you begin to build non-trivial systems. Issues like race conditions, where you cannot depend in which event handlers fire; dealing with situations where you need to wait for multiple events to trigger before proceeding; blocking operations locking up your process, etc.
The problem of race conditions is not special to event driven systems though, just to concurrent and distributed systems. I think that event based systems make it easier to solve race conditions, but it certainly does take some thought in terms of design, as it does with non-event-based distributed systems.
I'd love more content about all the fun real-world problems. Thanks Dave and team!
@@ContinuousDelivery Also, it's about the right tool for the right job. For example at my company we've got an EDA serverless system, but when there's sequences of events that _must_ happen in a set order, well, there's another tool for that. Step Functions and state machines.
Hi Dave,
I recently came across some of Adam Dymitruk’s work on Event Modeling, and I’m intrigued by his claims, but turned off by his tone.
He argues that traditional development practices like TDD and Agile are outdated and that Event Modeling is the future. My understanding is that Event Modeling is somewhat like Event Storming but uses the Muriel board as the basis for essentially all communication.
He suggests that by modeling systems as events, we can break down work into uniform slices, simplify the development process, and even implement a pay-per-slice compensation model, which he claims results in very low churn despite significant pay differences. I interpret that as meaning that the employees view it as a "tough but fair" pay model.
Given your experience with the actor model approach at LMAX (that you talked about in earlier videos, not this one), I’m curious about your opinion on Adam’s perspective.
Specifically:
- Are you familiar with Event Modeling as described by Adam Dymitruk?
- What do you think about Adam's claim that Event Modeling renders TDD, Integration Testing, and Agile wasteful and redundant due to the easy-to-understand scope of each slice?
He seems to have so much in common with you regarding what he would likely consider canonical works in the world of software, and yet he is coming to these radically different conclusions. And he is talking with this kind of "wake up sheeple" tone.
He claims to have loads of experience, but I trust your perspective because you have actually earned my trust, and apart from that, I know that you are both open-minded and able to smell if something seems fishy, yet his purported evidence has raised my eyebrows.
This was the video that I watched a few days ago th-cam.com/video/Pin_B-AbdXE/w-d-xo.htmlfeature=shared I have also read articles from his website eventmodeling.org/
I know it's not your job to be the arbiter of truth and who has it, it's just that it has been nagging my brain of how to interpret what this guy is saying for the last few years on and off. And I would rather not put you in a position to say something negative about someone else, but still I felt compelled to seek out your opinion, and today you are talking about Event Driven Architecture and I thought "that's it! I am going to ask"
Thank you!
FWIW, I'd like to return to DDD. Event based architectures simplify the Bounded Contexts. But there's much more responsibility in the Context Map, which I feel is the responsibility of the architects and design leads.
I confess that I have struggled a bit with Adam's response to my stuff, which seems like a shame because I think that we would probably agree on quite a lot, but the tone of his response to my stuff is more like a troll, than anything else, so he is one of about 3 people that I have muted on my social media. I have no objection to people disagreeing, but I prefer reasoned arguments, and Adam and I didn't seem to be able to achieve that. So I tend to ignore him, which is a shame because I think otherwise we'd probably be allies in the DDD, event based systems cause.
I watched the video that you linked and I agreed with lots of what he said, I am not sure that I see the distinction he makes between Event Modelling and Event Storming, at the level of detail that matters to me, the way that I use Event Storming sounds pretty similar to his use of it to me. Of course, I don't agree with his take on automated testing, though I do agree with his stance on designing systems from small, focused, independent pieces of code. It is my view that the best way to get people to design code like that though is to prefer testable code, which is what TDD does. He clearly doesn't like that. I accept that he may have something better, but I don't see it, and that is not how I and my teams worked, and successfully built systems.
Weird take. EDA and TDD go hand in hand.
I'm not sure where you draw those conclusions. I've made a post that includes multiple issues with TDD. I'm up for a discussion in a public forum any time. I've arrived at different conclusions but those that don't want to hear it because it doesn't align with how they see things are quick to dismiss as trolling, tone or whatever. It's to the point that my replies get deleted. I'm saving this one and posting it elsewhere too just in case.
I think that'd be a fascinating interview on either of your platforms to unpack it, dig into Event Modeling, and how Adam still has confidence without TDD.
Also, kudos for the excellent question and framing in the OP. 👏
Great video as usual, Dave!! I'm also glad that you touched upon the point of how intentional design often ends up requiring more code, because I see that criticism pop up a lot whenever design patterns are discussed and usually as an argument against applying them.
I find that silly because in my mind it's a one-to-one analogy to physical organization in the real world. When we organize objects around a physical space, they'll always take more space than they would as a single pile, and keeping them organized will always take more work than the alternative. And yet, in this analogous scenario most people intuitively see the advantage in things being organized because of the time you save looking for things when you need them.
Good stuff, I appreciate the overview of the benefits of applying event driven architecture to more than GUI's.
I think the biggest challenge with event driven systems is bridging the gap between imperative code and event driven code. For instance, how do you build a REST API if everything in the backend is event driven? How do you wait for events in your http handler to produce the proper response to your client? How do you even know which event you should wait for without coupling your API with the processes it might trigger? Building that translation layer is pretty hard, and it's often where I fail to sell event driven designs to other more junior people in my team.
Use a queue.
Omg i love these types of videos. I find it's very difficult to find resources i trust on design topics
I do like these design-centered videos. I also feel, like you said, it only scratches the surface. I feel like there's a tension there of breadth vs depth.
Thank you very much for this. I needed a reference video for this very topic for my company’s junior engineers, and you absolutely nailed it.
Thank you for a new perspective on a possible point of evolution in system architecture. I had never seen events as possible extension points in a system.
Good introduction to EDA, but what about its inherent asynchronous nature ?
I feel it requires to completely change how you approach business processes, going from a synchronous transactional approach to a responsive asynchronous approach.
In the given exemple, it means going from "making sure payment is approved before accepting the order" to "what to do with the order you registered if payment fails".
You implement poor man's rdbms 🤡
I think there is a line where you need to determine what can I defer or let something else handle and what do I need to know right now.
You can build event systems that are async or not, I didn't want to confuse things here. My preference for more complex systems, certainly for complex distributed systems is for async and event based. So I do think that they go together very well, by Windows is a non-async event-based architecture for example.
I talk more specifically about async systems here: th-cam.com/video/IaVPAJQ7iwA/w-d-xo.html
and here: th-cam.com/video/eRxLfUIMJwk/w-d-xo.html
I just started playing with this design method last week - maybe I will give it a more focused effort.
Been a huge fan of domain driven design and this sounds like a great way to orchestrate the system.
Excellent video, excellent format.
I am a big fan of EDA. One of the most underrated things in Software Development is the ease to reason about your code in a way that the business side also understands. EDA helps a ton with that, especially if combined with stuff like Event Storming as you mentioned, but I would mention Event Modeling as well.
Yes. It's a shame. In another comment it's dismissed as something that's the same as event storming. It's not. This was a conclusion with Alberto at the event storming summit in bologna, Italy in 2018
Even driven is like any other tool. Its good for something and not ment for other things. For me the most important is that you can follow the executing code from start to the end without having to analyze the whole code base.
I agree. Imho, using lots of events makes the code harder to follow and very hard to reason about. Also very hard to optimize if you find yourself in that spot. Though events can be great at some places.
Great video. I was working on a project, where we tried to implement event driven arch on aws using Lambdas, Sqs, Dynamo, Event buses etc. It was hard for developers to write code not in the imperative way and it often ended up with another service direct lambda invokes, or millions of retries(which is a pain to handle when some parts of system are not idempotent). Another problem was working with logs of the system. It was the first experience and I liked it very much. Waiting for new videos on this topic!
I personally like very much design and architecture topics! Approved 👊🏼
Nice vid! Nice to hear about a different topic from you 😃
Great introduction to the world of event driven architecture. Love the ability to design systems in this way, and have enjoyed the benefits, especially when dealing with worker processes due to the single thread behavior in node.
However, I generally try to avoid using events for the micros services until there is identifiable benefit of going for the more complex solution. I think that there is often a case of over engineering rather than focusing on simple testable solutions.
This couldn't come at a better time :)) pitched an event system to our app team and could not make clear what I meant. Will link them to this video that clearly makes a better job at explaining than I did.
Yes I would love more like this. I have always been on the side of add events only when you need that scale but I see your side now and how it can make most systems that get complex less so.
A nice simple intro, but I wish you'd chosen a different example, or went into a little more detail. My brain immediately went to "what if they don't have the money?" Adding the dispatch directly off order placed seems to open up a significant hole there. It would have been useful to either choose an example that didn't have this, or even better, show how additional events based on the results of operations like debit can be used to handle these cases. It's still a lot simpler and more decoupled in the EDA case.
Easy, so what is the correct event after which it is ok to dispatch? "OrderPaidFor" perhaps?
Great summary, thank you!
Very informative and well delivered, thank you. More system architecture would be very appreciated.
Great Stuff! Keep it coming
The key to making your system 'growable' is to make sure the events are business events, real world logic. Anything you are asked to add at short notice later will naturally call on real world business events, rather than idiosyncratic programming tweaks you happened to use earlier.
Yes. Thank you.
Yes, please: Do more videos about architecture.
Top stuff, thank you!
Hi Dave,
Thanks for another great video, I love your work.
It would be nice if you could expand on the topic, because I have a lot of questions regarding distributed EDA :
What about
- events that must be processed in chronological order, but do not arrive at the consumer in chronological order
- events that are no longer up to date (newer events made older events no more relevant or accurate)
- confidentiality of events (some event listeners shouldn't have access to some of the content of some events, or even know that some events have happened)
- keep understanding what is going on in the system (when many team members or multiple teams are adding event consumers)
- handling unavailable message broker (the event must be published in order for the necessary logic be executed, so the publisher must handle unavalabilities, hence additional work and complexity there),
- aren't events "duplication of information" at multiple places (publisher, broker, consumers), so a root of problems and to avoid?
These difficulties let me to conclude that distributed EDA should be pull-based (consumers pull events from sources) instead of push-based (with a message broker), but that is less performant, especially if there are many consumers of the same message.
I would love to hear your thoughts.
Best regards,
Great video! In java, Spring makes this super easy. You can setup a bean that returns all types of Interface X, and then that becomes a dependency of the event triggerer, and it just triggers on all registered events. Then you can build a fair amount of your system in a modular monolith, and split it at any time when you've discovered the clear boundary between services. Or, if you're not in need of super scalability, delay that decision as long as you want.
is there a difference between an app that uses events as an implementation detail, e.g. an image editor's undo stack, and an app which models the domain as events?
I used to support/use a CRM that would handle people's address history as a definitive record of state - a location and a start date and an optional end date: from 2018 to 2022 you lived in Bournemouth, from 2022 to present you live in Rotherham, etc. was a nightmare since "don't know" wasn't an option for start dates, no way to distinguish "don't know so am using today's date" from "this is the actual start date", could not insert historic addresses without rewriting the whole address history, etc. my brilliant idea is instead of having an end date field, you'd just record what you're told as structured communication events: "as of today they live in Rotherham", "they moved to Rotherham in 2022", "as of 2020 they lived in Bournemouth" etc. you could record these events in any order and the system would synthesise an address timeline from this.
(this was a social care CRM - needed the "who lived where when" for things like establishing who cohabited or lived near subjects of vulnerable persons protection investigations and the like)
I have built an event-driven platform 15 years ago that was very successful (i.e. still being used right now in two companies). I found that the hardest thing is to robustly implement the request-response semantic. We still used it, sparingly, but that is my single regret. We should have done without it completely.
I definitely enjoy the design and architecture videos more than the usual, which is also enjoyable, but less so.
Great video! There is one thing that I wish would get more focus when discussing event driven architecture - Ordering and aggregates with lifecycle (snapshot or delta events). To get really high performance most queue systems, like AWS SQS, do not garantere ordering. If you are broadcasting events about an aggregate with lifecycle, like an order (placed, updated, filled), then you need to be careful on the consumer side. If the consumer is an active record system, the events need to be versioned snapshot, to prevent overriding an already processed newer state. If the consumer is event sourced, it will probably rely on the producers timestamps to reorder at read. If the consumer is something that needs to process the events in order, in order to produce other events or state derived from multiple aggregate events, then things are not trivial at all. You mentioned GUI event loops and LMAX, the key feature of both is that they give you a globally ordered stream, which is why writing the handler code is trivial. If the events can arrive unordered, it get exponentially harder to write the code that processed the event stream. Unordered queued are good for notifications, or events that can be processed in isolations. If you produce events for aggregate with lifecycle, then use a system like Kafka where ordering is established at ingest time.
Definitely!
Yes I enjoyed this style video
9:16 for that example it's very important that if one operation fails the others do too (eg if the item is out of stock and the order dispatch fails, the payment should fail too, or if the payment fails the dispatch should fail). So maybe in this case you want to do the operations in order (first check if there is order capacity, if there is then charge, if the payment goes through then dispatch), so therefore maybe imperative code would be better for this scenario? or maybe we can model this with event driven design too, and the dispatch service would respond to a paymentSuccessful event, etc? I guess this is where you would bring in some of the things shown at 14:16
I do like this design & architecture style of video
Having recently written a gui based application in C# using events, one sort of trap with events is that once you star having events in multiple layers it starts to become difficult to know how they interact and it gets easy to for example have some gui component update multiple times from a single deeper event resulting in degraded performance. For example, Function 1 listens for event A and once it receves it, it mutates some data and sends a new event B. This event is then picked up by function 2 which further mutates the data and sends event C. Now the gui shows both some part of the data from event A and some from event C. At this point it's very easy to update the gui twice for the same data.
Further, imagine function 2 takes input from two sources, so it now listens to both event B and D. We now want to figure out why the GUI triggers twice each time, so we must backtrack event C into function 2, then backtrack events B and D into their respective functions and so on. It gets complicated, especially if not familiar with the initial code.
@@andreaszetterstrom7418 Event structure requires thought like anything else. You need to create a rational model to have a comprehensible result.
Yes it doesn't eliminate the need for thoughtful design, you can still build messy systems this way, but they are different rather than more prone to messiness IMO.
Or even better, you’ve created an event loop where A triggers B which eventually triggers A again.
Would be interesting to hear how Event Based Architecture fits in with Actors.
I tend to aim at event _sourcing_ for information systems, FRP and dmbarbour's "why not events" for some projects (especially UI), and E's vat model for anything distributed and multiparty, where coordination matters.
thank you
My previous job was maintaining 25y old Delphi5 desktop app and the VCL event bases architecture (which was the basis of the C#/Winforms architecture) never felt dated. Having a bunch of .OnFoo event properties was enough for most of the extensibility needs and for some key parts we made inherited extensions. Not a big fan of the Node.js interpretation, feels a bit clunky but does the job I guess (always liked how the VCL/C# event properties was very selfdocumenting, "this is how you build interaction").
EDA reduces complexity and coupling at the expense of increased awareness of orchestration. The goos thing is that it can be largely mitigated by understanding the business domain and applying frameworks like DDD. EDA requires effectively very good developers, with deep understanding of the principles of systems design.
Great stuff 👏, if you ever want to explore some open source EDA stuff I would love to connect
This pattern is very elegant, I hope you could help me because I'm trying to apply it and I found some things a bit confusing.
Let's say I have two services:
- a generic suggestion service which host a number of suggestions for my admin users. These suggestions are regularly created by different modules for a big number of reasons, and they're centralized here.
- a contract service that handles which contracts is assigned to user.
What I want to implement via EDA is: When a suggestion to change the contract of the user is applied, the contract of the user is changed, AND the suggestion disappears.
My issue is that I don't know where I should start after the user accepted the suggestion via the UI:
1°) (User accepts) -> (suggestion system "applies" the suggestion (change its state to "applied" in DB)) -> (emits a "SUGGESTION_APPLIED { type: "contract-change", id: xxx }" event).
The contract system then consumes this event and changes the contract.
=> OK but
- What about the possibility that the contract change failed at the end, should i emit a "SUGGESTION_FAILED" event, or should I increase monitoring and stay in this gray failed state ?
- How should I take this into account in the UX design to handle these gray states ? Should I just not show the suggestion (because it is applied) or should I add intermediary states to suggestions (like "accepted" -> then emit APPLIED or FAILED events from modules consuming the SUGGESTION_ACCEPTED event) ? This complexifies the subject quite a bit.
2°) (User accepts) -> (contract system changes the contract) -> (emits SUGGESTION_APPLIED event)
=> The emitted event by the contract system should (for my understanding) be "CONTRACT_CHANGED". But a change in contract could happen without a suggestion (via import, UI, etc.), so I'm a bit bothered to consume "CONTRACT_CHANGED" events in the suggestion system. Should I build more specific events like "CONTRACT_CHANGE_SUGGESTION_APPLIED" ?
Event Driven Architecture feels very much like Functional Programming at the system level.
More of ths please.
Would you cover data oriented design. Unity has been about to show some amazing performance improvements with this programming paradigm.
Inversion of control is the essence of not knowing who will execute something. Not sure which comes first though.
Assuming all data contracts need to be versioned then I sent see a big difference as long as an API gateway is involved and you don't care about the response. It's a semantic difference announcing that something happened or telling it.
What is in my opinion the biggest benefit is the retry which the queue concept offers. It's so good that makes the pattern irreplaceable. Everything else is a specialization of the basic principles.
Nice and clear discussion. Happy you stayed away from defining EDAs as "everything is an event", which unfortunately comes up way too often.
The most annoying of which is "everything is a kafka topic"
spent last 2 years doing event driven and pub sub... with wpf and c#
Is EDD standard for offline-first? Maybe you'll answer - I'll keep watching ☺️
Event Driven code is a lot more difficult to read and debug so there definitely is an overhead associated with it. Just like there is a massive overhead added when moving from monolith to a distributed system. There are advantages to event driven code but you have to pay the price.
I think that this is a matter of perspective, and familiarity. Certainly when I first started writing event based systems, I found it difficult to read, but now, to be honest I think it is easier. I think it is an illusion, at least for complex systems, that you can hold everything in your head, you *have* to read and understand them progressively, I think event based systems make that easier.
Eh? Was doing that in the 80's.
Maybe not formalized as such but basically clock interrupts, device interrupts, data inputs (serial or other) drove the majority of processing.
Though (of course) scheduled sampling or signalling also occurred.
IOW the main body of the program was a loop that did nothing other than check some flags or counters to invoke procedures linked by table. A scheduler checks flags or time as set by the clock interrupt and invokes procedures from there.
In some cases queues are used to put actions in order - but (in my case) not all that much.
Now I do much of this in separate threads (main body sleeps) which makes things even "cleaner" in most ways, but can have inter-thread data exchange issues (Interlocked exchanges, etc.) making test more complex.
Anyone who has spent time closer to the metal knows that computers are event-driven beasts. Heck, msdos was invoked as an x86 interrupt. And it goes way farther back and deeper than that. XEROX PARC etc.
@@brownhorsesoftware3605 Indeed, lots of assembler and even Pascal in the past working at the h/w level - no O/S.
Today people write real-time (ish) s/w under various OS's that are not especially realtime, but get away with it due to over resourcing (10 - 100x more CPU than needed).
I avoided the MS-DOS so-called interrupts (they're system calls, fer cripes sake) as much as possible - preferring to read/write around them and direct to the h/w. And real interrupts (KB, serial) I just took over.
Long time ago!
Eric Evans never mentioned Events in the blue DDD book. Ive heard it advised to avoid "event pinball" and tactical events (mentioned in the video).
i havent had a chance to use EDA in a large system, but it seems like good advice to try to use events as updates _between_ bounded contexts at the more strategic scale.
Does Mr Farley have any input on that? It seems like too granular events (userFirstNameUpdated) could be dangerous.
I agree, Eric never talked about the at the Blue Book, I am pretty sure that he came to appreciate their value later. Now in Modern DDD events are very much a part of the approach. My analysis is that this is for the reasons that I mention in the video - Events enable the separation of accidental and essential complexity better than any other approach that I know of. So our services can implement the essential complexity of our system - the core domain stuff, and mostly with good infrastructure we can generalise and delegate most of the accidental complexity outside of that core-domain logic.
I do agree that part of the secret to this is in the choice of events. In general I'd also agree that avoiding events that are too granular is a good guideline. High-level, broader interest, events are ideal, but sometimes a fine-grained "userFirstNameChanged" event maybe just what you are looking for. The good news is that in a good event based systems events are cheap! but it is distributed comms so cheap != free. Thoughtful application is needed!
Yes, is one part of the future today
Everything on a phone is event-driven. I think that fact and the complete absence of msft are why I like to work in that space so much. This comes up against a strong bias amongst developers favoring the command model. My experience on Android is their urge to 'simplify' things by hiding event interaction in the platform. All that does is add cycles and create thread production mayhem. I don't know this because I use these features, I know this by observing what gets deprecated. By taking advantage of the event architecture, I have never had a performance problem.
HAHAHA
I'm really really surprised to see after 15 years somebody talking about CQRS... from the almost forgotten Greg Young
But Event Driven Architecture isn't the same as Event Sourcing as some comments mixed
There is no "exactly once" (see CAP and Two Generals' Problem), *especially* when using external messaging infrastructure as stated in the video around the six minute mark. The best you could do is "effectively once" via "at least once" and some real careful coding of the consumers, but even then it's not a delivery semantic but a processing one.
No, I know, but there are close approximations. You can achieve "exactly once" in practice with a combination of "at least once" and "idempotence"
At the risk of being the least cool person here… the unified modeling methodology from the UNECE also does a useful job of breaking business processes down into activities that correspond to message exchange patterns I can build events around. It’s handy in places where post it notes are not allowed.
would EDD work well with TDD?
Yes they work very well together!
I've been a software engineer for 25 years now.. the main lesson I've learnt is that everyone's trying to overcomplicate stuff needlessly all the time. I say needlessly, it's actually so they can either 1) Sell something 2) Validate their career. Well worth being aware of this.
Web development today works like this due to reactivity and components. Need to watch out though that component emits are labeled like events (this happened) and not instructions (do this).
Nope, All of the first GUIs in the late 70's and early 80's were event based, I first learned to program Windows on V1.0, it had an event loop and each window had a general event handler that was implemented as a big 'switch statement' switching on the type of the event. nothing to do with react or reactive programming, that came MUCH later.
It has pros and cons. But the GUI comparison isn't accurate, precisely at the point where it can go wrong. UI events come from a message queue that exists in a "serial world". They are serially placed and serially dispatched. (distributed) Pub/sub has no such guarantee. It suffers from the same problem as TCP, where a newer "message" can arrive before an "older message". So, your system must NOT care about order OR implement mitigation. And now you might find yourself with a situation akin to the PentiumM/Pentium4 debacle where your new "deeper" architecture is having such deleterious effects that the older architecture was faster.
As with everything, its a "yes, if it fits". As an example, it your "deposit/withdrawal" system is OK with going negative, receiving "withdraw x" before "deposit x" is the same as the other way around. If it's not... you're left with money in your account and a failed withdrawal. Try again? ;)
There is a distinction between event based systems and async event based systems. I think that the combination of async and event based is the strongest one, but it is not essential to the definition of an event based architecture. Windows is an event based architecture, and at its core it still works on a synchronous event loop.
en.wikipedia.org/wiki/Message_loop_in_Microsoft_Windows
Isn't SOA (service oriented architecture) an implementation of EDA?
No, you can validly implement SOA without using the concept, or technology of event based systems.
@@ContinuousDelivery Hmm. Aren't messages that are sent over a message queue equivalent to events?
What's the difference between EDA and Reactive Systems?
There are a few differences, EDA is optionally async, Reactive Systems are Async. In the Reactive Manifesto we said communication was "via Messages" rather than "Events" the manifesto distinguished between them like this...
"A message is an item of data that is sent to a specific destination. An event is a signal emitted by a component upon reaching a given state"
My preference is to use Events in reactive systems, but sometimes a Message is what you want.
Reactive systems also talk about several other ideas that aren't, by definition, part of EDA.
I agree that both are closer to each other than to other type of systems though.
Thanks for clearing that up for me Dave.👍
Engineers will take a completely simple problem, look at it, and go "hmmm, I wonder how I can make the solution so complex that nobody will notice the problem's inherent complexity anymore" sigh :)
A job for everyone on the team!
@@Squeeeez Bad engineers may tend to design complex systems, good engineers will always try to reduce complexity as much as possible.
Event driven is not suitable for ever problem but it does solve some complex problems. Use it appropriately. But that's true for any type of solution.
How can i make it abstract so it’ll be able to solve every other problem in the world!!
"How can I be so dumb to make a stupid post"
To me Visual Basic was one of the first ones to do this
Nope, this approach has a very long history. For example, all device drivers ever were event based systems, all GUIs from Xerox Parc through to Mac OSX are event based systems too, and the first versions of all of these things pre-dated VB by a long way.
Hello, can you talk about the dark side of this? There are tooo less information of the dark side and the problems you get with this approach.
Weird how so much stuff I've been doing since the late '80s is now fashional buzz-words.
No mention of #eventsourcing. And no mention of #eventmodeling. This EDA definition is decades old still engrained in microservices glued by a service bus. The coupling is still there, you just get run time failures instead of at compile time.
My response in Justin's thread (he keeps deleting them): I'm not sure where you draw those conclusions. I've made a post that includes multiple issues with TDD. I'm up for a discussion in a public forum any time. I've arrived at different conclusions but those that don't want to hear it because it doesn't align with how they see things are quick to dismiss as trolling, tone or whatever. It's to the point that my replies get deleted. I'm saving this one and posting it elsewhere too just in case.
@@AdamDymitruk where can I find your post about TDD issues?
Thank your for the video. IMO for small and medium products Event Driven Architecture may be too much. You haven't told about all the difficulties compared to transactional-style approach. You would need more servers, more skilled personal, solving more network problems, availability problems, and many other
@@malchikovma I think I get where you are coming from but more servers? Do you mean a messaging infrastructure? For small projects you don't have to use multiple services, right? But even then, just buy it. Any major cloud provider has excellent services and there's also SaaS rabbitmq or redpanda or what have you.
It's gonna be a shiet show in a couple of months
There are certainly some complexities, but they are fewer, and you certainly don't "need more servers" you can write very good event based systems that run on a single machine, or on many - with the same code.
To me it's always been "gooey", not "gee you eye". Sounds so weird to me.
Really threw me the first time someone pronounced URL as "earl", instead of "you are ell" 😄
@@bc4198 haha, yikes. From now on I'll pronounce HTTP as "hittipee" and TCP as teacup.
I'm going to use "gooey" in my docs from now on.
c.f. SQL etc
Decades ago, I'm talking over 30 years or so, we did pronounce every letter in GUI. That changed after the Internet started gaining popularity.
I remember that time very well.
Ye EDA is fine, if you don't have any sequence of steps in the process and can treat things as fire and forgot. And btw you still have design time coupling in form of event schema.... In my opinion, before an organisation can do proper EDA, they should have several important components such as schema discovery, goverannce, documentation etc in place. PS the example you give is over simplified, the order fulfillment is never that easy and always requires sequence of steps.
In situations where a person A gives input to a system - but A is also responsible for this input to have effect - how is this handled in event driven design in an elegant way?
With no feedback A has to poll all the underlying databases until he sees the effects that corresponds to his input - which is not very nice.
This is irrelevant for casual webshop applications, but a forensic genetics lab assistant would want to see the lasting effect of his input - every time.
In this case you simply implement an event that is generated when the state is as A is interested in. Then A just registers a listener for that event. NO POLLING! Polling is the opposite of event driven.
@@stengrafflarsen are you talking about CDC? I think it has a very narrow use case.
Or are we talking about some kind of organizational responsibility? What's hindering A from just doing whatever after emitting some event? Or why would A be responsible for some downstream reaction? The whole point is not to care about the reaction of your system from the angle of a publisher. The responsibility either moves into "the ether" or to some monitor.
@@brownhorsesoftware3605 That is elegant. You would have to implement listeners for every possible effect in interest - though. In some system there are none or few - in others it may be every effect. I'm thinking pharma industry systems
@@stengrafflarsen You can define a listener that receives any or all or n of any set of events you wish to implement. There is no limit to how the interactions behave. It depends on the use case. A push client can handle any number of events.
@@black-snow A user input event is an intention and it might not have the intended lasting effect because of errors, queues, transactions etc.. In some use cases the user - and not the system - is responsible for the effect. Pharma, court and space are areas where proffesionals verify every change they make. I put the giraffe in the drawer; and I need to see the giraffe was put in the drawer - by Me.
past few years lol.
First