The example of the dark room made a lot of sense to me to explain the way in which a person is acquiring knowledge and understanding boundaries. I just finished my first year in a new job and the onboarding was quite difficult, because it is not enough just to listen to a good explanation with a bunch of unknown terms... at that moment I only see general shapes without details and with the passing of time I was be able to recognize the differences between each shape. Thank you!
I've been having similar thoughts recently. "don't do microservices until it's clearly obvious you should do so" I keep thinking. As long as you structure you're monolith like you would as services, then when you're ready to break out then it's fairly simple and easy to pull those pieces a part. However if it remains mostly a monolith over the long term, then that's a win, as you've just dodged the massive costs/complexity associated with distributed systems.
I've talked Logical vs Physical in this video (th-cam.com/video/kDVcAcCSvdI/w-d-xo.html) which really a lot of what I'm describing here this video. And yes, agree, once you start deploying (physically) then you have to deal with a whole another set of complexities.
One caveat: the situation that makes it "clearly obvious" you should do microservices is when your application fails to scale. That's a dire situation to be in, so the threshold should probably be a bit lower for breaking things into microservices.
Great points all, but most of us are not working on greenfield projects in a new and unfamiliar domain but on legacy systems where the domain is well-understood by the business but less so by the developers.In such a case, creating the bounded contexts around business capabilities must be the first step in any modernization. The service boundaries then follow, from which an evolutionary path (e.g. strangler pattern) can be pursued that results in a collection of (micro)services. One final point I want to add is that keeping schemas separate inside a common physical database requires extreme discipline. People will create views, queries, and such, joining tables from different schemas when it is convenient to them instead of creating an abstraction (dare I say: service) to keep the schemas decoupled. Having different database services for different bounded contexts eliminates this temptation even on a green field, as well as allowing polyglot persistence employing the optimal database technology for each bounded context.
FWIW, I also created this video the discussing moving from a Monolith to (micro)Services. Also a lot of the other videos I've done revolving around a Loosely Coupled Monolith talk about boundaries. th-cam.com/video/Inscvakv5XI/w-d-xo.html
Imo the sweet spot for 2021 is inbetween: - Identity Provider - API Service - SPA that requires login - SSR Website The Identity Provider can be 3rd party from Microsoft or Google and the SSR Website something simple.
I'd call it 'Micro-service ready monolith'. 3:12 My worry here is that people listening to this will associate bounded contexts with micro services, which sometimes may or may not be the case. I would strongly discourage this association, as a bounded context can make use of many micro-services. Love your vids man, keep it up.
Thanks. Well if they make that association between a boundary and a microservice, they need to watch my video on physical boundaries aren't logical boundaries.
I think the term has been associated with a Bill Ball of Mud. However, "microservices" that are interconnected by synchronous HTTP are a distributed Ball of Mud.
Great video. I'm not entirely clear on the message broker. What am I getting in return for the added complexity apart from less coupling? Maybe you can do a video on it.
That's it, coupling. If you choose to couple via in-process calls between boundaries, say via interfaces, then if you want to carve out a boundary into it's own service, you then need to make those in-process calls over the wire (RPC).
This was an excellent video, but while this is the BEST way to go right now, it's not the way it should be. We go to monolith first because we understand them better and have absolutely great tools to design and test with. I don't even know of a tool for microservices that would allow me to design and test in an effective manner. Until those tools exist, you've described the best way to get there, IMO... Thanks!
Good old formal modular design that's been around for at least 40 years... Modula-2, as an example, even had coroutines to allow each part to apparently execute autonomously even on a uniprocessor with an OS that didn't support threading. Many programmers need to read more history about their profession. ;-)
Correct (ish). Above that I'm describing SOA and boundaries of service (modules) at a conceptual level that can be transferred to the physical level. At the end of the day it's coupling and cohesion.
Hi Derek, I really like the idea of a vertical slice architecture but I’m having trouble with fully working out how to work with referential data between boundaries. It’s easy to join a table if both of them are in the same context, but what do you do if they are not? I have worked with microservice solutions where each microservice had it’s own copy of, let’s say, a user table, containing minimal information about them, while the user service was the boundary for CRUD user operations. But if that data is already in the same database, would it make sense to have a minimal representation of user in a non-user context and join it for read only operations? That way I’d still be touching other contexts tables though. I hope I’m not explaining it clearly. Perhaps you have a video about it?
You can have multiple commands and queries (slices) that share the same underlying database, data models, and domain models. A group of features (slices) may exist within a logical boundary. When you want to cross that logical boundary and get data from another boundary, you have a few options. If you're in a monolith, then expose an interface/delegate/etc to that returns read-only objects.
If you use a microservice architecture, and want to make a transaction that relates to two or more microservices (e.g. sales and warehouse). How do you ensure that the transaction is atomic?
Can you create a discord channel for the 'developer members' to discuss the detail/coding level to help each other to solve very specific problems? I think it's great way to support your channel and also grow this community.
I think there are several irrational motives why developers choose to use Microservices which we should not overlook. Partial list of which includes 1. The experience - I can't recall the last time i saw a Back-End Developers position where Microservices are not listed at least in a Desired category. 2. Hype - Microservices are shiny new thing that everyone talks about with an excitement, developers naturally are attracted and want to be part of that 3. Challenge - Talented developers are tend to seek for the complexity and tackle them, it produces lot of pure joy from the process. From the Project's Success point of view, arguably, above listed motives have less priority then the 1) Time Project takes to be implemented 2) How easy it is for a newcomer to join a project 3) How easy it is to investigate a bug, etc. Irrational motives of our decision making (generally as a human being) play an important role, there is an an interesting discipline called Behavioural Economics which studies such.
Great video, just a little unclear on how to separate different entities that reference each other. For example, let's say there's a Vendor, and it has one or more Users. If the User can not reference the Vendor schema, how would they know about each other? Or is that something that is recorded in the event?
@@CodeOpinion What I got out of the video (might be possible I misunderstood it), is that the best way to set up a Monolith, prior to refactoring it into Microservices, is to have clear boundaries. I also saw that you have a separate database, or schema for each entity, to have it separate from each other. However, how does that translate to when two entities have a relation with each other. Let's say there's a one-to-many relation between a User and an Address. Is then advices to have a foreign key for user_id in the address table? Or is this information stored somewhere else? TLDR; how can, or should, you keep two entities separate when t here is a relation to each other.
@@Bakhtaryan The answer is events. For exemple imagine you have entities User and Order. In a Monolith these two has a relationship, maybe you will have another entity Product, and Order will contain the user_id and the product_id. In Microservices you could just create an event, every event has a type for example orderCreated, and the order service would listen to this event, which will give him the user_id and the product_id...
@@jaysizmir9432 yes that's that's write part of it. but what about the read? Let's say I have an event OrderCreated with {"userId": 123, "orderId": 456, "products: [1, 2]"}. Assuming each has it's own database, the respective data gets written to it. But let's say I want to show all the orders that userId 123 has. How can I show this? I saw in the projections video how each product had an updated storage, based on the event [added, shipped or adjusted]. But those event only affected a single entity. How does this work when multiple entities are depending on each other and you want to read data.
@@Bakhtaryan Nice question. You have for example to create another service which will be only responsible for this task. So the service will listen to some events tho. To be clear this is Asynchronous communication. Details for our new service : you will listen whenever a user is created (the event type userCreated for example), you will store the user and an empty array for his orders since he has not ordered anything yet. When an event order occurs, we will go to the service and push in the user array the product infos that we need (product_id, image, description....) . Advantage : Imagine the user service is down, we will still be able to show to the users all the products that he ordered with the service that we just created...
First Building a monolith with microservices principles: Who's going to pay for? the budget is a critical point in software process. one maybe has time/money to kill:). However that is interesting view point.
Great point, absolutely budget is in play! Hence why I think having boundaries within a monolith are important with less complexity then separately deployed services. Even if you don't use async messaging between boundaries, but rather in-process method calls, boundaries are important. I appreciate the feedback.
The example of the dark room made a lot of sense to me to explain the way in which a person is acquiring knowledge and understanding boundaries. I just finished my first year in a new job and the onboarding was quite difficult, because it is not enough just to listen to a good explanation with a bunch of unknown terms... at that moment I only see general shapes without details and with the passing of time I was be able to recognize the differences between each shape. Thank you!
As time goes on, even in a different domains, you'll draw on past experience to relate to the new domain. Glad it was helpful.
i made this mistake hard. I wish I had watched this video before going crazy with the microservices. Thank you.
I Agree, I have my Green Field Project in Monolithic as well. But I have the bounderiea in Projects with Shared Kernel.
Yup, common. Just be careful what's in the shared kernel. Often times can be just a dumping round. Just food for thought.
That's Great Idea or way to learn Microservices..Thanks for sharing this thought.👍
Thanks for watching!
I've been having similar thoughts recently. "don't do microservices until it's clearly obvious you should do so" I keep thinking. As long as you structure you're monolith like you would as services, then when you're ready to break out then it's fairly simple and easy to pull those pieces a part. However if it remains mostly a monolith over the long term, then that's a win, as you've just dodged the massive costs/complexity associated with distributed systems.
I've talked Logical vs Physical in this video (th-cam.com/video/kDVcAcCSvdI/w-d-xo.html) which really a lot of what I'm describing here this video. And yes, agree, once you start deploying (physically) then you have to deal with a whole another set of complexities.
One caveat: the situation that makes it "clearly obvious" you should do microservices is when your application fails to scale. That's a dire situation to be in, so the threshold should probably be a bit lower for breaking things into microservices.
Now THIS I can jive with! (hugely prefer monoliths, even for medium sized teams, with my limited experience) ty for the videos!
Thanks for watching them!
Thank you so much for sharing your knowledge and insights! Love your content so much and the quality is insanely good!
Glad you enjoy it!
Great points all, but most of us are not working on greenfield projects in a new and unfamiliar domain but on legacy systems where the domain is well-understood by the business but less so by the developers.In such a case, creating the bounded contexts around business capabilities must be the first step in any modernization. The service boundaries then follow, from which an evolutionary path (e.g. strangler pattern) can be pursued that results in a collection of (micro)services.
One final point I want to add is that keeping schemas separate inside a common physical database requires extreme discipline. People will create views, queries, and such, joining tables from different schemas when it is convenient to them instead of creating an abstraction (dare I say: service) to keep the schemas decoupled. Having different database services for different bounded contexts eliminates this temptation even on a green field, as well as allowing polyglot persistence employing the optimal database technology for each bounded context.
Yes good points! Thank you for the comment. I will likely follow up on some of these in other videos, especially working in a legacy system.
FWIW, I also created this video the discussing moving from a Monolith to (micro)Services. Also a lot of the other videos I've done revolving around a Loosely Coupled Monolith talk about boundaries.
th-cam.com/video/Inscvakv5XI/w-d-xo.html
Imo the sweet spot for 2021 is inbetween:
- Identity Provider
- API Service
- SPA that requires login
- SSR Website
The Identity Provider can be 3rd party from Microsoft or Google and the SSR Website something simple.
I'd call it 'Micro-service ready monolith'.
3:12 My worry here is that people listening to this will associate bounded contexts with micro services, which sometimes may or may not be the case. I would strongly discourage this association, as a bounded context can make use of many micro-services.
Love your vids man, keep it up.
Thanks. Well if they make that association between a boundary and a microservice, they need to watch my video on physical boundaries aren't logical boundaries.
Esse vídeo era exatamente o que eu estava procurando
Good to hear the Monolith is alive, for some people it has become a bad word
I think the term has been associated with a Bill Ball of Mud. However, "microservices" that are interconnected by synchronous HTTP are a distributed Ball of Mud.
Great video. I'm not entirely clear on the message broker. What am I getting in return for the added complexity apart from less coupling? Maybe you can do a video on it.
That's it, coupling. If you choose to couple via in-process calls between boundaries, say via interfaces, then if you want to carve out a boundary into it's own service, you then need to make those in-process calls over the wire (RPC).
I believe you're referring to what Simon Brown popularized as "Modular Monolith". Easier to remember. ;)
This was an excellent video, but while this is the BEST way to go right now, it's not the way it should be.
We go to monolith first because we understand them better and have absolutely great tools to design and test with.
I don't even know of a tool for microservices that would allow me to design and test in an effective manner.
Until those tools exist, you've described the best way to get there, IMO... Thanks!
Ya for sure. Things change over time. We'll see where things land in the next 10 years. I appreciate the comment!
wonderful!! thanks!!
Also it seems like all the benefits the monolith gives to us could be achieved with monorepos
Good old formal modular design that's been around for at least 40 years... Modula-2, as an example, even had coroutines to allow each part to apparently execute autonomously even on a uniprocessor with an OS that didn't support threading. Many programmers need to read more history about their profession. ;-)
Correct (ish). Above that I'm describing SOA and boundaries of service (modules) at a conceptual level that can be transferred to the physical level. At the end of the day it's coupling and cohesion.
Hi Derek, I really like the idea of a vertical slice architecture but I’m having trouble with fully working out how to work with referential data between boundaries. It’s easy to join a table if both of them are in the same context, but what do you do if they are not? I have worked with microservice solutions where each microservice had it’s own copy of, let’s say, a user table, containing minimal information about them, while the user service was the boundary for CRUD user operations. But if that data is already in the same database, would it make sense to have a minimal representation of user in a non-user context and join it for read only operations? That way I’d still be touching other contexts tables though. I hope I’m not explaining it clearly. Perhaps you have a video about it?
You can have multiple commands and queries (slices) that share the same underlying database, data models, and domain models. A group of features (slices) may exist within a logical boundary. When you want to cross that logical boundary and get data from another boundary, you have a few options. If you're in a monolith, then expose an interface/delegate/etc to that returns read-only objects.
Thank you for the great video! What is "small" team in numbers?
I'd say under 10.
needed this
Hope it helps!
If you use a microservice architecture, and want to make a transaction that relates to two or more microservices (e.g. sales and warehouse). How do you ensure that the transaction is atomic?
You are looking for 2-phase commit
Check out microservices.io/patterns/data/saga.html
You want to implement event orchestration with compensating actions. Check out the video I did on it:
th-cam.com/channels/3RKA4vunFAfrfxiJhPEplw.html
Can you create a discord channel for the 'developer members' to discuss the detail/coding level to help each other to solve very specific problems? I think it's great way to support your channel and also grow this community.
This is a great idea! I'm going to try to put something together soon to get that up and running.
@@CodeOpinion Awesome! Looking forward to it!
I think there are several irrational motives why developers choose to use Microservices which we should not overlook. Partial list of which includes
1. The experience - I can't recall the last time i saw a Back-End Developers position where Microservices are not listed at least in a Desired category.
2. Hype - Microservices are shiny new thing that everyone talks about with an excitement, developers naturally are attracted and want to be part of that
3. Challenge - Talented developers are tend to seek for the complexity and tackle them, it produces lot of pure joy from the process.
From the Project's Success point of view, arguably, above listed motives have less priority then the 1) Time Project takes to be implemented 2) How easy it is for a newcomer to join a project 3) How easy it is to investigate a bug, etc.
Irrational motives of our decision making (generally as a human being) play an important role, there is an an interesting discipline called Behavioural Economics which studies such.
Agree, a bunch of good observations that I think are very true. I appreciate the comment.
Great video, just a little unclear on how to separate different entities that reference each other. For example, let's say there's a Vendor, and it has one or more Users. If the User can not reference the Vendor schema, how would they know about each other? Or is that something that is recorded in the event?
Not exactly sure what you mean. Can you elaborate a little more?
@@CodeOpinion What I got out of the video (might be possible I misunderstood it), is that the best way to set up a Monolith, prior to refactoring it into Microservices, is to have clear boundaries. I also saw that you have a separate database, or schema for each entity, to have it separate from each other. However, how does that translate to when two entities have a relation with each other. Let's say there's a one-to-many relation between a User and an Address. Is then advices to have a foreign key for user_id in the address table? Or is this information stored somewhere else? TLDR; how can, or should, you keep two entities separate when t here is a relation to each other.
@@Bakhtaryan The answer is events. For exemple imagine you have entities User and Order. In a Monolith these two has a relationship, maybe you will have another entity Product, and Order will contain the user_id and the product_id. In Microservices you could just create an event, every event has a type for example orderCreated, and the order service would listen to this event, which will give him the user_id and the product_id...
@@jaysizmir9432 yes that's that's write part of it. but what about the read? Let's say I have an event OrderCreated with {"userId": 123, "orderId": 456, "products: [1, 2]"}. Assuming each has it's own database, the respective data gets written to it. But let's say I want to show all the orders that userId 123 has. How can I show this? I saw in the projections video how each product had an updated storage, based on the event [added, shipped or adjusted]. But those event only affected a single entity. How does this work when multiple entities are depending on each other and you want to read data.
@@Bakhtaryan Nice question. You have for example to create another service which will be only responsible for this task. So the service will listen to some events tho. To be clear this is Asynchronous communication.
Details for our new service : you will listen whenever a user is created (the event type userCreated for example), you will store the user and an empty array for his orders since he has not ordered anything yet. When an event order occurs, we will go to the service and push in the user array the product infos that we need (product_id, image, description....) .
Advantage : Imagine the user service is down, we will still be able to show to the users all the products that he ordered with the service that we just created...
First Building a monolith with microservices principles: Who's going to pay for? the budget is a critical point in software process. one maybe has time/money to kill:). However that is interesting view point.
Great point, absolutely budget is in play! Hence why I think having boundaries within a monolith are important with less complexity then separately deployed services. Even if you don't use async messaging between boundaries, but rather in-process method calls, boundaries are important. I appreciate the feedback.
100% agree
👍👍
Who can explain me this video by doing meet?)