most microservice implementations just replace function calls on the CPU stack with network calls and not much else. They are not decoupled, can't handle failures, have no solution for distributed transactions if you have a separate database for each service. Maybe service mesh can help a bit, but it would be better to just not do it in the first place. I think if you don't have more than 20 devs in that project there is no reason to use microservices. They should only be used to scale the organization, not for technical reasons, as performance and scalability will usually be worse with microservices (shocking, I know).
Even when it's stated that the data isn't shared between services it is shared, but through the underlying database, not through direct calls. It's just the keys that are passed that ties everything together so that the data is properly accessed. All that is done when "microservices" is done is to slap a label on top of a good design pattern. How it's realized is a completely different issue. In a way you can do microservices using programs in the cgi-bin area of an Apache web server, servlets in a Tomcat web server or for that matter run individual programs through a text UI. But even then you will have some common objects between the services, most often objects representing data records in a database. In recent language updates like Java 14 there's now a new type called "record" that can be used. This seems to come from the Functional Programming language F#. A bit of a limitation or feature depending on your perspective is that this can't be extended from a base record. Just consider that you have a database where you have a pattern where you always have a primary key column that always shall have the same pattern. Then it would be convenient if all records declare this the same way and that could then constrain the use of the record type. In an ultimate service separation design all data including the key information about what was updated is only provided in the database and the only interface between the services would be a notification to the other service stating that there's new data to process. But regardless of what design pattern that's used you will have to consider that data exchanged between services have to have a consistent format, so if you change the data definition it's still the same problem that has existed since we started to use computers. Adding a new column to a table is usually a minor issue unless you put in a database constraint on it then all writers have to be aware of that, but changing the size of a column or the type of the data in a column in the database is still a potentially breaking issue for all services.
> data isn't shared between services it is shared, but through the underlying database I'm not sure where you got that idea. A service is responsible for everything from database to user interface (or api, or ...) and it doesn't have knowledge of other services, except for the fact that events exists and that identifiers are passed around. Other than that, no data is shared. In other words, they don't need "data exchanged between services" nor does the problem "if you change the data definition it's still the same problem that has existed since we started to use computers" exist. Because of data isnt exchanged, there is no data definition. So it can't be changed. Can't result in issues.
I hoped to hear about the publish/subscribe media he recommends to use as it is crucial for the whole architecture. It is quite hard to manage complexity there. So the event happened, but who processed it, was it processed successfully, if not what to do. In general the approach is good, but some important details are missing.
Thanks for the comment. Follow the link in the end if you want to comment and/or get in touch. I'd love to discuss more. You have several questions and most require very long answers. In short, it doesn't matter to the publisher if any subscriber processes it or not. The idea behind pub/sub is that publishers are decoupled from subscribers and you can add & remove them without the publisher knowing. The idea behind queuing is that message arrive and it varies per queuing technology how reliable this is, whether or not transactions are involved. MSMQ + SQL can do distributed transactions, but most DBA don't allow you to do distributed transactions. Azure Service Bus and RabbitMQ don't support transactions like that and solving that issue can become very, very difficult. 100% reliability is not possible, but we try and do our best. Again, if you want to know more, contact me! :-)
Hello Could You please explain how to reduce complexity if You need a list data that contains data from different bounded contexts? For example - Order 1-*(relation) OrderItem 1-1(relation) ItemPrice You need next columns OrderId | OrderItemCount | TotalPrice If You do not share data You can not use JOIN. And aggregating data via http couse huge coplexity comparing to JOIN.
4 ปีที่แล้ว +1
This seems like you need to compose data and this is usually done in very specific scenarios: either composing to send data over to an external system or composing data on an API, like a BFF. Since they all share an Id, you can get data from different boundaries and compose them under the same root id.
@ You are correct. Mauro Servienti explains this in a series of blogposts here: milestone.topics.it/categories/view-model-composition The thing is, that data in the database doesn't look like what it used to be. There aren't many joins to be made. And of course it is a trade-off. But imagine that your database is full of locks and retrieving data from a database alone is already expensive, versus lightning fast queries but a bit more latency. As an extra reduced coupling and far better maintainability. Not every system benefits from it, but if you're building systems that make you watch this video, it's likely a huge win. But also complex to learn and master.
Given that the events just pass IDs or the listener cares/needs only the ID, how do they get other info if for example they need the product name or product weight in this example. How does the shipping context knows about the product weight? Does it have its own database of products with their weight and other info being encoded? Or does it listen to events from Product Catalog like ProductCreated and stores it own copy of the product info that was just created?
@P Rivacy If you publish data throughout the system, business logic will follow. Bring all the highly-cohesive stuff together, because that has coupling that can't and should not be broken. A lot of other things have accidental coupling and might be separated. Bring it all together again in the user interface using composition. This was discussed by Mauro in this session: th-cam.com/video/hev65ozmYPI/w-d-xo.html
In your case when you create a product, the data should be decomposed by ui micro controllers which belong to their bounded contexts (shipping, invoicing) (like hyenas) and send info they interested in to bounded context data storage for future fulfilling their capabilities. As a shipping isolated module, in order to calculate shipping cost, I need product weight, hence I should tear away that info when user inputs that data on UI.
> Or does it listen to events from Product Catalog like ProductCreated and stores it own copy of the product info that was just created? The read only caches in services are the only practical way, if you limit to ONLY sharing identifiers you basically will only have 1 service because there is no way to implement any interesting business logic with high cohesion otherwise. Like you might want to have some special discount when it's customer's birthday or have shipping add a picture of cake on the box if it was ordered on their birthday. Are you just gonna merge shipping, pricing and crm together into 1 service only because of this, or are you going to cache birthdays in shipping and pricing? Only toy examples are so black and white, in real world you only have 1 service or share some things, but the sharing doesn't have to be a synchronous call but exactly like you suggested a cache that is updated from events. The coupling added is extremely minimal from this compared to the huge loss of cohesion when merging those 3 otherwise independent services together.
Q: Transactions & rollbacks might become a gordion knot: if last action in process fails, callbacks must handle that as rollback for the whole transaction. What if calling service is already dead by that time? Function as a Service is great for cloud & load balancing and to scale processes, but finally sync concurrent data with main-db makes everything worse (Event-Driven or not; and there will be contextual overlapping in an RDBMS somewhere). @32:04 Each widget to receive it's own data is cool. I am just afraid that the (stateless) Init/Open/Call/Await/Close TCP-Connection calls will produce alot of offset (time+bytes)?! Though, this is very interesting stuff - truly not the ultimate answer to everything (but a question how to make things better).
Not sure what you mean by the first question? If a publisher publishes an event, it's happened and can't be rolled back. You should not want to. Especially others cannot decide to roll it back. I'm not sure what you mean with the functions as a service remarks. On the UI composition, I've seen it work several times. It can be even faster than the 'old' way, especially because of the clear boundaries and separation of data having less locks, etc. If you want to contact me, reach out using the link at the very end of the video. My contact details are in it.
most microservice implementations just replace function calls on the CPU stack with network calls and not much else. They are not decoupled, can't handle failures, have no solution for distributed transactions if you have a separate database for each service. Maybe service mesh can help a bit, but it would be better to just not do it in the first place.
I think if you don't have more than 20 devs in that project there is no reason to use microservices. They should only be used to scale the organization, not for technical reasons, as performance and scalability will usually be worse with microservices (shocking, I know).
Most would say distributed transactions aren't a solution either. Often they do more harm than they solve problems.
What is the github link/information for the solution that was shown at the end?
Even when it's stated that the data isn't shared between services it is shared, but through the underlying database, not through direct calls. It's just the keys that are passed that ties everything together so that the data is properly accessed.
All that is done when "microservices" is done is to slap a label on top of a good design pattern. How it's realized is a completely different issue. In a way you can do microservices using programs in the cgi-bin area of an Apache web server, servlets in a Tomcat web server or for that matter run individual programs through a text UI.
But even then you will have some common objects between the services, most often objects representing data records in a database. In recent language updates like Java 14 there's now a new type called "record" that can be used. This seems to come from the Functional Programming language F#. A bit of a limitation or feature depending on your perspective is that this can't be extended from a base record. Just consider that you have a database where you have a pattern where you always have a primary key column that always shall have the same pattern. Then it would be convenient if all records declare this the same way and that could then constrain the use of the record type.
In an ultimate service separation design all data including the key information about what was updated is only provided in the database and the only interface between the services would be a notification to the other service stating that there's new data to process.
But regardless of what design pattern that's used you will have to consider that data exchanged between services have to have a consistent format, so if you change the data definition it's still the same problem that has existed since we started to use computers. Adding a new column to a table is usually a minor issue unless you put in a database constraint on it then all writers have to be aware of that, but changing the size of a column or the type of the data in a column in the database is still a potentially breaking issue for all services.
> data isn't shared between services it is shared, but through the underlying database
I'm not sure where you got that idea. A service is responsible for everything from database to user interface (or api, or ...) and it doesn't have knowledge of other services, except for the fact that events exists and that identifiers are passed around. Other than that, no data is shared.
In other words, they don't need "data exchanged between services" nor does the problem "if you change the data definition it's still the same problem that has existed since we started to use computers" exist. Because of data isnt exchanged, there is no data definition. So it can't be changed. Can't result in issues.
I like how over 20 minutes we are going back from last 19 years super features like web services, micro services to old database foreign keys.
How did you watch the same video as me? That's not really what he said
I hoped to hear about the publish/subscribe media he recommends to use as it is crucial for the whole architecture. It is quite hard to manage complexity there. So the event happened, but who processed it, was it processed successfully, if not what to do. In general the approach is good, but some important details are missing.
Thanks for the comment. Follow the link in the end if you want to comment and/or get in touch. I'd love to discuss more.
You have several questions and most require very long answers. In short, it doesn't matter to the publisher if any subscriber processes it or not. The idea behind pub/sub is that publishers are decoupled from subscribers and you can add & remove them without the publisher knowing. The idea behind queuing is that message arrive and it varies per queuing technology how reliable this is, whether or not transactions are involved. MSMQ + SQL can do distributed transactions, but most DBA don't allow you to do distributed transactions. Azure Service Bus and RabbitMQ don't support transactions like that and solving that issue can become very, very difficult. 100% reliability is not possible, but we try and do our best.
Again, if you want to know more, contact me! :-)
Dennis, what program did you use for your deck, PowerPoint? Asking b/c this deck kicks ass. It doesn't look for feel like PowerPoint...
Hello
Could You please explain how to reduce complexity if You need a list data that contains data from different bounded contexts?
For example -
Order 1-*(relation) OrderItem 1-1(relation) ItemPrice
You need next columns
OrderId | OrderItemCount | TotalPrice
If You do not share data You can not use JOIN.
And aggregating data via http couse huge coplexity comparing to JOIN.
This seems like you need to compose data and this is usually done in very specific scenarios: either composing to send data over to an external system or composing data on an API, like a BFF.
Since they all share an Id, you can get data from different boundaries and compose them under the same root id.
@ You are correct.
Mauro Servienti explains this in a series of blogposts here: milestone.topics.it/categories/view-model-composition
The thing is, that data in the database doesn't look like what it used to be. There aren't many joins to be made. And of course it is a trade-off. But imagine that your database is full of locks and retrieving data from a database alone is already expensive, versus lightning fast queries but a bit more latency. As an extra reduced coupling and far better maintainability. Not every system benefits from it, but if you're building systems that make you watch this video, it's likely a huge win. But also complex to learn and master.
Given that the events just pass IDs or the listener cares/needs only the ID, how do they get other info if for example they need the product name or product weight in this example. How does the shipping context knows about the product weight? Does it have its own database of products with their weight and other info being encoded? Or does it listen to events from Product Catalog like ProductCreated and stores it own copy of the product info that was just created?
@P Rivacy If you publish data throughout the system, business logic will follow. Bring all the highly-cohesive stuff together, because that has coupling that can't and should not be broken. A lot of other things have accidental coupling and might be separated. Bring it all together again in the user interface using composition. This was discussed by Mauro in this session: th-cam.com/video/hev65ozmYPI/w-d-xo.html
In your case when you create a product, the data should be decomposed by ui micro controllers which belong to their bounded contexts (shipping, invoicing) (like hyenas) and send info they interested in to bounded context data storage for future fulfilling their capabilities. As a shipping isolated module, in order to calculate shipping cost, I need product weight, hence I should tear away that info when user inputs that data on UI.
> Or does it listen to events from Product Catalog like ProductCreated and stores it own copy of the product info that was just created?
The read only caches in services are the only practical way, if you limit to ONLY sharing identifiers you basically will only have 1 service because there is no way to implement any interesting business logic with high cohesion otherwise. Like you might want to have some special discount when it's customer's birthday or have shipping add a picture of cake on the box if it was ordered on their birthday.
Are you just gonna merge shipping, pricing and crm together into 1 service only because of this, or are you going to cache birthdays in shipping and pricing? Only toy examples are so black and white, in real world you only have 1 service or share some things, but the sharing doesn't have to be a synchronous call but exactly like you suggested a cache that is updated from events. The coupling added is extremely minimal from this compared to the huge loss of cohesion when merging those 3 otherwise independent services together.
Q: Transactions & rollbacks might become a gordion knot: if last action in process fails, callbacks must handle that as rollback for the whole transaction. What if calling service is already dead by that time? Function as a Service is great for cloud & load balancing and to scale processes, but finally sync concurrent data with main-db makes everything worse (Event-Driven or not; and there will be contextual overlapping in an RDBMS somewhere). @32:04 Each widget to receive it's own data is cool. I am just afraid that the (stateless) Init/Open/Call/Await/Close TCP-Connection calls will produce alot of offset (time+bytes)?! Though, this is very interesting stuff - truly not the ultimate answer to everything (but a question how to make things better).
Not sure what you mean by the first question? If a publisher publishes an event, it's happened and can't be rolled back. You should not want to. Especially others cannot decide to roll it back. I'm not sure what you mean with the functions as a service remarks. On the UI composition, I've seen it work several times. It can be even faster than the 'old' way, especially because of the clear boundaries and separation of data having less locks, etc.
If you want to contact me, reach out using the link at the very end of the video. My contact details are in it.
Sorry, got it entirely wrong first (no more parallel tetris!). I like it! Q: Will all IHandleMessages-Handler stay under the same hood/assembly? .
The actually microservices talk starts and 11:00
@P Rivacy Cuz it's annoying and irrelevant.
@P Rivacy agreed if you don't have the context of what came before today's microservices, it's is easy to make the same mistakes of the past.
I know MSX :) BTW, Nice video!