Awesome Video Just to understand payment flow too. what should be the order for payment here 1. Reserve -> Payment-(if payment fails then remove reservation, exit)> Assign (if assign fails then refund) 2. Success Payment->Reserve(if reserve fails then refund, exit) -> Assign (if assign fails then refund)
Payment flows are slightly different. Depends on the API exposed by then payment gateways. They do expose orders which acts as reservations but not concrete like distributed transaction. By the way, you should never model payments as a distributed transaction. It is always designed with eventual consistency in mind given how banks work.
@AsliEngineering - what if in the placeOrder() we are able to make 3 api calls but the app instance crashes down before we book the agent for the order? I guess this case is not going to be handled.
I'm having difficulty understanding, how is the code representing distributed transactions?If I take it on literal face-value, there are no 2 different datasources so current scenario could have been done under same transaction, why going through the pain of taking locks and everything? Distributed transactions are hard because there are 2 or more services/datasources etc which don't share any transaction id/object/etc. This example doesn't fits the scenario. Please correct where I'm wrong. Thanks.
Distributed does not just mean having two machines. It means having completely isolated processes/threads. The example I took mimics that. You can use this very example across machines and it would work. also,.I never ran entrie thing in one transaction.
Quick question out of curiosity... There are scenarios when 1 agent gets 2 orders due to nearby geological of order delivery address ... How such things can be handled??
Delivery service's delivery_agent/reserve api can have this logic in real scenario. And the same api can return the agent id so that the order service can book the same agent in the 2nd phase of the commit
Could you please explain why did you use Select for update and then update it? I know it will lock that row in a write exclusive lock so that no other transaction can update that row. But what would happen if we directly update that row by using "update agents set is_reserved=true and order_id=1 where is_reserved = false and order_id = null"?
Consider that there is one delivery agent and if we do not lock it then both the transaction might update the same row and both would think that it reserved the agent. In reality one ohs while the other has not.
@@AsliEngineering How does it work in the case of replicas? Let's suppose a transaction updates is_reserved=true but other transactions might read stale data and assign it to some other agent
But find and update is atomic so not sure if select for update is really needed here. In case of replicated dbs writes go to primary so select for update is run on primary not on secondary
Hi Aprit, when we reserve a food packet and delivery agent is not available, then should we not rollback the reservation since the order itself is not placed?
This channel is a gold mine, appreciate the efforts you put in.👏
🙌
thank you for the code demonstration. it really solidified the understanding. I really need to learn go.
Your videos have so much information and to the point details in them, I am lucky that youtube has recommended me your videos
🙌🙌
This channel is a gold mine indeed
Great video, keep adding more videos !
Awesome Video
Just to understand payment flow too.
what should be the order for payment here
1. Reserve -> Payment-(if payment fails then remove reservation, exit)> Assign (if assign fails then refund)
2. Success Payment->Reserve(if reserve fails then refund, exit) -> Assign (if assign fails then refund)
Payment flows are slightly different. Depends on the API exposed by then payment gateways.
They do expose orders which acts as reservations but not concrete like distributed transaction.
By the way, you should never model payments as a distributed transaction. It is always designed with eventual consistency in mind given how banks work.
You channel has an addiction. Amazing content, loved it ❤️
This was top notch, mazza aaya!
Good example, nice explanation, thank you.
Literally this was an amazing hit
Good video. Looking forward for implementation of Paxos and Raft as well.
Absolutely beautiful!!
Amazing man, just amazing!
Where is the link to the previous video? I can't seem to find it
Very Informative channel and content . Keep it Up !! 👏👏👏
Thanks 🙏
This video got me subscribed to your channel, Amazing content. Loved the implementation bit :)
Thank you Nidhi!
we can make transaction in only commit phase itself....if both commit phases goes fine then accept order or else throw unacceptable error...…isn't it?
@AsliEngineering - what if in the placeOrder() we are able to make 3 api calls but the app instance crashes down before we book the agent for the order? I guess this case is not going to be handled.
Thanks awesome! This is high quality explanation.
Thanks 🙌
Appreciate your efforts 👏👏👏
Mazaa aa gya bhai sahab
Great video loved it. btw which vs code theme your are using?
Material. Mostly.
Thanks nice explaination but how we can implement timer thing..
1. Threads and executors 2. Distributed Event Generator 3. Specialised database like Redis
I'm having difficulty understanding, how is the code representing distributed transactions?If I take it on literal face-value, there are no 2 different datasources so current scenario could have been done under same transaction, why going through the pain of taking locks and everything? Distributed transactions are hard because there are 2 or more services/datasources etc which don't share any transaction id/object/etc. This example doesn't fits the scenario.
Please correct where I'm wrong. Thanks.
Distributed does not just mean having two machines. It means having completely isolated processes/threads. The example I took mimics that. You can use this very example across machines and it would work.
also,.I never ran entrie thing in one transaction.
Quick question out of curiosity... There are scenarios when 1 agent gets 2 orders due to nearby geological of order delivery address ... How such things can be handled??
Delivery service's delivery_agent/reserve api can have this logic in real scenario. And the same api can return the agent id so that the order service can book the same agent in the 2nd phase of the commit
12:45 order service is acting as an orchestrator [* link: orchestration vs choreography]
Could you please explain why did you use Select for update and then update it? I know it will lock that row in a write exclusive lock so that no other transaction can update that row. But what would happen if we directly update that row by using "update agents set is_reserved=true and order_id=1 where is_reserved = false and order_id = null"?
Consider that there is one delivery agent and if we do not lock it then both the transaction might update the same row and both would think that it reserved the agent. In reality one ohs while the other has not.
@@AsliEngineering How does it work in the case of replicas? Let's suppose a transaction updates is_reserved=true but other transactions might read stale data and assign it to some other agent
I believe, in this case, reading a right state is critical and critical read or writes should go to master instead of read replicas.
But find and update is atomic so not sure if select for update is really needed here.
In case of replicated dbs writes go to primary so select for update is run on primary not on secondary
Is it synchronous or asynchronous? Is it scalable in real toy environment?
distributed txn have to be synchronous. if you need strong consistency across machines, you have to do this.
💥
Hi Aprit, when we reserve a food packet and delivery agent is not available, then should we not rollback the reservation since the order itself is not placed?
Yes. We would roll back. Plus the timer would ensure that in case we don't the lock is auto-released.
@@AsliEngineering Okay
awesome content .. could you please suggest some beginner friendly resource?
This is beginner friendly 🙂 would recommend going through it twice or thrice. More importantly prototype it locally.
both parts of videos could have been under 30 min (total) if there is no repetitive talk.
Sahi hai. Reel daalna chalu karta hu mai ab.
@@AsliEngineering 😃
Is there an github repo for this code
Don't remember. Check my GitHub profile or description of this video.
👌👌
Tried 2PC logically However, it's difficult in practice.
can you please send the github link of this code
There is no GutHub repo holding this code. It was a one time short code that I wrote.
I've fallen to the right place probably 🤷🏽