What is API Idempotency and Why Is It Important?
ฝัง
- เผยแพร่เมื่อ 27 ก.ค. 2024
- Idempotency in APIs is a critical requirement in APIs that create or modify data. In this video, learn what an Idempotent API is, why its important, and how you can create an API that conforms to idempotency. This video is based on an excellent article Malcolm Featonby on 'Making Retries Safe with Idempotency' - aws.amazon.com/builders-libra...
Looking to get hands on experience building on AWS with a REAL project? Check out my course - The AWS Learning Accelerator! courses.beabetterdev.com/cour...
🎉SUPPORT BE A BETTER DEV🎉
Become a Patron: / beabetterdev
📚 MY RECOMMENDED READING LIST FOR SOFTWARE DEVELOPERS📚
Clean Code - amzn.to/37T7xdP
Clean Architecture - amzn.to/3sCEGCe
Head First Design Patterns - amzn.to/37WXAMy
Domain Driven Design - amzn.to/3aWSW2W
Code Complete - amzn.to/3ksQDrB
The Pragmatic Programmer - amzn.to/3uH4kaQ
Algorithms - amzn.to/3syvyP5
Working Effectively with Legacy Code - amzn.to/3kvMza7
Refactoring - amzn.to/3r6FQ8U
🎙 MY RECORDING EQUIPMENT 🎙
Shure SM58 Microphone - amzn.to/3r5Hrf9
Behringer UM2 Audio Interface - amzn.to/2MuEllM
XLR Cable - amzn.to/3uGyZFx
Acoustic Sound Absorbing Foam Panels - amzn.to/3ktIrY6
Desk Microphone Mount - amzn.to/3qXMVIO
Logitech C920s Webcam - amzn.to/303zGu9
Fujilm XS10 Camera - amzn.to/3uGa30E
Fujifilm XF 35mm F2 Lens - amzn.to/3rentPe
Neewer 2 Piece Studio Lights - amzn.to/3uyoa8p
💻 MY DESKTOP EQUIPMENT 💻
Dell 34 inch Ultrawide Monitor - amzn.to/2NJwph6
Autonomous ErgoChair 2 - bit.ly/2YzomEm
Autonomous SmartDesk 2 Standing Desk - bit.ly/2YzomEm
MX Master 3 Productivity Mouse - amzn.to/3aYwKVZ
Das Keyboard Prime 13 MX Brown Mechanical- amzn.to/3uH6VBF
Veikk A15 Drawing Tablet - amzn.to/3uBRWsN
🌎 Find me here:
Twitter - / beabetterdevv
Instagram - / beabetterdevv
Patreon - Donations help fund additional content - / beabetterdev
#SoftwareEngineer
#API
#Idempotency
![[TH] 2024 PMWC x EWC Main Tournament Day 1 | PUBG MOBILE WORLD CUP x ESPORTS WORLD CUP](http://i.ytimg.com/vi/f5Coja1zCzo/mqdefault.jpg)
The Simplicity with which you explain things is just awesome!, also thanks for sharing the reference article.
So clear and concise. Thank you!
They never teach this in schools. Saves me alot of time and headaches. Im trying to work out Payment channel apis to my app and I don't know what the hell is the x-idempotency-key in request header stated in the documentation. You earned a subscriber.
Just such a simple and great explanation. Thanks!
Thanks for this video !simple and clean explanation
Super clear explanation, thank you!
Awesome walk through! Thanks for the great content.
Glad you enjoyed it Malcolm and thank you!
I have already implemented this thing but I don't know it is actually a concept in programming.
Thank you Daniel.
Now I can show to my boss that I have used something unusual. And which is called idempotent.
😃😃
Great video 👍 Perfect explanation right to the point!
Thanks Max! Glad you enjoyed !
Great video!!!! thanks Daniel... this channel is one of the best!!!
Thanks Christian! Glad you enjoyed :)
Your videos are so good I can feel my brain consuming and enjoying the information.
Great video as usual 👍
Awesome explanation AND presentation! I’ve built countless rest endpoints and consumed countless more clientside but always felt there’s too much room for these edge cases and furthermore wondered how could clients compensate for the edge cases. Thanks! Subscribed!🎉
Glad it was helpful!
Great explanation!
Glad it was helpful!
Amazing video, thanks
Glad you liked it!
Really nice 👍 simply explained
You're very welcome!
Thanks for sharing!!
My pleasure!!
Very nice!
Thanks
Nice explanation
Thank you!
thank you
You're very welcome !
How the client in the second call it's supposed to generate the same token id as the first one? Won't just generate a brand new one? Didn't get that part
I didn't get that part either...
without proper rate limit that can lead to DOS by creating infinite requests with new tokens, overwhelming the database. you can intercept the request and change the token!
Thank you, it was a very clear explanation. I have a doubt, about the case the retry is made before the first request has been processed by the server. I only think of returning an response stating the request is in process. What should the server return in that case?
Client-side retries only make sense in very fast operations, and in this case, the retry time should be configured to be long enough. If the client's request is stimulating a long-running process, then you would need to design the solution differently. One common way is making the client's call be asynchronous and the server be the one responsible for doing everything that needs to be done (and retrying if it needs to). The server would also offer API's that inform in which stage the processing of the request is.
Keep in mind if you did use a cache, it would need to be distributed . A cache on a web server in a load balanced and distributed environment would lead to a miss. Without a distributed cache, the database would need to be the source of record (a uniqueness constraint would do it).
Can you make a similar video on the same thing but now with external api’s that you not control? For me that is an issue. Where the external api is not idempotent and I need to handle this myself. Currently when i need to retry i first do a get request to see if it is there, if yes mark completed. If no send request again.
I don't understsand the logic behind when the client creates a new token/uuid? If the client wanted to create 2 accounts they'd create a different token on the 2nd request, but if the client wanted to create 1 account and didn't receive a response on the first try and they clicked again wouldn't that create a different token and the server wouldn't know it's the same request?
Hello I am really learning from you aws videos but I don't know how to reach you. I am facing a cuncurrency limit issue after running glue job. I am new to my project but I don't know what can help. can you suggest me something.
I am assuming the tokens maintained in the stale store need to outlive the user session? Because if the user remains logged in and retries after the stored tokens are deleted, we'd run into the same issue all over again.
Or we can also have a acknowledgement from client indicating that request is successfully handled and discard the token, along with your solution. Acknowledgement solution on itself may not work as acknowledgement itself can get lost.
User session shouldn't have anything to do with the token here. Token should be stored with your DB record.
How does this work when there is a load balancer. Meaning, the first time token is saved in a different server but the 2nd request goes to a different server because of a load balancer?
really nice explanation
Glad you think so!
how exactly is the client supposed to generate the request token, is it the same idea as before (i.e. using some specific hash function)? if so, how would multiple clients know what function to use if this were a publicly exposed API? sorry if this is a basic question, but that was the only part confusing me. thanks in advance, awesome video.
Actually client won’t call the hash function, api gateway have the function (assume like lambda function) so any client calls the Api that function will execute . The hash will be generated based on payload basically all this hash function code will be in backend. If payload is same it know the request came before as stale store have this hash.
@@RAJU9622 are u saying im terms of flow,
1. Client call api
2. Lambda hash func executes
3. Lambda calls or fowards request to the actual api
Is this the flow? Seems expensive
@@kaypakaipa8559 yes it’s expensive but need to do
This can be developed too with some kind of "cache" approach, right? Let's say I don't want to store my tokens in the DB because they refresh after certain periods of time, and instead, I use a temporal cache to store them and as long as the token hasn't refreshed, whenever the client makes an API request I check my cache and if the token already exists I can return a { "success": true } response to the client. Is that correct?
+1
Hi Nahuel, good question. A cache is certainly a viable solution to store the tokens. I would just make sure you use a distributed cache so that any host will be able to assess if it has seen a similar request in the past (even if it was processed on a different host). A TTL feature on the cache is also a good idea!
@@BeABetterDev nice! Thanks for the insight :) and thanks for the quick response and the content! I hope you are having a nice day.
But do you still want to make the API idempotency after cache expires?
Thank you for sharing this.
A small doubt.. let's say my response is huge with many json fields which are the result of many operations at service layer. If any retry happens, my server can check for the token in store but how can I generate the identical response in this case? I think I shouldn't store each response in any kind of store and map it with token otherwise these big blobs may take significant space.
Can you please suggest any way around this?
Thank you once again sir.
Your response should be generated by your DB record.
I like my entity-creation APIs to return the created entity. I don't want them to just return { "success": true }. Is that possible with these tokens?
Hi serggie, yes thats a much more standard pattern and what I do in my usual API implementations.
Maybe you can link or attach the resource that has been created (or the response as it is supposed to be sent to the client) , then when the token comes in again, you could retrieve the complete response ...
Return id of resource, and success true
Could you tell me what software is used to draw the picture？
Hi this is done with Google slides.
OIDC!
Is there a code example of this concept?
#wan
Apparently, I've been pronouncing idempotency COMPLETELY wrong for a long time.
witn another word, you are NOT making it 100% idempotent by simply using PUT, you are just saying your api consumer that "this is a put request, so you should expect the same result as long as you do the same request" but actual idempotency is depent on your logic. If your API method has a logic which creates a record in database every single time its requested, then having a PUT method does NOT provide any idempotency automagically. Again its just a convention, like a documentation of your API. You actual implementation of the API method is the one if it is actually idempotent or not.
thank you
Welcome!