IT would certainly be helpful to also understand in which scenarios each of these would be used. For example, at-least once maybe in a payment service (idempotency handled appropriately), at-most once (comments for a post) etc..
It's to optimize performance, sometimes you can give up accuracy in favor of higher throughput. Exactly once has the lowest throughput of all 3 options, so you should only choose it if you need your data to be super accurate.
Say you're collecting the location coordinates of some objects moving on the map in each second (e.g. taxi cars on Uber, airplanes flying from one location to another). So you can choose "At-most-once" because it's okay to loose some coordinate data in between some seconds and also you need a high throughput in such scenarios.
Bit late too the show, but think about a banking application, having 700€ on the account, and due to a bug 600€ are requested twice. The server will in case of a second duplicate request still reply with an ACK.
message can be delivered more than once, but we have idempotency due to which we don't have to accept duplicate message. It gives once processing. You are giving misinformation
I think the point is that you have to ensure idempotency to guarantee "exactly-once". Or, you can use Kafka Transactional API, you can read about www.baeldung.com/kafka-exactly-once
Awesome video man. Keep up the good work. Lots of love and support. ♥️
Thank you! Will do! Appreciate it.
IT would certainly be helpful to also understand in which scenarios each of these would be used. For example, at-least once maybe in a payment service (idempotency handled appropriately), at-most once (comments for a post) etc..
I appreciate the feedback. Will work on it.
Very clear, thank you!
Great to hear!
Excellent video
What is the advantage of keeping three types
At-most once, at-least once, and exactly once
cant we have exactly once only?
It's to optimize performance, sometimes you can give up accuracy in favor of higher throughput. Exactly once has the lowest throughput of all 3 options, so you should only choose it if you need your data to be super accurate.
++ to everything said here. Thank you!
And how do we do cost estimation?@@irtizahafiz
Hey thanks for the regular uploading. It would be great if you can make videos on job scheduler, and live coding platforms like hacker rank.
Hi! I added this to my content backlog. Can't promise anything soon though. Thank you for watching! : )
Any real-world examples of when one would use 'Atmost once'?
It totally depends on the application tbh. Off the top of my head, I can't think of any.
"At-least-once" is the most common one in my opinion.
maybe when you're consuming access logs for real-time analysis, when the delivery of all the message is not obligatory.
Say you're collecting the location coordinates of some objects moving on the map in each second (e.g. taxi cars on Uber, airplanes flying from one location to another). So you can choose "At-most-once" because it's okay to loose some coordinate data in between some seconds and also you need a high throughput in such scenarios.
Bit late too the show, but think about a banking application, having 700€ on the account, and due to a bug 600€ are requested twice. The server will in case of a second duplicate request still reply with an ACK.
What is the default delivery method in Kafka if there is any ?
I believe it's "at-least-once". But you can easily configure it to whatever you like.
So helpful
message can be delivered more than once, but we have idempotency due to which we don't have to accept duplicate message. It gives once processing. You are giving misinformation
I think the point is that you have to ensure idempotency to guarantee "exactly-once".
Or, you can use Kafka Transactional API, you can read about www.baeldung.com/kafka-exactly-once
Good