Notification Service System Design Interview Question to handle Billions of users & Notifications
ฝัง
- เผยแพร่เมื่อ 27 ส.ค. 2024
- Solution for System Design Interview Question - "Design a Notification Service, that would be used to send Email/ SMS/ Push notifications to billions of users."
This is being asked by a lot of companies lately like Twitter, Uber, Google, Amazon, etc.
Prerequisites:
How to select the right Database for a Large Scale System: • Database Design Tips |...
Summary of the video: www.codekarle....
Architecture diagram: github.com/cod...
Author: / sandeep1904
If you like this video, please help us grow by sharing this video with your friends on Facebook, connections on LinkedIn and anyone who can benefit from this.
PS: This is not the real architecture of any such platform. This is my take on how I would answer that problem.
#codekarle #systemdesign #notification #system #design #interview #amazon #faang #sms #email
Just came across your channel. One word "FABULOUS". Keep up the good work.
Future suggestion:
1. Web crawler
2. Yelp
3. Api limiter and BOT blocker
Thanks for the kind words. Do share it in your circle.
Crawler is coming soon. Rest I'll try to do when there is some free time from work :)
@@codeKarle Api limiter is also a very common thing being asked and would love to see your extensive design perspective on the same :)
Could please share video link for these topics
why have you stopped making videos ? your content is of very high quality. so much condense & clear information. you're a great teacher.
I agree. No unnecessary noise and bulshit only hit on bull's eye. Great content
Yes. He is the best. But not sure why he has stopped and also not able to reach him. Absolutely awesome explanation with crystal clear clarity.
@@Nexatek He is working at Facebook . He might be busy. But , he really best
only TH-camr who talks about the components E2E with so much clarity, really appreciate it.
Super and correct - I myself was part of Notifications Team in a big product base company. And the architecture was same.
This is to "notify" everyone that this video is the best system design video for notifications I have seen - Every possible scenario is analyzed and a solution is provided - very comprehensive and logical making it easy to understand - Kudos to Sandeep for doing a wonderful job
Amazing, please keep posting more.... It's not just for interviews but even for implementation ideas!
I enjoy watching your videos among all youtuber on system design. Please don't stop giving us more content on system design sir. Thanks
That's why it is better to use a notification infra like knock, engagespot, novus instead of building the whole thing from scratch
One of the best system design channels on youtube.. you are very precise with your explanation and your channel is helping me a lot..Content is so deep and I really like the way you put things together..
I saw your website and and your videos. I really really liked it . You're working very hard for teaching system design. I wish you get millions of subscribers in your channel.
I don't have words for appreciation..nice explanation.
Thanks! Good to hear that you liked it!
Do share this with your group :)
Mind Blowing. Very well explained.
Very well explained, the best system design content in youtube
Superb content! I have looked at lot of contents but your content hits the nail hard :)
One suggestion could be that we could have an admin dashboard for the all templates. That means that we can update those templates on the fly and all the clients send that key to us.
Maybe we can also add a leads service to target users with specific notifications. Great job with the amazing explanation!!
I was wondering if we could add a rate limiter with some kind of authentication at the beginning .. it could solve 2 purpose
1) If we are using this system as a SAAS then, in that case, we will certainly provide the end-user with some API key so on the basis of the API key we can authenticate the user and then proceed forward.
2) If the user has already exceeded his notification sending limit then we can stop the notification then and there only so that this kind of messages don't propagate further ahead in the system.
+1. SaaS customer level rate limiting will be better served if done right at the start. User level rate limiting can be done later.
the whole bulk send notification recipient filtering module should NOT be part of the notification system. One of the first priorities of microservice / saas design is to define service boundary. The notification recipient gathering logic should be provided from external services/clients. Also can use some extensive topics (P2 or PN): maintain message order; What if associate is offline / message handlers throw exception; can we guarantee at least once delivery? what if publisher needs only once delivery?
Your content is good but sound quality needs to be improved
Hello @codeKarle, Thank you for really making our life easy when it comes to system design. One request would be , if you can elaborate more on rate limiting and create a system design video on rate limiting.
How would you handle failures? If sending notification fails, how would you retry? If it's not retryable, how would you notify the client that notification sending has failed?
may be we can have callback from notification handler and callback topic in kafka which will notify failure and then our system will retry sending notification
I think this can be very well implemented with the existing solution as well. Just enabling auto commit offset to false, if the notification delivery failed, can help us with re-attempts. To add to it, we could have some monitoring enabled on top of it, which would show how many notifications deliveries have failed, and as such alert the frontend service about the failure. This would work something similar to Hysterix.
Your videos are amazing, thanks for the teachings ! . You should continue adding more ^^
Such a great video - precise and comprehensive
Explained well !
But in Bulk notification scenario, do we want to send the notification to all target users at once? Should we have some kind of rate limiter to not overwhelm our service? Or since it is event driven architecture, it will process all requests eventually?
Outstanding explanation and you seems donned in this system design
Sadly to see you go Karle. Really like your content!
As usual, great lecture! Very informative I've learned a lot from you.
Can you add a lecture on a web crawler design, it will be great (I'm sure). Thank you in advance. Keep the hard and best work!
@codeKarl One important thinng is missed: templates handling. It should be somewhere Notifications Tamplates Repository component which should store a view of concrete type of the messages. Then a content filler for those templates, which should fill chosen template with the data provided by a Client (e.g. User name, amount of order, etc)
Or you suggest receive fully ready to send notification from Client? Then won't it be an overhead? For tempo, we send marketing emails in HTML format with images.
Wow bro, really impressed.
Can you add captions? Between the bad microphone quality and the accent, it makes it difficult to follow
A suggestion:
We can have SNS type of thing in notification handler and all the queues can be subscribed there
Awesome...
Plz make video on payment service... wallet payment like patym/phone pe....
1) How do I implement pause/resume here? Example: In some countries, you have specific time window to send promotional notification after (lets say 10am to 8pm). So my service should be flexible enough to pause notification @8pm and resume @10am next day.
2) How we are maintaining ACK? Also, want to resend to those users who has not received notification as per ack service.
1. Typically, this thing can be achieved by staging the messages before dispatch to handler in some storage (or if Kafka has capacity, then in some specific topic). Basically it can be handled as "kind" of back-pressure. Now there would be some meta information flowing along with message to be delivered that would help identify this restriction and using the same, such messages can be segregated separately. Note that final delivery of messages happen via 3p service, unless complete integration (e.g. for SMS with telecom operators, for emails with SMTP platform and so on), so from this notification engine at max, the dispatch to such 3ps can be controlled. This is typical business rule enforcement.
2. When the SMS or Email is sent, again 3p service provider can send back ACK. Now that may depend on 3p vendor, if it wants to send ACK to some webhook exposed by this platform or by other mechanism. But ACK is quite important and SLA is enforced with 3p vendor so that for high priority messages (OTP e.g.) if ACK is not received within certain interval, the message is retried. This is mostly driven by 3p vendor agreement. In case of email, generally postfix logs from SMTP or equivalent servers are parsed. You can assume webhook as mechanism for ACK with message id embedded and can do the aggregation (from above example, using data from Cassandra) to drive certain intelligence or you can have separate topic in queue/kafka for messages pending ACK for retrial purpose if SLA is violated.
Hope it helps!
U a System Design God
This is a really good design when interviewing for a mid level SDE role, but for a senior engineer , we need to talk in terms of Publisher / Subscriber model.
Can you pls Clarify more?
Thanks for the great video, codeKarle
I have one question:
what would you do if there is a need to deliver let's say to 1 M users an email with 10Mb attachment.
as far as I understood from you design you use bulk service to enrich all users data and based on this generate a bunch of messages to kafka. Does it mean that you want to create a 1M copies and send through Kafka to email-handler?
P.S. it would be nice to use a better microphone. content is great but hard to listen because of sound quality.
There are two scenarios here:
1. If you have the same message( or message template) with same attachment(or attachment template) being sent to millions of users. If that's a use case that we need to handle, then I'll modify this a bit, and keep a message generator component that creates that message, and kafka would just have a message Id. once the email handler processes the message, before sending out, it can query the message generator with the message Id, and fill in the content and send out to the user either directly or via a vendor.
2. If there are different messages being sent out to different users, then i'll stick to the current design, and we'll have to have all the data being put into Kafka. I'll still keep the attachments separate though, in an attachment service.
Also, as far as putting 1M messages in Kafka is concerned, that can be decided based on how frequently this use case comes in. If it's a frequent thing, it's better to have each component process bulk requests in one message. If it's not, the regular flow can be used.
PS: By message template I mean replacing just the user email, name etc and everything else remains the same.
And Thanks for the feedback on Audio. We are trying out a few different microphones and if the channel grows big, we'll be investing in a proper recording studio, but I guess that's far away :)
such a great explanation ❤🔥
Query -> taking an example of social media platform example. so if a celebrity uploading a post. and he has millions of followers, will the notification sending process will be the same. bulk notification service will copy Millions of task in Queue ?
Amazing videos, thank you very much
Since this is proposed as a SaaS notification system, there should be some mechanism for clients to get an Ack on the message delivery, or a means to query the status of the notification.
Why do we need query engine? Why not use a elasticsearch dsl type of library and extract data from elastic search system?
you are awesome bro!
Thanks for this video!
what happens for order transaction notifcation we would want to avoid rate limit in that case right?
Good video . Thanks a lot.
Good Explanation !!!
Thanks! Glad that you liked it 😊
Nice explanation. highly appreciated. can you also add more on how can the receipt/delivery confirmation of a notification be tracked by the asking customer.
Nice Explanation 👌 🙂
Thanks :)
Thanks for the wonderful videos.
Good Explanation
I saw there is Kafka is many parts of the diagram which makes sense, but should there also be Kafka between Client and Notification Service?
Sound Quality is can be better..
"the consumers while picking the requests will first pick the message from a high priority topic, then medium, then low priority messages." In kafka, as far as I know, high priority topics will have it's own consumer. That part was vague for me.
Or you can have dedicated partitions for different priorities
Hi @codeKarle
I think you didn't talk about the case when the notifications failed and hence we need to use db to save it and retry it (using a cron job or any scheduler ) once its done. We can add max retry over there then .
Very good content but sound quality is very poor.
One thing is missed imo. How to handle backbounce? What if we were not able to deliver a message due to reasonable error (e.g. email or phone number are not available), how to notify publishers about it?
Is there a AI/filter type of program to make the voice clearer or understandable? It is pity that this golden content appears to have been recorded using a potato mic. I am hoping there is some automated way to clean up the audio.
how this system will recover/work in case we get an exception in one of the adapters like SMS? how can we re trigger the whole flow?
nice design
Great explanation. Keep up the Good Work.
One Question: How would you have handled prioritization, such that priority notifications are processed first and promotional are processed with some delay?
A reasonable solution to this could be to use different topics for different priorities, and having more consumers on topics that are having high priority messages. Low priority messages can be worked upon by a fewer number of consumers at their own pace.
We could do something more smarter by keeping a running counter of messages of all priorities, and then at runtime spawn more workers for dealing with high priority messages if the traffic surges, but that might be too much over-engineering for a notification system.
@@codeKarle please make more videos
I was just wondering if we should also persist the user transaction data? That is, it's okay we decide that only past 3 days data will be stored for us to send notifications. But typically, how does it happen? Since Kafka won't allow us to persist the data beyond a certain period (depending on the TTL we set though), but do you think we might need a NoSQL/SQL DB for storing even user transaction data?
We'll be storing this data in a NoSQL data store. In this video we have chosen a Mongo as a datastore to store all user transaction info. Reason was to do bulk queries like find all users who bought any mobile phone and send them a notification.
Hi , can you create a video on how to design a platform like Hacker rank/Leet code .. i've been asked this question at one of the interview,
Thanks for the video. Let's say from the bulk notification UI I wanted to send a message to everyone who bought milk in the last one month. Or something along those lines which yields a lot of users maybe hundred thousand. How would we relay this user list to the kafka or the service which acts on these inputs to send notifications
Fantastic use case!!
I think chunking it and then putting it into Kafka would be a decent idea here.
We can say that each message would have let's say 100-200 users, and we can put a few 1000 odd messages into Kafka.
This might cause an issue in case some of the messages error'ed out while putting to kafka or during processing. Retry might be a bit tricky to implement in this case, because we wouldn't want to send the same notification to the same user again.
But I think it's still a good way to implement this use case.
Great explanation. I have question about load balancer. Why have you used at bulk notification service and not at notification service. And also on random question: What happens when we have multiple (backup) load balancers at a point and one of them fails? Do we lose all the requests that were handled by that particular load balancer?
see consistent hashing
3:48 😅got anxios when hld appeared
Great explanation, thank you!
Thanks !!
Interviewer may ask how would you send bulk notifications. If there are millions of user and you want to send notification to all of them then how would you do that.
Can you please make a separate video about it in detail? This video doesn't talk about bulk notification service in details
15:49 you broke Single responsibility. if you want bulk notifications, just send an array of user ids, or TO addresses with the relevant mail merge fields. why should the notification service know how to query ot have connection to the Query engine and internal data. as you said this is usually a generic 3rd party solution.
I dont get why/how you use notification service block (connecting to clients 1,2) to design a notification service. Sounds like recursion. Is that a simple API end point?
How is single point of failure handled in this design in case of any kafka goes down?
Good explanation!.. could u pls explain why Cassandra is chosen ?
Let's say this system needs to scale upto many millions of Notifications per minute kind of a scale, and if we have a requirement to store all the notifications that we are sending out.
Then we need a data store that can scale upto that many number of writes, one for each notification. This becomes a dumping ground for all the notifications in our system. Cassandra works well for that kind of a use case because it can scale horizontally.
You can find a more detailed explanation here: th-cam.com/video/cODCpXtPHbQ/w-d-xo.html
@@codeKarle thank you.
Thanks for video. I have one question here hope you can help me on this.Let say this notification has to be send to more than 1 million/billion people as email notification .Probably it will take lot of time for entire people to receive the email notification. Please help me in this case.
It would take time surely. Two use cases can happen:
(1) Transactional system with massive traffic: I'll throw more hardware to scale it. Since we have millions of transactions, we can afford some extra hardware.
(2) Promotional messages: I would rather stagger it and send the messages during those hours of day when there is lesser traffic.
Normally you'll have a combination of these in any real world system, so for that you can use a hybrid approach.
Does anyone have any ideas how to add schedule push notification system to handle about 1000 notifications/minute?
Can you explain cost estimation of the system where there are a million users and 10s of thousands of notifications pushed per second?
+ १ subscriber, thank you!!!
OTP is authorization flow and not a notification flow .
Please let me know what kind of API it will be using ?
Can you have a video explaining how email sending, SMS, GCM/FCM work?
Sure, We'll try to make that, but it'll take a while.. a bunch of other videos are planned before that.
Query engine will not be able to handle the volume by polling.
Do you have a plan to create a video of food delivery App design?
Why do we need these many Kafkas?
Hidden Gem
What tool did you use to design that Architecture diagram that is on Github?
Most of them were using draw.io
Thanks @@codeKarle
Rate limiter write to Reddis ? Reddis is Single point of failure.
You can use a clustered Redis to avoid Single point of failure, not just here, in a lot of other scenarios as well.
How would you handle message de-duplication?
Why there is no subtitle on this video..
We are in process of adding that. There are subtitles on most other videos now!
The Automatic ones are very incorrect!! But some of the videos now have proper subtitles.
我懂你
half the components here are irrelevant for a SAAS service.
Ticketmaster please
Good design. But in a typical system like this, there is data mining happening, there is ranking logic of notifications. This seems to be one particular flow.
have to focus more on trying to understand wtf he's saying..should of had an interpreter
It's so hard to listen to because of the strong accent.
Awesome 🔥
Plz make video on web crawler @codeKarle
@codekarle we need web crawler
Thanks :)
How would you handle failures? If sending notification fails, how would you retry? If it's not retryable, how would you notify the client that notification sending has failed?
We are using Kafka, so i dont think you meant network failure here, if the end service is not up the message will just stay in the queue until its back up and read again. Now the final delivery is handled by the end services, all the end services have to be fault tolerant and present with a retry mechanism (Services like SMTP for email and messaging are assumed to usually not fail), but just in case if they do i think the source will be present as one of the destinations and through the same notification service we can send them another notification about the failure.