This is to "notify" everyone that this video is the best system design video for notifications I have seen - Every possible scenario is analyzed and a solution is provided - very comprehensive and logical making it easy to understand - Kudos to Sandeep for doing a wonderful job
I saw your website and and your videos. I really really liked it . You're working very hard for teaching system design. I wish you get millions of subscribers in your channel.
One of the best system design channels on youtube.. you are very precise with your explanation and your channel is helping me a lot..Content is so deep and I really like the way you put things together..
One suggestion could be that we could have an admin dashboard for the all templates. That means that we can update those templates on the fly and all the clients send that key to us.
I was wondering if we could add a rate limiter with some kind of authentication at the beginning .. it could solve 2 purpose 1) If we are using this system as a SAAS then, in that case, we will certainly provide the end-user with some API key so on the basis of the API key we can authenticate the user and then proceed forward. 2) If the user has already exceeded his notification sending limit then we can stop the notification then and there only so that this kind of messages don't propagate further ahead in the system.
the whole bulk send notification recipient filtering module should NOT be part of the notification system. One of the first priorities of microservice / saas design is to define service boundary. The notification recipient gathering logic should be provided from external services/clients. Also can use some extensive topics (P2 or PN): maintain message order; What if associate is offline / message handlers throw exception; can we guarantee at least once delivery? what if publisher needs only once delivery?
@codeKarl One important thinng is missed: templates handling. It should be somewhere Notifications Tamplates Repository component which should store a view of concrete type of the messages. Then a content filler for those templates, which should fill chosen template with the data provided by a Client (e.g. User name, amount of order, etc) Or you suggest receive fully ready to send notification from Client? Then won't it be an overhead? For tempo, we send marketing emails in HTML format with images.
Hello @codeKarle, Thank you for really making our life easy when it comes to system design. One request would be , if you can elaborate more on rate limiting and create a system design video on rate limiting.
such a great explanation ❤🔥 Query -> taking an example of social media platform example. so if a celebrity uploading a post. and he has millions of followers, will the notification sending process will be the same. bulk notification service will copy Millions of task in Queue ?
Explained well ! But in Bulk notification scenario, do we want to send the notification to all target users at once? Should we have some kind of rate limiter to not overwhelm our service? Or since it is event driven architecture, it will process all requests eventually?
1) How do I implement pause/resume here? Example: In some countries, you have specific time window to send promotional notification after (lets say 10am to 8pm). So my service should be flexible enough to pause notification @8pm and resume @10am next day. 2) How we are maintaining ACK? Also, want to resend to those users who has not received notification as per ack service.
1. Typically, this thing can be achieved by staging the messages before dispatch to handler in some storage (or if Kafka has capacity, then in some specific topic). Basically it can be handled as "kind" of back-pressure. Now there would be some meta information flowing along with message to be delivered that would help identify this restriction and using the same, such messages can be segregated separately. Note that final delivery of messages happen via 3p service, unless complete integration (e.g. for SMS with telecom operators, for emails with SMTP platform and so on), so from this notification engine at max, the dispatch to such 3ps can be controlled. This is typical business rule enforcement. 2. When the SMS or Email is sent, again 3p service provider can send back ACK. Now that may depend on 3p vendor, if it wants to send ACK to some webhook exposed by this platform or by other mechanism. But ACK is quite important and SLA is enforced with 3p vendor so that for high priority messages (OTP e.g.) if ACK is not received within certain interval, the message is retried. This is mostly driven by 3p vendor agreement. In case of email, generally postfix logs from SMTP or equivalent servers are parsed. You can assume webhook as mechanism for ACK with message id embedded and can do the aggregation (from above example, using data from Cassandra) to drive certain intelligence or you can have separate topic in queue/kafka for messages pending ACK for retrial purpose if SLA is violated. Hope it helps!
Since this is proposed as a SaaS notification system, there should be some mechanism for clients to get an Ack on the message delivery, or a means to query the status of the notification.
Thanks for the great video, codeKarle I have one question: what would you do if there is a need to deliver let's say to 1 M users an email with 10Mb attachment. as far as I understood from you design you use bulk service to enrich all users data and based on this generate a bunch of messages to kafka. Does it mean that you want to create a 1M copies and send through Kafka to email-handler? P.S. it would be nice to use a better microphone. content is great but hard to listen because of sound quality.
There are two scenarios here: 1. If you have the same message( or message template) with same attachment(or attachment template) being sent to millions of users. If that's a use case that we need to handle, then I'll modify this a bit, and keep a message generator component that creates that message, and kafka would just have a message Id. once the email handler processes the message, before sending out, it can query the message generator with the message Id, and fill in the content and send out to the user either directly or via a vendor. 2. If there are different messages being sent out to different users, then i'll stick to the current design, and we'll have to have all the data being put into Kafka. I'll still keep the attachments separate though, in an attachment service. Also, as far as putting 1M messages in Kafka is concerned, that can be decided based on how frequently this use case comes in. If it's a frequent thing, it's better to have each component process bulk requests in one message. If it's not, the regular flow can be used. PS: By message template I mean replacing just the user email, name etc and everything else remains the same. And Thanks for the feedback on Audio. We are trying out a few different microphones and if the channel grows big, we'll be investing in a proper recording studio, but I guess that's far away :)
@@codeKarle for the first as u explained in the video , notification handler and user preferences component /service will be responsible to send multiple copies of template message direct to each targeted user based on the meta data notiication message has right ?
This is a really good design when interviewing for a mid level SDE role, but for a senior engineer , we need to talk in terms of Publisher / Subscriber model.
Hi @codeKarle I think you didn't talk about the case when the notifications failed and hence we need to use db to save it and retry it (using a cron job or any scheduler ) once its done. We can add max retry over there then .
As usual, great lecture! Very informative I've learned a lot from you. Can you add a lecture on a web crawler design, it will be great (I'm sure). Thank you in advance. Keep the hard and best work!
Nice explanation. highly appreciated. can you also add more on how can the receipt/delivery confirmation of a notification be tracked by the asking customer.
Great explanation. I have question about load balancer. Why have you used at bulk notification service and not at notification service. And also on random question: What happens when we have multiple (backup) load balancers at a point and one of them fails? Do we lose all the requests that were handled by that particular load balancer?
How would you handle failures? If sending notification fails, how would you retry? If it's not retryable, how would you notify the client that notification sending has failed?
may be we can have callback from notification handler and callback topic in kafka which will notify failure and then our system will retry sending notification
I think this can be very well implemented with the existing solution as well. Just enabling auto commit offset to false, if the notification delivery failed, can help us with re-attempts. To add to it, we could have some monitoring enabled on top of it, which would show how many notifications deliveries have failed, and as such alert the frontend service about the failure. This would work something similar to Hysterix.
"the consumers while picking the requests will first pick the message from a high priority topic, then medium, then low priority messages." In kafka, as far as I know, high priority topics will have it's own consumer. That part was vague for me.
Let's say this system needs to scale upto many millions of Notifications per minute kind of a scale, and if we have a requirement to store all the notifications that we are sending out. Then we need a data store that can scale upto that many number of writes, one for each notification. This becomes a dumping ground for all the notifications in our system. Cassandra works well for that kind of a use case because it can scale horizontally. You can find a more detailed explanation here: th-cam.com/video/cODCpXtPHbQ/w-d-xo.html
Thanks for the video. Let's say from the bulk notification UI I wanted to send a message to everyone who bought milk in the last one month. Or something along those lines which yields a lot of users maybe hundred thousand. How would we relay this user list to the kafka or the service which acts on these inputs to send notifications
Fantastic use case!! I think chunking it and then putting it into Kafka would be a decent idea here. We can say that each message would have let's say 100-200 users, and we can put a few 1000 odd messages into Kafka. This might cause an issue in case some of the messages error'ed out while putting to kafka or during processing. Retry might be a bit tricky to implement in this case, because we wouldn't want to send the same notification to the same user again. But I think it's still a good way to implement this use case.
Is there a AI/filter type of program to make the voice clearer or understandable? It is pity that this golden content appears to have been recorded using a potato mic. I am hoping there is some automated way to clean up the audio.
Great explanation. Keep up the Good Work. One Question: How would you have handled prioritization, such that priority notifications are processed first and promotional are processed with some delay?
A reasonable solution to this could be to use different topics for different priorities, and having more consumers on topics that are having high priority messages. Low priority messages can be worked upon by a fewer number of consumers at their own pace. We could do something more smarter by keeping a running counter of messages of all priorities, and then at runtime spawn more workers for dealing with high priority messages if the traffic surges, but that might be too much over-engineering for a notification system.
I was just wondering if we should also persist the user transaction data? That is, it's okay we decide that only past 3 days data will be stored for us to send notifications. But typically, how does it happen? Since Kafka won't allow us to persist the data beyond a certain period (depending on the TTL we set though), but do you think we might need a NoSQL/SQL DB for storing even user transaction data?
We'll be storing this data in a NoSQL data store. In this video we have chosen a Mongo as a datastore to store all user transaction info. Reason was to do bulk queries like find all users who bought any mobile phone and send them a notification.
One thing is missed imo. How to handle backbounce? What if we were not able to deliver a message due to reasonable error (e.g. email or phone number are not available), how to notify publishers about it?
I think the proposed solution assumes that there will be load balancer in front of multiple notifucation service clusters each just forwarding/publishing the message to kafka queue . The rety logic must be kept at client side itself . Another solution as u have proposed where we directly send the message to kafka itself , then i dont think we need notification service as metioned in the video because as far as i understand the only role of notification service is to send/push the message into kafka. Correct me if im wrong
Interviewer may ask how would you send bulk notifications. If there are millions of user and you want to send notification to all of them then how would you do that. Can you please make a separate video about it in detail? This video doesn't talk about bulk notification service in details
Thanks for video. I have one question here hope you can help me on this.Let say this notification has to be send to more than 1 million/billion people as email notification .Probably it will take lot of time for entire people to receive the email notification. Please help me in this case.
It would take time surely. Two use cases can happen: (1) Transactional system with massive traffic: I'll throw more hardware to scale it. Since we have millions of transactions, we can afford some extra hardware. (2) Promotional messages: I would rather stagger it and send the messages during those hours of day when there is lesser traffic. Normally you'll have a combination of these in any real world system, so for that you can use a hybrid approach.
I dont get why/how you use notification service block (connecting to clients 1,2) to design a notification service. Sounds like recursion. Is that a simple API end point?
15:49 you broke Single responsibility. if you want bulk notifications, just send an array of user ids, or TO addresses with the relevant mail merge fields. why should the notification service know how to query ot have connection to the Query engine and internal data. as you said this is usually a generic 3rd party solution.
We are in process of adding that. There are subtitles on most other videos now! The Automatic ones are very incorrect!! But some of the videos now have proper subtitles.
Good design. But in a typical system like this, there is data mining happening, there is ranking logic of notifications. This seems to be one particular flow.
![ลองสุ่มไอโฟน 990 โกงมั้ย? [ โกงมั้ยครับ ep.101 ] | DOM](http://i.ytimg.com/vi/bfsWe0j492k/mqdefault.jpg)
Just came across your channel. One word "FABULOUS". Keep up the good work.
Future suggestion:
1. Web crawler
2. Yelp
3. Api limiter and BOT blocker
Thanks for the kind words. Do share it in your circle.
Crawler is coming soon. Rest I'll try to do when there is some free time from work :)
@@codeKarle Api limiter is also a very common thing being asked and would love to see your extensive design perspective on the same :)
Could please share video link for these topics
only TH-camr who talks about the components E2E with so much clarity, really appreciate it.
why have you stopped making videos ? your content is of very high quality. so much condense & clear information. you're a great teacher.
I agree. No unnecessary noise and bulshit only hit on bull's eye. Great content
Yes. He is the best. But not sure why he has stopped and also not able to reach him. Absolutely awesome explanation with crystal clear clarity.
@@Nexatek He is working at Facebook . He might be busy. But , he really best
Super and correct - I myself was part of Notifications Team in a big product base company. And the architecture was same.
This question was asked me in interview - after watching this video i realise how dum response i gave. - awesome informative video sir
This is to "notify" everyone that this video is the best system design video for notifications I have seen - Every possible scenario is analyzed and a solution is provided - very comprehensive and logical making it easy to understand - Kudos to Sandeep for doing a wonderful job
I enjoy watching your videos among all youtuber on system design. Please don't stop giving us more content on system design sir. Thanks
Amazing, please keep posting more.... It's not just for interviews but even for implementation ideas!
I saw your website and and your videos. I really really liked it . You're working very hard for teaching system design. I wish you get millions of subscribers in your channel.
One of the best system design channels on youtube.. you are very precise with your explanation and your channel is helping me a lot..Content is so deep and I really like the way you put things together..
Always a great content on this channel but this time I felt that the system has been complicated this time.
One suggestion could be that we could have an admin dashboard for the all templates. That means that we can update those templates on the fly and all the clients send that key to us.
Very well explained, the best system design content in youtube
Superb content! I have looked at lot of contents but your content hits the nail hard :)
I was wondering if we could add a rate limiter with some kind of authentication at the beginning .. it could solve 2 purpose
1) If we are using this system as a SAAS then, in that case, we will certainly provide the end-user with some API key so on the basis of the API key we can authenticate the user and then proceed forward.
2) If the user has already exceeded his notification sending limit then we can stop the notification then and there only so that this kind of messages don't propagate further ahead in the system.
+1. SaaS customer level rate limiting will be better served if done right at the start. User level rate limiting can be done later.
I don't have words for appreciation..nice explanation.
Thanks! Good to hear that you liked it!
Do share this with your group :)
Mind Blowing. Very well explained.
the whole bulk send notification recipient filtering module should NOT be part of the notification system. One of the first priorities of microservice / saas design is to define service boundary. The notification recipient gathering logic should be provided from external services/clients. Also can use some extensive topics (P2 or PN): maintain message order; What if associate is offline / message handlers throw exception; can we guarantee at least once delivery? what if publisher needs only once delivery?
That's why it is better to use a notification infra like knock, engagespot, novus instead of building the whole thing from scratch
Maybe we can also add a leads service to target users with specific notifications. Great job with the amazing explanation!!
@codeKarl One important thinng is missed: templates handling. It should be somewhere Notifications Tamplates Repository component which should store a view of concrete type of the messages. Then a content filler for those templates, which should fill chosen template with the data provided by a Client (e.g. User name, amount of order, etc)
Or you suggest receive fully ready to send notification from Client? Then won't it be an overhead? For tempo, we send marketing emails in HTML format with images.
Thanks for the efforts, it would also help if you could explain/justify the Database chosen for each usecase in your designs
Hello @codeKarle, Thank you for really making our life easy when it comes to system design. One request would be , if you can elaborate more on rate limiting and create a system design video on rate limiting.
such a great explanation ❤🔥
Query -> taking an example of social media platform example. so if a celebrity uploading a post. and he has millions of followers, will the notification sending process will be the same. bulk notification service will copy Millions of task in Queue ?
Such a great video - precise and comprehensive
Explained well !
But in Bulk notification scenario, do we want to send the notification to all target users at once? Should we have some kind of rate limiter to not overwhelm our service? Or since it is event driven architecture, it will process all requests eventually?
A suggestion:
We can have SNS type of thing in notification handler and all the queues can be subscribed there
Your videos are amazing, thanks for the teachings ! . You should continue adding more ^^
Your content is good but sound quality needs to be improved
Sadly to see you go Karle. Really like your content!
Outstanding explanation and you seems donned in this system design
Wow bro, really impressed.
1) How do I implement pause/resume here? Example: In some countries, you have specific time window to send promotional notification after (lets say 10am to 8pm). So my service should be flexible enough to pause notification @8pm and resume @10am next day.
2) How we are maintaining ACK? Also, want to resend to those users who has not received notification as per ack service.
1. Typically, this thing can be achieved by staging the messages before dispatch to handler in some storage (or if Kafka has capacity, then in some specific topic). Basically it can be handled as "kind" of back-pressure. Now there would be some meta information flowing along with message to be delivered that would help identify this restriction and using the same, such messages can be segregated separately. Note that final delivery of messages happen via 3p service, unless complete integration (e.g. for SMS with telecom operators, for emails with SMTP platform and so on), so from this notification engine at max, the dispatch to such 3ps can be controlled. This is typical business rule enforcement.
2. When the SMS or Email is sent, again 3p service provider can send back ACK. Now that may depend on 3p vendor, if it wants to send ACK to some webhook exposed by this platform or by other mechanism. But ACK is quite important and SLA is enforced with 3p vendor so that for high priority messages (OTP e.g.) if ACK is not received within certain interval, the message is retried. This is mostly driven by 3p vendor agreement. In case of email, generally postfix logs from SMTP or equivalent servers are parsed. You can assume webhook as mechanism for ACK with message id embedded and can do the aggregation (from above example, using data from Cassandra) to drive certain intelligence or you can have separate topic in queue/kafka for messages pending ACK for retrial purpose if SLA is violated.
Hope it helps!
Awesome...
Plz make video on payment service... wallet payment like patym/phone pe....
Amazing videos, thank you very much
Since this is proposed as a SaaS notification system, there should be some mechanism for clients to get an Ack on the message delivery, or a means to query the status of the notification.
Thanks for this video!
Thanks for the great video, codeKarle
I have one question:
what would you do if there is a need to deliver let's say to 1 M users an email with 10Mb attachment.
as far as I understood from you design you use bulk service to enrich all users data and based on this generate a bunch of messages to kafka. Does it mean that you want to create a 1M copies and send through Kafka to email-handler?
P.S. it would be nice to use a better microphone. content is great but hard to listen because of sound quality.
There are two scenarios here:
1. If you have the same message( or message template) with same attachment(or attachment template) being sent to millions of users. If that's a use case that we need to handle, then I'll modify this a bit, and keep a message generator component that creates that message, and kafka would just have a message Id. once the email handler processes the message, before sending out, it can query the message generator with the message Id, and fill in the content and send out to the user either directly or via a vendor.
2. If there are different messages being sent out to different users, then i'll stick to the current design, and we'll have to have all the data being put into Kafka. I'll still keep the attachments separate though, in an attachment service.
Also, as far as putting 1M messages in Kafka is concerned, that can be decided based on how frequently this use case comes in. If it's a frequent thing, it's better to have each component process bulk requests in one message. If it's not, the regular flow can be used.
PS: By message template I mean replacing just the user email, name etc and everything else remains the same.
And Thanks for the feedback on Audio. We are trying out a few different microphones and if the channel grows big, we'll be investing in a proper recording studio, but I guess that's far away :)
@@codeKarle for the first as u explained in the video , notification handler and user preferences component /service will be responsible to send multiple copies of template message direct to each targeted user based on the meta data notiication message has right ?
This is a really good design when interviewing for a mid level SDE role, but for a senior engineer , we need to talk in terms of Publisher / Subscriber model.
Can you pls Clarify more?
Good Explanation
U a System Design God
you are awesome bro!
Hi @codeKarle
I think you didn't talk about the case when the notifications failed and hence we need to use db to save it and retry it (using a cron job or any scheduler ) once its done. We can add max retry over there then .
Thanks for the wonderful videos.
Nice Explanation 👌 🙂
Thanks :)
As usual, great lecture! Very informative I've learned a lot from you.
Can you add a lecture on a web crawler design, it will be great (I'm sure). Thank you in advance. Keep the hard and best work!
Nice explanation. highly appreciated. can you also add more on how can the receipt/delivery confirmation of a notification be tracked by the asking customer.
Good video . Thanks a lot.
Good Explanation !!!
Thanks! Glad that you liked it 😊
Great explanation. I have question about load balancer. Why have you used at bulk notification service and not at notification service. And also on random question: What happens when we have multiple (backup) load balancers at a point and one of them fails? Do we lose all the requests that were handled by that particular load balancer?
see consistent hashing
How would you handle failures? If sending notification fails, how would you retry? If it's not retryable, how would you notify the client that notification sending has failed?
may be we can have callback from notification handler and callback topic in kafka which will notify failure and then our system will retry sending notification
I think this can be very well implemented with the existing solution as well. Just enabling auto commit offset to false, if the notification delivery failed, can help us with re-attempts. To add to it, we could have some monitoring enabled on top of it, which would show how many notifications deliveries have failed, and as such alert the frontend service about the failure. This would work something similar to Hysterix.
Hi , can you create a video on how to design a platform like Hacker rank/Leet code .. i've been asked this question at one of the interview,
"the consumers while picking the requests will first pick the message from a high priority topic, then medium, then low priority messages." In kafka, as far as I know, high priority topics will have it's own consumer. That part was vague for me.
Or you can have dedicated partitions for different priorities
Good explanation!.. could u pls explain why Cassandra is chosen ?
Let's say this system needs to scale upto many millions of Notifications per minute kind of a scale, and if we have a requirement to store all the notifications that we are sending out.
Then we need a data store that can scale upto that many number of writes, one for each notification. This becomes a dumping ground for all the notifications in our system. Cassandra works well for that kind of a use case because it can scale horizontally.
You can find a more detailed explanation here: th-cam.com/video/cODCpXtPHbQ/w-d-xo.html
@@codeKarle thank you.
Thanks for the video. Let's say from the bulk notification UI I wanted to send a message to everyone who bought milk in the last one month. Or something along those lines which yields a lot of users maybe hundred thousand. How would we relay this user list to the kafka or the service which acts on these inputs to send notifications
Fantastic use case!!
I think chunking it and then putting it into Kafka would be a decent idea here.
We can say that each message would have let's say 100-200 users, and we can put a few 1000 odd messages into Kafka.
This might cause an issue in case some of the messages error'ed out while putting to kafka or during processing. Retry might be a bit tricky to implement in this case, because we wouldn't want to send the same notification to the same user again.
But I think it's still a good way to implement this use case.
nice design
@codekarle you are the best sir ❤️
Is there a AI/filter type of program to make the voice clearer or understandable? It is pity that this golden content appears to have been recorded using a potato mic. I am hoping there is some automated way to clean up the audio.
Great explanation, thank you!
Great explanation. Keep up the Good Work.
One Question: How would you have handled prioritization, such that priority notifications are processed first and promotional are processed with some delay?
A reasonable solution to this could be to use different topics for different priorities, and having more consumers on topics that are having high priority messages. Low priority messages can be worked upon by a fewer number of consumers at their own pace.
We could do something more smarter by keeping a running counter of messages of all priorities, and then at runtime spawn more workers for dealing with high priority messages if the traffic surges, but that might be too much over-engineering for a notification system.
@@codeKarle please make more videos
what happens for order transaction notifcation we would want to avoid rate limit in that case right?
Very good content but sound quality is very poor.
Why do we need query engine? Why not use a elasticsearch dsl type of library and extract data from elastic search system?
what about failover mechanism? if a high priority message gets failed or undelivered how do we add retry?
I was just wondering if we should also persist the user transaction data? That is, it's okay we decide that only past 3 days data will be stored for us to send notifications. But typically, how does it happen? Since Kafka won't allow us to persist the data beyond a certain period (depending on the TTL we set though), but do you think we might need a NoSQL/SQL DB for storing even user transaction data?
We'll be storing this data in a NoSQL data store. In this video we have chosen a Mongo as a datastore to store all user transaction info. Reason was to do bulk queries like find all users who bought any mobile phone and send them a notification.
Thanks !!
Thanks :)
+ १ subscriber, thank you!!!
One thing is missed imo. How to handle backbounce? What if we were not able to deliver a message due to reasonable error (e.g. email or phone number are not available), how to notify publishers about it?
Sound Quality is can be better..
Please design a system for Stock Notification alert when value is matching
I saw there is Kafka is many parts of the diagram which makes sense, but should there also be Kafka between Client and Notification Service?
I think the proposed solution assumes that there will be load balancer in front of multiple notifucation service clusters each just forwarding/publishing the message to kafka queue . The rety logic must be kept at client side itself . Another solution as u have proposed where we directly send the message to kafka itself , then i dont think we need notification service as metioned in the video because as far as i understand the only role of notification service is to send/push the message into kafka. Correct me if im wrong
Do you have a plan to create a video of food delivery App design?
How is single point of failure handled in this design in case of any kafka goes down?
Please let me know what kind of API it will be using ?
how this system will recover/work in case we get an exception in one of the adapters like SMS? how can we re trigger the whole flow?
Interviewer may ask how would you send bulk notifications. If there are millions of user and you want to send notification to all of them then how would you do that.
Can you please make a separate video about it in detail? This video doesn't talk about bulk notification service in details
Thanks for video. I have one question here hope you can help me on this.Let say this notification has to be send to more than 1 million/billion people as email notification .Probably it will take lot of time for entire people to receive the email notification. Please help me in this case.
It would take time surely. Two use cases can happen:
(1) Transactional system with massive traffic: I'll throw more hardware to scale it. Since we have millions of transactions, we can afford some extra hardware.
(2) Promotional messages: I would rather stagger it and send the messages during those hours of day when there is lesser traffic.
Normally you'll have a combination of these in any real world system, so for that you can use a hybrid approach.
Can you explain cost estimation of the system where there are a million users and 10s of thousands of notifications pushed per second?
Can you have a video explaining how email sending, SMS, GCM/FCM work?
Sure, We'll try to make that, but it'll take a while.. a bunch of other videos are planned before that.
I dont get why/how you use notification service block (connecting to clients 1,2) to design a notification service. Sounds like recursion. Is that a simple API end point?
Hidden Gem
3:48 😅got anxios when hld appeared
15:49 you broke Single responsibility. if you want bulk notifications, just send an array of user ids, or TO addresses with the relevant mail merge fields. why should the notification service know how to query ot have connection to the Query engine and internal data. as you said this is usually a generic 3rd party solution.
OTP is authorization flow and not a notification flow .
Can you add captions? Between the bad microphone quality and the accent, it makes it difficult to follow
What tool did you use to design that Architecture diagram that is on Github?
Most of them were using draw.io
Thanks @@codeKarle
Query engine will not be able to handle the volume by polling.
Does anyone have any ideas how to add schedule push notification system to handle about 1000 notifications/minute?
Why do we need these many Kafkas?
Rate limiter write to Reddis ? Reddis is Single point of failure.
You can use a clustered Redis to avoid Single point of failure, not just here, in a lot of other scenarios as well.
Why there is no subtitle on this video..
We are in process of adding that. There are subtitles on most other videos now!
The Automatic ones are very incorrect!! But some of the videos now have proper subtitles.
我懂你
How would you handle message de-duplication?
Ticketmaster please
half the components here are irrelevant for a SAAS service.
Good design. But in a typical system like this, there is data mining happening, there is ranking logic of notifications. This seems to be one particular flow.
have to focus more on trying to understand wtf he's saying..should of had an interpreter
It's so hard to listen to because of the strong accent.
Awesome 🔥
Plz make video on web crawler @codeKarle
@codekarle we need web crawler