Great video! Had a question: - Doesn't socket master now become a choking point with scale up? The sheer amount of work-to-be-done will increase with every new user subscription. - In case we add more instances of socket masters how do multiple socket masters decide which subscriptions to work upon and how is the list rebalanced in case of a scale-up/down? I had worked upon a system with a similar problem where we solved it via Leader Election among all nodes and consequent subscription assignment by the Leader node. Curious as to how Atlassian solved this. NOTE: Haven't read the actual blog yet where this would already be answered but still posting for the curious minds.
The socket master can be turned into a cluster and they all need to have the same copy of mappings. Adding a load balancer in front would solve this since there doesn't need to be a leader in socket master as they just do routing according to mapping
The Websocket Server resides inside the RabbitMQ Broker(Exchange + Queue + Websocker Server). Since RabbitMQ Broker's Queue and Consumer are connected via websockets, so when Queue recieve messages they are pulled by Websocker Server and pushed to the consumers which are listening to it. Hope it cleared your doubt.
@@AsliEngineering Can you share an official source where it says RMQ also supports pull mechanism? Afaik this is the major distinction between Kafka and RMQ that RMQ is push based and Kafka is pull based.
It is simple Pull stuff with no fancy logic. Events pushed, server pulled, filtered, and related to WS server which then uses the mapping of user to WS Connection related further to end user.
Hi Arpit, nice content as always. It was also possible that the socket clients could directly become Kafka consumers right? What could be the reason for a socket master in between Kafka and clients?
To reduce direct coupling between the event transport system (Kafka) and the client connection system (web socket). This way you'll be able to swap Kafka with any other message broker and the rest of the system don't need to be touched. Plus you can add more capabilities to the socket master for observability and auditing.
Maybe, but I guess it can also be tackled with how you would handle the load balancer's single point of failure, maybe by having multiple socket master nodes available, and keeping one as a master by election method.
I didn't get it, now the socket master became a single point of failure, what if the messages were read and error occurred while pushing to the ws server? How do they handle duplicate reads (if the socket master also cluster)?
The WS servers are already bound according to the user's topics. So if you are connected, your client will connect to a WS server and that will in-turn connect to all applicable queues. If I'm connected to a different server then your server won't be subscribed to my events
A separate machine that accepts the updates, filters, and relays them to socket servers over TCP (may be an HTTP API, they have not mentioned the details). Consider it as a filtering and relaying machine.
Hey Arpit, I wanted to share some constructive feedback regarding the audio quality of your content. I believe you have a great microphone (it seems like a Yeti), but there appears to be an issue with the audio quality. To me, it sounds like there is an echo. You might want to consider putting some sound-absorbing mesh on the walls. Also there are many AI tools available these days that can tune the audio and even remove construction noise. Your content is excellent, and as a long-time subscriber, I feel it's my duty to let you know about this issue. After coming back to the channel after a few months, I really noticed the audio quality problem. If you can fix this, I believe it will significantly boost the overall quality of the content.
Thanks a ton for sharing. Feedback taken. This time it actually sounded really bad. May be because I was recording very late at night. I will get some sound absorbing mesh. Thanks for the suggestion.
I checked a few websites and found that my mic settings were not ideal and the mic positioning was not ideal. I have changed it and the audio will be better the next time. I have recorded one more video, so the next one will also be poor, but subsequent ones will be much better. Thanks for the nudge and constructive feedback. It means a ton.
Great video! Had a question:
- Doesn't socket master now become a choking point with scale up? The sheer amount of work-to-be-done will increase with every new user subscription.
- In case we add more instances of socket masters how do multiple socket masters decide which subscriptions to work upon and how is the list rebalanced in case of a scale-up/down?
I had worked upon a system with a similar problem where we solved it via Leader Election among all nodes and consequent subscription assignment by the Leader node. Curious as to how Atlassian solved this.
NOTE: Haven't read the actual blog yet where this would already be answered but still posting for the curious minds.
The socket master can be turned into a cluster and they all need to have the same copy of mappings. Adding a load balancer in front would solve this since there doesn't need to be a leader in socket master as they just do routing according to mapping
Why can't we do the same with RabbitMQ ? Push message -> read from socket master -> route to client.
They were unhappy with Rabbit. Unstable for them, split brain issues, lack of high availability. They wanted to move out.
@@AsliEngineering Thank you
At around 02:50 you mention websockets pull events from RabbitMQ but isn't RabbitMQ works on push model?
The Websocket Server resides inside the RabbitMQ Broker(Exchange + Queue + Websocker Server). Since RabbitMQ Broker's Queue and Consumer are connected via websockets, so when Queue recieve messages they are pulled by Websocker Server and pushed to the consumers which are listening to it. Hope it cleared your doubt.
Both are possible. As per the blog, they pulled the data, but you can go push all throughout.
@@AsliEngineering Can you share an official source where it says RMQ also supports pull mechanism? Afaik this is the major distinction between Kafka and RMQ that RMQ is push based and Kafka is pull based.
Lot of details are skipped for new Kafka architecture
It is simple Pull stuff with no fancy logic. Events pushed, server pulled, filtered, and related to WS server which then uses the mapping of user to WS Connection related further to end user.
Hi Arpit, nice content as always.
It was also possible that the socket clients could directly become Kafka consumers right? What could be the reason for a socket master in between Kafka and clients?
To reduce direct coupling between the event transport system (Kafka) and the client connection system (web socket). This way you'll be able to swap Kafka with any other message broker and the rest of the system don't need to be touched. Plus you can add more capabilities to the socket master for observability and auditing.
Can the socket master be a single point of failure ?
Maybe, but I guess it can also be tackled with how you would handle the load balancer's single point of failure, maybe by having multiple socket master nodes available, and keeping one as a master by election method.
I didn't get it, now the socket master became a single point of failure, what if the messages were read and error occurred while pushing to the ws server? How do they handle duplicate reads (if the socket master also cluster)?
ofc, if it needs to be scaled beyond a single machine, consider sharding it and each shard handles its own subset of tenants.
Why is websocket server pulling all updates instead of only the updates that the connected user is subscribed to?
9:16
The WS servers are already bound according to the user's topics. So if you are connected, your client will connect to a WS server and that will in-turn connect to all applicable queues. If I'm connected to a different server then your server won't be subscribed to my events
Both are possible. As per the blog, they pulled the data, but you can go push all throughout.
Hello sir,
If possible, please do one video on docker architecture.
What is the socket master written in or running on?
A separate machine that accepts the updates, filters, and relays them to socket servers over TCP (may be an HTTP API, they have not mentioned the details).
Consider it as a filtering and relaying machine.
Very informative. What tool are you using to write your notes?
GoodNotes.
Instead of resolving the message with socket master cannot we create partition in kafka itself? What is primary use of the socket master then in case?
Filtering and relaying. This business logic is with Socket master.
Hey Arpit,
I wanted to share some constructive feedback regarding the audio quality of your content. I believe you have a great microphone (it seems like a Yeti), but there appears to be an issue with the audio quality.
To me, it sounds like there is an echo. You might want to consider putting some sound-absorbing mesh on the walls. Also there are many AI tools available these days that can tune the audio and even remove construction noise.
Your content is excellent, and as a long-time subscriber, I feel it's my duty to let you know about this issue. After coming back to the channel after a few months, I really noticed the audio quality problem. If you can fix this, I believe it will significantly boost the overall quality of the content.
Thanks a ton for sharing. Feedback taken. This time it actually sounded really bad. May be because I was recording very late at night.
I will get some sound absorbing mesh. Thanks for the suggestion.
I checked a few websites and found that my mic settings were not ideal and the mic positioning was not ideal.
I have changed it and the audio will be better the next time. I have recorded one more video, so the next one will also be poor, but subsequent ones will be much better.
Thanks for the nudge and constructive feedback. It means a ton.
👍
now socket master also becoming a choking point for the entire setup like rmq inbound cluster
Consider sharding it.
CPU 2x ?
Why Cost not gone up ?
lesser. It reduced, and not increased.
Some big tech company migrated some piece of software from X to Y. Hurry! Everybody migrate everything to Y now!
Precisely what should not be done. If it works for you, it works for you.