Websockets are horizontally scalable, you use a message bus where the message is sent to each websocket host also known as a backplane, and you will need to setup sticky sessions with the load balancer. The reason for this is that you don't know what server the client is on, so you send it to all of them. You can also chain websockets together e.g. a websocket server is a client of another websocket server, it's the same concept as the message bus. Also you can downscale due to redistrobution of connection via the load balancer.
Yes, I was also thinking same way. Why can't we use Kafka here...and assign a key to each client so that we will know to which server it can map to...that way we don't have to send it to every server
@jwbonnett suggestion is the correct approach, leverage message brokers and pub/sub for horizontally scaled instances of a server to publish messages for changes it makes which get broadcast by the message broker to the instances subscribing to that event. Those instances will then send those updates its client connections. This is how all clients know about changes by other clients in real-time. Load balancer tracks the client sessions using sticky sessions feature which uses least connection algorithm to load balance the connections to the replicated server instances. This video covers the topic well: th-cam.com/video/vXJsJ52vwAA/w-d-xo.html
I want to add 2 points to your video. The rise of anycast based network architectures and edge/serverless edge based architectures make us think differently about websockets and scaling them. I've seen some interesting deployments of self hosted pbx, voip and even office tools like collabra over anycast architectures and they work quite seamlessly. There definitely will be issues/drops during scaling activity but a quick reconnection usually solves the problem. Edge/edge serverless(like cloudflare workers) also have the same ideology as aws api gateway but the added advantage is the session is on the edge. This becomes a game changer for deployments in the above mentioned scenarios. That said, it's not perfect. It has its problems. But the pros outweigh the cons by a big margin for about 70% of the workloads that I deal with.
@@codedamn at a high level, anycast is a faster backbone to your origin server. In aws you can do it with global accelerator. You can setup global accelerator + ec2 or nlb as origin with multiregion deployment. In Google cloud, you can use Google load balancer and that also does the same thing. Throw in a random websocket server and play around with multi region with peak load. There is a noticeable amount of difference. Btw, you have some great content in your channel. Hats off for your effort.
We scaled our socket solution with a redis middleware that persists the session id. And it worked great. Very surprised to see no mention of this approach.
This video doesn't show how to scale persistent connections. Who uses websockets with AWS lambda functions? We use websockets to stream data between server and client and lambda functions are stateless and cannot remain up indefinitely. Most applications want to maintain a connection for days, weeks or indefinitely so how does this architecture fix anything in those situations. The gateway can allow you to load balance those persistent connections but that's it.
Hey there great clip. I think another way of doing this (not as scalable as this) would be to have an nginx as a reverse proxi in front of multiple node servers that would distribute the sokets among multiple node servers. This solution would unfortunately require you to change your code quite a bit since you wouldn't be able to guarantee that a socket is connected to the server that got a request, so the app instances would need to negotiate among themselves using a message system (maybe redis) and ask the node instance that has a connection with that particular client to send a message.
How can maintain the business logics and all sync with different servers. I have already used the redis pub/sub but i can't sync a local objects are sync.
Yesterday I faced cores issue. TH-cam recommended your video. Not searched in youtube.that time I am not watching tech videos , but youtube recommended. Today websocket also same happened. Currently I am building real time chat application. Thanks for your video
Thanks for the nice video! One question, how API gateway knows to which client the message needs to be delivered? Is there an internal queue per client automatically created by API gateway?
Just a helpful info. AWS Api Gateway needs to respond within 30 sec. If the backend behind the gateway takes more than 30sec then the API Gateway will directly respond with TIMEOUT json.
Thank you so much for this, your videos are really awesome with great video quality and content too. With help of your videos now I am able to build web projects. Thanks a lot!!
I always used to think if websockets are that of much fast, why use http, but now understood they are not reliable and scalable so http is better, thanks codedamn. Can you please tell us which language is better to us websockets on, php node or python....
Your are giving a misleading information (not in purpose, I know) but WS are as scalable as TCP connections are. You should say that node.js is not scalable, instead.
Hey! Congrats on Winning the codedamn T Shirt Giveaway. Please reply to this comment by mentioning your email for further communication. Or just go on your channel page and set an email address so that we can contact you that way. Or just reply here without using @ gmail . com.
Websockets are hard to scale but not impossible to scale. Using a service as recommended here doesn't really solve it, but rather just passes the problem on to AWS. That might or might not be acceptable depending on your business needs.
Websockets are pretty scalable, just for some solutions using them is not best for scaling and some is best for scaling. The only thing you have worry about is tcp sockets.
Websockets are scalable. Just use multiple of them. And keep a database for which user group is using which socket. That’s how I solved it in my application.
Websockets maintain state, so if you remove the statefulness of the connection in API gateway how would you handle a case when server wants to send a push request to clients. How does the lambda know which ws connection to try? The http connection is stateless
I recently implemented a tic-tak-toe game using websockets with room functionality and I didn't know websockets are hard to scale well thanks for this information, your content is great 👍
I have a friend who has a socket based business that grew 100x faster than they expected. His life was hell scaling that software lol I also had a socket based business that I just recently sold. Only had a few thousand users but i never had a single issue with the sockets.
You talk a lot about connection limits but I would have thought the main limiting factor would be how much traffic is going back and forth in the open connection.
Learn WebSockets and other Full Stack Technologies on codedamn: cdm.sh/fullstack
Hey Mehul, can you please show us how to implement this by a project maybe a simple chat app or something.. mostly we all use sockets for chat app ig
Websockets are horizontally scalable, you use a message bus where the message is sent to each websocket host also known as a backplane, and you will need to setup sticky sessions with the load balancer. The reason for this is that you don't know what server the client is on, so you send it to all of them. You can also chain websockets together e.g. a websocket server is a client of another websocket server, it's the same concept as the message bus. Also you can downscale due to redistrobution of connection via the load balancer.
This is the right answer. The solution in this video isn't a good one
I am really confused by this one. Is there any resource from which I can actually read this up?
Yes, I was also thinking same way. Why can't we use Kafka here...and assign a key to each client so that we will know to which server it can map to...that way we don't have to send it to every server
@jwbonnett suggestion is the correct approach, leverage message brokers and pub/sub for horizontally scaled instances of a server to publish messages for changes it makes which get broadcast by the message broker to the instances subscribing to that event. Those instances will then send those updates its client connections. This is how all clients know about changes by other clients in real-time. Load balancer tracks the client sessions using sticky sessions feature which uses least connection algorithm to load balance the connections to the replicated server instances.
This video covers the topic well: th-cam.com/video/vXJsJ52vwAA/w-d-xo.html
I want to add 2 points to your video. The rise of anycast based network architectures and edge/serverless edge based architectures make us think differently about websockets and scaling them.
I've seen some interesting deployments of self hosted pbx, voip and even office tools like collabra over anycast architectures and they work quite seamlessly. There definitely will be issues/drops during scaling activity but a quick reconnection usually solves the problem.
Edge/edge serverless(like cloudflare workers) also have the same ideology as aws api gateway but the added advantage is the session is on the edge. This becomes a game changer for deployments in the above mentioned scenarios.
That said, it's not perfect. It has its problems. But the pros outweigh the cons by a big margin for about 70% of the workloads that I deal with.
I don’t know a lot about anycast networks but this sounds very interesting. Thank you for sharing. Do you have any reading materials on this?
@@codedamn at a high level, anycast is a faster backbone to your origin server.
In aws you can do it with global accelerator. You can setup global accelerator + ec2 or nlb as origin with multiregion deployment.
In Google cloud, you can use Google load balancer and that also does the same thing.
Throw in a random websocket server and play around with multi region with peak load. There is a noticeable amount of difference.
Btw, you have some great content in your channel. Hats off for your effort.
We scaled our socket solution with a redis middleware that persists the session id. And it worked great. Very surprised to see no mention of this approach.
This video doesn't show how to scale persistent connections. Who uses websockets with AWS lambda functions? We use websockets to stream data between server and client and lambda functions are stateless and cannot remain up indefinitely.
Most applications want to maintain a connection for days, weeks or indefinitely so how does this architecture fix anything in those situations. The gateway can allow you to load balance those persistent connections but that's it.
Hey there great clip. I think another way of doing this (not as scalable as this) would be to have an nginx as a reverse proxi in front of multiple node servers that would distribute the sokets among multiple node servers. This solution would unfortunately require you to change your code quite a bit since you wouldn't be able to guarantee that a socket is connected to the server that got a request, so the app instances would need to negotiate among themselves using a message system (maybe redis) and ask the node instance that has a connection with that particular client to send a message.
Redis, bingo.
in the aws solution you mentioned, how would you initiate and send a message/data from server (lambdas in your case) to the client?
I was watching a WhatsApp system design video and was confused about the gateway. Luckily I found your video. Very well explained. 😊
How can maintain the business logics and all sync with different servers.
I have already used the redis pub/sub but i can't sync a local objects are sync.
Yesterday I faced cores issue. TH-cam recommended your video. Not searched in youtube.that time I am not watching tech videos , but youtube recommended. Today websocket also same happened. Currently I am building real time chat application. Thanks for your video
Thanks for the nice video!
One question, how API gateway knows to which client the message needs to be delivered? Is there an internal queue per client automatically created by API gateway?
Just a helpful info.
AWS Api Gateway needs to respond within 30 sec.
If the backend behind the gateway takes more than 30sec then the API Gateway will directly respond with TIMEOUT json.
sir i am an architrct and i dont agree tht sockets arent horzntly scalable ... simply use backolane to keep connections and memory and sessions .
Code Damn is just an amazing channel of an amazing person...I'm really excited for this...
Your content is always amazing!
Another alternative is switching to gRPC which can provide bidirectional communication.
I ran into the same problem..what I did was run a sister socket concurrently so if one socket fails..the sister socket continues to run.
Those AWS heads tend to forget that Nginx could balance WebSocket connections with ease.
Serverless will absolutely allow horizontally scaling of web sockets. Each message is dispatched to a serverless function for processing.
Use message broker to publish message centrally to every instance of web socket server
Api Gateway is underdeveloped at the moment, missing WS features. It also have a 2 hour timeout which is very unfortunate.
great explanation. thanks.
Thank you so much for this, your videos are really awesome with great video quality and content too.
With help of your videos now I am able to build web projects.
Thanks a lot!!
Good explanation. Thanks!
Every time i get notified of your video, i come here as soon as I can
I always used to think if websockets are that of much fast, why use http, but now understood they are not reliable and scalable so http is better, thanks codedamn. Can you please tell us which language is better to us websockets on, php node or python....
Your are giving a misleading information (not in purpose, I know) but WS are as scalable as TCP connections are.
You should say that node.js is not scalable, instead.
Also, I think what he should say is that websockets are not downscalable which seems to be the only problem he mentions.
Yes the downscale issue seems to be the main concern
2:21 Until the world runs out of silicon 😂
ERLANG VM entered the chat 🤣
Bro you can vertical..the the idea is spread the load over the ports
codedamn forever 🔥
Hey! Congrats on Winning the codedamn T Shirt Giveaway. Please reply to this comment by mentioning your email for further communication.
Or just go on your channel page and set an email address so that we can contact you that way.
Or just reply here without using @ gmail . com.
Real good explanation 👍
Great video.
Please make a video on message broker
Websockets are hard to scale but not impossible to scale. Using a service as recommended here doesn't really solve it, but rather just passes the problem on to AWS. That might or might not be acceptable depending on your business needs.
New Java feature introduced in Java 18 as a virtual thread makes websockets very scalable, you can run hundred of thousands thread without a problem.
doesn't phoenix channels use very horizontally scaleable websockets?
The BEAM erlang vm basically
International Level Content only available in Indian Tech "Codedamn" Channel 🔥🔥🔥
Marker cap moved from little finger to ring finger @9.57
If I'm to have a solution that's not cloud based, I would use SSE's for as long as the event period is at least three seconds.
Websockets are pretty scalable, just for some solutions using them is not best for scaling and some is best for scaling. The only thing you have worry about is tcp sockets.
Just use long polling
You probably missed the part when the server connects to the client.
thanks
Awesome video
Websockets are scalable. Just use multiple of them. And keep a database for which user group is using which socket. That’s how I solved it in my application.
Great
I dont want t shirt i want more knowledge 💥💥 thankyou sir
ty
amazing video
All good but one thing COST.
i need this tshirt
This is not a good solution, it doesn't apply to on prem infrastructure.
COMMENTED FOR T SHIRT GIVEAWAY!!!! :)
Wow 🔥
Nice. Now give me a T-Shirt
JESUS CHRIST. Shut the music down.
You never said how to scale it??.What if my app gets used by millions of people everyday
Use Erlang
Redis.
🔥🔥
Ajinkya Rahane
Websockets maintain state, so if you remove the statefulness of the connection in API gateway how would you handle a case when server wants to send a push request to clients. How does the lambda know which ws connection to try? The http connection is stateless
I recently implemented a tic-tak-toe game using websockets with room functionality and I didn't know websockets are hard to scale well thanks for this information, your content is great 👍
I have a friend who has a socket based business that grew 100x faster than they expected. His life was hell scaling that software lol
I also had a socket based business that I just recently sold. Only had a few thousand users but i never had a single issue with the sockets.
@@reprovedcandy then, what is alternative?
Amazing, as always🔥
As a full stack dev, how should one start learning these important cloud concepts and technologies?
So many minutes to conclude into stupid cloud.
Really?
Discord scaled 5million concurrent connections on web sockets
Did you not watch the video?
@@codedamn Thanks for the swift reply 🙏
I did.
Can you shed some light how they did it?
There is actually a blog written by discord's cto where he mentioned about this along with why and how they migrated from mongodb to cassandra
@@surajvijay1937 thanks
it's amazing can you make live experiment
video, so it will be more helpful
topics like webSockets, server sent events are hard to find and it's a mess in your channel make special playlist for talking about strategies
You basically just proved why websockets are horizontally scalable. Amazon isn't using magic, you know. They just have a bunch of computers.
Miss your videos
You talk a lot about connection limits but I would have thought the main limiting factor would be how much traffic is going back and forth in the open connection.
"codedamn" are damn Crazy 🔥
Good knowledge. BTW, my T shirt size is L :)
Great video!
Your information is much appreciated
Thanks, i was trying to learn sockets, it helps!
Is this related to system design?
Thank you for giving solutions to problems. It was really a good concept to understand and know.
This is great! thank you.
awesome explanation
Thank you!