A beginner here. Would you suggest starting with node when I know task is compute intensive? (Was planning to use Django restful) Also can you suggest some resources for advanced topic like this. Thanks in advance.
I would suggest first implementing your app,. and then thinking about scaling it - not the other way around. For a starting point you can check articles on HighScalability: highscalability.com/ - it can give architectural insights and real-life use cases about how performance optimization is done. Other than that, google is your friend ;) Hope that helps.
Hi Shasak. - You can always simulate a scalable environment with docker or vagrant on your local machine and then gradually scale up. That’s what I normally do with my side projects: I create a approximation of what the architecture looks like using docker container and docker networking on my local machine. - that costs zero bucks. Then I start with a single server and gradually scale up. If you don’t to give an arm and a kidney to AWS, you might want to try linode.com - their machines are much performant than AWS, and their plans start from $5. Hope that helps.
Thank you! Great talk by the way, learned a lot from it. I was wondering what your take on scaling an app that requires persistent connections (example: a multiplayer HTML5 game server) using web-sockets would be? What would you change (if anything) about your approach as described in this talk?
Thanks, glad that the talk was helpful. websockets is a good-enough choice for anything realtime. What I would do would be to outsource the websocket implementation. - Creating a simple websocket server is something; scaling websockets to millions is something else. I’ve managed a startup related to that, and at peak times we had ~100K concurrent users. - We had to rewrite the open source socket library we had, in C, based on our needs, to scale. Plus, in certain cases websockets will not work; so you might want to downgrade, or find the best communication proxy for the situation. I’d use pubnub, or pusher, or something similar to outsource that. That said, when you think you need persistent connections; sometime you don’t actually need. More often than not, your users will not die of you don’t check a service status in realtime; so you can poll it every half a minute or so etc. So unless you are doing gaming, or stock trade or something similar realtime is a burden. Bottom line: I’d stay away from realtime persistent full duplex connections as much as I can; and when I need to use it, I’d outsource it if I can to “as a service” solutions who know what they are doing. Trust me, passing pointers to persistent socket connection in C, and debugging memory leaks and kernel crashes is not fun :)
Great talk! Thank you for sharing!
Glad it was helpful.
If anyone watching this video considered themselves to be decent Backend developers, their ego probably regresses to zero by the end of this video.
Happy to be of use.
Stunned by the amount of knowledge you possess.
Thanks Rausan; I’m glad it was helpful.
A beginner here. Would you suggest starting with node when I know task is compute intensive? (Was planning to use Django restful) Also can you suggest some resources for advanced topic like this. Thanks in advance.
I would suggest first implementing your app,. and then thinking about scaling it - not the other way around.
For a starting point you can check articles on HighScalability: highscalability.com/ - it can give architectural insights and real-life use cases about how performance optimization is done.
Other than that, google is your friend ;)
Hope that helps.
Any mini project that one can do to employ these techniques, on their own system not requiring multiple machines or paid VMs?
Hi Shasak. - You can always simulate a scalable environment with docker or vagrant on your local machine and then gradually scale up.
That’s what I normally do with my side projects:
I create a approximation of what the architecture looks like using docker container and docker networking on my local machine. - that costs zero bucks.
Then I start with a single server and gradually scale up.
If you don’t to give an arm and a kidney to AWS, you might want to try linode.com - their machines are much performant than AWS, and their plans start from $5.
Hope that helps.
Thank you! Great talk by the way, learned a lot from it. I was wondering what your take on scaling an app that requires persistent connections (example: a multiplayer HTML5 game server) using web-sockets would be? What would you change (if anything) about your approach as described in this talk?
Thanks, glad that the talk was helpful.
websockets is a good-enough choice for anything realtime.
What I would do would be to outsource the websocket implementation. - Creating a simple websocket server is something; scaling websockets to millions is something else.
I’ve managed a startup related to that, and at peak times we had ~100K concurrent users. - We had to rewrite the open source socket library we had, in C, based on our needs, to scale.
Plus, in certain cases websockets will not work; so you might want to downgrade, or find the best communication proxy for the situation.
I’d use pubnub, or pusher, or something similar to outsource that.
That said, when you think you need persistent connections; sometime you don’t actually need.
More often than not, your users will not die of you don’t check a service status in realtime; so you can poll it every half a minute or so etc.
So unless you are doing gaming, or stock trade or something similar realtime is a burden.
Bottom line: I’d stay away from realtime persistent full duplex connections as much as I can;
and when I need to use it, I’d outsource it if I can to “as a service” solutions who know what they are doing.
Trust me, passing pointers to persistent socket connection in C, and debugging memory leaks and kernel crashes is not fun :)
Where can i find this ppt and associated links ?
Hi Arun, you can get them here » speakerdeck.com/volkan
I lost him on 5th minute