Back in the 90's, when web app servers were new, we thought it was odd too: Round Robin and Random were so close to the best algorithms we could come up with, we decided not to bother with anything more complicated.
While this is a layer or three deeper into the stack I develop (falling into the "not practical in my day-to-day life" category mentioned), I still found this very interesting and informative (as much as I could not fully comprehending about 51% of what was presented) since I do deal with API requests and page load performance concerns.
What if each server puts a token for each of its workers in a pool, and on each request the load balancer randomly takes a token from this pool (so it also removes it from the pool), then sends the request to the chosen server, and once the request has been processed the server puts back a token in the pool?
It is slightly difficult in a distributed setting to do that. That said, you might want to google Token Bucket Algorithm for rate limiting, for more pointers, in case you haven't come across already. Also, this is a good starting point: stripe.com/blog/rate-limiters
wow, this is awesome!! Randomized JSQ makes such a big difference!! Thank you for this wonderful talk!!
Back in the 90's, when web app servers were new, we thought it was odd too: Round Robin and Random were so close to the best algorithms we could come up with, we decided not to bother with anything more complicated.
I know that hair :)
While this is a layer or three deeper into the stack I develop (falling into the "not practical in my day-to-day life" category mentioned), I still found this very interesting and informative (as much as I could not fully comprehending about 51% of what was presented) since I do deal with API requests and page load performance concerns.
I'm like bothered by the fact that he, like, says "like" litteraly every like, three words. Really gets on my nerves. Sorry, had to mention it.
By mentioning it, you've made it worse. I can't un-hear it now :(
Great talk.
What if each server puts a token for each of its workers in a pool, and on each request the load balancer randomly takes a token from this pool (so it also removes it from the pool), then sends the request to the chosen server, and once the request has been processed the server puts back a token in the pool?
It is slightly difficult in a distributed setting to do that.
That said, you might want to google Token Bucket Algorithm for rate limiting, for more pointers, in case you haven't come across already.
Also, this is a good starting point: stripe.com/blog/rate-limiters
this is JIQ basically kind of :)