Finally! A video with that gives a clear background of why hashing is even used in the first place in load balancing! Other videos just mind-numbingly just start like "you have a circle of servers and requests and you hash this and that..." (what!?). Thank you for the awesome explanation!
It's shockingly surprising why so less views and so less feed for this awesome video for this concept introduction 101. Excellent video Hannah, purely for the simplicity of words and excellent explanation approach. Looking forward to many more! Thanks!
Super easy to understand! Clean and simply explained! Thank you!! Reminded me of Einstein's quote, "if you can't explain it simply, you don't understand it well enough".
take a bow!!!!! what an easy explanation, nicely presented. I had been googling but couldn't find straight forward way to understand. THANKS A LOT. Cheers.
Hey Hannah, Why didn't you put more videos ?..Your explanation is super crisp and soothing to ears. Very unlikely these days. Thanks for the awesome video though.
its like you having more postal addresses. so more the number of postal address, lesser is the chance that you will get a missed courier. of course you will not be present in all the postal addresses. someone else might take it on your behalf and route to ur actual physical location. so the postal addresses are "virtual" instances of yours, mapped to a "physical" instance of yours. so more addresses you have (aka, computational power of a server), more chances of you collecting deliveries (aka, requests from a user). this way you can adopt a skewed and staggered load-balancing strategy based on the computational powers of your servers.
Awesome and simple explanation! Hannah, given K hash functions and a request id, how to decide which function to apply on the request id? Do you just take one of the K function randomly?
My understanding is you're hashing the server id here, in order that you get multiple copies of it across the hash ring (we're still in the same id space). This is different to the request hashing function, requests will just get mapped as usual. However instead one segment for each server in the first case, we have loads of segments that should more evenly distribute to all the servers
Also, in practice, instead of using K hash functions, K server id variants are used with a single hash function (e.g., by appending some prefix or suffix to the server id). It seems to be easier to maintain consistency and a good uniform distribution that way.
Great video... Just that I was expecting some practical example with hashing on circular ring part. Apart from that awesome. Keep it up and thanks a lot! :)
vinaykumar15 the example makes sure all request for a session go to the same sever (you would not hash the request Id but the session Id or source ip+port). But typically consistent hashing is not used on client load balancing applications.
The purpose of consistent hashing in this case is to ensure that cache misses are minimized when a machine goes down or when a new one comes up. Load balancing is a somewhat orthogonal issue, but there are some algorithms to get the best of both worlds (because vanilla consistent hashing may suffer from overload, as you suspect). Vimeo ran into this problem and implemented one such algorithm in HAProxy (sending its PR to the maintainers), and it is now available for everyone to use.
The servers S0, S1, S2 etc. are now hashed with different hashing algorithms H1, H2, H3 to increase randomness. Are the requests R0, R1, R2 etc also hashed with H1, H2, H3 ? or just with a single hashing algorithm ?
You will hash the request with only a single hashing function, you don't want to duplicate the request. The idea is to distribute the requests evenly among multiple servers, that is why we use multiple hash functions. Also note, virtual nodes are not real nodes if you have 3 different hash functions then you will get 3 virtual nodes for every actual node.
Thank you for the introduction. If, for eg., the second server was down, the data got saved into next server on the ring. What will happen when the second server gets back on? Will it go through all the data in the third server and move whatever belongs to second server back?
Actually In normal hashing, when second server goes down, We need to rehash immediately in order to redirect the request on first and third server. otherwise all the request which mod value referring to second server, will get lost.. So, when 2nd server back, we need to rehash again and redirect the request to the second server. For Consistent Hashing, data will not back from third server to second server unless we migrate it manually(through some procedure). Also it depends, If you are using it for caching, generally cache data may expire after sometime, so in that case, migration might be costly operation, but if consistent hashing applied on database level, then we need to migrate the data from third server to second server.
Finally! A video with that gives a clear background of why hashing is even used in the first place in load balancing! Other videos just mind-numbingly just start like "you have a circle of servers and requests and you hash this and that..." (what!?). Thank you for the awesome explanation!
I just pictured the amazon data centre engineers moving the servers in a circle and being very proud they got it perfect.
I realize I am kind of off topic but does anyone know a good site to stream newly released series online?
@Aldo Melvin Lately I have been using FlixZone. You can find it by googling :)
I totally agree, Thank you Hannah for explaining the complex concept in a very easy way
Short and sweet. After couple of hours of sifting through all that junk out there, found the best one.
best explanation I've found of this. Most people just draw a circle but you're explanation was concise and very well explained. Thanks a ton!
It's shockingly surprising why so less views and so less feed for this awesome video for this concept introduction 101.
Excellent video Hannah, purely for the simplicity of words and excellent explanation approach.
Looking forward to many more!
Thanks!
Super easy to understand! Clean and simply explained! Thank you!!
Reminded me of Einstein's quote, "if you can't explain it simply, you don't understand it well enough".
The best of all CH videos hands down!
this is the best video on 'Consistent Hashing' . Thanks a lot for making this.
take a bow!!!!! what an easy explanation, nicely presented. I had been googling but couldn't find straight forward way to understand. THANKS A LOT. Cheers.
thanks for authentically short and precise info.
Thank you so much for this explanation! I've been all over youtube and couldn't understand one thing. You saved me!
Amazing explanation, truly appreciate it!
this was so great, thanks!! A little sad you don't have more videos like this, cheers!
Nice clear presentation. Thank you. Brief and to the point is very welcome in an educational video.
This is the best video on Consistent Hashing.
Can you post videos on other topics as well ?
Many teachers out there miss out this basic way of building fundamentals into complex system. Basics first ! Great job
This was so helpful and clear compared to other resources I've seen. Thank you!
Well done.. You focussee strongly on the problem statement
Really an awesome video on consistent hashing.Great work.Love from India
Consistent Hashing is a fairy tale told by Akamai to get a patent. The ring that everyone draws is really an ordered map, invented around 1960.
Amazing Explanation!!!
Excellent Explanation 👏
Thank you for simple explanation of this concept. Please continue to make system design explanation videos! You are very good at it!
Thank you! Amazing video.
Finally a good video on consistent hashing, thanks a lot!
Great explanation! Thanks!
Hey Hannah,
Why didn't you put more videos ?..Your explanation is super crisp and soothing to ears. Very unlikely these days. Thanks for the awesome video though.
best video so far, crisp & clear, thanks @Hannah
clear and crisp information. Good work!!
sweet and concise.
brief & clear explanation of an important subject. thanks!
Thanks for explaination but got confused at end. Need explaination with example
its like you having more postal addresses. so more the number of postal address, lesser is the chance that you will get a missed courier. of course you will not be present in all the postal addresses. someone else might take it on your behalf and route to ur actual physical location. so the postal addresses are "virtual" instances of yours, mapped to a "physical" instance of yours.
so more addresses you have (aka, computational power of a server), more chances of you collecting deliveries (aka, requests from a user). this way you can adopt a skewed and staggered load-balancing strategy based on the computational powers of your servers.
Very clear explanation. Thanks!
Great video, thank you! You should do more videos, your explanation was very clear.
Thank you!
Fantastic. You explain this so well!
best explanation on consistent hashing out there
Great presentation! And a special thank you for clear English. Think about making more presentations.
Very nice and to the point explanation. Thanks!
Great and precise video, please post more discussion like these
Nice Explanation. clear and crisp.
Nice video, clean and concise explanation, many thanks!
simple and elegant :)
Very clear explanation. Thank you
Thanks for the video! Could you please make more? :)
great explanation! could you make more videos?
clear and concise and short.
Great video
Really well explained, please make more videos!
Thank you. Explained very well and easy to follow. great video
Nicely put. Thanks Hannah.
Thank you for the video. I heard that adding more virtual nodes distributes the load evenly. I am not sure how? Could you please clarify that?
Thanks for explanation of consistent hashing. Good video!
you did an amazing job at teaching, but this seems to be the only one video uploaded in your channel. please keep going.
Super clean explanation. Thank You. I would encourage you to make some more videos like ths :)
Awesome and simple explanation!
Hannah, given K hash functions and a request id, how to decide which function to apply on the request id? Do you just take one of the K function randomly?
My understanding is you're hashing the server id here, in order that you get multiple copies of it across the hash ring (we're still in the same id space). This is different to the request hashing function, requests will just get mapped as usual. However instead one segment for each server in the first case, we have loads of segments that should more evenly distribute to all the servers
Also, in practice, instead of using K hash functions, K server id variants are used with a single hash function (e.g., by appending some prefix or suffix to the server id). It seems to be easier to maintain consistency and a good uniform distribution that way.
Did not understand the part after 6:11. What are these three hash functions within a server?
Excellent video! Congratulation!
very well explained and good Virtual Node introduction
How do we distribute the server nodes throughout the ring?
Great stuff... Please create more such videos (y)
Sweet, thanks for this!
Amazing! Thank you Hannah.
You have introduced me to a new concept called Virtual Nodes. Feels like enlightened. Thanks much.
Great video... Just that I was expecting some practical example with hashing on circular ring part. Apart from that awesome. Keep it up and thanks a lot! :)
thanks for sharing the knowledge. Lots of Love.
you didn't explain why consistent hash using server id as input instead of request id
You've made a grand total of only ONE video??? Why.....
Can you explain if load balancer uses round robin algorithm to evenly route the incoming request why do we even need consistent hashing?
vinaykumar15 the example makes sure all request for a session go to the same sever (you would not hash the request Id but the session Id or source ip+port). But typically consistent hashing is not used on client load balancing applications.
The purpose of consistent hashing in this case is to ensure that cache misses are minimized when a machine goes down or when a new one comes up. Load balancing is a somewhat orthogonal issue, but there are some algorithms to get the best of both worlds (because vanilla consistent hashing may suffer from overload, as you suspect). Vimeo ran into this problem and implemented one such algorithm in HAProxy (sending its PR to the maintainers), and it is now available for everyone to use.
Here’s the video where they explain it: th-cam.com/video/jk6oiBJxcaA/w-d-xo.html
The servers S0, S1, S2 etc. are now hashed with different hashing algorithms H1, H2, H3 to increase randomness. Are the requests R0, R1, R2 etc also hashed with H1, H2, H3 ? or just with a single hashing algorithm ?
You will hash the request with only a single hashing function, you don't want to duplicate the request. The idea is to distribute the requests evenly among multiple servers, that is why we use multiple hash functions. Also note, virtual nodes are not real nodes if you have 3 different hash functions then you will get 3 virtual nodes for every actual node.
Thank you for the introduction. If, for eg., the second server was down, the data got saved into next server on the ring. What will happen when the second server gets back on? Will it go through all the data in the third server and move whatever belongs to second server back?
Actually In normal hashing, when second server goes down, We need to rehash immediately in order to redirect the request on first and third server. otherwise all the request which mod value referring to second server, will get lost..
So, when 2nd server back, we need to rehash again and redirect the request to the second server.
For Consistent Hashing, data will not back from third server to second server unless we migrate it manually(through some procedure). Also it depends, If you are using it for caching, generally cache data may expire after sometime, so in that case, migration might be costly operation, but if consistent hashing applied on database level, then we need to migrate the data from third server to second server.
greate video! thank u veeery much
Thanks for the video; appreciate it.
Very helpful ! Keep it up.
such nice small and informative vdo :D .... Plz continue the good work :)
Started out really promising but totally vague towards the end. No takeaway indeed.
Really educative!
Thanks !
Треба більше подібних відео.
After watching this video, I am wondering why I didn't understand from other sources. Its a shame that there's just 1 video
greate video
nice!
virtual node explanation is not clear
Finally samjh nhi aya
Everything was good till virtual nodes.
you waste me 5 minutes in watching your useless explaination
Thank you!
Great video