What is Database Sharding?

🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - [e-mail]
▬▬▬▬▬ Experience & Location 💼 ▬▬▬▬▬
► I’m a Senior Software Engineer at Juniper Networks (12+ years of experience)
► Located in San Francisco Bay Area, CA (US citizen)
▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
► LinkedIn: / anton-putra
► Twitter/X: / antonvputra
► GitHub: github.com/antonputra
► Email: [e-mail]
▬▬▬▬▬▬ Related videos 👨‍🏫 ▬▬▬▬▬▬
👉 [Playlist] Kubernetes Tutorials: • Kubernetes Tutorials
👉 [Playlist] Terraform Tutorials: • Terraform Tutorials fo...
👉 [Playlist] Network Tutorials: • Network Tutorials
👉 [Playlist] Apache Kafka Tutorials: • Apache Kafka Tutorials
👉 [Playlist] Performance Benchmarks: • Performance Benchmarks
👉 [Playlist] Database Tutorials: • Database Tutorials
▬▬▬▬▬▬▬ Timestamps ⏰ ▬▬▬▬▬▬▬
0:00 What is database sharding?
0:12 Why is database sharding important?
0:58 What are the benefits of database sharding?
1:57 How does database sharding work?
3:42 What are the methods of database sharding?
3:47 Range-based sharding
4:59 Hashed sharding
5:52 Directory sharding
6:38 Geo sharding
7:25 How to optimize database sharding for even data distribution?
7:45 Cardinality
8:05 Frequency
8:24 Monotonic change
▬▬▬▬▬▬▬ Source Code 📚 ▬▬▬▬▬▬▬
► GitHub: github.com/antonputra/tutorials
#kubernetes #devops #cloud

[@AntonPutra]

🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - [e-mail]

[@jasper5016]

Hard to believe this top-notch content has very less views. Thanks a lot, Anton!!

[@AntonPutra]

❤️

[@jackfrost8969]

probably coz it basically has no volumn

[@aslan1504]

As for the hashing, you probably will take hash only of a subset of columns of a record, most probably - only primary key, because changing any field of any record will result the hash to change also, which leads to data losses.

[@ashutoshtiwari4398]

No Nonsense, direct to point, covering all cases.
Well-compiled video!

[@AntonPutra]

thank you!

[@alonsoalcantar7847]

Ive tried to understand sharding for crypto purposes but every "crypto sharding" video display vague descriptions. I stumbled upon this by accident and it was great. Amazing work 🙌

[@AntonPutra]

thanks!

[@bunnybal]

Really very well explained, thank you very much.

[@AntonPutra]

thanks!

[@nero4581]

Great Video, as always, Anton!

[@AntonPutra]

Thanks, if you think anything can be improved, please let me know!

[@agun21st]

Thank you sir for detail explaination of database sharding. We hope a practical handson of Database sharding will publish soon.

[@xardiannon5038]

Superb explanation, and never strayed off topic.

[@Xaoticex]

Nice, exhaustive and short video considering it covers a lot.

[@GabrielPozo]

Great video! Always an important topic when we think about scale our systems.

[@AntonPutra]

Thank you!

[@helciopandelo]

Truly awesome and simple to learn!!! Thank you!!!

[@AntonPutra]

Thank you for the feedback!

[@PhillyHank]

Very helpful. Very confused and to the point! I hope your colleagues who do technical videos would follow your framework. 👍🏾👍🏾👍🏾

[@ab_azmi5584]

Great video. Clear & easy to understand.

[@AntonPutra]

Thanks Ab!

[@dmitriydiachenko5648]

Awesome video! thanks for explaining it

[@AntonPutra]

Thanks!

[@shanchessmetilda5550]

Great Explanation....Thanks for the efforts

[@AntonPutra]

Thanks!

[@meron6913]

Great video Anton.

[@AntonPutra]

Thank you!

[@charlesopuoro5295]

Thank you so much for this clear, insightful explanation of Database Sharding.

[@AntonPutra]

Thank you, Charles!

[@bossgd100]

thank you for these explanations

[@AntonPutra]

thanks for visiting

[@cptn-nemo]

Good explanation, Thanks

[@erkanakgul6954]

Great explaination! Thanks

[@AntonPutra]

thank you!

[@dhruvpatel6604]

Excellent tutorial

[@AntonPutra]

Thank you!

[@LinuxForLife]

Very interesting! Thanks! 👍

[@AntonPutra]

welcome! my pleasure

[@yourname-scorpion]

Great video, thank you

[@AntonPutra]

thanks!

[@egitim10]

Short,nice,clear

[@rayaalkhateeb874]

thanks a lot for explaining

[@AntonPutra]

my pleasure!

[@vikasgoel7529]

Excellent described

[@AntonPutra]

Thanks!

[@69k_gold]

Let's say I'm using a shard-nothing architecture, now let's say there's a relationship between customers table, payments table and orders table.
Customers and orders tables are linked by the foreign key customers->id ~ orders->customer_id
Orders table and payments table has the foreign key
orders->id ~ payments->order_id
Now how would you shard this database? You can't use a single shard key, because both customer_id and order_id are important that ensure all the related rows are in a single shard.
So how would you solve this problem?

[@EpoxyArt]

Hi, Anton! How can I search by the field that is not shard key? I need to go thru all the shards?
And what if I need to scale it up or down (change shards number)?

[@nicgeorge6126]

Fortunately I’ve been able to get by with two read databases and a write by using table partitioning up until this point. Hopefully I don’t have to tackle sharding any time soon! Great video and thanks for sharing

[@AntonPutra]

Thanks! There a lot of distributed databases based on postgres that can shard for you.

[@tejapolisettysai6950]

sir thanks for the video, what do you use for editing, its really good.

[@AntonPutra]

thanks, adobe suite

[@user-yv6ti2wf7c]

thanks, but i have a question if i use range-based sharding and conside 4 shard what happen if i want to convert to 40 shard? what happen for previous data, and new data [ first i have 3 shard 1(a-h) 2(i-p) 3(q,z)) now need to make it 40.

[@AntonPutra]

If you shard manually at the application level, you need to write logic to rebalance it yourself. It's easier to use built-in mechanisms for sharding.

[@user-yv6ti2wf7c]

@@AntonPutra thanks

[@ProSunnySharma]

Excellent! What tool do you use to do animations?

[@AntonPutra]

Thanks Adobe suite

[@ricardorqr]

How do you create the animation for your videos? They are so cool!!!!! 💪🏼

[@AntonPutra]

Thanks! I use adobe suite.

[@user-ui5yw6pf5p]

you said sharding have unique data sets if one sharding not respond then other sharding response you but if customer search record and that record will be in sharding 1 . After that sharding 1 will not respond then what we have to show for customer

[@gcheese25]

great video! start subscribing now

[@AntonPutra]

Thank you!

[@rahuldinesh2840]

Can I do sharding in WordPress database?

[@AntonPutra]

WordPress uses a MySQL database under the hood. Take a look at Vitess.

[@noahgsolomon]

w video

[@AntonPutra]

thanks

[@andherium]

I think 99.9% use case are served fine by a monolith database server. Heck even stack overflow is fully powered on a single server

[@cariyaputta]

Yes. Still, it's useful to have a knowledge of whatever jargons uppermamagements are throwing at you.

[@AntonPutra]

For personal projects, sure, but in the enterprise, you frequently have to deal with sharding.

[@thewaygoesup]

your example confusing between database shard and table partitioning
range-base sharding is about one table sharding not about database sharding.

[@AntonPutra]

Noted, will improve

[@APoIIy]

So, sharding is a pain in the ass and requires a lot of configuration, analytics and also business logic to manage shards in an respectable way.
This is also why NoSQL Databases come in handy as they can scale better horizontaly without this extensive configuration activities you have with traditional SQL databases.
But to be said SQL Databases will probably cover 90% of all usecases anyway without you getting into sharding.

[@AntonPutra]

Nowadays, once a year, I see a new distributed database based on PostgreSQL come out, lol.

[@yoggg932]

Feels like a bot is reading the script. Good content, but please act it out a bit.

[@AntonPutra]

thanks for the feedback