What is DATABASE SHARDING?
ฝัง
- เผยแพร่เมื่อ 28 เม.ย. 2024
- Sharding a database is a common scalability strategy for designing server-side systems. The server-side system architecture uses concepts like sharding to make systems more scalable, reliable, and performant.
Sharding is the horizontal partitioning of data according to a shard key. This shard key determines which database the entry to be persisted is sent to. Some common strategies for this are reverse proxies.
Database interviews ask for concepts like sharding to make databases more performant and available. This makes horizontal partitioning a logical choice.
Looking to ace your next interview? Try this System Design video course! 🔥
interviewready.io
00:00 Introduction
00:52 Sharding - The problem
01:59 Horizontal Partitioning
03:20 Considerations
04:30 Potential Drawbacks
08:46 A challenge!
Video lectures, architecture diagrams, capacity planning, API contracts, and evaluation tests. It's a complete package.
Horizontal vs. Vertical partitioning: stackoverflow.com/questions/1...
Consistent Hashing:
en.wikipedia.org/wiki/Consist...
• What is CONSISTENT HAS...
Sources:
/ how-sharding-works
/ why-i-love-databases
www.quora.com/What-is-the-dif...
en.wikipedia.org/wiki/Partiti...)
Designing Data-Intensive Applications - amzn.to/2yQIrxH
#database #sharding #systemdesign
Notes to self:
* Sharding is basically a hiearchical way to index databases.
* One problem is that you have to split the database somehow. What do you split on?
* You only shard shards when the shard grow too big.
* When shard fails you use the master/slave architecture. Writes always go to master, reads are distributed across the slaves. When the master fails one of the slaves become master.
Nicely put :)
I think another important question to ask is what happens if your shard exceeds its capacity. The way it works is that it further splits the shard into smaller shards (DynamoDB does this behind the scenes)
What will do when both master and slave failed?
@@sarathbaiju6040 replicate them atleast 3 (general practice)
@@user-eq4oy6bk5p I just have one question. When we further break a shard into multiple shards, does those shards again go into different database servers? Or they reside in the same database server?
You know you're a programmer when you count pizza slices from zero :P
Hahaha!
Or really hungry
U r wrong... Person might be a Matlab programmer where count doesn't strts with zero.
@@gkcs Did you said "No we're not going to learn RDBMS" at t=40? BTW a great video Thanks!
You know you’re a DBA when you count them from one ;)
Can't just believe, you're uploading this for free... Please continue and we sure do want to sponsor the channel if that is in the cards!
I see 67 likes so far. I doubt all 67 people are going to donate. XD
Keep doing what you're doing..You explain too nicely...and your channel is the most handy updated and to the point..learning was never so easy..
@@dhruvilpatel856 12k likes :)
You could be a greater asset to this country if you pour yourself into research. I studied this course from the best of the best faculty in my college, who literally knows things, but you! Hands down man :)
Thanks 😍
You should have learned from best in world from internet..
He could be a teacher in your University, but I wouldn't see him from Chile.
Very nice videos in general. The intro of this one !!! Good job !
lol research. you can be an asset in the industry as well.
Now I seriously doubt about your College Quality :P :P :P - You should have more focused during your JEE time. :P
I can't able to resist myself without commenting after I see your enthusiasm to teach to unknown people around the world... Hats off to all the vloggers who are sharing knowledge freely through TH-cam.
The initial reaction of you saying sharding and your expression is awesome which makes someone really remember the video and the content by just imagining while asked in interview. Thanks gaurav :)
Your System Design Playlist is my favorite. I've got so much to learn.
Thank you!
Glad to hear that 😁
You are simply amazing man.
I used to dread system design, but just because of you, slowly getting comfortable with it.
I have actually allotted a particular share of my daily todo, just to watch your videos everyday.
good precise and short. exactly what is required in tutorial video with a bit of fun element in start.. awesome job
That little skit at the beginning is so funny lol. Thank you for making these system design videos, they're really helpful!
You are a living legend Gaurav! I learned more from your videos than any of my previous teachers
I love this play list and I'm learning so much but it the humor 😂 and your jokes that keep me learning great job man! Do you have a book recommendation on system design?
I am a non-tech person, looking to learn system design concepts. You do a phenomenal job explaining these concepts & also super fun. Great job Gaurav & thank you very much!
Thank you!
For the first time in life I'm actually enjoying theory
I like the way you present all the topics. It's so captivating, and it shows how passionate you're at solving System Design problems. Thank you for sharing your knowledge.
I don't know how I didn't come across your videos before! What an easy way to learn, well done mate!!
Never been so interested into database sharding just from that intro. You rock! Thank you for the videos
😁
The way you slipped the "Like & Subscirbe" part into the video, just one word - S L I C K
The topics covered were organized,relevant to me, and easy to follow . I loved it so much
Initially I was not able to catch this concept when I watched this video 2-3 years back. But as of now I am with a base knowledge I able to understand. The problem with me was not really a base knowledge, but a lack of perspective towards these system, I was sticked to my perspective and was trying to build over it, so sometimes it is better to gain knowledge with clear slate.
What is Database Sharding?: Effectively partitioning the pizza
You are too good brother. Your passion towards computer science can be clearly seen.
Thanks Sandeep!
You are one of the best. I can't wait to watch all your tutorials for system design! :) Keep going!
Thank you for sharing this Gaurav. It means a lot.
Woah opening was super cool
I am an Oracle database developer. I loved your content man. Learned something new today. Thanks.
Greetings from your new subscriber.👌👌😊
Love your vids. The way you deliver your information is really clear and concise. Amazing quality content!
That was a great job, can you do a video on what is a microservice and differences between soa and microservice?
God I so miss eating Pizza in pre-lockdown days!! I have just started watching your videos for an upcoming interview and I am finding them quite useful. Thank you so much :)
You’ve explained this concept so well! At work, I’ve always heard my manager use this word when explaining his responsibilities with MongoDB but I’d never understand it. Now I do, thanks to you dude!
PS: LOVED the ending 😂
Sir,Your Work is too much Valuable and appreciated,
Thank you so much for Doing this amazing work.
This inspired me to watch all of your videos. Subscribed
I recently found your channel and I just want to say I love your work. Thanks for creating fun informative content like this
Thank you 😁
You make life simple! The intro had me hooked in till the very end! Bravo!!! 👏
Last line is ethusiastically official way of building interest in users...very well explained sir.
Your way of explanation is really cool.
Super video sir.
That interview clip with doppelganger was awesome!
Hahahaha! He's almost as good as me 😋
He can easily send shivers down the spine of an interviewee, that's for sure 😅
I had to stop the video immediately and come down here in the comments section 😊..what an intro...what a playlist..kudos Gaurav for these amazing videos
The ending statement, That was the best
I just discovered your channel and I'm a huge fan. I have been studying to get certified in AWS and your videos have been helping me understand larger concepts.
Thanks!
Interesting name you have there, Hotmustardgas :P
@@gkcs Lol thanks I'm glad that you like it. Do you have any experience with the major cloud platforms that are out there? If so you should make a video on them.
"Let's say you have pizza, and you can't have the entire thing by yourself..." you've obviously never seen me eat pizza...
I got sharing vs partitioning question in an interview. This image & whiteboard notes flashed in my head instantly, I did crack the interview but got better retention offer. Thanks man !
PS - I should've used pizza as an example starting with 0 index, maybe next time...
Awesome bro...while watching ur videos it feels like I came to movie
"indexing is cute" hahahaha
I had to pause the video and laugh for that :D
Cracked me up 😂😂😂
Cuter when all sharding can be personified as is indexing over the network, wouldn’t it? 😂
Me spending days to implement indexes 😭
You are crazy bro . Excellent video
Great work guys! I just stumble upon on this video but it turns out to be great! loved your explanation and confidence. Keep it up. I have just subscribed this channel and would love to see more videos.
Thanks!
Love the explanation. Digestible while still retaining the core information of this utility and its implications.
The first minute was Killer!
"Let's say you have a pizza" This guy's a genius. Thumbs up for helping us relate with an actual example.
First time I was watching some video tutorials and I did not sleep.
OMG!!!Today's interview went exactly like this!!!!! except the sharding which led me to this video!!!!!
I am thinking about the 255 others who have disliked this video, now who on earth would dislike such a well explained video!!!
The ending was superb ...Enjoying the series very much
This cat is smart. Thanks for dumbing it down for me. This is fantastic stuff Gaurav!
This is really fun to watch!
I work at Oracle and my first project was related to implementing XA Transactions in WLS + Sharding, 6 months ago, I was a recent graduate so it was very complicated 😂😭
Went to Yogita"s videos ,came back here again , both are excellent masha allah
You have made it look easy! Thanks bro. I want to hear more from you :)
You are an amazing person with amazing talent. Once you happened to come our clg NIT raipur, that was wonderful lectures. Please make videos frequently
Thanks 😁
Amazing explanation on database sharding, thank you for sharing.
Cassandra uses consistent hashing out of the box to reach the shard that contains the data you're looking for. It uses the partition key part of the primary key
Going through some of the videos. I can definitely see they are more entertaining as they go.
Hitting like and subscribe button was pretty easy for me therefore sharding is ..?
Hi Gaurav,
Your videos are really very helpful. Apart from videos the references you have provided are mind blowing. Thank you so much.
Simple explanation and to the point. Keep up the good work bro. Cheers
That's exactly how my System Design interview went down. I got the job too. 😉
Congratulations!
8:45 🤣 I hit that notification button. The only other channel I have that turned on for is Kurzgesagt.
Thank you!
Your videos are really good. Editing and delivery are on point! Keep it up.
Thanks!
Nice video. Main problem i faced with Sharding is when you want to scale horizontally without impacting live system performance or creating downtime
Absolutely. That's almost as hard as a live DB migration.
Clear & crisp, to help understand the core concept of Sharding😊. And no, I wasn't able to click Like & Subscribe at a time, so had to do it sequentially. 😁
Thankyou! I learnt something new today because of you.
Congratulations! 😁
Your creativity and content just get's better.
Starting part of the video was cute and hilarious 😂. Keep up that thing.
Hahaha thanks!
Thanks a lot! you are doing an amazing job and teaching complex concepts in a interesting way! :)
Hey Gaurav
Your videos are really helpful, easy to understand and so much fun to watch 😂
Keep doing the good work 👍
Bhaai toone toh rock kar diya, thank you , love you
Dude you are doing really great; learned a lot from you!!!!! Liked, subscribed, shared!!!!!
Excellent video! Some videos about the Map Reduce concept would be amazing!
Thank you!
bro your soo amazing
thankq soo much
please conitinue to post awesome contents like these which could be understood by non CS person like me.
Great stuff!!
This channel is a gold mind for entrepreneurs in the pizza-business.
I just love all your videos
can you post faster those competitive programming videoss
that interview part was toooo good. especially when you say , we are looking for somethig more serious , we have huge hugeeeeee data. :-)
Ty bro for this amazing video ❤️❤️
First thing, i will never call my friend to help me finish the pizza, i can eat the pizza alone🤣🤣🤣
thanks a lot for such an awesome content for free
Thank you for this great video again. Better than many university professors.
Thank you!
Gaurav,, great work. Appreciate all your hardwork. It helps a lot.
oh god! the reaction when said "Sharding" by interviewee is humourous man :D
Beautifully explained, keep up the good work, hope to meet you one day in person.
Continue the great work!
Good video here are my questions
1. How do you make a practical application without Joins?
2. Any code example of the concept?
O bhai kahan sy le aye ho itni age ma itna knowledge. I really wanna know your learning strategies. Great man! Thumbs up, form Pakistan
“The key thing is we couldn’t finish the pizza by ourselves, we needed friends”
As an American I feel uniquely equipped to refute this assertion 😂😤😅
Awesome explanation, expecting more form you. I am big fan your computer science, knowledge...
Intro is awesome man!! Love it
Oh my goodness, keep going again so nice.
Really awesome.
I find your videos educating and entertaining as well.... Well done
Thank you Aakash!
Clean and simple explanation. Great work
U r amazing gaurav... thanks for the upload
Well Explained concept. 👍Understood it easy with those visuals in the starting of this video. It made the concepts more practical to understand. 🙌🙌
Wao..role playing an interview is an awesome concept
😎