So you created a channel, made 6 videos, showed us how a system design interview video should really be like, then dropped the mic and left :-) I am really sad that there aren't any more videos, but at least I hope the reason is that you are in a very happy place, and you don't have time anymore, because you enjoy your life :-) In any case, thanks a lot for the videos that you did make! There are other decent videos on system design, but none matches the ones on your channel
@@avraamtolmidis5188how is it? Is the 1st module good? I'm still waiting for the 2nd module before I buy it. He mentioned on Leetcode that he's 60% done with the 2nd module.
I recently interviewed at 2 top companies and I just nailed the system design interviews. In total, I had to go through 4 such 45 min interviews. I have around 10 years of engineering experience but my background is not strictly distributed systems. I am good at DS/algo but system design was my achilles' heel. Mikhail made it simpler for me to catch up. I did read a lot of books to understand how systems like Kafka, Zookeeper, Cassandra etc. are implemented. I had lots of bits and pieces of information from lots of sources. This channel helped me put all that understanding together in a coherent way. I also followed the structure described by Mikhail in the interviews and it helped me keep the discussion on point. The interviews were difficult, interviewers asked tough questions but listening to these videos multiple times helped me calm my nerves. I haven't heard back from those companies yet but initial signs look positive. We will see how it pans out. Regardless of the result, I feel satisfied with my preparation and a big chunk of credit goes to Mikhail. Why am I writing such a big comment here? Well, I want to obviously thank Mikhail but I want people to realize how valuable this channel can be for interview preparation. Please try to make it more popular by sharing it with your friends.
Hi primespace! Did I pay you to write this post? )) To be serious, I am really glad for you. To know that these videos helped you both sharpen your knowledge and give you a boost in confidence, makes me feel happy. Regardless of the result, you should be proud of yourself. Watching these videos thoroughly multiple times and reading through a lot of resources is a hard work. I know it. So, give yourself a credit. If you feel inclined, please let us know the final result of your interviews.
@@SystemDesignInterview Hahaha! No money was involved and it is coming straight from heart :). It should be the other way around. People should be paying you for this excellent content. I compared it with paid content like "Grokking the system design interview" and your content is far more practical and miles ahead. Yes, I will share the outcome once I hear back from them. One unique pattern I observed in these interviews was that the interviewers forced me to design a system which does not need to scale and will run only on one machine. They wanted to discuss the threading model, how would I use mutexes and locks etc. It almost seemed like they wanted me to keep things very simple and instead of saying big things, come up with simple solutions like - I will use SQLite because requirements are simple. Just something to watch out for. People shouldn't fall in the trap of assuming the interviewer is asking to design a highly scalable system. I almost fell for it but corrected myself in one interview. I said things as simple as I'd simply store the data in XML file if it is only 100 users and the interviewer was very happy to hear my response. Once we spent 15 minutes on very simple stuff then only they moved to scaling aspects. Surprisingly they drilled me on things like how DNS works, how TCP handshake happens and what is SSL etc. These questions were asked at random times while I was talking about something unrelated. In any case, these were easy enough to answer so all good.
@@SystemDesignInterview The results came back. Yes from both. Still has to match teams and officially get offers but so far so good. Thank you again, Mikhail! It would not have been possible without your help.
@@abby297in thank you for sharing your experience!! did you actually know the answers for those networking layer-specific questions and do you think familiarity with the OSI model would be useful to study at all?
Continued from someone's comment : 1:30 Why you need to ask your interviewer questions? Why requirements clarification is so important? 3:19 What we may want to ask about? Requirements clarification (User/Scale/Performance/Cost) 7:14 Define functional requirements-API 9:19 Define non-functional requirements 10:32 High-level architecture 11:56 Define Data model (what we store) 16:09 Where we store: List of non-functional requirements 17:20 Where we store: How SQL database handle these requirements 20:40 Where to store: About NoSql: Cassandra 25:00 How we store: data modelling for sql v nosql 29:03 Data processing path 30:51 Data processing: Data aggregation basics 34:42 Data procesing : Processing service detailed design 41:08 Data Ingestion pipeline 1:03:00 Data Retrieval path 1:06:35 Data flow simulation 1:08:26 Technology stack 1:13:22 Bottleneck, tradeoffs and more 1:09:06 Summary Kudos to the creator. Happy Learning.
don't look at it as a 83 min video, it contains enough contents to use a week to digest it :) it is absolutely fantastic, comprehensive, understandable, and maybe the best video in System Design Interview on TH-cam and anywhere else. Thanks for all the efforts to providing these contents.
This is pure gold. I have slept several times, woke up, rewinded several times, made notes, still watching. You are a legend. Hats off to you, you spent 1.23 hours talking non stop on a serious and important topic, covering everything. You explained several topics in so simple language which was hard to grasp. One was data consistency and availability and other was Eventual consistency and many more. Thanks once again. May God Bless you.
It took me a few days to get through this video. As a software engineer with 8 years of experience, you have very clearly demonstrated to me that I know nothing. Thanks D: Cram time...
1:30 Why you need to ask your interviewer questions? Why requirements clarification is so important? 3:19 What we may want to ask about? Requirements clarification (User/Scale/Performance/Cost) 7:14 Define functional requirements-API 9:19 Define non-functional requirements 10:32 High-level architecture 11:56 Define Data model (what we store/Where we store)
This is hands-down the best system design interview prep channel. I failed many system design rounds until I watched this video, and nailed the next one.
I almost never comment on TH-cam videos, but I just had to let you know how helpful this was. I can't even imagine how much time and effort was put into this. So detailed and explained beautifully with such clarity, thank you!
This is mind blowing. I didn't expected this kind of system design tutorial on TH-cam. Your style to explain HLD and then deep dive into each component, Its awesome and hidden gem. Hats off to you. I know it takes lots of time to create such content but please try to upload more videos on system design.
I think you are the only qualified person I've found to talk on this topic on TH-cam. So many people draw boxes with zero justification, depth or discussion of tradeoffs. Thank you for the tremendous effort
I'm so touched by your words when you said, "If you are still with me watching this video you should be proud of yourself. Seriously. (82:02)" because I felt really proud.
This is the best 1h23min I've ever spent on youtube. You are so detailed, so clear. I am absolutely humbled but your delivery and the depth of consideration you bring. Please keep at it, keep this simple, factual format.
I hope you continue creating content. This is the kind of content that needs to be out there more. It's not something we all learn on the job, in college, or even in our spare time with pet-projects. (Who has the money or time to toy with distributed systems of this complexity and scale as a side-project to learn about them? Practically no one!) Yet, it's something that is part of interviews now very regularly even for people with a few years of experience even if they'll never touch a system like this in their entire career. Interesting to learn about at least - just so dense though. Each little service or idea could be an entire chapter in a textbook itself and you've covered well over 30 or 40 of these in this video.
I wanted to thank Mikhail for the amazing job he does and for the top-quality content he makes on this channel. I just got an offer from Amazon and I owe my success at the system design interview to the content of this channel. Keep up the good work! Спасибо!
I have never designed a system before, but going through this video has made me feel like I am ready to design someone's system. Thank you, Mikhail, this video is well put together.
I cannot thank you enough for putting these system design videos up, they helped me a ton while preparing for my SDE2 interview at Amazon and developing the right mind set for system design . I'd like to share the good news with you that I landed my dream job offer with your help. This channel is indeed the best system design interview prep on youtube. Thank you again !
Always glad to hear stories like this! Thank you, Ethan, for sharing it. It is your hard work that helped you land your dream job. Wish you all the best in your new role at Amazon!
I just found this channel, I don't know why you stopped making videos but you are godsent. I learned more in these 5-6 videos than I have learnt in 100s of pages of books and hours of videos. This is just gold. If you find time please do make more!.
I'm preparing for a Senior TPM interview at Amazon next week, and your videos are hands down the most helpful videos on youtube. Thanks for all your hard work!
I am actually looking for a Lead Design Engineer to build a scalable distributed systems with a minimum server capacity for 1 billion users. Must be willing to travel, manage production team, and be prepared for a fast progression of project integration. If you're not CTO capable, not looking long term big money with Equity potential. Well....it's not a good fit. I need someone who wants to build something amazing. Why... because It's an amazing project. If you know anyone please reply me.
This video is a complete overview book for any system design. It took me more than 5 hours to complete the whole video. Great content, Thank you so much sir for making this video. It is very helpful.
33:20 how to use offsets for fault tolerance if we’re doing in memory batching (if machine crashes while it has some in memory items that would otherwise be lost). After processing a batch of messages, we update our offset in some persistent storage. If we crashed while we have some in memory items not yet flushed to db, then the offset will not be updated and the replacement can re-read from the previous offset 1:00:00 continue from this
By far the best video on system design interview questions, it saved at least 10 hours of effort. Great part is that you can apply these same concepts in a lot of questions , because of the depth that you covered. Liked and Subscribed !!
At 37:33 Internal queue is not necessary for data consistency/correctness. People just love using the same message bus for internal communication too, for simplicity. The only case when such queue before DB is necessary is when the DB can't handle concurrent writes and we are using message queue for write linearization. Though it can be a case, it's not a commons case of systems design. Another case is when Database Writer is stateful and makes some caching/preaggregations/etc in-memory. Then it can use locking or a queue (lock-free) for incoming update linerarization. But usually instead of doing such magic, it's better to leave this stuff for the DB. If your DB can't handle your workloads, you are in troubles anyway (scaling unscalable DB or building caches above it is the same nightmare).
I just wanted to say that I came across this channel by coincidence after looking at many system design channels and your videos are the best. I rarely comment but this channel is a hidden gem and I wanted to thank you Mikhail for the time you put into making these videos.
I've been in the interview loops several times in my 8year software engineering career and as part of interviewing, I've followed multiple different system design interview preparation sources. This video is the best source of information that one needs when preparing for a system design interview. The way the concepts are put in a clear, concise and defined manner is commendable. Really appreciate the hard work put behind creating such an informative video.
This is absolutely the best one and everyone should watch to get a grasp on the System Design interviews. We are very thankful to Mikhail to make this and available to everyone free.
This is awesome! I really miss this video during my preparation to my first interview I had failed. This video really made me believe in myself. Thank you Mikhail, your way of explaining such complicated topic as a system design is the best I have ever seen.
As a beginner, I have to say that this is the best and easy to understand tutorial, watching several times along the learning process helped a lot . Appreciated.
I'm preparing for the incoming interviews, got to say that your system design videos are top among the best youtube can offer. Would love to hear more from you.
Thanks god I found this channel by accident while I was preparing for my upcoming system design interview. Best material one could find on the web, very structured, detailed and covers a lot of topics. I didn't get the offer but system design was my best interview and this video (and others) really helped me to lead conversation and structure my thoughts. I really hope that you would continue making videos because it's the best resource for preparation out there. Thanks a lot!
One brilliant thing about this video is that that the teacher is literally designing another Kafka when talking about partitions piece of the system(the one which is used by partition service). Rather than abstracting that piece by just calling it Kafka, the author is talking about the barebones structure of that piece thereby also teaching us how Kafka itself works(on some level).
God, how wrong I was when dedicated only 2 hours for watching that. I spend couple weeks absorbing and digesting information, and just cannot express my gratitude enough. It is very useful, very well explained!
It is rare to have your attention span > 15 mins, I was skeptical to start the long video. I'd like to personally share with everyone, this is a well-drafted & organized video! The content is atomic, well-stitched and flows seamlessly throughout duration. Keep producing more such content, Kudos!
I watched a number of system design interview prep videos. This one really stands out and I wished I had watched this one earlier. He seems to be a real architect and knows the real stuff, unlike some other fluffy ones. Great, hats off, Mikhail!
You are a superb presenter and COACH. (I am very tough on people who claim to be teachers but can't do it effectively so this is intended as the highest compliment.) Obviously you also know the material widely and deeply, and yet understand how to organize it for people who don't have your skills. Thank you
Truly THE best video on System design. It is so detailed and thorough. It was an amazing learning video. Thank you, Mikhail! Appreciate all your effort in putting this up.
1:01:20 - oh!! the famous serialization cost!! You are getting into all nooks and corners of the system elegantly! Enjoying your video!!! And message schema upgrades - oh no!!!
31:45 The second option is better as it minimizes the interaction between different components and thus reducing system and networking latency. Compared to network/system calls, the time it takes to increment a variable is trivial and can be ignored. This left us 3 network/system calls in the first scenario and 1 network/system calls in the second, thus improving the performance three-folds. In general, we should always consider pack as much data into request as possible without loss of information to minimize the number of interactions. We should also consider asynchronously calling API to further parallelize these operations to improve efficiency.
Hi! First of all, do not be discouraged that the interview didn't go as expected. Interview is a lottery. Preparation may help to increase our chances a lot. But never to 100%. Keep pushing and don't give up! Second, thank you for the feedback! Let me know if you have any questions. I am here to help.
By far the best video on System design i have seen on TH-cam till now , thank you sir for sharing such valuable information in an organized and detailed manner.
@@SystemDesignInterview Please make more videos, like the different types of database and advantage and disadvantage of them. The content is really great !
This is one of the most comprehensive work done ever to explain system design fundamentals. Thanks a lot and I really wish that you keep posting such wonderful videos.
This is GOLD. Please release more materials even if you want to monetize it or add more subscribers. Its even better than what is there in educative. Requesting Mikhail to post system design for applications like Twitter, Facebook Feed, Ticket booking, Uber.Lyft backend etc . Thank you Mikhail
This video explains *so many terms and concepts*. I wish this video existed when I first joined AWS Kinesis in 2015, because I kept hearing all these terms and not understanding them. Such a tremendous resource, thank you for this monumental effort!
Spent hours learning this video😃 The knowledge is explained very clearly and the content is perfectly designed. Thanks so much for the wonderful tutorials!
32:00 , why second approach ? It will reduce number of database writes, so less database calls is better for scalability. Counting in memory is much faster. It will write to database periodically.
I worked in support for 12 years and trying to get into SRE. Your System design videos are LIT. Please continue the good work. I already learned quite a bit from this video. The technique is neat.
My goodness! I am watching this video while taking notes! I am totally blown away with quality content it has. I am watching, pausing, rewinding and taking notes since last 5 hours and I have reached only till 41:15 . It's such an epic feeling of getting to learn all these concepts with a real life problem :) Thanks a ton Mikhail, I can't thank you enough :) Thank you so much!
😱 This is literally a shock-content. Awesome, Brilliant, Many thanks, Mikhail! Please keep making videos, I can imagine how much time and effort you spent on this, but still, it was a pleasure watching all the videos from the first second! Ps. appreciate you little jokes you put in those videos :)
Much appreciate this upload. I think I speak for anyone who's watched this video fully that this is a thoroughly informative and practical video covering the salient spectrum of a complex topic. Very well done and looking forward to more videos or resources. Many thanks!
@42:11 - I liked that part where the presenter says to take a 15 minutes break. Grab yourself a cup of coffee and come back It showed how practical the presenter is. He knew its bad to dump everything at once
This video is extremely good. The other 5-6 videos are good as well, but the structure and breakdown of how to approach each section of the problems is crucial in learning and being able to present your case. It makes me realize that I know nothing compared to what you could and should know concerning different technologies.
Thank you Mikhail ! I went through these high quality videos multiple times and these enabled me to clear multiple system design rounds including a dream offer from a faanG company. Thanks again.
For the reason the second approach is better at 31:50, it is because by batching the calls together the database has less writes than the first method. This leads to less cost. Another reason is that the database will spend less time actually writing, thus in some databases it will actually make the database usage go down as well.
@@SystemDesignInterview Just a thought on choice of data store. One idea is to store all video related metadata and its associations with channels etc in a relational store like mysql, however since counting is a specific problem and other metadata is not needed, we may just use a simple key-value store like dynamo for storing videoId, timestamp(normalized to minute/hour depending on problem statement), view count . Here videoId is the partition key and timestamp is the sort key.Last 1 hour stats are obtained very fast as there is an index on sort key. Thoughts?
Also I prefer the in memory aggregation route. Reason is it 1) Fast 2) Decreases network latencies incurred during database writes for every event 3) Decreases disk writes incurred during database writes.
Hi Tej. Yes, we can make it work with DynamoDB as well. Instead of creating columns, as in Cassandra case, we may need to create tables for each time period (e.g. hour, day). And store information for all videos for that day in this table.
the Best system design guide out there, the perfect video stressing on the way to think, rather than what to do, the recaps were aptly timed too. I owe you big time.
As usual my most sincere gratitude for your interest in sharing your knowledge in such a crisp way. Just the first time I am watching this video, let's see how many times I end up watching this video :)
Even a video like this is getting 27 dislikes ? From competing SD channels ? Dude awesome work. You are really helping me aggregate information in a cohesive manner.
If it is possible to like this video 100 times, I would have done that.. Very useful for any one who is learning system design concepts for interview. Thanks a lot brother.
This was a greta video. Keep up the good work! One small thing that might come up during the interview is how can we create unique IDs for video, channel, like, comment etc. Think about how will you assign IDs: 1.If we use the generated id to do consistent hashing, we need to generate this id first and then goto a particular node We can use Key Generation Service: already created ids in the DB for next several years. Some keys can be stored in the service in the service in memory and can be retrieved from the service host directly. We don't care even if we lose some keys as we have a lot ^_^. The caveat here is that these might be random IDs and we cannot expect any sort of ordering from them and we might have to store an explicit timestamp in the DB. 2. We can use logical shards mapped to a few nodes, do round robin and generate the IDs from them: a. Time in milliseconds 41 bits: for next 20 years in millisecond b. Unique per Node: 13 bits: we can have around 8000 logical node c. Node Sequencer: 10 bits: 1024 seq per node per millisecond Total size: 64 bits Also the IDs stored like this will be sorted according to time per node and we can directly leverage this when we fetch the data from the DB Caveat here is that the IDs here are not strictly sorted, they are K sorted I.e. we expect 'K' IDs to be out of sync per second.
So you created a channel, made 6 videos, showed us how a system design interview video should really be like, then dropped the mic and left :-) I am really sad that there aren't any more videos, but at least I hope the reason is that you are in a very happy place, and you don't have time anymore, because you enjoy your life :-) In any case, thanks a lot for the videos that you did make! There are other decent videos on system design, but none matches the ones on your channel
He has a course. Has some more detailed content
@@arun_kannan what's the name? I'd love to look it up
@@arun_kannan I would also like to know what it is 🙂
ok, course bought 🙂link is now in the channel description, from what I noticed
@@avraamtolmidis5188how is it? Is the 1st module good?
I'm still waiting for the 2nd module before I buy it. He mentioned on Leetcode that he's 60% done with the 2nd module.
I recently interviewed at 2 top companies and I just nailed the system design interviews. In total, I had to go through 4 such 45 min interviews. I have around 10 years of engineering experience but my background is not strictly distributed systems. I am good at DS/algo but system design was my achilles' heel. Mikhail made it simpler for me to catch up. I did read a lot of books to understand how systems like Kafka, Zookeeper, Cassandra etc. are implemented. I had lots of bits and pieces of information from lots of sources. This channel helped me put all that understanding together in a coherent way. I also followed the structure described by Mikhail in the interviews and it helped me keep the discussion on point. The interviews were difficult, interviewers asked tough questions but listening to these videos multiple times helped me calm my nerves.
I haven't heard back from those companies yet but initial signs look positive. We will see how it pans out. Regardless of the result, I feel satisfied with my preparation and a big chunk of credit goes to Mikhail.
Why am I writing such a big comment here? Well, I want to obviously thank Mikhail but I want people to realize how valuable this channel can be for interview preparation. Please try to make it more popular by sharing it with your friends.
Hi primespace! Did I pay you to write this post? ))
To be serious, I am really glad for you. To know that these videos helped you both sharpen your knowledge and give you a boost in confidence, makes me feel happy.
Regardless of the result, you should be proud of yourself. Watching these videos thoroughly multiple times and reading through a lot of resources is a hard work. I know it. So, give yourself a credit.
If you feel inclined, please let us know the final result of your interviews.
@@SystemDesignInterview Hahaha! No money was involved and it is coming straight from heart :). It should be the other way around. People should be paying you for this excellent content. I compared it with paid content like "Grokking the system design interview" and your content is far more practical and miles ahead.
Yes, I will share the outcome once I hear back from them.
One unique pattern I observed in these interviews was that the interviewers forced me to design a system which does not need to scale and will run only on one machine. They wanted to discuss the threading model, how would I use mutexes and locks etc. It almost seemed like they wanted me to keep things very simple and instead of saying big things, come up with simple solutions like - I will use SQLite because requirements are simple. Just something to watch out for. People shouldn't fall in the trap of assuming the interviewer is asking to design a highly scalable system. I almost fell for it but corrected myself in one interview. I said things as simple as I'd simply store the data in XML file if it is only 100 users and the interviewer was very happy to hear my response. Once we spent 15 minutes on very simple stuff then only they moved to scaling aspects. Surprisingly they drilled me on things like how DNS works, how TCP handshake happens and what is SSL etc. These questions were asked at random times while I was talking about something unrelated. In any case, these were easy enough to answer so all good.
@@SystemDesignInterview The results came back. Yes from both. Still has to match teams and officially get offers but so far so good. Thank you again, Mikhail! It would not have been possible without your help.
Well done, primespace, well done.
@@abby297in thank you for sharing your experience!! did you actually know the answers for those networking layer-specific questions and do you think familiarity with the OSI model would be useful to study at all?
Continued from someone's comment :
1:30 Why you need to ask your interviewer questions? Why requirements clarification is so important?
3:19 What we may want to ask about? Requirements clarification (User/Scale/Performance/Cost)
7:14 Define functional requirements-API
9:19 Define non-functional requirements
10:32 High-level architecture
11:56 Define Data model (what we store)
16:09 Where we store: List of non-functional requirements
17:20 Where we store: How SQL database handle these requirements
20:40 Where to store: About NoSql: Cassandra
25:00 How we store: data modelling for sql v nosql
29:03 Data processing path
30:51 Data processing: Data aggregation basics
34:42 Data procesing : Processing service detailed design
41:08 Data Ingestion pipeline
1:03:00 Data Retrieval path
1:06:35 Data flow simulation
1:08:26 Technology stack
1:13:22 Bottleneck, tradeoffs and more
1:09:06 Summary
Kudos to the creator. Happy Learning.
Can this comment be pinned above?
1:19:07 Summary
don't look at it as a 83 min video, it contains enough contents to use a week to digest it :) it is absolutely fantastic, comprehensive, understandable, and maybe the best video in System Design Interview on TH-cam and anywhere else. Thanks for all the efforts to providing these contents.
By far the only person who talks about system design after mastering it in youtube!
This is pure gold. I have slept several times, woke up, rewinded several times, made notes, still watching. You are a legend. Hats off to you, you spent 1.23 hours talking non stop on a serious and important topic, covering everything. You explained several topics in so simple language which was hard to grasp. One was data consistency and availability and other was Eventual consistency and many more. Thanks once again. May God Bless you.
Hey man, please continue releasing videos. It's the best material I've seen so far.
The quality and clarity is amazing
@@dhawang.gayash4080 popppoppopppppooppppppoipooiipipioqqqqqqqqqqqqq
Unfortunately no new content.
It took me a few days to get through this video. As a software engineer with 8 years of experience, you have very clearly demonstrated to me that I know nothing.
Thanks
D:
Cram time...
1:30 Why you need to ask your interviewer questions? Why requirements clarification is so important?
3:19 What we may want to ask about? Requirements clarification (User/Scale/Performance/Cost)
7:14 Define functional requirements-API
9:19 Define non-functional requirements
10:32 High-level architecture
11:56 Define Data model (what we store/Where we store)
after 12 minute you closed video
@@moscowhits3227 lol
How can give up after 12min? Later on is the real hard part
Just watch the video, this is pure gold
Mikhail, you should win a TH-cam Creator award for this channel. The content is just so exceptional.
This is hands-down the best system design interview prep channel. I failed many system design rounds until I watched this video, and nailed the next one.
I almost never comment on TH-cam videos, but I just had to let you know how helpful this was. I can't even imagine how much time and effort was put into this. So detailed and explained beautifully with such clarity, thank you!
Best System Design Interview Preps. Builds up the system slowly and explains in breadth and depth. I would pay for this
This is mind blowing. I didn't expected this kind of system design tutorial on TH-cam. Your style to explain HLD and then deep dive into each component, Its awesome and hidden gem. Hats off to you. I know it takes lots of time to create such content but please try to upload more videos on system design.
Appreciate the feedback, Dheeraj! Thank you.
exactly
I think you are the only qualified person I've found to talk on this topic on TH-cam. So many people draw boxes with zero justification, depth or discussion of tradeoffs. Thank you for the tremendous effort
Yesterday I gave my first FAANG interview for Amazon, I watched all of your videos "at least" twice. It was a great help. THANK YOU!
Hope it went well! Wish you all the luck at the interviews, Komal!
The best channel for system design interview questions. 🔥
Glad you like the channel, Ashish. Thank you for the feedback.
I'm so touched by your words when you said, "If you are still with me watching this video you should be proud of yourself. Seriously. (82:02)" because I felt really proud.
Thank you so much, Mikhail
This is the best 1h23min I've ever spent on youtube. You are so detailed, so clear. I am absolutely humbled but your delivery and the depth of consideration you bring. Please keep at it, keep this simple, factual format.
I hope you continue creating content. This is the kind of content that needs to be out there more. It's not something we all learn on the job, in college, or even in our spare time with pet-projects. (Who has the money or time to toy with distributed systems of this complexity and scale as a side-project to learn about them? Practically no one!) Yet, it's something that is part of interviews now very regularly even for people with a few years of experience even if they'll never touch a system like this in their entire career. Interesting to learn about at least - just so dense though. Each little service or idea could be an entire chapter in a textbook itself and you've covered well over 30 or 40 of these in this video.
I wanted to thank Mikhail for the amazing job he does and for the top-quality content he makes on this channel. I just got an offer from Amazon and I owe my success at the system design interview to the content of this channel. Keep up the good work!
Спасибо!
Congrats, Maxim! Happy to hear that!
I have never designed a system before, but going through this video has made me feel like I am ready to design someone's system.
Thank you, Mikhail, this video is well put together.
Finally, I have completed the video, two days. I enjoyed every single piece of information. Thank you so much.
I cannot thank you enough for putting these system design videos up, they helped me a ton while preparing for my SDE2 interview at Amazon and developing the right mind set for system design . I'd like to share the good news with you that I landed my dream job offer with your help. This channel is indeed the best system design interview prep on youtube. Thank you again !
Always glad to hear stories like this! Thank you, Ethan, for sharing it.
It is your hard work that helped you land your dream job. Wish you all the best in your new role at Amazon!
Hey man, do you have any tips for gaining the technical expertise to be a sde2? (Other than leetcode and system design)
I just found this channel, I don't know why you stopped making videos but you are godsent. I learned more in these 5-6 videos than I have learnt in 100s of pages of books and hours of videos. This is just gold. If you find time please do make more!.
I'm preparing for a Senior TPM interview at Amazon next week, and your videos are hands down the most helpful videos on youtube. Thanks for all your hard work!
Hi Sean! Appreciate the feedback! And wish you all the luck on your interview!
Master of System Design is back.
Missed you all as well! ))
I am actually looking for a Lead Design Engineer to build a scalable distributed systems with a minimum server capacity for 1 billion users.
Must be willing to travel, manage production team, and be prepared for a fast progression of project integration.
If you're not CTO capable, not looking long term big money with Equity potential. Well....it's not a good fit.
I need someone who wants to build something amazing.
Why... because It's an amazing project.
If you know anyone please reply me.
@@CestuiQueTrustBeneficiary-KING I am interested. My email id is prgupta8891@gmail.com
This video is a complete overview book for any system design. It took me more than 5 hours to complete the whole video. Great content, Thank you so much sir for making this video. It is very helpful.
Thank you, Ankit, for the feedback! Really appreciate.
Each word he says is gold mine and every sentence deserves to be seperate video for other youtubers to explain.
No kidding, I pause to write down the gold nuggets of information. lol
Thanks to you I landed a job in Manga. Words can't express my gratitude. We miss you!
Took me 5 hours to take notes and get through this video, but it was worth it. Literally every second is important
33:20 how to use offsets for fault tolerance if we’re doing in memory batching (if machine crashes while it has some in memory items that would otherwise be lost). After processing a batch of messages, we update our offset in some persistent storage. If we crashed while we have some in memory items not yet flushed to db, then the offset will not be updated and the replacement can re-read from the previous offset
1:00:00 continue from this
I've learnt hours of System Design videos and I have to say, this guy is best of the all! Clear, very clear! Please keep up good work!
Thank you, yrs207, for the feedback! Glad you liked the video!
Just got my offer from FB. Thanks for your videos!
This is great news, yrs207! Glad for you!
This channel is the best of all available on TH-cam for System Design.
Glad to hear back from you, Algorithm Implementer! Have not seen you for a while.
This is the single best video on system design on youtube thus far. Please keep making more!
Appreciate your feedback, Santosh.
By far the best video on system design interview questions, it saved at least 10 hours of effort. Great part is that you can apply these same concepts in a lot of questions , because of the depth that you covered. Liked and Subscribed !!
Glad to be helpful!
At 37:33 Internal queue is not necessary for data consistency/correctness. People just love using the same message bus for internal communication too, for simplicity. The only case when such queue before DB is necessary is when the DB can't handle concurrent writes and we are using message queue for write linearization. Though it can be a case, it's not a commons case of systems design.
Another case is when Database Writer is stateful and makes some caching/preaggregations/etc in-memory. Then it can use locking or a queue (lock-free) for incoming update linerarization. But usually instead of doing such magic, it's better to leave this stuff for the DB. If your DB can't handle your workloads, you are in troubles anyway (scaling unscalable DB or building caches above it is the same nightmare).
Watching the first 6 mins of the vid took me 30 mins. This is Gold.
I just wanted to say that I came across this channel by coincidence after looking at many system design channels and your videos are the best.
I rarely comment but this channel is a hidden gem and I wanted to thank you Mikhail for the time you put into making these videos.
Thank you, Adel Ali, for all the kind words! Appreciate the feedback!
You are a life saver 🙌 you made me so confident in my interview. I got asked a question that was here and I attacked it out of instinct.
I've been in the interview loops several times in my 8year software engineering career and as part of interviewing, I've followed multiple different system design interview preparation sources. This video is the best source of information that one needs when preparing for a system design interview. The way the concepts are put in a clear, concise and defined manner is commendable. Really appreciate the hard work put behind creating such an informative video.
This is absolutely the best one and everyone should watch to get a grasp on the System Design interviews. We are very thankful to Mikhail to make this and available to everyone free.
Glad you liked it, thunrou. Thank you for the feedback!
This is awesome!
I really miss this video during my preparation to my first interview I had failed.
This video really made me believe in myself. Thank you Mikhail, your way of explaining such complicated topic as a system design is the best I have ever seen.
As a beginner, I have to say that this is the best and easy to understand tutorial, watching several times along the learning process helped a lot . Appreciated.
seriously we need to be united and force him to release more videos, this work is precious then gold
Hands down.. one of the best videos on system design I came across so far..
I'm preparing for the incoming interviews, got to say that your system design videos are top among the best youtube can offer. Would love to hear more from you.
The best video on the system design I have seen so far. Thank you for your effort on making such a detailed and structured video!
Thanks god I found this channel by accident while I was preparing for my upcoming system design interview. Best material one could find on the web, very structured, detailed and covers a lot of topics. I didn't get the offer but system design was my best interview and this video (and others) really helped me to lead conversation and structure my thoughts. I really hope that you would continue making videos because it's the best resource for preparation out there. Thanks a lot!
We miss you! Still watch you video occasionally to refresh the memory. Please produce more videos!
One brilliant thing about this video is that that the teacher is literally designing another Kafka when talking about partitions piece of the system(the one which is used by partition service). Rather than abstracting that piece by just calling it Kafka, the author is talking about the barebones structure of that piece thereby also teaching us how Kafka itself works(on some level).
God, how wrong I was when dedicated only 2 hours for watching that. I spend couple weeks absorbing and digesting information, and just cannot express my gratitude enough. It is very useful, very well explained!
It is rare to have your attention span > 15 mins, I was skeptical to start the long video. I'd like to personally share with everyone, this is a well-drafted & organized video! The content is atomic, well-stitched and flows seamlessly throughout duration. Keep producing more such content, Kudos!
I watched a number of system design interview prep videos. This one really stands out and I wished I had watched this one earlier. He seems to be a real architect and knows the real stuff, unlike some other fluffy ones. Great, hats off, Mikhail!
Man this is awesome channel !!
Why did you stop ? 😭
He finally passed the interview
Yes I think there is definitely a niche for system design content
@@winrid If he can't pass a system design interview, we are all doomed.
You are a superb presenter and COACH. (I am very tough on people who claim to be teachers but can't do it effectively so this is intended as the highest compliment.)
Obviously you also know the material widely and deeply, and yet understand how to organize it for people who don't have your skills.
Thank you
Truly THE best video on System design. It is so detailed and thorough. It was an amazing learning video. Thank you, Mikhail! Appreciate all your effort in putting this up.
1:01:20 - oh!! the famous serialization cost!! You are getting into all nooks and corners of the system elegantly! Enjoying your video!!!
And message schema upgrades - oh no!!!
No one deep dives like Mikhail. Best system design videos on YT.
Truly one of the best videos I have come across for System Design concepts and approach. Please continue posting videos of such quality.
Thank you, Ashish!
Best system design interview channel on TH-cam
He did System Design of System Design Interview👏
31:45 The second option is better as it minimizes the interaction between different components and thus reducing system and networking latency. Compared to network/system calls, the time it takes to increment a variable is trivial and can be ignored. This left us 3 network/system calls in the first scenario and 1 network/system calls in the second, thus improving the performance three-folds.
In general, we should always consider pack as much data into request as possible without loss of information to minimize the number of interactions. We should also consider asynchronously calling API to further parallelize these operations to improve efficiency.
You are back! Thank you very much! Just failed system design on facebook. I hope this video helps me with the next interview! You are the best!
Hi!
First of all, do not be discouraged that the interview didn't go as expected. Interview is a lottery. Preparation may help to increase our chances a lot. But never to 100%. Keep pushing and don't give up!
Second, thank you for the feedback! Let me know if you have any questions. I am here to help.
By far the best video on System design i have seen on TH-cam till now , thank you sir for sharing such valuable information in an organized and detailed manner.
Thank you, Pranov, for the feedback! Glad you liked the video!
@@SystemDesignInterview Please make more videos, like the different types of database and advantage and disadvantage of them. The content is really great !
A top notch system design video.
I fail to understand why this channel is so underRated.
I have been waiting for almost 6 months. Finally you are back. Definitely the best system design video I have watched so far.
Hi ichbinas. Yes, this video took me a while to prepare. Glad you liked it!
This is one of the best videos to start with system design basics.
The best system design interview channel on TH-cam!
You are setting the bar high here, Q )) But I will try to keep it high.
Thank you for the feedback! Much appreciated!
After reading so many youtubers for sys design, this is THE best one.
For some reason this is the best system design course I have ever seen
For some reason I believe I have the most grateful viewers on TH-cam. Thank you!
This is one of the most comprehensive work done ever to explain system design fundamentals. Thanks a lot and I really wish that you keep posting such wonderful videos.
This is GOLD. Please release more materials even if you want to monetize it or add more subscribers. Its even better than what is there in educative. Requesting Mikhail to post system design for applications like Twitter, Facebook Feed, Ticket booking, Uber.Lyft backend etc . Thank you Mikhail
This video explains *so many terms and concepts*. I wish this video existed when I first joined AWS Kinesis in 2015, because I kept hearing all these terms and not understanding them. Such a tremendous resource, thank you for this monumental effort!
Thanks! Even though I'm a new grad I have a system design interview to prepare for.. this video is very helpful!
The best video I've seen so far about the system design question approach! Watch over and over again. Thank you from your fan :-)
You are back, Memfis! Glad to see you. And thank you for the feedback!
Spent hours learning this video😃 The knowledge is explained very clearly and the content is perfectly designed. Thanks so much for the wonderful tutorials!
this is the best channel about system design. i definitly willing to pay for paid content if there have one.
Thank you for the kind words, paper_plane!
32:00 , why second approach ? It will reduce number of database writes, so less database calls is better for scalability. Counting in memory is much faster. It will write to database periodically.
I worked in support for 12 years and trying to get into SRE. Your System design videos are LIT. Please continue the good work. I already learned quite a bit from this video. The technique is neat.
The best video in System Design I have seen so far!
My goodness! I am watching this video while taking notes! I am totally blown away with quality content it has. I am watching, pausing, rewinding and taking notes since last 5 hours and I have reached only till 41:15 . It's such an epic feeling of getting to learn all these concepts with a real life problem :) Thanks a ton Mikhail, I can't thank you enough :) Thank you so much!
Hey @Ramendu, can you share your notes if possible ? :)
OMG..where was this video. Such a valuable information given in a crisp and clear manner. Thank for creating this video.
Easily the best system design interview video I have seen.
Hi Aleksey. Thank you for the feedback!
Do we need to have prior knowledge of System design?
Just 17 mins in the video, I have made 40 lines of notes. This is awesome.
That's the spirit! ))
Hi Karthik, did you complete the notes ? Can you share them if possible?
@@avikaran23 when you write your own, more chances you will learn the material better.
Can you please share your notes ?
😱 This is literally a shock-content.
Awesome, Brilliant, Many thanks, Mikhail! Please keep making videos, I can imagine how much time and effort you spent on this, but still, it was a pleasure watching all the videos from the first second!
Ps. appreciate you little jokes you put in those videos :)
I have a System Design Interview tomorrow afternoon - Im cramming using this course. Definitely will help a lot. Thanks!
Much appreciate this upload. I think I speak for anyone who's watched this video fully that this is a thoroughly informative and practical video covering the salient spectrum of a complex topic.
Very well done and looking forward to more videos or resources. Many thanks!
@42:11 - I liked that part where the presenter says to take a 15 minutes break. Grab yourself a cup of coffee and come back
It showed how practical the presenter is. He knew its bad to dump everything at once
This video is extremely good. The other 5-6 videos are good as well, but the structure and breakdown of how to approach each section of the problems is crucial in learning and being able to present your case.
It makes me realize that I know nothing compared to what you could and should know concerning different technologies.
Hi Kevin. Thank you a lot for the feedback! Glad you liked the videos.
Regarding knowledge, it will come, just give it some time.
Thank you Mikhail ! I went through these high quality videos multiple times and these enabled me to clear multiple system design rounds including a dream offer from a faanG company. Thanks again.
For the reason the second approach is better at 31:50, it is because by batching the calls together the database has less writes than the first method. This leads to less cost. Another reason is that the database will spend less time actually writing, thus in some databases it will actually make the database usage go down as well.
I got offers from Amazon and Uber with the help of this video. Thanks Mikhail!
Incredible ! The breakdown of what to focus, how to drive discussions, tradeoffs of choices, generic template etc is very impressive! Thanks a lot!
Thank you, Tej, for the feedback!
@@SystemDesignInterview Just a thought on choice of data store. One idea is to store all video related metadata and its associations with channels etc in a relational store like mysql, however since counting is a specific problem and other metadata is not needed, we may just use a simple key-value store like dynamo for storing videoId, timestamp(normalized to minute/hour depending on problem statement), view count . Here videoId is the partition key and timestamp is the sort key.Last 1 hour stats are obtained very fast as there is an index on sort key. Thoughts?
Also I prefer the in memory aggregation route. Reason is it 1) Fast 2) Decreases network latencies incurred during database writes for every event 3) Decreases disk writes incurred during database writes.
Hi Tej. Yes, we can make it work with DynamoDB as well. Instead of creating columns, as in Cassandra case, we may need to create tables for each time period (e.g. hour, day). And store information for all videos for that day in this table.
Those are good points! I would also add cost to this list. Calling database per event is very costly.
the Best system design guide out there, the perfect video stressing on the way to think, rather than what to do, the recaps were aptly timed too. I owe you big time.
Glad to know that, Aravamuthan! Thank you for the feedback.
Best system design channel I've ever seen on youtube, thanks so much, I'm so excited that I fount it. Great work!
One of the best (if not the best) video for System Design Preparation. Thank you so much!
Thank you, Chrisy, for the feedback!
@@SystemDesignInterview There's no doubt that this is the best resource on system design.
As usual my most sincere gratitude for your interest in sharing your knowledge in such a crisp way. Just the first time I am watching this video, let's see how many times I end up watching this video :)
My sincere gratitude for your, Wabi Sabi, for giving me feedback consistently! You are great! Glad I can help.
Even a video like this is getting 27 dislikes ? From competing SD channels ?
Dude awesome work. You are really helping me aggregate information in a cohesive manner.
If it is possible to like this video 100 times, I would have done that.. Very useful for any one who is learning system design concepts for interview. Thanks a lot brother.
This was a greta video. Keep up the good work!
One small thing that might come up during the interview is how can we create unique IDs for video, channel, like, comment etc.
Think about how will you assign IDs:
1.If we use the generated id to do consistent hashing, we need to generate this id first and then goto a particular node
We can use Key Generation Service: already created ids in the DB for next several years. Some keys can be stored in the service in the service in memory and can be retrieved from the service host directly. We don't care even if we lose some keys as we have a lot ^_^.
The caveat here is that these might be random IDs and we cannot expect any sort of ordering from them and we might have to store an explicit timestamp in the DB.
2. We can use logical shards mapped to a few nodes, do round robin and generate the IDs from them:
a. Time in milliseconds
41 bits: for next 20 years in millisecond
b. Unique per Node:
13 bits: we can have around 8000 logical node
c. Node Sequencer:
10 bits: 1024 seq per node per millisecond
Total size: 64 bits
Also the IDs stored like this will be sorted according to time per node and we can directly leverage this when we fetch the data from the DB
Caveat here is that the IDs here are not strictly sorted, they are K sorted I.e. we expect 'K' IDs to be out of sync per second.
Thanks for sharing this with all of us, Vishar!
great! but how to generate those unique keys beforehand?