To be honest, I immediately liked the video just by reading the title and without even watching the video. I was like "finally something new and not even new but also really interesting topic". Would like more of those kind of videos in the future, thank you
In fact, the implementation of leader election is needed even without the use of web workers. And many browsers do not have a shared worker. You can use a broadcast channel, "active tab" events, and local storage to elect a leader. And make sure that all requests to the server come only from the leading tab. Especially in the case of sockets, this reduces the load on the server hundreds of times -- people have a habit of not closing tabs -- in collaborative web application. And sometimes there can be several "read leaders" if you have 2+ active tabs/monitors at the same time.
Great great video! I have stumbled upon the "How Notion Cut Latency" posts on Linkedin multiple times previously, but the textual material always seemed too heavy or contained too much jargon to comprehend easily without losing patience. This video format & someone explaining as we go - is SOOO SOO much better! Please keep doing this. There's a lot of DSA content on TH-cam but almost no content such as this video. This could also be NeetCode's near future niche.
Graphics make all the complex concepts you cover super approachable. More 20-minute videos please! This was excellent. You covered all the important points concisely and only cut out details that can be found in the newsletter. I would suggest a TLDR of some kind at the start.
The cause of your website performance problems is database latency. Over the last 20 years, this has consistently been true for every system I've had to maintain. It's extra funny when they have gone to the effort to rice, using whatever meme language everyone is claiming to be fastest. Reducing the query count has taken operations from minutes to microseconds, and I keep having this experience every time I change teams.
Checks out to me. No matter how much flashy a software piece is, you will be limited by your hardware, and something that strictly enters disk manipulation for read & write (aka the DB) makes sense that will drastically slow your app
Feedbacks: 1- pictures are really good. I think you should keep it. It really helped visually internalizing some of concepts or design approaches. 2- The time was all right. For more complicated videos it makes sense to go longer. 3- Tying the content of the current video with entry level/beginner stuff (for eg ACID compliance at the end) is really good. That helped unveiling more of the complexity of what is happening. 4- I was originally not happy with the flow of the video (moving from subparts of the videos) but after reflecting I think it was fine Thank you for going that deep in the details of what exactly is happening and what is going wrong. That is what sets your system design video apart from the other content on TH-cam. Excited about where this is going.
I love the way you explain. Definitely keep the visuals. Everytime i think wth he is talking about you immediately goes to basics of it and i really like this aspect of your explaining complex problems/designs. Thank you for providing such value on TH-cam.
I watched the whole video, I think the length is appropriate for the depth, thank you! I'd love to know how Notion does, or how you would, implement the sqlite part.
I usually dont comment on the videos, but this made me comment because of your effort you put in the video creating it! Amazing video dude! I really liked the flow of the video and everything else.
This was super helpful and incredible video. I love learning about these kinds of stories since it mimics what engineers actually do on the job. It's also something I've wondered a lot myself!
One thing to note is this only improves the write queuing performance (which is fine as it's the bottleneck) as sqlite does not support concurrent writers as stated in the "when to use" docs: "SQLite only supports one writer at a time per database file. But in most cases, a write transaction only takes milliseconds and so multiple writers can simply take turns. SQLite will handle more write concurrency than many people suspect. Nevertheless, client/server database systems, because they have a long-running server process at hand to coordinate access, can usually handle far more write concurrency than SQLite ever will."
You are f$$king fantastic. The depth vs breadth covered is amazingly on point. You point out the gotchas (e.g. SQLite is ACID compliant - so how come...?) which a lot of videos don't (creating a 'perfect system where everything just works' illusion). Your scribbling is even nice - does highlight and help my brain hold on to key concepts while it's dynamic enough that I don't lose focus. Overall, just keep killing it, man
great content i loved it but by the way at 11:35 i wanna say that basically threads share the same memory but processes don't , this is why processes they have to communicate through channels
ah good catch, in this case i believe while web workers run in a separate thread, from a programming perspective to me it still feels like they communicate with the main js thread the way different processes would communicate with each other. this is such a subtle point tho so someone please correct me if im wrong
I loved all the visuals and the explanations. I would prefer if these videos were 15 to 20 mins and not longer. The blog post can provide additional details. Also, this was one the best explanations I have seen to date.
the browser file API being the problem reminds of a talk by a database creator. He said that many people assume that if you write bytes to disk, and then immediately read them back, you get the same thing. Turns out that when you are creating a database, that's not "true enough" - from time to time the SSD or HDD gets corrupted, and you have take that into accound.
Very cool video. Liked those hand-on explanation. The way you actually showed application tab, it's memory usage and the js file with log statements. Pls continue your good work. Looking forward to learn more from you!
Naive Question: does Notion solve for race between tabs? I.e. let’s say you have two tabs open on the same task but they are in two different states. one is newer - let’s say 2 paragraphs in the task description One with staler data. Focus is on the tab with the new data, and now you move to the old tab, add one char to your stale description and navigate to a different task. Will you lose your paragraphs ? (I don’t use notion… but intrigued)
Loved the visuals loved the explanation and drawings. For the video, if anything I would love more depth. I feel that's what's mainly missing in all of TH-cam. Totally in for much longer videos or series where you fully cover a concept in high level and actual code/tech level. Hope to see that! For the newsletter I want to read it, but wonder can we only read it in the emails ? nowhere else? Just like to read in light mode or reading mode of some browsers if possible.
Awesome awesome video! Thank you for taking the time to draw and give a great explanation of their system, I found it really helped and connected all the pieces together! You are the GOAT. I love these types of videos. Keep up the great work!
Liked the diagram pictures used & the length of the video was nice, I learned that this was possible from this video & knowing the pitfalls like the OPFS causing data corruption will save a lot of headaches I imagine.
Awesome !!! The details, the code and yes the pictures - very very useful ... I have also signed up for your newsletter. Your leetcode videos are gold standard, and so was this.
Awsome video, awsome topic as well. I wanted to learn more about using sqlite for caching, particularly now with WASM tools like the sqlite-vec extension for vector search!! This was great! Visuals were great!
the visuals are good. obviously could always be better but i dont have further feedback on how. for the vid length could definitely cut it a bit. Thanks as always
Everything is good 📈😊, honestly It makes me stick to the video to the end. You should add the visuals like these to persuade the System design and very informative video BTW, I hope like me everyone expect this like understandable and knowledgeable video from you in the future . Thank you.
Dunno about cutting latency, but they did a lot if shite to make notion slow right after initial release. I remember how at first I was pretty happy how it was snappy as hell, but when our teams started to fill more documents it become slower. And then they've released a new update with new features and everything turned into a turtle - everything was slow as hell. Including simple selection and opening context menu.
This was extremely awesome and kept my attention intact and curiosity raised, drawing and images way of explanation is better (atleast for e) than rambling while showing up bunch of text. Thankyou so much. Btw might be a naive question, how do you get to know about news as such ? what are your sources?
Hi nav, I really enjoied a lot your video. I'm mind of tech lead in the places I work. We are doing aí stuff. And it's very good to me to see how others experts are solving their problems
Keep doing these videos, i learned a lot from this. Also this just made me realize than as a self taught sometimes you can invent your own solutions to overcome programming problems like workarounds same as what notion did with race requests etc
Great video man! If you do get time make more videos on these concepts, atleast for things in you newsletter that you fell are video worthy! Again, great video man!
Loved the video, I like how you explained system design Often with bytebytego (which I do like) I do feel a little stumped at times Also if I understand correctly we are literally having a database on the client side?!? That seems wild, but why didn't they just use indexedDB
I actually prefer Informative videos and 'Hour-long-videos' actually doesn't matter to me yk, I'd like if you produce such contents as I really feel your ability to explain is pretty neat. Oh and also, really nice and informative visuals I really appreciate it.
Pictures and diagrams are a requirement. Rather than creating your own you can use some icon library and grab what you need. Boring standards is better than flashy when learning, it's more important that everyone is on the same page.
Honestly, had a lot of fun making this one. How'd you like it and do you wanna see more?
Written version: neetcode.io/newsletter
Finally what we have all been waiting for. Neetdesign 🎉
Sure NeetCode. I really like the way you explain everything :)
yes , watched the video twice. its pure Gold. Need more and more videos.
For sure!
Yep really like this style of content, primeagen and t3gg theo are some of my favourite types specially as a beginner
To be honest, I immediately liked the video just by reading the title and without even watching the video. I was like "finally something new and not even new but also really interesting topic". Would like more of those kind of videos in the future, thank you
I love it.. the format, the video length, the topics coverage and of course the hands-on explanation. Looking forward to this series.
same here
In fact, the implementation of leader election is needed even without the use of web workers. And many browsers do not have a shared worker. You can use a broadcast channel, "active tab" events, and local storage to elect a leader. And make sure that all requests to the server come only from the leading tab. Especially in the case of sockets, this reduces the load on the server hundreds of times -- people have a habit of not closing tabs -- in collaborative web application. And sometimes there can be several "read leaders" if you have 2+ active tabs/monitors at the same time.
Great great video! I have stumbled upon the "How Notion Cut Latency" posts on Linkedin multiple times previously, but the textual material always seemed too heavy or contained too much jargon to comprehend easily without losing patience. This video format & someone explaining as we go - is SOOO SOO much better! Please keep doing this. There's a lot of DSA content on TH-cam but almost no content such as this video. This could also be NeetCode's near future niche.
Great video. I loved the visuals and 10-20 minutes is the sweet spot for optimal length imo
Agreed, sweet spot that makes you watch it now, instead of saving it for later (and forget about it)
Graphics make all the complex concepts you cover super approachable. More 20-minute videos please!
This was excellent. You covered all the important points concisely and only cut out details that can be found in the newsletter.
I would suggest a TLDR of some kind at the start.
The cause of your website performance problems is database latency. Over the last 20 years, this has consistently been true for every system I've had to maintain. It's extra funny when they have gone to the effort to rice, using whatever meme language everyone is claiming to be fastest. Reducing the query count has taken operations from minutes to microseconds, and I keep having this experience every time I change teams.
Checks out to me. No matter how much flashy a software piece is, you will be limited by your hardware, and something that strictly enters disk manipulation for read & write (aka the DB) makes sense that will drastically slow your app
Feedbacks:
1- pictures are really good. I think you should keep it. It really helped visually internalizing some of concepts or design approaches.
2- The time was all right. For more complicated videos it makes sense to go longer.
3- Tying the content of the current video with entry level/beginner stuff (for eg ACID compliance at the end) is really good. That helped unveiling more of the complexity of what is happening.
4- I was originally not happy with the flow of the video (moving from subparts of the videos) but after reflecting I think it was fine
Thank you for going that deep in the details of what exactly is happening and what is going wrong. That is what sets your system design video apart from the other content on TH-cam. Excited about where this is going.
I love the way you explain. Definitely keep the visuals. Everytime i think wth he is talking about you immediately goes to basics of it and i really like this aspect of your explaining complex problems/designs. Thank you for providing such value on TH-cam.
I watched the whole video, I think the length is appropriate for the depth, thank you!
I'd love to know how Notion does, or how you would, implement the sqlite part.
I usually dont comment on the videos, but this made me comment because of your effort you put in the video creating it! Amazing video dude! I really liked the flow of the video and everything else.
appreciate it 🙏🙏
This was super helpful and incredible video. I love learning about these kinds of stories since it mimics what engineers actually do on the job. It's also something I've wondered a lot myself!
Visuals are great. Lengthy videos are also nice, I'm not here for dopamine, I'm here for education. Keep up the good work man! 🙂
One thing to note is this only improves the write queuing performance (which is fine as it's the bottleneck) as sqlite does not support concurrent writers as stated in the "when to use" docs: "SQLite only supports one writer at a time per database file. But in most cases, a write transaction only takes milliseconds and so multiple writers can simply take turns. SQLite will handle more write concurrency than many people suspect. Nevertheless, client/server database systems, because they have a long-running server process at hand to coordinate access, can usually handle far more write concurrency than SQLite ever will."
Loved this. Please make more of this system design / optimisation type video!
Yes! Visuals are fantastic, especially for breaking down abstract concepts. Love this initiative.
You are f$$king fantastic. The depth vs breadth covered is amazingly on point. You point out the gotchas (e.g. SQLite is ACID compliant - so how come...?) which a lot of videos don't (creating a 'perfect system where everything just works' illusion). Your scribbling is even nice - does highlight and help my brain hold on to key concepts while it's dynamic enough that I don't lose focus. Overall, just keep killing it, man
One interesting thing about this is OPFS. I've never actually used it before.
great content i loved it but by the way at 11:35 i wanna say that basically threads share the same memory but processes don't , this is why processes they have to communicate through channels
ah good catch, in this case i believe while web workers run in a separate thread, from a programming perspective to me it still feels like they communicate with the main js thread the way different processes would communicate with each other.
this is such a subtle point tho so someone please correct me if im wrong
There is so much entry level stuff posted for tech in general its hard to find good sources for Mid + level development.
This is huge.
The visuals were awesome, I made sure to subscribe to the newsletter!
I loved all the visuals and the explanations. I would prefer if these videos were 15 to 20 mins and not longer. The blog post can provide additional details.
Also, this was one the best explanations I have seen to date.
the browser file API being the problem reminds of a talk by a database creator. He said that many people assume that if you write bytes to disk, and then immediately read them back, you get the same thing.
Turns out that when you are creating a database, that's not "true enough" - from time to time the SSD or HDD gets corrupted, and you have take that into accound.
Very cool video. Liked those hand-on explanation. The way you actually showed application tab, it's memory usage and the js file with log statements. Pls continue your good work. Looking forward to learn more from you!
Naive Question: does Notion solve for race between tabs?
I.e. let’s say you have two tabs open on the same task but they are in two different states.
one is newer - let’s say 2 paragraphs in the task description
One with staler data.
Focus is on the tab with the new data, and now you move to the old tab, add one char to your stale description and navigate to a different task.
Will you lose your paragraphs ? (I don’t use notion… but intrigued)
Great video! I admire how you simplify complex concepts, making them easy to understand.
Loved the visuals loved the explanation and drawings.
For the video, if anything I would love more depth. I feel that's what's mainly missing in all of TH-cam. Totally in for much longer videos or series where you fully cover a concept in high level and actual code/tech level. Hope to see that!
For the newsletter I want to read it, but wonder can we only read it in the emails ? nowhere else? Just like to read in light mode or reading mode of some browsers if possible.
Nice thumbnail. The dark and faded blue looks really good against the black background, very easy to read.
I love this format so much new things to learn. Would definetly love more of these.
I love the video and the explanation and the detail and everything. You genuinely want your viewers to learn these things and it shows. Thanks
This video hit me at the perfect time. That shared web worker strategy is a surprise to me. Might save some bugs in my own code!
Awesome awesome video! Thank you for taking the time to draw and give a great explanation of their system, I found it really helped and connected all the pieces together! You are the GOAT. I love these types of videos. Keep up the great work!
Liked the diagram pictures used & the length of the video was nice, I learned that this was possible from this video & knowing the pitfalls like the OPFS causing data corruption will save a lot of headaches I imagine.
Awesome !!! The details, the code and yes the pictures - very very useful ... I have also signed up for your newsletter. Your leetcode videos are gold standard, and so was this.
Man, you are so good in explaining things. Thank you for your hard work!
Honestly, Everything about your content, the way you share is just awesome for understanding. Looking forward for even more of these content.
love the content.
really interesting to see a breakdown of optimization of a platform I use and a bunch of other people
Loved the pictures. Explaination was amazing as well. And yup 20 minutes is ideal. Thanks for making this newsletter.
I rarely put comments on videos. But have to say, this is good. Length is good. Content is non-generic. Learnt something that might come handy someday
loved the video, it's perfect. Concise and letting me know the things possible as opposed to exactly how to do it.
Clean explanation. Need more high quality content like this. Good job my man🎉
Loved your way of explaining up the things, 19 minutes totaly worth to watch
sqlite is possibly the best piece of software written in the 21th century
loved videos that shows drawing to explain the process and thought on how its done, please do more like this! thanks
Visuals are key to understanding any topic much faster and better. So, add more if necessary.
Love the content! Keep the visuals coming. they make things much easier to understand.
This is awesome! I like the length of your video, your explanation and thought are clear as always! Please keep making these!
Awsome video, awsome topic as well. I wanted to learn more about using sqlite for caching, particularly now with WASM tools like the sqlite-vec extension for vector search!! This was great!
Visuals were great!
the visuals are good. obviously could always be better but i dont have further feedback on how. for the vid length could definitely cut it a bit.
Thanks as always
Great video as usual, i love your explanation and diagrams! Keep up the good work, THANKS
Love the vid! Visuals help a lot, thanks for taking the time to draw them!
Love this. Love your channel and most importantly way you explain things. Even for folks with 10+ yoe, it’s so useful.
This is so good keep doing it please ur the only person i can listen to talk
Keep the style! This works really well and I understood everything. Thank you
Everything is good 📈😊, honestly It makes me stick to the video to the end.
You should add the visuals like these to persuade the System design and very informative video BTW,
I hope like me everyone expect this like understandable and knowledgeable video from you in the future .
Thank you.
Dunno about cutting latency, but they did a lot if shite to make notion slow right after initial release. I remember how at first I was pretty happy how it was snappy as hell, but when our teams started to fill more documents it become slower. And then they've released a new update with new features and everything turned into a turtle - everything was slow as hell. Including simple selection and opening context menu.
Can't wait to study this, but shouldn't you be posting this on the main channel?
My guess is: His main account probably maxed subscribers in cs job market, so opening another channel would bring more exposure and double income.
Great job, man! I will check out your newsletter. Thanks!
This was extremely awesome and kept my attention intact and curiosity raised, drawing and images way of explanation is better (atleast for e) than rambling while showing up bunch of text. Thankyou so much.
Btw might be a naive question, how do you get to know about news as such ? what are your sources?
Love the thorough run through and easy explanation!
Cool video! I think the shared worker is called "service worker"?
BTW, you forgot to link the newsletter in the description
Very refreshing video, between leetcode problem solving videos.
looking forward to see more system design lecture , Thank you.
never had someone taught me this clean, you are the best thanks
Hi nav, I really enjoied a lot your video. I'm mind of tech lead in the places I work. We are doing aí stuff.
And it's very good to me to see how others experts are solving their problems
loved the video with the details , images and everyhting. it was soo clear and eays to understand, lots of knowledge as well!
great video! love the visuals, length, and overall format 👍
That was 20 minutes??? Time flys when you are having fun. The exact issue I’m writing thru too rn. Love it give me 14 more
learnt something within 5 mins of the video. Quality content, thank you !
I really like this video! I like the visuals, the length is perfect.
Excellent video, looking forward to have more like this. Really need some practical system design knowledge. Thanks for your videos as always
The visuals were awesome, Keep up the good work, THANKS
sick video. with a hands on explanation it was easy for a novice like me to understand it. i need moreeee
I like the coding, it really adds to the typical system design videos.
Love stuff like this. Signed up for the newsletter.
I have a software interview tomorrow. Watching this to see if I can learn some system design lingo to trick them into thinking I know stuff.
Love seeing these system design videos, keep it up (but also keep up leetcode questions, I'm still unemployed)
Great video man!!! Keep up the good work!!!
I really enjoy the explanation...
Keep doing these videos, i learned a lot from this. Also this just made me realize than as a self taught sometimes you can invent your own solutions to overcome programming problems like workarounds same as what notion did with race requests etc
"Latency is bad but you know what is worse..." I was totally expecting a sponsor plug there 😅
There is a key-value database like Firestore built into the browser (IndexedDB) without any limits. I guess Notion team's wants to write SQL
Great video, good length and details enough to be awesome watch and educational keep it up
your effort is worth it coz i watched the whole video till the end, your narrations visuals everything is top notch. Liked and subscribed
The visual is amazing, keep doing it 🔥
I love the video . I love the practical aspect. Kudos to you ....
Great video man! If you do get time make more videos on these concepts, atleast for things in you newsletter that you fell are video worthy!
Again, great video man!
Great deep dive. I appreciate the visuals.
OPFS allows you to write TB of dstaz it's not exclusive of SQLITE
This deserves more likes.
need some more videos like this my man!!!!!! keep up the good work!
Loved the video, I like how you explained system design
Often with bytebytego (which I do like) I do feel a little stumped at times
Also if I understand correctly we are literally having a database on the client side?!? That seems wild, but why didn't they just use indexedDB
Love the Length. Love the pictures. Just subbed.
Nice video with pretty good illustrations. Thanks mate
Please do more videos like this! Please don't leave daily Leetcode problems too =] These 2x are "must have" to interview.
I actually prefer Informative videos and 'Hour-long-videos' actually doesn't matter to me yk, I'd like if you produce such contents as I really feel your ability to explain is pretty neat.
Oh and also, really nice and informative visuals I really appreciate it.
Great video you should keep the images and cover more details on each topic.
what prompted Notion team to use SQLite instead of SQL-like storage already available in the browser called indexDB?
Do more of these videos plz!! Joined your newsletter!
Great video and super clear explanations, thanks a lot!
Pictures and diagrams are a requirement. Rather than creating your own you can use some icon library and grab what you need. Boring standards is better than flashy when learning, it's more important that everyone is on the same page.
This is fire ! Keep more of these coming !