Great video Gaurav. You have simplified it so much. I have one doubt though... At 6.20 when you are mentioning the 4th point, you say NoSQL are good for metrics/analysis, etc. because it easier to perform operations like average age, total salary, etc. At 7.30 you are saying these are not read optimized because data will have to be read from each blob of data and then perform some operation like sum or average. I am confused about this part.
This is misleading. Not all NoSQL store data as a key-value pair. What you are talking about is only a subset of NoSQL. There are 4 meain types of NoSQL databases: Document, key-value, wide-column and graph.
This is the first video of yours that I saw but amazing way of explaining bro. This video is great for someone like me who had absolutely zero idea about NoSQL Databases since I have always worked with Relational Databases only. Subscribed!
How I look RDBMS and NoSQL databases are used based on the requirements or use cases. So, while designing any application we need to understand them first which @GS has done very well. I love your presentation skills @GS and I won't mind mentioning the same in your videos. Keep it up👍
Gaurav, you have tremendous ability to articulate modern day computer science concepts. Its great to see someone so young having this charisma and tech flair which is a rare combo. I have been in software for 20 years and sadly i was never taught like this or then around early 2000's there were no youtube channels like yours. You are redefining online learning with your videos. Keep it up mate.
Great Video! A small point to add, Most of the NoSQL offerings also offer, Consistency levels for users to choose. So, If I want to make sure my users read always consistent data I can have strong consistency which means a write is acknowledged only when a quorum of replicas have also acknowledged it. This makes sure, consistency is present even when one of the replicas go down. But obviously the tradeoff is the writes are slow. If availability is preferred over consistency, then Eventual consistency can be choosen in which the writes are acknowledged when the present replica writes it in memory, hoping that all other replicas catch up with the write "eventually" .
Agree! This is what allows us to aggregate and read fast on NoSQL. I made a mistake in the video by stating that reads are slow. Reads are in fact faster in NoSQL than standard RDBMS as long as consistency requirements are relaxed.
Writes are fast in Cassandra if replication factor is local_One. Although if you change it to quorum then obviously it adds to the throughput of the transaction. It’s all about the system requirement at the end of the day. 😃
@@gkcs Has anything changed with MongoDB 4.2, are the writes any faster considering we get to keep our consistency? Also, We know that NoSQLs go for Availability over Consistency but with MongoDB 4.2, you can guarantee the consistency and also I get to keep my availability by scaling across many shards. MongoDB 4.2 (FULLY ACID th-cam.com/video/iuj4Hh5EQvo/w-d-xo.html) They claim that they're "the only database to fully combine the ACID guarantees of traditional relational databases with the speed, flexibility, and power of the document model, and an intelligent distributed systems design to scale-out and place data where you need it." (www.mongodb.com/collateral/mongodb-multi-document-acid-transactions) Which also puts me to a question that why should I use RDBMS over MongoDB when with horizontal scaling I will have a hard time but with MongoDB 4.2 it's built-in with all it brings. Does MongoDB > RDBMS in 2020 after the 4.2?
It is a difficult skill to be able to understand/comprehend the lower layers of a given technology AND also be able to present it in a clear, concise manner that many can grasp. You have this skill and are able to present the data in a way that is simple with stacks that are complex. This is why being an "instructor" or "presenter" requires skills beyond just knowing the technology really well. Anyway, I appreciate the videos as its a wealth of valuable information!
Great video as always :D, Just one correction , data is kept in self sorted structures like (AVL/ Red-Black Trees) in memory, and once the memory is past some threshold value (say ~50kb), then the entire memtable(the self sorted trees) are dumped into a SSTable (on disk) which is efficient as the data is already sorted.
In Cassandra, “strong consistency” is typically expressed as W + R > RF, where W is the write consistency level, R is the read consistency level, and RF is the replication factor.
Listen guys... this way of classifying a database as either "SQL" or "NoSQL" is meaningless. The relational datamodel is pretty well defined, but there are countless of non-relational databases each with their own datamodel and consistency guarantees. It's meaningless to lump them all into 1 category and talk about them as if there were all MongoDB.
I am kind of confused that how No-SQL databases can have the read time as a disadvantage along with data aggregation as an advantage. Isn't a lot of read required for aggregating the data?
I think that the key is database sharding and partitions. If you don't take this into account, read across partitions its a very expensive operation. If your database is well design, then you are ok.
@@amythpaddy8527 I think what Gaurav means is most of NoSQL databases give inbuilt support for aggregation like mongo. Please go through this link you might get your answer docs.mongodb.com/manual/core/aggregation-pipeline/
I think read performance can vary a lot for each NoSQL database. To further complicate the matter small reads vs large batch reads could have vastly different performances. Data aggregation tends to be large batch reads I would assume. Perhaps that is the reason for his statements?
Hey, 1. When you say that NoSQL is better for insertion and retrieval, you say that Relational DBs are slower because you will have to join the two tables with address as foreign key. If that's the scenario, we should not normalize the table here and have address also in the same table for faster reads. Also, locks are still present in implementations of NoSQL DBs because of concurrency. So both are comparable actually and anyone can be slower than the other as it depends on the way you have decided the structure to store data. Ofcourse, retrieving data with a join in SQL will turn out to be slower. 2. Then at 7:58 you say NoSQL is not read-optimized and in advantages you mentioned its used for aggregations as well as. I think it makes sense not to have such comparisons made out as NoSQL databases can be implemented in variety of ways. In-memory databases are also subset of NoSQL and pretty pretty fast to read because of RAM coming into play. Talking about consistency at scale, one is user's choice of stronger v/s eventual but we can't say that Relational DBs will be slow because locks will still be there in implementations of NoSQL databases because of concurrency. To win that, we have concept of granular locks in Relational Databases so that lock is placed only on necessary part. I think the journey to scale any database is pretty complex and only experience can teach us exactly how to do that. TH-cam is running on Relational DB after all - but I am sure its much more complex with portions of NoSQL, CDNs, caching, and what not flying around in their backend architecture.
Hey Rachit, thanks for the good insights. Here are my thoughts: 1. We "shouldn't" normalize a table is a difficult decision to make. Normalizing offers a logical break up of the data, ACID properties and is the way SQL databases are designed to run. It's true that a lot of systems run their analytics on NoSQL databases. The reason for this is the non-normalized form of data in these tables. 2. I made a mistake here, you can have a look at the pinned comment for clarifications. Reads and writes are usually faster in NoSQL, because it is rare to take a ton of locks in these tables. Aggregations are faster if you have a columnar database, and that along with faster read times contribute to the performance. The implementation decisions influence a lot of how a database performs, but the core ideas on which it runs are very important. The NoSQL databases are, by definition, denormalized and expected to take fewer locks. Eventual consistency, Quorum and Fast updates are selling points of databases like Cassandra. Using them for diametrically opposite goals wouldn't be wise, in my opinion.
@@gkcs I don't get the part that aggregates are faster in NOSQL than SQL at 06:21. How is total salary at 06:21 in NOSQL faster than find all ages of all employees in company at 7:30? Both will require getting each blob, parsing etc which won't happen in SQL columnar table. appreciate your reply.
21:50 Two corrections 1) Cassandra do not store log file in-memory. Rather it stores on disk. That's how it can recover from during failures 2) Cassandra do not append in-memory it rather appends to commit log on disk. So, in-memory the data is sorted Memtable. When it reaches certain limit then it flushes to the disk on SSTable
Love your videos dude! Just watched about 8 of them and I now use them in the gym because you can pretty much follow along just listening to you :-) Quick Tip: Set your camera to manual focus and increase the aperture (will keep everything in focus) a little as your camera 'hunt's for focus and is a bit distracting on the eye. Love the quality your producing so please keep them coming!
Thanks for video @Gaurav. query: If read times are slower in NoSql, how is it that it is good for aggregations? If I want average age per city, I still have to go through all records and and entire blob for each record, this is expensive right? Can you elaborate more on this?
thanks very match for this great video ,i have a question please answer me :i want to do android app social media but i still struggle with myself for chosing type of database i search in stackoverflow i found answer for my question : that you should use graph database :is the graph database is a great solution for my problem ?? (english not my original language)
Just curious how old are you? I'm 23 and I am not this smart, this is a great channel I discovered that I can relate to (shirts/intros) and actually enjoy listening quality content ^_^
When a Non DB guy can understand this.... there is nothing else as better ! Thanks a Ton for you Videos. Your Videos are one of the things that helped me through 2020 Lockdown.
Your Videos are always Amazing Let me know how can we Develope programming skills on Algorithms and all advanced level coding skills apart from a average programming . I am in BCA 2nd year learnt C, C++, now learning DS. Can i go with learning competitive programmings . Please Provide Best way to Improve Coading Skills which really worth for future.
Hi Gaurav, Thanks for this great video and all other videos. I'm extremely benefiting by your videos. Basically, I'm a Mechanical Engineering graduate, with zero CS/dev background but currently work as a Technical Writer with one of top technology Giant in the world. I want to transition to product management and one of areas I lack is technical design. Yours videos are helping me in those aspects. Kudos your great effort. Appreciate every bit of it.
Thank you so much for your video. It is amazing and I would have never been able to understand how NoSQL works without it. Just a minor thing that I noticed and that might be a mistake. In the cons for NoSQL you say that relations are not implicit. But shouldn't it be that they are not explicit or that they are implicit?
Thanks MrAntraxico! Yes, a NoSQL database doesn't define relationships like foreign keys so well. Which means the relationships are not explicitly defined 👍
wow, I am understanding now many things which I had already worked on and faced the technical issues and never used to get the "why" part from my architect's talks. Thank you Gaurav.
Then you were never an actual architect. Most people think they are architects, it takes time... Unless you got 20 years in multiple industries, companies, environments you can't be truly an architect. Working in MS, JAVA, WEB, Service, Networking, Infrastructure, CICD, UX, Security from every possible way I think I have a better understanding. This video is your novices or juniors not architects
Hello, Thanks for the excellent video. can you answer the following scenario: Scenario 1: User enters address and leaves the post code field blank : RDBMS: The program inserts null in the post code field. JSON: The program creates a JSON document with all the fields as keyvalue pairs except for post code. Scenario: User now updates his address and enters the post code RDBMS: The program updates the address by inserting the value for postcode. JSON: what happens in this case? Will it update the same document using the object ID? Do we have a duplicate check like we can have in RDBMS so that we can decide if its a update or a new addition?
@@gkcs Thanks. I tried to upload the same document in JSON format by simple adding one more attribute. However, I receive an error message stating that "MyApp_DB.Users index: _id_ dup key: { : ObjectId('5c9e845c2335513b784e96bc') }"
While mentioning cons, in 2nd point when you said that reads are slower in comparison to Relational databases. I don't think it's rue always, as in columnar database like cassandra, you can directly fetch a column values faster compared to RDMS. The reason being in Relational database, for every row it has to parse to that column to get value, while in columnar just fetch that column and done. Please correct me, if I am wrong 🙂
Hey, Gaurav thanks for excellent video! I wanted to ask as per your video on sharding: th-cam.com/video/5faMjKuB9bc/w-d-xo.html, for replication you specified master-slave architecture. However, in Cassandra, we do replication in cyclic order (order may be any but lets use the same as described in video). Can you describe trade-off between two choices? Hope, you will answer this!
Hey Gaurav, Love your content I have one doubt though: You mentioned that read times are comparatively slower for NoSQL DB as compared to RDBMS. Also you mentioned that NoSQL DB are built for aggregation/ analytics. How is that possible when reads are slower ? Can you please throw some light here ?
You do have a clean way to "denormalise things" and it's simple. Just SET the address COLUMN type to JSON if you think joining the address table is expensive and you don't want to have a separate table for the addresses. But then if more than one person can be from the same address you'll be storing (address * number of persons). You can also create a table with the same structure as your NoSQL example; CREATE TABLE users { id int unsigned AUTO_INCREMENT PRIMARY KEY, value JSON } Note that if your developer is always requesting all columns (*) when they don't need to then you have a bad developer and if an SQL Database can deal with unstructured data like a NoSQL DB then which of em is the best? 🤔🤔🤔
Guarav, thank you for the awesome videos. I am applying for entry-level software engineering jobs and this is helping me a lot for companies which test system design skill of their candidates.
THANK YOU SO MUCH!! Read my reading assignment for my NoSQL class and understood most of the things but needed to see all these new concepts in action! Thank you for providing an excellent run through and example! LIKED AND SUBSCRIBED!!!
I have a doubt. While discussing the advantages of the NoSQL database you said that aggregate analysis will be performed in a much better way here. But I think, SQL databases will be doing this in a much better way. SQL databases are designed with the relational model in mind and are well-suited for complex queries, aggregations, and analytical processing. Please guide me about it.
Hey Gaurav, I have a doubt. if all the shards have master and slave, wont it be better? if master goes down slave will take its place. Cassandra might not have this, but generally can we design something like this?
Was just randomly browsing to know about NOSQL and I must say i couldn't move forward without watching full video and I feel confident with the concept. Thank you Gaurav
Thanks for a nice video, I have a question though, i thought the key values are stored as hash functions within the memory as well, so for example if the request id is 123 and the server is 2 which handles the request, then in server 2 all key values are stored as hash map for fast retrieval, i a confused with sequential key value store (similar to a log file). Please can you clarify
Hey, For me it is false, you speak specifically of MongoDB and not of the family of noSQL, which gathers much more, and much better. MongoDB is really useless. Try a multimodel NoSQL database, such as OrientDB (Graph type) and we'll talk about it again, it is free. Compare performance, robustness, scalability, and there we can talk about a real solution to get out of ancestral databases and obsolete SQL relation
Small doubt when does Quorum thing takes place? is it for every read operation or only when main shard where the data is present fails? For first case, there is no need of voting, because the main shard is our base to search.
Hey Gaurav, great video. I have a different question. Do big companies like facebook take backups of huge user data? If they do i'm assuming a backup would be a heavy load (read time) on the database. How would they do that while simultaneously serving user requests. My initial assumption is that they don't do the whole backup at once, but rather in parts at different times. Also a different strategy i can think of is a time based solution , essentially do your backups for a given country when it's night time as we can assume the load will be significantly lesser and avoid potential conflicts. Would love to hear your thoughts .....
Hey Gaurav, at @15:33 you mention we want to replicate 2's data in 3, but 2's data should be replicated in 1 right ? Because if 2 goes down all of 2's requests will go to 1 ? Or is it just using data from 1 ?
Regarding quorum of 3 and missing data, if you want your system to be highly consistent, you can do it while writing the data. Say, if you have 3 replicas, make sure the write succeeds only if they data is written on all the three nodes. This obviously has some drawbacks of delay in write but if consistency is all you care about, this approach would help overcome missing data. In CAP, you are compromising on availability here.
thank you. how to scale my MySQL database? i am using 1vcpu,2gb ram digital ocean droplet, which contains a node js API and MySQL server, it works fine, but after 10-12 days MySQL queries become slower. how to scale MySQL? i have 50% ram free and only 3-4 % cpu is using most of the time.
omg bhai..u jus like my frnd who teaches me one day b4 exam and i get salvation enlightened whole universe gets cleared which i try to understand and learn whole 6 months with teachers faculty staff and didn't understand :)
Also, I am a bit confused on your two stmts.. NoSQLs are built for getting meaningful info like aggregates.. another stmt is I"t is not recommended when select happens on only a few columns"? Aggregate should happen on age or salary only right? May be some more info can make me understand this stmt..
If we can skip the address in NoSQL(4:30) straightaway then whats the difference between error mentioning Error 1 : No address found Error 2 : Address field is empty ???
Great video Gaurav. You have simplified it so much. I have one doubt though... At 6.20 when you are mentioning the 4th point, you say NoSQL are good for metrics/analysis, etc. because it easier to perform operations like average age, total salary, etc. At 7.30 you are saying these are not read optimized because data will have to be read from each blob of data and then perform some operation like sum or average.
I am confused about this part.
This is misleading. Not all NoSQL store data as a key-value pair. What you are talking about is only a subset of NoSQL. There are 4 meain types of NoSQL databases: Document, key-value, wide-column and graph.
This is the first video of yours that I saw but amazing way of explaining bro. This video is great for someone like me who had absolutely zero idea about NoSQL Databases since I have always worked with Relational Databases only. Subscribed!
How I look RDBMS and NoSQL databases are used based on the requirements or use cases. So, while designing any application we need to understand them first which @GS has done very well.
I love your presentation skills @GS and I won't mind mentioning the same in your videos. Keep it up👍
The intro was extra fun😀 and educative
I LOVE IT❤😂
Schema is always changable can be a big disadvantage as well if we need to update retroactively..
True 🙂
MongoDB has full ACID transactions and supports joins
Too good.
Thank you!
Great Teaching Skills. As a 63 year old, I found learning from a youngster, really cool!
Thank you!
I m 44 but wanted to say the same.Now no need.
i m 30+ and fan of them too
You're 64 and still leaning. I got to learn from you. Take care, stay safe.
Me too; learning at 55:-) from GKCS
Gaurav, you have tremendous ability to articulate modern day computer science concepts. Its great to see someone so young having this charisma and tech flair which is a rare combo. I have been in software for 20 years and sadly i was never taught like this or then around early 2000's there were no youtube channels like yours. You are redefining online learning with your videos. Keep it up mate.
Thank you Suraj!
Great Video!
A small point to add, Most of the NoSQL offerings also offer, Consistency levels for users to choose. So, If I want to make sure my users read always consistent data I can have strong consistency which means a write is acknowledged only when a quorum of replicas have also acknowledged it. This makes sure, consistency is present even when one of the replicas go down. But obviously the tradeoff is the writes are slow. If availability is preferred over consistency, then Eventual consistency can be choosen in which the writes are acknowledged when the present replica writes it in memory, hoping that all other replicas catch up with the write "eventually" .
Agree! This is what allows us to aggregate and read fast on NoSQL.
I made a mistake in the video by stating that reads are slow. Reads are in fact faster in NoSQL than standard RDBMS as long as consistency requirements are relaxed.
a great summary of cap ❤️✔️
Writes are fast in Cassandra if replication factor is local_One. Although if you change it to quorum then obviously it adds to the throughput of the transaction. It’s all about the system requirement at the end of the day. 😃
@@gkcs
Has anything changed with MongoDB 4.2, are the writes any faster considering we get to keep our consistency?
Also, We know that NoSQLs go for Availability over Consistency but with MongoDB 4.2, you can guarantee the consistency and also I get to keep my availability by scaling across many shards. MongoDB 4.2 (FULLY ACID th-cam.com/video/iuj4Hh5EQvo/w-d-xo.html)
They claim that they're "the only database to fully combine the ACID guarantees of traditional relational databases with the speed, flexibility, and power of the document model, and an intelligent distributed systems design to scale-out and place data where you need it."
(www.mongodb.com/collateral/mongodb-multi-document-acid-transactions)
Which also puts me to a question that why should I use RDBMS over MongoDB when with horizontal scaling I will have a hard time but with MongoDB 4.2 it's built-in with all it brings. Does MongoDB > RDBMS in 2020 after the 4.2?
@@ananyasharma1201 "why should I use RDBMS over MongoDB " -> because MongoDB is buggy as fuck. It's not reliable.
It is a difficult skill to be able to understand/comprehend the lower layers of a given technology AND also be able to present it in a clear, concise manner that many can grasp. You have this skill and are able to present the data in a way that is simple with stacks that are complex. This is why being an "instructor" or "presenter" requires skills beyond just knowing the technology really well. Anyway, I appreciate the videos as its a wealth of valuable information!
Thank you!
Apart from technical expertise you are also trying to specialise in video effects 😊
Yup!
In a strange way, I watched this Video like a proud father lol
You are amazing and it shows how much you care to explain rather than show off.
Thanks Ramzi!
"Why don't we become optimists as engineers?" 🤣🤣 Yea right I wish. Thanks for the upload !
Hahaha 😛
Great little bro..you looks so young but good skill on explanation..
Thanks!
Truth has been spoken at 9:26 "inner join,outer join,left join the things that we didn't read in college"
very true bro...after working in a job only we start to get to understand the real use of join and group by😜
College mai toh kuch bhi nhi batate hume khud hi krna padta h..😅😅😅 gfg h na
Great video as always :D, Just one correction , data is kept in self sorted structures like (AVL/ Red-Black Trees) in memory, and once the memory is past some threshold value (say ~50kb), then the entire memtable(the self sorted trees) are dumped into a SSTable (on disk) which is efficient as the data is already sorted.
In Cassandra, “strong consistency” is typically expressed as W + R > RF, where W is the write consistency level, R is the read consistency level, and RF is the replication factor.
Greetings from Munich, Germany. Great video Gaurav. It needs a lot of preparation behind such great content. Kudos man!
Listen guys... this way of classifying a database as either "SQL" or "NoSQL" is meaningless.
The relational datamodel is pretty well defined, but there are countless of non-relational databases each with their own datamodel and consistency guarantees. It's meaningless to lump them all into 1 category and talk about them as if there were all MongoDB.
I am kind of confused that how No-SQL databases can have the read time as a disadvantage along with data aggregation as an advantage. Isn't a lot of read required for aggregating the data?
Same question. And from my experience I did not find data aggregation easy for nosql data. Am I doing something wrong?
I think that the key is database sharding and partitions. If you don't take this into account, read across partitions its a very expensive operation. If your database is well design, then you are ok.
@@amythpaddy8527 I think what Gaurav means is most of NoSQL databases give inbuilt support for aggregation like mongo. Please go through this link you might get your answer docs.mongodb.com/manual/core/aggregation-pipeline/
I think read performance can vary a lot for each NoSQL database. To further complicate the matter small reads vs large batch reads could have vastly different performances. Data aggregation tends to be large batch reads I would assume. Perhaps that is the reason for his statements?
if consistency is relaxed, then read is faster in Nosql, else consistency adds an overhead .
Damn!! why does liking something complex gives me a vicarious feel of being intelligent?
This boy is really good.Great content
Hey,
1. When you say that NoSQL is better for insertion and retrieval, you say that Relational DBs are slower because you will have to join the two tables with address as foreign key. If that's the scenario, we should not normalize the table here and have address also in the same table for faster reads. Also, locks are still present in implementations of NoSQL DBs because of concurrency. So both are comparable actually and anyone can be slower than the other as it depends on the way you have decided the structure to store data. Ofcourse, retrieving data with a join in SQL will turn out to be slower.
2. Then at 7:58 you say NoSQL is not read-optimized and in advantages you mentioned its used for aggregations as well as.
I think it makes sense not to have such comparisons made out as NoSQL databases can be implemented in variety of ways. In-memory databases are also subset of NoSQL and pretty pretty fast to read because of RAM coming into play.
Talking about consistency at scale, one is user's choice of stronger v/s eventual but we can't say that Relational DBs will be slow because locks will still be there in implementations of NoSQL databases because of concurrency. To win that, we have concept of granular locks in Relational Databases so that lock is placed only on necessary part.
I think the journey to scale any database is pretty complex and only experience can teach us exactly how to do that. TH-cam is running on Relational DB after all - but I am sure its much more complex with portions of NoSQL, CDNs, caching, and what not flying around in their backend architecture.
Hey Rachit, thanks for the good insights. Here are my thoughts:
1. We "shouldn't" normalize a table is a difficult decision to make. Normalizing offers a logical break up of the data, ACID properties and is the way SQL databases are designed to run. It's true that a lot of systems run their analytics on NoSQL databases. The reason for this is the non-normalized form of data in these tables.
2. I made a mistake here, you can have a look at the pinned comment for clarifications. Reads and writes are usually faster in NoSQL, because it is rare to take a ton of locks in these tables. Aggregations are faster if you have a columnar database, and that along with faster read times contribute to the performance.
The implementation decisions influence a lot of how a database performs, but the core ideas on which it runs are very important. The NoSQL databases are, by definition, denormalized and expected to take fewer locks. Eventual consistency, Quorum and Fast updates are selling points of databases like Cassandra. Using them for diametrically opposite goals wouldn't be wise, in my opinion.
So finally, which is faster to retrieve all values in Age-column?
@@indiansoftwareengineer4899 NoSQL.
@@gkcs
Thanks Gaurav Bhai for reply.
@@gkcs I don't get the part that aggregates are faster in NOSQL than SQL at 06:21.
How is total salary at 06:21 in NOSQL faster than find all ages of all employees in company at 7:30? Both will require getting each blob, parsing etc which won't happen in SQL columnar table.
appreciate your reply.
Just to put an example, Royal Bank of Scotland uses MongoDB, a No-SQL database. So both SQL and No-SQL are in present day screnario, equally lucrative
10:30 Looks like Request sperm trying to penetrate to Cassandra egg. Not sure what will be reproduced.. :D
@Gaurav_Sen bother hope you are doing well. I just saw you on TH-cam.. I really appreciate for what you are doing... Sharing knowledge.. 🙏 🙏 🙏 🙏 🙏.
Thank you!
if reading is not optimised in NoSQL how are aggreagations optimised? If you try to find average of ages you have to read them. can you explain?
Sir, Are you twins?
21:50
Two corrections
1) Cassandra do not store log file in-memory. Rather it stores on disk. That's how it can recover from during failures
2) Cassandra do not append in-memory it rather appends to commit log on disk. So, in-memory the data is sorted Memtable. When it reaches certain limit then it flushes to the disk on SSTable
"Why don't we become more optimists as engineers" - gkcs, 2019
NoSQL databases isn't limited to Document databases and Cassandra...
MongoDb now supports transactions.
Apart from jokes .. i really love no sql databases ( mongo db )
I prefer to use them always
Ahem, good choice 😁
Try to find() based on a populated() data. Its a nightmare.
@@claudiob3161 .find('data': {$exists: true})?
Sir, you are truly a great teacher. Thanks for sharing your knowledge so wisely.
Thank you!
is it true whatsapp doesn't have database? how is that possible? or is it a joke which i don't get?
if it is not read optimized , then how is it good for analytics ? could you please comment
@ 7:56 you say the read times are slower and I get the reasoning for it. But if that's the case then how are NoSQL DBs optimized for analytics?
Love your videos dude! Just watched about 8 of them and I now use them in the gym because you can pretty much follow along just listening to you :-)
Quick Tip: Set your camera to manual focus and increase the aperture (will keep everything in focus) a little as your camera 'hunt's for focus and is a bit distracting on the eye. Love the quality your producing so please keep them coming!
Thanks!
I'll make sure to apply these tips 😁
Tomorrow is my Big data and data analytics exam and You uploaded this...........what a coincidence!
Yey!
kaisa hua exam XD
Thanks for video @Gaurav. query: If read times are slower in NoSql, how is it that it is good for aggregations? If I want average age per city, I still have to go through all records and and entire blob for each record, this is expensive right? Can you elaborate more on this?
the beginning was some ninja editing skills..!! the video as always very informative.. Thank you..!
Thank you!
thanks very match for this great video ,i have a question please answer me :i want to do android app social media but i still struggle with myself for chosing type of database i search in stackoverflow i found answer for my question : that you should use graph database :is the graph database is a great solution for my problem ?? (english not my original language)
That depends on your problem.
the don.
I liked how everyone ignored the WhatsApp roast :p
You've made System design a real fun concept to read about! Thanks a lot
Thanks Pallavi!
I'm sorry but wanted to ask whether its an effect or your twin brother?
Hahaha
I got this video as a suggestion and just started watching...
seriously thought - Twin brothers started a channel and how much sync they are in :D
I agree with the brother who has the winterfell shirt
10 seconds in subscribed
First video I am watching.
Know I will enjoy your content
Thanks!
0:56 lmao "WhatsApp doesn't HAVE a database!"
One of the best explanation for nosql over youtube...so far...
Keep up the good work
Thanks!
Just curious how old are you? I'm 23 and I am not this smart, this is a great channel I discovered that I can relate to (shirts/intros) and actually enjoy listening quality content ^_^
Thanks Mario!
I have been working for 4 years now, so it's been some time since I started off :)
How old are you though?
@@zorocheese3939 Mid 20s
@@gkcs Aaah... I get it... you're almost 30. I didn't expect that you'd be worried about revealing it. Age is just a number.
Great video. Please make a detailed video on Redis or distributed cache system
Coming up 😁
No one saw Tim Sort? Maybe call it Gaurav Sort? then you might get views :P
Hahaha
When a Non DB guy can understand this.... there is nothing else as better !
Thanks a Ton for you Videos.
Your Videos are one of the things that helped me through 2020 Lockdown.
Amazing explaination for a noob. Nice job
Thanks!
I am convinced to switch! :D
😁
NoSQL = Not Only SQL.
Truly amazing video. So perfectly described NoSql and RDBMS.
Thank you so much.
11 seconds passed and I gave a like to the video. Great job!
😁
Your Videos are always Amazing Let me know how can we Develope programming skills on Algorithms and all advanced level coding skills apart from a average programming . I am in BCA 2nd year learnt C, C++, now learning DS. Can i go with learning competitive programmings . Please Provide Best way to Improve Coading Skills which really worth for future.
Upcoming video idea : Give a brief on all current trending databases, pros and cons and when to pick each
Can any tell what does it mean whatsapp doesn't have database 🤔
Hi Gaurav, Thanks for this great video and all other videos. I'm extremely benefiting by your videos. Basically, I'm a Mechanical Engineering graduate, with zero CS/dev background but currently work as a Technical Writer with one of top technology Giant in the world. I want to transition to product management and one of areas I lack is technical design. Yours videos are helping me in those aspects. Kudos your great effort. Appreciate every bit of it.
Thanks Karthik!
Thank you so much for your video. It is amazing and I would have never been able to understand how NoSQL works without it.
Just a minor thing that I noticed and that might be a mistake. In the cons for NoSQL you say that relations are not implicit. But shouldn't it be that they are not explicit or that they are implicit?
Thanks MrAntraxico!
Yes, a NoSQL database doesn't define relationships like foreign keys so well. Which means the relationships are not explicitly defined 👍
@@gkcs Awesome
wow, I am understanding now many things which I had already worked on and faced the technical issues and never used to get the "why" part from my architect's talks. Thank you Gaurav.
Then you were never an actual architect. Most people think they are architects, it takes time... Unless you got 20 years in multiple industries, companies, environments you can't be truly an architect. Working in MS, JAVA, WEB, Service, Networking, Infrastructure, CICD, UX, Security from every possible way I think I have a better understanding. This video is your novices or juniors not architects
@@paulkersey2424 I'm not.. I was a developer. I mean to say.. my project's architect - my boss. 😳
Do u personally like each of our comments or u wrote some sort of program to do it automatically 😂
Hello, Thanks for the excellent video. can you answer the following scenario:
Scenario 1: User enters address and leaves the post code field blank :
RDBMS: The program inserts null in the post code field.
JSON: The program creates a JSON document with all the fields as keyvalue pairs except for post code.
Scenario: User now updates his address and enters the post code
RDBMS: The program updates the address by inserting the value for postcode.
JSON: what happens in this case? Will it update the same document using the object ID? Do we have a duplicate check like we can have in RDBMS so that we can decide if its a update or a new addition?
The user address must have an id for an update. It will be overwritten when they update their postal code.
@@gkcs Thanks. I tried to upload the same document in JSON format by simple adding one more attribute. However, I receive an error message stating that "MyApp_DB.Users index: _id_ dup key: { : ObjectId('5c9e845c2335513b784e96bc') }"
While mentioning cons, in 2nd point when you said that reads are slower in comparison to Relational databases. I don't think it's rue always, as in columnar database like cassandra, you can directly fetch a column values faster compared to RDMS. The reason being in Relational database, for every row it has to parse to that column to get value, while in columnar just fetch that column and done. Please correct me, if I am wrong 🙂
That's true, and I did mention it in the advantages. But when looking for a select query with limited columns, RDBMS performs better 🙂
A song of ice and fire
Hey, Gaurav thanks for excellent video!
I wanted to ask as per your video on sharding: th-cam.com/video/5faMjKuB9bc/w-d-xo.html, for replication you specified master-slave architecture.
However, in Cassandra, we do replication in cyclic order (order may be any but lets use the same as described in video).
Can you describe trade-off between two choices?
Hope, you will answer this!
Hey Gaurav,
Love your content
I have one doubt though: You mentioned that read times are comparatively slower for NoSQL DB as compared to RDBMS. Also you mentioned that NoSQL DB are built for aggregation/ analytics. How is that possible when reads are slower ? Can you please throw some light here ?
Brooo I had the same doubt. I also commented this same doubt and then I found your comment. If you got this cleared, could you please explain me?
8:43 I think you got the referential integrity backwards there.
You do have a clean way to "denormalise things" and it's simple. Just SET the address COLUMN type to JSON if you think joining the address table is expensive and you don't want to have a separate table for the addresses. But then if more than one person can be from the same address you'll be storing (address * number of persons).
You can also create a table with the same structure as your NoSQL example;
CREATE TABLE users {
id int unsigned AUTO_INCREMENT PRIMARY KEY,
value JSON
}
Note that if your developer is always requesting all columns (*) when they don't need to then you have a bad developer and if an SQL Database can deal with unstructured data like a NoSQL DB then which of em is the best? 🤔🤔🤔
At the first I was shocked when he said whatsapp doesn't have database. I was like wtf from where I got my old message.. 😅😅😅
Guarav is so cute
Guarav, thank you for the awesome videos. I am applying for entry-level software engineering jobs and this is helping me a lot for companies which test system design skill of their candidates.
Great to hear that 😁
THANK YOU SO MUCH!! Read my reading assignment for my NoSQL class and understood most of the things but needed to see all these new concepts in action! Thank you for providing an excellent run through and example! LIKED AND SUBSCRIBED!!!
Glad to hear that!
They asked me about NoSQL in my interview today and I wished I watched the whole video before. 🙁
There is always a next time 😁
I have a doubt. While discussing the advantages of the NoSQL database you said that aggregate analysis will be performed in a much better way here. But I think, SQL databases will be doing this in a much better way. SQL databases are designed with the relational model in mind and are well-suited for complex queries, aggregations, and analytical processing.
Please guide me about it.
Hey Gaurav, I have a doubt. if all the shards have master and slave, wont it be better? if master goes down slave will take its place. Cassandra might not have this, but generally can we design something like this?
this is the first IT explanation video made by Indian guy, that I enjoyed watching
I hope you find many more in future 😁
Maybe start looking at people as the person instead of viewing them in terms of race, nationality, gender. etc.
if your are in java u should watch tutorial from java brains .. ur perception will change.
Can we say that in Cassandra DATACENTER is what is generally called a NODE in NoSQL and NODE in CASSANDRA would we a SHARD ? Thanks
Was just randomly browsing to know about NOSQL and I must say i couldn't move forward without watching full video and I feel confident with the concept. Thank you Gaurav
Thanks for a nice video, I have a question though, i thought the key values are stored as hash functions within the memory as well, so for example if the request id is 123 and the server is 2 which handles the request, then in server 2 all key values are stored as hash map for fast retrieval, i a confused with sequential key value store (similar to a log file). Please can you clarify
Hey,
For me it is false, you speak specifically of MongoDB and not of the family of noSQL, which gathers much more, and much better.
MongoDB is really useless.
Try a multimodel NoSQL database, such as OrientDB (Graph type) and we'll talk about it again, it is free.
Compare performance, robustness, scalability, and there we can talk about a real solution to get out of ancestral databases and obsolete SQL relation
Small doubt when does Quorum thing takes place? is it for every read operation or only when main shard where the data is present fails? For first case, there is no need of voting, because the main shard is our base to search.
Hey Gaurav, great video.
I have a different question.
Do big companies like facebook take backups of huge user data?
If they do i'm assuming a backup would be a heavy load (read time) on the database. How would they do that while simultaneously serving user requests.
My initial assumption is that they don't do the whole backup at once, but rather in parts at different times.
Also a different strategy i can think of is a time based solution , essentially do your backups for a given country when it's night time as we can assume the load will be significantly lesser and avoid potential conflicts.
Would love to hear your thoughts .....
This reminds me of RAIDs.
What if what'sapp has a database 🤷🏼♂️ .... they gotta know how many people using their stuff can't rely on a in memory count variable 😂😂😂😂
Hey Gaurav,
at @15:33 you mention we want to replicate 2's data in 3, but 2's data should be replicated in 1 right ? Because if 2 goes down all of 2's requests will go to 1 ? Or is it just using data from 1 ?
Bro you should upload 1 min or 30 sec separate video of that intro clip which discusses Difference between NOSQL & SQL!! It is great🤣🤑
3:02 "Unless of course its a very big table or if you have some column which is pretty big".... like a BLOB!!!!!
Regarding quorum of 3 and missing data, if you want your system to be highly consistent, you can do it while writing the data. Say, if you have 3 replicas, make sure the write succeeds only if they data is written on all the three nodes. This obviously has some drawbacks of delay in write but if consistency is all you care about, this approach would help overcome missing data. In CAP, you are compromising on availability here.
That's correct.
thank you. how to scale my MySQL database?
i am using 1vcpu,2gb ram digital ocean droplet,
which contains a node js API and MySQL server,
it works fine, but after 10-12 days MySQL queries become slower.
how to scale MySQL? i have 50% ram free and only 3-4 % cpu is using most of the time.
omg bhai..u jus like my frnd who teaches me one day b4 exam and i get salvation enlightened whole universe gets cleared which i try to understand and learn whole 6 months with teachers faculty staff and didn't understand :)
Also, I am a bit confused on your two stmts.. NoSQLs are built for getting meaningful info like aggregates.. another stmt is I"t is not recommended when select happens on only a few columns"? Aggregate should happen on age or salary only right? May be some more info can make me understand this stmt..
7:35 - usually most NoSQL databases have secondary indexes that make that kind of querying faster. Great video, thanks
If we can skip the address in NoSQL(4:30) straightaway then whats the difference between error mentioning
Error 1 : No address found
Error 2 : Address field is empty ???