can we write recent changes in the cache in when it comes to discard the entry from the cache we write that changes to the database, this way our db does not get behind because first we will be checking for cache, if it is in cache then it is latest, by the way great video sir i admire you
Caches are used to cache repeated queries so that you do not invoke hard compute and return pre calculated values, its usually not designed to reduce latencies. Latency reduction is the by-product of the less compute required to answer the question. Keep that in mind. An SQL query requires alot more compute than returning it from file based or in-memory cache, it's faster and very efficient. Poor cache performance is often due to bad invalidation logic, limited cache sizes or taxing data structures stores inside a cache system. Caching, Deadlocks, Synchronisation, Memory Management and Scheduling are the most simplest yet the hardest concepts in computer science if you don't do it right. The introduction only speaks about keywords at this point, how to make them work for you is the real engineering esp in a "distributed system" where caching is completely different concept comapred to single node or centralised caches. Redis as displayed in this illustration is still a centralised cache if all servers use it as server will see it as single entity behind sentinel cluster. That being said, go and try yourself and see how it works rather than rely on half baked information.
@@gkcs "At a high level Caching reduces latency by just using more storage", that's not even close to compute use-case. You can have a single 4 / 8 byte counter read by thousands of nodes relying for citicial infrastructure information or crutial service data to function correctly, good luck syncing it in distributed system / caches while taking about little space. You simply forgot to talk about actual limitations and now pointing out parts where it does not make sense. There are tons of other usecases where data storage is so small while being cached but it's importance and chance of causing impotency to system is very high, caches do store both kinds of data wherever applicable and they are equally important while designing realworld systems.
@hmmmm4193 Practice, implement, and read docs and source of whatever stack you're working. If you're not familiar with programming concepts like SOLID. DRY concepts familiarize with it first. Learn as much as OS concepts like Filesystem, Processes, Networking, Memory Management, and Process Scheduling in order to know where you can make your code faster or how the application will behave under load. After learning all these along with high level stack functions such as how Application server works, How Web servers are different from App servers, the protocols behind these servers, message protocols and message brokers, async concepts in systems and many other things like database abstraction layers, service containers and service mesh, service registries, frontend application layer ( the API layer, not to be confused with web app frontend ), service layers and db and storage layers to name a few. If you have worked even a bit with these, you'd have enough knowledge to go deeper in each sub application to know their strengths and weaknesses and utilise them at your advantage. That's where software engineering ends and system design starts. So first be a good programmer > Followed by becoming a good software engineer and then transitioning into a system designer. It's a long journey. The more you know about different systems implementing the same stuff, the better, as you can see, tradeoffs between them, and choose the right one for usecase. The more you explore and exercise over the years, the better. While doing it learn how algorithms are utilised at core of each solution. Books and docs are your resources. The more you read about these, the more you'll learn. Some things will ne learned by practice and by experience. Don't be afraid to fail and implement code over and over with a different approach until you find a good fit as theories may not be the same as practical solutions.
Please keep making these videos and continue this journey of system design in depth they're extremely helpful
Waiting for this video for a long time 🎉
Thank you
Best video on Caching
Bought your system design course after this video, I really hope it will be worth it and I am excited for this journey!!
Cheers!
Really like your examples used while explaining something. Aspiring to become an engineering like you😀
Beautifully explained. Please keep sharing such imp videos and if possible attach PDF links too where can read further about the topic. Thanks.
One correction: Videos are not stored in the database; instead, they are stored in the file system.
a database can be an RDBMS, or an object-store, or a file/collection/document store.
How to handle data updates on cache? Would help if you show strategies too
can we write recent changes in the cache in when it comes to discard the entry from the cache we write that changes to the database, this way our db does not get behind because first we will be checking for cache, if it is in cache then it is latest,
by the way great video sir i admire you
i have one doubt that can we call cache as an database ?
Cache is just data. It maybe stored in a database, but it is just storing data in a way so that we can access it faster for future requests.
@@failureboy8993 you can't because database persistence is different from cache store. E.g ACID
Caches are used to cache repeated queries so that you do not invoke hard compute and return pre calculated values, its usually not designed to reduce latencies. Latency reduction is the by-product of the less compute required to answer the question. Keep that in mind.
An SQL query requires alot more compute than returning it from file based or in-memory cache, it's faster and very efficient. Poor cache performance is often due to bad invalidation logic, limited cache sizes or taxing data structures stores inside a cache system.
Caching, Deadlocks, Synchronisation, Memory Management and Scheduling are the most simplest yet the hardest concepts in computer science if you don't do it right. The introduction only speaks about keywords at this point, how to make them work for you is the real engineering esp in a "distributed system" where caching is completely different concept comapred to single node or centralised caches. Redis as displayed in this illustration is still a centralised cache if all servers use it as server will see it as single entity behind sentinel cluster.
That being said, go and try yourself and see how it works rather than rely on half baked information.
3:40
@@gkcs "At a high level Caching reduces latency by just using more storage", that's not even close to compute use-case.
You can have a single 4 / 8 byte counter read by thousands of nodes relying for citicial infrastructure information or crutial service data to function correctly, good luck syncing it in distributed system / caches while taking about little space.
You simply forgot to talk about actual limitations and now pointing out parts where it does not make sense.
There are tons of other usecases where data storage is so small while being cached but it's importance and chance of causing impotency to system is very high, caches do store both kinds of data wherever applicable and they are equally important while designing realworld systems.
@@MaulikParmar210 Man, stfu. The video made total sense to me. You sound like a person desperate to explain how intelligent you are.
@hmmmm4193 Practice, implement, and read docs and source of whatever stack you're working. If you're not familiar with programming concepts like SOLID. DRY concepts familiarize with it first.
Learn as much as OS concepts like Filesystem, Processes, Networking, Memory Management, and Process Scheduling in order to know where you can make your code faster or how the application will behave under load.
After learning all these along with high level stack functions such as how Application server works, How Web servers are different from App servers, the protocols behind these servers, message protocols and message brokers, async concepts in systems and many other things like database abstraction layers, service containers and service mesh, service registries, frontend application layer ( the API layer, not to be confused with web app frontend ), service layers and db and storage layers to name a few.
If you have worked even a bit with these, you'd have enough knowledge to go deeper in each sub application to know their strengths and weaknesses and utilise them at your advantage. That's where software engineering ends and system design starts. So first be a good programmer > Followed by becoming a good software engineer and then transitioning into a system designer. It's a long journey. The more you know about different systems implementing the same stuff, the better, as you can see, tradeoffs between them, and choose the right one for usecase.
The more you explore and exercise over the years, the better. While doing it learn how algorithms are utilised at core of each solution.
Books and docs are your resources. The more you read about these, the more you'll learn. Some things will ne learned by practice and by experience. Don't be afraid to fail and implement code over and over with a different approach until you find a good fit as theories may not be the same as practical solutions.