Folks, apologies for the background noise. I never seem to get the tech right 😅 Thank you for watching, I am looking forward to seeing you again soon 🦸
Hey Man, can we get a discount back on the course. I wanted to buy during Festival Sale, but missed it. It hasn't come down since then. Hoping in Diwali there would be some discount.
I'm an AWS engineeer and it's so interesting to see how most of the building blocks are actually used in a similar way for Amazon products. The whole mcsqueal idea is very similar to the Alf journal which is a tier 0 service at AWS.. everything from SQS to Aurora (amazon's spin on the sql db backed by an append only log) to S3 depend on Alf.
Great summary video Gaurav 👏 Few pointers you may consider to cover: * Lease concept to mitigate stale sets and thundering herd to persistent DB * McRouter intermediary component to batch invalidation requests and minimise network congestion * Remote Marker concept to tackle stale set problem arising from eventual consistency during cross-region replication from leader to follower
In final section (data consistency ) at 29:30 , when we are using Bin Logs , how do they resolve data conflicts b/w ind server and us server ? does McSQUEAL handle that or its just rollback ?
Most popular question these days, Design distributed counter where we can see burst of write on counter, multiple solutions 1. range distribution -> 1.1. Once range distributor exhausted all ranges and some range are available on other app server how to borrow order id from neighbor app servers. 1.2. Commit of order since we want atomicity as well. 2. Sharding It is Good topic to cover :)
Replication makes more sense than sharding 🤣 What? How long will you vertically scale each replica? There has to be sharding, replication can't replace it.
Folks, apologies for the background noise. I never seem to get the tech right 😅
Thank you for watching, I am looking forward to seeing you again soon 🦸
Hey Man, can we get a discount back on the course. I wanted to buy during Festival Sale, but missed it. It hasn't come down since then.
Hoping in Diwali there would be some discount.
Thanks for the effort man….your videos are great and are of great help
Cheers!
I'm an AWS engineeer and it's so interesting to see how most of the building blocks are actually used in a similar way for Amazon products. The whole mcsqueal idea is very similar to the Alf journal which is a tier 0 service at AWS.. everything from SQS to Aurora (amazon's spin on the sql db backed by an append only log) to S3 depend on Alf.
Great summary video Gaurav 👏
Few pointers you may consider to cover:
* Lease concept to mitigate stale sets and thundering herd to persistent DB
* McRouter intermediary component to batch invalidation requests and minimise network congestion
* Remote Marker concept to tackle stale set problem arising from eventual consistency during cross-region replication from leader to follower
Thanks Piyush!
thank you Gaurav for teaching us 🙏🏽 This is kind of knowledge is out of bound for us older and self-taught developers.
Cheers!
Beautifully explained, thanks a lot!
Amazing video! Loved it! Hoping to get more videos on whitepaper series soon!
8:49 Engineers are like everyone else [pause] ..... are lazy, that killed me
Great video 👏🏼
Cheers 😁
Thanks a lot Gaurav 🙏 It's always some value addition to my design knowledge 👌 Thanks a lot ❤
Thank you 😁
Thank you sir for this awesome video.
Thank you!
Great video. Could you explain about choosing cache sizes and if its use case dependent or how will it adapt to changing use cases .
What's better than Gaurav explaining a concept? Two Garurav's XD
Cheers :D
why cant they use redis? was redis not there in 2010? or was it not feasible for their usecase?
Redis didn't exist at that time. Memcached came out in 2003, redis took till 2009.
The facebook team was well-versed with Memcached by 2010.
How can sharding be replaced by replication?
Thank you very much bro❤
Amazing paper 😮 31:40
Great content!! Thank you!
Thank you!
wht if they had used configuration provider like kafka for the sharding approach? obvio it was not available then... just a thought...
I don't see how that would help. Could you elaborate on the thought?
So, in replication - we will have replication of whole Facebook database in a cache (muiltple times)? Can you please clarify
We will have as much data from the DB as we can store in-memory.
@@gkcs Incase we don't have that in Cache - we will get it via DB query and get it updated in Cache ?
❤ you brother.
In final section (data consistency ) at 29:30 , when we are using Bin Logs , how do they resolve data conflicts b/w ind server and us server ?
does McSQUEAL handle that or its just rollback ?
They wait for the problem to resolve itself. Eventual consistency.
Most popular question these days,
Design distributed counter where we can see burst of write on counter,
multiple solutions
1. range distribution ->
1.1. Once range distributor exhausted all ranges and some range are available on other app server how to borrow order id from neighbor app servers.
1.2. Commit of order since we want atomicity as well.
2. Sharding
It is Good topic to cover :)
8:51 : shahrukh khan vibes
🙂👍🏻💯
Replication makes more sense than sharding 🤣 What?
How long will you vertically scale each replica? There has to be sharding, replication can't replace it.