How NETFLIX onboards new content: Video Processing at scale 🎥
ฝัง
- เผยแพร่เมื่อ 5 มิ.ย. 2024
- Everyday, #Netflix handles billions of requests regarding movies, trailers and other video content. Delivering at such a large scale needs an #engineering marvel. This #video will talk about how Netflix is able to onboard new video content onto their platform. We go from video chunking to collating 4 second shots into scenes.
Amazon S3 is used to store the video chunks. Netflix also provides open connect servers to internet service providers, which acts like a cache of movies. Most requests to Netflix can be served by this cache, and the remaining are sent over the network. This reduces the bandwidth and time required for Netflix to operate at scale. Synergy at it's finest.
00:00 Problem Description
00:32 Video formats and resolutions
03:18 Chunk processing
05:52 Storage
06:19 OpenConnect for video caching
10:13 Summary
Looking to ace your next interview? Try this System Design video course! 🔥
interviewready.io
With video lectures, architecture diagrams, capacity planning, API contracts and evaluation tests. It's a complete package.
References:
/ high-quality-video-enc...
/ optimized-shot-based-e...
/ video
/ stream
/ keystone-real-time-str...
media.netflix.com/en/company-...
System Design Playlist: • System Design for Begi...
You can follow me on:
LinkedIn: / gaurav-sen-56b6a941
Instagram: / applepie404
Quora: www.quora.com/profile/Gaurav-...
Twitter: / gkcs_
External content used:
Fast and Furious 7 (Car chase scene)
Boone Loves Video: • 🌎Create 3D PATH ANIMAT...
Sacred Games (Available on Netflix)
Johnny English Reborn (What's it called?)
Netflix Logo
I believe there is a variation to Gaurav's understanding on open-connect caching mechanism. A new uploaded video DOES NOT goes directly to the caching server, instead they are resolved to a 'a small Netflix DB replica" in that region, which in-term populates the caching layer for new videos or most frequently accessed videos. The distinction is important because otherwise the design is assumed to directly populate OC CDNs, which is not the case. So for example, if a viewer in Australia wants to stream a non-cached US content:
Instead of directing all the Netflix traffic through under-sea cables, Netflix copies the video file (transcoded) from US-based storage to a storage location in Australia once during off-peak hours. Once the video has reached the continent, it’s copied to all the Open Connect servers present in the ISP networks.
This is not to point out a flaw in the video, just a clarification.
Thank you for this amazing clarification 😁
@@gkcs I am glad. Keep making fun content😄
Content apart , the new animations that you've added are really nice. Extra efforts are appreciated!
Keep em coming !
Man, you know how to teach with fun! I must say presentation and video editing is very well done in your videos.
Moral : Steal that Open Connect Box
Wow, excellent video. Love your unique style of breaking down the topics in chunks and explaining them neatly. Keep them coming.
holy shit! Just fall for these videos; Thank you from bottom of my heart! Please make more video on this topic!
Bro - Thanks for the great video! Learned a lot. Just subscribed. Keep the great content coming!
It's amazing to see how Guarav improves the quality of his content, and I can tell you, guys, as a newbie tech TH-camr, it's a big deal. Keep it up, bro! ;-)
Thanks Victor!
What a beast! Loved your reference to Amazon S3 out of nowhere! 😎✔️
The material you have in this playlist is gold! I did not expect to learn so much from it. Thanks a lot!
You're welcome!
Keywords = CDN , predictive content delivery , content pre-fetch and so on...
Yes Content Delivery Network plays a really important role if the geographic location of physical server is far and also I think what these OTTs would be doing is what we are getting in recommendation might be based on geo location rather than your watch history , this way if one guy from ur location has seen some movie it gets cached to a CDN and then when u request for the same content , since it comes in ur recommendation its extremely fast
Showing Gupt in local popular movie would make Booby Deol really Proud. Savage AF Gaurav! :D
Blown by the simplicity you explained such a complex topic!
Very helpful and simple for anyone who is preparing for software design interview. Your way of explaining is interesting and funny 😊
Whoa!!! Thank you so much for making such unique and great content videos ;)
Another informative video from you. Nice explanation. Thanks. Just wanted to add something on the caching part. Many websites take help of CDN providers like Akamai to do the caching on behalf of them. The CDN providers have the required infrastructure across the globe wherein they have placed their caching servers in most of the countries.
That's interesting, thanks!
Love the energy. Your videos are great.
Don't be surprised if you hit 500K subscribers by the end of 2019. Your content quality is skyrocketing.
Thank you!
@@talesara74 It's system design and it's not for everyone to understand, what did you expect? People mainly use this channel for technical system design interviews.
You said that from your perspective. But the sad reality is People really don't like good contents😅😂. They can watch a 3hrs movie, but couldn't watch this type videos for 10mins😑
Gaurav I love your content and soon I guess will be joining your InterviewReady platform...Please offer me some special discount bhai 😍
It's almost 2023 and he still has 488k 🥲
Hi Gaurav, thanks for the video. Impeccable. Keep doing the great work!!
You have the most unique content on TH-cam, man. No bs, pure knowledge. Keep going.
Thanks!
Thanks for your descriptions
Cheers!
Awesome man! Simple and to the point. The best kind of online content.
To be honest learned something useful today, and willing to learn from you more like this kinda of topic!
I'm very impressed! Great explanation! I just subscribed! Keep going!
1:23 AWESOME GAURAV, perfect example of what you are explaining, Huge respect bro
You’re awesome. Thanks 😁😍 this was so helpful
Have been working in this arena for a while now, you got everything correct man, other stuff you post usually goes over my head because I haven't dabbled in a lot of those things but for once, felt nice to already know what you were gonna say. Haha.
Thanks! Your replies on the comments are excellent btw :D
As i have some experience in this area, i would like to say that you delivered crisp content on the same. Good job!
Thank you!
Thank You for sharing. Very nice details and very good description
Gaurav, I can't tell how interesting this video was ! Thanks a lot ! I totally enjoyed learning this ! Keep making more such videos ! :)
Thanks!
Wow this is sweet! Super smart solution to the issues of video streaming, will be interesting to see where it is improved upon in the future too!
great efforts thanks a lot ! i remember my childhood days after seeing the GUPT poster! :-)
I was waiting for this video, awesome
Thanks for this wonderful insight on the engineering side of Netflix, looking forward to more system design videos.
Thanks!
That was a nice one !
Thanks for the video.
I just started watching your videos. Your content is Amazing.
Champion presentation. Just wanted to see a graphical representation of the adaptive bit rate ladder through Netflix's eyes.
That bit at the beginning was hilarious 😂
Awesome bro ..!! Simple and very informative.. loved it .. keep it up 👍
Short, simple and full of interesting concepts. Thanks!
Thank you!
Your videos are pure delight to watch !
Thank you!
Nice and depth knowledge bro!
Very nice explained and made it interesting. Will be waiting for more good system design question.
Very detailed. Great work!
must say u made it really Interesting with editing :P and great content!
Thank you!
ISP cache is really interesting, wondering how they have handled security on top of it across multiple geography??
Thank you Gaurav for the content . Hope to see many more useful content .
that's pretty interesting concept... very well explained 👏👏.....
Thank you for your reading on this topic...this is enlightening (at least for me)
Nice explaination, Thank you so much.
this isp cache thing is just simply amazing,
Thank you for the explanation
This was a very informative video! Awesome job.
Great, Nicely explained !!!
Great video ! Thanks for sharing
Brief and precise.. Nice.
Excellent video, well explained!
As always one another brilliant content
I just love your system design videos.
Thanks 😁
Please do some lectures on docker internals like Cgroups, namespaces, union capable file systems
OMG!!!! the opening😂😂
Good job explaining!!
Can you explain some of the advanced concepts of video streaming too? For eg., Bandersnatch? I wonder how Netflix is providing conditional streaming of video chunks based on user selection.
Outstanding 😨😱
Great content always!
Thanks!!
Paaji ek dum kamaal ka video hai ...even understoodable for non cs
The content of this video was very dense. The breather at 5:29 was very much wanted :-). You have very good audience engaging skills. Good Luck !!
This is great work Gaurav. Thank you for this!
😁
Thanks Gaurav. I am not an engineer but always wanted to get some resource to learn the concepts. I would love to see more of your videos and on advance concepts.
Thanks Ankit!
Superb explanation bro
Fantastic explanation
Thank you!
That's just awesome
Keep it up
real effort here, good job mate
Great breakdown!
Hey Gaurav,
I was just wondering how the Open Connect or TH-cam Red boxes are any different from the long existing concept of CDN (Content Delivery Network). The only change is, being big companies they have the money and resources to create their own personal CDNs and brand them which is kind of building your own house instead of renting one. Is there something I am missing here ?
They are very similar, yes.
I was having the same question as how these Oppen connect boxes are different from CDNs.
Happy Teachers Day Sir! U taught me a lot! I really appreciate that!
Thank you!
Bro u are great am a 14+ software engineer really appreciate ur content
Very well explained, thank you, Gaurav!
Thanks Veronica 😁
Awesome!!
Video -> Fragmentation -> ABR -> DASH + HLS -> Packaging -> Cache On CDN -> Play
Very informative!
So well explained 🤟
Really helpful man!
Not gonna lie, you got me in first second, sorting in O(n) lol
:) Thanks this is amazing
Great Sir,
Great presentation skills and content🔥🔥🔥
Nice work.
I was the first one to watch this video after him!
The content had no views but a like!!
That's a improper system design for TH-cam, probably slow.
Too good... sir
loved it
Thats a really great explanation. Thanks
Thank you!
Gupt yrr, haha Very wonderfully explained Thanks for that
Awesome Video. Very good explanation. Video content caching is not an easy job. The video is really good.
Software Architecture is the most important thing today. No matter how much you you improve on ML algos or buy hardware etc... It is the archtecture that defines quality and reliability. How you connect components together using a good deisgn tactic and make each component work flawlessly by the deisgn pattern matters the most. It is archtecture everywhere. The fact that you make videos on it is really a good thing. Developers today must understand the imporatnce of it. It is not just writing code but many other things too.You are doing a great job by making such videos.
You are God of System Design!!
Good explanation!
Great content.. keep up the good work..
For some reason, the sparse movies that I've watched were also A-rated.
Legends understood what you mean.