Serving Netflix Video at 400Gbps on FreeBSD by Drew Gallatin

In this talk, I will discuss the efforts to serve TLS encrypted Netflix video at 400Gb/s from a single server. This will be a follow-on to 2 talks at the 2019 EuroBSDCon: “Numa Optimizations in the FreeBSD Network Stack” and “Kernel TLS and TLS Hardware Offload“. I will provide background on the Netflix video workload, and define key technologies such as NUMA, kernel TLS and hardware kTLS.
I will describe encountering bottlenecks such as:
- Memory bandwidth limits for software kTLS
- PCIe issues with hardware kTLS
- NUMA for software vs hardware kTLS
I will present current and historical performance results from at least:
- AMD “Rome” 2nd generation EPYC systems
- AMD “Milan” 3rd generation EPYC systems
- Ampere Altra arm64 systems
Drew Gallatin
Drew started working on FreeBSD at Duke in the 90s, and was one of the people behind the FreeBSD/alpha port. He worked on zero-copy TCP optimizations for FreeBSD and was sending data at over 1Gb/s before gigabit Ethernet was generally available. He spent a decade at Myricom, optimizing their drivers. After a brief hiatus at Google, he landed at Netflix, where he works on optimizing the FreeBSD kernel and network stack for content delivery. He worked on the optimizations to serve unencrypted Netflix traffic at 100Gb/s, and then on more optimizations to send encrypted traffic at ever increasing speeds, from 100Gb/s to 400Gb/s and beyond.

[@Kozical]

Audio is misaligned but very good talk.

[@TheOpenBSDguy]

Very interesting, thanks

[@kavya1638]

FreeBSD is the king of Operating Systems.

[@NickDyers]

... said no one

[@diegonayalazo]

Thanks

[@NickDyers]

What an intricate sorcery behind Netflix stream!
now, would you kindly explain to this unfortunate muggle how to watch Netflix from my FreeBSD box?

[@mikejakubik]

Anyone know what file system/RAID solution they use?

[@kristofprovost]

They're on FFS, and each drive is its own filesystem. They don't care about data integrity because these are cache devices. If they lose data clients can just get it elsewhere. All that matters is performance.

[@mikejakubik]

@@kristofprovost Makes sense, ty.

[@petegiant]

UFS for filesystem