Thanks! To be fair nobody (even me as a programmer) are suddenly switching a reverse proxy like Nignx and built their own using Pingora. What about dealing with specific location/paths? What about implementing rate limiting? What about static content? If you use a reverse proxy like Nginx across multiple domains and projects, you don't gonna implement a reverse proxy from the ground up. These are just two fully different ways. Mostlikely most engineers already need to spend a lot of time on the programming on the applications itself, Pingora would be a project on it own.
@@AntonPutra So Ingress could be Nginx, HAProxy, Traefik and more.. But even k8s for a homelab is for some people already next level. Some just have bare metal or a simple Proxmox VM with Nginx..
@@stxnw I'm not "just" building a homelab. First of all I work for a big enterprise. And second of all, I basically run a datacenter at home within server racks.
Great video as usual! would be a bit different from performance benchmark. but I'd really love the comparison between serverless (e.g. AWS Lambda or Fargate) vs serverfull (e.g. EC2) in terms of performance, latency, and ultimately cost. I think when it comes to hundreds or even thousands of RPS, it makes sense to compare between these two.
Great video! I'd be curious to see how h2o compares to Nginx; their benchmarks show them beating Nginx, and it ranks highly on the Techempower framework benchmarks.
Small tip for Graphs, set Tooltip mode to "All" and Values sort order to "Descending". That way you can very easily see values of all lines on the graph when you hover over it. Making it way easier to see the actual differences. Edit: I see you did that with Network traffic, but it would be nice to have on all graphs.
@@vuongphaminh2293 He'll get to it eventually. It has its own configuration learning curve since it has more features than NGINX. Plus running the test for hours on end.
Very interesting. Never heard of Pingora before, but being programmable is certainly an advantage. Instead of waiting for the supplier to add that feature to their custom config DSL you just add it yourself. Looks very robust and reliable.
@@j-p-d-e-vyeah its mainly used by cloudflare, the maintainers. i wish they'd document more but i can imagine that will slowly improve since it was only open sourced not too long ago
This is wonderful! I think it would be cool to benchmark nginx versus river (which is a new proxy the isrg is building to be a feature complete off the shelf proxy to replace nginx) I think it's in beta but river is based on pingora and would be a suitable alternative for someone who's wanting a prre-built binary
What I miss here, the way of build the proxy is really imperative you really need to program, It will be nice a framework base on Pingora which you just provide a yml or json file with the configuration and Pingora just do the Magic. Really nice benchmarks. Thank you.
Thank you, this benchmark comparison between Pingora and Nginx was incredibly insightful. I appreciate the detailed analysis and the effort you put into making this video.
i'd hoped the test would continue until the breaking point of pingora to see its overloaded behavior. iirc nginx is more preferable due to connection blocking when overloaded
its main claim to fame is that basically all of cloudflare runs on it, they're the ones who made it. it was only open sourced recently so its alsp pretty new.
Python is slow and a pain to setup in general for webservers. Last time I was working with FastAPI Gunicorn using Uvicorn (asgi) workers was vastly faster than uWSGI. Don't use uWSGI if you can do Async/Await.
It would be interesting to see results when HTTP is used instead of HTTPS, because HTTPS could take significant percent of CPU. With HTTP differences will be probably more emphasized.
@twitchizle CPU usage is based on the average core usage. If your app can only use one core but two ar available, your app will have a bottleneck with only 50% CPU usage.
Bs, you confusing CPU utilization with CPU usage. And in current test , rust based proxy is less effective both by cpu usage and per watt effectiveness
Try to make python again but using socketify it is much faster compared to go fiber, gin, gorilla and others, socketify is a python framework that uses c++
no way it is faster than go lmao, if you just directly return json then you dont replicate real world load because in real world you write logic first and you write it in pure python, no c/c++ help anymore, and then in the end of the day your app is always slower than compiled languages because python's interpreting proccessing in runtime that slow down your app, and python's dynamic type that also slowdown your app, the more logic you write the more your app become slower again and here we know who is the real winner in suct a reality workload not suct imaginable workload
@@aryindra2931 you should learn that only some function that will call c++ code. all your python code, the logic, etc will be executed by python interpreter which is slow, adding the memory management and garbage collection overheat, dynamic varible checking, the more you create python code, the more your app will be very slow
So nginx is almost as fast while having a thousand times more plugins, features, community, business support and LLM training. Pingora would have to be 10x more performant to be worth considering. Like some Specialized C/C++ servers.
Pingora is a framework, you can do tons of stuff with it and load balancer was only one usecase. Nginx is nothing more than just a reverse proxy and creating new plugins for it is not that easy.
Agree. Nginx does the job and does it well while having a much larger community for support. If you need more performance, you can scale it up. If you're running an application that needs this kind of throughput, you can definitely allocate more resources to your proxy.
not only that, you'd have to learn rust and learn the pingora api to make it work. It would take you like 5 years to write the perfect pingora proxy config :D
considering Cloudflare built pingora since nginx wasn't enough from a performance perspective but also ability to quickly implement new protocols etc., i'd expect it to perform well on high workload.
🔴 To support my channel, I'd like to offer Mentorship/On-the-Job Support/Consulting (me@antonputra.com)
Would like to request HAproxy with nginx
Did you use a release build with pingora? Aka `cargo build --release`
Please test Pingora against Oxy, both are from Cloudflare, both in Rust
nginx ingress controller vs istio
Thanks for these benchmarks. Now I know what framework to use for my BLAZINGLY FAST, high-performance web app (it’s a CRUD app with 10 users)
No it's 0 user
it's a todo app
i got 10 devs in my team and about 3-5 daily users :D
⚠If you can improve Nginx or Pingora, please send me a PR, and I'll rerun the test! ⚠
With the mention of disabling the logs to create a fair comparison, why don't you compare nginx with and without logging?
@@gizlas I think each test cycle costs $50 bucks?
Thank you for making this benchmark video. I had been wondering how these would compare.
Thanks! To be fair nobody (even me as a programmer) are suddenly switching a reverse proxy like Nignx and built their own using Pingora. What about dealing with specific location/paths? What about implementing rate limiting? What about static content? If you use a reverse proxy like Nginx across multiple domains and projects, you don't gonna implement a reverse proxy from the ground up. These are just two fully different ways. Mostlikely most engineers already need to spend a lot of time on the programming on the applications itself, Pingora would be a project on it own.
agree, if you are at the scale of cloudflare then it makes sense, otherwise just use k8s + ingress when needed
@@AntonPutra So Ingress could be Nginx, HAProxy, Traefik and more.. But even k8s for a homelab is for some people already next level. Some just have bare metal or a simple Proxmox VM with Nginx..
@@MelroyvandenBerglol you shouldn’t be on this channel if you are only building a homelab
@@stxnw I'm not "just" building a homelab. First of all I work for a big enterprise. And second of all, I basically run a datacenter at home within server racks.
@@MelroyvandenBergSame but just the last part, lmao
Nice!!! :)
I was waiting for this one, thanks ^^
Great video as usual!
would be a bit different from performance benchmark. but I'd really love the comparison between serverless (e.g. AWS Lambda or Fargate) vs serverfull (e.g. EC2) in terms of performance, latency, and ultimately cost. I think when it comes to hundreds or even thousands of RPS, it makes sense to compare between these two.
Would really love to see this as well
I love the way how you explains things in your video! Very easy for to understand.
Your name kind a Indonesian name :)
Great video! I'd be curious to see how h2o compares to Nginx; their benchmarks show them beating Nginx, and it ranks highly on the Techempower framework benchmarks.
I would like to see h2o's benchmark too!
Small tip for Graphs, set Tooltip mode to "All" and Values sort order to "Descending".
That way you can very easily see values of all lines on the graph when you hover over it. Making it way easier to see the actual differences.
Edit: I see you did that with Network traffic, but it would be nice to have on all graphs.
can you do HAproxy vs Nginx ?
we've asked repeatedly but... he didn't do it
@@vuongphaminh2293 we need HAproxy since it's used more than Pingora and HAproxy is a direct competitor of Nginx as reverse proxy
@@vuongphaminh2293 He'll get to it eventually. It has its own configuration learning curve since it has more features than NGINX. Plus running the test for hours on end.
I'm interested in this also since those two are C based.
HAproxy is faster in every metric. No comparison.
Please @AntonPutra, benchmarks Swift hummingBird/vapor vs golang or etc.
ok :)
Very interesting. Never heard of Pingora before, but being programmable is certainly an advantage. Instead of waiting for the supplier to add that feature to their custom config DSL you just add it yourself. Looks very robust and reliable.
its nice but the documentation of it sucks.
@@j-p-d-e-vyeah its mainly used by cloudflare, the maintainers. i wish they'd document more but i can imagine that will slowly improve since it was only open sourced not too long ago
Thank you that wass perfect as always.
Thank you Anton
This is wonderful! I think it would be cool to benchmark nginx versus river (which is a new proxy the isrg is building to be a feature complete off the shelf proxy to replace nginx) I think it's in beta but river is based on pingora and would be a suitable alternative for someone who's wanting a prre-built binary
This is interesting, in terms of absolute numbers 40k request per second doesn't seem much. Are proxies more resource intensive than regular servers ?
What I miss here, the way of build the proxy is really imperative you really need to program, It will be nice a framework base on Pingora which you just provide a yml or json file with the configuration and Pingora just do the Magic. Really nice benchmarks. Thank you.
thanks! yes, using pingora is a project on its own
please create SQL vs NOSQL comparison! e.g. Postgres vs Mongo
would be great comperision.
Keep up the good work. Respect 😊
Test haproxy , nginx and pingora
nice job 👍
nginx vs envoy please
0:30 I can see from blurred images that Nginx is green 😅
Congrats, now watch it until end
Hey, have you heard River ?
a Reverse Proxy build on top of Pingora. Please compare with nginx.
Webflux vs. Virtual Threads next?
ok soon
awesome! this one is really useful info!
Thank you, this benchmark comparison between Pingora and Nginx was incredibly insightful. I appreciate the detailed analysis and the effort you put into making this video.
Why does nginx request per second plateau then nosedive around 60% cpu utilisation? I would expect it to scale like pingora.
i'd hoped the test would continue until the breaking point of pingora to see its overloaded behavior. iirc nginx is more preferable due to connection blocking when overloaded
It did reach a breaking point at around 47k RPS
terima kasih anton
HAProxy vs Pingora next?
Standard, tried and proven things always beat new and shiny :)
The only exception is when there's 10x improvement.
wow, nice thing, never heard about pingora
its main claim to fame is that basically all of cloudflare runs on it, they're the ones who made it. it was only open sourced recently so its alsp pretty new.
Prometheus vs InfluxDB, performace, memory usage, disk usage🥰
can we do Actix with actors (default) vs Actix with websocket
ok i'll take a look
Took a peak at the sources; how do you build aputra/load-tester ?
it's a rust client instrumented with prometheus metrics, i have similar clients in my repo
@@AntonPutra Are the sources public for it? I looked in the repo but couldn't find.
Hello Anton,
Can you make a video comparing NGINX with OPENLiteSpeed?
Thank you in advance.
sure, added to my list
🐍 uWSGI VS Gunicorn
Python is slow and a pain to setup in general for webservers. Last time I was working with FastAPI Gunicorn using Uvicorn (asgi) workers was vastly faster than uWSGI. Don't use uWSGI if you can do Async/Await.
It would be interesting to see results when HTTP is used instead of HTTPS, because HTTPS could take significant percent of CPU. With HTTP differences will be probably more emphasized.
🍿 Benchmarks: th-cam.com/play/PLiMWaCMwGJXmcDLvMQeORJ-j_jayKaLVn.html&si=p-UOaVM_6_SFx52H
good work
Thank you.....
Раст - сила, JS - могила
haha
Keep in mind that higher CPU usage sometimes mean that it has a better resources usage
What, no. This aint game FPS benchmark
true
@twitchizle CPU usage is based on the average core usage. If your app can only use one core but two ar available, your app will have a bottleneck with only 50% CPU usage.
Bs, you confusing CPU utilization with CPU usage. And in current test , rust based proxy is less effective both by cpu usage and per watt effectiveness
@@s.i.m.c.athis proxy is multithreaded unlike nginx that single threaded. this proxy is used in cloudfare that server many milions trafic
Proxies are so complicated, to swap another due latency, especially on complicated systems with load balancers etc...
true, most companies don't need 'another' proxy. it only makes sense for those that have business around it
finally a worthy opponent for nginx
Not really. Nginx is still much more advanced
It is so over for nginx 😂
Rust is top!
Facts! 🦀
Try to make python again but using socketify it is much faster compared to go fiber, gin, gorilla and others, socketify is a python framework that uses c++
no way it is faster than go lmao, if you just directly return json then you dont replicate real world load because in real world you write logic first and you write it in pure python, no c/c++ help anymore, and then in the end of the day your app is always slower than compiled languages because python's interpreting proccessing in runtime that slow down your app, and python's dynamic type that also slowdown your app, the more logic you write the more your app become slower again and here we know who is the real winner in suct a reality workload not suct imaginable workload
Why don't you try it first?
@@aryindra2931 because i already know. if you know how a language work, you dont need to try it
@@aryindra2931 you should learn that only some function that will call c++ code. all your python code, the logic, etc will be executed by python interpreter which is slow, adding the memory management and garbage collection overheat, dynamic varible checking, the more you create python code, the more your app will be very slow
Pingora (Rust) vs YARP (C#)
obviously rust beat the shit out of yarp lol. pingora is used in cloudfare services that serve many many milions traffic
ScyllaDB
Pls do C# .NET 8 vs Node.js
So nginx is almost as fast while having a thousand times more plugins, features, community, business support and LLM training.
Pingora would have to be 10x more performant to be worth considering. Like some Specialized C/C++ servers.
Pingora is a framework, you can do tons of stuff with it and load balancer was only one usecase. Nginx is nothing more than just a reverse proxy and creating new plugins for it is not that easy.
I agree with you that for 99% of the cases it is true, but if somebody have the time and money to get something more performant than why not
Agree. Nginx does the job and does it well while having a much larger community for support.
If you need more performance, you can scale it up. If you're running an application that needs this kind of throughput, you can definitely allocate more resources to your proxy.
not only that, you'd have to learn rust and learn the pingora api to make it work. It would take you like 5 years to write the perfect pingora proxy config :D
@@isaacdruin
scale up cost money
stick to caddy if you have a life and sometimes a girlfriend
wow pingora is way too powerful
It’s not pronounced thouthand, it’s thousand
Ok, my conclusion: nginx started using more cpu, was slower (latency) at the end , so on high workload -pingora works better.
considering Cloudflare built pingora since nginx wasn't enough from a performance perspective but also ability to quickly implement new protocols etc., i'd expect it to perform well on high workload.