► What should I test next? ► AWS is expensive - Infra Support Fund: buymeacoffee.com/antonputra ► Benchmarks: th-cam.com/play/PLiMWaCMwGJXmcDLvMQeORJ-j_jayKaLVn.html&si=p-UOaVM_6_SFx52H
I’d like to see the performance of an Golang Echo server in a comparison please. Templating html or doing a database query would be realistic tests IMO!
Hello, the second test feels sus! Why does the s3 upload take so much time? Is the Upload blocking? Can you replace it with an async wait and try again?
I think this is a perfect analogy for go vs rust. You can get nearly as far as any language with Go with (relatively) little knowledge. You can get even further with deep knowledge of Rust.
@@curio78 of course Java is slower than C or Rust.. Java is generally 7th to 12th fastest, depending on the benchmark (and the depth of languages tested) & is usually sitting behind Go. More code for a slower result.. I'm not seeing the draw card of Java. People are generally moving away from Java, not to it. Doesn't mean it doesn't serve a purpose, but speed & simplicity are not its' strong points.
It is amazing to me how well go keeps up even being garbage collected. Glad you did this video. I think it’s much more fair to what rust is capable of. That said I would argue that most teams would be better served by go especially if you don’t have +10k rps problems and an extremely well defined problem space for your application. That’s because it was easier for you (and most devs) to get a fair well optimized application with go first time around. I find in the real world most projects only get one serious pass at the time it’s initially created.
@@anonymousalexander6005 I don’t think that’s true. As someone who’s doing backend js begrudging at work if you are running in kubernetes golang start up time is a real advantage. No JIT needed either. Not to mention the memory usage. I can run a 3 go servers on the smallest ec2 instance no problem. Can’t say the same for JavaScript.
Depends on the kind of work you do. Complex logic is better served by a compiler able to do more compile time checks and a language which has a much higher expressive power. Few real world workloads are as simple as serving constants or uploading some stuff to multiple places.
The cost of writing and maintaining software is more paramount than the perf metrics in respect to, say, a 5% difference - for most software projects. There are systems software and infrastructure software that can warrant being written in a manner to achieve maximal perf, and/or efficiency of resource usage, and/or have stringent requirements on deterministic behavior. But that’s a rather narrow category of software projects. The majority of software projects are of a nature that can tolerate an automatic garbage collector - ala Golang
@@TheSulross You're right. You don't learn Rust just to write such code that is served well enough by Go. You use Java for that 😀 OTOH, once you have learned Rust, it's a much better language than Go for just about anything. My gripe with Go is very specifically its rather poor expressiveness. More expressive languages allow you to write less code overall. Effort aside (which is only poorly correlated to lines of code anyway), fewer lines of code translate to fewer bugs - there are studies that show that all other things being equal, regardless of language used, bug density is proportional to lines of code.
I found Axum to be really simple, if a bit immature, didnt feel like I was much slower that with other languages and frameworks. But maybe that just means I suck at all of them 😂😂
@@jonnyso1No Axum is pretty straightforward. It has a really good balance of exposing features (not hiding everything behind macros like rocket), and being simple enough. I think that (As long as you’re working with REST APIs and not holding state), Axum and Go are ~equal in difficulty here. If you need to get more complicated Go will likely be easier to use but rethinking your design might also help.
@@houstonbova3136I gave up on frameworks because I didn’t like any of them. I just use hyper directly (what all these frameworks for Rust use anyway), and get absolutely *silly* performance out of it. With just a few simple macros I wrote myself it does everything it needs to do, and I have absolute control over everything.
@@jonnyso1using axum may as well be easy and straightforward, but it does not mean that all other parts of your Rust program are going to be the same.
I believe the kubernetes CPU throttling is giving this benchmark an unfair comparison of the latency. Go is not aware of the cpu limit that you gave it. in general you should avoid cpu limits since how the kernel applies this limit will cause high tail latency, which can also explain why the throughput goes from 12K/s to 7K/s when throttling starts. Would like to see the same test but with rust and go on their own nodes which they can use fully (so without any cpu limits). And would also be interesting to see them running on one node where they both get the same amount of cpu request but again no limit to avoid throttling, the requests will still make sure that both can equally use the cpu
@@erkintek You can set the GOMAXPROCS and GOMEMLIMIT env vars according to the pod cpu and memory limits respectively (rounded down to the nearest integer in the case of GOMAXPROCS). I do this regularly with GOMEMLIMIT (sometimes also tweaking GOGC) but haven't yet tested GOMAXPROCS. But the theory seems sound: Goroutines will be scheduled over only that many OS threads, so there should be less thread contention which improves tail latency. Uber has a package called automaxprocs to handle this automatically, and they measure significantly better p99.9 latency. Anton, this could be an interesting thing to test.
Hi Anton! I’d love to see a comprehensive course on observability and monitoring from you, covering everything from basics to advanced topics. It would be amazing if you could include: what to monitor (applications and clusters), how to monitor (tools and techniques), key metrics for applications and clusters, insights from 12+ years of experience, and an overview of metrics, logs, traces, OpenTelemetry, Jaeger, and how to create custom dashboards in Grafana. Also, a section on implementing monitoring for different programming languages would be incredibly valuable. Your approach and expertise in monitoring are unmatched on TH-cam-I'm sure your course would offer insights that no one else provides. Thanks!
Hey! awesome one! I took a look into the rust code and I saw that each time a new prepared statement is being created, while it could reuse statements. I'll take a look into S3 as well sometime and try to give it PR.
Amazing video! I’m just starting to learn cloud as well so I’m glad I found your channel. An extra comparison that I think could be interesting would be adding in a node.js or even something like a python server (or both) as a comparison to show the concrete performance benefits we are achieving with go and rust. I imagine it’ll be a sort of Olympic athlete vs regular civilian comparison but could add good perspective to how well both of these are already doing despite already each having relatively similar limitations.
This is actually really great video on monitoring pods in production, thanks for creating this video, this will be very helpful resources for more junior engineer to benchmark service (regardless of the content).
Thanks for all the work you've done to produce these tests. I would like to add a small modification to the Rust code, you are using Serde for serialization and deserialization, use sonic-rs instead, it will give you 20% more performance.
Thank bro for doing that second test scenario, because that is a more frequent use case where you are talking to other services doing file and network stuff.
Thanks for doing Rust vs (non-Fiber) Go. Would be interesting to see how PHP ecosystem compares to this. Not only classic frameworks (for example Symfony vs Laravel), but also newer solutions like ReactPHP and FrankenPHP.
@@npcemprove6016 PHP's strength lies in established and developed ecosystem, which can help deliver product faster. Either way, I am curious how it compares, but it obviously doesn't win in performance.
@@npcemprove6016I mean, I wouldn't write a application for a local business with Rust, php or rails on that case are just overall great, its on the name, "personal home page", it was never meant to be a meta level language
@@edism in few words, ok. REST API, several middlewares, same database, same functionality for php (Lumen) and Go (stdlib). Lumen handles 1000 req/sec, Go 20000 req/sec. x20 at latency and much more throughput (run K6 grafana for tests, Go can handle 15-20 parallel loads-threads while PHP got sick at 2-3). Another example real-world use case. I wrote a Go service for XLSX generation. For phpspreadsheet it took x1000 times longer than for go (excelize lib). PHP itself slower than Go + PHP libs written by community without performance in mind. Hey, phpspreadsheet doesn't have stream writing mode! These 2 factors significantly slow down PHP apps.
This was huge hard work and I appreciate the effort you put here. Can you do this test with Laravel PHP to see how it fares against these large RPS? That will be a good comparison to understand how much a batteries-included framework slows or increases latency.
The big question - "Is the extra cost of developing a large app in rust worth the (minor) additional server costs if you choose go?" If you build a project in Rust and it is successful, finding good rust devs to scale it is much more difficult (and expensive) vs go devs
It's just a matter of time before many properly skilled rust programmers will become available. It's too good a language to pass up. There's evidence (a study performed internally by Google as it has started rewriting C++ stuff in rust) that once you've spent a couple of months in rust you start writing code much faster that with other languages - no surprise, I'd say, given rust's expressive power. Go's expressive power, OTOH, is awful, compared to most modern languages - which makes it a lot easier to learn, but once you're past that initial learning there's not much that you can improve further.
In the real world, to solve business problems, 9 out 10 times you should pick Go. Rust would be the right pick for niche cases where your specific use case benefits from the performance. I keep reading how popular Rust is but the reality is that the job market for it is tiny, you would certainly have issues hiring to maintain that codebase.
@@hyper_channel Provided you already know both languages, why would you _ever_ pick Go over Rust? Rust is faster, uses less memory and allows you to write the same thing in fewer lines of code. Also, the Rust compiler detects many more problems at compile time than the Go compiler. What would be even a single advantage of Go? That a half-retarded code monkey who you wouldn't trust writing a hello world on his own could possibly understand the code? That's not relevant, in a professional setting. The only situation in which Go would be a better choice is when your team already knows Go but doesn't know Rust, and the task at hand is a small one-off. If it's a larger project that's performance-critical, learning Rust already pays off. If it's a larger project without stringent performance requirements, Java or C# will do just fine, and still allow you to write less code faster. Go filled a niche at a time when there was nothing to fill it: a relatively simple language adequate for writing robust and not overly complex systems software. Rust fully covers that, and much more, and does so better than Go. Recently, Google started to rewrite its massive C++ codebase in Rust. They're the ones who created Go in the first place. How come _they_ didn't pick Go over Rust? Oh, and Google also translated some code from Go into Rust - leading to lower memory usage and lower bug count. Google hires tens of thousands of programmers, in case you missed it. Also, there's some pushback, but Linus is supporting Rust being used in the Linux kernel. Amazon and Microsoft are also using Rust. You won't need to search much for a job if you know Rust well. The only reason there are more jobs for Go is that Go is an older language.
The use of Rust can be a example of premature optimization. With Go your can iterate faster with good performance, only if your project scale a lot, at the point in which that minor difference is translated in hundreds of thousands of dollars, maybe makes sense search for something more low level, but in that case makes more sense use something like Zig instead of Rust, because Zig has better performance (more grained use of resources) than Rust and is more simpler (like Go) to implement.
@@SilasDuarte-e9k There's no optimization if your team already knows Rust. Unless you write the code in certain ways which cause Rust to generate inefficient binary code, Rust and Zig provide comparable performance. Zig or Go may make sense if your team doesn't know Rust already and if your project asks for performance but is expected to remain small and simple long term. For larger and more complex projects, even if your team has to learn Rust first it is still worth it.
I would LOVE to see the optimizations benchmarked next to each other as another video. Eg. Changing to cope on write improved speed by 20%. That way the audience would learn some optimization techniques vs just seeing what frameworks / languages are the most efficient.
Thanks! I keep getting PRs to improve Rust, Go, and other frameworks. I'll consider creating another video to go over each optimization technique I've received so far.
This is great esp the ecosystem of erlang and telecommunication. I prefer Go, coz it has better ROI on what I do(APIs). My ROI is balance btw devtime and speed.
I work with some people that think Node is good enough. I’d love to see the same comparison, with Node and/or Python mixed in. I know they’re not in the same league, I’m curious just how far out performed Node would be in this test. Very cool stuff, thanks for doing this!
Thanks for your awesome work!, It would be great if you could add a final time section at the end where you talk about conclusions drawn or summary of the finding.
axum is great and more intuitive than the other rust frameworks but i wish there was a website. the docs are great but it would attract more developers if there was a website. especially for new developers because the rust docs are very difficult to read unless you are familiar with reading them.
It looks like the go postgres library automatically caches the prepared sql statements, but the rust tokio_postgres connection don't (you can opt-in to this with prepare_cache instead of prepare though).
Good job! But in second case you test minio and postgresql. For complete picture you should show CPU and RAM for minio and postgresql. Because they are real bottleneck. Anyway thank you for video!
Thank you for the advice. I will include database CPU usage in the next test. I have Prometheus exporters for each database, but they have hundreds of metrics, and it's hard to fit them on the screen.
Hey, Anton. Love your content. I'm quite new to programming - how are you running these tests? Are you using a prometheus library inside the code which exposes metrics so it can bee scrapped? Would be awesome if you did a beginner introduction so we could learn how to do some of these tests ourselfs
Have you set the GOMAXPROCS environment ? By default golang is not aware of cpu usage limit and will try to run as many threads as there is physical cpus.
Very interesting! One benchmark i would be really interested on would be replicating floats vectors over udp; for moving 3d objects; ( Actually it could be any kind of vectorized data going through udp ) it should be pretty straightforward but could also make use of several optimisation like cpu intrinsics; and it would also be interesting to compare it over different cpu architecture and see if arm is really that big of a deal in terms of costs; Let’s say that one client have one vector of data that can change ( no need to add dirty bit / compression algorithms ) something like 60 time a second and it must be replicated to other clients; how many clients can it handle and what’s the error rate Would also be nice to have the comparison between go / rust / ebpf? / other?
Many thanks for revisiting this test. These new results are much more in line with my expectations. The updated Rust code is also extremely easy to read.
Is surrealdb really as fast as they claim? I know it’s fast with little data, but I’d love to see some benchmarks vs other databases with lots of rows.
Антон привет, спасибо за тесты и подробные разъяснения. Могу ли я где то увидеть примеры для аналогичных развертываний собственных нагрузочных тестов ? возможно есть готовые докер стеки(понятное дело интересуют конфиги для них, сами технологии понятны)
da konechno na moem kanale vse est vkluchaya EKS tutorial, chto imenno interesuet? K8s stuff - github.com/antonputra/tutorials/tree/main/lessons/212/k8s vse open source
I wonder how this compares to real life scenarios. Like what's the scale of the app that constantly has 14k requests per second. How many users is that typically?
I'm a bit confused why comparing Axum with Go std is more fair than using fiber. The rust version is highly optimized because of the options that rust gives you, so I think it's only fair to do the same in the Go version. I understand that fiber isn't the best option for all cases, but for these benchmarks it would certainly be faster. I think for a benchmark like this there are two options: Use the most naive implementation in both languages or optimize both implementations as much as you can. I feel like in this test you optimized the Rust version a lot but went with a pretty naive implementation in Go. I don't see how that is a fair comparison tbh
I think you need to show the network metrics. Are you saturating network. Since rust isn't using all of the cpu or memory, i think you are hitting limits in the network. Also, for the last test, you are getting latency spikes in both rust and go at similar times again, implying network limits.
Thanks for the informative video! I don't understand why Rust in 1st test cannot fully utilize more CPU than 60% and RAM when number of requests reach 25k/s! What are the resources that saturated here instead of CPU and RAM that makes the application slower?
Which json library did you use for serializing JSON. Most of the time when I saw the profile of benchmarks envolving JSON, the JSON marshal/unmarshal part always takes the most CPU. Could if you try a different library for it?
Bro is so cool, but reccomend draw scheme every test and text load test profile on video example: client -> our app -> database client -> our app -> (database + s3) client -> our app -> s3 and with steps: test 1: 1. post request with random boyd size between X Y test 2: 1. upload file to s3 with size xxx sometimes i look graphics other test and try look different but its not correctly, and mb need create 3-4 load test profile with names and with descriptions + steps + scheme and use this profiles without changes every test other languages? and i think more best if result was in the end of video as tables
Most large-ish apps I see have much more CPU usage, so you should add that in somehow. Maybe rendering a template dynamically. Also, p90 is a better metric here than p99. Go will benefit from p99 due to GC not running in the 1%
It's interesting if you benchmark not only by language programming, but also by Technology such as implementing grpc or graphql based on go vs rust vs c# vs python
@@AntonPutra interesting, thanks for letting me know. FYI This idea came from my experience with minio in on prem clusters with a not so fast Netapp as backing storage.
Nowadays there's a lot of discourse (in some places) about moving from JS to PHP, especially Laravel. A performance benchmark between those two would be great!!
When we are talking about go, rust.. it means that we are trying to match them with C... Js and php are 10000 times slower than these languages in congestion.. use case is different for js and php now a days.. not sure if it is really worth to compare js and php.
It seems that developing real-world web applications in Rust, with databases and storage in S3, is similar to developing them in Golang. I think even Python might offer similar "performance" because the database is often the bottleneck.
But what if in the test where go started failing at 14k rps you re-ran it but gave the go pod 500mb or even 1gb. out of interest maybe performance would be close ? using more RAM at the cost of an easier language than rust could be worth it ?
if you know javascript i would go the wasm route for learning with leptos or dixious. i did rustlings, went through the book, and then wasm. it was a lot easier that way.
Can you do the same tests but not setting a CPU limit in k8s (only request if you like it). We noticed that the k8s throttling is a Problem and using or restricting too much resources. We had 100% CPU spikes with a limit and without the services took only 1/3 of the production resources. Its only s problem if you run the service close to the limit of cou resources
► What should I test next?
► AWS is expensive - Infra Support Fund: buymeacoffee.com/antonputra
► Benchmarks: th-cam.com/play/PLiMWaCMwGJXmcDLvMQeORJ-j_jayKaLVn.html&si=p-UOaVM_6_SFx52H
I’d like to see the performance of an Golang Echo server in a comparison please.
Templating html or doing a database query would be realistic tests IMO!
c#/dotnet vs java/springboot，thank you
Axum vs acrix?
For go html/template vs Templ or encoding/json vs Jsoniter could also be interesting.
Hello, the second test feels sus! Why does the s3 upload take so much time? Is the Upload blocking? Can you replace it with an async wait and try again?
Glad you went back and did it again after feedback from the other video. Nice
🫡
I think this is a perfect analogy for go vs rust. You can get nearly as far as any language with Go with (relatively) little knowledge. You can get even further with deep knowledge of Rust.
true
Fair assessment!
@@curio78 😂
@@curio78 256MB ram maybe not enough for 1 java spring application to run
@@curio78 of course Java is slower than C or Rust.. Java is generally 7th to 12th fastest, depending on the benchmark (and the depth of languages tested) & is usually sitting behind Go.
More code for a slower result.. I'm not seeing the draw card of Java. People are generally moving away from Java, not to it.
Doesn't mean it doesn't serve a purpose, but speed & simplicity are not its' strong points.
It is amazing to me how well go keeps up even being garbage collected. Glad you did this video. I think it’s much more fair to what rust is capable of. That said I would argue that most teams would be better served by go especially if you don’t have +10k rps problems and an extremely well defined problem space for your application. That’s because it was easier for you (and most devs) to get a fair well optimized application with go first time around. I find in the real world most projects only get one serious pass at the time it’s initially created.
yeah, go is beginner friendly😂
@@anonymousalexander6005 I don’t think that’s true. As someone who’s doing backend js begrudging at work if you are running in kubernetes golang start up time is a real advantage. No JIT needed either. Not to mention the memory usage. I can run a 3 go servers on the smallest ec2 instance no problem. Can’t say the same for JavaScript.
Depends on the kind of work you do. Complex logic is better served by a compiler able to do more compile time checks and a language which has a much higher expressive power. Few real world workloads are as simple as serving constants or uploading some stuff to multiple places.
The cost of writing and maintaining software is more paramount than the perf metrics in respect to, say, a 5% difference - for most software projects. There are systems software and infrastructure software that can warrant being written in a manner to achieve maximal perf, and/or efficiency of resource usage, and/or have stringent requirements on deterministic behavior. But that’s a rather narrow category of software projects.
The majority of software projects are of a nature that can tolerate an automatic garbage collector - ala Golang
@@TheSulross You're right. You don't learn Rust just to write such code that is served well enough by Go. You use Java for that 😀
OTOH, once you have learned Rust, it's a much better language than Go for just about anything.
My gripe with Go is very specifically its rather poor expressiveness. More expressive languages allow you to write less code overall. Effort aside (which is only poorly correlated to lines of code anyway), fewer lines of code translate to fewer bugs - there are studies that show that all other things being equal, regardless of language used, bug density is proportional to lines of code.
Love watching your benchmark videos.
thanks!
Woha, benchmarks fights ! nice videos ! I love your channel
thank you! :)
It's nice how fast you can do something in Go while it still runs pretty fast
yesah, beginner friendly 😊
I found Axum to be really simple, if a bit immature, didnt feel like I was much slower that with other languages and frameworks. But maybe that just means I suck at all of them 😂😂
@@jonnyso1No Axum is pretty straightforward. It has a really good balance of exposing features (not hiding everything behind macros like rocket), and being simple enough.
I think that (As long as you’re working with REST APIs and not holding state), Axum and Go are ~equal in difficulty here.
If you need to get more complicated Go will likely be easier to use but rethinking your design might also help.
@@houstonbova3136I gave up on frameworks because I didn’t like any of them.
I just use hyper directly (what all these frameworks for Rust use anyway), and get absolutely *silly* performance out of it.
With just a few simple macros I wrote myself it does everything it needs to do, and I have absolute control over everything.
@@jonnyso1using axum may as well be easy and straightforward, but it does not mean that all other parts of your Rust program are going to be the same.
I believe the kubernetes CPU throttling is giving this benchmark an unfair comparison of the latency. Go is not aware of the cpu limit that you gave it. in general you should avoid cpu limits since how the kernel applies this limit will cause high tail latency, which can also explain why the throughput goes from 12K/s to 7K/s when throttling starts.
Would like to see the same test but with rust and go on their own nodes which they can use fully (so without any cpu limits). And would also be interesting to see them running on one node where they both get the same amount of cpu request but again no limit to avoid throttling, the requests will still make sure that both can equally use the cpu
ok i can do it, i was thinking about vm vs k8s benchmark due to throttling. i'll run few tests and see
this is normal cgroup cpu limit nothing special, just golang don't care how to calculate GOMAXPROCS inside cgroup
I think in real world app can be deployed like that, not under the perfect conditions. Go Lang should correct itself for limited usage.
@@AntonPutra consider adding GOMAXPROCS env with fieldRef to your container manifest
env:
- name: GOMAXPROCS
valueFrom:
resourceFieldRef:
resource: limits.cpu
@@erkintek You can set the GOMAXPROCS and GOMEMLIMIT env vars according to the pod cpu and memory limits respectively (rounded down to the nearest integer in the case of GOMAXPROCS). I do this regularly with GOMEMLIMIT (sometimes also tweaking GOGC) but haven't yet tested GOMAXPROCS. But the theory seems sound: Goroutines will be scheduled over only that many OS threads, so there should be less thread contention which improves tail latency. Uber has a package called automaxprocs to handle this automatically, and they measure significantly better p99.9 latency. Anton, this could be an interesting thing to test.
Hi Anton! I’d love to see a comprehensive course on observability and monitoring from you, covering everything from basics to advanced topics. It would be amazing if you could include: what to monitor (applications and clusters), how to monitor (tools and techniques), key metrics for applications and clusters, insights from 12+ years of experience, and an overview of metrics, logs, traces, OpenTelemetry, Jaeger, and how to create custom dashboards in Grafana. Also, a section on implementing monitoring for different programming languages would be incredibly valuable.
Your approach and expertise in monitoring are unmatched on TH-cam-I'm sure your course would offer insights that no one else provides. Thanks!
thank you! yes it is coming!
@@AntonPutra Can't wait for it!
Great video! Very clear and concise! I really liked the recap, as I didn't watch the previous video.
thanks!
Hey! awesome one! I took a look into the rust code and I saw that each time a new prepared statement is being created, while it could reuse statements. I'll take a look into S3 as well sometime and try to give it PR.
Thanks! I appreciate it!
@@AntonPutra The go code doesn't call prepare - I wonder if that's the cause for the CPU difference?
Amazing video! I’m just starting to learn cloud as well so I’m glad I found your channel. An extra comparison that I think could be interesting would be adding in a node.js or even something like a python server (or both) as a comparison to show the concrete performance benefits we are achieving with go and rust. I imagine it’ll be a sort of Olympic athlete vs regular civilian comparison but could add good perspective to how well both of these are already doing despite already each having relatively similar limitations.
Rust is really incredible, it's shocking to see how fast it is and still offers so many features.
yeap!
Nice! A lot more in line with what I saw with other tests. 👍
thanks!
This is actually really great video on monitoring pods in production, thanks for creating this video, this will be very helpful resources for more junior engineer to benchmark service (regardless of the content).
thank you
Thanks for all the work you've done to produce these tests. I would like to add a small modification to the Rust code, you are using Serde for serialization and deserialization, use sonic-rs instead, it will give you 20% more performance.
I would guess not since rust doesn’t seem to be CPU bound, but could be interesting 😊
Thank bro for doing that second test scenario, because that is a more frequent use case where you are talking to other services doing file and network stuff.
Thanks for doing Rust vs (non-Fiber) Go.
Would be interesting to see how PHP ecosystem compares to this. Not only classic frameworks (for example Symfony vs Laravel), but also newer solutions like ReactPHP and FrankenPHP.
PHP is not about speed. I compared go vs php and php totally lose for all types of benchmark. I dont see any reasons to write new applications in php.
@@npcemprove6016 PHP's strength lies in established and developed ecosystem, which can help deliver product faster. Either way, I am curious how it compares, but it obviously doesn't win in performance.
@@npcemprove6016I mean, I wouldn't write a application for a local business with Rust, php or rails on that case are just overall great, its on the name, "personal home page", it was never meant to be a meta level language
@@npcemprove6016 tell me more about the tests you ran, some specifics would be useful.
@@edism in few words, ok. REST API, several middlewares, same database, same functionality for php (Lumen) and Go (stdlib). Lumen handles 1000 req/sec, Go 20000 req/sec. x20 at latency and much more throughput (run K6 grafana for tests, Go can handle 15-20 parallel loads-threads while PHP got sick at 2-3).
Another example real-world use case. I wrote a Go service for XLSX generation. For phpspreadsheet it took x1000 times longer than for go (excelize lib). PHP itself slower than Go + PHP libs written by community without performance in mind. Hey, phpspreadsheet doesn't have stream writing mode! These 2 factors significantly slow down PHP apps.
This was huge hard work and I appreciate the effort you put here. Can you do this test with Laravel PHP to see how it fares against these large RPS? That will be a good comparison to understand how much a batteries-included framework slows or increases latency.
thanks! i'll do php soon
The big question - "Is the extra cost of developing a large app in rust worth the (minor) additional server costs if you choose go?" If you build a project in Rust and it is successful, finding good rust devs to scale it is much more difficult (and expensive) vs go devs
It's just a matter of time before many properly skilled rust programmers will become available. It's too good a language to pass up. There's evidence (a study performed internally by Google as it has started rewriting C++ stuff in rust) that once you've spent a couple of months in rust you start writing code much faster that with other languages - no surprise, I'd say, given rust's expressive power. Go's expressive power, OTOH, is awful, compared to most modern languages - which makes it a lot easier to learn, but once you're past that initial learning there's not much that you can improve further.
In the real world, to solve business problems, 9 out 10 times you should pick Go. Rust would be the right pick for niche cases where your specific use case benefits from the performance. I keep reading how popular Rust is but the reality is that the job market for it is tiny, you would certainly have issues hiring to maintain that codebase.
@@hyper_channel Provided you already know both languages, why would you _ever_ pick Go over Rust? Rust is faster, uses less memory and allows you to write the same thing in fewer lines of code. Also, the Rust compiler detects many more problems at compile time than the Go compiler.
What would be even a single advantage of Go? That a half-retarded code monkey who you wouldn't trust writing a hello world on his own could possibly understand the code? That's not relevant, in a professional setting.
The only situation in which Go would be a better choice is when your team already knows Go but doesn't know Rust, and the task at hand is a small one-off. If it's a larger project that's performance-critical, learning Rust already pays off. If it's a larger project without stringent performance requirements, Java or C# will do just fine, and still allow you to write less code faster.
Go filled a niche at a time when there was nothing to fill it: a relatively simple language adequate for writing robust and not overly complex systems software. Rust fully covers that, and much more, and does so better than Go.
Recently, Google started to rewrite its massive C++ codebase in Rust. They're the ones who created Go in the first place. How come _they_ didn't pick Go over Rust? Oh, and Google also translated some code from Go into Rust - leading to lower memory usage and lower bug count. Google hires tens of thousands of programmers, in case you missed it. Also, there's some pushback, but Linus is supporting Rust being used in the Linux kernel. Amazon and Microsoft are also using Rust. You won't need to search much for a job if you know Rust well. The only reason there are more jobs for Go is that Go is an older language.
The use of Rust can be a example of premature optimization. With Go your can iterate faster with good performance, only if your project scale a lot, at the point in which that minor difference is translated in hundreds of thousands of dollars, maybe makes sense search for something more low level, but in that case makes more sense use something like Zig instead of Rust, because Zig has better performance (more grained use of resources) than Rust and is more simpler (like Go) to implement.
@@SilasDuarte-e9k There's no optimization if your team already knows Rust. Unless you write the code in certain ways which cause Rust to generate inefficient binary code, Rust and Zig provide comparable performance. Zig or Go may make sense if your team doesn't know Rust already and if your project asks for performance but is expected to remain small and simple long term. For larger and more complex projects, even if your team has to learn Rust first it is still worth it.
I would LOVE to see the optimizations benchmarked next to each other as another video. Eg. Changing to cope on write improved speed by 20%. That way the audience would learn some optimization techniques vs just seeing what frameworks / languages are the most efficient.
Thanks! I keep getting PRs to improve Rust, Go, and other frameworks. I'll consider creating another video to go over each optimization technique I've received so far.
@@AntonPutra Awesome! thanks - I'm really enjoying your videos! super cool stuff here!
Very useful video, thanks for your hardwork.
thank you!
PR sent with some basic improvements to the Go version.
Go vs Elixir, Specifically for real time communication, where you have to maintain ws connections and send and receive messages from them.
This is great esp the ecosystem of erlang and telecommunication. I prefer Go, coz it has better ROI on what I do(APIs). My ROI is balance btw devtime and speed.
I love watching your comparisons; I really enjoy your videos. I hope you can make a comparison between Fiber, Iris, and Gorilla Mux. Best regards!
thank you! noted! never heard about iris before :)
Add Hertz to the list too!
I work with some people that think Node is good enough. I’d love to see the same comparison, with Node and/or Python mixed in. I know they’re not in the same league, I’m curious just how far out performed Node would be in this test.
Very cool stuff, thanks for doing this!
Good job. Would love to see more i/o centric middleware framework comparisons
thank you! will do in the future
Thank You SIR, ❤ I learned real world technology from a great teacher, prayers for you! JazakAllah ❤ 🙏 🤲 👍
Thanks for your awesome work!, It would be great if you could add a final time section at the end where you talk about conclusions drawn or summary of the finding.
I think it would be nice to test with Postgres reads/ writes only, as it will show how well the two handle async operations in general.
ok i'll keep this in mind
great learning about profiling n optimizations
thank you!
Great video! Because I`m not that deep into k8s, rust and go, a conculsion / summary at the end of the video would be nice. Thank you
thanks! noted
axum is great and more intuitive than the other rust frameworks but i wish there was a website. the docs are great but it would attract more developers if there was a website. especially for new developers because the rust docs are very difficult to read unless you are familiar with reading them.
Спасибо за тесты. Особенно за реальный тест.
bez problem 😊
Love ur videos man, really useful
thank you!
Great video!! Thanks!!!
It looks like the go postgres library automatically caches the prepared sql statements, but the rust tokio_postgres connection don't (you can opt-in to this with prepare_cache instead of prepare though).
thanks, i got your PR
Great work 🙌
thank you!
thanks!
Good job! But in second case you test minio and postgresql. For complete picture you should show CPU and RAM for minio and postgresql. Because they are real bottleneck. Anyway thank you for video!
Thank you for the advice. I will include database CPU usage in the next test. I have Prometheus exporters for each database, but they have hundreds of metrics, and it's hard to fit them on the screen.
Hey, Anton. Love your content. I'm quite new to programming - how are you running these tests? Are you using a prometheus library inside the code which exposes metrics so it can bee scrapped? Would be awesome if you did a beginner introduction so we could learn how to do some of these tests ourselfs
Excellent testing, thank you.
Have you set the GOMAXPROCS environment ? By default golang is not aware of cpu usage limit and will try to run as many threads as there is physical cpus.
not in that particular video, but i've improved since then. thanks for the advice anyway
Please make a Laravel vs express benchmark.
Loving your videos ❤
thank you! will do!
Very interesting!
One benchmark i would be really interested on would be replicating floats vectors over udp; for moving 3d objects; ( Actually it could be any kind of vectorized data going through udp )
it should be pretty straightforward but could also make use of several optimisation like cpu intrinsics; and it would also be interesting to compare it over different cpu architecture and see if arm is really that big of a deal in terms of costs;
Let’s say that one client have one vector of data that can change ( no need to add dirty bit / compression algorithms ) something like 60 time a second and it must be replicated to other clients; how many clients can it handle and what’s the error rate
Would also be nice to have the comparison between go / rust / ebpf? / other?
Many thanks for revisiting this test. These new results are much more in line with my expectations. The updated Rust code is also extremely easy to read.
my pleasure, but fiber is still faster than rust 🙈 i was trying to find analog in rust to test and so far i got Ntex
Is surrealdb really as fast as they claim? I know it’s fast with little data, but I’d love to see some benchmarks vs other databases with lots of rows.
Would love to see how rocksdb vs the tikv cluster for data storage effects it.
Would be interesting to see how some more alternatives would compare like Hono, Elysia, Fastify, Express and .Net :D
some of them next
Awesome!
thanks!
Thanks, good test
thanks
Thank you for this great content
AWS has recently deprecated the GO runtime. And recently released the Rust sdk out of beta
Go sdk v2 exists in 2024, if you mean lambda, provided.al2 or provided.al2023 are replacement of go runtime
This video is interesting, but no more interesting than "apples vs oranges: the truth". That was a hit, last year.
Dhanyavad
Gin vs Echo benchmark please
thanks, noted!
Антон привет, спасибо за тесты и подробные разъяснения.
Могу ли я где то увидеть примеры для аналогичных развертываний собственных нагрузочных тестов ? возможно есть готовые докер стеки(понятное дело интересуют конфиги для них, сами технологии понятны)
da konechno na moem kanale vse est vkluchaya EKS tutorial, chto imenno interesuet? K8s stuff - github.com/antonputra/tutorials/tree/main/lessons/212/k8s
vse open source
I wonder how this compares to real life scenarios. Like what's the scale of the app that constantly has 14k requests per second. How many users is that typically?
it is possible that you have internal api which provides specific data that many your microservices would request, it's not necessary external api..
For a complex backend service. How do you benchmark it? Do you pick some of the most used endpoint to test?
well, the most common use case is to interact with a database, aws services, and messaging systems like kafka
Awesome video as usual 📺👍🏻
_Just a little unrelated note:_
Noun: *Degradation*
Verb: *Degrade* NOT -Degradate- 🚫
thank you! 😊
Did you consider using axum framework instead of actix
yes, next video "actix vs axum vs rocket"
@@AntonPutra looking forward to thi :) Thank you.
@@AntonPutra nice 🔥
Mantulll😊@@AntonPutra
I'm a bit confused why comparing Axum with Go std is more fair than using fiber. The rust version is highly optimized because of the options that rust gives you, so I think it's only fair to do the same in the Go version.
I understand that fiber isn't the best option for all cases, but for these benchmarks it would certainly be faster.
I think for a benchmark like this there are two options: Use the most naive implementation in both languages or optimize both implementations as much as you can. I feel like in this test you optimized the Rust version a lot but went with a pretty naive implementation in Go. I don't see how that is a fair comparison tbh
Thanks for the feedback. I'll run some tests between the most performant Go and Rust frameworks independently, and then compare the winners.
It would be interesting to see Java using Vert.x vs Go and Rust.
noted!
fiber is best for performance. because we already saw the compression between go std/net vs fiber.
I think you need to show the network metrics. Are you saturating network. Since rust isn't using all of the cpu or memory, i think you are hitting limits in the network. Also, for the last test, you are getting latency spikes in both rust and go at similar times again, implying network limits.
ok, i have all the metrics. i will include them in the following videos, as well as packet drops
Thanks, was looking for a while for a comment realizing that the bottleneck must be somewhere else when rust doesn’t max out the CPU
what a "surprising" results!
😊
Thanks for the informative video! I don't understand why Rust in 1st test cannot fully utilize more CPU than 60% and RAM when number of requests reach 25k/s! What are the resources that saturated here instead of CPU and RAM that makes the application slower?
thanks! i try to improve the tests with each new video
Sir, can you please make a video on "Deno" vs "Node" vs "Bun". You are our trusted one. we don't trust company benchmark. they always lies to us.
Which json library did you use for serializing JSON.
Most of the time when I saw the profile of benchmarks envolving JSON, the JSON marshal/unmarshal part always takes the most CPU.
Could if you try a different library for it?
which one you recommend?
Bro is so cool, but reccomend draw scheme every test and text load test profile on video example:
client -> our app -> database
client -> our app -> (database + s3)
client -> our app -> s3
and with steps:
test 1:
1. post request with random boyd size between X Y
test 2:
1. upload file to s3 with size xxx
sometimes i look graphics other test and try look different but its not correctly,
and mb need create 3-4 load test profile with names and with descriptions + steps + scheme and use this profiles without changes every test other languages?
and i think more best if result was in the end of video as tables
Oh, you are parsing uuid every single request last time in rust, no wonder it uses so much cpu last time
What charts library are you using? echarts?
no it's just grafana charts
Most large-ish apps I see have much more CPU usage, so you should add that in somehow. Maybe rendering a template dynamically. Also, p90 is a better metric here than p99. Go will benefit from p99 due to GC not running in the 1%
Hope that Rust fan won't complain about Go anymore
haha
it's insane how rust handle memory
well golang has garbage collection and rust doesn't
@@AntonPutra yep that's my point too. tho props for golang to be pretty efficient being a language with gc
I would like to see the comparison between the rust(actix) with the optimization and go (stdlib) and go(fiber
)
do c/c++ vs zig vs rust next time ;P
yes something from this list definitely :)
Could you test rust vs elixir? Thanks
With Rust? I'll do Elixir vs Go first, and then we'll see. It depends on how it performs.
Wow, this is unexpected .... so golang is better for file operations while rust is better for handle web apps querys ?
I expected exactly the opposite
It's interesting if you benchmark not only by language programming, but also by Technology such as implementing grpc or graphql based on go vs rust vs c# vs python
thanks for the tip, grpc vs rest vs graphql is coming. i tested grpc vs rest in the past as well and kafka vs grpc, lol
If you used minio as s3 setup in your eks cluster the bottleneck might be the storage backend that is used by your minio instance
Actually, I spent some time debugging that issue, and it turned out that the database and the size of the connection pool were the main problems.
@@AntonPutra interesting, thanks for letting me know.
FYI This idea came from my experience with minio in on prem clusters with a not so fast Netapp as backing storage.
Nowadays there's a lot of discourse (in some places) about moving from JS to PHP, especially Laravel. A performance benchmark between those two would be great!!
When we are talking about go, rust.. it means that we are trying to match them with C... Js and php are 10000 times slower than these languages in congestion.. use case is different for js and php now a days.. not sure if it is really worth to compare js and php.
Reference php on a rust & go benchmark is a joke lmao
@@trumvkl exactly 😅🤣😂
Absolutely no one is moving towards php
You just made that up, nobody would move anything from anything else to PHP
I'd be very interested to see results with TinyGo.
Interesting, I've never heard of TinyGo before
Would be nice to see bottleneck in profiler instead of just client time
are there any articles on that topic🧐?
@@AntonPutra a lot, yes
@@AntonPutra"how to profile go/rust code"
@@CuriousSpy 🫡
>>super optimized crate
>>versus super stable, simple, not that optimized stdlib
>WOW rust is sooooo much better!!
I believe in you Axum
i'll do some more testing soon
It seems that developing real-world web applications in Rust, with databases and storage in S3, is similar to developing them in Golang. I think even Python might offer similar "performance" because the database is often the bottleneck.
yes, but try to develop one in zig (zap) 😊
At first test I think rust is not the limit, with %60cpu, and plenty of ram. K8s or requests may be?
I'm curious if the cpu and memory differences in the second test point to not using a buffered file strategy in rust with go using one.
maybe i need to test
Does this mean, Rust is better than go in all benchmark?
I hope fireship or prime watches this.
But what if in the test where go started failing at 14k rps you re-ran it but gave the go pod 500mb or even 1gb. out of interest maybe performance would be close ? using more RAM at the cost of an easier language than rust could be worth it ?
i don't think memory matters in this case but i'll check next time
Good video.
compare to fastapi?
Well, when I compare Rust with Fiber/Fasthttp, the Rust community gets upset :)
Man I would like to be a competent rust developer so bad
if you know javascript i would go the wasm route for learning with leptos or dixious. i did rustlings, went through the book, and then wasm. it was a lot easier that way.
could you please make comparison c++ to rust ?
найс инглиш бро
Can you do the same tests but not setting a CPU limit in k8s (only request if you like it). We noticed that the k8s throttling is a Problem and using or restricting too much resources. We had 100% CPU spikes with a limit and without the services took only 1/3 of the production resources. Its only s problem if you run the service close to the limit of cou resources
What we need right now is list result of th e benchmark for every language , so whois better for some cases , idlove to see the web
They deprecated the lambda runtime for golang too, a while back
yeah just saw in in supported runtimes page, well i guess generic runtime for binaries is good enough
So in real world scenario (upload s3, sql query...) with the same expertise level at both runtime they have the same performance ?
With Rust, you still get a better price to CPU ratio. I even rewrote my clients from Go to Rust to save 50% on infrastructure costs. It matters to me.
@@AntonPutra Pretty rare where the cost of +50% of CPUs is more expensive than hourly senior Rust developer !
I would love to see a comparision between golang grpc vs tonic