🔴IMPORTANT🔴 - For the next 2 or 3 days, I'll collect all the tips from Zig experts with optimization techniques. If it makes a significant difference, I'll release an updated video; if not, I'll just create a TH-cam post with the results. UPDATED - th-cam.com/video/SR2LRhnL1AQ/w-d-xo.html
@@AntonPutra This is a great thought and it's fair by all accounts. I only fear that you may not get the same level of input as other communities. Not sure if there's anything to do about it though. Just pointing out what may be natural biases that exist due to culture and time.
I got 1 PR already, I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results. github.com/antonputra/tutorials/pull/268
Looking at the Dockerfile, you turned on release on Rust, but not on Zig! RUN cargo build --release versus RUN /opt/zig/zig build (missing -Doptimize=ReleaseFast here)
thanks for the tip, I already got 1 pull request - github.com/antonputra/tutorials/pull/268 I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
Thanks, @ndykhng! The guy doesn't know how to compile Zig, compares apples with oranges, and obviously, it's Zig's fault. He has no idea how Zig works. He asks in the video if there is no garbage collection in Zig? Of course not. 🤦♂ I'm asking YT to not recommend this channel again. 👎 with no hesitation.
we do almost 2 million TPS in HFT with java 8 lol on one core wire to wire measured (but yes we do not use retarded transport protocols or serialization formats so its not very fair) its just not impressive...
@@krellin Congrats. That's impressive. Are you sure you're actually hitting the CPU core and not serving requests off FPGAs directly onto the wire? It's hard to imagine you could even get a handful of bytes on and off the NIC into a single CPU core at that rate.
@@krellin You have a budget of 50ns at 2M rps. That's effectively 20 add instructions or a single L3 cache hit. That's not at all comparable to what the video is about. Both are impressive results in their own right. Yours in being able to pre-compute a response and program hardware to respond for you, and the video for processing a more reasonable workload on the fly, like the vast majority of companies rely on
@@ryanseipp6944 Since he was talking about HFT I am assuming that he has some event driven system that processes 2 million events per second, which is fairly straightforward to do
Rust and Zig are so unconstrained (same for C, C++, etc) that when doing benchmarks like these, it’s always a framework benchmark and not a language benchmark. Comparing slower languages, it is more of a language benchmark, for example boxing values in Python takes a big chunk of the total time. In Rust/Zig it’s more down to "does the framework allocate or not im this codepath" or "is the framework using a radix trie for matching URLs or something less efficient".
After taking a look at the dockerfile, seems like you arent doing thing properly. The zig build should be: RUN /opt/zig/zig build -Doptimize=ReleaseFast
Yes i got a PR already - github.com/antonputra/tutorials/pull/268 I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
@@anonymousalexander6005 That depends on what the goal is. ReleaseSafe includes lots of runtime checks that Rust doesnt have. It also includes debug info. In this case, ReleaseFast is the correct one imo.
Thanks! If you have any practical tips, flags I need to use, or options, please let me know. I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
Can you possibly add some query parameters to the requests so the returned content is not static? I suspect that zig can compute the json at comptime and replace it with const slice in release build.
This is a good comparison video. One thing that would make it better is if you could group (or visually mark) which graphs where lower is better vs. higher is better. I can read each graph and tell one-by-one, but since they the graphs are a mix of higher/lower is better, it takes me longer to read and comprehend than it would if it were clearer. Thanks for this analysis! I haven’t tried Zig, but my interest is growing.
It’s hard to draw meaningful results out of micro benchmarks sometimes. Aside from the “/healthz” route, which is effectively hello world - most endpoints do significant work. It’s the “significant work” bit that shows how some languages and approaches are better or worse than others Not suggesting you expand your tests in that area, since the choice of “significant work” then becomes awfully subjective :) Even in this case here, it’s hard to say whether it’s handling IO vs json encoding that’s giving different results Maybe adding some complex endpoints that .. push a big struct of data through a template and output HTML or something Maybe decode a JWT too, as that can be CPU intensive and it should be super common
ok, noted. i mean, i could potentially measure JSON serialization and other stuff, but i may end up with a lot of graphs. in other benchmarks, i also add the AWS SDK to upload files to S3 and Postgres. this benchmark was an exception because there aren't many SDKs for Zig :)
@@AntonPutrafair call. Give us a lesson number that does the S3 and Postgres updates.. and will have a go at doing a (pure) zig version for you. The AWS updates have been messy since they have slowly been changing TLS versions over the last 12 months :) The Postgres libs are pretty solid now I’m not sure about the zap / debug build issue though, since it’s facil doing the actual work here .. the zig bits are just wrapping that. Changing optimisation mode might not end up changing much .. I’m not sure Probs best to start with a clean slate and do another different run using pure zig. (Will help out here) It’s early days still .. but keep in mind there are a lot of senior devs using zig for this sort of load already, and because it’s not as “mainstream” as other options- we need very good technical reasons to justify the risk. We are definitely getting good numbers - it would be great to see someone outside that community duplicate the results in their own setup Might take a bit of time over this and get it right. Zig being what it is, we kinda like taking our time vs knocking out quick hacks ;)
@@steveoc64 that would be great! Here is go vs rust source code with s3 and postgres - github.com/antonputra/tutorials/tree/main/lessons/205 please use minio, you can run it locally using docker, also aws sdk needs to use "path" style - github.com/antonputra/tutorials/blob/main/lessons/205/go-app/config.yaml#L8 you can add zig verion to github.com/antonputra/tutorials/tree/main/lessons/207
@@AntonPutra awesome! Thx PS: thanks for the docker file and tf setups too - devops is totally not my thing, and your repo has been super helpful for me ;)
It would get clapped by a lot of language in this test by a long short. Erlang, by it nature, isn't good at raw performance on limited single instance, but rather the scalability (both vertically and horizontally) that is crazy good.
@@OnFireByte everybody know the BEAM vm isnt particularly performant i want to see how well it handles latency and how graciously it breaks down as you overload the system.
I got 1 PR already, I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results. github.com/antonputra/tutorials/pull/268
Man, i really love your video! it give me exactly what i imagine of comparing and benchmarking two different language performance. i love to see if you also compare them with go!
@@AntonPutra i watch your other video, i try to replicate the kubernetes monitoring but somehow i cannot make the cadvisor and prometheus showing my docker containers metrics. why? do you have video that do metrics but only for docker? no kubernetes ?
I think Zig results are worse because of, well, maturity. Zap is over 1 year old, while Actix has been around a few years already. There was time to multiple rounds of bug fixing and optimizations. I would say Zap is relatively young. To add to the subject, on the Rust side, you're still not using the "fastest framework out there". According to the TechEmpower Web Framework Benchmark there are multiple faster frameworks for Rust, including: May, Xitca and Ntex. Generally, in time we should se those scores come closer to one another.
I've seen the new results on the Git repository, there's a remark, for the same number of Req/s , actix sends more Bytes than Zig, so Zig sends requests with less complete metadata (headers) than Actix, so the test is flawed.
Zig can be a nice choice for replace C in the server, specially for teams with C developers already because of the inter op and syntax, but Rust sounds more attractive for general use. Try Rust (Actix) vs C++ (CrowCpp) in a future test, will be a nice comparison.
Maybe Zig allocated all required memory just once, and not performing re-allocation later? (To the question, why Rust using less when 1 MB, and Zig - 22,5 MB.)
Maybe I'll wait for the Zig experts with optimization tips, and I'll update the results if there is a significant difference, or I'll just create a TH-cam post.
Collecting optimization tips from Zap (Zig) experts over the next couple of days, and I will release an updated video if there is a significant improvement, or just a TH-cam post.
Rust has an extremely mature asynchronous runtime, Zig doesn't exactly have concurrency as a core value proposition and it only promises to be a bit more convenient than C
Your Zig is NOT optimized whereas you have used Rust in release mode. Such a silly miss! Why would anyone trust this benchmark and the follow-on benchmark when you do not have the sense of doing the basic stuff!
I guess the existing bench is useful in showing how fully optimised Rust is able to hang in there with a debug mode Zig wrapper over an external C lib Should keep the video, but change the misleading title :) To give Anton credit where it’s due .. his devops skills are very excellent. That tutes repo is a great resource for learning more about k8s and Docker configs for different things Not necessarily the same skill set as programming
will update the results in a couple of days, but i’ve only got this so far to optimize Zig. i hope it makes a big difference github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L15C14-L15C23
A lot of people complain that you are not using the languages the right way. I think it would be a good idea to consult with willing and knowledgeable people for each language. So that the programs are idiomatic and represent apps that you could see in he wild
After looking at zap vs actix, I would say Zig implementation is much "low level" than Rust ie. - Actix handles the routing for you while you hand write it in Zig. - In rust the serialization is done by the framework directly (with macro) but in Zig you manually allocate and call stringifyBuf. - In Zig, calling stringifyBuf could fail when the buffer overflow, this is not the same API parity with Rust which *practically* never fail. To make this fair, please update Zig implement to use a growable buffer or inline Rust json serialization to use array buffer. - Like other commenter have said, you forgot to use release mode in zig
My assumption here is this like the initial Go vs Rust where Go won. The biggest difference might end up being that you won't have zig experts come to tell you how you can improve it as it's still a very young language.
Agree! Thanks! I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
Looking at the benchmarks it seems that you should set a standard for optimizations for the comparison to be fair. Obviously ideally you want to be using PGO but not all compilers have easy support for that. So I think a default of the highest optimization setting that doesn't break fp math, (as in no -Ofast or -ffast-math) so in the case of gcc/clang that would be -O3 combined with true link time optimizations would be a sane default. Not sure if you set true LTO for rust this time, so as rust by default doesn't optimize across crate boundaries this could be losing performance. To summarise LTO + O3 would be sane. Along with building for the architecture of the server you are running on (-march and -mtune equivalents). Edit: I see that you already set lto to true in rust and panic to abort.
@@anonymousalexander6005You would be surprised. LTO is not niche and should be a default. In terms of PGO what it does is it allows the compiler to know what are the hot paths in the code. Depending on the application PGO could even yield 10% performance or more. Getting around 5-10% is not uncommon. This is talking about recent clang and LLVM 18 and above. The compiler without LTO is only able to guess as to what the correct path to inline is, or if some loop gets called enough to be worth vectorizing or unrolling.
@@anonymousalexander6005You would be surprised. LTO is not niche and should be a default. In terms of PGO what it does is it allows the compiler to know what are the hot paths in the code. Depending on the application PGO could even yield 10% performance or more. Getting around 5-10% is not uncommon. This is talking about recent LLVM 18 and above. The compiler without LTO is only able to guess as to what the correct path to inline is, or if some loop gets called enough to be worth vectorizing or unrolling.
that thumbnail could give you trouble with the german authority so be careful. But cool video and as others mentioned zig should compile with releasefast
Also, regarding Zig, I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
У зига стабильная версия 0.13 вроде? Учитывая состояние документации и новизну языка, вполне себе хорошие результаты. Единственное что странно - использование памяти. Скорее всего "skill issue", но типа это не в упрёк, ты итак пишешь на куче языков, надеюсь, знающие зиг люди помогут хорошими коммитами =)
one govorya 0.13 is stale, so they suggest to use latest master - github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L3 I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
well i have a lot of tutorials on my channel how to monitor containers with prometheus and grafana one of them - github.com/antonputra/tutorials/tree/main/lessons/135
I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
well, thank you for your infomation, i am learning Rust, but i got advertising from zig that they said Zig is faster than Rust, it make confuse, but now i known Rust is faster and more stable
One important thing to note here is that Zig is not async hence must be using a thread pool, An async runtime has been proven to be more performant, using lesser resources and handle more concurrent connections.
yes will release updated video, but how come "./zig build" defaults to debug? it's a bit confusing updated - github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L15C14-L15C23
well, i wouldn't say that i built it in debug mode - github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L15 but i get your point and will rerun the test with some optimizations, including this - github.com/antonputra/tutorials/pull/268
🔴 To support my channel, I'd like to offer Mentorship/On-the-Job Support/Consulting (me@antonputra.com)
Zig vs Go
Nomad (Hashicorp)
actix (rust) vs salvo (rust)
rust vs. go. vs. js vs php hehe
Time of writing (creation) of a new web-service on Go, Rust and Zig.
🔴IMPORTANT🔴 - For the next 2 or 3 days, I'll collect all the tips from Zig experts with optimization techniques. If it makes a significant difference, I'll release an updated video; if not, I'll just create a TH-cam post with the results. UPDATED - th-cam.com/video/SR2LRhnL1AQ/w-d-xo.html
@@AntonPutra This is a great thought and it's fair by all accounts. I only fear that you may not get the same level of input as other communities. Not sure if there's anything to do about it though. Just pointing out what may be natural biases that exist due to culture and time.
You should post this video in zig subreddit too. Will probably get helpful inputs there
I looked at your dockerfile,. you are not testing a release build of Zig...
I got 1 PR already, I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
github.com/antonputra/tutorials/pull/268
Yes. Man it’s like you read my mind. I was gonna suggest a comparison using zig.
i may update results if i get enough tips to improve Zig code, i already got 1 PR - github.com/antonputra/tutorials/pull/268
Looking at the Dockerfile, you turned on release on Rust, but not on Zig!
RUN cargo build --release
versus
RUN /opt/zig/zig build (missing -Doptimize=ReleaseFast here)
Up!
thanks for the tip, I already got 1 pull request - github.com/antonputra/tutorials/pull/268
I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
@@AntonPutra Make updated video
Thanks, @ndykhng! The guy doesn't know how to compile Zig, compares apples with oranges, and obviously, it's Zig's fault. He has no idea how Zig works. He asks in the video if there is no garbage collection in Zig? Of course not. 🤦♂ I'm asking YT to not recommend this channel again. 👎 with no hesitation.
Such a silly miss! Why would anyone trust this benchmark and the follow-on benchmark when you do not have the sense of doing the basic stuff!
134K/s requests is pretty impressive
160k max so far
UCall Python library claims 210k requests/s
@@snippletrap who care?
@@dxlaam004 Anyone who cares about server performance? Like the people who clicked on this video?
@@snippletrapucall is a python wrapper for a C/C++ implementation.
Very cool! 130-160k rps on two cores is nuts. Glad you revamped the clients to generate enough load
Yeah, I'm trying to improve each time
we do almost 2 million TPS in HFT with java 8 lol on one core
wire to wire measured (but yes we do not use retarded transport protocols or serialization formats so its not very fair) its just not impressive...
@@krellin Congrats. That's impressive. Are you sure you're actually hitting the CPU core and not serving requests off FPGAs directly onto the wire? It's hard to imagine you could even get a handful of bytes on and off the NIC into a single CPU core at that rate.
@@krellin You have a budget of 50ns at 2M rps. That's effectively 20 add instructions or a single L3 cache hit. That's not at all comparable to what the video is about. Both are impressive results in their own right. Yours in being able to pre-compute a response and program hardware to respond for you, and the video for processing a more reasonable workload on the fly, like the vast majority of companies rely on
@@ryanseipp6944 Since he was talking about HFT I am assuming that he has some event driven system that processes 2 million events per second, which is fairly straightforward to do
Great analysis! Thank you
thank you!!
Zig framework probably preallocates a bunch of memory to reduce allocations during runtime
maybe
Next logical step is Odin vs Rust :)
🫡 😂
I think Odin doesn't have a HTTP server yet.
you should do Java & C# vs Golang. People say the two are as good as Golang is, but I'm not sure if that's true
They are better
@@EzequielRegaldo least delusional bytecode user
ok, will do!
@@realoctavian tested here and NET won by far
@@EzequielRegaldo delusional
Great comparison as always. Would be good to compare rust vs c and c++
thanks, yes will do. do you have any frameworks in mind?
@@AntonPutraThere is the framework Drogon for C++
uwebsockts
@@AntonPutramaybe you could try boost beast, though its pretty low level compared to other libraries
Yes dragon is a mysterious framework @@germandavid2520
Rust and Zig are so unconstrained (same for C, C++, etc) that when doing benchmarks like these, it’s always a framework benchmark and not a language benchmark. Comparing slower languages, it is more of a language benchmark, for example boxing values in Python takes a big chunk of the total time. In Rust/Zig it’s more down to "does the framework allocate or not im this codepath" or "is the framework using a radix trie for matching URLs or something less efficient".
agree, it is mostly frameworks, especially since zig is a wrapper around a c lib.
Are you using ReleaseSafe or ReleaseFast for the zig build?
Edit: The benchmark is inherently flawed, and shouldnt be trusted.
After taking a look at the dockerfile, seems like you arent doing thing properly. The zig build should be:
RUN /opt/zig/zig build -Doptimize=ReleaseFast
Yes i got a PR already - github.com/antonputra/tutorials/pull/268
I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
@@AntonPutra thats me ;)
@@uzaaftThe test is for production environment, ReleaseSafe is the proper release flag.
@@anonymousalexander6005 That depends on what the goal is. ReleaseSafe includes lots of runtime checks that Rust doesnt have. It also includes debug info. In this case, ReleaseFast is the correct one imo.
you didn't enable optimizations for zig
Thanks! If you have any practical tips, flags I need to use, or options, please let me know. I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
It would be nice to include Nim and its caprese framework. It currently sits at the top of web frameworks benchmark
ok noted!
Can you possibly add some query parameters to the requests so the returned content is not static? I suspect that zig can compute the json at comptime and replace it with const slice in release build.
i was thinking about it
This is a good comparison video. One thing that would make it better is if you could group (or visually mark) which graphs where lower is better vs. higher is better. I can read each graph and tell one-by-one, but since they the graphs are a mix of higher/lower is better, it takes me longer to read and comprehend than it would if it were clearer.
Thanks for this analysis! I haven’t tried Zig, but my interest is growing.
thanks for the advice
@@AntonPutra Is that dashboard with all the graphs created by you? Or provided by a service?
Are you **able** to customize it?
@@tonyr.6637 By me, it's just open-source Prometheus and Grafana to visualize metrics. I have some tutorials on my channel.
Hope we get an update on the Zig results when someone tells you whiy it might've failed :D
yes, will update soon
Add external storage layer to make benchmark more realistic
well, i'm not an expert in zig, if you can create a PR for zig, i'll take care of rust
Do you have a tutorial how to setup a distributed database for horizontal scaling sir? Such as a mysql distributed database. I mean a cluster
no, but there are some solutions like Citus that you can use out of the box
It’s hard to draw meaningful results out of micro benchmarks sometimes.
Aside from the “/healthz” route, which is effectively hello world - most endpoints do significant work. It’s the “significant work” bit that shows how some languages and approaches are better or worse than others
Not suggesting you expand your tests in that area, since the choice of “significant work” then becomes awfully subjective :)
Even in this case here, it’s hard to say whether it’s handling IO vs json encoding that’s giving different results
Maybe adding some complex endpoints that .. push a big struct of data through a template and output HTML or something
Maybe decode a JWT too, as that can be CPU intensive and it should be super common
ok, noted. i mean, i could potentially measure JSON serialization and other stuff, but i may end up with a lot of graphs. in other benchmarks, i also add the AWS SDK to upload files to S3 and Postgres. this benchmark was an exception because there aren't many SDKs for Zig :)
@@AntonPutrafair call. Give us a lesson number that does the S3 and Postgres updates.. and will have a go at doing a (pure) zig version for you.
The AWS updates have been messy since they have slowly been changing TLS versions over the last 12 months :)
The Postgres libs are pretty solid now
I’m not sure about the zap / debug build issue though, since it’s facil doing the actual work here .. the zig bits are just wrapping that. Changing optimisation mode might not end up changing much .. I’m not sure
Probs best to start with a clean slate and do another different run using pure zig. (Will help out here)
It’s early days still .. but keep in mind there are a lot of senior devs using zig for this sort of load already, and because it’s not as “mainstream” as other options- we need very good technical reasons to justify the risk. We are definitely getting good numbers - it would be great to see someone outside that community duplicate the results in their own setup
Might take a bit of time over this and get it right. Zig being what it is, we kinda like taking our time vs knocking out quick hacks ;)
@@steveoc64 that would be great! Here is go vs rust source code with s3 and postgres - github.com/antonputra/tutorials/tree/main/lessons/205
please use minio, you can run it locally using docker, also aws sdk needs to use "path" style - github.com/antonputra/tutorials/blob/main/lessons/205/go-app/config.yaml#L8
you can add zig verion to github.com/antonputra/tutorials/tree/main/lessons/207
@@AntonPutra awesome!
Thx
PS: thanks for the docker file and tf setups too - devops is totally not my thing, and your repo has been super helpful for me ;)
Q: what is the db schema ? Zig Postgres is pretty picky about getting types right (like uuid - you using the uuid ext or a text field ? etc)
isnt erlang exactly made for this type of work? it would be interesting to see results.
I'll test it in the future
It would get clapped by a lot of language in this test by a long short.
Erlang, by it nature, isn't good at raw performance on limited single instance, but rather the scalability (both vertically and horizontally) that is crazy good.
@@OnFireByte everybody know the BEAM vm isnt particularly performant i want to see how well it handles latency and how graciously it breaks down as you overload the system.
It might sound stupid, but can you try Go vs Node in a AWS (DynamoDb) Scenario?
sure will do in the future
Please please, compare the javascirpt runtimes. Bun vs Deno vs Node .. pleasee
very soon :)
another great comparison. thanks for the video
my pleasure!
This is very misleading, taking in consideration zig one is not running on release mode
I got 1 PR already, I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
github.com/antonputra/tutorials/pull/268
please try swift and vapor versus go. and whatever but I think vapor is like spring, because of this gin is the best candidates to compare
ok will do
Is the zap instance prealocating buffer and handler pools for requests? Is there another reason why it has such a comparatively large idle footprint?
i got 1 PR - github.com/antonputra/tutorials/pull/268/files
maybe with this optimization it will not allocate so much, will see in a couple of days
It's running a debug build of zap. The results are meaningless.
Man, i really love your video! it give me exactly what i imagine of comparing and benchmarking two different language performance. i love to see if you also compare them with go!
thanks will do at some point
@@AntonPutra i watch your other video, i try to replicate the kubernetes monitoring but somehow i cannot make the cadvisor and prometheus showing my docker containers metrics. why? do you have video that do metrics but only for docker? no kubernetes ?
@@gneryze double check metric names, sometimes they change names
If you use Sonic-rs instead of Serde, you'll gain about ~20% in serialization and deserialization on Rust.
thanks! i'll try it out
yo i'm learning phoenix these days and idk if it deserve it so if you could share a benchmark and thx.
ok noted
awesome content, subscribed
thank you!
big fan of what you're doiing, i'm looking to do some benchmarks like this with my team and i was wondering what are you using for the client tests?
a simple http client written in rust
is there any plan to do compare between Go fiber vs echo. will appreciate that.
yes soon!
Great video!
Can you share which tool do you use for your diagram animations?
sure, adobe suite
Please try overhyped frameworks: nextjs/remix vs svelte vs qwik vs the fast frameworks: rust actix vs go fiber
ok will do!
Can you do rust(actix) vs nodejs(express) so I can convince my team and manager to use rust over nodejs 😅
noted :)
I think Zig results are worse because of, well, maturity. Zap is over 1 year old, while Actix has been around a few years already. There was time to multiple rounds of bug fixing and optimizations. I would say Zap is relatively young. To add to the subject, on the Rust side, you're still not using the "fastest framework out there". According to the TechEmpower Web Framework Benchmark there are multiple faster frameworks for Rust, including: May, Xitca and Ntex. Generally, in time we should se those scores come closer to one another.
thanks for the feedback, i'll try Ntex in near future
My takeaway from this is NOT to think Rust is so much better than Zig but to wonder what is wrong with Zap?
i built it without additional optimization flags as suggested here - github.com/antonputra/tutorials/pull/268
i'll retest it again in a couple of days
I've seen the new results on the Git repository, there's a remark, for the same number of Req/s , actix sends more Bytes than Zig, so Zig sends requests with less complete metadata (headers) than Actix, so the test is flawed.
yes, Actix send more headers
i love your tests :)
my pleasure :)
Ok thank you for this video. So, waiting for the update with zig build release.
yes soon
Is the zig version using a bit more memory because of arena allocation or something?
interesting, i use the latest version - github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L3
Using raw zig, it think it's better to using rust with hyper or warp
I don't think Zap is written from scratch in Zig. It binds to an underlying C library.
yes, it is a wrapper around facil.io c lib
He said it in the video. I agree it's not very meaningful if Zig is only a wrapper.
Will be interesting to see comparison between Gleam, Rust and Go
ok noted
Im stuck at 1000 requests per second using axum
it can do more :)
Zig can be a nice choice for replace C in the server, specially for teams with C developers already because of the inter op and syntax, but Rust sounds more attractive for general use.
Try Rust (Actix) vs C++ (CrowCpp) in a future test, will be a nice comparison.
Hope rust loses, I'm not a fan of cults
just merged the PR to optimize Zig and will rerun the test maybe tomorrow
haha, it will still exist and grow, the loser is you 😂
You should consider doing a test against gleam lang at some point.
ok, noted! never heard about gleam before
@@AntonPutra it compiles to Erlang and runs on the BEAM VM
If you did Erlang vs Elixir vs Gleam - should expect almost identical results
Maybe Zig allocated all required memory just once, and not performing re-allocation later? (To the question, why Rust using less when 1 MB, and Zig - 22,5 MB.)
Maybe I'll wait for the Zig experts with optimization tips, and I'll update the results if there is a significant difference, or I'll just create a TH-cam post.
You should include runq latency
runq latency??
Gleam and Go please!
ok noted
I was expecting Rust to sweat here but maybe that Zap framework isn't good
Collecting optimization tips from Zap (Zig) experts over the next couple of days, and I will release an updated video if there is a significant improvement, or just a TH-cam post.
other people commented about zig not being compiled in release mode, expect Anton to re-do this video 😆😁
Rust has an extremely mature asynchronous runtime, Zig doesn't exactly have concurrency as a core value proposition and it only promises to be a bit more convenient than C
Your Zig is NOT optimized whereas you have used Rust in release mode. Such a silly miss! Why would anyone trust this benchmark and the follow-on benchmark when you do not have the sense of doing the basic stuff!
I guess the existing bench is useful in showing how fully optimised Rust is able to hang in there with a debug mode Zig wrapper over an external C lib
Should keep the video, but change the misleading title :)
To give Anton credit where it’s due .. his devops skills are very excellent. That tutes repo is a great resource for learning more about k8s and Docker configs for different things
Not necessarily the same skill set as programming
will update the results in a couple of days, but i’ve only got this so far to optimize Zig. i hope it makes a big difference github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L15C14-L15C23
Next please do zig vs go
yes will do, but first i need to wait for a couple of days to get optimization tips from Rust experts
A lot of people complain that you are not using the languages the right way. I think it would be a good idea to consult with willing and knowledgeable people for each language. So that the programs are idiomatic and represent apps that you could see in he wild
yes, after getting a lot of advice just released updated video
Nice video🎉
thank you!
Can you do a bench mark on nodejs nestjs vs golang
yes soon
anyone know a good resource to learn zig?
this one is pretty good - zig.guide/
After looking at zap vs actix, I would say Zig implementation is much "low level" than Rust ie.
- Actix handles the routing for you while you hand write it in Zig.
- In rust the serialization is done by the framework directly (with macro) but in Zig you manually allocate and call stringifyBuf.
- In Zig, calling stringifyBuf could fail when the buffer overflow, this is not the same API parity with Rust which *practically* never fail. To make this fair, please update Zig implement to use a growable buffer or inline Rust json serialization to use array buffer.
- Like other commenter have said, you forgot to use release mode in zig
thank you! i just release a new video with updated results using all optimizations
try elixir and gleam sir. thanks
yes will do elixir soon!
My assumption here is this like the initial Go vs Rust where Go won. The biggest difference might end up being that you won't have zig experts come to tell you how you can improve it as it's still a very young language.
Agree! Thanks! I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
Nice idea!
you are comparing frameworks not languages
yes
Please do a laravel vs express video
ok soon
your laravel will got beaten by express easily
🍿 Benchmarks: th-cam.com/play/PLiMWaCMwGJXmcDLvMQeORJ-j_jayKaLVn.html&si=p-UOaVM_6_SFx52H
Here's an idea. It would be cool if you compared Rust with V lang.
ok interesting
Funny how the C# version here is faster than Go :D
?
@@AntonPutra 3:16 C# is > Go
Looking at the benchmarks it seems that you should set a standard for optimizations for the comparison to be fair. Obviously ideally you want to be using PGO but not all compilers have easy support for that. So I think a default of the highest optimization setting that doesn't break fp math, (as in no -Ofast or -ffast-math) so in the case of gcc/clang that would be -O3 combined with true link time optimizations would be a sane default. Not sure if you set true LTO for rust this time, so as rust by default doesn't optimize across crate boundaries this could be losing performance. To summarise LTO + O3 would be sane. Along with building for the architecture of the server you are running on (-march and -mtune equivalents). Edit: I see that you already set lto to true in rust and panic to abort.
@@anonymousalexander6005You would be surprised. LTO is not niche and should be a default. In terms of PGO what it does is it allows the compiler to know what are the hot paths in the code. Depending on the application PGO could even yield 10% performance or more. Getting around 5-10% is not uncommon. This is talking about recent clang and LLVM 18 and above. The compiler without LTO is only able to guess as to what the correct path to inline is, or if some loop gets called enough to be worth vectorizing or unrolling.
@@anonymousalexander6005You would be surprised. LTO is not niche and should be a default. In terms of PGO what it does is it allows the compiler to know what are the hot paths in the code. Depending on the application PGO could even yield 10% performance or more. Getting around 5-10% is not uncommon. This is talking about recent LLVM 18 and above. The compiler without LTO is only able to guess as to what the correct path to inline is, or if some loop gets called enough to be worth vectorizing or unrolling.
thanks for the feedback and yes i used lto for Rust - github.com/antonputra/tutorials/blob/main/lessons/207/actix-app/Cargo.toml#L11
go next hell level: different rust vs c++ vs carbon?
ok will do
Actix (rust) vs uwebsockets (js/c++)
Would be the only fair comparison if you want to compare JavaScript frameworks to Rust lol
ok noted
130K req/s - impressive
160k actually :)
@@AntonPutra right. That's insane
that thumbnail could give you trouble with the german authority so be careful. But cool video and as others mentioned zig should compile with releasefast
@@berserk.4121 they banned the use of red triangle and especially and upsidedown one
haha i didn't know
Also, regarding Zig, I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
@@salim444why?
@@AntonPutra thanks! I couldn't find any memory leak so I don't know why zap has a higher memory footprint
So the trick that Zig is using is to use more CPU usage and use more memory... :)) that is not good.. Rust does not need more CPU and memory...
i'll release updated video in a day with all optimizations on zig side
Wow, expected absolutely different results, looks like indeed Zap is too young to compete 😅
I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
У зига стабильная версия 0.13 вроде? Учитывая состояние документации и новизну языка, вполне себе хорошие результаты. Единственное что странно - использование памяти. Скорее всего "skill issue", но типа это не в упрёк, ты итак пишешь на куче языков, надеюсь, знающие зиг люди помогут хорошими коммитами =)
one govorya 0.13 is stale, so they suggest to use latest master - github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L3
I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
Pls can u tell me how i can create benchmark like this
well i have a lot of tutorials on my channel how to monitor containers with prometheus and grafana
one of them - github.com/antonputra/tutorials/tree/main/lessons/135
So rust is better?
idk yet, i got some tips to improve zig, will rerun the test in a day or two
Omg this crab 🦀 is eating everything 😭
Haha YES! 🦀🦀🦀
😂
Common rust W
Rust wins ... against an unoptimised release lol
I'll collect all the tips I can, including pull requests, over the next 2 or 3 days and will release an updated video or a TH-cam post with the results.
@@vercolit Literally a debug build
well, thank you for your infomation, i am learning Rust, but i got advertising from zig that they said Zig is faster than Rust, it make confuse, but now i known Rust is faster and more stable
waiting for the optimization tips... will update in couple of days
One important thing to note here is that Zig is not async hence must be using a thread pool, An async runtime has been proven to be more performant, using lesser resources and handle more concurrent connections.
is there anything i can do to improve Zig performance? more threads? container only see 1 core (2 vCPU)
First comment, wow long awaited for this😊
😊
man, rust is killing it
Rust on Release build, ZigZap on Debug build. The update video should be interesting!
yes will release updated video, but how come "./zig build" defaults to debug? it's a bit confusing
updated - github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L15C14-L15C23
Oherenchick ! krasavo
haha
Rust powerful 😂
will see
i notice zap kinda performs bad so bad it think zap either performs head to head or better at least
Zap was running on Debug build, while Actix on Release build. The table should be turned upside down once it's fixed.
well, i wouldn't say that i built it in debug mode - github.com/antonputra/tutorials/blob/main/lessons/207/zap-app/Dockerfile#L15
but i get your point and will rerun the test with some optimizations, including this - github.com/antonputra/tutorials/pull/268