The idle memory usage difference has its reasons. By default dotnet runtime just doesnt release its reserved memory unless there's memory pressure. If you want it to release it more aggressively, you should run it as a desktop workload instead of a server workload (default).
Oooo reasons, reasons, reasons, come one, face it, Go lang with simpler approach without OOP studpidity rulset has shown it can perform. We kewn all a long, that is a case. Face it, Go lang is the new "C#" as new popular language for beginners. Modern C# is just what C++ was in the 80s. Even C# author said, C# is more C++ replacement that Java replacement.
@@Sam-gd4xp oh, I like golang as well and I've used it alot ❤️ But personally I wouldn't have courage to start developing a large scale enterprise-y system with it. Everything being so simple and imperative means there are more statements you'll have to manage, which means you'll have more state to manage, which means there'll be more chance for bugs. Golang's range requires much more statements, state and ceremony than C#'s Select(...).Where(...).GroupBy(...).Join(...).OrderBy(...)... I know there are some more functional slice utils, but still a lot is missing. C#'s LINQ (with and without EF), expressions and way superior generics is something I personally couldn't develop a large scale system without... But I do love to use golang for more algorithmic stuff and I'm also super interested in how golang's generics and itetators will turn out and improve 😋
Your methodology is excellent. I much prefer this to the gamified mess that techempower has become. Startup time/cost, container size, memory/cpu usage, baseline memory/cpu usage are extremely valuable to us.
Cool test actually. A few possible improvements except trying the AOT: 1. Try enabling a full PGO (tiered PGO is enabled in .NET 8): # Full PGO ENV DOTNET_TieredPGO=1 ENV DOTNET_TC_QuickJitForLoops=1 ENV DOTNET_ReadyToRun=0 2. Use slim app configuration: WebApplication.CreateBuilder() vs WebApplication.CreateSlimBuilder() 3. Try increasing the minimal number of threads: ThreadPool.GetMinThreads(out _, out int completionPortThreads); ThreadPool.SetMinThreads(100, completionPortThreads);
also the two async operations (await cmd.ExecuteNonQueryAsync() and await amazonS3.Upload(appConfig.S3Bucket, image.ObjKey, appConfig.S3ImgPath)) are running sequentially, while they could be started both together and awaited after they both have been started; sure, it's a small to really small improvement, but this is the point of knowing well the language and doing a test well: it's not easy *at all*
@@AntonPutra I actually think you tested it fine. AOT is only for startup time. Tiered PGO is the default, so it makes sense to keep it here. My one complaint is that if you want this to be a "real world" test case your code should use Entity Framework as that's the most standard thing. I took a look at the code and you're doing it with Npgsql's built-in datasource. I don't know anyone who would do this in a real world application. There's all sorts of caching that you're missing from Entity Framework. Note that I think the way you did it is actually faster. EF has overhead. But I would still very much like to see that comparison instead.
None of these are needed, with vanilla configuration .NET is strictly better than Go. Anyone who ever looked at Go's compiler output and GC throughput knows that both are underwhelming. Note that for *small* deployments like these using NativeAOT is a good idea. .NET 9 also enables DATAS by defvault which is a new GC mode for more dynamic scaling of active heaps and their sizes based on load profile. This massively improves low heapsize scenarios like this one. Generally speaking you have to understad that for high throughput workloads 256mi is anemic. Bump it up to 512mi and you will see a completely different picture. Same applies to more core which .NET's threadpool and task system can utilize much more effectively than fixed runtime threads setup in Go. Lastly, I suspect the culprit here is S3 SDK, as it has quite room for improvemet and Amazon posted like a week ago that the have new preview version which is faster and has much less wasted allocations. The open question is also about Go using connection pool while .NET seemingly avoids opting into that, which would impact the behavior significantly and make it apples to oranges comparison.
11 years with .NET at my main jobs (I'm a solution architect), 6 years with Golang (side projects for other clients' companies). I think .NET is more suitable for fast development, quick adaptation to changes, and Enterprise-focused products. Golang is better for cost-efficient projects, but requires more maintenance.
@@MrSamisack it's often start with system thinking, learn how a system works, what are components, how to make it secure and realiablely. 4th-8th years try to build all skillset that you target to be master. .NET ecosystem , Cloud Provider, Database, AI... I spent almost 12-15 hours per days for learn and practices.
@@deado7282 it is an ok default if you deploy to piss weak anemic container configurations and then have to scale with replica count and nodes, because go cant optimally do otherwise, instead of scaling up individual container resources, the ecosystem is in a different place now anyway
@@deado7282 Fiber also limits you. That would be far more fair. Not to mention that there's something bad with the C# build, because my projects are way bigger in code but way smaller when built. He did something bad.
At 8:01 you mention that theres a significant higher latency for C#. It looks worse than it actually is, since the graph doesn't start at zero (unlike the other graphs in this view). The difference is still significant, but being mindful about these things adds clarity for the viewer and potentially saves you from a wrong conclusion in the future :)
@@AntonPutra startup time will increase by a miniscule amount as it needs to decompress before running. For this use case, using UPX is a good move as it will reducesdownload time.
@@ArnabAnimeshDas got it. unfortunately, it's hard to measure boot time in Kubernetes because the minimal interval is 1 second, and I want to focus these benchmarks on Kubernetes.
Also the images are already compressed by the container registry and runtime. So double compression is not all that effective in general. Something like the trimming and Native AOT in .NET does help a bunch though. Go already does a fairly okay job.
One thing to add - about 10% of men have reduced color vision. For those of us, the difference between these two graphs are almost impossible to distinguish.
This is the first test video I've seen of yours and I love how thorough you are, from methodology and tools, to the results. However, as someone who is color deficient, I cannot tell the difference between low-contrast colors easily or at all. For your next test, please use high-contrast colors for all test subjects.
My thoughts exactly. Something in the setup of both versions are bad, neither go and c# seems to utitlize resources nowhere near 100%, yet they all start failing request. Maybe they can't see the resources they have available or similar and they are throtthling themselves by using less thread/connection/something that they could. Go (fiber) ofc will win anyways, but based on my very similar tests, the difference is only 10-15% in requests per second and this was with net6, not net8 (which can be significantly faster).
@@metaltyphoon yes, which means the margin should be even smaller for him compared to what I measured in net6 era, since net8 is faster then net6. My tests were capping cpu, but here we can see both frameworks failing without using the available resources, so something seems off.
no vanila kubernetes cluster with containerd runtime. 2cpu and 256mb memory - github.com/antonputra/tutorials/blob/main/lessons/202/deploy/cs-app/deployment.yaml#L26-L32 have you tried it in k8s or just standalone? default file descriptors 1024 could play a role as well
I was also surprised, I did some local test during spring for our new services and could easily handle 15 000 requests per second on my machine, though I did not run under k8s and did have more memory and CPU. BUT I did not test 15 000 clients though. @AntonPutra did your test use concurrent clients doing separate calls or just requests? That could make a difference I think, especially if the clients connect using HTTP/2 in which case one client would be able to do many many more requests in the same time since you save on TCS setup, and that might account for the higher numbers I got.
Hey, great job! One comment I have is: a lot will come down to how good the Amazon S3 and Postgres are optimized, and in DOTNET MS SQL and Azure Blobs will probably be a priority. Could you do a dedicated test for streaming back local file by id (from disk)? That would show how much HTTP connections specifically each app can handle.
I have few tutorials that can help 1. github.com/antonputra/tutorials/tree/main/lessons/135 2. github.com/antonputra/tutorials/tree/main/lessons/136 3. github.com/antonputra/tutorials/tree/main/lessons/137
Well, I used the available documentation that most people would use to compile and build Docker images. I'm pretty sure you can optimize both Go and C# further. But I get your point, and next time I'll test with AOT as well. If you have anything that could help me optimize .NET, please share it, and I'll definitely use it!
@@AntonPutra unless you change the algorithm you uses, there is not a lot of options that can be used to optimize your Go app, you only have the PGO (which also can be used with C#). In C# you just have to compile the program like Go to get a better performance.
@@AntonPutra luckily when you create a .NET solution you have the choice of selecting web api with AOT, which is a very good starting point, i strongly recommend you to check it out as it has a huge performance difference and the boot time is reduced, the only downside i would say is the image size. I made a POC for my company, it was a stress test of 1000 concurrent users making 1000 requests each and the memory stayed at 30mb with cpu to 1.5%
Nice work, man. One thing is I wonder why dotnet starts to drop packets and latency having so few of the cpu time and memory consumed. I managed to make it serve smth like several thousands of requests per second on a single thread with 200mb of ram and batched sql server inserts, on a windows machine though
Once a couple of years ago I had to use .NET CORE C# on Linux and it ate up a lot of memory and CPU. Since then nothing has changed. I can be told that I haven't optimized my .NET environment, but I don't understand why it isn't optimized out of the box
Once again a spectacularly well thought out test. We can say that C# wins a proud silver medal in this head to head :D Would you be interested in a fresh comparison of some of the different javascript runtimes, like node.js, deno and bun?
You should ask community to create most efficient app they can and give them time to do that on github. Then test them. That would be the best compression in existence
Excellent video with a very very good testing methodology! Just one note: next time can you use colors that are better differentiated? (i'm colorblind). Thanks!
Very informative as always, thanks also just a thought, maybe revisit apache vs nginx. mpm_event module for apache with default config sounds pretty strong I'd like to see how they would compare now keep up the good work
well i have all source code, including terraform, helm charts everything even dashboards in the repo, link in the description - github.com/antonputra/tutorials/tree/main/lessons/202
@@AntonPutra while at it. I see more an more job postings for python django, fast api, flask. I would expect these to perform "bad", but so many people choose them. I wonder if they are good or not
@@jozsab1 It's a very reliable and well-tested framework. We've been running Django as the main API gateway at the company where I work for the last few years with no issues at all. However, most of our other microservices are written in Go :)
It seems you didn't read any of the comments from the Go/Java comparison, the Go binary can be further reduced by removing debugging symbols with the linker options `-s -w`. You are also using an outdated AWS SDK.
it is in this video, and the difference is only 6 MB - th-cam.com/video/56TUfwejKfo/w-d-xo.html I'll update sdk in the next lesson, but it does not affect performance it's just a wrapper around rest api
interesting, looks like they integrate cloud sdks to the standard library.. so it sort of wrapper around all clouds? how well it is supported? I'll take a look...
Hey Anton can you redo the tests now with .NET 9 that is out? Microsoft claims that is 10 times faster then NodeJS(Express) and 5 times Faster then Go(GIN)
Which memory usage do you measure? Reserved or Committed? .NET GC eagerly reserves a lot of memory upfront, but does not commit it until necessary. in .net 8 they introduced Dynamic Adaptive GC mode, but it's on by default only for AOT application. DOTNET_gcServer=1 DOTNET_GCDynamicAdaptationMode=1 Or System.GC.DynamicAdaptationMode
well the same as kubernetes uses to oomkill my applications :) to be more precises container_memory_working_set_bytes / container_spec_memory_limit_bytes
Автор плодит холивары. Почему-то он взял AspNetCore старой версии, не запублишил(то есть нет компайла в AOT нейтив), без тримминга без ничего и сравнивает производительность с Go(который по умолчанию AOT), ближайшим аналогом которого для тестирования подобных задач больше подошел бы Blazor из мира Dotnet.
@@akknaodinden странно. Ютьюб как-то через раз ответы отправляет. Писал, что интересно еще сравнить производительность, когда не CreateBuilder, а CreateSlimBuilder
Your videos are very interesting ... Keep doing it. I have one question. Is it possible for you to create videos for gin, fiber and echo frameworks? thank you
I had a look at the source code a and configuration after the last video and I wondered why you used minIO. Thanks for giving the explanation. I did not know it supports (partially) the S3 API. At work, we have an application that uses it but more as a cloud agnostic object storage. The use cases here are more realistic than the tech empower benchmarks ones. The none fixed size of the DB pool for DotNet impacts the latency, opening a connection to the DB (especially if TLS was used) is expensive.
thanks! minio can also be used as a replacement for hadoop for some data lake stuff, etc. It uses significantly less disk space while supporting the same replication, but it's harder to scale. If you are on-prem, it's a viable alternative. I agree with the pool size, just the defaults, no more, no less.
4 CPUs and 8 GB of memory. From now on, I'm only testing apps in production-ready EKS clusters to make it similar to what we actually run in production environments. I use m6a.2xlarge EC2 instances for the nodes...
@@AntonPutra because Dart serverpod is new framework, and we wanna see how it compares against popular alternative like Gin. Comparing it with Fiber or std I think is useless, because we already know that they are too fast :).
Sounds reasonable as a whole. .Net isn't particularly efficient unless you do a LOT of massaging. Then again, like ThePrimagen usually puts it, do you have more microservices than users? If I'm not doing high performance, I'd probably sacrifice efficiency for "ease of use and support", especially in a MS centric environment. If I am, then "who cares about your experience, fps for life" (old competitive Quake joke, where you'd sacrifice ANYTHING for more fps because max_speed was tied to fps).
Why C# fanboys always crying when they dont get the result that they want. If you want more optimization the same thing can happen in go. Do your own test without biases.
Is it fair to compare a web framework specifically designed for maximum performance with many limitations and unimplemented web features to a standard enterprise application framework that has everything you need? At least this doesn't exactly match the title of the video.
The comparison seem kind of fair, but they're room for improvement, but i think that a good general "default" setup representation. But small precision (with a little history): It's not dotnet api or dotnet core, it's the aspnetcore framework just like gin/fiber are web framework that you're testing against, they may exist other web framework for .NET, but i agree that is most popular and used one. Nowadays it's just .NET (or dotnet) no "Framework", when we say .NET framework, it mean the old .NET Stack that was only working/designed to work on windows (yeah.. yeah...I know about MONO, but that out of the scope for this simple history) dotnet core was/is the rewrite of the .NET stack from scratch. Microsoft choose to call it dotnet core, because at first it was only a subset of library (the core part), and apsnet core was also the rewrite from scratch of the web framework stack that only supported a subset of the aspnet MVC api, why they didn't change aspnetcore to aspnet, because it's already exist (they were (as far as i know) 3 Microsoft's web framework: ASPNET, ASPNET WebForm, ASPNET MVC (with 3 version of that framework is recall correctly)). So why did they change .NET core to .NET ? i don't know the real reason, but i have some idea like stopping people to not use the new improve version of .net because in their head .NET framework is the supported version of Microsoft versus .NET Core was not as feature complete and not maintainted as good by microsoft because it's now not part of the automatic windows update. Fun fact that most people don't know, but a certain point in time, you were able to start a apsnetcore project using the .NET framework and this was not a hack, it was a valid use case, but today the required version of .NET for aspnercore is too high and .NET framework support was dropped, but switching from aspnetcore using .NET framework to aspnetcore using .NET Core (now know as .NET) was really easy.
Thanks for the clarification. I didn't have any experience with .NET, but someone suggested trying it out with Minimal API, so I did. I'm learning a lot from the feedback I get, lol.
@@AntonPutra Did you really use Release mode? I checked Dockerfiles in your repo and I see that you use dotnet publish command without -c Release parameter. dotnet publish command builds app in Debug mode by default. Edit: I rechecked, since .NET 8, publish uses Release by default, so it seems everything is ok.
Yeah. I think it's no surprise that Golang is faster - but .NET is well established and really powerful. And in real use case, it's still REALLY fast. Performance should only be the deciding factor when you REALLY need it. Majority of projects don't. In most cases, there's more to it - from ecosystem to team/personal experience and preference.
@@Sam-gd4xp You're either trolling or you simply never comprehended the idea of "different tools for different jobs". I've never seen anyone mad that Golang is faster. Everyone knows it, everyone accepts it, and people happily pick it when they need it. Both languages are tools, and only immatures do these kind of "performance wars" and call each other mad over it.
@@TehGM if you're the head of small startup and small firm, and you learn basic accounting and project management properly, you will see. That's why Google push so hard for kotlin multiplatform bro. But will it come? After so much Java VS kotlin legal battle
Basically, if you want something that works and easy, go with Java, C#, Node. If you want improved performance and efficiency without too much work, go with Go. If you want the most performance, speed and efficiency you can get, with the downside of complexity, go with C, C++, Rust
Rust is not ready for the cloud, with poorly maintained SDKs, and many of them are just slower than Go implementations. But in theory, Rust should be second after C.
🔴 To support my channel, I'd like to offer Mentorship/On-the-Job Support/Consulting (me@antonputra.com)
Amazing. Bro where can I find your C# and Go code used for testing. Any GitHub links?
Rust vs Go
Haskell please 🙏 ChatGPT can help🫠
Scala sttp is a great wrapper, and akka/pekko perform very strongly in general with small heaps🤫
zig vs Rust
Zig vs Go
Haskell vs Elixir Vs Gleam
@@ulrich-tonmoy ok, just added to my list!
.NET should not be dropping requests whole using so little CPU. Something about the setup is wrong here.
Not only that but it doesn't make sense to have so many DB connections.
I haven't looked at the code but I suspect bad quality code.
Author of Fiber here, keep up the good work. If you are interested you should try enabling the Prefork option for even more performance.
Now that .NET 9 is out, this would need a refresh. New server mode is biased towards low memory consumption + plethora of perf improvements
in fact，this benchmark has some issues，he's code show he was use dotnet 8.0 but he not open server gc model，in workstation gc QPS too low。
@@mysteriouseyes6960 the code in go as well. so why not do your own test instead
I like these real world test scenarios. And I love that you increased RPS until failure. Please keep doing that with every benchmark
I wish you would increase RPS in second test as well
thanks, will do! i started to use eks to run those tests...
The idle memory usage difference has its reasons. By default dotnet runtime just doesnt release its reserved memory unless there's memory pressure. If you want it to release it more aggressively, you should run it as a desktop workload instead of a server workload (default).
noted, memory does not play a significant role in these benchmarks
Oooo reasons, reasons, reasons, come one, face it, Go lang with simpler approach without OOP studpidity rulset has shown it can perform. We kewn all a long, that is a case. Face it, Go lang is the new "C#" as new popular language for beginners. Modern C# is just what C++ was in the 80s. Even C# author said, C# is more C++ replacement that Java replacement.
@@Sam-gd4xp oh, I like golang as well and I've used it alot ❤️ But personally I wouldn't have courage to start developing a large scale enterprise-y system with it. Everything being so simple and imperative means there are more statements you'll have to manage, which means you'll have more state to manage, which means there'll be more chance for bugs. Golang's range requires much more statements, state and ceremony than C#'s Select(...).Where(...).GroupBy(...).Join(...).OrderBy(...)... I know there are some more functional slice utils, but still a lot is missing. C#'s LINQ (with and without EF), expressions and way superior generics is something I personally couldn't develop a large scale system without... But I do love to use golang for more algorithmic stuff and I'm also super interested in how golang's generics and itetators will turn out and improve 😋
@@Sam-gd4xp Don't be a fanboi
@@Sam-gd4xp No, C# is so flexible that it can and needs to be optimized for a particular task.
Your methodology is excellent. I much prefer this to the gamified mess that techempower has become. Startup time/cost, container size, memory/cpu usage, baseline memory/cpu usage are extremely valuable to us.
thank you!
@@AntonPutra mas anton, ayo bikin site kaya techempower. Hasilnya ditunjukkan di site. Di run setiap tahun sekali utk bahasa2 populer
Cool test actually.
A few possible improvements except trying the AOT:
1. Try enabling a full PGO (tiered PGO is enabled in .NET 8):
# Full PGO
ENV DOTNET_TieredPGO=1
ENV DOTNET_TC_QuickJitForLoops=1
ENV DOTNET_ReadyToRun=0
2. Use slim app configuration:
WebApplication.CreateBuilder() vs WebApplication.CreateSlimBuilder()
3. Try increasing the minimal number of threads:
ThreadPool.GetMinThreads(out _, out int completionPortThreads);
ThreadPool.SetMinThreads(100, completionPortThreads);
thanks for the tip! I'll definitely try it next time!
@@AntonPutra yeap, it would be interesting with Native AOT too
also the two async operations (await cmd.ExecuteNonQueryAsync() and await amazonS3.Upload(appConfig.S3Bucket, image.ObjKey, appConfig.S3ImgPath)) are running sequentially, while they could be started both together and awaited after they both have been started; sure, it's a small to really small improvement, but this is the point of knowing well the language and doing a test well: it's not easy *at all*
@@AntonPutra I actually think you tested it fine. AOT is only for startup time. Tiered PGO is the default, so it makes sense to keep it here. My one complaint is that if you want this to be a "real world" test case your code should use Entity Framework as that's the most standard thing. I took a look at the code and you're doing it with Npgsql's built-in datasource. I don't know anyone who would do this in a real world application. There's all sorts of caching that you're missing from Entity Framework.
Note that I think the way you did it is actually faster. EF has overhead. But I would still very much like to see that comparison instead.
None of these are needed, with vanilla configuration .NET is strictly better than Go. Anyone who ever looked at Go's compiler output and GC throughput knows that both are underwhelming. Note that for *small* deployments like these using NativeAOT is a good idea. .NET 9 also enables DATAS by defvault which is a new GC mode for more dynamic scaling of active heaps and their sizes based on load profile. This massively improves low heapsize scenarios like this one. Generally speaking you have to understad that for high throughput workloads 256mi is anemic. Bump it up to 512mi and you will see a completely different picture. Same applies to more core which .NET's threadpool and task system can utilize much more effectively than fixed runtime threads setup in Go. Lastly, I suspect the culprit here is S3 SDK, as it has quite room for improvemet and Amazon posted like a week ago that the have new preview version which is faster and has much less wasted allocations. The open question is also about Go using connection pool while .NET seemingly avoids opting into that, which would impact the behavior significantly and make it apples to oranges comparison.
11 years with .NET at my main jobs (I'm a solution architect), 6 years with Golang (side projects for other clients' companies). I think .NET is more suitable for fast development, quick adaptation to changes, and Enterprise-focused products. Golang is better for cost-efficient projects, but requires more maintenance.
hi! what do i need to lear to become a .net solution architect?
@@MrSamisack it's often start with system thinking, learn how a system works, what are components, how to make it secure and realiablely. 4th-8th years try to build all skillset that you target to be master.
.NET ecosystem , Cloud Provider, Database, AI... I spent almost 12-15 hours per days for learn and practices.
@@anhcoder thanks 🫂
why does goland requires more maintenance
@ more code, more simple, more customization, more dead and opensource libs then more maintenance cost.
I don’t understand why both start dropping request while cpu usage is nowhere near 100 and memory usage is less than 50%
Imagine this might be due to OS syscall bottleneck or physical hardware bottleneck (network card throughput)
maybe one core of processor is used on 100% and other at 20%
the problem is in the Config.cs, he is loading the entire configuration each time he needs a configuration value.
bogus tests
Use native aot for C#. Memory usage will be way less, and startup time will be very fast
AOT is not a sensitive default since it limits you in terms of 3rd party packages.
@@deado7282 it is an ok default if you deploy to piss weak anemic container configurations and then have to scale with replica count and nodes, because go cant optimally do otherwise, instead of scaling up individual container resources, the ecosystem is in a different place now anyway
i'll try
@@deado7282 Fiber also limits you. That would be far more fair. Not to mention that there's something bad with the C# build, because my projects are way bigger in code but way smaller when built. He did something bad.
This would be a fair comparison to fiber as it is the fastest option to run aspnetcore.
At 8:01 you mention that theres a significant higher latency for C#. It looks worse than it actually is, since the graph doesn't start at zero (unlike the other graphs in this view). The difference is still significant, but being mindful about these things adds clarity for the viewer and potentially saves you from a wrong conclusion in the future :)
To further reduce the image size of Golang, you can use UPX to further compress the executable which usually results in 40-50% file size reduction.
cool, thanks for the tip! i'll try it out. Does it affect performance, or just in case you need to debug something?
@@AntonPutra startup time will increase by a miniscule amount as it needs to decompress before running. For this use case, using UPX is a good move as it will reducesdownload time.
@@ArnabAnimeshDas got it. unfortunately, it's hard to measure boot time in Kubernetes because the minimal interval is 1 second, and I want to focus these benchmarks on Kubernetes.
Also the images are already compressed by the container registry and runtime. So double compression is not all that effective in general. Something like the trimming and Native AOT in .NET does help a bunch though. Go already does a fairly okay job.
@@EraYaN 👌
Keep the benchmarks coming! Love it!
thanks! will do, rust vs go coming in couple of days :)
I have never seen dotnet to drop requests if is not 95% plus on cpu ram etc this is just weird.
i'll add more metrics next time from cadvisor - github.com/google/cadvisor/blob/master/docs/storage/prometheus.md
Thanks for the thorough test! Why do you use .NET 6 while the latest LTS version is .NET 8 ?
my pleasure! i used 8.0, you can see it from the source code - github.com/antonputra/tutorials/blob/main/lessons/202/cs-app/Dockerfile#L1
One thing to add - about 10% of men have reduced color vision. For those of us, the difference between these two graphs are almost impossible to distinguish.
Can you please create a comparison video between go and PHP Swoole?
ok!
Congrats you got new subs!!
Love with your test, please add more C# comparison
thanks! will do more in the future
I mean, it is gin, one of the slowest go libraries. You used Fiber in the last video.
microsoft compared .net with gin, i used fiber in this video - github.com/antonputra/tutorials/blob/main/lessons/202/go-app/main.go#L12
Gin is not slow…
@@einstein_god I've been using it for benchmarks, it is definitely slower than fiber..
This is really Informative... superb work, super setup, i would really like to learn to do this
This is the first test video I've seen of yours and I love how thorough you are, from methodology and tools, to the results. However, as someone who is color deficient, I cannot tell the difference between low-contrast colors easily or at all. For your next test, please use high-contrast colors for all test subjects.
thank you! it's just default colors that Grafana uses for two different subjects, i'll see if i can improve
I am loving these videos
Why some requests start to drop? I see the CPU utilization and memory usage are very far from 100%. Where is the bottle neck coming from?
it depends on the implementation of the http server, some prefer to drop others try to process every single request...
My thoughts exactly. Something in the setup of both versions are bad, neither go and c# seems to utitlize resources nowhere near 100%, yet they all start failing request. Maybe they can't see the resources they have available or similar and they are throtthling themselves by using less thread/connection/something that they could. Go (fiber) ofc will win anyways, but based on my very similar tests, the difference is only 10-15% in requests per second and this was with net6, not net8 (which can be significantly faster).
@@hupett his repo has .net 8 not 6
@@metaltyphoon yes, which means the margin should be even smaller for him compared to what I measured in net6 era, since net8 is faster then net6. My tests were capping cpu, but here we can see both frameworks failing without using the available resources, so something seems off.
This is where GC starts to kick in. If we use something like dotnet-monitor for monitoring GC in Prometheus that might show us the real picture
Can you try again with .net 9
yes in the new benchmark i'll use .9 version
@@AntonPutra do you use ef core, if you do, use dbcontextfactory in offical doc
I'm sure you didn't make any adjustments to the operating system. .net core doesn't give me any problems until it reaches 50k/sec.
no vanila kubernetes cluster with containerd runtime. 2cpu and 256mb memory - github.com/antonputra/tutorials/blob/main/lessons/202/deploy/cs-app/deployment.yaml#L26-L32
have you tried it in k8s or just standalone? default file descriptors 1024 could play a role as well
I was also surprised, I did some local test during spring for our new services and could easily handle 15 000 requests per second on my machine, though I did not run under k8s and did have more memory and CPU.
BUT I did not test 15 000 clients though.
@AntonPutra did your test use concurrent clients doing separate calls or just requests?
That could make a difference I think, especially if the clients connect using HTTP/2 in which case one client would be able to do many many more requests in the same time since you save on TCS setup, and that might account for the higher numbers I got.
Hey, great job! One comment I have is: a lot will come down to how good the Amazon S3 and Postgres are optimized, and in DOTNET MS SQL and Azure Blobs will probably be a priority. Could you do a dedicated test for streaming back local file by id (from disk)? That would show how much HTTP connections specifically each app can handle.
Thanks! I was actually thinking about Kafka consumer/producer implementations and maybe a simple ETL pipeline. What do you think?
Can you do a video on how to create these graphs to test apis. Thanks
I have few tutorials that can help
1. github.com/antonputra/tutorials/tree/main/lessons/135
2. github.com/antonputra/tutorials/tree/main/lessons/136
3. github.com/antonputra/tutorials/tree/main/lessons/137
I like this kind of content, you've got one more subscriber! 😊
thank you! rust vs go coming in couple of days :)
I am curious about the performance of .NET Orleans compared to Go
ok, noted!
Nice work
Can you do Java vs C# next ?
thanks, maybe not next but i'll do it for sure
@@AntonPutra Please do! Looking forward to seeing the same test between Java vs C#. Also would be great to use .Net Native AOT.
@@Tolg i got PR for aot so i'll be definitely testing it soon.. - github.com/antonputra/tutorials/tree/main/lessons/202/cs-app-aot so i'll be testing
This comparison is not fair, as you didnt compile the minimal api using AOT
still c# have a very good perform, i think it has the best performance for a general us language
Well, I used the available documentation that most people would use to compile and build Docker images. I'm pretty sure you can optimize both Go and C# further. But I get your point, and next time I'll test with AOT as well. If you have anything that could help me optimize .NET, please share it, and I'll definitely use it!
have you tried rust? :)
@@AntonPutra unless you change the algorithm you uses, there is not a lot of options that can be used to optimize your Go app, you only have the PGO (which also can be used with C#). In C# you just have to compile the program like Go to get a better performance.
@@AntonPutra luckily when you create a .NET solution you have the choice of selecting web api with AOT, which is a very good starting point, i strongly recommend you to check it out as it has a huge performance difference and the boot time is reduced, the only downside i would say is the image size.
I made a POC for my company, it was a stress test of 1000 concurrent users making 1000 requests each and the memory stayed at 30mb with cpu to 1.5%
You could have used distroless image for .NET as well. 8.0-noble-chiseled
Will you test with Native AOT C# app?
yes, next week. first python then c#aot
Nice work, man. One thing is I wonder why dotnet starts to drop packets and latency having so few of the cpu time and memory consumed. I managed to make it serve smth like several thousands of requests per second on a single thread with 200mb of ram and batched sql server inserts, on a windows machine though
Once a couple of years ago I had to use .NET CORE C# on Linux and it ate up a lot of memory and CPU. Since then nothing has changed. I can be told that I haven't optimized my .NET environment, but I don't understand why it isn't optimized out of the box
interesting, i'll be doing a new one soon with .net 9
Once again a spectacularly well thought out test. We can say that C# wins a proud silver medal in this head to head :D
Would you be interested in a fresh comparison of some of the different javascript runtimes, like node.js, deno and bun?
thanks! yes but next rust vs go with new updated frameworks and sdks :)
You should ask community to create most efficient app they can and give them time to do that on github. Then test them. That would be the best compression in existence
It's crazy how closely aligned the GO and C# RPS was to start. There has to be an external limiting factor.
Excellent test & very thorough....thanks.
thank you!
Do a Go vs PHP please. We know Go is faster, but it would be interesting to see the differences. You can use FrankenPHP or Symfony Flex.
ok will do in a week or so
Seems unfair because PHP does not run by itself. Lower level of socket connection is handled by either Apache or Nginx.
C# VS Java?
ok coming
Interesting, thanks! What about Temporal vs Kafka next?
ok noted!
You have found some kind of holy grail of programming content for TH-cam 😄 Very addictive videos -> I subscribed
thank you! :)
Excellent video with a very very good testing methodology! Just one note: next time can you use colors that are better differentiated? (i'm colorblind). Thanks!
Very informative as always, thanks
also just a thought, maybe revisit apache vs nginx. mpm_event module for apache with default config sounds pretty strong
I'd like to see how they would compare now
keep up the good work
thank you! will do! is mpm_event module enabled by default or i need to turn it on?
Hey, can you please compare latest version of dotnet 9 with golang again?
yes will do soon
Would be interesting to know how to setup all this
well i have all source code, including terraform, helm charts everything even dashboards in the repo, link in the description - github.com/antonputra/tutorials/tree/main/lessons/202
I personally would love some exotic test like Rust, c++ ( oatpp, crow.. ), apache compared to GO.
noted!
@@AntonPutra after i made the comment youtube recomended your rust, node, java ... comparisons. awesome work !
@@AntonPutra while at it. I see more an more job postings for python django, fast api, flask. I would expect these to perform "bad", but so many people choose them. I wonder if they are good or not
@@jozsab1 It's a very reliable and well-tested framework. We've been running Django as the main API gateway at the company where I work for the last few years with no issues at all. However, most of our other microservices are written in Go :)
@@AntonPutra I'm not a webdev, for me it's just raw numbers vs raw numbers. For you, maybe a topic to explore / make another video
maybe try some functional or new language? i suggest elixir and gleam for next video:)
okay, will do. i hope they have up to date aws sdks. :)
How realistic is it to use Fiber for production workloads? What is a more realistic framework for enterprise prod workloads in Go?
i think standard library is goon enough since 1.21
Nice stuff! Thanks
thank you!
Please repeat the test in November with .NET 9
will do, i got a lot of feedback how to optimize it as well, haha
It seems you didn't read any of the comments from the Go/Java comparison, the Go binary can be further reduced by removing debugging symbols with the linker options `-s -w`.
You are also using an outdated AWS SDK.
it is in this video, and the difference is only 6 MB - th-cam.com/video/56TUfwejKfo/w-d-xo.html
I'll update sdk in the next lesson, but it does not affect performance it's just a wrapper around rest api
.NET framework is an overloaded term in the .NET ecosystem since it refers to the legacy framework
Can you uncover cloud language winglang? What is your opinion about this approach ?
interesting, looks like they integrate cloud sdks to the standard library.. so it sort of wrapper around all clouds? how well it is supported? I'll take a look...
is it Delphi from Embarcadero good in 2024?
Delphi is still one of the best UI library and IDE. But the price is so crazy
Hey Anton can you redo the tests now with .NET 9 that is out?
Microsoft claims that is 10 times faster then NodeJS(Express) and 5 times Faster then Go(GIN)
Which memory usage do you measure? Reserved or Committed? .NET GC eagerly reserves a lot of memory upfront, but does not commit it until necessary. in .net 8 they introduced Dynamic Adaptive GC mode, but it's on by default only for AOT application.
DOTNET_gcServer=1
DOTNET_GCDynamicAdaptationMode=1
Or System.GC.DynamicAdaptationMode
Also, C# compiler interns strings by default, AFAIK Go does not, so you give slight advantage to C# in this case.=)
The repo is in his profile under "Tutorials". It's the most recent commit atm.
well the same as kubernetes uses to oomkill my applications :) to be more precises container_memory_working_set_bytes / container_spec_memory_limit_bytes
Please test drogon framework 🙏
this one? drogon.org/
@@AntonPutra it would ne interesting to see a c++ web framework and go comparison.
@@suyulmaz4 Sure, any specific C++ framework in mind? Also, would you add any new test scenarios?
Автор плодит холивары. Почему-то он взял AspNetCore старой версии, не запублишил(то есть нет компайла в AOT нейтив), без тримминга без ничего и сравнивает производительность с Go(который по умолчанию AOT), ближайшим аналогом которого для тестирования подобных задач больше подошел бы Blazor из мира Dotnet.
AOT не надо, PGO после "прогрева" должен дать более оптимизированный код, чем получится при AOT.
Интересно другое: (из доков)
When multiple containerized apps are running on one machine, Workstation GC might be more performant than Server GC
@@VoroninPavel У меня фобия что после PGO что-то отпадёт.
@@akknaodinden ну это только если там бага какая суровая.
Еще интересно бы посмотреть на тест не с CreateBuilder, а CreateSlimBuilder()
@@akknaodinden странно. Ютьюб как-то через раз ответы отправляет.
Писал, что интересно еще сравнить производительность, когда не CreateBuilder, а CreateSlimBuilder
I like this comparison keep up like this for benchmark maybe with aot make some difference
thanks, I'll try AOT next time
Please can we get an update for .net 9?
What diagram tools are you using
should also compare the image size
noted
Please test elixir phoenixframework compare to golang fiber and rust actix web. Thanks. 😁
ok noted!
In test 1, in C# , the json serialization can a improve by setting the context
noted! i'll try next time when i prepare a test with c#
Your videos are very interesting ... Keep doing it.
I have one question.
Is it possible for you to create videos for gin, fiber and echo frameworks?
thank you
thanks you! i'll see what i can do and when
I had a look at the source code a and configuration after the last video and I wondered why you used minIO. Thanks for giving the explanation. I did not know it supports (partially) the S3 API. At work, we have an application that uses it but more as a cloud agnostic object storage.
The use cases here are more realistic than the tech empower benchmarks ones. The none fixed size of the DB pool for DotNet impacts the latency, opening a connection to the DB (especially if TLS was used) is expensive.
thanks! minio can also be used as a replacement for hadoop for some data lake stuff, etc. It uses significantly less disk space while supporting the same replication, but it's harder to scale. If you are on-prem, it's a viable alternative. I agree with the pool size, just the defaults, no more, no less.
Genuinely shocked by these results.
Great video, thank you!
Could you do a performance comparison between Rust Actix and Golang GoMicro?
Thank you.
Well, I just finished testing Rust Actix with Fiber in EKS and will upload in a couple of days. Why GoMicro? Anything special about that framework?
Nice video
You might enjoy using Nim as well. Could be an interesting comparison
thanks, i'll take a look
So c# easily defeats node nestjs ?
i'll make some more tests in the near future
HI CAN YOU DO one for golang and swift - hummingbird?
What specs do each of the nodes have and the controllers ?
4 CPUs and 8 GB of memory. From now on, I'm only testing apps in production-ready EKS clusters to make it similar to what we actually run in production environments. I use m6a.2xlarge EC2 instances for the nodes...
can you do go (fiber) vs rust (actix)
coming next..
🍿 Benchmarks: th-cam.com/play/PLiMWaCMwGJXmcDLvMQeORJ-j_jayKaLVn.html&si=p-UOaVM_6_SFx52H
Can you compare nodejs to bunjs? Thanks
ok, noted!
great video
thank you!
Can you also compare with rust-lang please ?
well, i can say that Golang is a beast
Can you do rust vs. go?
yes, coming next
Really cool benchmark. Can you do Dart Serverpod vs Go Gin too?
Gin? It's kind of slow, and I've already tested it. Maybe Fiber or standard library?
@@AntonPutra because Dart serverpod is new framework, and we wanna see how it compares against popular alternative like Gin. Comparing it with Fiber or std I think is useless, because we already know that they are too fast :).
@@billn98 noted
Sounds reasonable as a whole. .Net isn't particularly efficient unless you do a LOT of massaging. Then again, like ThePrimagen usually puts it, do you have more microservices than users? If I'm not doing high performance, I'd probably sacrifice efficiency for "ease of use and support", especially in a MS centric environment. If I am, then "who cares about your experience, fps for life" (old competitive Quake joke, where you'd sacrifice ANYTHING for more fps because max_speed was tied to fps).
Why C# fanboys always crying when they dont get the result that they want. If you want more optimization the same thing can happen in go. Do your own test without biases.
Hi there! Nice video, Thanks!!! Would be possible to do the same test, but using Java instead of .NET ? Maybe also Java x .NET as well 🙂🙂
thank you! noted!
Is it fair to compare a web framework specifically designed for maximum performance with many limitations and unimplemented web features to a standard enterprise application framework that has everything you need?
At least this doesn't exactly match the title of the video.
Hi, which libraries are missing at go side for enterprise application needs, can you give some samples if possible please.
Well, internally most of the microservices we have are written in Go, some in Python, and data pipelines in Scala, Java, etc.
can you do Dart vs Js vs python ?
ok noted!
The comparison seem kind of fair, but they're room for improvement, but i think that a good general "default" setup representation.
But small precision (with a little history):
It's not dotnet api or dotnet core, it's the aspnetcore framework just like gin/fiber are web framework that you're testing against, they may exist other web framework for .NET, but i agree that is most popular and used one.
Nowadays it's just .NET (or dotnet) no "Framework", when we say .NET framework, it mean the old .NET Stack that was only working/designed to work on windows (yeah.. yeah...I know about MONO, but that out of the scope for this simple history)
dotnet core was/is the rewrite of the .NET stack from scratch. Microsoft choose to call it dotnet core, because at first it was only a subset of library (the core part), and apsnet core was also the rewrite from scratch of the web framework stack that only supported a subset of the aspnet MVC api, why they didn't change aspnetcore to aspnet, because it's already exist (they were (as far as i know) 3 Microsoft's web framework: ASPNET, ASPNET WebForm, ASPNET MVC (with 3 version of that framework is recall correctly)). So why did they change .NET core to .NET ? i don't know the real reason, but i have some idea like stopping people to not use the new improve version of .net because in their head .NET framework is the supported version of Microsoft versus .NET Core was not as feature complete and not maintainted as good by microsoft because it's now not part of the automatic windows update.
Fun fact that most people don't know, but a certain point in time, you were able to start a apsnetcore project using the .NET framework and this was not a hack, it was a valid use case, but today the required version of .NET for aspnercore is too high and .NET framework support was dropped, but switching from aspnetcore using .NET framework to aspnetcore using .NET Core (now know as .NET) was really easy.
Thanks for the clarification. I didn't have any experience with .NET, but someone suggested trying it out with Minimal API, so I did. I'm learning a lot from the feedback I get, lol.
Шарп в AOT скомпилить и потом размеры контейнеров сравнить, не?
Hi Anton - waiting for monitoring setup in compose config from you :)
ok :)
I would love to see an experiment like this with Zig someday.
noted!
C# .NET 9 vs Java 🔥 Would be awesome!
Thank you!
Is C# code built with configuration of Release or Debug mode. I can not see that.
release, i also got AOT verion will be testing soon
@@AntonPutra Did you really use Release mode? I checked Dockerfiles in your repo and I see that you use dotnet publish command without -c Release parameter. dotnet publish command builds app in Debug mode by default.
Edit: I rechecked, since .NET 8, publish uses Release by default, so it seems everything is ok.
Best if you can draw conclusions.
thanks, i'll try it
C# view engine 10 years ahead of Golang. And for enterprise, consistent changing of go framework sure is a headache
Yeah. I think it's no surprise that Golang is faster - but .NET is well established and really powerful. And in real use case, it's still REALLY fast.
Performance should only be the deciding factor when you REALLY need it. Majority of projects don't. In most cases, there's more to it - from ecosystem to team/personal experience and preference.
@@TehGM Not really! PRoblem is that developers are mad, that simpler idea like Go lang flies better that begemut C# with .Net
@@Sam-gd4xp You're either trolling or you simply never comprehended the idea of "different tools for different jobs".
I've never seen anyone mad that Golang is faster. Everyone knows it, everyone accepts it, and people happily pick it when they need it. Both languages are tools, and only immatures do these kind of "performance wars" and call each other mad over it.
@@TehGM if you're the head of small startup and small firm, and you learn basic accounting and project management properly, you will see.
That's why Google push so hard for kotlin multiplatform bro. But will it come? After so much Java VS kotlin legal battle
Go is known for it's backwards compatibility, i'm not sure what do you mean here
Go is solid!
it is, with very little optimization, and it's very fast
Basically, if you want something that works and easy, go with Java, C#, Node. If you want improved performance and efficiency without too much work, go with Go. If you want the most performance, speed and efficiency you can get, with the downside of complexity, go with C, C++, Rust
Rust is not ready for the cloud, with poorly maintained SDKs, and many of them are just slower than Go implementations. But in theory, Rust should be second after C.
@@AntonPutrareally? What did you use? That has not been my experience with Rust at all!
@@Comeyd actix with aws sdk and postgres driver + prom client, i'll publish in a day or so including source code
Is there a Node vs C# comparison ?
Not yet, but I'll make one soon. :) I do have other Node.js benchmarks, though in that playlist.
@@AntonPutra thanks, I've checked your playlist. Keep it up
@@salman2070 🫡