Writing C# without allocating ANY memory
ฝัง
- เผยแพร่เมื่อ 30 มี.ค. 2022
- Check out my courses: dometrain.com
Become a Patreon and get source code access: / nickchapsas
Hello everybody I'm Nick and in this video I will show you how you can use features such Span, ReadOnlySpan and stackalloc to write allocation free C# that is very fast. This video will just focus on a single example but I will explain every decision as I go and hopefully you will find areas where you can use the technique you see here to optimize your own code.
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: bit.ly/ChapsasGitHub
Follow me on Twitter: bit.ly/ChapsasTwitter
Connect on LinkedIn: bit.ly/ChapsasLinkedIn
Keep coding merch: keepcoding.shop
#csharp #dotnet #performance
![เกิดใหม่ทั้งทีก็เป็นสไลม์ไปซะแล้ว ซีซั่น 3 - ตอนที่ 62 [ซับไทย]](http://i.ytimg.com/vi/fsIY2fKn38Y/mqdefault.jpg)
Definitely want to see more content like this, thanks Nick for constantly expanding my mind. I just want to say I'm probably about 10 years older than you but I have no shame in saying that you are a great teacher and I learn a lot from you, I always promote your videos at work.
I also promote his video's at work. 😁
Yep, I third this, lol
I equally do share his videos with my colleagues and other programmer friends.
Me Too, also promoting his courses
Great example! Well done Nick!
BTW: when your process runs in 32 Bit mode, the default thread stack size is 1MB. In 64 Bit mode it's 4MB.
Source?
I would be interested in seeing json serialization/deserialization techniques that are more memory efficient. Especially when receiving back large json payloads that need to be deserialized to an object.
exactly what I was thinking about!
I reckon you’d use System.Text.Json as it is built with similar optimisations as this video.
No matter how much I think I know, I always learn something from your videos! Thanks for continuing to put out great content!!
Prior to 2019, allocation-free code was *essential* in Unity (the game engine), which ran on a 12-year-old version of Mono with single-threaded garbage collection. Having too much garbage would result in stuttering.
To work around this issue, in 2015 Unity introduced "IL2CPP", a compiler which takes the compiled C# IL code and compiled it into C++, and then compiled *that* into a native binary. No, this is not a joke; it's a real thing that still exists.
However in 2019 they finally upgraded to a version of Mono with multi-threaded GC, so IL2CPP is less useful. It's still a good idea to minimize allocations, though.
I don't think that was or is the primary reason for IL2CPP....
Yeah definitely not the only reason for IL2CPP. iOS sticks out, as you can currently ONLY build Unity games for iOS using IL2CPP. Android supports the Mono runtime or IL2CPP.
still an older version of mono, but at least not ancient
Now you have burst wich converts IL/.NET bytecode to highly optimized native code using LLVM, but to get the most of it, you have to write in a subset of c# called high performance c#. It is c# but with no reference types.
Now with .NET 7 we have Native AOT!
Even without it, RyuJit can output some very optimized assembly if you know what you're doing.
C# is starting to feel more and more like Rust. Thanks for taking the time to walk us through this.
Add first-class sum types and late interface implementation/orphan rules to the lang, and ideally offer niceties like the One True Constructor and a true no-null mode, and C# becomes very attractive indeed
@@berylliosis5250 i give another name to this, feature hell. The language is getting more and more complex, if i want rust, i use rust instead, is far more performant from scratch.
@@rafaelrosa3841 That's a fair point, but the whole "small and simple language" ship has _very_ much already sailed for C#.
As for using Rust instead - sure, but C# has different design constraints that are genuinely better at some things, so providing the "common" things that are really nice makes it nicer to use when necessary. (E.g. for use with Godot, which has a threading model that broadly disagrees with safe Rust)
@@rafaelrosa3841 Rust has the borrow checker and with that another memory management model. C# is and will always be a Garbage Collected Language. This makes for very different ergonomics, ever if they share features.
One's hell is the other persons heaven. The recent additions to the language have all been aimed at reducing boilerplate and reducing error potential. And for all of them, I think one thing applies: If you read the code using these features you'd be hard pressed to assume it does something else than what it does.. And that's imho the most important thing.
@@RedHotBagel I don't agree with this, more features to do the same things differently only will kill the homogeneous code base to a more extensive and hard to read language, for me give a different name to a Apple killing one p (aple) Wich mean now i writing less to describe the same thing will not help, I already know what apple means it's just now have two ways of describe it, and more ways to describe the same thing will increase the learning curve of the language.
the new span and memory types are making the old C programmer in me happy
Some tips on this video:
1) there is a Guid.TryWriteBytes() method that acts without allocation
2) you can use string.Create() to fast allocate the strings based on Span, instead of calling string constructor
One thing the Primeagen always mentions about garbage collected languages is that the performance gains of having fewer allocations won't show up in spot tests like this, because the garbage-collector pass didn't get timed in the stopwatch.
So the optimized methods clocked in way faster but also shorten the timing on the garbage collector. Not sure if that also applies to C# like it does to JS
First of all, I just discovered your channel and I think it is really good and informative. Kudos.
Now, as an old C developer (now a manager :P ) the "allocate memory" concept rang so many bells in my mind. I have a general comment on this. If you want speed, you need to go low. If you do all this you may as well go writting unsafe in my opinion. Still it is great for people to see these kind of tutorials in order to actually see how things work!
Nick, I really value your content. Every time I listen to you talk about some topic, I realize I don't know enough, but I finish the video knowing more than I did before. Cannot thank you enough
I'd definitely like to see more videos like this. Great example in this video. Keep up the great content.
This is amazing! Learnt a lot of things that I’ve heard about but didn’t really know, keep up the good work!
Another thing worth mentioning is if you want your URL-safe base64 to be RFC 4648 compliant, you need to swap the replacements. Pluses should be replaced with hyphens, not underscores. Slashes should be replaced with underscores.
Brilliant as usual. Thank you. Your videos push people to make their minds more efficient
I never use structs but thanks to this video I will use them a lot more. I am starting to see the real power of structs when you just use them to compute stuff and don't store them in memory. thanks Nick great video!
About the constant characters; I'd even consider it worse to create character constants mapping them to their names. You already know that '=' is the equals sign, and it's the glyph itself you care about when replacing them in base 64. It would be good practice if that '=' character represented some special sentinel that acts like a magic value, a special value encoded in the same data type.
I was just going to say the same. It would make sense to define PaddingReplacement, PlusReplacement and SlashReplacement because that's where you're deviating from / overriding the standard and you may want to make it configurable. But this way you just make the reader doubt if the constants actually contain what their names claim to contain.
I assume we all know and agree that const’s provide some benefits including being included in the assembly at build time and also being able to view occurrences referencing the const, allowing easy renaming/ better refactoring as opposed to hard coded literals.
@@NedMorse I agree on the occurrence and refactoring argument, but it doesn't matter if you have a const field, const variable, or simply use the literal. The resulting assembly code is the same. IMO as these constants are private and they are only used once, it is a choice of style, and I tend prefer AlFas's and Benjamin's suggestion.
I'm going to concur with AlFas and Benjamin Schug as well. Characters are self-identifying constants already. If you're going to give them another name, give them the name of what they're representing. Replacing `'=`` with `Equals` is just obfuscation. Replacing it with `Base64PaddingCharacter` tells the reader why you're appending an equals without actually needing to tell them what the character even is.
Great video. I learned something very valuable today! Now I've got to look up stack vs heap. Hopefully Nick Chapsas is in the results. :)
Wow, I can already think of where to use it at work. Great stuff!
This is great video! please do more of those! I never thought about the size of the guid.
I am willing to pay for a course with this level of details!
Yes. More like this. Very good optimisation walkthrough.
The most insightful TH-cam channel!!
Please more content like this, also Unsafe code :)
Thanks very very much Nick!!!
Thanks for great video! I love Span and Memory.
Memory, array, and string builder pools
I wanna see more of those kind of videos for sure, thanks for sharing.
Thank you! For both this video and the one on the Span.
I really like using Span and Memory when appropriate. I remember using Memory for creating a zip archive from file streams. With a maximum compression I was able to make my application 2.5 times faster because of using Memory (2.5 seconds instead of 6-7)
Is extracting the strings to constants necessary? The compiler performs automatic string interning on literals and should only allocate once for the life of the program.
It's a style thing. An ordinary idea in programming is to avoid "magic strings" in your methods, and have all constant values at the top. As you said it doesn't do anything at all to impact speed or memory consumption.
@@AndersJohansen92 As a style thing, this video shows exactly how NOT to do it correctly. A couple of points:
- Replacing '/' with `Slash` is akin to replacing `42` with `FourthyTwo`. It doesn't provide any value and definitely doesn't respect the idea behind the no-magic-literals rule which is to make sure that all of your literals are *described* in your code. A better name for `Slash` would be `Base64_Value63` which describes what the code expects the variable to contain.
- The cast to byte should also reuse this variable and not redeclare it from '/' again to highlight the relationship between the char and byte version. In this example, one could decide to replace the '/' with another char but forget to do it on the other way around. This would prevent decoding from working correctly.
- The fact that the compiler already handles this for you is not mentioned in the video. This means that most people will watch this video and think that those const extractions actually helps with the performance of this code, which is a very common myth.
- '+' comes before '/' in base 64, so they should be declared in that order as well. In fact, a proper variable name would also highlight this.
@@KawazoeMasahiro Awesome points man. This youtuber is a total noob, he should not have a youtube channel. You would do a much better job.
@@ZelenoJabko 🤡
This is because of rule #1: You write code for developers, not for computers. While defining Equals = '=' is kinda pointless, it comes from a good habit where you leave not constant unchecked, so you fan find wherever you use them. Try finding where use used = as a constant with "Find in Files"
extremely high level content, thanks!
I reallly like this kind of content, pitty i didnt run into this channel earlier
Props to you, you really make it look easy!
I would be interested in seeing if what difference replacing new string() with string.Create() would make in both memory and performance.
Of course it depends on hardware, OS and .NET version, but my tests (i7-9850H, win 10, .net 6) shown that string.Create is slower. No difference in memory allocation.
Thank you for sharing your knowledge!
I don't use C# anymore but I'm impressed how far it's gotten since I used it many years ago.
Wonderful .. truly high value stuff ... keep it up. 👍
Great video!
But what if I want to remove some characters from a Span?
And what if I want to replace a sequence of chars with one specific char?
This is absolutely great! This'd be an amazing way to optimize azure function apps
Thank you for teaching us this
Cool content,
Thanks for your great videos
Great video nick !!!!
Well done! Thank you!
You’re literally god of c#, thanks for existing
This is very useful! Thank you for sharing..
Hi Nick. Amazing video as always. Do you have any safe vs unsafe code video for C#? That would really help us folks out. Thanks.
Love how you emphasized "when you need to"...
Amazing content. More like this please
Really videos like this opens yours eyes on you do wrong 😅. Thank you
seeing chained replaces... i think i know where this is going lol. great content.
Thank for the video, Nick.
It’s a pity that Span doesn’t work with async methods. (Can’t be members of heap objects which async state machine is)
The state machine is a struct in release mode, it’s just not a ref struct
This is great! Thanks a lot.
Tiktokers: Say something without saying it.
Nick: Access something without allocating it.
Hey Nick, How do we preserve benchmarked code and keep code duplication low? Do we create these methods in test project?
Awesome content!!
I write different type of C# apps, worked since VS 2001 with C#, I had to use strange memory allocations twice and that was for a multichannel (80+) audio streaming application.
Hey Nick,
I feel like I've become pretty proficient in safe memory optimizations (which usually is more than enough).
Especially since span got added.
However, I never really trained the unsafe muscle and I was wondering if you can point me towards some good videos / blogs to start looking into that?
Same for me :)
Le dot
wow! thats cool! thank you!
Excelente content!!
very informative!
Do we can update base64span array without creating finalChars? Iterate and update special chars.
Great video. I was actually discussing url safe Id's with my professor today say..do you have tutorials for ASP.NET and Memory management in C# I really want to dive into this
Up until a few weeks ago, it'd been a long time since I'd touched C#, and I've only recently been doing Windows Forms in Mono on Linux. I'd spent a long time working on C/C++ and microcontroller assembly, so seeing these highly performant improvements to C# is very exciting! I'm loving the safe stackallocs to Span! If only it were implemented in Mono. 😢
You don't need Mono. .NET is fully cross platform.
@@nickchapsas Yeah but Windows Forms isn't. And yeah I know Avalonia is a thing, but I just wanted to quickly whip up this personal project in something I was familiar with. (In retrospect, maybe I should've spent the time to learn Avalonia and the newer .NET and C# features, but alas.)
@@c4ashley You can use MAUI instead of WinForms.
@@nickchapsas I wish. It's still unsupported on Linux, and the linux-maui community fork is a total bust in my case. Maybe one day soon. Or maybe if I spend long enough trying to solve installation problems. For now, Mono is fine. A bit buggy, a bit dated, but at least it actually works.
Anyway, this is all tangental; I was just saying I like the features you're pointing out in this video, and I'm excited at the prospect of using them for future (non-Mono) projects! 😀
Will introducing a variable for indexers' calls in switch statements improve the performance?
That youtube link example 🤦♂, it got me
Wow, you took the time to type that out and got punished 🤣
Love it.
Is such conversion of guid to bytes correct? What about BE/LE systems? Won't they mess bytes?
well done!
Would casting char to byte within the function allocate memory as well? Or you could have worked directly with const char?
Also when was that switch introduced? First time seeing it
Char and byte are value types so there would be no allocation. Switch expressions have been out for a few years now
is it possible to use Benchmark in existing application to measure parts of code that are not so easy to extract into separate console test app? use case would be to collect measurements of code that you suspect is the root of all evil before refactoring that would allow to benchmark those parts in more isolated way. of course you can use dotTrace or similar profiling tools but maybe Benchmark or other package could be of use as well here?
and as always, this video was great, keep them coming!
That's next level stuff
I was waiting for a pointer I'm any time, but we still safe.
Hey Nick, for some of your courses, if you have questions, what is the best way to reach out on those?
Linked in, Twitter DMs or email works
Great video, but I got a bit lost when you said Span was allocated on the stack (so I thought it generally was) and still used stackalloc (in order to achieve something I thought was already the case?). What am I misunderstanding?
Broadly speaking: Span only contains a 'pointer' to memory and bounds. That memory can still be on the stack or on the heap. Using a stackalloc you allocate memory on the stack, and then create new Span object (also on the stack) that points to that memory
Also it would be even faster if you convert to/from base64. It's just a lookup.. You were walking through the bytes/chars for replacement. You may use that time you to convert with a lookup table..
When using the default case in a switch you do a second array lookup. Would it be faster to use a binding match instead of an underscore? Like char i => i instead of _ => array[pos]
There is no array lookup at all with the switch. It's being lowered to if checks
@@nickchapsas I had the same reaction as @Michael Chen and I must admit I don't understand your answer...
@@Krimog Sorry I misunderstood the question. No that can't happen because list patterns don't work like that. You can't do i => i.
I like your low_level optimization videos in particular, but why are there no built-in URI-safe GUID methods which use safe characters as their native lexicon?
Could you make a video about how dispose works (IDisposable) and how to implement it correctly?
Keep in mind there's also IDisposableAsync. Dispose is used to release resources. Disposable objects should be put in a using statement when possible.
The whole point of IDisposable is to use it in a using statement so it will get called when the using statement ends, even if an exception gets thrown.
Why did you create a const variable for backslash and plus and such? is it for maintainability?
MORE PLEASE
very useful, but this way for reference type, have any way for value type ?
Tip for writing allocation free code: At least in Rider you can set a different highlight color for struct types. I have structs green and class orange.
Additionally the `Heap Allocations Viewer` plugin is really great.
Just because something is a value type it doesn't mean it will be a stack allocation so the colour highlighting advice is misleading. HAV is good though
@@nickchapsas no, but knowing if something is a value or a ref type is essential to be able to reason about allocation and performance characteristics and having different highlights makes this a lot easier.
I do this too. Most types are the standard teal color, but structs are a neon greenish color. (dark theme).
Nice!
It is any chance to get stackoverfow exception in methods with stackalloc, if they will be frequently called at the same time?
No because this memory is only allocated in the scope of the method so after the method of called it’s gone.
Having a full grown api already i am curious if this would improve the performance so drastically.
I would think you have the string converted after each call of an endpoint, so basically every Guid you use now would be string something like B2yOjLyEZk0.
Then you'd convert it back to guid and use the guid for querying database. Would be interesting to see how you would implement that into an existing api.
I am also curious if the DI would slow things down once you implemented the function to convert guids.
edit: i just saw you're using static.. so forget the DI.. :D
You will really see a benefit if you are using the conversion in many areas. For example if you were allocating 200 bytes (like in the video) per conversion, then if you’re dealing with 1000 requests per second (which I do more of that) then you’re instantly on 20kb or wasted memory per second which adds up quickly and triggers collection by memory pressure. I think you should profile your apps memory and if that part is hot then optimise it otherwise, ignore it and go for the low hanging fruit. You wanna be pragmatic here
I very much like optimized code. Just wondering what is the business role of the server that needs such id-optimization? You mentioned the id being used in URLs -- I would assume that even parsing the http request and headers is orders of magnitude less efficient than the conversion, not even including any business logic. Or if there is a need for such optimization, is the C# the best language for the purpose? Just to be clear, I am not criticizing, just wondering about the use case.
It is very common for client facing ids to be represented as base64 strings but have a different internal representation. Some even have prefixes and suffixes. This is very common. Now if you’re APIs are dealing with thousands or requests per second then this is saving thousands of GC triggers
Where can I learn stuff like this? I missed a lot of this kind of stuff in my degree.
1:24 Wasn't the URL I was expecting but still got the expected result.
Out of curiosity, would it be possible to avoid using "string"?
Is there a "struct" that could be used in it's place? Or an array of "char" would be the only/best option?
Arrays are reference types so they would still allocate the same thing. You can return a ReadOnlySpan instead
It's a great idea to use stackalloc and Span. But my question is why not use StringBuilder which is nearly as fast and is using Span and refs underneath? I am also interested to see practical performance benefit of unsafe to avoid GC.
I'm curious how you can tell if some code like this should be optimized in a real application. Because it's pretty hard to tell such a thing, it's hard to have such metrics from production env for methods with average response time like nanoseconds.
putting aside the exact math, 'statistically speaking' if your method takes only 100 ns to run, but largely contributes to the runtime of the app because it's called frequently (let's say 10% of the hot path you're interested in) then profiling with cpu sampling should be lucky enough to 'hit' it often enough to show it in the report.
The more frequent sampling rate you use, the more accurate the results would be (but more overhead).
You would typically run such benchmarks in controlled environment, potentially with data copied (and anonymised) from production, but running such profiling on a selection of your production servers might not be too bad, assuming you're fine with temporarily slowing down the app a bit for some of your users.
And obviously you wouldn't spend time trying to find such optimization opportunities if you didn't have performance problems in the first place and after ruling out more obvious performance bottlenecks (wrong algo, bad multithreading, frequent/long gc etc etc)
Amazing
Sorry for the silly question what is IDE u are using that shows allocated memory..? Please let me know
It's JetBrains Rider
Your intro sounded like a aprils fool - glad I knew about Spans and stuck around ;)
What's the purpose of putting =, /, - in fields instead of just in the switch?
Looking forward to seeing unsafe version of this optimization
Does it help performance wise to replace the 22 in the for loop by a .Length property to prevent boundary checking (under the hood)?
No because they are both lowered to a while look. In fact, having 22 there is a tiny bit faster
@@nickchapsas I was referring to something that happens under the hood by the = id[i] statement. Are you also referring to that? From what I understood, the compiler will not insert IndexOutOfRangeException checks, when it can detect that the index can not go over the boundaries of the array. When it cannot, it inserts a min and a max check. I would say that applies here. for the same reason foreach is preferred. But perhaps my knowledge is outdated :).
@@donnerkebab I understood what you meant but no that's not what the compiler will do. Adding the length will make it 1.5-4ns slower compared to the inlined int for accessing the value of length
Can you do a video on improving Entity Framework performance?
It's interesting idea, how we can optimize for example update entity which has child and they have own child. What about batch operations and so on
Dapper :)
@@briankarcher8338 🤣🤣
Instead of copying the bytes in the Guid into a Span, can't you just cast it into a byte* in unsafe mode?
We don't wanna resort to unsafe code
Hi, i wont to send GzipCompressed data Stream to RabbitMQ body message, which accept ReadOnlyMemory. Is any way to convert Stream to ReadOnlyMemory with minimum memory allocation?
If i want to optimize my (real time)C# code i'd rather port it to C++. My boss doesnt think that way tho. Thanks for the knowledge
Awesome
Cool!