A long time ago I made something similar to that with Dictionary and it did the job for me. I haven't used the ConditionalWeakTable type yet, but it does look handy in rare situations. Thanks for showing it.
I have not, but I have dabbled with WeakReference to prevent event subscriptions from keeping objects from being GCd, but I eventually abandoned that for a better structure/model. Things that intend to be 'helpful' by being 'magical' and often just end up being more confusing for junior devs and just create different kinds of bugs.
I used to use ConditionalWeakTable in a project and it was a big failure. Once you have milions of objects there which are being created and removed over time, the "Dictionary" behind the scenes grows and gets into Large Object Heap. It is the part of memory which is never garbage-collected. The dictionary grows, shrinks over time, and it always allocate new space in LOH. This is the best way how to fragmet your memory. After some time we started having OOM Exceptions even though we had plenty of free memory. Plenty of free and unusable fragments. It also caused not garbage-collecting of gen2 memory, because it didn't hit 80% memory usage due to this fragmentation. So use ConditionalWeakTable with caution.
I'm not quite buying this. I could not reproduce any OOM issues even after extensive performance testing. It was so consistent that I didn't include this section in the video. I am suspecting that something was misconfigured or missused in your case or the latest .NET releases have fixed the problem. Do you have a code sample in latest .NET that I can use to reproduce the issue?
@@nickchapsas I am a colleague of Jan, still working on the project. Sadly, this is an enterprise application and we do not have permission to share the code. The goal was to implement deep equality on many objects, some of them quite large, all designed to be immutable. We used to override Equals and GetHashCode in them to call our reflection-based helpers which made the implementation universal. Any collections were wrapped into our own types with overridden equality to keep deep equality working even for collection-typed properties. The helpers used a ConditionalWeakTable for memoizing the hash codes. After discovering what Jan described, we switched to our modified version of Amadevus.RecordGenerator, which implemented deep equality and added an extra field memoizing the hash code to each record -- this solved our memory fragmentation issues, we got rid of the hash code wrappers (ConditionalWeakTable supports only classes as values) and I believe the improved locality of reference even further improved the performance. This was back in mid-2020, the project was targeting .NET Core 3.1 at that time, with C# 8. C# records were not yet a thing back then, but we are still reluctant to use them now on .NET 6 with C# 10, because we need the GetHashCode memoized and we see no way to add that feature to them. We believe Amadevus.RecordGenerator is no longer under development and we would like to migrate our fork to .NET 6 and source generators so that we can maintain it long-term.
@@nickchapsas By the way, the OOM issues were most definitely NOT just the fault of ConditionalWeakTable. The app was developed on a very tight deadline and it was quite complex. But ConditionalWeakTable was one of the factors that seemed to contribute. Several minutes ago, I looked into the old helpers and found this one weird part that might be related: ``` private static CachedValue GetOrCreateValue(object obj) { try { return WeakTable.GetOrCreateValue(obj); } catch (InvalidOperationException ex) { // WeakTable was corrupted due to underlying issue // typically, OutOfMemory exception corrupts the state of WeakTable and make all future calls unusable // to fix it we have to recreate WeakTable instance. // stackoverflow.com/questions/7150140/what-caused-this-invalidoperationexception-using-linq-to-sql // gist.github.com/pmunin/f5381a0226c4c00c8c972e051f2fdb39 #pragma warning disable CA1303 // Do not pass literals as localized parameters Logger?.LogError(ex, "WeakTable instance is corrupted due to an underlying issue, recreating."); #pragma warning restore CA1303 // Do not pass literals as localized parameters WeakTable = new ConditionalWeakTable(); return WeakTable.GetOrCreateValue(obj); } } ``` Still, the LOH fragmentation issue was real and getting rid of this enormous object greatly improved the memory management of the app.
@@nickchapsas I wanted to isolate the problem this weekend but with no success. Our solution was based on .NET Core 3.1. It was deployed in Kubernetes cluster on a machine with 32 cores with lot of memory available but the POD had memory limited. Also, the application used server GC. It might have been a combination of those curcumstances. What we knew, the problem was mitigated by replacing ConditionalWeakTable with different solution.
I use the ConditionalWeakTable in a ReferenceManager class that I use to automatically dispose shared large IDisposable objects (image data). When an IDisposable is shared (onto a different thread), I increase the refcounter in the CWT. When a shared item is no longer needed, the refcounter is decreased. When refcounter reaches zero the item is disposed.
This is cool when you are dealing with sealed classes I guess. But, what about the just old and simple inheritance approach? Is there something that I'm missing? This also adheres to the Open-Closed principle.
I was thinking the same thing. The only place this would be better than inheritance is if you don't control where the objects get constructed. If they're constructed by the 3rd party DLL, you can't have it construct your inherited type instead.
I have been aware of this thing for a few days, coincidentally I stumbled across it in a hobby project. I wanted to continue on it next weekend, you saved me a little research time with your video, so thanks 😄
I actually used this. I had a library to enable automatic chaining of NotifyPropertyChanged events based on attributes (for UI view models, which could have nested types). Had to weakly subscribe to events, and also keep track of which objects were being managed by my library. I think I used this for the latter. Memory management was definitely the hardest part. If I did that over again, I'd definitely use source generators.
Think I prefer to just wrap in a class/struct with the extra properties and access the base with a Value property. They you do not need the lookup and there is also no extra keeping track of the objects in the dictionary. Something like this, also more explicit what you are passing around. public class ExtraPropsFor { public int MyExtraNumber { get; set } public T Value { get; init; } }
I wish I knew this before. This is super useful if you want to track changes in a table (say you updated, deleted, created a new object) which is owned by another code base (which you might not want to change for only this use case). Thanks for the video!
seriously I'm new to programming but I really don't know that why I always watch your videos. Just because I love your videos and wish to be a PRO like you. Love from Afghanistan.
I "know" that I looked into those Weakly referenced collection types before. However, it's one of those things which isn't in my memory muscle and I kind of forgot about them. Future me, will most likely forget this approach again. It's a very usefull approach iso wrapping or subclassing. Will have to look into more real world scenarios to be able to start using it, and to make it part of my memory muscle.
Last i used it was to keep reference equals working on objects encapsulating an implementation through an older interface. So if a class implemented V1, it was encapsulated in a class implementing V2. When needing a V2, just check the ConditionalWeakTable for an existing instance before creating a new. Does require thread synchronization of course. So in my toolbox, but used so infrequent I always have to Google for the name when I do need it.
Used it in the past to reduce the memory footprint of a class by externalizing rarely used properties, resulted in too much pressure on the garbage collector (long finalizer queue). Perhaps you could make a follow-up video with benchmarks.
In the example I don't understand why removing an element in the list should affect the dictionary elements though. The elements pointers are the same but if I remove one element from the list I would still expect to have it in the dictionary. I think I am missing the point. WeakReference looks interesting! I didn't know it. Thanks!!!
This is cool, but as others have mentioned the dispose "issue". It would be nice if there was some place to have a "CacheItemRemovedCallback" concept, so that you could provide a way to handle the remove event. But since this weak reference concept is something that the GC understands and not the "user level" code, I guess this is not doable.
I tend to prefer using the "is false" syntax over using the boolean inversion operator nowadays, it always sounds a lot more readable to me, what you think personally about this approach?
I used to think "== false" was ugly. But I've witnessed so many people misunderstand code because their eyes glossed over the "!". I'm starting to become a fan of "== false" and now "is false". C++ has an optional set of keywords if your keyboard doesn't support symbols. In that case, you can write "if( not variable ) {}", which is what I wish C# would adopt.
@@WilliamLeeSims I was never a big fan of "== false", but I really like the "is false" syntax. In C#, "== false" can produce false positives, as our equality operators are overloadable, but "is false" compares the value directly, it always looks cleaner too due to being more fluid. About your question, I believe you can do something similar currently, in C# you can use a syntax like: "if(variable is not const_value)", it is far from perfect but you got it
@@BenjaminBrienen Yeah, that is true, btw, you can do something like "!(nullable_bool ?? false)" when handling bool?, but the is false syntax appears to be much more sweet
I prefer the full legibility of not using !. I saw a post years ago that also proved that it was slightly more efficient too but I have been trying to find it without success since then. The compiler might have evolved to make it not be any different since then.
@@nickchapsas Just clarifying that, for folks like myself who are coming to .NET from other languages like C++ & PHP, I was unaware of this obvious information. Still, it makes sense since it isn't technically the class's destructor.
In the use case given this solution seems to introduce complexity that is simply not needed unless the ExternalObject is sealed. Favour composition over inheritance does not mean inheritance loses its value completely. ExternalObjectEx externalObject = new ExternalObjectEx(); Console.WriteLine($"{externalObject.Id} {externalObject.text}"); public class ExternalObject { public Guid Id => Guid.NewGuid(); } public class ExternalObjectEx : ExternalObject { public string text => "Hello World"; }
I’m developing my own game (and a game engine) and I use “WeakLists” and Weak reference event handlers. On one hand I have full control over game objects from the engine, and on the other hand storing data about game objects doesn’t prevent GC from collecting them if the game has no need in them
I think this is the first time I've ever seen a WeakReference ever having a usecase. I've always known of them, but as I've never really used lower level languages I've never seen a usecase for them.
Aren't you assuming that the type you don't control correctly implements GetHashCode for this behaviour to work? Otherwise there's no guarantee that the added properties are actually correctly associated with the instances you want them to be associated with (Aside from all the other issues with GC). I'd say ideally this is a case for inheritance (If the class isn't sealed). Next best is a partial (Requires that the class you don't control was declared as partial). Finally I'd consider a wrapper with pass-through methods / properties if I had to, hopefully can consolidate that with the original type at an interface level, or just work with the wrapper within your original code.
Probably obvious, but if an object implements IEquatable and overrides GetHashCode(), this may fail if the only difference between two objects are supposed to be the extending properties.
I think we have been using static ConcurrentDictionary, but I think based on the conditions, replacing with ConditionalWeakTable in some cases might be a good step? Time for investigation...
It just occurred to me the IEnumerable Count() method name is a verb but the IList .Count property name is a noun. I need to remember weak references exist and to use them more for things like caches. I think I've maybe used them in my code once maybe.
I have a custom cancelation token, and I use this as a cache to convert to/from the TPL cancellation token without creating a new object every time it's converted.
Hi I would love to attend one of the conferences. Unfortunately I have not been to a conference yet and would like to experience it. Greetings from Germany
If the class is not sealed, wouldn't it be better to use a subclass to contain the extended properties? By using a subclass, we can share the reference of the instance by using the Unsafe.As(ref TFrom source) method, for example: "public static Subclass GetExtensions(this Baseclass instance) => Unsafe.As(ref instance);". This provides the benefit of having an object that provides access to both the base and subclass' members.
I don't know, in my opinion just creating a wrapper for this would be cleaner. The problems I see are the following: - How do you know that you need to call GetProps? - CoditionalWeekTable creates more garbage than just creating a wrapper class - Accessing these properties has more overhead than accessing properties from the wrapper class
- You communicate/document it - No it doesn't. Except for the table itself, it's the same amount of object that the system needs to create. - Does it? I would assume you just get the props object once which is an O(1) operation and access all the properties that you need
Hey Nick, thanks for answering :D Wasn't really expecting it :D - That's the thing, you need to document it. Just making a wrapper seems (at least to me) something that is sefl-document and understandable right away. - Oh my bad, here I thought that it creates a WeekReference object for every new entry, but I have checked and it doesn't, but then the actual problem with this approach is that it doesn't work with structs. For structs you would either need to box it or again create a wrapper, which would need to be a class. With a wrapper you could create a new struct wrapper. - This is then another thing you need to know, that you need to cache the properties. Just accessing GetProps in a loop without caching it can lead to significant performance differences compared to the wrapper. And I agree that most of the time it should be fine.
Why couldn't we just use inheritance for that ? Create a child object, and create a constructor that takes the base class as an argument, and now you can add whatever you want Sure, it will not work on structs and sealed classes, but I've rarely if ever seen them on my (admittedly limited) experience in C#
Yeah I forgot to mention this but assume it as sealed. Also, this isn't just about properties but it's the easiest example. It is also about object assosiations in a weak reference way
There might be existing code that you don't know that is passing around a reference to the external object. Since you can't change the type reference, you basically need to bolt additional state to the instance which is what the weak reference table does.
humm, ok and how about composition + adapter? You compose your new class to embark the immutable one and add what behavior is needed. And if needed, add an interface to the mix. This video is quite interesting but it seems a little too overcomplicated. But definitively interesting, thnak you
Hmmm. As someone slightly unfamiliar with .NET internals, rather than a sort-of weakref table, my first instinct would've been to look and see if there's an event or similar thing in which to register a callback for when the object is freed. Does something like that exist? "When this object is freed, do X where X is removing the object from this Dictionary," i.e. allowing a user of the object to append customized cleanup operations for the instance.
Hmm I actually know a use case. If there is like a proxy which is generating references and you cant change them (even though you bloody own the proxy, dont ask) then this kind of thing could perhaps had worked. How ever, we built a new service instead because that design was an aweful mess. I guess if you have no choice it's good to know you have multiple ways to expand classes. That container was new to me and it's always nice to learn new things
I can't seem to successfully add properties that don't make infinite loops on my OWN classes,and here I am watching how to do it in classes i dont own :P
Its an interesting approach and its the first time seeing the use of ConditionalWeakTable, so thanks for sharing that. But i dont like it. If i want properties for a class that i dont own, than i create a wrapper. Thats totally fine by me, because it may be most likely be replaced by something else anyway. For me this is most likely an indicating that the external class may be not really suitable for that problem i want to solve or have simple bad API design.
How smart is it about circular references? E.g. if you insert the Key x with Value List{x} into the dictionary, will that never get collected, because there is a strong reference? Even though if you deleted the key as if there were no strong references then there would cease to be any strong references?
How does this work if the external object is factoried from a DI container with a lifetime of scoped? The external properties are static right ? Will the external property get disposed when the external object goes out of scope?
Why would you not own EF classes? They're yours, which is why you can mark them as partial. It would not work with Nick's example, as he has no control over that ExternalObject class.
@@michaldivismusic My understanding is that it would make no difference if the external class is marked as partial, because it is external, and the "partial" part does not exist on the IL level (try it on sharplab). So a partial class will only solve the issue for you if you can build it yourself at the same time as your partial extension.
I'm at 3:15, and I'm going to try and predict what method you'll use... ExpandoObjects with a dynamic casting... (If you see this message edited, I was wrong) :D --- I was wrong :(
Hello Nick! Do you have any tip about WebRTC using C# ? I've been following this topic for a long-long time and it still a nightmare doing anything that uses WebRTC. Anyway, Very nice video!
I would like to see extention props but really extention methods are same thing. In Java or C++ there are get and set methods. Just because we like props in c# doesn't mean we can't ever use methods
Doesn't this ruin sealed? Also, this may have changed but there was a time when my program was under heavy load, and the GC can be a bit lazy itself. The fix was to put in Gc.WaitForPendingFinalizer(), actually two of them, ah and at the time it was someone on the runtime who suggested the fix... While it was back in .Net Framework 3.x days i.e. Orcas or Whidbey...
Sort of. It will fullfill one use case, but weakly referenced object association is still a valid usecase. Also I wouldn't hold my breath for roles making it into C# 12
If you have the class avaiable in your code through nuget or otherwise, couldn't a good use case be to inherit from it and add the extra properties in your own super-class and use Automapper to map it and maybe fill the new extended properties too?
@@nickchapsas Then it won't work, but there are not that often you step onto sealed classes that is ment for an external API or that you want to extend. So if it is Sealed I think it can be a good idea to consider if it is meant to be extended or if you should map it to your own Class instead. If I look at i.e the CosmosDb SDK which I know you have worked a lot against too and see the sealed classes there with CosmosSerializationOptions and such things. I think I would create my own classes if I needed to extend that class, but maybe this way you show would be better? But if it is not Sealed, would you prefer to inherit and add properties to super class then or would you still prefer this way?
You might not be able to. You might work with sealed class. You might work with a collection of data, that you cannot change (readonly collections) or any other stuff. This is cleaner and simpler to create a new collection of data of new wrapper classes, of data you want to work with. Combine this with extensions and its very very neat and clean in upper level code.
Isn't this approach broken by implicilty assuming that GetHashCode of that external object must be stable througout whole app lifetime? Example: public class ExternalObject { public int Id { get; set; } public override int GetHashCode() { return Id.GetHashCode(); } } var dict = new Dictionary(); var extObj = new ExternalObject { Id = 1 }; dict.Add(extObj, "test"); extObj.Id = 2; var b = dict.ContainsKey(extObj); /*b is false*/
No. ConditionalWeakTable keys based on reference equality, not object equality. As such, conditionalWeakTable uses RuntimeHelpers.GetHashCode(key) instead of Object.GetHashCode.
So i added an extension to object so you can add a property to any object using this method.. public static class ObjectExt { public class ExtensionProperty { public T Property { get; set; } } private static readonly ConditionalWeakTable Data = new ConditionalWeakTable(); public static ExtensionProperty GetExtension(this object subject) => Data.GetOrCreateValue(subject); } Used like this: object myObject = new object(); myObject.GetExtension().Property = "myData"; var myData = myObject.GetExtension().Property; any thoughts?
Just out of curiosity, what is the cost of the ConditionalWeakTable? Until now, I've never had to use this approach for any problems, as I always find more elegant options, however, I'm curious to know how much it costs
It's a keyvaluepair extension with a special key. There is no inherit cost apart form that. The only thing I didn't mention is that it is fully thread safe via a lock, so that aspect does affer the get and set performance of the type
@@nickchapsas I mean, the cost of removal, I imagine for something like this to work it has to be deeply tied to the CLR, or make some use of finalizers and maybe some hacky things, but it's interesting to know that it's thread safe.
@@diadetediotedio6918 I didn't look at the inner implementation. Probably they just store the memory address thats points to the object and they validate it, when somebody tries to access that key.
@@diadetediotedio6918 there's no hook into an object's removal and there's no active tracking of an object's existence. The reference is validated upon access and removed if it's dead.
@@obinnaokafor6252 That's the joke. It was supposed to be in C# 10, then it got postponed to C# 11 and then to C# 12. What I'm saying is don't hold your breath for any "announced" feature to make it into the next C# version
A long time ago I made something similar to that with Dictionary and it did the job for me. I haven't used the ConditionalWeakTable type yet, but it does look handy in rare situations. Thanks for showing it.
I have not, but I have dabbled with WeakReference to prevent event subscriptions from keeping objects from being GCd, but I eventually abandoned that for a better structure/model. Things that intend to be 'helpful' by being 'magical' and often just end up being more confusing for junior devs and just create different kinds of bugs.
If it requires a big brain, make sure you picked the right approach.
I'm sure .NET already has some types to help with this. Look for “weak events” in Microsoft's documentation.
Didn't know about ConditionalWeakTable class.
Very useful technique, thanks!
I used to use ConditionalWeakTable in a project and it was a big failure. Once you have milions of objects there which are being created and removed over time, the "Dictionary" behind the scenes grows and gets into Large Object Heap. It is the part of memory which is never garbage-collected. The dictionary grows, shrinks over time, and it always allocate new space in LOH. This is the best way how to fragmet your memory. After some time we started having OOM Exceptions even though we had plenty of free memory. Plenty of free and unusable fragments. It also caused not garbage-collecting of gen2 memory, because it didn't hit 80% memory usage due to this fragmentation.
So use ConditionalWeakTable with caution.
I'm not quite buying this. I could not reproduce any OOM issues even after extensive performance testing. It was so consistent that I didn't include this section in the video. I am suspecting that something was misconfigured or missused in your case or the latest .NET releases have fixed the problem. Do you have a code sample in latest .NET that I can use to reproduce the issue?
@@nickchapsas WeakReference have one solid use case and that is to resolve circular references, everything else will lead most likely to a perf hit.
@@nickchapsas I am a colleague of Jan, still working on the project. Sadly, this is an enterprise application and we do not have permission to share the code.
The goal was to implement deep equality on many objects, some of them quite large, all designed to be immutable. We used to override Equals and GetHashCode in them to call our reflection-based helpers which made the implementation universal. Any collections were wrapped into our own types with overridden equality to keep deep equality working even for collection-typed properties. The helpers used a ConditionalWeakTable for memoizing the hash codes. After discovering what Jan described, we switched to our modified version of Amadevus.RecordGenerator, which implemented deep equality and added an extra field memoizing the hash code to each record -- this solved our memory fragmentation issues, we got rid of the hash code wrappers (ConditionalWeakTable supports only classes as values) and I believe the improved locality of reference even further improved the performance.
This was back in mid-2020, the project was targeting .NET Core 3.1 at that time, with C# 8. C# records were not yet a thing back then, but we are still reluctant to use them now on .NET 6 with C# 10, because we need the GetHashCode memoized and we see no way to add that feature to them.
We believe Amadevus.RecordGenerator is no longer under development and we would like to migrate our fork to .NET 6 and source generators so that we can maintain it long-term.
@@nickchapsas By the way, the OOM issues were most definitely NOT just the fault of ConditionalWeakTable. The app was developed on a very tight deadline and it was quite complex. But ConditionalWeakTable was one of the factors that seemed to contribute. Several minutes ago, I looked into the old helpers and found this one weird part that might be related:
```
private static CachedValue GetOrCreateValue(object obj)
{
try
{
return WeakTable.GetOrCreateValue(obj);
}
catch (InvalidOperationException ex)
{
// WeakTable was corrupted due to underlying issue
// typically, OutOfMemory exception corrupts the state of WeakTable and make all future calls unusable
// to fix it we have to recreate WeakTable instance.
// stackoverflow.com/questions/7150140/what-caused-this-invalidoperationexception-using-linq-to-sql
// gist.github.com/pmunin/f5381a0226c4c00c8c972e051f2fdb39
#pragma warning disable CA1303 // Do not pass literals as localized parameters
Logger?.LogError(ex, "WeakTable instance is corrupted due to an underlying issue, recreating.");
#pragma warning restore CA1303 // Do not pass literals as localized parameters
WeakTable = new ConditionalWeakTable();
return WeakTable.GetOrCreateValue(obj);
}
}
```
Still, the LOH fragmentation issue was real and getting rid of this enormous object greatly improved the memory management of the app.
@@nickchapsas I wanted to isolate the problem this weekend but with no success. Our solution was based on .NET Core 3.1. It was deployed in Kubernetes cluster on a machine with 32 cores with lot of memory available but the POD had memory limited. Also, the application used server GC. It might have been a combination of those curcumstances. What we knew, the problem was mitigated by replacing ConditionalWeakTable with different solution.
I use the ConditionalWeakTable in a ReferenceManager class that I use to automatically dispose shared large IDisposable objects (image data). When an IDisposable is shared (onto a different thread), I increase the refcounter in the CWT. When a shared item is no longer needed, the refcounter is decreased. When refcounter reaches zero the item is disposed.
This is cool when you are dealing with sealed classes I guess. But, what about the just old and simple inheritance approach? Is there something that I'm missing? This also adheres to the Open-Closed principle.
I was thinking the same thing. A simple class that inherits from ExternalObject could have all the properties you need. Also, it's way simpler.
It's a trend to hate inheritance in favour of composition
Same. This gives no advantage over inheritance or even a wrapper class. Just creates unnecessary problems.
I believe Nick meant that for, as u said it, sealed classes
I was thinking the same thing. The only place this would be better than inheritance is if you don't control where the objects get constructed. If they're constructed by the 3rd party DLL, you can't have it construct your inherited type instead.
Always have something new to learn. Keep Nicking
I have been aware of this thing for a few days, coincidentally I stumbled across it in a hobby project. I wanted to continue on it next weekend, you saved me a little research time with your video, so thanks 😄
I actually used this. I had a library to enable automatic chaining of NotifyPropertyChanged events based on attributes (for UI view models, which could have nested types). Had to weakly subscribe to events, and also keep track of which objects were being managed by my library. I think I used this for the latter. Memory management was definitely the hardest part. If I did that over again, I'd definitely use source generators.
Extension methods, using this from long time. Very good concept
Χρόνια πολλά Νίκο! Να χαίρεσαι το όνομα σου! Πάντα γερός και ευτυχισμένος!
Think I prefer to just wrap in a class/struct with the extra properties and access the base with a Value property. They you do not need the lookup and there is also no extra keeping track of the objects in the dictionary. Something like this, also more explicit what you are passing around.
public class ExtraPropsFor
{
public int MyExtraNumber { get; set }
public T Value { get; init; }
}
Never heard of this thingy but looks great! Gracias 🙏
I wish I knew this before. This is super useful if you want to track changes in a table (say you updated, deleted, created a new object) which is owned by another code base (which you might not want to change for only this use case).
Thanks for the video!
seriously I'm new to programming but I really don't know that why I always watch your videos.
Just because I love your videos and wish to be a PRO like you.
Love from Afghanistan.
@nickchapsas awesome video, maybe could you deep dive a bit on how ConditionalWeaktable works under the hood? I think this can be very useful
I "know" that I looked into those Weakly referenced collection types before. However, it's one of those things which isn't in my memory muscle and I kind of forgot about them. Future me, will most likely forget this approach again. It's a very usefull approach iso wrapping or subclassing. Will have to look into more real world scenarios to be able to start using it, and to make it part of my memory muscle.
Last i used it was to keep reference equals working on objects encapsulating an implementation through an older interface. So if a class implemented V1, it was encapsulated in a class implementing V2. When needing a V2, just check the ConditionalWeakTable for an existing instance before creating a new. Does require thread synchronization of course.
So in my toolbox, but used so infrequent I always have to Google for the name when I do need it.
I feel wrapper classes are more handy for this sort of work, but it's a preference I guess.
I'm gonna need some coffee before I try to wrap my head around this.
Used it in the past to reduce the memory footprint of a class by externalizing rarely used properties, resulted in too much pressure on the garbage collector (long finalizer queue). Perhaps you could make a follow-up video with benchmarks.
Can you explain ? How many objects did you have ?
I suggest Microsoft to implement "Extension properties"
It's been a suggestion for as long as extension methods exist but sadly it's probably not even on the radar.
Thank you Nick, I needed this, you're so helpful. You received a sub!
In the example I don't understand why removing an element in the list should affect the dictionary elements though. The elements pointers are the same but if I remove one element from the list I would still expect to have it in the dictionary. I think I am missing the point.
WeakReference looks interesting! I didn't know it. Thanks!!!
This is cool, but as others have mentioned the dispose "issue". It would be nice if there was some place to have a "CacheItemRemovedCallback" concept, so that you could provide a way to handle the remove event.
But since this weak reference concept is something that the GC understands and not the "user level" code, I guess this is not doable.
Very interesting. But why not simple inherit from the "ExternalClass" ?
Sure it's not possible when it's sealed.
I tend to prefer using the "is false" syntax over using the boolean inversion operator nowadays, it always sounds a lot more readable to me, what you think personally about this approach?
I used to think "== false" was ugly. But I've witnessed so many people misunderstand code because their eyes glossed over the "!". I'm starting to become a fan of "== false" and now "is false". C++ has an optional set of keywords if your keyboard doesn't support symbols. In that case, you can write "if( not variable ) {}", which is what I wish C# would adopt.
@@WilliamLeeSims
I was never a big fan of "== false", but I really like the "is false" syntax. In C#, "== false" can produce false positives, as our equality operators are overloadable, but "is false" compares the value directly, it always looks cleaner too due to being more fluid.
About your question, I believe you can do something similar currently, in C# you can use a syntax like: "if(variable is not const_value)", it is far from perfect but you got it
It is actually much more readable, and is actually necessary when using bool? instead of bool.
@@BenjaminBrienen
Yeah, that is true, btw, you can do something like "!(nullable_bool ?? false)" when handling bool?, but the is false syntax appears to be much more sweet
I prefer the full legibility of not using !. I saw a post years ago that also proved that it was slightly more efficient too but I have been trying to find it without success since then. The compiler might have evolved to make it not be any different since then.
When items are removed it’s important to know that IDisposable classes do NOT have their dispose method called.
Which is obvious because calling the Dispose method of IDispose objects isn't part of garbage collection
@@nickchapsas Just clarifying that, for folks like myself who are coming to .NET from other languages like C++ & PHP, I was unaware of this obvious information. Still, it makes sense since it isn't technically the class's destructor.
@@nickchapsas I've been a .Net developer for 15 years, and I didn't know this. I thought that Dispose WAS called on a GC event.
@@flurinbruhwiler9783 I honestly thought it was just a way to free up memory sooner, without having to wait for the garbage collector.
Classes that implement IDisposable should override the finalizer and call dispose in it anyways
In the use case given this solution seems to introduce complexity that is simply not needed unless the ExternalObject is sealed. Favour composition over inheritance does not mean inheritance loses its value completely.
ExternalObjectEx externalObject = new ExternalObjectEx();
Console.WriteLine($"{externalObject.Id} {externalObject.text}");
public class ExternalObject
{
public Guid Id => Guid.NewGuid();
}
public class ExternalObjectEx : ExternalObject
{
public string text => "Hello World";
}
I’m developing my own game (and a game engine) and I use “WeakLists” and Weak reference event handlers. On one hand I have full control over game objects from the engine, and on the other hand storing data about game objects doesn’t prevent GC from collecting them if the game has no need in them
But then you have to clean up the list periodically, remove lost refs.
@@Christobanistan sure. I made both WeakList and Weak event handler clean themselves periodically
I think this is the first time I've ever seen a WeakReference ever having a usecase.
I've always known of them, but as I've never really used lower level languages I've never seen a usecase for them.
There is a good usecase from Steve Gordon here: www.stevejgordon.co.uk/accessing-state-in-system-text-json-custom-converters
I am using this to add implicit interface implementation to interfaces in pre C# 8.0 (pretty much UWP projects) where it is not allowed.
You can use the same method to write stateful interfaces for a pseduo mixin experience
Aren't you assuming that the type you don't control correctly implements GetHashCode for this behaviour to work?
Otherwise there's no guarantee that the added properties are actually correctly associated with the instances you want them to be associated with (Aside from all the other issues with GC).
I'd say ideally this is a case for inheritance (If the class isn't sealed).
Next best is a partial (Requires that the class you don't control was declared as partial).
Finally I'd consider a wrapper with pass-through methods / properties if I had to, hopefully can consolidate that with the original type at an interface level, or just work with the wrapper within your original code.
Just wanna mention that Roles & Extensions will allow you to define extension properties, a territory this concepts leans quite a bit into.
When we eventually get it
@@nickchapsas In theory it's already possible, PoC here:
github.com/BlinD-HuNTeR/ExtensionEverything-POC
Probably obvious, but if an object implements IEquatable and overrides GetHashCode(), this may fail if the only difference between two objects are supposed to be the extending properties.
I think we have been using static ConcurrentDictionary, but I think based on the conditions, replacing with ConditionalWeakTable in some cases might be a good step? Time for investigation...
It just occurred to me the IEnumerable Count() method name is a verb but the IList .Count property name is a noun.
I need to remember weak references exist and to use them more for things like caches. I think I've maybe used them in my code once maybe.
Hi nick
Can you explain the difference between await and ConfigureAwait(false)?
Thanks Man!
I have a custom cancelation token, and I use this as a cache to convert to/from the TPL cancellation token without creating a new object every time it's converted.
IIRC this is something like how dependency properties in WPF are implemented.
Hi I would love to attend one of the conferences. Unfortunately I have not been to a conference yet and would like to experience it. Greetings from Germany
If the class is not sealed, wouldn't it be better to use a subclass to contain the extended properties?
By using a subclass, we can share the reference of the instance by using the Unsafe.As(ref TFrom source) method, for example: "public static Subclass GetExtensions(this Baseclass instance) => Unsafe.As(ref instance);". This provides the benefit of having an object that provides access to both the base and subclass' members.
Sure but assume that it is sealed. I forgot to mention that in the video
Maybe you don't know the type at compile time
I don't know, in my opinion just creating a wrapper for this would be cleaner. The problems I see are the following:
- How do you know that you need to call GetProps?
- CoditionalWeekTable creates more garbage than just creating a wrapper class
- Accessing these properties has more overhead than accessing properties from the wrapper class
- You communicate/document it
- No it doesn't. Except for the table itself, it's the same amount of object that the system needs to create.
- Does it? I would assume you just get the props object once which is an O(1) operation and access all the properties that you need
Hey Nick, thanks for answering :D Wasn't really expecting it :D
- That's the thing, you need to document it. Just making a wrapper seems (at least to me) something that is sefl-document and understandable right away.
- Oh my bad, here I thought that it creates a WeekReference object for every new entry, but I have checked and it doesn't, but then the actual problem with this approach is that it doesn't work with structs. For structs you would either need to box it or again create a wrapper, which would need to be a class. With a wrapper you could create a new struct wrapper.
- This is then another thing you need to know, that you need to cache the properties. Just accessing GetProps in a loop without caching it can lead to significant performance differences compared to the wrapper. And I agree that most of the time it should be fine.
Why couldn't we just use inheritance for that ?
Create a child object, and create a constructor that takes the base class as an argument, and now you can add whatever you want
Sure, it will not work on structs and sealed classes, but I've rarely if ever seen them on my (admittedly limited) experience in C#
Thanks for showing this approach)
But why cant we just use an inheritance? Or should we treats ExternalObject class as sealed?
Yeah I forgot to mention this but assume it as sealed. Also, this isn't just about properties but it's the easiest example. It is also about object assosiations in a weak reference way
There might be existing code that you don't know that is passing around a reference to the external object. Since you can't change the type reference, you basically need to bolt additional state to the instance which is what the weak reference table does.
humm, ok and how about composition + adapter? You compose your new class to embark the immutable one and add what behavior is needed. And if needed, add an interface to the mix. This video is quite interesting but it seems a little too overcomplicated. But definitively interesting, thnak you
You may not be the who creates an instance
Hmmm. As someone slightly unfamiliar with .NET internals, rather than a sort-of weakref table, my first instinct would've been to look and see if there's an event or similar thing in which to register a callback for when the object is freed. Does something like that exist? "When this object is freed, do X where X is removing the object from this Dictionary," i.e. allowing a user of the object to append customized cleanup operations for the instance.
Hmm I actually know a use case. If there is like a proxy which is generating references and you cant change them (even though you bloody own the proxy, dont ask) then this kind of thing could perhaps had worked. How ever, we built a new service instead because that design was an aweful mess. I guess if you have no choice it's good to know you have multiple ways to expand classes. That container was new to me and it's always nice to learn new things
I can't seem to successfully add properties that don't make infinite loops on my OWN classes,and here I am watching how to do it in classes i dont own :P
Yes, I was aware. No, I have not used.
Its an interesting approach and its the first time seeing the use of ConditionalWeakTable, so thanks for sharing that. But i dont like it. If i want properties for a class that i dont own, than i create a wrapper. Thats totally fine by me, because it may be most likely be replaced by something else anyway. For me this is most likely an indicating that the external class may be not really suitable for that problem i want to solve or have simple bad API design.
How smart is it about circular references? E.g. if you insert the Key x with Value List{x} into the dictionary, will that never get collected, because there is a strong reference? Even though if you deleted the key as if there were no strong references then there would cease to be any strong references?
How does this work if the external object is factoried from a DI container with a lifetime of scoped? The external properties are static right ? Will the external property get disposed when the external object goes out of scope?
When the main object gets GCed, the entry in the table will also be removed
What if we don't call GC.Collect manually? Does the GC eventually gets rid of the weak table's unreferenced element?
Yes. Calling the gc collect was just to force it for video.
I've extended a class I don't own using the "partial" keyword (i.e. EF6 object and added some additional properties).
Why would you not own EF classes? They're yours, which is why you can mark them as partial.
It would not work with Nick's example, as he has no control over that ExternalObject class.
That's only possible when the external class is defined as partial, which is usually (at least in my work) not the case.
@@michaldivismusic My understanding is that it would make no difference if the external class is marked as partial, because it is external, and the "partial" part does not exist on the IL level (try it on sharplab). So a partial class will only solve the issue for you if you can build it yourself at the same time as your partial extension.
@@zabustifu wow, thanks for the tip, I had no idea.
I'm at 3:15, and I'm going to try and predict what method you'll use...
ExpandoObjects with a dynamic casting... (If you see this message edited, I was wrong)
:D
---
I was wrong :(
xD
Hello Nick! Do you have any tip about WebRTC using C# ? I've been following this topic for a long-long time and it still a nightmare doing anything that uses WebRTC. Anyway, Very nice video!
I would prefer Extensions properties... like extensions methods..
Yeah, I just don't understand why Microsoft won't add extension properties. Would be very useful.
Apparently F# already has them
Is it possible to data bind the text property in wpf, and if yes, how?
If you had used release configuration insted of debug, you would have got a stable zero reference count due GC implementation details.
It is also completely thread safe
Yeah there are locks in every operation
I would like to see extention props but really extention methods are same thing. In Java or C++ there are get and set methods. Just because we like props in c# doesn't mean we can't ever use methods
Doesn't this ruin sealed? Also, this may have changed but there was a time when my program was under heavy load, and the GC can be a bit lazy itself. The fix was to put in Gc.WaitForPendingFinalizer(), actually two of them, ah and at the time it was someone on the runtime who suggested the fix... While it was back in .Net Framework 3.x days i.e. Orcas or Whidbey...
Looks to me like you're trying to implement ECS, but with only one component.
When you would need this?
Here is a real world example: www.stevejgordon.co.uk/accessing-state-in-system-text-json-custom-converters
Is it the kind of use case that Roles will fulfill in C# 12, if it makes the cut?
Sort of. It will fullfill one use case, but weakly referenced object association is still a valid usecase. Also I wouldn't hold my breath for roles making it into C# 12
@@nickchapsas I sure hope it does. But I guess you're right
Still waiting for extension properties as in f#
F# has extension properties????
@@nickchapsas
yes
open System.Collections.Generic
type List
If you have the class avaiable in your code through nuget or otherwise, couldn't a good use case be to inherit from it and add the extra properties in your own super-class and use Automapper to map it and maybe fill the new extended properties too?
What if it's sealed?
@@nickchapsas Then it won't work, but there are not that often you step onto sealed classes that is ment for an external API or that you want to extend. So if it is Sealed I think it can be a good idea to consider if it is meant to be extended or if you should map it to your own Class instead. If I look at i.e the CosmosDb SDK which I know you have worked a lot against too and see the sealed classes there with CosmosSerializationOptions and such things. I think I would create my own classes if I needed to extend that class, but maybe this way you show would be better? But if it is not Sealed, would you prefer to inherit and add properties to super class then or would you still prefer this way?
@@jonasgranlund4427 You are thinking this the wrong way. This isn't about class extention. It is abou data association. It's very different
Why would you do this instead of just declaring a subclass? Not sure what I'm missing
Assume it's sealed
You might not be able to. You might work with sealed class. You might work with a collection of data, that you cannot change (readonly collections) or any other stuff. This is cleaner and simpler to create a new collection of data of new wrapper classes, of data you want to work with. Combine this with extensions and its very very neat and clean in upper level code.
Isn't this approach broken by implicilty assuming that GetHashCode of that external object must be stable througout whole app lifetime? Example:
public class ExternalObject
{
public int Id { get; set; }
public override int GetHashCode()
{
return Id.GetHashCode();
}
}
var dict = new Dictionary();
var extObj = new ExternalObject { Id = 1 };
dict.Add(extObj, "test");
extObj.Id = 2;
var b = dict.ContainsKey(extObj); /*b is false*/
No. ConditionalWeakTable keys based on reference equality, not object equality. As such, conditionalWeakTable uses RuntimeHelpers.GetHashCode(key) instead of Object.GetHashCode.
@@xxgn thanks for clarification 😀
So i added an extension to object so you can add a property to any object using this method..
public static class ObjectExt
{
public class ExtensionProperty
{
public T Property { get; set; }
}
private static readonly ConditionalWeakTable Data = new ConditionalWeakTable();
public static ExtensionProperty GetExtension(this object subject)
=> Data.GetOrCreateValue(subject);
}
Used like this:
object myObject = new object();
myObject.GetExtension().Property = "myData";
var myData = myObject.GetExtension().Property;
any thoughts?
In this video “C# community invents JavaScript” 😂
Just out of curiosity, what is the cost of the ConditionalWeakTable? Until now, I've never had to use this approach for any problems, as I always find more elegant options, however, I'm curious to know how much it costs
It's a keyvaluepair extension with a special key. There is no inherit cost apart form that. The only thing I didn't mention is that it is fully thread safe via a lock, so that aspect does affer the get and set performance of the type
@@nickchapsas
I mean, the cost of removal, I imagine for something like this to work it has to be deeply tied to the CLR, or make some use of finalizers and maybe some hacky things, but it's interesting to know that it's thread safe.
@@diadetediotedio6918 I didn't look at the inner implementation. Probably they just store the memory address thats points to the object and they validate it, when somebody tries to access that key.
@@diadetediotedio6918 there's no hook into an object's removal and there's no active tracking of an object's existence. The reference is validated upon access and removed if it's dead.
@@CrippleX89
Oooh, fine fine fine, but then it will not bring the cost of constantly keeping empty memory entries over time?
Roles and Extension everything coming in C# 12
Yeah also the field keyword is coming in C# 10
@@nickchapsas Yea. You meant C# 12?
@@obinnaokafor6252 I meant 10
@@nickchapsas I thought field keyword was postponed?
@@obinnaokafor6252 That's the joke. It was supposed to be in C# 10, then it got postponed to C# 11 and then to C# 12. What I'm saying is don't hold your breath for any "announced" feature to make it into the next C# version
I'd only use this as a last resort, preferring simple inheritance.
i would rather create a wrapper...
Can't I just Extends ":" it ?
Using an object rather than an ID as a key in the dictionary/table seems ... bloated
couldn't you have used reflection here? This seems too complicated
Reflection would have been a terrible solution
@@nickchapsas writing terrible code is funny
ConditionalWeakTable is very very slow.
Ciuld you PLEASE talk not so fast... very hard to follow-up when you are not natie english spoken.
Use the playback speed feature
Console.WriteLine("Nick Chapsas");