Thanks for the video. Didn't even know CreateWeakLambda exists and I avoided all lambda bindings in my code. One thing to note is that not all "regular" timer delegate bindings are safe. For instance the BindUFunction one also has the same memory leak problem, at least it's the case in 5.2.
Thank you so much! I found this video by coming from another one of your videos and then binge watching some of your videos lol. I didn't even know how I could search for my problem but this was exactly it. :D
Great video. Thanks for explaining this so well. I was researching the topic and everybody seems to agree that when you need multiple objects to be valid for a Lambda (Weak Lambda) it is preferable to use TWeakObjectPtrs for the other objects not captured. But is it really the case? wouldn't "IsValid(MySecondaryObject)" would be sufficient?
"wouldn't "IsValid(MySecondaryObject)" would be sufficient?" Hey, that's a great question. In short, it isn't sufficient. Looking into IsValid() definition, it appears to first check that the pointer isn't null, then check flags on the thing pointed to (at least in source provided with the 5.3 download, I need to update to 5.4). But this isn't actually able to detect a garbage collected pointer (from my testing). And IsValid() can return true even though the pointer's contents have been garbage collected. So, you can have IsValid() return true for something garbage collected. So, where is IsValid useful? The engine has a "MarkAsGarbage()" function call, that effectively flags an object to be garbage collected, even though something has a reference to it (ie a UPROPERTY()). The IsValid function will return "false" if an object has been marked for garbage, but has not yet been garbage collected. In the context of lambda captures, Lambda captures are not like UPROPERTY() references. A lambda-captured-object-ptr may have no references, and never be marked for garbage. But a captured object will be garbage collected eventually (if its only usages is the capture), since the engine isn't aware of any UPROPERTY references. In this situation (at least in a 5.3 project I have tested), the IsValid on a stale pointer will return true, despite that pointer being garbage collected. So I would not use IsValid as a way to try avoid GC issues. TWeakObjectPtr however uses a different mechanism to validate its contents are still valid, and can be safely used instead of IsValid. Alternatively, if you capture a `this` pointer as a WeakObject, eg a WeakThis, and all other objects needing captured are proper UPROPERTIES member variabels of the captured object, you just validate the captured WeakThis is still valid, and use the object pointers off of that. I should make a video on the IsValid function. But for now was hoping to explain that bit so you don't have to wait for a video.
Hey@@enigma_dev , many thanks for your thorough response. I'm a bit stubborn, so I wanted to dive deeper into the various options. I agree that TWeakObjectPtr::IsValid() is what it should be used for lambdas. I spent some more time investigating this. I even created a test project (feel free to check it out if you'd like) to see how the Engine handles the different options. Summary Key Points: IsValid() checks if an object is null or marked as PendingKill, but it won't always detect if the object has already been garbage collected. This can lead to cases where IsValid() returns true even though the object has been partially cleaned up. TWeakObjectPtr::IsValid() It automatically nullifies the weak reference when the object is garbage collected. IsValidLowLevel() is a lower-level check that validate an object's memory state. It is very expensive but reliable. IsValidLowLevelFast() Cheaper than IsValidLowLevel(), but it might still return true for objects that have been garbage collected, unreliable. Please check: github.com/BionicApe/GarbageCollectorTest
@@bionicape683 Nice, very professional looking test. TIL about IAutomationLatentCommand. I wouldn't call that stubborn, rather professional information gathering. :) Empirical tests are often a good source of information. Your test is more robust than what I am about to suggest, but if you ever want to make a quick bespoke empirical test, I suggest looking into auto console commands. I often find myself sanity checking things by using a snippet to generate one, and writing a quick test. Btw, I did some prep to make a video on IsValid, but didn't get into the IsValidLowLevel stuff for TWeakObjectPtr. Hoping to find some time in the coming weeks to turn that into a video, but I have a busy fall. IIRC IsValid is also useful for particle and sound systems that are set to autodestroy. As it quickly detects when they're going to be garbage collected, vs when they're still around but disabled. But I need to make sure I am right on that and write some tests to test that aspect, before I make a video.
@@enigma_dev Thanks so much. To be honest, I’m really glad I found your videos. It’s really difficult to find someone who is truly passionate. Hive mentality is very common in the game dev scene, especially in Unreal Engine. Having someone like you who truly cares about what they’re talking about is refreshing.
@@enigma_dev Btw, what do you mean by auto console commands? The good thing about unit testing is that you don't need to launch the editor, and you can iterate very fast, you can even launch them inside the IDE. As far as I know, you do need to launch the engine if you use the console commands, unless we are talking about different things.
Hey, you mean like I did in the video? I have a video titled: "How to make a console command in Unreal Engine - UE C++ Tutorial" th-cam.com/video/zK43zinv8Cg/w-d-xo.html It goes over setting up bespoke console commands. If you want the "Exec" system, I have a video: "Unreal C++ Exec Functions vs ke * - UE C++ Tutorial 6 Pt7" th-cam.com/video/5B-5qWFKEpQ/w-d-xo.html That covers exec functions and an laternative using ke* cheats.
Pretty cool video, in your description you state that this doesnt happen with regular bindings because they're associated with an object. Can you explain why that is? Why is the bound lambda with the captured "this" not associated to the object pointed to by "this" by default?
@@zeez7777 Yes, the lambda is a copy of the object where the this pointer is a pointer to the object you've captured which allows you to modify the object. This is the 'closure' part of the lambda. You can start to see the problem if those objects you're pointing at through the lambda's this pointer are no longer there.
Hey, great question. I should have elaborated a bit more on that. tl;dr My understanding: Unreal delegate bindings are Unreal code, and so are it's the concept of UObjects and garbage collection. They're separate from the C++ language. The C++ language has lambdas with the ability to capture arbitrary addresses. In C++, it is hard (if not impossible) to query what is in the captures from outside the lambda's body. So you must explicitly write code to be safe with lambda's capturing memory addresses, since the engine doesn't know about what you have captured in your lambdas. Part1: why a regular `Delegate:CreateUObject` binding is safe even though it requires a `this` pointer too: Here's the way I think of it. Recall that if you have something like UPROPERTY() UObject* MyObject; Unreal will track a reference to MyObject preventing it from being garbage collected. I think that same engine machinery for object-reference-tracking, seems to be used here, but it actually allows the object to be GC'd, -- the delegate just doesn't call the binding if the bound object is no longer valid. When you bind a delegate with Delegate::CreateUObject (or via AddUObject) you must give it a raw pointer the UObject that is being bound to the event. (see example at 7:48) Unreal behind the scenes tracks this object, and starts monitoring if it is a valid pointer, ie has it been garbage collected. I'm not familiar with what code doing that tracking behind the scenes, but it might be similar to the internals of TWeakObjectPtr. Anyways, the important bit is delegate bindings like above are apparently aware of when the `this` object has been deleted via the garbage collector, via some mechanism. And when it comes time to invoke/broadcast the delegate (in the case, when the timer's time is up) it checks to see if the reference has gone stale, and only invokes the delegate if it is still valid. This is all very similar to the protection BindWeakLambda gives, see the section of this video at timestamp 2:14 Part2: why a raw lambda binding, that captures `this`, is not also safe, why it is not associated with the objected pointed to be `this`: So, Delegate::CreateLambda is unaware of the contents in a lambda capture. The lambda could capture `this`, or the lambda might not capture `this`. The delegate binding is unaware in either case. Note: the lambda is raw C++, and this problem exists in regular C++ as well, it isn't just specific to Unreal (in regular C++ it happens with delete keyword) Anyways, the `this` ptr is just an address, for example let's just say it is address 0x12345678. The lambda is just capturing that address number, variable like it would capture an int number variable. When the lambda binding is invoked/broadcast, it is trusting that you are taking the precautions that all the captures are still in scope and valid. It doesn't know you captured "this", or anything else. I think that's why CreateWeakLambda exists, to explicitly communicate to the engine you're capturing "this" and want the engine to keep track of whether or not it got freed (hence the reason you have to type `this` two times, the first argument and the second argument). (at least, that is my inference, I don't actually know for certain that is why CreateWeakLambda exists, but it makes sense to me trying to reason about this) Even if you could look at the captured address, it isn't truly enough information. An object at address 0x12345678 may have been freed from the garbage collector. And a new object was created at address 0x12345678. In this case, you have a different bug than operating on freed memory. You have a lambda callback operating on a different object than it thought it captured! The memory at the captured address changed without the lambda ever knowing it changed. (TWeakObjectPtr solves this problem I believe, so you can look at that code to see how, I'm not sure) IMO: when the memory is deleted from the garbage collector, It is really the same problem as the following code using ints in raw C++, which you can test on the compiler explorer website godbolt: """ #include int main() { int* value = new int; *value = 5; auto lambda_read_value = [value](){ std::cout
@@enigma_dev Thanks for the extremely detailed response :) I see what you're saying now. Unreal isn't aware of the captures and indeed now that i think about it i dont know how it would be, apart from its solution which is calling a function and explicitly specifying the captures. This begs the question that if you want to capture more than just "this" and want to capture another pointer (not just local variable) then we'd need another function to pass all those captures manually. As far as how i would write code that inspects lambda captures, i'm not sure. I'm vaguely aware that C++ doesn't offer reflection. My UE journey just started 3 months ago and im still struggling understanding all of the dynamics between "real c++" and generated "magic" from the UHT so i was assuming it would be possible for it to parse my code and generate wrapper code around those bindings. Some sort of reflection at compile time where it would do the work of manually specifying the captures.
Thanks for the video. Didn't even know CreateWeakLambda exists and I avoided all lambda bindings in my code.
One thing to note is that not all "regular" timer delegate bindings are safe. For instance the BindUFunction one also has the same memory leak problem, at least it's the case in 5.2.
Thank you so much! I found this video by coming from another one of your videos and then binge watching some of your videos lol. I didn't even know how I could search for my problem but this was exactly it. :D
Thank you. The videos are cool and informative. Love it.
awesome explanation
great as usual
Great video. Thanks for explaining this so well.
I was researching the topic and everybody seems to agree that when you need multiple objects to be valid for a Lambda (Weak Lambda) it is preferable to use TWeakObjectPtrs for the other objects not captured. But is it really the case? wouldn't "IsValid(MySecondaryObject)" would be sufficient?
"wouldn't "IsValid(MySecondaryObject)" would be sufficient?"
Hey, that's a great question.
In short, it isn't sufficient.
Looking into IsValid() definition, it appears to first check that the pointer isn't null, then check flags on the thing pointed to (at least in source provided with the 5.3 download, I need to update to 5.4).
But this isn't actually able to detect a garbage collected pointer (from my testing).
And IsValid() can return true even though the pointer's contents have been garbage collected.
So, you can have IsValid() return true for something garbage collected.
So, where is IsValid useful?
The engine has a "MarkAsGarbage()" function call, that effectively flags an object to be garbage collected, even though something has a reference to it (ie a UPROPERTY()).
The IsValid function will return "false" if an object has been marked for garbage, but has not yet been garbage collected.
In the context of lambda captures, Lambda captures are not like UPROPERTY() references.
A lambda-captured-object-ptr may have no references, and never be marked for garbage.
But a captured object will be garbage collected eventually
(if its only usages is the capture), since the engine isn't aware of any UPROPERTY references.
In this situation (at least in a 5.3 project I have tested), the IsValid on a stale pointer will return true, despite that pointer being garbage collected.
So I would not use IsValid as a way to try avoid GC issues.
TWeakObjectPtr however uses a different mechanism to validate its contents are still valid, and can be safely used instead of IsValid.
Alternatively, if you capture a `this` pointer as a WeakObject, eg a WeakThis, and all other objects needing captured are proper UPROPERTIES member variabels of the captured object, you just validate the captured WeakThis is still valid, and use the object pointers off of that.
I should make a video on the IsValid function.
But for now was hoping to explain that bit so you don't have to wait for a video.
Hey@@enigma_dev , many thanks for your thorough response. I'm a bit stubborn, so I wanted to dive deeper into the various options. I agree that TWeakObjectPtr::IsValid() is what it should be used for lambdas.
I spent some more time investigating this. I even created a test project (feel free to check it out if you'd like) to see how the Engine handles the different options.
Summary Key Points:
IsValid() checks if an object is null or marked as PendingKill, but it won't always detect if the object has already been garbage collected. This can lead to cases where IsValid() returns true even though the object has been partially cleaned up.
TWeakObjectPtr::IsValid() It automatically nullifies the weak reference when the object is garbage collected.
IsValidLowLevel() is a lower-level check that validate an object's memory state. It is very expensive but reliable.
IsValidLowLevelFast() Cheaper than IsValidLowLevel(), but it might still return true for objects that have been garbage collected, unreliable.
Please check:
github.com/BionicApe/GarbageCollectorTest
@@bionicape683 Nice, very professional looking test. TIL about IAutomationLatentCommand. I wouldn't call that stubborn, rather professional information gathering. :) Empirical tests are often a good source of information.
Your test is more robust than what I am about to suggest, but if you ever want to make a quick bespoke empirical test, I suggest looking into auto console commands. I often find myself sanity checking things by using a snippet to generate one, and writing a quick test.
Btw, I did some prep to make a video on IsValid, but didn't get into the IsValidLowLevel stuff for TWeakObjectPtr. Hoping to find some time in the coming weeks to turn that into a video, but I have a busy fall. IIRC IsValid is also useful for particle and sound systems that are set to autodestroy. As it quickly detects when they're going to be garbage collected, vs when they're still around but disabled. But I need to make sure I am right on that and write some tests to test that aspect, before I make a video.
@@enigma_dev Thanks so much. To be honest, I’m really glad I found your videos. It’s really difficult to find someone who is truly passionate. Hive mentality is very common in the game dev scene, especially in Unreal Engine. Having someone like you who truly cares about what they’re talking about is refreshing.
@@enigma_dev Btw, what do you mean by auto console commands? The good thing about unit testing is that you don't need to launch the editor, and you can iterate very fast, you can even launch them inside the IDE. As far as I know, you do need to launch the engine if you use the console commands, unless we are talking about different things.
Hello, could you please explain how do use execute function from unreal console ?
Hey, you mean like I did in the video?
I have a video titled:
"How to make a console command in Unreal Engine - UE C++ Tutorial"
th-cam.com/video/zK43zinv8Cg/w-d-xo.html
It goes over setting up bespoke console commands.
If you want the "Exec" system, I have a video:
"Unreal C++ Exec Functions vs ke * - UE C++ Tutorial 6 Pt7"
th-cam.com/video/5B-5qWFKEpQ/w-d-xo.html
That covers exec functions and an laternative using ke* cheats.
Pretty cool video, in your description you state that this doesnt happen with regular bindings because they're associated with an object.
Can you explain why that is? Why is the bound lambda with the captured "this" not associated to the object pointed to by "this" by default?
You're capturing a copy so you aren't binding to any specific object instance when invoked
@@softwaretools5140 Copying the this pointer you could still infer the object instance though?
@@zeez7777 Yes, the lambda is a copy of the object where the this pointer is a pointer to the object you've captured which allows you to modify the object. This is the 'closure' part of the lambda. You can start to see the problem if those objects you're pointing at through the lambda's this pointer are no longer there.
Hey, great question. I should have elaborated a bit more on that.
tl;dr My understanding: Unreal delegate bindings are Unreal code, and so are it's the concept of UObjects and garbage collection. They're separate from the C++ language. The C++ language has lambdas with the ability to capture arbitrary addresses. In C++, it is hard (if not impossible) to query what is in the captures from outside the lambda's body. So you must explicitly write code to be safe with lambda's capturing memory addresses, since the engine doesn't know about what you have captured in your lambdas.
Part1: why a regular `Delegate:CreateUObject` binding is safe even though it requires a `this` pointer too:
Here's the way I think of it.
Recall that if you have something like
UPROPERTY()
UObject* MyObject;
Unreal will track a reference to MyObject preventing it from being garbage collected.
I think that same engine machinery for object-reference-tracking, seems to be used here, but it actually allows the object to be GC'd, -- the delegate just doesn't call the binding if the bound object is no longer valid.
When you bind a delegate with Delegate::CreateUObject (or via AddUObject) you must give it a raw pointer the UObject that is being bound to the event. (see example at 7:48)
Unreal behind the scenes tracks this object, and starts monitoring if it is a valid pointer, ie has it been garbage collected.
I'm not familiar with what code doing that tracking behind the scenes, but it might be similar to the internals of TWeakObjectPtr.
Anyways, the important bit is delegate bindings like above are apparently aware of when the `this` object has been deleted via the garbage collector, via some mechanism.
And when it comes time to invoke/broadcast the delegate (in the case, when the timer's time is up) it checks to see if the reference has gone stale, and only invokes the delegate if it is still valid.
This is all very similar to the protection BindWeakLambda gives, see the section of this video at timestamp 2:14
Part2: why a raw lambda binding, that captures `this`, is not also safe, why it is not associated with the objected pointed to be `this`:
So, Delegate::CreateLambda is unaware of the contents in a lambda capture.
The lambda could capture `this`, or the lambda might not capture `this`.
The delegate binding is unaware in either case.
Note: the lambda is raw C++, and this problem exists in regular C++ as well, it isn't just specific to Unreal (in regular C++ it happens with delete keyword)
Anyways, the `this` ptr is just an address, for example let's just say it is address 0x12345678.
The lambda is just capturing that address number, variable like it would capture an int number variable.
When the lambda binding is invoked/broadcast, it is trusting that you are taking the precautions that all the captures are still in scope and valid.
It doesn't know you captured "this", or anything else.
I think that's why CreateWeakLambda exists, to explicitly communicate to the engine you're capturing "this" and want the engine to keep track of whether or not it got freed (hence the reason you have to type `this` two times, the first argument and the second argument). (at least, that is my inference, I don't actually know for certain that is why CreateWeakLambda exists, but it makes sense to me trying to reason about this)
Even if you could look at the captured address, it isn't truly enough information.
An object at address 0x12345678 may have been freed from the garbage collector.
And a new object was created at address 0x12345678.
In this case, you have a different bug than operating on freed memory.
You have a lambda callback operating on a different object than it thought it captured!
The memory at the captured address changed without the lambda ever knowing it changed.
(TWeakObjectPtr solves this problem I believe, so you can look at that code to see how, I'm not sure)
IMO: when the memory is deleted from the garbage collector, It is really the same problem as the following code using ints in raw C++, which you can test on the compiler explorer website godbolt:
"""
#include
int main()
{
int* value = new int;
*value = 5;
auto lambda_read_value = [value](){
std::cout
@@enigma_dev Thanks for the extremely detailed response :)
I see what you're saying now. Unreal isn't aware of the captures and indeed now that i think about it i dont know how it would be, apart from its solution which is calling a function and explicitly specifying the captures. This begs the question that if you want to capture more than just "this" and want to capture another pointer (not just local variable) then we'd need another function to pass all those captures manually.
As far as how i would write code that inspects lambda captures, i'm not sure. I'm vaguely aware that C++ doesn't offer reflection. My UE journey just started 3 months ago and im still struggling understanding all of the dynamics between "real c++" and generated "magic" from the UHT so i was assuming it would be possible for it to parse my code and generate wrapper code around those bindings. Some sort of reflection at compile time where it would do the work of manually specifying the captures.