I've found this Crust of Rust series after the fact. Just wanted to comment and thank you for taking the time to put these streams up to youtube. This is all very good information that is well delivered, and it is helping me greatly in my quest to learn rust; and indeed I think making me a better programmer overall.
I love this Crust of Rust series, thank you so much for making this! I have a couple of questions about this one: 1. Why not explicitly declare that a type parameter is "dropped on Drop", instead of using PhantomData hack? 2. Why does the compiler require every generic parameter T to appear in the type definition? If it didn't we would get rid of the other reason for PhantomData.
Yet another amazing stream! I really enjoy the content you produce @Jon Gjengset and learn a lot from it. Thank you! (favourite moment 1:00:33 - 1:00:42 🤣 (would make for a hilarious meme))
Regarding the "unnecessary complexity", rust as a whole is "unnecessarily" complex. Rust is designed to restrict what code you can write, to prevent certain bugs and to semi-automate memory management. It does so by always erring on the side of caution. The complexity is there to communicate properties and behavior of your code, to the compiler. So that it knows when caution was already taken by the programmer.
@@jonhoo That's why I used the quote marks. It's unnecessary in a sense, that a programing language can function without it. It just wouldn't have the specific features Rust has.
I think Empty(PhantomData) compiles because not implementing std::ops::Drop means that it does not take a &mut self (mutable reference to self) at the end of a block, so the compiler can know that it can take a immutable reference safely, yes? Maybe the actual drop (not the side effects we write in std::ops::Drop::drop) would probably just consume self, not take a mutable reference?
I believe that if you use `PhantomData` you can maintain the const-compatibility, covariance, and `Send` as opposed to `*const T` or `fn() -> T`, while still letting the drop check know that you won’t be dropping an actual `T`.
So why isn't variance included as part of the type signature? I was under the impression that changing the private fields of a type does not count as a breaking change, but it seems like converting Boks from using a *mut to a NonNull (or vice versa) would fundamentally break a bunch of code, even though there was no direct change to type signature. I feel like the compiler is doing some magic by looking inside the Boks, something which I feel like it shouldn't be allowed to do, in the same way that its not allowed to look inside a function when that function is called (all the necessary information about the contents of a function should be encoded in its signature).
There are already other examples where changing private details of a type isn't backwards-compatible. Some examples: - Adding a private field to a type that only has public fields. - Adding a non-Send field to a type that was Send (without an explicit Send impl) - the same applies to Unpin and Sync - (as you observed) Changing the variance of a type parameter away from covariant/contravariant - changing it _from_ invariant is fine - Changing a type such that a type parameter is subject to the drop check where it wasn't before (the other way is fine). - Implementing a pre-existing trait (such as one from std) for a pre-existing type. In general, the backwards-compatibility guarantees for a type includes a bunch of private details, and it has never been true that _just_ the public-facing type signatures are sufficient.
Can you clarify why, "accessing the T" is not fine in the Drop, but "dropping it" is? I am trying to square this logically in my head, and no matter how I do, I can't understand it. It seems to me, that the "promise" or "guarantee" of not touching/accessing the T, is due to how Rust works, to keep aliasing rules and all that good stuff, but doesn't dropping something *essentially* "do" something to/with the T? I mean, once the value/data is dropped, it's gone, therefore any mutable reference to it, can't continue to exist, and since drop is called when it goes out of scope, nothing else (unless parallel from another thread or whatever) can access it at that point, at least not in sequential code. Which also means, that whatever access to the T there is, will always be the last thing that happens to it, inside of the drop function, thus no mutable reference can see that change anyhow. I know this is not how it works, I'm trying to make sense of the reasoning that's all.
Yeah, that confused me a lot in the beginning too. However, as you will see later in the video (with the Oisann example) that dropping T can, indeed, access T, if T has a implements Drop. Nevertheless, the initial example with "Boks::ny(&mut y)" works, because T is actually &mut i32, which is a reference. When you drop a reference, it is a no-op - you are not dropping the value that got referenced to, because you are only dropping the reference itself. It is like doing this: let mut a = 100; let reference = &mut a; *reference = 300; drop(reference); println!("{}", a); (I am still new to Rust. If anyone spot an error in my explanation, please let me know.)
@@rintepis9290 Of course, I had glossed over that entirely that it was a reference, that obviously makes much more sense. I was thinking about it owning the T, not holding a reference to it
Ah, no, Rust does not protect you from _leaking_ memory. It protects you from memory unsoundness (like use-after-free), but leaking memory isn't really an error, it's just an efficiency problem.
Man I really got to try Rust, it has way too many benefits, the only bad thing I see so far is that the compiler it’s really strict but I guess that’s the price to pay for a safe non GC language, also I love the sintax it’s very similar to Kotlin for the most part
Same amount of time debugging, just in Rust you spend it fixing compiler errors instead of diagnosing segfaults. Once you finally fix all 893 compiler errors and it compiles, I have never had a program crash.
@@bananarobotoverlord I prefer fixing compile time errors rather than runtime errors, haven’t tried rust yet, but I will probably use it for a small game with the bevy engine
I wonder if "MetaData" wouldn't have been better, than PhantomData as a name? It seems to me, that's exactly what it is, because it *isn't* data, but it is meta data. I suppose it could've been called "SpiritualData" 🤣 Or named it something even more concise, like "CompilerMetaTag" or "CompilerMetaData" or something like, to *really* hammer home what it's used for. Or am I misunderstanding the purpose of PhantomData?
From Wikipedia: Metadata is "data that provides information about other data". If you say "_p: MetaData", that implies "_p" is metadata. However, "_p" is definitely not data (because it is 0 sized), therefore not metadata.
@@rintepis9290 Which I suppose is why I provided the alternative "CompilerMetaTag" or "CompilerMetaData". And I don't see how _p is meta data, when the type itself in it's description is the meta data, (i.e., the meta data for T, which *is* what it is, if I understand it correctly - otherwise you wouldn't have PhantomData, and just have PhantomData). I think your argument fails to reason about PhantomData as well, since it also suggests that _p is meta data, as it is used as a struct, or trait and not an attribute, which would be even clearer. Thus an attribute like #[CompilerMetaData] or #[PhantomData] would signal even clearer what it is meant for, or that it does something specific, which is not related to the final binary, it's in a sense, meta programming, because you are telling the compiler things.
@@simonfarre4907 I honestly think using attribute makes sense too. However, the reason I think PhantomData also works is that, to me, phantom means "not real". So, in my eyes, "_p: PhantomData" means "_p will appear (in the eyes of the compiler) to contain T, but in reality, it doesn't".
@@rintepis9290 Yeah, I suppose the argument against using attributes is also a good one; essentially when using a bunch of attributes, you introduce sort of a sub-language within the language (which, beyond macros would make it more than 1, that is), which I think is a strong argument against using it as an attribute, although, I still think PhantomData kind of trips people up, since it isn't really clear what "Phantom" is suppose to mean here. Even though I suppose, it could be interpreted as meaning, "not really T"
PhantomData is a sensible name. It accurately describes what it is - a data that's not actually there, but acts as if it was, for the sake of compiling. The only issue is, that it's not immediately obvious what purpose does it serve. When I first came across it in docs I was like "Why the fuck would I ever need this?" At the end of the day, the entire source code is just metadata about the machine code (the actual data) that is supposed to be produced by the compiler. In most languages this fact is less obvious, because the source code mostly just describes what the machine code should do. In Rust, the source code also largely describes how it can (and can't) be integrated into other source code. Other languages do this rather minimally, with basic stuff like method signatures and static typing.
It's easy to get rid of annoying rust-analyzer message. Download binary and add this to CocConfig ``` "rust-analyzer.serverPath": "/path/to/rust-analyzer-linux" ```
Since Jan 6, 2022, Empty now contains PhantomData T>
57:20 Current definition of std::iter::Empty is finally "pub struct Empty(marker::PhantomData T>);"
31:13 How can anyone not love this compiler? Look at that message! (both, the warning and the error)
I've found this Crust of Rust series after the fact. Just wanted to comment and thank you for taking the time to put these streams up to youtube. This is all very good information that is well delivered, and it is helping me greatly in my quest to learn rust; and indeed I think making me a better programmer overall.
I absolutely love the "nom nom nom nom" in the beginning XD
o
I love this Crust of Rust series, thank you so much for making this! I have a couple of questions about this one:
1. Why not explicitly declare that a type parameter is "dropped on Drop", instead of using PhantomData hack?
2. Why does the compiler require every generic parameter T to appear in the type definition? If it didn't we would get rid of the other reason for PhantomData.
This felt shorter than it was, great video, really enjoyed it! :-)
40:12 "Ahh this gets to work for silly other reasons" - probably the scariest thing to say to a programmer.
Yet another amazing stream! I really enjoy the content you produce @Jon Gjengset and learn a lot from it. Thank you!
(favourite moment 1:00:33 - 1:00:42 🤣 (would make for a hilarious meme))
Regarding the "unnecessary complexity", rust as a whole is "unnecessarily" complex. Rust is designed to restrict what code you can write, to prevent certain bugs and to semi-automate memory management. It does so by always erring on the side of caution. The complexity is there to communicate properties and behavior of your code, to the compiler. So that it knows when caution was already taken by the programmer.
I would argue that for exactly the reason you outline, it isn't unnecessary :)
@@jonhoo That's why I used the quote marks. It's unnecessary in a sense, that a programing language can function without it. It just wouldn't have the specific features Rust has.
@@KohuGaly I would use the term "added" complexity then; it's not unnecessary.
I think Empty(PhantomData) compiles because not implementing std::ops::Drop means that it does not take a &mut self (mutable reference to self) at the end of a block, so the compiler can know that it can take a immutable reference safely, yes?
Maybe the actual drop (not the side effects we write in std::ops::Drop::drop) would probably just consume self, not take a mutable reference?
If you write struct Unit(Oisann); without impl Drop, it will still implicitly drop all fields.
As of today TaPå is a valid type name! yay 1.53.0
I believe that if you use `PhantomData` you can maintain the const-compatibility, covariance, and `Send` as opposed to `*const T` or `fn() -> T`, while still letting the drop check know that you won’t be dropping an actual `T`.
1:17:42 Exploring the cxx crate can be very interesting for people who needs to integrate C++ code
A little late, but the definition for std::iter::Empty is (now)
`pub struct Empty(marker::PhantomData);`
So why isn't variance included as part of the type signature? I was under the impression that changing the private fields of a type does not count as a breaking change, but it seems like converting Boks from using a *mut to a NonNull (or vice versa) would fundamentally break a bunch of code, even though there was no direct change to type signature. I feel like the compiler is doing some magic by looking inside the Boks, something which I feel like it shouldn't be allowed to do, in the same way that its not allowed to look inside a function when that function is called (all the necessary information about the contents of a function should be encoded in its signature).
There are already other examples where changing private details of a type isn't backwards-compatible. Some examples:
- Adding a private field to a type that only has public fields.
- Adding a non-Send field to a type that was Send (without an explicit Send impl) - the same applies to Unpin and Sync
- (as you observed) Changing the variance of a type parameter away from covariant/contravariant - changing it _from_ invariant is fine
- Changing a type such that a type parameter is subject to the drop check where it wasn't before (the other way is fine).
- Implementing a pre-existing trait (such as one from std) for a pre-existing type.
In general, the backwards-compatibility guarantees for a type includes a bunch of private details, and it has never been true that _just_ the public-facing type signatures are sufficient.
Can you clarify why, "accessing the T" is not fine in the Drop, but "dropping it" is? I am trying to square this logically in my head, and no matter how I do, I can't understand it. It seems to me, that the "promise" or "guarantee" of not touching/accessing the T, is due to how Rust works, to keep aliasing rules and all that good stuff, but doesn't dropping something *essentially* "do" something to/with the T? I mean, once the value/data is dropped, it's gone, therefore any mutable reference to it, can't continue to exist, and since drop is called when it goes out of scope, nothing else (unless parallel from another thread or whatever) can access it at that point, at least not in sequential code. Which also means, that whatever access to the T there is, will always be the last thing that happens to it, inside of the drop function, thus no mutable reference can see that change anyhow. I know this is not how it works, I'm trying to make sense of the reasoning that's all.
Yeah, that confused me a lot in the beginning too. However, as you will see later in the video (with the Oisann example) that dropping T can, indeed, access T, if T has a implements Drop.
Nevertheless, the initial example with "Boks::ny(&mut y)" works, because T is actually &mut i32, which is a reference. When you drop a reference, it is a no-op - you are not dropping the value that got referenced to, because you are only dropping the reference itself. It is like doing this:
let mut a = 100;
let reference = &mut a;
*reference = 300;
drop(reference);
println!("{}", a);
(I am still new to Rust. If anyone spot an error in my explanation, please let me know.)
@@rintepis9290 Of course, I had glossed over that entirely that it was a reference, that obviously makes much more sense. I was thinking about it owning the T, not holding a reference to it
What are your vim plugins and configuration? It's really awesome! I am more familiar with Vscode but I would like to try it.
Thank you, very clear now
Hi Jon. I love your videos and they are really insightful. How can I support your work (financially?)?
Why is it only applicable if the struct is generic over T? Couldn't I still access the inner types in Drop even if the struct isn't generic?
Oh yeah all the references on a non-generic struct can only be 'static, which hence is not a problem.
Wait, the first example, before implementing the Drop trait, leaked memory, without a single line of unsafe? I thought rust protects you from those?
Ah, no, Rust does not protect you from _leaking_ memory. It protects you from memory unsoundness (like use-after-free), but leaking memory isn't really an error, it's just an efficiency problem.
+1 For Norwegian comments 😁🇳🇴
Is your vim / shell config available anywhere?
In th-cam.com/video/ycMiMDHopNc/w-d-xo.html he made a video about his setup. The video is from 2018, but it seems that most stuff is still valid.
Drop of a tuple struct is different from normal struct.
Why does "Drop Check" sound kinda brutal?
The drop check seems like an unnecessary thing to worry about if all the effort it prevents me from doing is inserting a drop(T) just before the use.
Skjønner ikke hva som står i sikkerhetskommentarene, pls halp!
Man I really got to try Rust, it has way too many benefits, the only bad thing I see so far is that the compiler it’s really strict but I guess that’s the price to pay for a safe non GC language, also I love the sintax it’s very similar to Kotlin for the most part
Same amount of time debugging, just in Rust you spend it fixing compiler errors instead of diagnosing segfaults. Once you finally fix all 893 compiler errors and it compiles, I have never had a program crash.
@@bananarobotoverlord I prefer fixing compile time errors rather than runtime errors, haven’t tried rust yet, but I will probably use it for a small game with the bevy engine
@@juanherrera9521 Agree 100%. Compiler errors are explanatory and didactic, while runtime errors are aloof and derisive. :)
I wonder if "MetaData" wouldn't have been better, than PhantomData as a name? It seems to me, that's exactly what it is, because it *isn't* data, but it is meta data. I suppose it could've been called "SpiritualData" 🤣 Or named it something even more concise, like "CompilerMetaTag" or "CompilerMetaData" or something like, to *really* hammer home what it's used for. Or am I misunderstanding the purpose of PhantomData?
From Wikipedia: Metadata is "data that provides information about other data". If you say "_p: MetaData", that implies "_p" is metadata. However, "_p" is definitely not data (because it is 0 sized), therefore not metadata.
@@rintepis9290 Which I suppose is why I provided the alternative "CompilerMetaTag" or "CompilerMetaData". And I don't see how _p is meta data, when the type itself in it's description is the meta data, (i.e., the meta data for T, which *is* what it is, if I understand it correctly - otherwise you wouldn't have PhantomData, and just have PhantomData). I think your argument fails to reason about PhantomData as well, since it also suggests that _p is meta data, as it is used as a struct, or trait and not an attribute, which would be even clearer. Thus an attribute like #[CompilerMetaData] or #[PhantomData] would signal even clearer what it is meant for, or that it does something specific, which is not related to the final binary, it's in a sense, meta programming, because you are telling the compiler things.
@@simonfarre4907 I honestly think using attribute makes sense too. However, the reason I think PhantomData also works is that, to me, phantom means "not real". So, in my eyes, "_p: PhantomData" means "_p will appear (in the eyes of the compiler) to contain T, but in reality, it doesn't".
@@rintepis9290 Yeah, I suppose the argument against using attributes is also a good one; essentially when using a bunch of attributes, you introduce sort of a sub-language within the language (which, beyond macros would make it more than 1, that is), which I think is a strong argument against using it as an attribute, although, I still think PhantomData kind of trips people up, since it isn't really clear what "Phantom" is suppose to mean here. Even though I suppose, it could be interpreted as meaning, "not really T"
PhantomData is a sensible name. It accurately describes what it is - a data that's not actually there, but acts as if it was, for the sake of compiling. The only issue is, that it's not immediately obvious what purpose does it serve. When I first came across it in docs I was like "Why the fuck would I ever need this?"
At the end of the day, the entire source code is just metadata about the machine code (the actual data) that is supposed to be produced by the compiler. In most languages this fact is less obvious, because the source code mostly just describes what the machine code should do. In Rust, the source code also largely describes how it can (and can't) be integrated into other source code. Other languages do this rather minimally, with basic stuff like method signatures and static typing.
nom nom nom nom
cat
It's easy to get rid of annoying rust-analyzer message. Download binary and add this to CocConfig ``` "rust-analyzer.serverPath": "/path/to/rust-analyzer-linux" ```
I don't want to do that though - normally I want my analyzer to be up to date, just not on stream 😅
@@jonhoo ah that makes sense.