Improve your Rust APIs with the type state pattern
ฝัง
- เผยแพร่เมื่อ 12 มิ.ย. 2024
- Today I'm teaching you how to build Rust APIs that are impossible to misuse by taking advantage of generics and zero-sized types!
FREE Rust Cheat Sheet: letsgetrusty.com/cheatsheet
Code: github.com/letsgetrusty/gener...
Chapters:
0:00 Intro
0:28 Example overview
2:30 First solution
4:20 Second solution
8:13 Third solution
14:18 Outro - วิทยาศาสตร์และเทคโนโลยี
![ช้างศึกสู้ต่อไป!! พาเปียโน ดูบอลไทยครั้งแรก ตกรอบแบบโคตรเสียดาย!! l [Nickynachat]](http://i.ytimg.com/vi/4WloIH645Lg/mqdefault.jpg)
📝Get your *FREE Rust cheat sheet* :
www.letsgetrusty.com/cheatsheet
Corrections:
10:00 - PhantomData isn't needed here. The Locked and Unlocked structs are already zero-sized types.
11:40 - The lock method should return PasswordManager
That was great, thank you! Amazing how it catches so many errors before even running; and, the hints are dynamic as well. Nicely demonstrated.
There is a bug in your final lock() method on lines 27 and 31, since it just returns an unlocked password manager. You should change the state to locked in both lines. Other than that minor issue, great vid.
I like the idea of using "types" as "markers" of sorts (regarding using generics on a struct just for differentiation). Better than using a field as that's runtime data.This was a great little video. Helped me see things a little different. A "rust patterns" mini-series would be good. Definitely some distinctions from other languages.
Lmao, this was done in OOP for past 50 years.
@@shinobuoshino5066What does OOP have to do with the topic of the video? This is about encapsulation verification (access control) at compile time via Rust's type system. How does OOP do the same via types?
You mistyped the lock method it still returns an unlocked password manager
Good catch, it should be locked.
@@letsgetrustyI suggest to always return Self to simplify refactoring and guard against such cases for the future. One can also construct and return a Self {}.
@@mihaigalos279 Isn't Self the wrong type? If you call unlock then Self is PasswordManager but you want to return a PasswordManager
@@mihaigalos279 can't use Self when changing the state since Self represent the type with the current state
Thank you, I thought I was going crazy.
Brilliant thank you! PyO3 (the rust crate for python extensions) uses this method to confirm the user has the GIL (Global interpreter Lock). At the time I gave up trying to understand the phantom data, but thanks to you I understand now! Now if only there was a way to make the macros that crate uses less mysterious...
Soon it might also be possible to define states in an enum using const generics. Currently this requires the unstable "adt_const_params" feature, otherwise you could only be allowed to use integer and char types which isn't really readable.
This would have the benefit that it's easier to see what states are possible currently there is no real connection between the Locked and Unlocked types and the PasswordManager other than the impl blocks.
This is the first time I understand PhantomData. Thanks!
Amazing video. Thank you! This solution is very elegant. Thank you for exploring the initial, non-optimal solutions first. It makes it easy to see the benefits of the final solution.
As an alternative implementation, we could have the unlock method return an owned token that is a required parameter for the list_passwords method. Upon locking, the token is consumed again by the PasswordManager.
This is definetly the proper way to solve this specific problem. Not only you get compile time checks for API usage, but your API is now closer to supporting being unlocked independently in two different functions.
@@GrzesiuG44 however, this requires allocating extra data, which isn't what the demo was trying to demonstrate.
@@tubebrocoli cannot this token be a zero-sized struct too? "struct Token;" If it can, how does passing a zero-sized struct as a function parameter affect the function's call stack?
The type state pattern, love it!
It’s a beautiful use of the type system. Definitely gives me something to think about.
Wow man. First 12 hours with Rust and it has already blown my mind so many times.
Recently I have came across a situation where I had to do something similar and this video immediately came to my mind. Nice work!
Fantastic. I love short, to the point tutorials like this.
This is awesome! Almost the same thing was implemented in ATS2 Postiats socket library, which is basically a header files for interop between "C". It uses something like the phantom types but with the refinement types. So not only you can describe the exact flow for you api, but also bind this flow with actual values you pass into functions. This is done via dependent type system
I'm pretty sure you don't have to specify the type of PhantomData since it would be done by the compiler, aside from that very small thing, this is one of your best videos so far. It's providing information on an intermidiate level while being explained very well!
Indeed, usage of `PhantomData` seems redundant since `State` is already a zero-sized type.
(BTW, there is another error: in the third solution, the method `lock()` should return a `PasswordManager`, not `PasswordManager`.)
@@julytikh Yeah, I also noticed both errors. But the video is still great!
I'm new to Rust, but I felt weird, when he said, that just using "State" would take up memory, since Locked amd Unlocked are zero sized types, just like PhantomData. I'm glad to see, that I wasn't mistaken.
What are idiomatic uses of PhantomData then? I know, it can "own" nonzero size types (e.g. PhantomData), but when is it useful compared to using just plain unit structs?
The only thing I can see, is if somebody by mistake created PasswordManager (which wouldn't be prevented by compiler), they'd be wasting some memory in case, where "state: State" ("state: i32": size is equal to size of i32), as opposed to "state: PhantomData" ("state: PhantomData": size is 0).
But there's a way to restrict the State generic to only include the right types, which would prevent you from creating idiotic types like PasswordManager, which would be even better, than using PhantomData to deal with it.
You can create a (possibly private) marker trait yourself, let's call LockState and have Locked and Unlocked implement it, and then restrict generic State: LockState. At that point there's no need to use PhantomData in this case, and it's absolutely impossible to create idiotic types (e.g. PasswordManager).
I guess PhantomData is useful in cases, where your intent is for your type to be able to be marked by any types user of your library wants (e.g. YourType, YourType, YourType). But I don't know of such scenario (because YourType would not actually contain values of those types, it's only marked by them).
@@tri99er_ One case where this comes up occasionally is when you need a type dependency for the working of your type (e.g. for a return type in one of its methods, for example) but the type isn't actually used in the struct itself. In that situation, you can use a PhantomData to satisfy the compiler that you are "using" the type in the struct so that you can use the parameter in your methods.
@@tri99er_ e.g. the implementation of 'dyn Trait'
That was super straightforward, thanks for sharing!
Your best video yet in my opinion, thanks for your work! :D
Beautiful! Thanks for sharing. Looking forward to more intermediate Rust contents like this. :)
Great video! I’m just starting to learn rust but everything was explained very clearly. Coming from typescript I’m very excited about the existence of PhantomData.
In some other languages like Haskell they call it the Indexed Monad pattern. Feels pretty similar to me.
The Rust library called Graph uses this in their builder pattern graph constructor, so it can infer at the type level if your graph is directed or undirected, and if the edges or nodes contains values.
Also, thanks for this. The non generic example was a nice touch. Should it be put in the lock impl, as it defaults to lock or is this better as it changes with the default state?
Indexed Monads are an extension of "plain"-er GADTs with DataKinds. Before those were usable, people used open phantom types. GADTs + DataKinds are more common than indexed monads because a lot of operations can be represented with simple non-monadic functions.
Nice, saw this once before in a "chaining builder" pattern that prevented setting the same property twice.
I personally would make UnlockedManager a newtype for &mut PasswordManager, that way, do don't need ownership and you get the re-lock for free when it drops.
Great! Great! Great! The best explanation of the Type State Pattern on TH-cam! You rock man! How can we support your work?
Would be interesting to expand on this further by implementing auto lock on drop, raii style.
Just make LockedPasswordManager a newtype for &mut PasswordManager. That way, it makes the locked manager avaliable after it gets dropped automatically thanks to Rust's ownership system.
u can just impl the Drop trait cant u
This video is hot sauce. I had seen Phantom data before, but did not fully comprehend it. Nice work Bogdan.
didnt know about zero types. really nice video. as always keep the great work up
Nice video, really useful, didn’t know about this pattern and state before. Cheers
I liked the general concepts of using types to represent the state.
But wouldn't one problem be that after locking the password manager one could still have a reference to the unlocked one. Since lock/unlock returns a new instance
I.e it's not like an FSM with one state but rather a struct where you can have multiple instances all with same content with different state.
That's the whole point: lock and unlock take their self argument not by reference but by ownership transfer (i.e., move). So the caller loses ownership, and there cannot be any references to the old instance or the whole thing does not typecheck.
Wow, that was genuinely an awesome vid, thank you!
Zero-size Types is new to me. I have learnt something. Thanks. I have used similar approaches in Scala / Java earlier, even though the effort was more. Moreover, in Akka-Typed, the approach is very similar, even though the implementation is cumbersome.
Thanks again, for uploading this.
That's the exact feature I needed the other day and I didn't know yet !
I learned a lot from this. Thank you!
how would using just state: State waste memory? Isn't it also 0-sized?
I'm also confused about this. I can certainly see PhantomData being useful if the struct holds some data, but in this case there really shouldn't be any additional memory used.
I checked it on goldbolt and both structures with :State and :PhantomData takes 72 bytes for me, so I guess it is optimized out in both variants
What he is saying is very confusing, if T is a ZST you don't need PhantomData to make it ZS because it already is, PhantomData exist primarly for lifetimes: for example there are structs that owns a pointer for optimizations, but still need to hold the lifetime of the backing data, so you add somewhere a PhantomData
It wouldn't, I made a mistake. Will point this out in the pinned comment.
this was a really good explanation!! thanks for this
I've been doing that instinctively for a long time now. But great example and explanation nontheless! The Rust specifics with the zero-size types was new and pretty informative!
This is so useful!
Thank you for sharing.
i like this kind of video. keep up the good work man
Nicely explained, thanks
I haven't seen this one before. Very cool!
I use this pattern for data sanitisation in my backend. Works great!
Always handy Bogdan appreciate the share!
Woow, that was very cool. Great teaching. Thanks
This been really useful. Thanks!
Beautiful. Will be looking for ways to use this, thanks!
you code your password vaults like this? 😱
This video is one of your best
Please use Self as a return from your methods, instead of rewriting all the time the struct name. It makes code reformatting wayyyy easier! :)
Why? If you rename the struct shouldn‘t the IDE rename that too?
lock() and unlock() do not return Self, they change the type
Interesting pattern! My default is to revert to state variables, usually enums.
I think this is maybe your best video
Very cool stuff man
This was very cool. Thanks !
thanks for this, I really wondered how typestate pattern works since it was said that it only works well with rust..
Enjoy this video so much!
Very interesting and illuminating ! A while back ago, I was heavily into generating code from UML state diagrams. Generated state machine code either would use a traditional state transition table or state classes. Now duplicated code isn't that much of a problem, if the code is generated anyway, however to avoid duplicatation I could also a use traditional inheritance. So "LockedPasswordManagerr" and "UnlockedPasswordManager" would both inherit from "PasswordManager" which would implement common code (here: "version()" and "encryption"). It never occurred to me, that I could do the same with generics.
There's no inheritance in Rust. And traits don't have access to implementor fields, so you'll end up with the same amount of duplication.
Loved this video! ❤
Thank you, now i know how to do something like inheritance in rust
I really like this kind of videos 🤛
Wtf. You can actually do this? So much potential! And head wacking because I need to make sense of the structure
Thank you, this can become handy
Till the end I have been waiting for raii cause I think it fits here well
Nice stuff, it'll help a lot
This was great!
Really cool! thanks Bogdan!
interesting, looks like a builder pattern, but returns an instance of same struct with just a state 0 size in memory and template reference thing. Great combo
This was relay helpful. I couldn't come up with such a smart system. However, you should have a State trait that is implemented for the two state types.
This was beautiful
great pattern. Man, I just love rust.
Awesome this is the idiomatic way of using Rust type system for states. It van be also extended to state machine or state chart.
Great video!
This looks amazing. But is only suitable for state that changes infrequently right? Whenever we transform from one state to another, we are creating a copy of the structs fields
In the last example, if you switch states, does rust have to move all the data to the new struct? So its basically allocating a whole new object when switching states?
Can you also create an impl block that defines methods for multiple states?
There might be methods that shouldn't exist in all states, but in some.
Does the compiler help make sure you handle all possible states for structs as it does in match statements?
This example is consuming previous state, which is fine. Can you make a proxy type holding reference instead?
it intentionally consumes self to prevent dangling references to previous states
I like TypeScript exactly because it provides similar functionality to implement this 👍
The problem I ran into is when I try to implement a trait for such state-parameterised struct, I either get "trait not implemented" or "duplicate definitions with the name " errors.
great video. thank you
The lock function returns an Unlocked PasswordManager, instead of a locked one.
Incredible!
Excellent. I really enjoyed learning this one.
What minimum version of Rust can you use this pattern on?
Looks like the PhantomData struct is since 1.0.0. I don’t see when default generics were added but there’s a spec for it from 2015.
Thanks 4 z great videos :)
thank you so much 🙏
It does not need both the default generic type and the last impl block. Just define `new()` in whatever state you want it, e.g. in the `impl PasswordManager` block.
In the third solution, you didn't update the lock() method to return a locked password manager. But it doesn't matter, you got the point across and I bet many people didn't even notice.
Nice video thanks
Since Locked and Unlocked are Zero-Sized Types (ZST's), why do we need to use PhantomData? What if our state needed to carry a little bit of information? (such as who unlocked it)
How did you get your "todo!()"s highlighted like that? Looks really helpful.
Seems like TODO Highlight vscode extension
The todo-tree VSCode extension + adding this line to settings.json:
"todo-tree.regex.regex": "(//|#|
@@letsgetrusty TY
Amazing !
How do you include another generic type with bounds? Say I want my password manager to also store another value, and I want that value to implement a list of traits. In each impl block I would have to specify that my generic type needs to implement the traits. How can I do this without repitition?
I think in the final solution, you can merge the first and 4th impl. PasswordManager and PasswordManager is the same
This is correct, but I think it's clearer, because we are not creating a locked manager, but just a manager(Even though it will be locked anyway)
Great video
why the lock method returns the state Unlocked? It should return the state Locked, right? Also, in rust should be possible to force the manager cannot be used after calling to lock or unlock? (to force do the switch of the variable?)
Given that you consume the original password manager, is it necessary to clone the members of the struct (password list and string)
How feasible would it be to do it the way Mutexes work, with the lock being held only until out of scope ?
this is what I thought the video would be about
You could normally do that by implementing drop on the unlocked password manager and calling the lock method, except his lock consumes self and you only have a &mut self when implementing drop so I'm not sure. I guess in a real use case, lock would have some other functionality that encrypts the passwords and writes it to a file for example, which would be available to call in the drop implementation, so you could do that at least.
This API prevents calling lock twice while for mutexes calling lock twice (usually from a different thread) is the point. I suppose you could create a Mutex API such that each client gets issued a reference wrapped in a way that prevents lock being callable the second time on the same reference.
Bogdan,
Rust neophyte here. I get confused sometimes when my variables get consumed when I don't expect them to be.
How do you know quickly when a variable is going to be consumed, either by using it as a function parameter, calling an implemented method on it, or even iterating over a vector? Do you just have to be aware of the exact syntax of every function call?
Also, in the Type-State pattern is there any way to avoid moving each member from one state of the type to the next? That could get tedious for a non-trivial number of states or members.
So how do we handle the case where the client gets the password wrong? The unlock method should either return an unlocked manager or a locked manager...or panic I guess?
Instead of zero sized structs you can use empty enums here. They are not only zero sized but actually not constructible.
This would solve the problem of user building the wrong state by itself
The issue is that you can't implement different functionality on a struct based on an enum variant. Therefore, we only know if the manager is locked/unlocked at runtime, resulting in panics or results like the first solution. As he showcased, generics allows multiple implementations. The only concern I have is restricting the available State variants, since his implementation allows arbitrary types to it. Perhaps having a sealed trait would solve this as a constraint => pub type PasswordManager {}
@@ErikCampobadal You didn't understand me correctly. I'm saying turn `struct Locked;` into `enum Locked {}`, an enum with 0 veriants which is a non-constructible type in Rust.
@@Artentus ahh interesting, indeed. This combined with a sealed trait could be a great solution
Would you say this is the better approach? Is it less common?
This perfectly explains Phantom Data types, I never really grokked this but this instantly made me understand and Now I can see how it can be used in other places! It kind of reminds me of how Two structs of the same Type but with different life-times are treated as two completely separated types so you can't return something with the wrong life-time.
Actually, `PhantomData` is redundant in this particular case. The types `Locked` and `Unlocked` are already zero-sized (because they have no fields), and `PhantomData` does not improve upon that.
@@julytikh can you explain how not to use the phantom data in this example
@@LukasCobblerxD just use `State` instead of `std::marker::PhantomData`.
Would an enum LockedStatus work well here too?
Best vid❤❤❤❤
This is great. Quick question (for anyone): why have an impl block with the generic state and then one for the type as a whole? E.g. why can’t the new method live in the generic impl block? Or vice versa - why can’t the shared methods live in the impl block for PasswordManager?
What's the advantage over just using an enumeration of the password manager?