Compiler-Driven Development in Rust
ฝัง
- เผยแพร่เมื่อ 27 ก.ย. 2024
- Compiler-Driven Development in Rust, including TDD, CDD, and the Typestate Pattern.
Today's sponsor: Quadratic, check them out at quadratichq.co...
When I write a new Rust program, I don't start with functions or methods or any runtime code. I start with the model of my application, expressed with Rust's rich type system.
Interacting with the real-world, on disk or through the network is too slow for me, at first, I need to iterate faster than that, to sketch out my ideas, unconstrained by the outside world.
This compile-checked whiteboarding I call Compiler-Driven Development.
❤️ If you would like to support what I do, I have set up a patreon here: / noboilerplate - Thank you!
📄 All my videos are built in compile-checked markdown, transcript sourcecode available here github.com/0at... this is also where you'll find links to everything mentioned.
🖊️ Corrections are in the pinned ERRATA comment.
🦀 Start your Rust journey here: • How to Learn Rust
👕 Bad shirts available here www.teepublic....
🙏🏻 CREDITS & PROMO
My name is Tris Oaten and I produce fast, technical videos.
Follow me here tech.lgbt/deck...
Website for the show: noboilerplate.org
Come chat to me on my discord server: / discord
If you like sci-fi, I also produce a hopepunk podcast narrated by a little AI, videos written in Rust! www.losttermin...
If urban fantasy is more your thing, I also produce a podcast of wonderful modern folktales www.modemprome...
👏🏻 Special thanks to my patreon sponsors:
- Jaycee,
- Gregory Taylor,
- Ything LLC
And to all my patrons!
ERRATA
- Check out today's sponsor:quadratichq.com/?
- Sorry about my voice in this video, I've delayed publishing for a month due to getting over Covid, and I simply can't wait any longer! I am totally fine (got all vaccines), but my voice is still croakier than I'd like!
- 'Geocache' is missing the final 'e' in example 1.
- 9:20 I LOVE Red Means Recording, Jeremy's videos have been an inspiration to me as a musician and video maker for years! In addition to the pinned video at this timecode, I made a few 'RMR'-style videos, my favourite of this is my cover of Love on a Real Train: th-cam.com/video/W1meeqQRR28/w-d-xo.html
- Do check the markdown source for all links, butthe typestate pattern article is here cliffle.com/blog/rust-typestate/
- 8:56 variants is misspelled (and likely in other places too, freakin' schwas ruining my day!)
Days since last errata: 0
"(got all vaccines)" hopefully you got super-autism now!!!
8:56 variants is misspelled
Nobody notices your voice is slightly different.
It's actually the other way around, autism causes vaccines!
Here's my video on autism btw: th-cam.com/video/DhY3fu-YgHU/w-d-xo.html
My headcannon is that every video on this channel are just snippets of Seth rambling about random topics to his friends.
Funny story, that's *ALSO* half of what Lost Terminal is!
(have you heard about this engine that burns Iron? Or this cool story from the early internet? or about this weird amateur radio trick? etc!)
@@NoBoilerplate Great show! I've been enjoying it! Just wanted to let you know that the recap section on the website is two seasons out of date (well, three if you want to recap season 16 before it's released ;D).
... And the shampoo bottles
In the alternate timeline, where Seth knows Rust
There's already rust easter eggs in Lost Terminal.... 🕵️
I've been subconsciously doing CDD the entire time I've beed doing Rust. Before learning Rust, I worked with Typescript wayyy too much and from there I subconsciously learned to let the type system handle a lot of runtime checks. In TS land, this doesn't help (hence my many frustrations with TS) because the types are literally a facade. Rust actually not letting you run a program until you fix ALL type issues beforehand and Rust having an extremely complex and easy to use type system is what makes CDD viable and enjoyable in it. I highly doubt CDD in many other compiled languages would be as fun and model-asserting as Rust.
RIGHT! Rust Doesn't have features that other languages dont, but it makes them coherant and MANDATORY!
i just picked up go and its definitely not
@@minnow1337 You can just as well choose JAVA or Kotlin or Swift. The main difference between those languages and C/C++/Rust is: you do not have to worry about memory management (with a Caveat when using languages like Swift or Objective-C which do reference counting instead of garbage collection). You can create data structures on the fly without feeling the cost to do so, but when measured vs. algorithms where developers do care about memory management (and Rust forces them to do so), you will find that your code performs worse by magnitudes: using more memory (by a lot) and more CPU time (by a lot). Which is even a cost factor when your code runs in the cloud.
What I love about Rust is the compiler gives so many errors that, once we are done fixing our mistakes, there's no room for bugs, unless our logic was wrong in the first place
if it compiels, it works!
My experience with Rust is that, if you spend time using the debugger at all, it is for finding flaws in your algorithm, whereas in C and C++ it is more often to find flaws in your data (i.e. memory) management or other pitfalls of the language (like why this goddammend template does not compile, but pesting me with totally inconclusive error messages instead?) . This not having to deal with the pitfalls of a programming language is what makes me fond of Rust, and enjoying to do programming in it as a hobby. I love things that get me focused to work on real problems, not tool issues. As I want to think on algorithms, not tools.
Every new pattern or feature of Rust I learn from these videos feels like a glimpse into the future of programming. This feels like what programming should feel like, and it's incredibly satisfying.
Thank you! This is why I can't *not* make these videos raving about this language. and it's #19th most popular in the world! YOU CAN GET PAID TO WRITE THIS!
You know there are lots of languages that already achieve the same? Rust is not discovering "CDD" and some could argue that is not the best at doing it too
@@jofla I honestly don't know of any. Do you have any examples?
I don't need Rust to be the best at it. I'd be writing Haskell or another ML language if I cared about 'best'. I want my language to be popular and powerful. Go is the former, Haskell is the latter, Rust is both. redmonk.com/rstephens/2024/03/08/top20-jan2024/
My assumption is that they assume compiled languages are all the same, which is reasonable if you've not tried an ML or Haskell style type system before!
loved the Red Means Recording reference!
Jeremy's a great guy!
This wasn’t a connection I was expecting, but I can’t say I’m disappointed! 😊
I mean, I'm much poorer (financially) because of how much I love his channel, but I'm rich in synthesisers 😀
@@NoBoilerplate haha, that's the spirit!
This further affirms my theory that every channel I watch on TH-cam is related in some way.
I should make a data visualization for that, now that I think about it.
Oh man you're finally back! Imma grab my popcorn and enjoy you tell us the best things about Rust 🍿
Thanks for being with us!
I'm sorry it's taken so long! RSI is a hell of a thing - I'm on the road to recovery finally :-)
@@NoBoilerplate Nah dw about it! Whether it's you taking a break or you gifting us a video, all of us are glad you're here either way 😁
Loved the video btw! the TypeState pattern literally had me going all 🤩
You are one of the fewest (if not any) people on TH-cam i watch immediately any new video i see. Thank you for doing good work and for the spark of the unusual your channel carry compared to the copy-paste, soulless corporate content so abundant on TH-cam.
Thank you so much, I really appreciate you saying so ☺️
"I want to roll for a new boilerplate upload.'
"Roll it."
"Nat 20."
🗿
I'm sorry it's taken so long! RSI is a hell of a thing - I'm on the road to recovery finally :-)
@@NoBoilerplate For better or worse, I stopped keeping up with code tubers since I have been on my own project (fantasy console) and writing for a novel. I have reached the point where my code feeds into my writing and vice versa lol.
Plus, glad to see you recovered
@@NoBoilerplate Hopefully you have a more ergonomic setup. Not sure about you, but switching to a split keyboard with a vertical mouse in the middle of the two halves helped me a lot when I began developing wrist pains. Oh, and good wrist rest pads too.
many thanks for the tips! my stupid self-imposed 16-hour days gave me RSI even using a split kb and mouse! oops! breaks saved me!
One thing to always keep in mind, and I'm guilty too sometimes cause I'm lazy, don't ever push your code assuming it's sound just because it compiles. Logic errors are always behind the corner and hiding under the carpet. This is why tests are also a good practice. For large projects it may be worth the effort to create a rich development environment with automatic testing tools specific to the job
Totally agree. For me, CDD is the whiteboard, TDD is the prototype, and functional testing gets you to production. But you must START with CDD 😀
@@NoBoilerplate Hmm I see, maybe I'm wrong but this video gave the impression CDD completely replaces the need for testing. I might be wrong! But then maybe testing is limited to logic errors and it's fully unnecessary to test anything else -- which is still amazing. Can you comment on this? 😄
That is why Rust makes writing tests so easy and executing them so accessible in all IDEs. Being a system programming language, where the ultimate goals are both correctness and performance, having a expressive test set is essential, as it allows you to verify whether your optimisations tried are sound or not in an instance. So I am more of a TDD guy. But I do not write any tests for a function until i believe it will work. Which it doesn't most of the time.... but fixing things in Rust is easy then, as every feedback of your toolchain is so conclusive.
"When you think about it, tests and types are similar" next step is having an embedded proof assistant DSL in Rust and it will unify the two :D
ugh, I'd LOVE that! Excitingly, it's absolutely possible to do so, someone just has to write a complex enough macro!
(deptypes has the right idea: lib.rs/crates/deptypes)
@@NoBoilerplate My reply got deleted (probably because of links) but there is also an interested project - based on a paper - called RefinedRust!
@@NoBoilerplate There is also prusti which has pretty good tooling (which is a problem I usually have with most for research software)
oh cool! What makes me excited is that whatever is built can be written as simple libraries using the macro system!
@@qexatKani would be quite close to that
Get better soon Triss! Your quality just keeps on being great, incredable work!
Thank you so much! Regular breaks are the core of my recovery!
I was wondering if you could make a video discussing how to shift from OOP-oriented thinking to whatever system works best with Rust. I've watched your "Rust data modeling with enums" (or whatever it's called) video, but I don't really think it shows what to use instead of OOP, rather it just shows how useful enums are. OOP structured programs have a graph-like structure, and it's pretty difficult to implement graph-like data structures in Rust. Glad you've had a smooth recovery from Covid, and excellent Rust video as usual
Re-watch that video, the part that allows the graph-like thinking you hope for is the normalisation section.
Think of it as tables in a database, not animal genetics.
It took me a long time to unlearn OOP, python helped me, Haskell was revolutionary, Rust finished my path ☺️
@@NoBoilerplate Alright. By the way, when I talk about "graph-like structures", I'm not referring to inheritance, I know that it's bad. Rather, I'm talking about the ability to have different objects directly referenced by various others as many times as you want. In Rust, you can only have one owner of a (mutable) reference normally to ensure program validity, but it's very limiting. I'm trying to program a transportation network where it would be nice to have one object be owned multiple times, but solutions like Rc make the program look ugly and much more complex. I assume Rust is purposely designed this way to discourage the patterns I was using.
To avoid this, I decided to assign items IDs and created connections between transportation sites using their IDs rather than direct references. Is this in line with what you were trying to communicate in your video?
Thanks
I think you're on the right path, here's how to fix your complexity issue - abstraction. Use Rc for the plumbing of your code, but abstract that complexity through layers of your code.
Imagine a web server like Actix or Rocket or Axum. You better believe they're using some GNARLY clever stuff behind the scenes, but I as the web developer never see that, because they've abstracted it away from me, as they damn well should.
unsafe abstracts pointer access, String abstracts bytes - to program is to compose layers of abstraction. Refactor with that in mind, and you'll find your solution. Same as in any language, I bet! Good luck - come ask for help in my discord if you need it, there are far smarter folks than I there ☺️
THE BROTHER DROPPEDDD!!!!!! 🐐🐐🐐🐐
Rumours of my death have been greatly exaggerated :-P
I love this. This is the truth that I’ve learned through intuition and experience, but that I never had words for it.
Thank you! It's so hard to put these kinds of ideas into simple terms, if I had more time, the video would have been shorter :-D
Gotta love it when you release a new banger
Thank you!
This channel is incredible. Thank you so much!
I feel like my rust code gets better every time I watch one of these
Thank you! Heh, my code gets better every time I make one of this videos!
I practice ADD Attention Driven Development. Basically I follow my fancy until I get to a state that does not motivate me anymore.
Once again great video.
Thank you so much! Also, you should check out my adhd video 😅
This might be the most complicated yet subtle No Boilerplate video Ive seen, or maybe I just havent had enough coffee yet. Definitely glad for the .md documents because I can tell pouring over those is going to be very useful
Love the video! I’ve always maintained that rustc is a greater asset to the language than any individual Option or #[proc_macro] could ever be. A mild nitpick is in the comment on the slide beginning at 8:41, you have varients (e), instead of the correctly spelled variants (a)
Thank you, added to the errata pinned comment
That type state pattern is interesting! cant wrap my heads around implementing it in real projects though, definitely need more examples
This isn't some novel concept, Scala devs have been doing this for ages. 'Builter pattern with phantom types' is a well-known blog post about it.
I completely agree with this. I practice and advocate for type-driven development with TypeScript on my team.
when I must code js, I write typescript - the correct default 👌
Who on earth would dislike this video?! C++ and Python developers worried about their career, maybe?
I imagine Python - the #1 language in the world - will be around for a while! Especially if they use github.com/PyO3/pyo3
@@NoBoilerplate I hadn't heard of pyo3 actually, thanks for the info!
I really don't get the rate. Python is a good enough interpreted language. Rust is a compiled language. They are totally different things. I like both. I will not be making my plots in Rust for a single use application. I don't need to fight with the compiler for plotting some data and formatting it well. Single use scripts are much better in Python. Large projects can become an unmaintainable mess. That is why Rust is better. Stop being monogamous with your programming languages.
@@vidal9747 Absolutely! The comment was a joke, but Python is definitely better for throwaway scripts (and I often use it for that).
15-years of writing python here - it's the most popular language in the world for a REASON! There's lots of applications I would not use it for any more, but there's still plenty I *would*!
As always I appreciate the new views you give on “other sides” of programming I dont usually encounter in daily work. However I seem to be missing the point where CDD is asserting actual behaviour and outcome of code, instead of testing wether or not your program compiles and handles every possible case (also a good thing, but different entirely).
Im sure if you want to *test and record the behaviour of your code*, you will have to end up writing tests still… or am I missing something?
Thx to anyone to answer or discuss. Cheers!
Quite right, CDD is the first step - quickly sketching out the core model of your code. One should RELUCTANTLY move on to TDD and so on only after exhausting CDD!
Although the compiler does a lot of checks in Rust, this does not mean that unit testing is completely meaningless. We still need to test the correctness of the logic and whether the results are consistent with expectations.
Absolutely right. I feel like the dream workflow is: CDD, TDD, Feature tests, E2E tests - KINDOF in that order!
I love how close you're getting to dependent types with the typestate pattern, you're so close :D
Imagine just lifting a `bool` to typelevel, instead of making the untyped On && Off structs (yes, you can make a trait to contain them, but that's kind of ugly, still working in the open set of types), just like you can with numbers (const generics mentioned)
"Compiler-checked whiteboarding" - thanks ill steal that
yoink! Just be sure to send folks my way 😉
@@NoBoilerplate but of course, also without stealing anything.. thanks for ur output :)
Pleasure, it's all public domain ☺️
Another one hit it out of the park. I would never have fallen in love with Rust if not for this channel. Please keep making videos, Tris! ❤
Thank you so much, that's really nice of you to say! There's not SO much left I want to say about Rust, but I'll certainly keep raving about features I love as I find them!
A stop sign is similar to a fire truck. Both made of metal, for safety and red in color.
what do you mean?
"So, at what time do you live? I'm alive past midnight in the clubs"
"I live at compile time."
"Oh? And how many did you 'compile'? Heheh"
"Well, there was this beauty last night..."
"And?"
"Well... when I compiled I got an error not found..."
You are just too good at talking. Love you.
Thank you! if you want HOURS MORE I'm half-way through season 16 of my audiofiction podcast, Lost Terminal:
th-cam.com/video/p3bDE9kszMc/w-d-xo.html
I'd love to know what you think!
yooo new boilerplate
:-D
No boilerplate
Consider compiler as a sort of assistant AI. Which never hallucinates, always correct, catches all of your mistakes, sometimes even suggests improvements.
rust and languages like it are so good to use with ai tools like copilot because of the extra context they give to the LLM, and the extra confidence they give to me in inspecting what my 1000 clueless interns have written for their group project today 😀
I’ve been skeptical of Rust for a while, as a C developer first and foremost, but I think I might try it. I fear it will demand I surrender more control than I am comfortable doing, but I hope it will optimize the code in ways any C compiler simply cannot.
Don't believe what people say, try it out for yourself. Here's how I recommend getting started: th-cam.com/video/2hXNd6x9sZs/w-d-xo.html
Thank you for all your videos man! You continuously inspire me to be a better developer. Can't thank you enough!
my pleasure!
12:00 I did something similar to this for Geometric Algebra.
If I multiply two versor types, the result is always a versor type. All versor types have the same implementations for all methods.
In this case, both versors and non-versor multivectors have the same methods, but some have different implementations.
I wasn't able to implement it in Rust, though. It requires compile time code execution and const generics.
The only language which is powerful enough is probably Scopes. Maybe C++ can do, too.
Rust *has* first-class compile-time execution, in the macro system, here's my video on it: th-cam.com/video/MWRPYBoCEaY/w-d-xo.html
And though I'm not familiar with exactly what you need, rust also has const generics for 7 years: practice.course.rs/generics-traits/const-generics.html
@@NoBoilerplate Rust macros are far from powerful enough.
I could technically create a macro which generates all types for N-Dimensional GA.
But then if multiple libraries use this macro, these libraries will be incompatible, since both generate their own multivector types.
Besides that, I have to choose a dimension in advance. The number of multivector types for each dimension N is at least 2^N.
It's technically possible, but it's not great.
What's great about scopes is that there are macros, which return types and memoize their results.
So if I have a multivector type as a macro (for example a quaternion MultiVector!), it would implicitly create the type definition only once at toplevel, and every usage of this macro will refer to the same type.
oh cool! First I heard of Scopes, very neat!
The behaviour you describe (creating a type at top-level then every usage of the macro re-using the top-level type) certainly could be written as a simple rust macro.
@@NoBoilerplate I'm pretty sure it's not possible. I would have to manually call some macro a toplevel to create the type first, and then maybe have another macro to use this type.
Really liked the light switch example, but could not find the link in the description. Perhaps you could add it?
I don't normally put links in the description (WAY to many! I put them in my original markdown script, linked on github). But you're right, I should make an exception for this one!
Haha I'm reading it right now and you're right, it's great!
this is kind of something i figured out on my own. in a text adventure io library i still haven’t finished, i had a Pen struct that handles printing text, which can be turned into a Prompt struct that handles reading commands and then turning that part of the text into just a summary of the command. i had not thought to use a generic parameter for this and i will look into that. maybe.
Games are FULL of state machines that would be amazing modelled using the typestate pattern - read the cliffle article I linked to to see all of it's power!
I actually use this in typescript. Types there also can be quite powerful if you know how to use them
Big fan of typescript, when I work in a js framework, this is what I use!
I understand the usefulness of this but I think a mix of both would be the best practice. Using cdd is good for the general structure of the program but for things that can't really be checked at compile time like some maths calculations or off by one errors might still need tdd. But I don't know if there is a way to do it with cdd I would be impressed
I too think that there's a clear workflow:
CDD, TDD, Functional Tests, Interactive Tests
pure code -> impure code.
CDD is the first step, for SURE. BTW, Rust can run arbitrary code at compile time, so you COULD if you want run unit-test-style code all ins your CDD workflow, check out my video on const: th-cam.com/video/voRBS0r4EyI/w-d-xo.html
TDD kinda reminds me of generative adversarial NN's, where you have a generator and a discriminator each having the goal of catching out the other, self adversarial programing if you will.
interesting!
Have you looked into inline const assertions, and what they mean for CDD?
Yeah, smells similar, right? I considered expanding CDD to include compile-time TDD using const, but that's just TDD running at compile-time, there's nothing super compelling there.
MODELLING your application using the type system is where the real power is!
Is there any significance to the roundabout in London the coordinates point to?
WELL SPOTTED! Trafalgar Square is often given as the centre of London, I suppose I pulled those from the london wiki page!
if you want the maximum help from the compiler, the coding style have to be changed too. For example, if there is a catch all arm in match enum like "_ => do_something()", then the compiler won't notify you if you add more variants to your enum later on.. So you need to be careful with those small pitfalls, same thing might happen with #[non_exhaustve] enum too
That's why I run clippy super strict so it keeps me honest! rust-lang.github.io/rust-clippy/master/index.html#/wildcard_enum_match_arm
Seeing 11:10, you might be interested in learning about the PhantomData marked in the std. If you only use the state for defining functions and you never read this value, making it a zero-sized type is an option (Maybe you're planning on making a billion lights, who knows!).
Quite right! PhantomData is mentioned in the linked Cliffle post - I stripped out as much as I could - traits, boxed state data, phantomdata etc - to make my demo as small as possible.
@@NoBoilerplate That's fair, a very informative video as always (:
Depending on if I’m writing the code for school/personal use that will never see a server/robot I’ll use CDD all day everyday.
If I’m getting paid for it, I’ll spend an hour writing a largely if a is true then test pass type test because it’s a requirement in the work place. I.e self evident tests
There's nothing worse than writing bullshit code, even if you're getting paid for it. You deserve more, annoy your colleagues with perfectly modelled code at work, too 😀
I think 11:30 should be enum On {} and enum Off {} with PhantomData as it does not make sense to have “an On”
All the code you see in my videos is stripped out and compiled as I write it, it's all valid! you can drop the {} in this case, which is super handy.
@@NoBoilerplate You misunderstood, I know you can create unit structs like that, but I am proposing zero-variant enums, Unit structs still have one value, the unit. But that doesn't make sense, you can't have "an on", so the type `On` should have zero values, as in it does not exist at runtime but only as a type, so it should be `enum On {}` instead of `struct On`
The difference being that you cannot construct an `enum On {}`, it cannot exist as a value, unlike a unit `struct On` which has one possible value being `On`
oh got it! yes that's a very semantically satisfying improvement 😀
@@NoBoilerplate Indeed it is :D, changed my life when I found that zero variant enums can be used like this.
very intriguing topic, maybe i'll try to incorporate it into my programming. in a serious production environment though, having tests, especially integration tests, is invaluable and the final sanity check between you and the client. one big advantage of traditional tests is that they are maintained separately from your developed program. in CDD, any refactoring work on the types or function signatures would leave me wondering if I haven't unwittingly created holes undetectable by the compiler's static analysis, right? i think it's a nice addition to existing good practices. leveraging the compiler to reduce the number of errors during development. but without tests i would feel my code flimsy and naked.
Oh of course, I agree. CDD is the first step, but once you've got the pure model of the code correct, developing runtime features using TDD, then putting into place feature tests and so on is of course essential.
"If it compiles, it works". Great quote! This has been my feeling as well, and it's awesome. the The light switch example in the end though... Not so convincing? What good does that really do? Maybe it was just a poor example?
Quite possibly, It's the absolute smallest example I could show. Maybe I should have gone with the classic traffic light? DO read the linked cliffle article, it's so powerful!
Thanks!
My pleasure!
To me TDD was less about shipping correct code and more about designing code that's ergonomic/easy to use/test and to show that the code is able to do what the (actual) user wants it to do. Having the tests around is just a nice side-effect.
Nevertheless, this looks very interesting. And great video as always :)
"what the actual user wants" I think you're talking about BDD tests there, larger tests that are based on user stories?
TDD is quite a pure, simple theory, at its heart is just "how to code without fooling yourself!".
@NoBoilerplate Well, user of the code or "actual" user are both very broad, and I meant it that way. User of the code is probably another dev or yourself in the future. TDD makes you write ergonomic interfaces because you'd be insane to think up a bad complicated interface for code that doesn't exist. You state what you want to do. And TDD tests can vary wildy in scope, depending on what you're working on. You might call them by different names like integration, acceptance or behaviour tests, or whatever, but at the core, you wrote a test first, made it pass and hopefully didn't forget to refactor, which counts as TDD in my book. If you enter the testcase from a function call or a call to a webapi for which you are writing a new endpoint doesn't matter, really. In fact, the latter case can be better because it's less coupled to the code you are/will be testing, which is super important to look out for.
I like the Lights example: it's code that doesn't produce any runtime. All structs are empty. Indeed, empty structs may help you to create semantic-only types that have no runtime effect, and only serve to guide the compiler.
I guess I do "compiler driven refactoring" when I'm writing some code where I think 'Hey...if I were to call that function now...things would crash. How can I protect myself from that?"
I will then refactor so the compiler no longer allows me to do this or that.
I guess this is the opposite of good old win32 functions where the documentations says: "This parameter is reserved. If you pass anything other than NULL, the function call will fail."
Oh, the cliffle article I linked to has a few options for storing other information in the state - I just kept it very tiny for my example. Do read it!
At work I very seldom actually run the services I'm writing locally. Just as you say: I write, I run clippy, then I push and let CI take care of the rest.
The dream! CDD first, then TDD, and finally some functional tests for production!
running the rust compiler takes same amount of time as running test suite for other platforms with time to get a snack 🤣
i mean, the first time sure. but the build caching is pretty damn good. so unless you build in release mode while testing - dont - its pretty fast the second, third, etc. time you compile some iterative changes.
Don't repeat stuff you've heard other people say - give it a try! Wouldn't it be amazing if what I'm saying was true? 😉 th-cam.com/video/2hXNd6x9sZs/w-d-xo.html
I was just messing, I like rust, zig, and c
Find someone to love you like this guy loves Rust.
I actually love loads of other things more than Rust - but Rust allows me to FINISH my code and go and go live my life 😀
Maybe I just need a better example to help me visualize generically how this could be applied because that link which talks about HTTP requests seems convoluted and full of too much boilerplate, which kind of seems like a natural negative consequence of how Rust implements enums. For instance, if I were implementing a mechanism to handle requests, I'd put everything in that one class, and there would be no need to return different structs or whatever.
This also kind of strikes me as one of the quirky flaws of method chaining because then you have to return different types to allow the chain to continue. Of course, since we're talking about HTTP requests, I'd probably just fill in default details and have functions to let the user replace the defaults or not if they shouldn't care. Certain things would obviously be an error, like a lack of target IP and so on, but there are things which the user might not want to care about, or shouldn't have to manually handle.
For instance, one of the things about the example that strikes me as wrong is the status codes being passed as magic numbers/strings instead of having a set of enumerated names so the user doesn't need to know the actual status code, such as just STATUS_OK instead of passing (200, "OK"). And on top of that the content length being set by the user passing anything at all. That's one of those things that would never be explicit in such a library were I writing it because the user could pass the wrong value and that's a huge flaw, where instead the library should calculate it for you and set it automatically.
The two examples he gives of where such things go wrong in C++ I agree with, but they're also kind of irrelevant. Sure, move semantics should probably be implemented better and more at the compiler level, but things like closing files really is better with RAII and it's just a matter of slightly modifying those classes. Of course, I don't agree that Rust actually makes these things easier or ergonomic as that code looks disgusting and makes me want to barf worse than reading STL code in C++.
It's been too long since you uploaded a Rust video! Ended up using Go
That sounds like a "you" problem ;-)
I really like Go too! If Rust didn't exist, I'd be happily using it. Many similar good decisions over there. And, of course, quite a few ones I don't agree with.
"The rust compiler is faster than a REPL" XD XD
To be clear, I said, "The compiler is faster than all these methods, but to be fair, and balanced, it's because it's doing less."
I've written production lisp at a bank, I LOVE repl-driven development, but compiler-driven is SO much faster!
@@NoBoilerplate I'm not sure why you'd believe that given how slow Rust compiles, even when working incrementally. Every REPL that I know of can use the full width and breadth of the language, even down to importing modules, and thus iteration is nearly instantaneous because it's a simple matter of :w from the other window and the up arrow key followed by enter to rerun the import line again.
Awesome that your videos are public domain now as well! Have you considered posting them to PeerTube and/or Odysee as well?
Odysee is a non-starter for me, as is anything blockchain. PeerTube is a real possibility - activitypub is a great option, and I've been delighted with Mastodon!
(Please excuse me, I must point out that everyone on my patreon and ko-fi can watch ad-free and without (google's) tracking.)
I love the idea of the TypeState pattern. I love compile-time checks and guarantees in general. Faster feedback and everything else. But I have some hesitance about using it, after having tried using it. I've been working a pretty large personal rust project (a game-adjacent app) for a while, and it had a lot of these kinds of invariants that on the surface I would have loved to encode in types. The problem is, as far as I've seen working on my project on and off for the past 4 years, that at some point you need _so_ many generics and types that not only do you run out of names for everything, more importantly, your code just ends up an unwieldy mess of deeply nested types. I ended up having to refactor my code to just check a lot of invariants on start up rather than have everything in the type system. What do you (or anyone reading this) think about how TypeState scales up with bigger projects? Genuinely curious what people's experiences are using this "in the wild".
BTW long time lurker. Love your videos. The videos are just so stylish and every time I get exposed to new ideas or less talked about ideas that makes me re-evaluate how I code. Keep up the great work!
Thank you so much, that's very kind of you to say!
Naming, program organisation, namespaces - Rust has the same design challenges as any other language, it's the hardest part, honestly! Come ask that question on my discord - there's genius overe there!
@@NoBoilerplate I just might. And you're very much welcome :>
0:44 Some more news reference?
I have a feeling that this guy is a fan of rust.
Garbage game for garbage people
@@NoBoilerplate game? I mean the programming language.
red means recording 😌
LOVE Jeremy (though my bank account doesn't!)
There was a lot to unpack here.
1) After years of TDD all I want to do is to find some weakness in your arguments. For instance, let say we have a "User" object. A user can have a name which is a String. Without tests a user with an empty string as a username. I do not think there is a way to enforce non empty strings as usernames via compiler. Or is there a way to do it?
2) Based on the previous example, when you are describing the CDD you are not saying that we should completely abandon unit tests, right?
3) It feels like in reality CDD could become an excuse for developers not to do unit tests. And their way of CDD will not be the same as you described. It will be much less sophisticated. Because nobody will bother to understand the concept. At work I encounter such excuses regarding TDD every single day. "There is no way my code can produce bugs because it is not complicated", "with our architecture there is no reason to test", "unit tests are a waste of time".
4) Feedback: your example with on/off light switch was honestly amazing. This is the only thing that allowed me to at least somewhat understand your idea. Enforcing business rules via types was very cool. But I wish you gave us more examples. Because the example with IDs did not allow me to understand the concept. Because of the problems I described in my first point. Because junior developers can always mess up assignment of values. A string can be an empty string by mistake. A number can be always zero.
Also, when I said that it would be useful to have more real life examples is because when I heard you talked about type state pattern it felt like you are saying "with fire you can cook a tasty soup", but you only showed us how to boil water. It was useful, but I feel like it will take me months of doing my own research before I will be able to apply your ideas about CDD
You can enforce specific invariants, like non-empty Strings or even clamping values between min and max, with a wrapping type around the original one. I recommend looking into Rusts NewTypes.
You've got it!
Paley, you're gonna really love Rust, I promise you. Let me prefix this all with my video on how I recommend you learn quick: th-cam.com/video/2hXNd6x9sZs/w-d-xo.html
I'll answer your points one by one:
1) The reason you don't think this is possible is you're not as familiar with statically-typed languages as dynamicly-typed, would be my guess? So there are 5 ways of doing this that come to mind:
1. Do runtime validation. `if name.len() > 0` - gross.
2. Use formal methods that use perfect modelling to prove that there are no execution paths where name is not an empty string. - esoteric and impractical.
3. Dependant types. Some languages can natively constrain types so that a string must be (say) between 1-128 characters long.
4. SOME languages (such as Rust) can use compile-time magic to do all of the above. But you're gonna be doing some magic, and most of the libraries are still alpha. (for example lib.rs/crates/deptypes%29
5. Use the "Parse don't validate" pattern (lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ don't be afraid of the haskell syntax in that article, it's simple and you'll get it even without knowing haskell. But do learn it!)
5 is the way most languages solve this. TLDR: 1. parse your input into a type that has no other ways of creating it other than being parsed. 2. use this type everywhere and you can safely assume it's got valid data inside. This is how Rust's strings are guarenteed to only contain valid utf-8 code points, there is no way of making them in the standard library that doesn't get parsed and valided at runtime, so you can assume the compile-time type is always valid.
2) A mercifully short answer: Yes, CDD just comes first. TDD later, normal testing after.
3) Don't let the bastards grind you down, my friend.
4) My Light example was stripped down to the very bones (to the point that rust developers are in my comments tellling me off!) To get what you desire, just read the linked Cliffle article, the final example he shows, the trait-based typestate example is incredible, and has everything you desire :-)
@@NoBoilerplate Thanks a lot for your response! Sorry for the late reply, I got a bit sidetracked and forgot to get back to you. I really appreciate the advice, especially the part about not letting bad developers bring me down-that was spot on. I’ve already checked out the resources you mentioned, and they were really helpful. Thanks again for taking the time to answer my questions!
fn five() -> i32 {
6
}
Ah yes, the perfect implementation with nothing left to be desired
so perfect!
remember rustaceans, if statements are ugly. match for the win!
yeah! ugly and less semantic!
The typestate pattern looks just like the bloc pattern's state definitions lol
ew, gross. Sorry reading the flutter docs made my eyes bleed.
I THINK the main difference is all states are proven by rust at compile time, and no unit testing is required because the compiler proves that there are no possible invalid states in your whole codebase.
I've missed your videos
Thank you! I'm so glad to be back making videos again ☺️
CDD + TypeScript => Copium-Driven Development?
Use zig!
I love this!!! P.S. also very vaccinated and very COVID'inated over here.
Thank you! get well soon!
Thanks! But yeah. I've described this as "locking in" logical pathways, so that you can make a determination and conversion in one place which truncates the semantic space which is even possible to describe at later points. This approach is literally part of how I sleep at night!
i swear there is a part of me, just wants to learn to code so i could understand develepors who i've subscribe to for their non coding related post.
why not? start somewhere, build things, don't get stuck in tutorial hell by actually expanding on the tutorials contents, there are tons of free resources online!
Do it! It's so easy to start, here's my video on how to th-cam.com/video/2hXNd6x9sZs/w-d-xo.html
Something I don't understand is why not use a language such as mojo which lets you keep a comfortably pythonic syntax?
Because popularity counts. I didn't talk about Rust before 2022 not because I didn't know about it, but because all new language have infinite potential when no-one's using them. Rust matters because it's in the top 20: redmonk.com/sogrady/2024/03/08/language-rankings-1-24/
@@NoBoilerplate I agree that popularity counts but in the specific case of mojo (and perhaps tons more) you do have compactibility with every python libary so it doesnt matter much, its like typescript: an optional superset that doesn't break anything.
i know this is an unpopular opinion, but my answer is because i *really* hate python's syntax
I've now reminded myself what mojo is. "python but native speed" isn't a feature for me. My whole career has been in python. 15 years. And I'm sick of it.
Python's core compromise of a GC, coupled with the GIL, whitespace syntax, are no longer things I value.
In short, the above core features make easy things easier, but hard things harder. I am no longer impressed by how small a hello world is, or how few characters I have to type to do simple things. Here's my video on why Rust is not a faster horse
th-cam.com/video/4YU_r70yGjQ/w-d-xo.html
@@NoBoilerplate
tbh whitespace syntax is one of the biggest "it doesn't technically matter all that much but I hate it anyways" feature of basically any programming language. any language with it loses a lot of points in my mind
Here is one thing where I think you are wrong: "The compiler is faster than all these methods". No it isn't and especially not the rust compiler. Real REPL-driven development has feedback cycles of less than a second. There is the 2 second rule for websites which I'll restate as "If it takes more than 2 seconds, it becomes annoying". Your short term memory lasts about 10 and up to 30 seconds but compilers even routinely break the upper limit of that. The compiler might be more efficient in terms of wasted energy, but for all practical purposes it is a very slow way to get feedback.
unless you do either enormous changes without compiling, or compile in release mode, its usually a few seconds max even with a lot of dependencies. the build caching is pretty great.
the real issue with REPL-driven development or TDD starts to appear once your applications start getting more complex and interconnected. Here CDD in Rust shines because it works on complex and interconnected applications the same way it works on simple problems.
@laundmo Yes, there are a lot of things that help. But still, a few seconds is enough to interrupt your reasoning because your short term memory degrades quickly. Compiler just aren't interactive.
At this point I'd like to highly recommend this talk: "Stop Writing Dead Programs" by Jack Rusher.
From my experience CDD, RDD and TDD all degrade with complexity. The bottleneck is always our ability to reason things out. With TDD we also constantly need to make context switches between test and program. With CDD we need to reason about types. And with RDD the issue is managing the "state" of the REPL. But all three also give something in return. None of them scale perfectly but CDD and RDD scale pretty well with complexity. That's one reason why managing complexity is so important (see "Simple Made Easy" by Rich Hickey). Functional and procedural code usually helps, class-based-OO usually hurts but types *can* also add a lot of complexity.
I like CDD and I like RDD. I have had very bad experiences with CDD (namely C, C++, Haskell) but also positive experiences (namely Rust). I've also had bad experiences with RDD (namely python, javascript, Haskell) but also great experiences (clojure, javascript/svelte). But I strongly disagree that CDD is always or even usually more effective than RDD, it always depends.
Very good thread here, thanks both. I have nothing but respect for RDD, which is why I brought it up in the video.
I'm very familiar with both Rusher and Hickey's work: The first video on this channel is a lightning talk about Clojure - a language I was paid to write at a startup bank about a decade ago, and one that I still have great fondness for.
I specifically said that CDD is faster than RDD, not better. My workflow is to start with modelling in the type system, then moving on to TDD and then more interactive testing.
My ideal language would support RDD and CDD, however, lisps seem to be allergic to a strong type system. Typed Clojure (like python's mypy, and even typescript) is a seldom-used add-on.
In Lisps, I cannot start with CDD, and in Rust, I cannot progress to RDD. HOWEVER, the deep runtime knowledge that the Rust type systems has makes many of the cases where I used to reach for RDD obsolete, I simply ask my ide about what this var or that var is going to be at runtime, and the rich type system constrains the runtime type SO MUCH that I'm good.
If it compiles, it works!
People complain rust is slow to compile but forget or doesn't know what its doing behind the scene
people who say that just repeat stuff they've heard, they've not actually compiled any rust 🤣
@@NoBoilerplate With C or Jai like compile times, CDD would actually be an enjoyable experience. Currently I'm using C++ for game dev and I don't want to endure compile times that are somehow even slower than Rust's.
I've tried compiling ggez and bevy and noticed with my own set of eyes that they are way slower in compilation speed
(and are also harder to use) than something like Raylib.
I can see Rust being an ideal language for general purpose and server development, but not for the domain I'm interested in.
"don't just assume what other people say" is not an argument and just ignores one of the few complaints I have about Rust.
RED MEANS RECORDING
LOVE Jeremy!
yey more rust
this is the way
Look up "Type Driven Development" ;-)
LOVE idris. I feel like Rust is an idris I can get paid to write 😀
God this is so hot.
🔥
🤩
Was almost convinced this man is dead
Reports of my death have been greatly exaggerated!
Covid + RSI suck. If you're ever worried, check my weekly scifi show lostterminal.com/
🤶
@@NoBoilerplate ah, being here in east Africa, i think of covid like something that came and went for about half a year
Hope you are doing well now 🙌
Thank you!
Uh, you realise C and C++ had CDT from the get go right?
C and cpp don't have all the pieces. You must have:
1. an elaborate ML-style type system to go along with
2. an advanced compiler, and - the most important reason why I am excited about Rust -
3. no possibility of disabling or not turning on these excellent features - they must be on and mandatory by default.
Rust doesn't do anything other languages can't. It's put together the best features of other languages and made them default. This is the special sauce. I explain more of this in a few of my videos, principly this one th-cam.com/video/v6RxJsk8otY/w-d-xo.html if you're interested! :-)
@@NoBoilerplate
#if !defined( __STDC_VERSION__ ) || __STDC_VERSION__ < 202300L
You don't need an ML-style type system, you just need a decent library. I'm working on the library part where the ABI can be upgraded without breaking older apps. It will have a launcher to go with it and will abuse the PATH variable along with makefiles and static libraries &/or gcc objects.
There will be no *.exe, *.elf etc in that devs need to build. They'll just build the objects and create a makeifle that uses the presented variables like MAIN_O and APP_EXT to link up to a native object provided by the launcher to produce the native binaries which it can then actually launch. An example command would look like this:
paw1 --start "/ "
Which will result in something like ~/.paw/paw1/all_/bin being prefixed to the PATH variable and then ~/.paw/paw1/all_/bin// being build from the objects (if it's not already) and launched. The native objects would reside in ~/.paw/paw1/_/bin/paw/
Simply put only the inattentive use rust, any other will see it's flaws and not touch it with a 10ft barge pole. C is the only language suitable for designing base level interfaces. C++ etc are supposed to be used as wrappers, not the baseline.
I hope it is a joke, especially with c...CDD with UB camping under every stone ? :D, Did you mean CoreDumpedDriven or DebuggerDriven or ValgrindSanitizerDriven ;)
C++ may give some HopeDrivenDevelopment if you stick to GuidelinesDrivenDevelopment ;)
Or do you meant that every compiled language has some kind of CDD ?
I heard Elm has it too, it transpiles to JavaScript but if it compiles it runs without errors.
Gleam is going this way to, but i think it is not battle tested yet.
@@AK-vx4dy Any type based language like C is CDT based. It don't compile then it failed the test. If you mean testing the code at runtime, then you're naive if you think it's even possible to prevent errors from ever happening. Corruption happens not just because of poorly done code, but also because the real world exists. Take the terminal fiasco that happened here in britain recently. The devs naively thought they could rule out every scenario and then reality took a giant dump on their code. The only option is to assume errors will happen and find ways to gracefully fail, not pretend they didn't happen.
@@zxuiji C mereley allows to define types, checking by compiler is weak (wich allows some crazy fast almost assembly like tricks but comes with a price).
Diffence is that in Rust you can forbid some
or better to say build strict guidelines for self and next programmers, in C you can bail out quickly, sometimes by accident, without a warning.
Problems caused by real world are from other category, we didn't talk about this kind here and we don't pretend anything ;)
With Elm example it was about that compiler has such strict control (like Rust) that any JavaScript quirks don't blow up at runtime.
Gleam natievely works on BEAM (VM wich drives Erlang) wich is build on gracefull fail philosophy.
Terminal fiasco has nothing to do with programmers or language, it it/was architectural flawa and not even an error or failure.. just stupidity to put hard quota on goverment account by BT and/or reusing the same account for diffrent pruproses...
bro finally uploaded, and within just two monthly polyphasic sleep cycles
RSI + Covid are a hell of a thing to get at once! Both getting better :-D
@@NoBoilerplate no way. you got covid after all the covid hype ended?
@@RenderingUser Looking the other way doesn't magically make the virus disappear.
(That's kinda why i still put precaution till this day)
@@RenderingUser Yeah unfortunately the actual virus is unaware of its hype status and continues to do its thing despite how hard we're trying to pretend it isn't.
The good news, if my personal experience is anything to go by, the trend for plague viruses to get less deadly and more contagious certainly seems to be the case with the variant I got. I almost didn't notice it for the last 3 weeks, unless I did anything strenuous on that day (then I'd be strangely tired)
This is such an amazing pattern I stumbled into last week. When refactoring Rust code, like changing the return type or number of arguments to a function, the compiler immediately tells you the exact location of each breaking change you need to fix. It is awesome 🤩
The compiler is so helpful!
the typestate pattern is really really cool and I want to see it being used more, I wish the language had a more streamlined way to write those tho
There's always way to improve things, certainly, but it's the best language for concisely expressing the typestate pattern that I have seen!
I could see having a derive macro that would automatically generate the states. You would define all the states and content using an enum, and it would generate from it.
For the impls I don't know enough to tell how, but a simple attribute above a function to filter only some states could be great.
Yeah, a simple macro would do it. That'd be cool. My Light example is the absolute simplest I could boil it down to - using it with traits (as the Cliffle article shows) is the best way to do it, and there's a bit more boilerplate there.
some work has begun already lib.rs/crates/typestate
How about Haskell, though? What I love about Haskell is that the compiler can actually ensure that the only valid type parameter you can apply to Light are On and Off. The equivalent of Light would be a compiler error.
@@kintrix007 In that case, you can't actually *make* Light because the new() method is only defined for Light
I mean, you _can_ declare a variable/field of having that type. But you can't actually assign anything to it, making it practically useless.
Rust's compiler is honestly just so great. I really hate the move towards dynamically-typed and interpreted languages like Python - you sacrifice so much and gain so little.
Keep up the excellent videos!
There's a killer feature for interpreted language - state preservation between code changes. The browser and how you use python for ML are two HUGE applications. But yes, if you don't need a long-running interpreter, it's time for Rust!
you gain so little? meh I think then you dont actually what you are talking about.
different languages are best for different applications.
python is the second best for all applications ;)
@@red_roy What's n°1? Haskell?
@@red_roy you just contradicted yourself with your second sentence.
Awesome video! Haskell's abd Rust's amazing type systems have made CDD the norm for me too. It feels awesome!
SUCH a comfortable way of living!
About the TypeState pattern. What happens if you send a light as parameter with Light and you want to call `flip` on it. It won't know which `flip` it should call. I wish you made an entire video about this pattern. Seems great :D
OH I COULD HAVE MADE A WHOLE VIDEO ON IT! It's my fav pattern!
I believe the answer is to use Rust's generics - this is what the trait system is for - many standard library functions can accept any kind of string (not just String or str or osstring etc) by specifying traits that parameters should adhere to, rather than concrete types.
This is just OOP. This is done in every language with a type system.
Not quite. There are type systems and there are type systems.
Certainly, you can do the basics of the typestate pattern in any language, and it's a good idea. But there are powerful parts of the ML-inspired Rust type system that lend themselves to compiler-driven development.
Here's one of my videos talking about some of the magic that the Rust compiler can do, I'd love to know what you think: th-cam.com/video/voRBS0r4EyI/w-d-xo.html
NEW RUST VIDEO YEAH! I love rust sooo much, thanks for all the brilliant videos :3
Thank you! I'm a big fan too! If you can believe it :-D
Are you paying a legacy to Eminem or am I playing this at 2x speed?
The reason I produce "Fast, technical videos" is that for accessibility, I don't put animations, music, sound effects, memes or other typical stuff in my scripts. So the only way to hold the audience's attention is to have a TIGHT, fast script. The pause button is there if you need it - and you may read all my scripts on github in the link in the description! :-)
One of the very few TH-camrs I watch/read at normal speed... And absolutely the only one I'll ever drop into reduced speed for...
The style of presentation encourages pausing and thinking.
Thx for this.
I'm delighted, thank you! ☺️
you have single handily convinced me to try and learn rust
let us know how it went
this is the way!
MISSION ACCOMPLISHED!
So excited for you, here's my video on how I recommend learning it: th-cam.com/video/2hXNd6x9sZs/w-d-xo.html
love the take, thx... yeh if it compiles pretty sure it works! (rust reminds me a lot of scala, without the JVM! - which is a good thing)
Ah Scala, my first love. Can you believe they dropped XML literals from the base install? Short sighted that is. Rust has that feature as a macro! Love that.
I talk about rust vs scala in this video, btw: th-cam.com/video/voRBS0r4EyI/w-d-xo.html
If the burden of writing tests is replaced with the burden of writing types, i think it's an improvement, but i don't know why
Red does indeed mean recording.
I don't like writing unit tests, because they go out of date, models change we needed to get 100% coverage for the build pipeline, so it was some stupid tests that didn't test what they needed to.
But man, I love the compiler and errors in there, I am all for CDD
I'm so happy to have a new video from you, my RSS feed needed the update
Woah, you have RSS for no boilerplate uploads? How would I set that up? Thanks :)
@@half_invisible I have a Nextcloud instance hosted on my HomeServer and the included RSS app can just take the link to a YT channel :)
I believe most RSS aggregator are able to do it
I use miniflux for that! RSS is truly wonderful.
Aaron Schwartz left us a wonderful gift and did not deserve what happened to him 💔
most rss readers can subscribe to youtube channels, RSS rules!
I really enjoy your videos, because of you i dipped my toe into rust between semesters last spring. I should have more free time soon to properly make something with rust.
So exciting! Make sure to run `bacon clippy` while you code 😀
Bravo. Yet another fantastic video on Rust! I really wish I could use the typestate pattern as elegantly in other languages as well.
Thank you very much!
Certainly, you can still use the pattern, not as elegantly at all :(