Rust's Witchcraft
ฝัง
- เผยแพร่เมื่อ 26 ม.ค. 2023
- You and I would not be here today if it weren't for my favourite feature of rust, and indeed of any language: Macros.
I've mentioned them in the past, but in today's video I'm explain why they're SO POWERFUL.
❤️ If you would like to support what I do, I have set up a patreon here: / noboilerplate - Thank you!
📄 All my videos are built in compile-checked markdown, transcript sourcecode available here github.com/0atman/noboilerplate this is also where you'll find links to everything mentioned.
🖊️ Corrections are in the pinned ERRATA comment.
🦀 Start your Rust journey here: doc.rust-lang.org/stable/book/
🙏🏻 CREDITS & PROMO
Thanks so much to Laund for helping me fix a critical error in this script!
My name is Tris Oaten and I produce fast, technical videos.
Follow me here / 0atman
Website for the show: noboilerplate.org
Come chat to me on my discord server: / discord
If you like sci-fi, I also produce a hopepunk podcast narrated by a little AI, videos written in Rust! www.lostterminal.com
If urban fantasy is more your thing, I also produce a podcast of wonderful modern folktales www.modemprometheus.com
👏🏻 Special thanks to my patreon sponsor:
- JC Andrever-Wright
And to all my patrons!
ERRATA
0:21- link should be github.com/0atman/noboilerplate
1:25 - "The whole language always available" actually means at runtime as well as compile time. So Rust does the latter, but does not have the former. Hmm.
6:27 - typo in title. Should be "Arbitrary compile time execution"
I panicked for a second there
0:21 it's com/0atman not com:0atman
@@CaptainOfDoom must have left something unwrapped then.
@@pedrobraz2809 well spotted! I muddled up git/http address obviouly, woops!
I was wondering if there was a new permissable link format
It's time for my regular dose of Rust appreciation
Likewise friend, thank you so much :-)
Lol
same
@@NoBoilerplate Thanks doc 👨⚕️
My Friend who doesnt like Rust that much went "Oh, its just Rust D sucking" when I sent her this channel lmao
The idea that you can write valid HTML in your Rust files, with syntax highlighting and full Rust style error checking is just 🤯. Love these videos.
Rust's incredible isn't it? Though the syntax highlighting is down to the editor, I found that most editors highlight it really well!
And of course once the web frameworks (like yew.rs) become more mature, folks will write editor plugins that highlight the html really well, just as JSX has!
@@NoBoilerplate rsx for the win
@@warpspeedscp Really great standard here crates.io/crates/syn-rsx/reverse_dependencies
@@NoBoilerplate wow, i didn't realise people were already working on thos
@@warpspeedscp Yeah! The macro I demoed in the video uses syn-rsx behind the scenes, loads do!
The most famous sentence in Rust is: "We can do so much more"
"Hold my beer"
and the most infamous sentence continues: "...once we resolve the 5-year-old soundness issues in this laundry list of systems"
@@KohuGaly example?
@@mannycalavera121 there is quite a bit of soundness issues, but they don’t really affect real code
@@qwertyqwerty-jp8pr give me an example
I love macro_rules because macros do indeed rule
that = true
Still watching declarative macros 2.0 though, stuff looks a lot cleaner, shinier, and more consistent with the module system.
@@raffimolero64 Yeah, declarative macros 2.0 looks so much better! Can't wait for it to be in stable
I though i was the only one that read it as "MACRO RULES!" It always sounded like a joke to me
@@daishes Half the time it's precisely that, other half it's like the fundamental rules of the universe.
When I realized that I can run shell commands at compile time and insert the output at comoile time into the source code, I knew that rust easily is the best language
RIGHT
It's nice to see someone talking about macros like this. I think a lot of "Rust influencer" types aren't really aware of exactly what it means? Coming from a C++ background, when I saw the section in the rust book about macros I _immediately_ told my friend (who's very much a rust evangelist) that they looked really powerful and I was impressed by them, but he just didn't care and then started going on about the borrow checker and how the compiler "does everything for you". Your videos are a breath of fresh air that honestly have done more to make me want to get into Rust than any of the endless proselytizing I find a lot of rust devs do.
Right! As I say in a few of my earlier videos, as a python developer, I didn't care about memory safety at all - I never had to! It's a bad selling point when there are 10 other LIFE CHANGING features to talk about :-)
@@NoBoilerplate Totally. Looking forward to watching more of your content! Again, just absolutely loving the no-nonsense, straight-forward and balanced approach you have to topics.
This video sent me over the edge: I'm adding Rust to my Exercism tracks now!
Fantastic! It's not the easiest language to learn, my biggest advice is to read the error message twice, it's usually got the answer! also:
- The Rust Book
- Rustlings
- fasterthanli.me
- the rest of my videos...!
Do come chat on my discord over in #newbie-advice if you get stuck, GOOD LUCK!
@@NoBoilerplate Great recommendations. Love fasterthanlime; also Jon Gjengset.
@@NoBoilerplate Don't forget Jon Gjengset! Love his and fasterthanlime's content, it's quite in-depth yet easy to follow
The borrow checker, macros and how generics are implemented are the bumpiest parts of learning the language. Everything else is easier to understand in Rust than other languages. You will love it when your code compiles and there is almost no bugs at all in your system.
After the obligatory headache in Rust it gets better :-)
After a year of Rust, I'm thoroughly impressed with what people have done with macros. It's just incredible.
Yeah! And There's SO much more we could do!
I used relatively complex macros for example to be able to declare rust structs and generate the OpenGL/Vulkan data automatically to upload them to the GPU, they are really amazing. If you write your own procedural macro, your errors probably won't be quite as nice (usually just highlighting the whole macro invocation) unless you add errors manually but for your own code that should already be enough. I have some problems with them but I think the Rust team is working on improving them and I can still recommend trying them out!
Absolutely, it's a developing area, as Macros usually are. But what a feature ALREADY!
Procedural macros really blew my mind. This is beyond what I've seen in any other programming language (perhaps lisp), and is SO USEFUL. Attribute macros and Derive macros save-me a lot of boilerplate and spaghetti code...
Thank you Rust developers for this bless
Such a great feature THAT NO-ONE TALKS ABOUT OMG!
Lisp, Forth, Converge, Nemerle, Template Haskell, and a lot more.
@@vitalyl1327 Yeah! Great feature that I don't see why couldn't be in every language.
Though Template Haskell has it's detractors.
@@vitalyl1327 Julia too! I think Julia and Rust are kindred spirits.
@@micahrufsvold yes! And Julia front-end is based on Scheme, which makes it pretty natural to extend.
7:55
I thought I just saw my own neovim setup right there and was shocked
But then I remembered I got the tool recommendations from you
XD
ha! AstroNvim *is* pretty great.
@@NoBoilerplate agreed
One of the best neovim setups
Just what I needed on a Friday morning
Captain, it's Wednesday
I made a vec3 object in rust and using macros really made implementing the operator traits really easy. Definitely worth learning!
That's so cool! Did you publish on crates.io?
@@NoBoilerplate While I could, i'm fairly certain there are better implementations out there. What I built is part of a ray tracer implementation which I learned to build from a tutorial.
I found the macros on stack overflow when trying to find an elegant solution to the problem.
@@aditeya1024 ohh, please link! I'm doing the exact same thing and my vec3 file is ~300 lines of `impl Add for Vec3` etc..
@@aditeya1024 Was the tutorial "Ray Tracing in One Weekend", by any chance?
@@ZKtheMAN yes it was
You have my utmost respect for some of, if not *the* most ideal short, sharp rust lessons.
Thank you so much Bill, that's so kind of you to say. I hope to write many many more!
blazingly fast!
Wow, even though I do not yet understand everything perfectly, I can feel the energy with which you produce these videos, thank you so much! :D
My pleasure! Thank you for saying so :-)
Be careful -- that's how you were tricked into getting "vaccines" of extremely dubious value. Macros may or may not be the right code injection for you!
This is the one. I've always thought "Yeah, Rust is fine, but other languages give me just as much of all the benefits". But this here. This converted me. Checking your HTML is valid and even running test SQL queries at compile time? Awesome
People LOVE to forget the innovations of lisp!
Do start with the book, it's really well written and approachable! doc.rust-lang.org/stable/book/
@@NoBoilerplate thanks. I never really looked much at Lisp. Too many parentheses to see the beauty beyond it. But made a few compilers and interpreters for fun so might take a jab at a Lisp one sometime too to get friendly with it.
Rust documentation generally seems nice and good.
So the TH-cam algorithm has been recommending me your videos now and then for a while now… But now I've literally had 30 seconds of brain lag where my mind *insisted* that I know your voice from somewhere else.
When I started to listen to Lost Terminal out of curiosity, I'd never have thought it'd come with a free podcast of Seth nerding out about programming details :D. I'm sold. Thank you for both!
oh amazing! I love it when the stars align for that!
Hey, I have two other podcasts you might like, one that does exist and one that doesn't yet but you should keep an eye on:
1. Modem Prometheus, urban fantasy written by my friend Neil and performed by my friend Kate. I produce and write music (also I read the credits) If you love Buffy or anything Neil Gaiman has been involved in, you'll love this first episode: th-cam.com/video/_SDnUVHAC44/w-d-xo.html
2. /usr/stories podcast. Similar to Lost Terminal, hard scifi but set this time in the past. Season 1 in pre-production, probably will drop in a month or so twitter.com/usrstories
Thank you!
@@NoBoilerplate Got it, thanks for the heads up! :)
FINALLY, after so many decades, other languages are beginning to look into what Lisp macros have to offer.
EXACTLY. I'm excited by Rust because it takes most of the features I love from Lisp and Haskell and hides it inside C's clothing :-D
@@NoBoilerplate Quite frankly, imo Macros (especially Macros as powerful as LISPs) are more of a downside than an upside when working together with a lot of people.
Seriously, is it so hard to use the same code for runtime and compile time generation (oh, wait, it's not, Zig has done it).
@@kuhluhOG You've missed the point: Rust macros have full access to the system - disk and network etc. SQLx can't be written in zig's comptime because it has no access to the system at compile time.
You can write zig's comptime inside rust macros, but not the other way around. Proof: crates.io/crates/comptime
BTW rust ALSO has what you want: doc.rust-lang.org/reference/const_eval.html
@@NoBoilerplate About Rust's sqlx crate: I think you mean the query! macro to not be possible because everything else is in fact possible with Zig's comptime.
Also, full network and disk access at compile time is not necessarily wanted by everyone (for example the C++ committee voted against giving std::embed because they thought of it as too insecure; yes, really...).
But I don't think you get why I dislike Macro's that much. It's not about the power of network access or the like, it's that you can embed or even invent your own language inside of it and that it has essentially it's own rules compared the rest of the language (to a point where one can consider it it's own PL). While this may not be too bad for a project (you can straight up outlaw these kinds of things), you can't do that on an ecosystem level. This leads to ecosystem divergence on not a choice, but understandability level.
Also, const_eval is quite limited compared to Zig's comptime
@@kuhluhOG Rust isn't a theoretical language, it's been out for 16 years with macros. Your fears have not come true. Macros are FINE, honestly 🙂
“The dream of Scala is alive in Rust", I love it
I am so sad that scala has deprecated xml literals - what a cool feature! Ah well, we have Rust now :-)
Procedural macros are unreasonably powerful. I'm working on one which can validate a SQL-like statements used by SurrealDB against a schema. So far it will correct your clause ordering, inject variables with proper escaping, ensure fields exist when you query against a schemaful table, and parse the results into any serde compatible type.
The possibilities are endless, and would be even more so if the tokeniser could parse backticks.
It's SO powerful, right? I love it!
I always love seeing your Rust talks, I've been into the language for a while but they have really helped me appreciate all that makes Rust so excellent. I have absolutely loved macros myself, they're a complete game changer and I didn't know how much I needed them in other languages until I had them in Rust. In any other language I would bemoan the hefty syntax, but in Rust I never actually mind my code getting large, since I know that I will only ever have to write it once. Once and ONLY once, because I can use the type system to build such powerful abstractions, and where even that falls into monotony, macros pick up the slack by letting me compress repetitive syntax like implement blocks into a single line.
That's right! But don't forget (I beg you) that proc macros can access the disk and network. What might you do with that power?
@@NoBoilerplate In a proc macro, I did parse the library dependencies (reading "Cargo.toml") with the "cargo_toml" crate because my advent of code binary needs a solver library for each puzzle, list that evolves with time.
@@philippecholet9484 so meta, I love it!
The most complex thing I did with macro_rules! was a macro I called "fluid" that (recursively) turned a series of statement-looking lines into a builder chain, including support for nested builders. The whole thing is only like 20 lines, but *relatively* parseable, if you're used to recursive code at least.
I cannot possibly imagine doing that on a text replacement-based "macro" engine.
omg like clojure's arrow macro? (->) I LOVE IT PLEASE TELL ME THE PACKAGE NAME (don't use urls, yt hates that)
@@NoBoilerplate I'll be honest, I never thought about publishing it (or anything, for that matter). I will try doing so over the weekend, and get back to you on that!
@@CaptainOfDoom Please post it here if/when you do publish! :)
Don't forget to share it
@@NoBoilerplate
I do know the macro "pipeline" from my searches, you can do things like:
let length = pipe!(
"abcd"
=> [len]
=> (as u32)
=> times(2)
=> [to_string]
); // from their documentation
I did not use it yet, but I saved as fav because it looks so cool
Procedural macros really allow you to do some black magic.
I once wrote one that allows the user to turn very verbose parser combinator code into a concise DSL with infix operators.
yaaaas did you publish?
I did, the crate is called langbox
wat?
@@Artentus Amazing! crates.io/crates/langbox
@@NoBoilerplate thanks!
Okay, I only understood like 10% of that, but I'm stoked to learn more.
It's a complex topic! Check out my other rust videos 😁
Yet another great video. I enjoyed every second. I just started to expand my knowledge about macro! and this is great to have in mind. 👏👏
thank you so much!
Thanks for the interesting video as always! Still halfway through the book but it’s really good to see so many new, fun things to learn.
Fantastic! You're gonna have a great time, and thanks for your support friend :-)
You are really good, the sound is so comfy that I can see the video and enjoy the information. Thanks for the great video
Thank you so much for saying :-)
I have a lot of practice speaking because I am currently on season 11 of my hopepunk/scifi podcast, Lost Terminal. I'd love for your feedback on it! Episode 1 is here th-cam.com/video/p3bDE9kszMc/w-d-xo.html
Love your videos, seeing the potential of procedural macros I would love you see you go in deapth for that topic and explain the syntax and how they are used. Amazing video man, love your work
Thank you! I have noted that, and it looks like I will indeed do a part 2 in the future!
I am here because ThePrimeagen allowed me to discover this channel and find curiosity in NeoVim and Rust hoping to enjoy this journey! Love the channel content!
Thank you so much! Primeagen is such a nice chap, welcome!
every time i miss hearing you talk about rust, you're right there! great video as always~
Thank you so much! I'm on a regular 2 week schedule now on fridays. (though Patrons get the video a day early!)
Perhaps you'd like to hear me talk about science and tech through the lens of a fictional AI? That's what my podcast Lost Terminal is about! th-cam.com/video/p3bDE9kszMc/w-d-xo.html
I really like the way you narrate this video. This really motivates me to finally have a deeper look at how to write Marcos in rust.
Aside have you ever had a look at type providers in rusts cousin F#? There is a SQL type provider that checks your query definitions at design and compile time. Also exists for CSV, XML and others. If Microsoft put a little more love into it it would blow C# out of the water.
This has grown to be my favorite channel on yt. Nice vid as always!
Thank you! You're too kind!
Thanks. Great addition to what I have read in chapter 19.5 of the The Rust Programming Language book.
The book goes a bit more into detail (and the Reference and other docs probably even a lot more, but I am not there yet), but this video has greater examples with HTML and SQL.
We keep learning. :)
Thank you so much! Yes, The Book is wonderful, as is Rust By Example and Rustlings. I imagine Amos over at @fasterthanlime (fasterthanli.me) will one day write the definitive article on Macros and then we'll all be enlitened!
love your videos ... always smart and thoughtful ... keep on keeping on; you make a difference.
That's so kind of you to say Ken, thank you, I will! :-)
You really make my day by uploading a new video, keep up the excellent work ❤️
That's so kind of you, thank you!
You convinced me to try using Rust (I was previously an avid golang user) and I'm loving it
Go is great! You're gonna have a great time with Rust, it punches through both go's ceiling and floor to be lower-level and higher level than it.
To get the async ergonomics you're used to with go, learn tokio, it's the de-facto standard.
Do watch my videos and read all of fasterthanli.me and The Rust Book! Have fun!
This is mind blowing!
I want to point out that C++ got constexpr and consteval, they are compile time expression and can be made into function that run in compile time.
Can't do side-effects though, right? Can they access the network at compile time?
@@NoBoilerplate i've seen a demo that printed ASCII art at compile time at different scales based on a size parameter, so some side effects can be done
@@somenameidk5278 I'm delighted to inform you that this sounds like a bug in the compiler's handling of constexpr
"neither std::cout
@@NoBoilerplateamazing
Hey, I just want to thank you for these videos. This channel is one of the few I've found that is targeted to experienced practitioners, presented even-handedly, and isn't just horrendously off-putting from an aesthetic standpoint, so I hope you'll keep going for a long time.
Re proc macros, I'm still a low-intermediate Rust user and hadn't previously encountered anything like Yew's html macro in the wild, and so I think I assumed when I saw that they consume TokenStreams that they necessarily took in basically Rust-adjacent syntactic forms. I'm glad to see that they're much more powerful than that! They do make me long for an intermediate point on the simplicity vs. power scale, since Rust has solutions for both extremes but nothing right in the middle, which is how I would characterize Common Lisp's or Clojure's "defmacro" since these allow you to transform and generate syntax with simple list operations and don't require all the machinery of parsing (CL of course has reader macros too, which are basically equivalent to proc macros in their flexibility to define new syntax entirely). Of course that's a consequence of Lisp syntax being composed of lists already and so there's likely no possible analog in Rust, I just miss that particular affordance. Still, it's amazing to have a feature this expressive in a language that also permits such explicit control of details like memory layout. It's truly an excellent environment to work in.
Thank you so much for saying, I really appreciate it. I am indeed targeting just those sort of users, and I aim to not just continue, but ramp up this year, with some tasteful adverts, I might even be able to go full-time youtube!
Check out macro_rules! for a simple solution that does the parsing for you, lisp style!
I avoid macros whenever I can.
The most complicated one I had to write is to verify that an array is sorted. And I will replace it with a const function whenever that becomes possible
I daresay we will get const functions, just as we got macro_rules! compile-time matchers, anyone can write this feature for us today, within a macro!
Man everything you made that TH-cam suggests is gold, love your content
My pleasure! Do check my other videos, they're in the same style, and I do interesting non-rust videos too!
This is just what I needed after a long day of fixing issues in a java codebase caused by over-reliance on boilerplate.
All those extra bloated features in java, all they need was proper macros in 1996!
as always an amazing new rust video. macros are so good.. i need to get more into it.
Amazing! Thank you. Would be cool to to see DSL with Rust macro video from you as well)
noted, thank you!
One of the best example of procedural macro i know, is the derive_xrb! macro from the WIP xrb crate. It includes custom syntax to implement the types needed for X11, as well as serialization and deserialization. This is awesome work from Antikyth !!!
Fantastic!
I wrote a macro library that mangels the correct jni name based on given parameter types and package info. It also encloses your code so you can return a result and it will throw a java exception if it's an error.
I never thought rust would be so easy yet so complicated. Love it.
It's terrific! Do check my high-level intro to Rust: th-cam.com/video/oY0XwMOSzq4/w-d-xo.html
Sorry to be picky (kind of my job!) but I think of Rust in the opposite terms:
It's not easy, but it is simple.
@@NoBoilerplate If I was able to understand it, then it is easy bro.
I don't know why I like these videos so much, I have never written a single character of anything resembling code in my entire life.
You're so kind! I wonder if you'd like a scifi story read by me, if that is the case? I'd love to know what you think of Lost Terminal th-cam.com/video/p3bDE9kszMc/w-d-xo.html
@@NoBoilerplate I watched the first three episodes. I'm very interested! I find it actually captivating to think about what an AI actually 'thinks' about. Though Seth is very personified for an AI, he is quite endearing. looks to me that I have something new to listen to on my drives to work! thanks for your great work and dedication to the story!
@@anamewithnoface1330 I'm so pleased! I too love listening to podcasts while on long journeys - perfection!
You can listen to LT on spotify, itunes or wherever you listen to music and audio - more handy than youtube when in the car!
If you'd like to chat about LT (or indeed NB), come and say hi on my Discord server!
I currently don't have the time or motivation jet to lern Rust and have a hard time to understand the syntax as a java user (because of school). So I like your funny words, magic man.
I wrote just the video for you, Rust is easy! th-cam.com/video/CJtvnepMVAU/w-d-xo.html
I love watching Rust videos. That one snippet for bounded_impl had just enough syntax hell to dissuade me from wanting to learn it though
Oh don't even worry about macros, they ALWAYS look nasty in whatever language you write! There's an old lisp saying "never do in a macro what could be done in a function". True in Rust too!
Rust syntax is only as complex as it has to be, I promise. There's loads of great features that simply require more syntax to work. You must make your programming language as simple as possible, but no simpler.
I'd love to know what you think of my video here th-cam.com/video/CJtvnepMVAU/w-d-xo.html
@@NoBoilerplate I'd seen the video before, just watched it again.
I'll never say the compiler isn't really handy since that's what everyone talks about with Rust and I'm sure you can get 90% of the way there with just the compiler messages.
My main reasons for not learning it are:
1. I just don't have a use for any compiled languages at the moment (I use Lua to make addons for various games and Python to get data off of websites) and when I write code I want it to run immediately without waiting for compile time
2. When I do need to use a compiled language, memory safety is usually not on my mind because I'm not doing anything important or to scale
3. (Especially for Rust and C++) when I look at other people's code and see a ton of mumbo jumbo(&forThe,compiler) => {} it's a real turn off, not that most languages don't have their moments of that but c++ was really nasty to look at and Rust is just c++ with added rules, syntax, etc
cool logo and nice idea though
@@lionbryce10101 Absolutely fair, lua's really great (I recently converted my nvim config from vimscript to lua and I'm so much happier!
If you want to play in the following domains, Rust might be worth the price of entry:
- webasm
- bare-metal without using C
- high-performance computing
My take is Rust is EXTREMELY different to what I was used to: No inheritance, new syntax for lifetime annotations and borrows, and the compiler hates a lot of the standard normal patterns we've been using for decades. I know how you feel there!
Back in 2020 I crashed out of learning Rust twice. First time due to multiple string types (Haskell's biggest mistake made again, I thought to myself) and second due to the lifetime syntax. At the time I had a great mentor who picked me up and helped me back on the path. Most people don't have this. This is why I made my Rust series. Yes Rust is very different from what you're used to. But that's the point.
Anyway, you sound like you're in a good place, no rush to learn new stuff. Python made my career for 15 years, no harm in that at all. BTW do check out bevyengine.org/examples/ if you're interested in game dev, impressive stuff there. Certainly, people on my discord rave about it! Cheers!
Your voice is super soothing and distracts me from focusing on what you're explaining (can't blame you though lol). Jokes apart, nice and enjoyable content packed with information. Could you also please make videos on low level/unsafe rust too because there aren't many.
That's so kind of you to say, thank you!
I'm afraid it'll be a while till I do a deep-dive on unsafe and low-level, because I don't work with that level typically.
I've talked about it in half of this video, and I'm very happy with the high-level description: th-cam.com/video/PuMXWc0xrK0/w-d-xo.html
If you would like to hear my voice in a format where I'm TRYING to be soothing, I'd love to know what you think of my AI/hopepunk scifi podcast, Lost Terminal th-cam.com/video/p3bDE9kszMc/w-d-xo.html
@@NoBoilerplate Hello, I've just listened to the lost terminal podcast and I could relate to the satellite because I'm lonely too. Again, could just listen the whole podcast just for your soothing voice, but as for the podcast content, I personally didn't find it really engaging and entertaining 😅 (super sorry, but please don't feel bad and think that you're making bad content, its just my personal opinion and there are many viewers apart from me who love the podcast).
@@formulaetor8686 Quite alright! I produce and write the music for Modem Prometheus, maybe you'd like it instead? It's very VERY different th-cam.com/video/_SDnUVHAC44/w-d-xo.html&embeds_euri=https%3A%2F%2Fmodemprometheus.com%2F&feature=emb_title
just new to this rust, really love this concept !!!
It's *amazing*, so much of what I like in one languages. Do check out this short playlist of my best Rust videos for more info th-cam.com/video/oY0XwMOSzq4/w-d-xo.html
i will surely thankyou dude@@NoBoilerplate
May we please have some of your fantastic stylistic approach towards wasm and the like? Your explanation process is excellent 👌
I have great news! I already wrote two videos on wasm!
First video's here, the second part is 2 videos later
th-cam.com/video/P4LMfkFLRsI/w-d-xo.html
@@NoBoilerplate 🙌 Many thanks!
Hey! I really appreciate your videos and this one as well! Would you mind spelling abbreviations you use out on your presentation cards? I find myself googling some of them.
Thank you so much! This is a great idea, I will put in the markdown scripts links to the first usages of all abbreviations going forward!
github.com/0atman/noboilerplate/
@@NoBoilerplate I can give that thank you right back! I really enjoy your videos and you being so open to feedback will improve on that even more.
@@thebaconbreadful this is the way!
I cannot escape Rust. Every time I try something else it calls me back like the siren from The Odyssey. And I have no earplugs.
Don't tie yourself to the mast - come in, the water's fine!
Interestingly I actually have a video in draft making an analogy to an Odyssean pact and Rust's type system...!
@Peter I strongly disagree. Rust has a single compiler that defines what valid rust is and for most projects and companies (by far), that is enough. It still makes sense to work on a spec but that should not stop people from using Rust right now. And it doesn't. Don't take my word, just look at all these large companies and FOSS projects that are already investing heavily in Rust and using it for crucial parts of their infrastructure (Google for Android, Microsoft for some low level stuff in windows, Cloudflare, Amazon, you name it). Cloudflare is a good example, they replaced nginx (!!!) with an inhouse solution written in rust, serving billions of requests every day. Even Linus pushed towards adding Rust to the kernel.
@Peter I have good news for you!
1. The Rust team are considering a formal specification (which is what I think you want, not standardization). and
2. Many industries don't require standardisation - web development being a huge one!
I'm excited for Rust to become even more useful, and would welcome a formal specification to allow use in organisations that have come to assume specification = correctness.
This video has made me decide to start using Rust.
Wonderful! Do check out my other videos, there's so much good stuff here! th-cam.com/video/ifaLk5v3W90/w-d-xo.html
I primarily code in Julia and it has this feature too. Really cool way of programming
The rust macro system is wonderful, but it does have its caveats.
Declarative macros have some limitations, since they always have to output some standalone valid code, which makes them hard to combine.
You end up having to use the weird TT muncher pattern, which is pretty impractical and hard to read.
Procedural macros are also limited by the complete lack of context to their input. If you want to make a procedural macros that feels idiomatic to rust, you'll quickly feel the lack of typing or general location of the call.
Oh yeah, I have so much respect for those who write macros. Even in lisp, macro's natural habitat, they're weird. Worth it though!
I like your funny words, magic man.
I was with you until about halfway through and then I had no idea what was happening, but it sure sounded impressive! I'm still kind of scared of rust, to be honest. Maybe I'll come around eventually lol
Ha! Macros are so weird aren't they! Don't worry, you don't have to write a single one to write normal Rust. It's like writing C extensions for python/js/ruby etc, it's mostly wizards who do that for us!
You sound like you need a relaxing video to watch after this one, I prescribe th-cam.com/video/CJtvnepMVAU/w-d-xo.html
(let me know what you think!)
I would have settled for Rust style error checking alone but this takes it so much further. I cannot wait to see this language with a little pedigree behind it.
Don't wait my friend, Rust's already replaced C in the Linux kernel, in Discord, in CDNs around the world, and even some of NPM's backend services.
It's achieved critical mass, and is ready!
What are you waiting for, what's not good for you?
yeah the sqlx query macro is actually insanely nice. It's everything i want from an ORM without any of the things i hate about ORMs
It's fun to think what postgres/whatever features sqlx supports.
All of them.
I love the power of the macros but I can imagine a scenario where the DB a macro pulls from changes or is missing and that breaks compilation.
Might be a great security feature though.
Sure, just like when you run your project and it tries to connect to a database and it says "db not found", or if the db changes and suddenly your schema is out of data in your running app. With sqlx you get that feedback *immediately*, which I love.
Early errors are much better than late errors! Honestly, that's the Rust thesis right there :-)
Loved this video! Traits are my favorite Rust feature, but macros are a very close second!
Though, I'd appreciate it if next time maybe you mentioned some bad things as well :P E.g. here would be the cost we pay in compilation time (esp. for procedural macros)
I've mentioned that in the past in more general Rust videos, it's a price I'm very happy to pay! :-D
Narrator: *talks about macros, HTML syntax within rust etc etc.
Me who started learning the language 3 days ago: o.O
You're going to have an *incredible* time learning Rust! Do ask questions in #newbie-advice on my discord if you get stuck, and read The Book! doc.rust-lang.org/stable/book/
Thanks a lot for covering this advanced issue !
I am impressed about what Rust macros can do !!
E..g. executing code at design time e.g.with the database crate. I suppose it is possible to write code that is to be executed as well at design time as in the compiled program, when executed.
But this is not a completely unparalleled feature.
*Delphi* - in it's combination of language / compiler, library, and IDE - also features database access at compile time to try out and configure the communication with a database server.
Moreover Delphi uses the same paradigm for it's "WYSIWYG" GUI designer. Here the code at compile time displays the GUI under constructions (allowing to use the mouse for editing the design) and writes a file (similar to jason) with the construction and functionality rules. As runtime the (essentially) same code reads that file (in fact embedded in the executable as a resource) and displays the GUI to be used by the business logic.
I do hope that something similar once will be provided by an appropriate Rust crate (of course based on Macros), e.g. based on egui / OpenGL. Same will allow to create projects that can be compiled for multiple platforms including Browser / webassembly.
Thanks for listening
The great thing with HTML, as a universal interface language, there are MANY visual designers that you can build your UI in, then just template them into your server code with (say) rocket.rs.
@@NoBoilerplate Can you comment on " I suppose it is possible to write code that is to be executed as well at design time as in the compiled program, when executed"
@@michaelschnell5633 What does "design time" mean? I'm Familiar with compile time and run time?
@@NoBoilerplate My dream always was a system that allows for a decent local GUI and and a bowser based GUI (i.e. local and remote GUI to choose from).
With Rust this seems to be possible by Webasm and WebGL (e.g. vi egui) . But nowadays it might be appropriate to use html for this by implementing a browser in the desktop app. Now for design time a WISYWIG HTML editor might be implemented by a Rust Macro.
I just don't yet see how to have the designer easily bind business logic functionality within the GUI design flow. But I am no expert on HTML stuff at all :)
-Michael
@@michaelschnell5633 Ah! I see! I'd recommend trying out tauri.app and doing their tutorial, getting a simple app working, and taking it from there! Lots of videos on youtube can help you design html interfaces, it's really not too complicated, I promise 🙂
3:20 ngl, creating implementation with declarative macros hasn't crossed my mind. it's genius.
I suspect we're only just (re)learning, as an industry, how to use macros!
I'm in awe!
Rust's pretty incredible! Do check out my other videos to get more excited about it :-)
I didn't understand everything but I was blown away
honestly same, that's why I'm here :-)
@@NoBoilerplate Thanks!
Súper bueno! i wan't aware Rust macros where so different. I am liking Rust a lot!
Cool right! If Rust didn't have macros (my favourite feature from lisp) - we wouldn't be speaking, I'd be using Haskell or Go - terrible! :-D
@@NoBoilerplate hahaha… I tried Go last year and really didn't like it a lot. Since I already know C++ and C#, I really needed something comparable. Rust makes a lot of sense to me.
@@AlexEscalante hehe, yes Ken Thompson and Co were rather blinkered when they made go, I think. They tried for "cpp but modern" and I believe they succeeded wildly. But I need more than that 😅
There's one thing that I feel is very much missing in Rust macros currently - the ability to "see" the surrounding namespace to be able to query it. As is, if you see an `Option` in a macro, you do not know if it actually *is* `core::option::Option` or any other type. On a related note, afaik you can't exactly tell the compiler in which order to expand the macros such that you can keep track of some state to effectively "annotate" other types with information about them that might be useful.
While you *can* do both of these things to *some* extent with traits and/or consts, it's rather difficult sometimes & in my opinion does limit the *reasonable* possibilities quite a bit.
This is where I think Zig actually does this rather well - Zig has compile time only functions, which you can use to generate new types or do compile time reflection on existing types.
Oh fascinating - this is something called macro hygiene, I think? Hygenic macros only act on the scope they've been provided, and are thus easier to reason about, unhygenic macros slurp up stuff outside there scope. I could be wrong there.
I'm not as familiar with zig compile time functions as you are (though I've read much). Can't you generate new times and do compile time reflection on existing types with rust macros? My understanding is that zig compile time functions can only do a subset of what rust macros can do?
@@NoBoilerplate macro hygiene isn't a way of "seeing" the environment, but rather is a way of making sure that macros don't end up having unwanted interactions with the scope in which they are expanded, by actually going it it's own "separate environment" to some extent. Say you have a macro that expands to `{ let x = foo(); bar($expr); baz() }`, then that `x` will not be visible to the expanded `$expr`, meaning it won't shadow an `x` defined in the scope in which the macro is expanded.
Regarding how powerful macros / Zig functions are, pretty sure it's the other way around, Zig functions can do everything macros can but not the other way around, except for custom syntax (since functions in Zig still operate on "values", be they compile time values (which include types) or not - and not token streams).
In general, in Zig you can do things like given a type find out if it's a tagged union, enum, union, struct or primitive, and even see the fields it has and their types etc. Beyond that you can also concatenate variable length slices at compile time without having to worry about allocations or lengths without issue, read files at build time, etc etc - the print function in Zig also uses the same compile time machinery: ziglang.org/documentation/0.10.1/#Case-Study-print-in-Zig
So you can do pretty much everything (and if you need a DSL you can parse it from a string at least, or put it in another file) at compile time in the actual context in which you know all the context of your types etc.
(It should be noted that you can get some similar stuff in Rust at least as it pertains to types with a ton of derives, but it's much more work to implement.)
Meanwhile in Rust, macros just turn token streams into other token streams - if you see the token `Option`, you only know it's an identifier, and not what that identifier refers to / what traits it may implement / etc
Which I should note though, is a hard problem to solve - the abstractions Rust chooses make it pretty much impossible without having a lot of metadata about each macro, especially where traits are involved.
I ultimately still prefer the safety and abstraction language via traits Rust provides over Zig for actual production level stuff, but I do very much enjoy the "reflective power" and compile time goodness Zig can provide that Rust is comparatively lacking.
@@codenamelambda Ah of course! Thank you for the explanation. I have enormous respect for Zig, though the minimalism of the language isn't quite for me (in the domains I work in).
I started learning Rust this past December and didn't really get how macros worked (aside from the simple text-substitution ones). The SQL example one helped me out a lot. Now I want to try making macros that execute at compile time and maybe integrate them into a project I'm working on.
Wonderful! Yes they're tricky to understand at first. I only understood their importance due to my lisp background.
Good luck!
Was just thinking about how you would probably release a video soon
That's Pavlovian conditioning: I release every other Friday :-)
Every time I stumble upon these videos my brain goes “Hey it’s the lost terminal voice” 😂 (even though I found that because of this channel, it made such an impression 😊)
Hehe!
Your channel is making me fall in love with Rust! Do you have any recommendations to land a job? Most of the positions I see requirer at least 3y of exp. Im currently freelancing Fronend and Web3 (Solidity and Js basically), but never worked in a company before. Looking forward to dev smart contracts in the NEAR Ecosystem, looks promissing.
There's a small but growing market for Rust developers, all the big players are taking rust seriously, look at this list that contains Microsoft, Aws, github and others www.rust-lang.org/sponsors - find their job boards and apply to them all!
Secondly, four of my videos were sponsored by companies actively recruiting for Rust devs, apply quick! Videos:
- th-cam.com/video/JIvKgSyvtxI/w-d-xo.html
- th-cam.com/video/P4LMfkFLRsI/w-d-xo.html
- th-cam.com/video/y10jJX35shE/w-d-xo.html
- th-cam.com/video/sbVxq7nNtgo/w-d-xo.html
Good luck! I'll try to add more sponsors who are recruiting for Rust, and maybe even do a "how to" video in the future!
So with the feature of embedding other language inside of rust, is there a way to type rust, but with the indent based syntax that python has without squiggly brackets?
I bet there is (search crates.io) but let me defend curly brackets:
I've written python professionally for 15 years. At the start, indentation scope is nice and easy. When you're at production-level, and you've got huge levels of indentation and debugging where the mistake is, you wish wish WISH you had curly braces XD
This is how I think about Rust's more complex syntax: there's more detail, but it's there to help you. My video on this here: th-cam.com/video/2hXNd6x9sZs/w-d-xo.html
Fantastic videos thanks :)
My pleasure!
I've been procrastinating the learning of macros. My SV3G repo would seriously benefit from them! (I need to validate CSS colors and SVG strings at compile-time)
oh that would be perfect for a macro! Make sure you're always using nightly, and try this to help understand what is happening github.com/dtolnay/cargo-expand
If you need help, post in #programming on my discord, loads of smart people live there! :-)
@@NoBoilerplate Thank you!
What? That's so cool! Why didn't I know about this?
YEAH OK FINE. I GET IT. YOUVE CONVINCED ME.
Next WASM experiment I try - or reverse engineering/modding experiment - will almost certainly be in Rust because your way of breaking down the features in a digestible manner and into layman’s terms is absolutely bloody brilliant.
Thank you, or heck you - time will tell (though I’m quite sure it’ll be the former)
I'm so pleased! You'll have such a great time, make sure you watch the rest of my rust series, starting with this one on wasm:
th-cam.com/video/P4LMfkFLRsI/w-d-xo.html
This is the one video that Really makes me want to get onboard. I love using SQL to generate SQL so this is Panacea.
But boy do you speak fast !
Thanks for everything.
My pleasure!
I think of macros as turning any language into a domain-specific language, excpept it's specific to every domain.
Exactly. Lisp is described as a "programmable programming language" and Rust is too!
The more I search the more awesome Rust gets!
Same!
In the future I also want to learn about the rust, and I have a question, does the rust have "full oop features - four principles of oop" like c++, java? thanks
p/s: sorry for my english, i used gg translate
Perfect english! Rust is not an OOP language, it is BETTER than that. As you learn more about languages, you may discover alternatives to OOP that are just as good - we often think of OOP as the *only* way to program, but it's simply *one way* to program.
If I'm reading this right, the 4 principles are abstraction, inheritance, encapsulation, and polymorphism.
Rust has all of those, but in very different ways than Java and C. polymorphism is the most different in Rust.
Rest assured, Rust has comprehensive techniques for building as large projects as you want. I would recommend reading The Book: doc.rust-lang.org/book/
Good luck, and please ask me more questions on my Discord if you need help :-) (links in the description)
@@NoBoilerplate thanks, what I am interested in is that rust has all 4 principles of oop, and surprisingly it still has those principles, about how to approach and implement them, I will research
wow, this one kinda blew my mind
I know right! I started to take Rust seriously when I discovered it had a lisp-style macro system! Did you see my previous video on it? th-cam.com/video/PuMXWc0xrK0/w-d-xo.html
I love every single video of yours ♥
That's so kind of you to say!
I wonder if you've watched my little hopepunk/scifi show, Lost Terminal? th-cam.com/video/p3bDE9kszMc/w-d-xo.html
I'd love your thoughts on it!
I thought Rust macros could not compete with Lisp macros (seriously, who would dare bootstrap object oriented classes in any language other than Lisp ?) because lisp relies on abstractions like dynamic typing or garbage collection, which are not possible in Rust, in order to maximize expressivity and minimize the syntax.
But this video proves me wrong, implementing html and sql as Rust DSLs seems to be as much power as one could get from Lisp.
Rust is indeed the ultimate language.
Actually, there is still one thing i haven't seen Rust do yet: continuations
But i think a continuation passing style macro would be no problem for Rust
Certainly you could implement them using macros, just as CL does - this is how async/await was prototyped in Rust at first, but it's now a native language feature!
rust-lang.github.io/async-book/03_async_await/01_chapter.html
this is just one of those things that you don't realize you need until you hear about it, and then you hear about it and wonder why it doesn't exist anywhere else
This is called The Blub Paradox, introduced in Paul Graham's fantastic essay on Lisp (which Rust implements 80% of!) paulgraham.com/avg.html
I can imagine the painless porting from the other programming language with just copy paste into the macro arguments.
An: yeah that's right! A macro for every language would be incredible! Completely possible, for the right kind of person XD
i'm buying whatever you're selling rn, man, such passion
My real passion is not getting paged by another syntax error in production at 4am
Holy shit.
I thought the feature interesting but failed to grasp it's real use (beyond i.e. a build system).
But the sql blew me away.
Awesome as always Tris.
Fantastic. I'm so pleased, this was just the sort of misunderstanding I was hoping to help with by making this video - Macros are great! XD
Greatly put! I should give Rust another chance 😂
It's really incredible!
i once wrote a proc_macro that would take an arbitrary amount of iterator and generate a sql bulk query from them because the library i was using didn't support it.
Perfect!
2:41 the space after the left bracket hurts me
Well spotted! You found the bug in my reveal.js plugin, it HATES those dollar signs, some kind of escaping bug? I had to hand-edit these macro slides to fix them. DISLIKE!
My bug report is here FYI github.com/MSzturc/obsidian-advanced-slides/issues/193
Macros are love, macros are life. May our syntax be free, may our code be proud, and may not a single phrase EVER be written more than once!
SO good right!?
I'd love to see opencl and CUDA kernels with macros. (May already exist I need to check again). I never got into gpgpu kernels because I couldn't stand typing string literals as syntax.
AFAIK, That is exactly what the asm!() macro does doc.rust-lang.org/rust-by-example/unsafe/asm.html
I hope there's one for CUDA!
Syntax changing macros, I cannot get behind, but arbitrary compiletime execution is what I got so fed up over the lack of in languages that can go to the metal that I started designing my own.
Let me try to sell you on syntax-changing macros:
If you don't have them, you must wait for the language authors to release functionality you want. (think, async). Other languages have to do this, if you want new python functionality, you must wait for the python developers to do it for you.
It's even worse in Python because the language is written in C, a different language to the one we're used to.
Rust is written in Rust, so we *could* make our own changes, but we'd still have to persuade the community to merge them.
With macros, YOU get to program the compiler per project. Take a look at this video of mine where I explain more th-cam.com/video/PuMXWc0xrK0/w-d-xo.html
@@NoBoilerplate
Yes, in LISPs, languages that only use monoids to deal with IO, and other ones that are intended to be detached from the machine or ISA they're executing on, it might well be impossible to add new functionality to the language without syntax changing macros, but with a language that goes down to the metal, any functionality can be added with compiletime execution of the normal language itself; you won't get the syntax you want with, say, that html macro, but I'd rather load an html file into a string or through a constructor at compiletime than switch the syntax of the language I'm writing in mid-file.
Another point about using a macro system is that it can only happen at compiletime; that splits the language and makes you write them completely differently. Exempli gratia, a macro must contain another macro, not a procedure, to do everything at compiletime. Having arbitrary compiletime execution, instead, allows you to write a procedure once, let it be called at compiletime or runtime, and, importantly, have the compiler propagate whether the expression is being executed at compiletime to sub-expressions and procedures that are called by procedures in the expression; this allows you to just have one implementation of an algorithm instead of a macro and procedure for the same thing.
Here's my language if you'd like to check it out: www.kennethpollick.com/software/y.html (a high level overview), th-cam.com/play/PLQm-teGlSiPQp9H5lIuQVqI_Glxx4YQdn.html (a playlist of streams I've done on twitch about Y (watch out; the audio's blown out on the first one)), github.com/KennethPollick/LibY (the standard library which needs an overhaul because of changes), discord.gg/AdjxzFjPNs (the Y discord).
constexpr in c++ is all about compile time code. Right now c++ is trying to discourage the use of the preprocessor and shift over to constexpr and other things instead for compile time evaluations and other preprocessor usecases
Can you access the network in constexpr?
@@NoBoilerplate sadly no. networking hasn't even made it into the stl yet and the TS for it has been kinda dead since 2018...
Constexpr and similar afaik aren't as powerful as rust macros but they have similar ideas behind them.
Just wanted to mention constexpr because the video mentioned c macros. So I wanted to mention what C++ is doing to replace them with something safer and more modern.
@@sinom Ah! Got it, thank you very much for pointing that out, so it's an attempt to do c macros but better? That's super!
No side-effects is a reasonable choice, macro_rules! can't do side-effects either.
Rust also has constexpr. They serve different purpose than macros. Constexpr is for compile-time evaluation. Macros are for pre-compile-time code generation. Just because you can use the latter to do the former does not mean it's a good idea to do so.
Common Lisp macros are also an example of good macros system
Oh absolutely, I mentioned lisp specifically because of it.
I actually wrote clojure professionally for 2 years a few years ago - that was fun!
when I discovered Rust is an expression-based language that has lisp-style macros, but is REALLY popular, I switched HARD!
Rust seems quiet beautiful - I would love to try. Also, at 8:40 an incorrect statement is made claiming "To get this compile time features in other languages, you must wait for the language or pre-compiler authors to update their code". Languages like common lisp and scheme have supported macros for many many decades now.
Ah lisp, my beloved! Yes of course, as I have said many times, Rust is like Haskell and Lisp snuck into the cool languages party in C's clothing!