To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/fasterthanlime - You’ll also get 20% off an annual premium subscription. Early access: fasterthanli.me/videos/h2spec-to-rust Errata: (nothing yet!)
Gotta say, I really expected a betterhelp sponsorship just how, I'm happy it was Brilliant instead but unless that misdirect was on purpose to make fun of Better help sponsorships this was a pretty strange way to promote Brilliant?
There is an old rule for creating engaging stories; You are allowed one lie. You can make up butt-faced aliens climbing up from the earths core on jetski's, one lie is forgiven by your audience. Every other event must have a logical reason, an objective "yes, how else would you suggest to do this" causality. Your audience will not forgive more lies, and boo at the cinema screen and popcorn will fly around the room! Amos wrote, acted in, and edited, a moving story with **zero** lies. 💪
Even Rust cannot escape TableGen cranelift has its own DSL for generating machine instructions that does a whole ton of code generation. GCC does the same and so does LLVM (where TableGen is from). Maybe one day proc macros can replace this code generation case but that day is not today.
i've done some ungodly things with proc macros to the point where i tried to come back to it 2 days later and i had no idea how to follow the flow of my own macros. rust foundation please finish introspection!
The purity of proc macros is a bit unfortunate even if understandable. A while ago I wanted to do some code injection into an asmjs app. I wanted to do this semi-automatically by annotating the rust library with proc macros and then parsing both my lib as well as the asmjs source and patching it based on metadata generated from the proc macros. In the end I ended up using a feature flag in the proc macro that would either modify the injected function and pass in some extra values or generate calls to the inventory crate. This would then be collected into a json by using the example(?) binary type in cargo.
re: git hooks @ 14:11, aren't git hooks "disabled" by default? I thought that they are not committed to the repo and you have to copy them over manually (or use something like the `pre-commit` tool). Which makes sense, because they can run code without you knowing.
@fasterthanlime couldn't you have made a proc macro at that you call at the crate level on `httpwg` that iterates through each module looking for tests matching your criteria and writes an impl block for some type that builds up your test suite? that way it recurses properly without being stateful and you only have to write one proc macro. I know it would still have to parse a lot of syntax but you could probably internally just feed syn individual modules instead of the whole crate
Awesome video, super interesting as always! Can't wait for another fasterthanlime 40-minute-video-saga-classic when you have the funding and time! Put it on the backburner and there'll be an opportunity eventually :) I appreciated the mouth trumpet (tuba?) at the end with the cloth as a pop-filter. Got a sock on my microphone and it works just fine.
There is, at least, one reason to regenerate Rust bindings from C header files, every time a *-sys crate is built: the generated Rust bindings are platform-specific. C types will be translated to different Rust types, depending on which platform is being targeted by the build. Also, the C preprocessor might define constants and types in different ways depending on platform-specific macros. For example, (1) the C type "long" will be translated sometimes to Rust i32, and other times to Rust i64. The only way I can think of avoiding this regeneration is by caching the bindings for every single supported OS/Architecture/ABI (i.e., target triple), and that doesn't seem like a sustainable idea, even if it would avoid the need for build dependencies and a C compiler tool chain.
The windows crate at least can know exactly what all the possible ABIs are, so it uses a macro that generates the correct extern decl for the current target. For other platforms, in theory you could do the same thing that bindgen does to figure out the in a proc macro, but... I'm pretty sure it actually just uses clang as a library to mostly compile the header down to the ABI that clang proper would use. Don't think we want to do that in a proc macro ...
@@SimonBuchanNz Windows is the easier case, because of how few ABIs it supports (5 or so). The *-sys crates I was talking about target more platforms than the x86_64-pc-windows-msvc ABI, including Linux, FreeBSD, MacOS, bare metal, etc.
Cool project, I have been trying to figure out using io_uring with hyper. But its IO model doesn't really map that well to zerocopy. You really need to let the io runtime/glue take over the buffers for maximum performance with zero copy, I believe. Because you can register pages with io_uring, so that they can stay memory-mapped by the kernel. Maybe this could be solved by an allocator interface (like the unstable `Alloc` trait) that hyper can use? This way you can allocate from the registered pool of memory. Also, when sending buffers you should hand over ownership to the kernel until it yields control of the buffer to the application again (TCP ACK? Or maybe the NIC copies it into it's own internal buffer, idk). And then I gave up😅 And is there even an async runtime that implements io_uring networking? I looked at glommio but it only seems to use io_uring for filesystem access
When working on the plane. How do you deal with cargo dependencies on a new project? One can use cargo fetch or cargo vendor for established projects, but how do you deal with that when experimenting on new projects? Local crates mirror?
You could also use libtest-mimic or similar to build a custom test harness that has test cases generated in runtime (from the central test registry you could have collected)
To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/fasterthanlime - You’ll also get 20% off an annual premium subscription.
Early access: fasterthanli.me/videos/h2spec-to-rust
Errata: (nothing yet!)
Gotta say, I really expected a betterhelp sponsorship just how, I'm happy it was Brilliant instead but unless that misdirect was on purpose to make fun of Better help sponsorships this was a pretty strange way to promote Brilliant?
There is an old rule for creating engaging stories; You are allowed one lie. You can make up butt-faced aliens climbing up from the earths core on jetski's, one lie is forgiven by your audience. Every other event must have a logical reason, an objective "yes, how else would you suggest to do this" causality. Your audience will not forgive more lies, and boo at the cinema screen and popcorn will fly around the room!
Amos wrote, acted in, and edited, a moving story with **zero** lies. 💪
Guys will literally do anything to avoid writing a unit test.
Cool, another HTTP implementation in Rust!
It's not a fasterthanlime video if I don't need to watch it twice in a row to understand what's going on 😎
still faster than reading his bible long articles lol
ahh if only ThePHD could have done their talk. goddamnit rust!!
Even Rust cannot escape TableGen
cranelift has its own DSL for generating machine instructions that does a whole ton of code generation. GCC does the same and so does LLVM (where TableGen is from). Maybe one day proc macros can replace this code generation case but that day is not today.
i've done some ungodly things with proc macros to the point where i tried to come back to it 2 days later and i had no idea how to follow the flow of my own macros. rust foundation please finish introspection!
The production value gets better with every new video! Even the sponsorship section was done really well.
wow... what an insane good skill to articulate your tree of thought. Impressive!
The purity of proc macros is a bit unfortunate even if understandable. A while ago I wanted to do some code injection into an asmjs app. I wanted to do this semi-automatically by annotating the rust library with proc macros and then parsing both my lib as well as the asmjs source and patching it based on metadata generated from the proc macros.
In the end I ended up using a feature flag in the proc macro that would either modify the injected function and pass in some extra values or generate calls to the inventory crate. This would then be collected into a json by using the example(?) binary type in cargo.
re: git hooks @ 14:11, aren't git hooks "disabled" by default? I thought that they are not committed to the repo and you have to copy them over manually (or use something like the `pre-commit` tool). Which makes sense, because they can run code without you knowing.
Now I see that you're using cargo-husky, so I guess you're actually referring to naughty contributors who don't run tests 😅
@fasterthanlime couldn't you have made a proc macro at that you call at the crate level on `httpwg` that iterates through each module looking for tests matching your criteria and writes an impl block for some type that builds up your test suite? that way it recurses properly without being stateful and you only have to write one proc macro. I know it would still have to parse a lot of syntax but you could probably internally just feed syn individual modules instead of the whole crate
Awesome video, super interesting as always! Can't wait for another fasterthanlime 40-minute-video-saga-classic when you have the funding and time! Put it on the backburner and there'll be an opportunity eventually :) I appreciated the mouth trumpet (tuba?) at the end with the cloth as a pop-filter. Got a sock on my microphone and it works just fine.
There is, at least, one reason to regenerate Rust bindings from C header files, every time a *-sys crate is built: the generated Rust bindings are platform-specific.
C types will be translated to different Rust types, depending on which platform is being targeted by the build. Also, the C preprocessor might define constants and types in different ways depending on platform-specific macros. For example, (1) the C type "long" will be translated sometimes to Rust i32, and other times to Rust i64.
The only way I can think of avoiding this regeneration is by caching the bindings for every single supported OS/Architecture/ABI (i.e., target triple), and that doesn't seem like a sustainable idea, even if it would avoid the need for build dependencies and a C compiler tool chain.
The windows crate at least can know exactly what all the possible ABIs are, so it uses a macro that generates the correct extern decl for the current target.
For other platforms, in theory you could do the same thing that bindgen does to figure out the in a proc macro, but... I'm pretty sure it actually just uses clang as a library to mostly compile the header down to the ABI that clang proper would use. Don't think we want to do that in a proc macro ...
@@SimonBuchanNz Windows is the easier case, because of how few ABIs it supports (5 or so). The *-sys crates I was talking about target more platforms than the x86_64-pc-windows-msvc ABI, including Linux, FreeBSD, MacOS, bare metal, etc.
Cool project, I have been trying to figure out using io_uring with hyper. But its IO model doesn't really map that well to zerocopy. You really need to let the io runtime/glue take over the buffers for maximum performance with zero copy, I believe. Because you can register pages with io_uring, so that they can stay memory-mapped by the kernel. Maybe this could be solved by an allocator interface (like the unstable `Alloc` trait) that hyper can use? This way you can allocate from the registered pool of memory. Also, when sending buffers you should hand over ownership to the kernel until it yields control of the buffer to the application again (TCP ACK? Or maybe the NIC copies it into it's own internal buffer, idk). And then I gave up😅
And is there even an async runtime that implements io_uring networking? I looked at glommio but it only seems to use io_uring for filesystem access
That little pink ear at 6:56 made me happy. I once had a lovely white cat that ALWAYS wanted to sit on top of people :)
cat hair flying around while he throws his hands around... there are only two kinds of people in this world I suppose. Cat vs Dogs. :)
No VS. Cats & Dogs. I love both 😊
have you ever thought about writing compilers or interpreters?
rustdoc's json output has saved me once before. Such a lovely feature to have :)
You have an amazing ability to de-mystify 100 things at a time for me and I love you for it
how do you learn about this concepts
When working on the plane. How do you deal with cargo dependencies on a new project?
One can use cargo fetch or cargo vendor for established projects, but how do you deal with that when experimenting on new projects?
Local crates mirror?
std only maybe?
Local crate hub?
The outro really caught me by surprise, really informative and interesting video
I understand the words individually, but I need to rewatch at least twice to understand them all together...
Big fan of using intermediate data formats for crimes. Beautiful work!
This is awesome. I absolutely love it.
You could also use libtest-mimic or similar to build a custom test harness that has test cases generated in runtime (from the central test registry you could have collected)
What monospace font are you using in the video?
“A series of unfortunate events” is such an understatement
Due that outro was first class!
Where did you get your shirt?
The bear necessities, the simple bear necessities...."
I don't usually sub before I check out a back catalogue, but with a description section that comprehensive I'm sold.
"I decided port it to rust" then "don't repeat yourself"
Okey okey
That was a great ad
I swear that "Life is too short for long builds" was a slogan for some product but for the life of me I can't find it.
wow
People also might not want to run "trust me bro" binaries on their dev environment...
Rust BTW😎
second
The constant over-gesturing is really difficult for me to deal with. I can basically only listen to the video and not watch it.
I'm the other way around 😂
autism is a spectrum my friend
Haha you would hate talking to me 😂
Interesting
third
first
Always impressive how much information you're able to fit into a (relatively speaking) short video, and still have a decent flow