CppCon 2018: Louis Dionne “Compile-time programming and reflection in C++20 and beyond”
ฝัง
- เผยแพร่เมื่อ 5 ก.ค. 2024
- CppCon.org
-
Presentation Slides, PDFs, Source Code and other presenter materials are available at: github.com/CppCon/CppCon2018
-
Compile-time programming and reflection have been getting a lot of attention recently. Indeed, from Herb Sutter's well-known Metaclasses proposal to the newly created Reflection TS, a lot of effort is being put into solving the general problem of programmatic code generation and program introspection. More recently, the C++ Standards Committee has also been evaluating and adopting proposals that vastly expand the realm of constructs available at compile-time, in constexpr functions: new-expressions, try-catch blocks, virtual functions, some standard containers, and more.
For most people, it is unclear how all these features, whether exploratory or voted into the Draft International Standard, relate together. Without being active in the Committee, it can be difficult to see the big picture, the unifying vision driving all these changes. Fortunately, there is one, and this is what this talk is about.
We will go over these features and explain how they might interact with each other by boiling them down to their essential parts. We will present how different use cases for compile-time programming will be solved in C++20, and how even more use cases can be unlocked in the future. Attendees will leave this talk with an understanding of the improvements to constexpr planned for C++20, of what's needed to unlock more advanced use cases and an unifying vision for how to get there.
-
Louis Dionne, Apple, C++ Standard Library Engineer
Louis is a math and computer science enthusiast who got swallowed by the C++ monster when he was a naive, unsuspecting student. He now works for Apple, where he is responsible for libc++, the Standard Library shipped with LLVM/Clang. He is a member of the C++ Standards Committee and of the Boost community, where he authored the Boost.Hana metaprogramming library.
-
Videos Filmed & Edited by Bash Films: www.BashFilms.com
*-----*
Register Now For CppCon 2022: cppcon.org/registration/
*-----*
![ช่วยพาฉันผ่านเวลา - วสันต์17 [Official MV]](http://i.ytimg.com/vi/HFnntWoqc54/mqdefault.jpg)
He makes this seem soo understandable and easy that its surprising it wasnt in the language already. !
incredible talk. i love how things are lining up over the years. especially "for..." is news to me, but looks very powerful to get rid of tons of repetition in the code.
Awesome presentation. The future looks promising! @Louis Dionne, it really like you love your job :)
You're making an amazing talk here Louis, as always.
Amazing talk. I think this stuff is very powerful and long overdue.
Incredibly exciting to see C++ start catching up with modern languages on compile time evaluation, reflection and hygienic macros. One day I'll have nothing to complain about any more! xD
"Please reserve questions to the end." Audience: "question question question question" chrissakes people learn to listen
Brilliant Louis always.
39:20, that's exactly what I've been waiting for.
Great question at 1:04:00. "Are constexpr expressions evaluated when a module is compiled or when the interface is imported by another module?" I was thinking this myself. My assumption is that it would be compiled into the module so it's included only once by that translation unit. However, expressions can be templatized and exposed to other modules. In those cases I would expect the expression to be evaluated at the point of use by that external module. Can anyone offer a concrete answer?
I have full confidence Louis will get it right, I've been using hana and it's an absolute pleasure compared to type based TMP.
So basically, the standards committee saw the compile-time execution available in D and Jai, and thought, "hey! We can make a worse version of that!"
Don't be an asshole. All langauges borrow things from each other. Not a single person has the best ideas. Alot languages have people working on it with sometimes great idea's. It's an evolution of all languages. It benefits all languages to share ideas. This is evolution. D probably also have borrowed ideas from other genius minds.
Regarding your very last statement in the video, that standardizing something to emit warnings (call it logging) during compile time isn't something that the committee would be interested in, I think it should be. We already have static_assert which breaks compilation with a message in case a predicate is not true. Why not have something that is always output. I think we'll need it. Call it static_log or something. A constexpr iostream "static_cout" is probably too much, but a static_log which takes a level and a constexpr string as a parameter - why not?
mfkman May be one can use static_cout for debugging as well.
Great talk!
Thanks for the great talk. Not in the committee, not really optimistic... but guess will use it in C++27 somehow.
Yeah, looks like even coroutines don't make it into C++20. And i would not bet money on concepts or modules either, given that no compiler implemented all of them and so we don't have enough feedback.
C++ 26 / C++29 ?
I worked alot with constexpr in the recent months and I can tell you we are far away from a perfect implementation. I can count atleast 4 breaking compiler bugs I encountered in msvc, the most recent one was that he believed that I left a for loop which I didnt. Clang was better but the error messages were horrific if there were any at all.
Well it is fun because it is new but actually implementing whole containers seems such a monumental task at the moment, that I'm not too optimistic about the near future.
Considering compiler and the code it generates may be running in different environment, if we are calling `new int[N]`, how does compiler know the target machine has enough memory to allocate the storage for it?
The list at 3:46 is so sweet :-)
hell yea, finally slowing down template hell, and staying with the core language. please continue with minimizing having to use template hellish language to achieve compilation time functionality. loved the constexpr!.
Great talk
I'm not sure how you can debug all the compile generated code.
Will we see this getting into the standard in Bjarn's lifetime ?
A constexpr unordered_set or unordered_map could actually use a perfect hash function. :-)
GNU Perf has been around for a very long time, and it isn't all that slow. Having that shipped as a header-only library for C++ would make it a lot easier to use and greatly simplify the build process.
www.gnu.org/software/gperf/manual/gperf.html
I guess that would depend on the size of your set. :)
Pretty easy to do in D programming language, i.e. github.com/skoppe/perfect-hash , it can be used both at runtime and compile time, directly from the compiler.
why was `constexpr` ever defined to mean "maybe constexpr"?
otherwise you'll have to write every constexpr function twice
And we still needed consteval >_>
1:02:20 is a good question, would be more work for library implementers but a valid and more minimal implementation
As already mentioned by Louis Dionne, I see a lot of potential in serialization and pre-compiling data. When starting with game programming it was a chore to implement all the binary serialization for transmitting objects over the network. I know there are things like protobuff, but just letting your data class inherit from a "BinarySerializable" interface with meta-class code behind it, that generates the functions for you, would be REALLY nice. And we could get rid of the Macro hacking in engines like Unreal (which was actually the point that drove me away from it).
Another question: Could we constexpr std::fstream? :-p
Then I could read files at compile-time and generate new code from it. In a way utilizing the compile-time abilities of C++ to make a compiler itself. E.g. Reading a JSON file and generating classes from it.
Not steal from Louis's great talk (who happens to be a friend of mine), almost everything in this topic already exists in the D programming language, has been used successfully for years, without the need of a special keyword like 'constexpr'. To answer your fstream question, assuming that there is a file named 'key_values' with the following content:
white beyaz
black siyah
the following D program would generate a hash table at compile time:
// D code:
// This table is constructed at compile time as a result of executing the following lambda
enum string[string] table = (){
import std.algorithm : splitter, map;
import std.range : chunks, array, assocArray;
import std.typecons : tuple;
return import("key_values") // contents of file read as string at compile time
.splitter
.chunks(2)
.map!array
.map!(kv => tuple(kv[0], kv[1]))
.assocArray;
}();
void main() {
// Accessible both at compile time ...
static assert(table["white"] == "beyaz");
// ... and at run time
assert(table["black"] == "siyah");
}
Ali
Just for your information. D programming language, has this feature. import("filename.txt"), will read the file at compile time and return as a string. you can do whatever you want with it then, using D's compile time evaluation functionalities, i.e. generate classes, code, tables, enums, whatever. Parsing protocol buffer description at compile time in D is rather easy, and then you can just generate the interfaces and code to do parsing. Crazy, but it does work, and in fact works very fast.
while i think this is going to be a pain in the ass to modify the ast of cpp source code, i do agree with the overall plan presented at 6:30. It definitely makes sense to me.
I hear a bunch of talk about constexpr, but I don't hear that much about why this is a good thing. A couple of real life use cases might be a good idea if you want to sell this.
Generally I would say, any computation that can be done at compile-time is better than at runtime. Runtime is the place everything should be done already.
My real-world example is a JSON serializer. I have a class, that contains key-value-pairs of the serialized JSON string. But to what I want them to serialize to is known at compile-time:
void deserialize (SerializationObject& serObj) {
var1 = serObj.get ("var1");
var2 = serObj.get ("var2");
var3 = serObj.get ("var3");
var4 = serObj.getArray ("var4");
}
So my "get" function looks like this:
template
T get (const std::string& key) {
if constexpr (std::is_same::value) {
std::string value;
auto& it = values.find (key);
if (it == values.end ()) {
return value;
}
value = it->second.value;
return value;
} else if constexpr (std::is_same::value) {
bool b;
std::string value;
auto& it = values.find (key);
if (it == values.end ()) {
return b;
}
...
And the "if constextr" makes it so, that only the one needed is compiled into the binary (at least to my knowledge).
This can be used for generating a pre-sorted read-only container holding information you know at compile time. Instead, the compiler bakes the pre-sorted container into the executable. Codebases with containers that fall into this pattern should be able to make use of constexpr with minimal changes. One side effect of this is reduced program startup times. Mark Elendt spent time in his CPPCon 2018 talk "Patterns and Techniques Used in the Houdini 3D Graphics Application" explaining how reducing startup times is one of their real world goals.
obvious uses are serializing/deserializing, parsing&compiling regular expressions, offline computations, reducing TMP usage in some cases.
it's a good thing, because it can potentially yield the fastest possible code.
Is constexpr! really necessary ? It seems you can achieve the same with static_assert(std::is_constant_evaluated());
static_assert takes a constant expression, so that would always evaluate to true. Its the same reason the if condition in vector::clear is not if constexpr(), as he explained when answering a question about that.
why caan't we just remove the constexpr keyword and make all code constexpr by default unless it's impossible? Maybe because it will make compile time much higher? hmmmm
This feature is called compile time function execution (CTFE) in D, without a special keyword. It does not add to compile-time speed more than what is already needed as the compiler uses it only when the result of an expression is needed at compile time e.g. literal values, template instantiation, etc.
That's awesome but often times the result is not needed at compile time; you just need it precalculated to avoid the overhead at runtime. Good to know about the feature in D though thanks!
I personally think that true dynamic allocations should not be allowed in a constant expression. No reason however, why placement-new should not be allowed. However, if we allow dynamic allocations in a constant expression If a constant expression allocates memory, it should either pass to Static-promotion, or has to be deallocated before the end of the constant expression. However, containers should be constexpr-ified, so that Constexpr Allocators can exist.
How would you implement custom allocators if you, as a developer cannot constexpr new? Your approach seriously restricts yourself to ONLY using STL containers or custom containers relying on std::allocator.
I said that placement new should be allowed (with restrictions).
A constexpr allocator would store Ts in internal uninitialized memory (with blocks aligned to T, and the same size as a T), and construction operations would use placement new.
constexpr bool isFirst = true;
constexpr auto _2nd = true;
That reflection syntax is very verbose. I kinda hoped that we use the dollar symbol as a new operator for accesing reflection features instead of having to rely on std::reflect or reflexpr, it'd make reflection in the code so much easier to spot and easier to read.
operator $ : Get reflection data of object directly to the right of it
.$ : Get reflection data from member
->$ : Get reflection data from pointed-to object
struct reflected Foo{int x; int* y;}; //struct declaration for Foo. "reflected" keyword says compiler should create reflection data/metadata for this struct, which can then be accessed via $
struct Meow{ float y;}; // no reflected keyword, no metadata will be created for this struct and no reflection on this class will be possible
Foo foo;
Meow meow;
$foo //get reflection data from foo
foo.$x // get reflection data from foo.x via new .$ operator
foo.$x.get_type() // get foo object, then grab reflect data from x member and get its type from the metadata
foo.$x.set_value(10) // set value of foo.x to 10 via reflection
foo.$bar.get_member("baz") = nullptr //get foo object, from that grab bar metadata, search for a bar member named baz as object and set its value to nullptr
foo->$y.get_type() //get foo object, get metadata of object pointed to by y and get its type
std::vector memberdata = $foo.get_member_data() // returns a std::vector with metadata of the object foo and its members
$foo.$get_member("x").get_type() // Get foos metadata, search for member named x, get x metadata and get its type
$meow // error, we didn't declare Meow as a reflectable class
I think Herb Sutter had the same idea initially with his meta-classes proposal and then the committee intervened. I could support an idea like this
he mentions it in the talk: iiuc it is currently implemented as TMP code just to see what functionality is needed. syntax decissions come later. i guess this is about making something work in a compiler with minimal modifications for now.
@@hansiraberAh, thank you for the info, i didn't catch that while watching the video.
@@OperationDarkside Herb Sutters proposal is the reason why i wrote my comment in the first place. :) I just agree with him thought that it'd be a really good idea to use a special operator for reflection to make it stand out more from the "non-reflection" code.
If you don't have exceptions in constexpr context, you don't need the try-catch in std::vector as they are all about exception guarantees. What you need is separate declarations for std::vector..
Or just admit C++ exceptions are bad and remove them from the standard library ;)
maybe in c++800, they will manage to achieve what smalltalk had achieved 40+ years ago.
maybe.
understandable 'boilerplate' code at 2:45 is too primitive, unreadable cryptography at 42:30 is way better (no)
I'm not convinced that running user-written C++ code inside the compiler is better than spawning python to do this (as he does now).
why?
Python are macros can generate arbitrary code that can be ill-formed. This can lead to very cryptic error messages during compilation. The proposed interface doesn't allow to inject arbitrary code, so it will be correct code (even though it can still be wrong)
Ah, the good old "it compiles so it must be bug-free" philosophy. Python runs in its own process and is specifically designed to compile and run quickly. C++ only does the last of those, and while it may do that better than Python it more than makes up for it in the compilation phase.
First, errors are localized and the compiler gets a hint of what you're trying to do, allowing it to associate errors to the code that does the generation (where the actual bug is), rather than the generated code.
Second, it's one external dependency less over which the standards committee and compiler writers have absolutely no control whatsoever. Using an external tool to add this feature is unacceptable.
Third, it greatly simplifies the build setup. Don't tell me you've never spent days debugging only to realize the build system had cached the generated stuff and used an old version.
Fourth, _you can still go the python route or use some other code generation system._ This new feature provides more options to choose from. It is not at all intended to entirely replace existing systems.
@@mozismobile I see your playing the advocate of the devil to get your answers. Nice tactic.
C++ is fucked ^^
And C++ continues to get more complicated, more verbose and less and less user friendly.
???
And well paid.
Then pls just go program something else. I think you should leave C++ and just move on to C# or Java. These are nice C++ programs that could be usefull for you?
"I work at Apple".
Instant downvote. :-þ
Apple pays him to work on Clang. Even if you don't like Apple, he benefits us all reguardless of who is paying him.
Yep. It's good. I like Swift too.