21:30 Go doesn't actually have a typical stack and heap in the same way that C programs have. Their stack is more like a chain of 4k blocks. Whether an object is allocated on the heap or the stack does not depend on it being declared with new or without new, instead it's based on escape analysis.
Odin's memory allocation stuff is interesting. You can choose your allocator, and either pass it explicitly into a procedure or have it passed implicitly via the context. Seems to be a good way to let calling code override allocation logic if needed
It is not the case with Zig. In the Zig standard library, functions that dynamically allocate will require the user to pass an allocator. Since Odin has optional function parameters with default values, you can opt not to pass allocators to functions, and the functions will use the allocator from the caller’s context.
@@Irremnent I haven't used zig, but Odin's approach so far seems quite elegant, and from what I've read, quite a few people seem to prefer the ergonomics of Odin. Zig seems to get plaudits in other areas.
@constantitus it's not about crediting anyone for the idea, I have no idea where it originated. Given jai is nowhere near finalisation as a language or syntax, doesn't even have a name yet, and isn't publicly available, it doesn't seem likely many people will have experience of it. All languages are a blend of ideas from many sources. If you want to experiment with the context approach using a language that is essentially complete with finalised syntax, Odin seems like a reasonable way to do it.
Good interview guys. For any Go users who want to explore arenas, they were added to Go in 1.20 NOTE: "Unfortunately, the proposal to add arenas to the standard library is on indefinite hold due to the fact that they compose poorly with the language and standard library." I think it was experimental for 1.20 and maybe a version later, then removed
in odin I would say its basically standardised, the default allocator is a heap allocator, but the temp allocator is an arena. so most of the time I just allocate to temporary, and its pretty much like using a gc language.
im usimg arenas in a web framework Im working on for modern C called MACH (Mustach, Apline, C, Htmx), the user of the framework doesn't have to worry about memory management, pointers, buffer overflows, and has very simple primitives and api, etc. It's fully async and supports sqlite, libsql (turso), postgres, smtp, fetch http calls, background jobs, hmr, single binary production binary, and more
I should have example apps in a repo on my github written in mach in coming weeks with links to them live and running. Mach itself I'm hoping for a spring release.
Nic Barker, creator of Clay, also has talked about Arenas a lot - even recommending it as your default memory management technique in C. th-cam.com/video/9UIIMBqq1D4/w-d-xo.htmlsi=1IJ08ATHIhlSl8C6
Not only is the podcast great, but the comments are awesome! Super informative and add to the content. What an excellent corner of the internet you've fostered.
I have a thick, hard back book that dates to the 80s and describes the implementation of a C compiler (including the back-end code generator). The implementation is based on using an arena allocator - which is a model of mem mgt that works perfectly with this compiler. So the concept of arena mem mgt is almost as old as C and has been a thing with C programmers like forever. In the early 90s I devised a drop in replacement of the malloc allocator and its arenas were collections of fixed-size blocks and a given arena was based on object size (object data type). Used a red-black tree to manage the free list. It was used with Aldus PageMaker (desktop publishing layout), which was written in C++ and allocated thousands of discreet objects. This custom mem mgr sped it up 20% across the board and some benchmarks were 70% faster. It’s really weird to hear a podcast like this and something like an arena mem mgr is talked about as though it’s a relatively recent, novel thing
I’m not sure if it was in the edited video, but in this conversation we did explicitly talk about that. There was a question about “why don’t people use them?” And the answer is that they aren’t new. They’ve been used for decades. But information transfer across generations is lossy, and I’m trying to help communicate the lessons I learned from older programmers about how you simplify things.
33:03 Not only pointers are not stable on slices, but ALL pointers are not stable in Go. You cannot convert a pointer to a uint and expect it to be stable. The GC can take the liberty to move things around, and it will updated your pointers behind your back, but if you had converted a pointer to a uint, the GC will not know about it, so it will not update it.
It's almost like that all the abstraction we have been using for years as some sort of good thing for the last 30 years has gone full circle to end up having all sorts of memory management issues. When you are working at low-level you have to think about this stuff from the get-go so you plan for it. I'm wondering if most of this stuff was already solved, we were simply told pass pointers down don't return them, unless you have really thought about what you are doing.
I dont know if this is a valid way to do them but I normaly have the arena return a "handle" instead of the raw pointer and like use the handle to access the data from that arena. I find it easier this way since I can resize the block of contigious memory without worrying about invalidating pointers. Basically a handle in this case is just an index and the arena is like a dynamic array.
If you are using the virtual memory technique he mentions you don't have to worry about invalidation either. There are many ways to go at it. Also don't know if I just made it up, be the prefectcher can know you are using a pointer and have that memory ready for you, not with a handle tho. Correct me on this last thing, the idea just popped into my head
If you watched from 31:42, pointer invalidation is not a problem because you aren't copying or moving chunks of memory when you expand. You can create another block then link in a linked list format.
@@wulf3n773 yeah but im trying to avoid having linked lists. I need one free call at the end of the lifetime and using linked lists for this makes this impossible.
@@joandotvandamme3340 I dont use the technique he mentioned. I like using handles instead of pointers, especially when I need to store some data that points to some other data in that arena. I use u8 and u16 as the pointer to the node / data that the structure instead of raw pointers which are bigger. I offset the handles by 1 so zero gets to be used as null. Then at the end of the lifetime, It's just 1 free call and I'm done. I dont know about prefetching. I mean the block is contigious, and the handles are just indexes. Idk for sure if this is efficeint or not.
Damn. I never knew about this guy. Ryan seems really knowledgeable and man he explains in such a great manner. I wish he was a Educator content creator as well.
Well the guys spoke a lot about malloc and free, but I don't think they mentioned that all of these calls have overhead. Basically malloc and free has to call to the OS, and the OS has to do it's thing to set aside the memory block, by doing some internal tracking things...like adding a small header right before that memory block. And for few big allocations it's fine I guess, but for many small ones(like what they mentioned), it takes toll on performance. Memory allocators like arena do this overhead OS calls once and from there on, an allocation at the arena is basically returning the "free" pointer inside the arena block and then just advancing that pointer forward by the size of the "allocated" block. So the performance benefit is obvious.
malloc doesn’t need to call the kernel in any way that an arena doesn’t. Good malloc implementations also call the OS sparingly, and for only large blocks. The problem is that the generality of the lifetimes & sizes enforce a number of constraints which have a high cost, both in performance & complexity, if paid for on every dynamic micro allocation.
@@RyanFleury well sure but just as you said, because it's a "one size fit's all" kind of a thing I would imagine there is a bit of an overhead that wouldn't be a thing in a custom allocator. Like checking the size , and deciding how to handle it, since as you said malloc can allocate big chunks and small chunks. I mean for a single allocation I would imagine that malloc would be better than an arena allocator, but after that, all other allocations are almost free for the arena, not to mention that the allocations are right next to each other on the block, while malloc cannot guarantee that. So I would imagine higher chances of a cache hits than misses with arenas. But yeah it's a cool topic. Especially if we can get better performance.
1:03:13 I am writting a custom web server in C, and I use arenas for memory management, I have a youtube video about it th-cam.com/video/NJhtKc3nnsQ/w-d-xo.html
Yeah and comparing them to arenas makes no sense, they are way different. In fact, there was a proposal for arenas in Go which was worked on but in the end was not implemented.
For my game engine project I made a wrapper for malloc/free (and aligned versiom) and have atomic counters for different memory categories so I can see if anything leaked when the program ends. I use pools for most types of objects so there aren't too many small allocations.
The first 30 minutes of the video could have been done in 5 minutes if they had done some code examples and actually tried to explain the concept of arenas. Instead, the host chose a wired podcast format where the host asks wired questions. This made the video difficult to follow and understand. As a result, the video is not very informative for someone who is new to the concept of arenas. Also the title is bad clickbait.
th-cam.com/video/UeJPyuVxL-o/w-d-xo.html Jai standardizes arenas into the language itself. In addition to your standard malloc/free equivalent allocator, you have what's called the 'temporary' allocator, which is just an arena that's always available from everywhere, and has the semantics of 'temporary' or not-long-lived. Up to you to release the arena. Extremely useful for things like the standard string 'library' - often basic operations with strings require annoying, small, yet dynamic allocations to be done, which if you call malloc/equivalent to do, it sucks, and passing an explicit arena of your own is cumbersome (imagine doing this for every string lib call you do). If you write libraries yourself it's nice to have available, because if you need an allocation that is semantically 'small-ish' and short-lived, you can just do it and not worry about it.
21:30 Go doesn't actually have a typical stack and heap in the same way that C programs have.
Their stack is more like a chain of 4k blocks. Whether an object is allocated on the heap or the stack does not depend on it being declared with new or without new, instead it's based on escape analysis.
Care to share any references? Thanks!
GO's mess with the stack is the reason cgo is slow.
@@abiiranathan the escape analysis stuff seems like quite a clever idea, though I don't know if it has any unwanted consequences
Odin's memory allocation stuff is interesting. You can choose your allocator, and either pass it explicitly into a procedure or have it passed implicitly via the context. Seems to be a good way to let calling code override allocation logic if needed
It's the case with Zig as well
It is not the case with Zig. In the Zig standard library, functions that dynamically allocate will require the user to pass an allocator. Since Odin has optional function parameters with default values, you can opt not to pass allocators to functions, and the functions will use the allocator from the caller’s context.
@@Irremnent I haven't used zig, but Odin's approach so far seems quite elegant, and from what I've read, quite a few people seem to prefer the ergonomics of Odin. Zig seems to get plaudits in other areas.
Pretty sad to see odin being credited the "context" idea considering it's just copied from jonathan blow's language
@constantitus it's not about crediting anyone for the idea, I have no idea where it originated. Given jai is nowhere near finalisation as a language or syntax, doesn't even have a name yet, and isn't publicly available, it doesn't seem likely many people will have experience of it. All languages are a blend of ideas from many sources. If you want to experiment with the context approach using a language that is essentially complete with finalised syntax, Odin seems like a reasonable way to do it.
I see Ryan and I click like
Love to see it
Good interview guys. For any Go users who want to explore arenas, they were added to Go in 1.20
NOTE: "Unfortunately, the proposal to add arenas to the standard library is on indefinite hold due to the fact that they compose poorly with the language and standard library."
I think it was experimental for 1.20 and maybe a version later, then removed
in odin I would say its basically standardised, the default allocator is a heap allocator, but the temp allocator is an arena. so most of the time I just allocate to temporary, and its pretty much like using a gc language.
im usimg arenas in a web framework Im working on for modern C called MACH (Mustach, Apline, C, Htmx), the user of the framework doesn't have to worry about memory management, pointers, buffer overflows, and has very simple primitives and api, etc. It's fully async and supports sqlite, libsql (turso), postgres, smtp, fetch http calls, background jobs, hmr, single binary production binary, and more
that's interesting I love working with HTMX and Alpine. Adding C to that combo sounds cool. Is the project public ?
I should have example apps in a repo on my github written in mach in coming weeks with links to them live and running. Mach itself I'm hoping for a spring release.
@@nightshade427 That's awesome, would love to contribute to it. Even though I am not very good at C programming, would love to learn.
Im planning to have an example repo of apps using mach in the coming weeks, hoping for a spring release of mach itself
it's kind of a bad name, it will be impossible to find on search engines
Nic Barker, creator of Clay, also has talked about Arenas a lot - even recommending it as your default memory management technique in C. th-cam.com/video/9UIIMBqq1D4/w-d-xo.htmlsi=1IJ08ATHIhlSl8C6
I love Nic's videos, he explains things so well, I code C in a very similar way, makes things so much easier
I think both Nic and Ryan were initially introduced to the concept from handmade hero.
Not only is the podcast great, but the comments are awesome! Super informative and add to the content. What an excellent corner of the internet you've fostered.
I have a thick, hard back book that dates to the 80s and describes the implementation of a C compiler (including the back-end code generator). The implementation is based on using an arena allocator - which is a model of mem mgt that works perfectly with this compiler. So the concept of arena mem mgt is almost as old as C and has been a thing with C programmers like forever.
In the early 90s I devised a drop in replacement of the malloc allocator and its arenas were collections of fixed-size blocks and a given arena was based on object size (object data type). Used a red-black tree to manage the free list. It was used with Aldus PageMaker (desktop publishing layout), which was written in C++ and allocated thousands of discreet objects. This custom mem mgr sped it up 20% across the board and some benchmarks were 70% faster.
It’s really weird to hear a podcast like this and something like an arena mem mgr is talked about as though it’s a relatively recent, novel thing
I’m not sure if it was in the edited video, but in this conversation we did explicitly talk about that. There was a question about “why don’t people use them?” And the answer is that they aren’t new. They’ve been used for decades. But information transfer across generations is lossy, and I’m trying to help communicate the lessons I learned from older programmers about how you simplify things.
Whats thr name of the book about implementing a C compiler?
Odin's creator, Ginger Bill, has a series of articles on memory management. Very good intro into the topic.
Odin itself is very nice to look at.
Really interesting to learn new perspectives on already established things.
33:03 Not only pointers are not stable on slices, but ALL pointers are not stable in Go. You cannot convert a pointer to a uint and expect it to be stable. The GC can take the liberty to move things around, and it will updated your pointers behind your back, but if you had converted a pointer to a uint, the GC will not know about it, so it will not update it.
It's almost like that all the abstraction we have been using for years as some sort of good thing for the last 30 years has gone full circle to end up having all sorts of memory management issues. When you are working at low-level you have to think about this stuff from the get-go so you plan for it. I'm wondering if most of this stuff was already solved, we were simply told pass pointers down don't return them, unless you have really thought about what you are doing.
You guys reinvented memory pools , nice :)
My opinion on a celebrity has never changed so drastically. Seth Rogan has really turned things around for the better
Hahahahahahahahahahahahahahaha
@@RyanFleury Lmao sense of humor on point. That's a sub to your page for that 😎
🤣
I dont know if this is a valid way to do them but I normaly have the arena return a "handle" instead of the raw pointer and like use the handle to access the data from that arena. I find it easier this way since I can resize the block of contigious memory without worrying about invalidating pointers. Basically a handle in this case is just an index and the arena is like a dynamic array.
If you are using the virtual memory technique he mentions you don't have to worry about invalidation either. There are many ways to go at it. Also don't know if I just made it up, be the prefectcher can know you are using a pointer and have that memory ready for you, not with a handle tho. Correct me on this last thing, the idea just popped into my head
Also, by chaining arenas like a linked list instead of copying into a newer bigger region, pointer also do not get invalidated
If you watched from 31:42, pointer invalidation is not a problem because you aren't copying or moving chunks of memory when you expand. You can create another block then link in a linked list format.
@@wulf3n773 yeah but im trying to avoid having linked lists. I need one free call at the end of the lifetime and using linked lists for this makes this impossible.
@@joandotvandamme3340 I dont use the technique he mentioned. I like using handles instead of pointers, especially when I need to store some data that points to some other data in that arena. I use u8 and u16 as the pointer to the node / data that the structure instead of raw pointers which are bigger. I offset the handles by 1 so zero gets to be used as null. Then at the end of the lifetime, It's just 1 free call and I'm done. I dont know about prefetching. I mean the block is contigious, and the handles are just indexes. Idk for sure if this is efficeint or not.
Damn. I never knew about this guy. Ryan seems really knowledgeable and man he explains in such a great manner. I wish he was a Educator content creator as well.
It's so good that Arenas are becoming more mainstream.
I'm surprised it has taken this long
Shout out to Ginger Bill for his memory allocation articles
Well the guys spoke a lot about malloc and free, but I don't think they mentioned that all of these calls have overhead. Basically malloc and free has to call to the OS, and the OS has to do it's thing to set aside the memory block, by doing some internal tracking things...like adding a small header right before that memory block. And for few big allocations it's fine I guess, but for many small ones(like what they mentioned), it takes toll on performance.
Memory allocators like arena do this overhead OS calls once and from there on, an allocation at the arena is basically returning the "free" pointer inside the arena block and then just advancing that pointer forward by the size of the "allocated" block. So the performance benefit is obvious.
malloc doesn’t need to call the kernel in any way that an arena doesn’t. Good malloc implementations also call the OS sparingly, and for only large blocks. The problem is that the generality of the lifetimes & sizes enforce a number of constraints which have a high cost, both in performance & complexity, if paid for on every dynamic micro allocation.
@@RyanFleury well sure but just as you said, because it's a "one size fit's all" kind of a thing I would imagine there is a bit of an overhead that wouldn't be a thing in a custom allocator. Like checking the size , and deciding how to handle it, since as you said malloc can allocate big chunks and small chunks.
I mean for a single allocation I would imagine that malloc would be better than an arena allocator, but after that, all other allocations are almost free for the arena, not to mention that the allocations are right next to each other on the block, while malloc cannot guarantee that. So I would imagine higher chances of a cache hits than misses with arenas.
But yeah it's a cool topic. Especially if we can get better performance.
I was just reading Ryan's arena blog after hearing about the concept from Nic Barker and then this video pops up
you can use alloca or vla to do dynamic stack allocations, I wouldn't recommend it though
You can also allocate an arena on the stack by giving it a backing stack allocated byte buffer and disallowing growth. It's essentially a sane VLA
Would love some more Lowe level courses on bootdev
1:03:13 I am writting a custom web server in C, and I use arenas for memory management, I have a youtube video about it th-cam.com/video/NJhtKc3nnsQ/w-d-xo.html
You botched the explanation of slices in Go. You could have just said they are dynamic arrays.
Yeah and comparing them to arenas makes no sense, they are way different. In fact, there was a proposal for arenas in Go which was worked on but in the end was not implemented.
For my game engine project I made a wrapper for malloc/free (and aligned versiom) and have atomic counters for different memory categories so I can see if anything leaked when the program ends. I use pools for most types of objects so there aren't too many small allocations.
The first 30 minutes of the video could have been done in 5 minutes if they had done some code examples and actually tried to explain the concept of arenas. Instead, the host chose a wired podcast format where the host asks wired questions. This made the video difficult to follow and understand.
As a result, the video is not very informative for someone who is new to the concept of arenas.
Also the title is bad clickbait.
Blockchain technology
🤍
th-cam.com/video/UeJPyuVxL-o/w-d-xo.html Jai standardizes arenas into the language itself. In addition to your standard malloc/free equivalent allocator, you have what's called the 'temporary' allocator, which is just an arena that's always available from everywhere, and has the semantics of 'temporary' or not-long-lived. Up to you to release the arena.
Extremely useful for things like the standard string 'library' - often basic operations with strings require annoying, small, yet dynamic allocations to be done, which if you call malloc/equivalent to do, it sucks, and passing an explicit arena of your own is cumbersome (imagine doing this for every string lib call you do).
If you write libraries yourself it's nice to have available, because if you need an allocation that is semantically 'small-ish' and short-lived, you can just do it and not worry about it.
Sounds like Odin