I don't think memory protection is part of the C specifications, it's more of an OS thing. This is also doable in any language that can call OS APIs. On Windows it's VirtualProtect (or the lower level NtProtectVirtualMemory)
4:44 Be careful! sysconf returns a 'long' data type which can overflow that declared 'integer' pagesize data type, under certain conditions. Recommended security patch: static constexpr long kPageSize = sysconf (_SC_PAGE_SIZE); Then, if kPageSize fits within the INT_MIN and INT_MAX int data type range, then assign it to the local variable.
I remember your lesson about making C code more abstract, we can use this for structures to make their data unchangeable after iniciation. Unless we make special function which will make it possible
ive delved into it a little bit; it can be intimidating, but there are enough analogues between the win32 & posix apis to get your feet wet without getting too uncomfortable. there’s even some aspects of win32 that i think are superior to posix, namely how it has asynchronous i/o built in and immediately available as an option. the biggest downsides for me are wrangling w/ COM or WinRT - its possible, but difficult in c; its more manageable in a language like c++ - and the at-times excessive amount of extra parameters. msdn & online code samples are your friend here. the other two pain points i have are more subjective &/or nitpicky; using wide strings (aka utf16) for unicode isn’t too bad, since stdc does come with wide string analogues to the normal narrow string utils out of the box; and writing code which is backward compatible with yet still mostly functional in older versions of windows is annoying, though that mostly holds true for every other operating system so 🤷
Microsoft hasn't worked on a C compiler in 25 years. Their latest C++ compiler happens to support compiling C files. MSVC is a C++ compiler with a C++ linker. Also, GCC is written in C++ since 2010.
@@anon_y_mousse I'm talking about at one point in the video where he uses a Linux or posix only function to allocate aligned memory. I'm wondering if the std library one could be used instead
Great! finally I can make own ram drive. What happens when program exists? Would it be still marked read-only? Say one program initialized ram drive and stayed resident in memory. How could another program access/modify the content of proposed ram drive?
Al that memory is "owned" by the process running this code. After exiting, the process will be cleaned up/destroyed by the OS. The memory will become unclaimed memory and the rights/protection is removed and set to the "protection" of unclaimed menory. It will result in a segmentation fault when accessed.
Just a quick question is there any limitation on branching out else if's (or, similarly, case's on switch) ? i.e., if (main cond).. else if (cond 1); else if (cond 2); .. else if (cond n). max number of lines/statements, n? Any thoughts? Thanks,.
Would you recommend this for ephemeral encryption keys? Can the memory page still be swapped out? Is it protected against malicious software in kernel-space; for example, root-kits? POSIX is just an API front-end, right? Is an OS's POSIX sub-system verified by independent groups? Can we ever be sure whats happening to these 'protected' spaces?
@@JacobSorber Ever since i read Ken Thompson's 'Reflections on Trusting Trust' I've been fascinated with the idea of protecting a user-space program from a malicious kernel. Whitfield Diffie, Martin Hellman, and Ralph Merkle devised brilliant ways to exchange cypher keys over insecure channels. I still think there's hope for a protected user-space.
Hi! Im new to C programming and I need to make a program which constantly shows in screen the word that the user types and can be stopped by the key word "END" like this: Type a word: "hello" Word is hello Type a word: "bye" Word is bye Type a word: "END" - It is for a class exercise so it would be amazing if we could use simple functions to make it. Thanks for reading 🙏
@@JacobSorber Thanks! For some reason I imagined that memory addresses that a program gets wouldn’t typically start with zero. I don’t have a justification for thinking that way though.
@@bob_kazamakis I know but it should be taught as a habit so new programmers don't forget. And especially if we are trying to deal with security issues.
There is a group of programmers that do not cleanup at the end of the program because the OS will do it anyway. No constructs like destructors, so technically no need to free/cleanup. The cleanup takes up time and codespace and can even break things due to bugs in the cleanup code I think it is always better to proper cleanup when the code is subject for update or reuse. In this case neither will, so for the simplicity of the demonstration code, better leave out the cleanup.
If you're referring to C23's `constexpr`, then it might help. However, it can only guarantee that the data is usable at compile-time. There is no guarantee that the data is stored in read-only memory (in the OS's perspective).
Just had an idea for protecting against "buffer" overrun/underrun, for starters instead of handing the allocation directly to the requester, instead hand over a object that reference the memory by index. The functions for read/write would be similar to the file functions and when handing over the pointer a reference count should be updated in a different page. The allocation itself would always be set to read only when this reference count is greater than 0, serving as a lock against writing. When it's 0 then the protection is removed from the page and the write calls can go ahead without issue. Here's a VERY rough example in code (with some assumed symbols): int main(void) { #define HELLO "Hello World" #define BYE "Goodbye World" /* Ensure we cannot accidentally write more than we need, the 0 indicates we do NOT want this memory to auto grow, the remaining 1s mean pad both the front and back of the memory with null bytes of the size mentioned where sizeof(char) is, this ensures we can always pass the string into any function that expects a null byte */ int strmem = mem_calloc( sizeof(HELLO), 0, sizeof(char), 1, 1 ); /* The 0 indicates that no, the pointer will not be handed to iconv() or some other 3rd party function that does not expect the custom interface */ char *str = NULL; /* This could segfault under different circumstances but on this occasion the count is at 1 so an exclusive lock is possible */ str = memputraw( strmem, STR, sizeof(STR) ); /* This will run without issue as it only reads the memory */ puts( str ); /* This will segfault because the mem_getptr( strmem, true ) was not called */ str[0] = 0; /* Assuming we didn't do the line above then this would start writing "Goodbye World" from position 0 (as indicated by the 0) however only "Goodbye Wor" will actually get written because we specified during allocation we do not want the memory to grow */ str = memputabs( strmem, BYE, sizeof(BYE), 0 ); puts( str ); /* 0 the memory 1st then release it, this will protect against hackers using the pointer after it was released to glean sensitive info */ mem_zero_and_free(strmem); return EXIT_SUCCESS; }
@@anon_y_mousse That was just an easier to get across example, I'm sure there's plenty more cases where null terminated/prefixed buffers are handy, for example a list of IDs where id 0 is invalid, or linked lists that use offsets from the buffer root instead of addresses. Just give it a think for yourself, I'm sure you can come up with other cases where it's useful to not have to worry about buffer lengths matching their capacity when you null out the final element (since the reported capacity would be less then actual capacity). Strings are just the easiest example to give so I used 'em
@@zxuiji Yeah, that's why I wrote and use my own data structures library. I have an abstract object that I don't manipulate directly and I call functions on it to work with it. Objects don't even have to be on the heap either because of how I've structured things. I'm sure a thousand other libraries exist to do the same things. If you don't want to write it yourself, then use one of those. Also, I'm just going to throw this out there, but get yourself a copy of RayLib and make a game.
@@zxuiji Might have just been how it was then, but that's okay. Have you ever written a game? Because I'm trying to push as many people that way as I can. The more I can get people into it without using a game engine I feel it's for the better.
One thing that no one ever covers though: how do I make memory write-only without having to implement a software solution myself, even if it's cheap? Every cycle counts.
Thanks. You always make me think, today you taught me something too. Fifty years coding in C and there is always another trick.
You're very welcome. I love new-trick days.
I absolutely adore this channel.
Thanks!
I don't think memory protection is part of the C specifications, it's more of an OS thing.
This is also doable in any language that can call OS APIs.
On Windows it's VirtualProtect (or the lower level NtProtectVirtualMemory)
I guess it put that memory pages as read only and that’s the protection so it generates page faults?
@@mihaicotin3261it generates segmentation faults. Segfaults are real observable errors (crash) and page faults are os internal problems.
Lol. Literally nothing is part of the C specifications😂
These videos are super high quality content ❤
4:44 Be careful! sysconf returns a 'long' data type which can overflow that declared 'integer' pagesize data type, under certain conditions.
Recommended security patch:
static constexpr long kPageSize = sysconf (_SC_PAGE_SIZE);
Then, if kPageSize fits within the INT_MIN and INT_MAX int data type range, then assign it to the local variable.
Good point. Thanks.
Great video! Thank you very much Jacob! 🍀
I remember your lesson about making C code more abstract, we can use this for structures to make their data unchangeable after iniciation. Unless we make special function which will make it possible
It’s requiring a system call though so it ain’t cheap
@@samhadi7972 unless it will be marked as inline
You mean mprotect? Inlining that won’t do anything it’s still a system call
@@samhadi7972 no, i mean you can create function like struct_get_something(struct *object) and make this function inline
Out of curiosity. Have you ever done windows API programming in C? I've never done so i am curious about how it looks and feels.
ive delved into it a little bit; it can be intimidating, but there are enough analogues between the win32 & posix apis to get your feet wet without getting too uncomfortable.
there’s even some aspects of win32 that i think are superior to posix, namely how it has asynchronous i/o built in and immediately available as an option.
the biggest downsides for me are wrangling w/ COM or WinRT - its possible, but difficult in c; its more manageable in a language like c++ - and the at-times excessive amount of extra parameters. msdn & online code samples are your friend here.
the other two pain points i have are more subjective &/or nitpicky; using wide strings (aka utf16) for unicode isn’t too bad, since stdc does come with wide string analogues to the normal narrow string utils out of the box; and writing code which is backward compatible with yet still mostly functional in older versions of windows is annoying, though that mostly holds true for every other operating system so 🤷
nearly the whole video can be done with VirtualProtect and VirtualAlloc Win32 API, this is most used in hooking
Yeah, my first programming job involved windows API programming in C, but it's been a while.
No. You don't wanna know how it looks and feels. It looks ugly and it feels like pulling teeth. 😂
Microsoft hasn't worked on a C compiler in 25 years. Their latest C++ compiler happens to support compiling C files. MSVC is a C++ compiler with a C++ linker. Also, GCC is written in C++ since 2010.
Thank you!👍👍👍
You're welcome.
couldn't you have used aligned_alloc from stdlib? what are the differences?
Sure for alignment purposes, but the whole point of the video is to make it read only.
@@anon_y_mousse I'm talking about at one point in the video where he uses a Linux or posix only function to allocate aligned memory. I'm wondering if the std library one could be used instead
@@MatthisDayer I'm pretty sure it can, but you'll have to test that.
I thought you just make your pointer point to const and it's done 😂
😂
Great! finally I can make own ram drive. What happens when program exists? Would it be still marked read-only? Say one program initialized ram drive and stayed resident in memory. How could another program access/modify the content of proposed ram drive?
Al that memory is "owned" by the process running this code.
After exiting, the process will be cleaned up/destroyed by the OS.
The memory will become unclaimed memory and the rights/protection is removed and set to the "protection" of unclaimed menory.
It will result in a segmentation fault when accessed.
Can you make brk vs mmap?
Haven't I already done that? I'll go back and check. It's hard to keep track sometimes. 🤔
Just a quick question is there any limitation on branching out else if's (or, similarly, case's on switch) ?
i.e., if (main cond).. else if (cond 1); else if (cond 2); .. else if (cond n). max number of lines/statements, n? Any thoughts? Thanks,.
Would you recommend this for ephemeral encryption keys? Can the memory page still be swapped out? Is it protected against malicious software in kernel-space; for example, root-kits? POSIX is just an API front-end, right? Is an OS's POSIX sub-system verified by independent groups? Can we ever be sure whats happening to these 'protected' spaces?
I'm pretty sure this won't necessarily protect you from a malicious or compromised kernel.
@@JacobSorber Ever since i read Ken Thompson's 'Reflections on Trusting Trust' I've been fascinated with the idea of protecting a user-space program from a malicious kernel. Whitfield Diffie, Martin Hellman, and Ralph Merkle devised brilliant ways to exchange cypher keys over insecure channels. I still think there's hope for a protected user-space.
Hi! Im new to C programming and I need to make a program which constantly shows in screen the word that the user types and can be stopped by the key word "END" like this:
Type a word: "hello"
Word is hello
Type a word: "bye"
Word is bye
Type a word: "END"
-
It is for a class exercise so it would be amazing if we could use simple functions to make it. Thanks for reading 🙏
9:04 why is p3 6 bytes long while p1 and p2 are 4?
Or are all 3 of them like, 8 bytes long and the leading zeros are just omitted?
Yep. All 8 with leading zeros omitted.
@@JacobSorber Thanks! For some reason I imagined that memory addresses that a program gets wouldn’t typically start with zero. I don’t have a justification for thinking that way though.
Hmm used malloc but didn't use free.
There’s no opportunity for a leak here since it’s not a real program
@@bob_kazamakis I know but it should be taught as a habit so new programmers don't forget. And especially if we are trying to deal with security issues.
There is a group of programmers that do not cleanup at the end of the program because the OS will do it anyway.
No constructs like destructors, so technically no need to free/cleanup.
The cleanup takes up time and codespace and can even break things due to bugs in the cleanup code
I think it is always better to proper cleanup when the code is subject for update or reuse.
In this case neither will, so for the simplicity of the demonstration code, better leave out the cleanup.
So is a segmentation fault always related to a read-only portion of a page in (virtual) memory being written to or are there other causes too?
Reading from memory your process doesn’t have permissions for should also cause it, I think.
accessing an unmapped page will cause a seg fault, as well.
If only the compiler had some keyword to help with this...
Is it final?
no wait! It's hardcode!
No wait, its variable!
If you're referring to C23's `constexpr`, then it might help. However, it can only guarantee that the data is usable at compile-time. There is no guarantee that the data is stored in read-only memory (in the OS's perspective).
make videos about binary programming, or more low level programming
I do like low-level stuff. Can you clarify what you mean by "binary programming"?
@@JacobSorber like manipulating data using bitwise, shifts, etc. As if we were developing a driver
Hmm, in real world code I would want something else than a segfault. I guess one could capture the signal somehow and do some error handling.
segfault here is for the sake of simplicity as it's not related to main subject
Just had an idea for protecting against "buffer" overrun/underrun, for starters instead of handing the allocation directly to the requester, instead hand over a object that reference the memory by index. The functions for read/write would be similar to the file functions and when handing over the pointer a reference count should be updated in a different page. The allocation itself would always be set to read only when this reference count is greater than 0, serving as a lock against writing. When it's 0 then the protection is removed from the page and the write calls can go ahead without issue. Here's a VERY rough example in code (with some assumed symbols):
int main(void)
{
#define HELLO "Hello World"
#define BYE "Goodbye World"
/* Ensure we cannot accidentally write more than we need, the 0 indicates we do NOT want this memory to auto grow,
the remaining 1s mean pad both the front and back of the memory with null bytes of the size mentioned where sizeof(char) is, this ensures we can always pass the string into any function that expects a null byte */
int strmem = mem_calloc( sizeof(HELLO), 0, sizeof(char), 1, 1 );
/* The 0 indicates that no, the pointer will not be handed to iconv() or some other 3rd party function that does not expect the custom interface */
char *str = NULL;
/* This could segfault under different circumstances but on this occasion the count is at 1 so an exclusive lock is possible */
str = memputraw( strmem, STR, sizeof(STR) );
/* This will run without issue as it only reads the memory */
puts( str );
/* This will segfault because the mem_getptr( strmem, true ) was not called */
str[0] = 0;
/* Assuming we didn't do the line above then this would start writing "Goodbye World" from position 0 (as indicated by the 0) however only "Goodbye Wor" will actually get written
because we specified during allocation we do not want the memory to grow */
str = memputabs( strmem, BYE, sizeof(BYE), 0 );
puts( str );
/* 0 the memory 1st then release it, this will protect against hackers using the pointer after it was released to glean sensitive info */
mem_zero_and_free(strmem);
return EXIT_SUCCESS;
}
Kind of sounds like you just want a managed string type. Have you considered using one of the thousand libraries that provide that in C?
@@anon_y_mousse That was just an easier to get across example, I'm sure there's plenty more cases where null terminated/prefixed buffers are handy, for example a list of IDs where id 0 is invalid, or linked lists that use offsets from the buffer root instead of addresses.
Just give it a think for yourself, I'm sure you can come up with other cases where it's useful to not have to worry about buffer lengths matching their capacity when you null out the final element (since the reported capacity would be less then actual capacity).
Strings are just the easiest example to give so I used 'em
@@zxuiji Yeah, that's why I wrote and use my own data structures library. I have an abstract object that I don't manipulate directly and I call functions on it to work with it. Objects don't even have to be on the heap either because of how I've structured things. I'm sure a thousand other libraries exist to do the same things. If you don't want to write it yourself, then use one of those. Also, I'm just going to throw this out there, but get yourself a copy of RayLib and make a game.
@@anon_y_mousse tried it, didn't like it, don't remember the reason as it was years ago I think
@@zxuiji Might have just been how it was then, but that's okay. Have you ever written a game? Because I'm trying to push as many people that way as I can. The more I can get people into it without using a game engine I feel it's for the better.
One thing that no one ever covers though: how do I make memory write-only without having to implement a software solution myself, even if it's cheap? Every cycle counts.
Making memory read-only feels harder than it should be.