Definitely would love to hear more about file systems. I’m still confused about COW, and how a file system can be better for NVMe (something like BTRFS) than others (EXT4, ZFS, etc).
mostly a bunch of optimizations, for instance, by having implicit compression at fs level, you reduce the overall amount of single byte writes and reads to the actual drive (at the expense of more cpu usage) which usually means better overall performance, since io tends to be the biggest bottleneck
This is not feature of C, but feature of the underlying platform. POSIX / glibc / Linux in this case. You could do async IO on Windows as well, but you need to use the overlapped functionality for it instead.
Files systems?! Yes please! Keen as to hear you talk about file systems! :D Also, just did a test during the first few minutes of your video, it made me think, is reading from a file, treated the same way as iterating through memory via like memory_location++ sort of thing, and I found, it does! That's AMAZING! :D :D :D, that's super exciting. So you can just like, read however many bytes you want to, do things with it, then read some more, etc etc. Sooooo cool!!! :D
This is quite fascinating seeing this done in C for file access.... I've done it before in assembly, waiting for a microcontroller's ADC to work it's magic or in JacaScript, waiting for a network call to complete..... interesting to see how it's done in "the middle ground".
Thank you, thank you, THANK YOU! I been working on a side project that went into the unfinished pile as I started something else which many of us are guilty of but since I need to refactor a lot of it because I was coding whilst bed ridden post surgery this makes me keen on finishing it, and now it will be far better with it being far faster than I thought possible. Looking forward in ticking complete on a project soon enough. Much gratitude!
If I may suggest something it would probably be cleaning up memory after ourselves (potential memleak). Here it is definitely not an issue, but having thousands or more little async reads may consume a significant amount of memory on some systems.
what exactly does async io do though? Like, if you have multiple cores, does it use that? Or is the "counting" happening while we wait for the disk's info to get into memory and available to the cpu? If neither, what runs parallel to what?
in this example, while the program is waiting for the file to be read, the counter is increased in the while loop, basically, it's about not needing to wait for the read to finish to do other stuff (though it has to be something that doesn't need the data from the file)
@@dcorderoch yes, that makes sense, but my question is more along the lines "what, if anything, in the actual hardware, is running in parallel?" Like, on one hand, the other computation might be running in parallel to the disk doing it's thing, but, on the other hand, it might be just time-sharing (as in threads running in a single core), and I guess there's a lot more possibilities (maybe even depending where the file is). Come to think of it, this might depend on the architecture as well and not be standardized?
As far as I can tell aio_read() calls __aio_enqueue_request (), which uses another thread to read the file. aio_read(): code.woboq.org/userspace/glibc/sysdeps/pthread/aio_read.c.html Where the magic happens: __aio_enqueue_request (): code.woboq.org/userspace/glibc/sysdeps/pthread/aio_misc.c.html#__aio_enqueue_request
Cool! I have never done any http requests on c (if that makes sense) or any socket programming. So does async programming exists there? I suppose everything is a file in linux, so socket programming is writing or reading from socket "file". Correct me, please, if I am wrong
Used it before when need to do other stuff while some data is being loading so i will no block the execution .it was amazing as Unix books teach lots stuff hard to discover by myself.
It would be great if you could show how to use io_uring. Nowadays this approach is becoming more and more popular, even microsoft in Windows 11 implemented this way of communication with kernel.
Interesting, didn't know about `io_read`, thank you! How does this relate to opening the file with the `O_NONBLOCK` flag and then calling `read` and again checking errno for `EAGAIN` / `EWOULDBLOCK`? Are they doing the same thing under the hood?
Yep, I thought exactly the same. I don't normally bother for basic file IO but I do often use read() rather than recvfrom() with non blocking sockets, I have also used non blocking IO with Async file sources such as video capture devices, that way I can do some processing with partial data without much added complexity
For Linux, according to the aio(7) man page this is all still handled in userspace using non-blocking file descriptors (and with the occasional help of threads) because the io_* syscalls provided by Linux 2.6+ only implement a fraction of the POSIX standard. On the other hand, on FreeBSD or macOS aio_read is a native syscall. The biggest difference is that AIO allows you to queue multiple requests at once.
Single threading, yes. This could be a way to avoid the overhead of threads. You can do some types of multi-tasking with asynchronous I/O. It just sometimes takes a lot of care to keep your code from becoming super complicated.
Hey Jacob, love the topics and videos as always! I do have a thought though.. At around the 10 minute mark, for the aio pointer member: I would consider it a lot more readable if it wasn't in these... unicode(?) characters.. but rather the actual two characters you typed. I seem to recall there being a point when you asked about it before and I can't remember what I thought at that time, but seeing this right now confuses me. Anyway, thank you for your great work explaining the topics you bring up!
mmap is usually slower than sync or async block read, but is way more convenient in some use cases. The fread-like stuff is optimized really well, and the page faults on mmap usually add up eventually. Even if you just memcpy to a buffer you allocated with mmap, it should recognize the linear access pattern and pre-populate the pages so that you don't hit page miss every time you advance 4k but it still ends up losing to block read. mmap is a pain in the ass when or if you still have to support 32 bit systems as the address space may not be available for mapping to success when trying to map a larger chunk at a time. mapping over ~200 MB may already fail in 32 bit system depending on the address space consumption of current process.. kind of sucks.. 64 bit
If a more complicated task than incrementing a counter is performed while the async read is executed how would it be possible to partly execute it and countine executing it after the read operation has returned?
A server for example could be reading files to serve and send data asynchronously, and while those two are busy it compresses 1024 byte blocks of data and checks between each block.if the Io is finished.
Also this is not asynchronous because you are still waiting on file reading completion albeit using a function from another library. You need a “wake up” feature to be called asynchronous. signals and condition variable with wait would do.
Keep in mind that you don't really need a poll loop. I used it as a convenient way to measure the time between starting and finishing the request, but if you put something in here that is definitely going to take longer than the request, you could just grab the result when you finished. To be safe, most people would still use a loop of some sort - just in case, but in many cases you can write the code so you never really loop.
Would'nt the whole process context-switch while performing syncronous IO operations. If this happens, the cpu will not be idle always. Could anyone shed some light on this?
I think it comes down to the question if my task is IO-bound or CPU-bound. If it is IO-bound, AsyncIO is a great option. If it is CPU-bound, AsyncIO does not really give any advantages over just using something like a thread pool. The real beauty of AsyncIO shines when a asynchronous runtime is used. Then it's the runtimes job to run the tasks than can currently make progress while other tasks are waiting on IO input. Of course I cannot end this comment without mentioning the awesome async integration into the Rust Programming Language. Together with a runtime like Tokio it is really pleasant to write async programs.
When looking into Linux backup systems, like borg and bup, I noticed that these use alternative ways to access files, like mmap. Could you make a video on how to use mmap?
In this brave new world of Node.js, asynchronous I/O is definitely no longer rare. OTOH POSIX aio is fairly unwieldy especially if you want to do more complex things.
C programming language has inbuilt structures and functions to do asynchronous work, using external library like aio is overkill also it make C similar to “package” based programming language.
7:56 that printf specifier looks suspect. Probably best to cast the parameter to the expected type (unsigned long) rather than assume that uint64_t happens to be an unsigned long. 13:54 If you wanted to squeeze as much processing out of your program as possible, you'd need to make it multi-threaded. Doing things with async I/O will only allow one core to be utilized maximally. And if you're making it multi-threaded, the other threads can do the CPU-bound work while this one does I/O and blocks.
As far as I know, the specification does not intend that. Note that the compiler would have to parse the string given as the fist argument in order to know the expected parameter types.
@@soniablanche5672 are you sure? I think it's non-blocking multiplexed I/O that is popular now. For example, using `select` of something like that. Or, the classic, multithreading is being used instead of non-blocking i/o
That is being done fully preemptively, with software interrupts. Threading can be considered a higher-level abstraction around that. But for applications which are not CPU intensive, this is all needlessly complicated. That’s why higher-languages are now adopting the old “coroutine” idea in the form of async/await constructs, e.g. in JavaScript and Python.
MSG_DONTWAIT does only work on sockets, not on files. Beside that, I expect that it is basically the same. aio is just some abstraction. (For files, the abstraction will probably use O_NONBLOCK to archive the same effect as using MSG_DONTWAIT on sockets.)
I think this is less of a C thing, and more of a reflection of the *nix culture (as opposed to the more verbose/expressive microsoft culture, for example). If you look at at the Windows C API, you'll see what I'm talking about. Might be an interesting topic for a future video. There is reason behind the different cultures and their decisions-not that the other side will find those reasons compelling.
@@JacobSorber sorry for the aggression in my earlier comments; i appreciate your videos. i just feel really strongly about the Linux world's hostility to improvement and self-reflection. Windows is *also* excruciatingly bad and inconsistent at naming things, just in sometimes different ways. i think it's totally unacceptable that these names (and conventions) are still present as anything other than back-compat aliases. it simply makes the language harder to learn and understand for newcomers. if people are naming things according to the number of keypresses it takes to write, that's a big waving red flag that they're neglecting to use proper tools (same-project autocomplete; pre-baked language/library completions; snippets etc.), and that they're prioritising the most immediate and transient convenience over maintainability. the Linux world is absolutely rife with really, really poor coding practices (not to mention acutely bad for certain broader social problems like misogyny) and a kind of ossified, hierarchical, rote-learned set of conventions which can only be described as "tradition", which i use in the most pejorative sense.
@@gloverelaxis No worries. I think I agree with everything here. Cultural inertia is a super powerful force. Of course, it's a much bigger issue than Linux/Unix (cue discussions about qwerty keyboards and carriage returns), but the *nix world definitely has a lot of room for improvement.
man I've been watching your videos and your audio is quite soft. Can you have your audio slightly amplified during editing or something? love your videos a lot though
I'm not 100% sure what Linux is doing under the hood (and it could be different from one OS to another), but there are definitely other ways to do this (besides creating OS threads under the hood). It would be hard to get too deep into it in a comment, but modern operating systems try to avoid threads and synchronous operation whenever possible, favoring a more interrupt-driven/event-style approach.
@@JacobSorber Well I you're right with that one. I just checked. It defaults to user-space threaded approach if no interrupt routine for that IO is available. Otherwise it uses kernel-space interrupt-driven code.
I'd LOVE for you to talk about file systems!
++
Definitely would love to hear more about file systems. I’m still confused about COW, and how a file system can be better for NVMe (something like BTRFS) than others (EXT4, ZFS, etc).
mostly a bunch of optimizations, for instance, by having implicit compression at fs level, you reduce the overall amount of single byte writes and reads to the actual drive (at the expense of more cpu usage) which usually means better overall performance, since io tends to be the biggest bottleneck
Could you compare aoi to other forms of doing asynchronous/non blocking io, like poll, epoll, io_uring, ...
Tbh, poll and epoll is old way to do it. AIO was latest before io_uring release. Much more interesting would be a video about io_uring
What is aoi?
@@31redorange08 did you not watch the video?
@@AlFredo-sx2yy I did. Did you?
@@31redorange08 i did, but unlike you, i paid attention. Rewatch it and you'll see what im talking about...
Didn't know asynchronous operations were possible in C.
Brilliant!
Anything that your hardware + OS can do is possible in C.
This is not feature of C, but feature of the underlying platform. POSIX / glibc / Linux in this case. You could do async IO on Windows as well, but you need to use the overlapped functionality for it instead.
Anything possible at all is possible in C first, before being ported
Comment for the algorithm, your audio is fine!
Nice video, really helpful. Would really love a video or videos on Filesystems. Nice work
Did you free the aiocb after reading the file?
Yes.
Files systems?! Yes please! Keen as to hear you talk about file systems! :D
Also, just did a test during the first few minutes of your video, it made me think, is reading from a file, treated the same way as iterating through memory via like memory_location++ sort of thing, and I found, it does! That's AMAZING! :D :D :D, that's super exciting. So you can just like, read however many bytes you want to, do things with it, then read some more, etc etc. Sooooo cool!!! :D
This is quite fascinating seeing this done in C for file access.... I've done it before in assembly, waiting for a microcontroller's ADC to work it's magic or in JacaScript, waiting for a network call to complete..... interesting to see how it's done in "the middle ground".
Thank you, thank you, THANK YOU! I been working on a side project that went into the unfinished pile as I started something else which many of us are guilty of but since I need to refactor a lot of it because I was coding whilst bed ridden post surgery this makes me keen on finishing it, and now it will be far better with it being far faster than I thought possible.
Looking forward in ticking complete on a project soon enough. Much gratitude!
3:36-3:41 “and of course let me know if you’d like me to talk more about file systems in future videos”. Sounds good yes.
If I may suggest something it would probably be cleaning up memory after ourselves (potential memleak). Here it is definitely not an issue, but having thousands or more little async reads may consume a significant amount of memory on some systems.
and the 6100 checks and increments were in a best case scenario for file access speed, where the file should already be cached in RAM.
I've always wondered how to use async io, thanks, this was incredibly useful
what exactly does async io do though?
Like, if you have multiple cores, does it use that? Or is the "counting" happening while we wait for the disk's info to get into memory and available to the cpu? If neither, what runs parallel to what?
up
in this example, while the program is waiting for the file to be read, the counter is increased in the while loop, basically, it's about not needing to wait for the read to finish to do other stuff (though it has to be something that doesn't need the data from the file)
@@dcorderoch yes, that makes sense, but my question is more along the lines "what, if anything, in the actual hardware, is running in parallel?"
Like, on one hand, the other computation might be running in parallel to the disk doing it's thing, but, on the other hand, it might be just time-sharing (as in threads running in a single core), and I guess there's a lot more possibilities (maybe even depending where the file is).
Come to think of it, this might depend on the architecture as well and not be standardized?
@Jacob Sorber
would love to see an explanation of how async IO works under the hood
As far as I can tell aio_read() calls __aio_enqueue_request (), which uses another thread to read the file.
aio_read():
code.woboq.org/userspace/glibc/sysdeps/pthread/aio_read.c.html
Where the magic happens:
__aio_enqueue_request ():
code.woboq.org/userspace/glibc/sysdeps/pthread/aio_misc.c.html#__aio_enqueue_request
Very well explained. Thanks.
Excited for this one!
Cool!
I have never done any http requests on c (if that makes sense) or any socket programming. So does async programming exists there?
I suppose everything is a file in linux, so socket programming is writing or reading from socket "file".
Correct me, please, if I am wrong
Used it before when need to do other stuff while some data is being loading so i will no block the execution .it was amazing as Unix books teach lots stuff hard to discover by myself.
Cool! And yes filesystems would be nice to hear more about
It would be great if you could show how to use io_uring.
Nowadays this approach is becoming more and more popular, even microsoft in Windows 11 implemented this way of communication with kernel.
perhaps it would be better to compare them on two separate files eliminating the cashing by the os
Interesting, didn't know about `io_read`, thank you!
How does this relate to opening the file with the `O_NONBLOCK` flag and then calling `read` and again checking errno for `EAGAIN` / `EWOULDBLOCK`? Are they doing the same thing under the hood?
Yep, I thought exactly the same. I don't normally bother for basic file IO but I do often use read() rather than recvfrom() with non blocking sockets, I have also used non blocking IO with Async file sources such as video capture devices, that way I can do some processing with partial data without much added complexity
For Linux, according to the aio(7) man page this is all still handled in userspace using non-blocking file descriptors (and with the occasional help of threads) because the io_* syscalls provided by Linux 2.6+ only implement a fraction of the POSIX standard. On the other hand, on FreeBSD or macOS aio_read is a native syscall.
The biggest difference is that AIO allows you to queue multiple requests at once.
It would be great to learn in detail about filesystems and various mechanisms that they use.
Nice. Good technique to know about.
2:38 char is guaranteed to take 1 byte at all times.
I guess this would be particularly good for single-tasking single-threading servers, right?
Single threading, yes. This could be a way to avoid the overhead of threads. You can do some types of multi-tasking with asynchronous I/O. It just sometimes takes a lot of care to keep your code from becoming super complicated.
Hey Jacob, love the topics and videos as always! I do have a thought though..
At around the 10 minute mark, for the aio pointer member: I would consider it a lot more readable if it wasn't in these... unicode(?) characters.. but rather the actual two characters you typed.
I seem to recall there being a point when you asked about it before and I can't remember what I thought at that time, but seeing this right now confuses me.
Anyway, thank you for your great work explaining the topics you bring up!
Ah, yes, the ligatures. They're definitely a point of contention. Some people love them. Some hate them. Thanks for the feedback.
How does this compare to reading files via mmap? Performance wise / use cases
mmap is usually slower than sync or async block read, but is way more convenient in some use cases. The fread-like stuff is optimized really well, and the page faults on mmap usually add up eventually. Even if you just memcpy to a buffer you allocated with mmap, it should recognize the linear access pattern and pre-populate the pages so that you don't hit page miss every time you advance 4k but it still ends up losing to block read. mmap is a pain in the ass when or if you still have to support 32 bit systems as the address space may not be available for mapping to success when trying to map a larger chunk at a time. mapping over ~200 MB may already fail in 32 bit system depending on the address space consumption of current process.. kind of sucks.. 64 bit
If a more complicated task than incrementing a counter is performed while the async read is executed how would it be possible to partly execute it and countine executing it after the read operation has returned?
A server for example could be reading files to serve and send data asynchronously, and while those two are busy it compresses 1024 byte blocks of data and checks between each block.if the Io is finished.
Also this is not asynchronous because you are still waiting on file reading completion albeit using a function from another library. You need a “wake up” feature to be called asynchronous. signals and condition variable with wait would do.
can you please shed some light on OS topics like memory allocators, buddy, slab, kzalloc in future videos ??
Is line 17 ok? Shouldn't it looke like memset(aio, 0. sizeof(aio))?
The poll loop is a bit annoying. It is much easier to kick off a worker thread, and sync at the end, though it would have more overhead.
Keep in mind that you don't really need a poll loop. I used it as a convenient way to measure the time between starting and finishing the request, but if you put something in here that is definitely going to take longer than the request, you could just grab the result when you finished. To be safe, most people would still use a loop of some sort - just in case, but in many cases you can write the code so you never really loop.
What if I already have a threading system. Can I use aio.h or it'd be too much overhead?
Would'nt the whole process context-switch while performing syncronous IO operations. If this happens, the cpu will not be idle always. Could anyone shed some light on this?
Hi Jacob, can you speak about memory barriers in your upcoming videos
Yeah, probably. I'll add it to the list.
I think it comes down to the question if my task is IO-bound or CPU-bound. If it is IO-bound, AsyncIO is a great option. If it is CPU-bound, AsyncIO does not really give any advantages over just using something like a thread pool.
The real beauty of AsyncIO shines when a asynchronous runtime is used. Then it's the runtimes job to run the tasks than can currently make progress while other tasks are waiting on IO input.
Of course I cannot end this comment without mentioning the awesome async integration into the Rust Programming Language. Together with a runtime like Tokio it is really pleasant to write async programs.
When looking into Linux backup systems, like borg and bup, I noticed that these use alternative ways to access files, like mmap. Could you make a video on how to use mmap?
th-cam.com/video/m7E9piHcfr4/w-d-xo.html
it seems you only call free on an error. when is free called after the file is completed reading?
everything is automatically freed when you exit the program
Not related to this video but I bought an Arduino (because of Jacob) just learn embedded SW.
Cool. Best of luck.
Every1 who suffers audio inconvenience try and use loudness limiter browser extension for dynamic equalization.
In this brave new world of Node.js, asynchronous I/O is definitely no longer rare. OTOH POSIX aio is fairly unwieldy especially if you want to do more complex things.
File systems please 🙏🏼
C programming language has inbuilt structures and functions to do asynchronous work, using external library like aio is overkill also it make C similar to “package” based programming language.
Wait, aio.h is not part of the standard if i see correctly? Is it open source?
According to the man pages, aio_read and aio_write are both part of the POSIX standard.
7:56 that printf specifier looks suspect. Probably best to cast the parameter to the expected type (unsigned long) rather than assume that uint64_t happens to be an unsigned long.
13:54 If you wanted to squeeze as much processing out of your program as possible, you'd need to make it multi-threaded. Doing things with async I/O will only allow one core to be utilized maximally. And if you're making it multi-threaded, the other threads can do the CPU-bound work while this one does I/O and blocks.
on x86_64 architecture I don't think it matters that much to cast from uint64_t to uint32_t since it's going to be passed through a 64 bit register
@@ahmadhadwan I thought it was portable code. No need to implicitly tie it to a specific compiler and architecture.
@@gblargg true, but I also think that on other architectures any good modern compiler will implicitly cast the parameters to their proper types.
@@ahmadhadwan Unless they've very recently added that, C requires that you pass the proper parameter types to varargs functions like printf.
As far as I know, the specification does not intend that. Note that the compiler would have to parse the string given as the fist argument in order to know the expected parameter types.
Is this AIO approach any good for network sockets?
most modern web servers use asynchronous code to listen to connections
@@soniablanche5672 are you sure? I think it's non-blocking multiplexed I/O that is popular now. For example, using `select` of something like that. Or, the classic, multithreading is being used instead of non-blocking i/o
I remember doing asynchronous I/O on the PDP11 with READ$/WRITE$ macros and event flags - a long, long time ago. 👴
Yes, and then moving on to VMS with QIO and AST's. Although I believe QIO was supported on some of the PDP-11 OS's as well.
@@andre0baskin I think you're right. I do remember QIO too, with and without a W, in RSX-11M.
That is being done fully preemptively, with software interrupts. Threading can be considered a higher-level abstraction around that.
But for applications which are not CPU intensive, this is all needlessly complicated. That’s why higher-languages are now adopting the old “coroutine” idea in the form of async/await constructs, e.g. in JavaScript and Python.
Why don't we free up allocated memory? Will aio_raturn do that?
What would be the benefit of using aio over MSG_DONTWAIT on read in the while loop?
MSG_DONTWAIT does only work on sockets, not on files. Beside that, I expect that it is basically the same. aio is just some abstraction. (For files, the abstraction will probably use O_NONBLOCK to archive the same effect as using MSG_DONTWAIT on sockets.)
I don't get why everyone is crying about the audio, there's nothing too messy or quiet.
so instead of AsyncIORequest or Request or AsyncRequestInfo or anything remotely useful, C calls it "aiocb"
I think this is less of a C thing, and more of a reflection of the *nix culture (as opposed to the more verbose/expressive microsoft culture, for example). If you look at at the Windows C API, you'll see what I'm talking about. Might be an interesting topic for a future video. There is reason behind the different cultures and their decisions-not that the other side will find those reasons compelling.
@@JacobSorber sorry for the aggression in my earlier comments; i appreciate your videos. i just feel really strongly about the Linux world's hostility to improvement and self-reflection. Windows is *also* excruciatingly bad and inconsistent at naming things, just in sometimes different ways. i think it's totally unacceptable that these names (and conventions) are still present as anything other than back-compat aliases. it simply makes the language harder to learn and understand for newcomers. if people are naming things according to the number of keypresses it takes to write, that's a big waving red flag that they're neglecting to use proper tools (same-project autocomplete; pre-baked language/library completions; snippets etc.), and that they're prioritising the most immediate and transient convenience over maintainability. the Linux world is absolutely rife with really, really poor coding practices (not to mention acutely bad for certain broader social problems like misogyny) and a kind of ossified, hierarchical, rote-learned set of conventions which can only be described as "tradition", which i use in the most pejorative sense.
@@gloverelaxis No worries. I think I agree with everything here. Cultural inertia is a super powerful force. Of course, it's a much bigger issue than Linux/Unix (cue discussions about qwerty keyboards and carriage returns), but the *nix world definitely has a lot of room for improvement.
You forgot to free the aio in main.
javascript developers be like : oh yes we did
Memory leaks detected
man I've been watching your videos and your audio is quite soft. Can you have your audio slightly amplified during editing or something? love your videos a lot though
Commenting on dead times without establishing that both programs really executed over similar time intervals, seems not conclusive.
I bet that async read creates a thread under the hood. There is literally no other way to do this.
I'm not 100% sure what Linux is doing under the hood (and it could be different from one OS to another), but there are definitely other ways to do this (besides creating OS threads under the hood). It would be hard to get too deep into it in a comment, but modern operating systems try to avoid threads and synchronous operation whenever possible, favoring a more interrupt-driven/event-style approach.
@@JacobSorber Well I you're right with that one. I just checked. It defaults to user-space threaded approach if no interrupt routine for that IO is available. Otherwise it uses kernel-space interrupt-driven code.
There are some OSes where it’s the other way round: all I/O calls are inherently async, and threading is an abstraction built on top of that.
C sucks, C++'s great
You're youtubing for a couple of years and still not fixed your volume level. Your audio is too quiet and intro way too loud.
Your headphones suck