David Beazley's ground up work on callbacks and coroutines were phenomenal. Your work on asyncio history concludes the whole story and connects all the dots. This is amazing and can't thank you enough for the gap you have filled. Perfect closure for me. BTW, I think personally dark theme (easy on eyes), professionally while presenting white background theme.
You're doing an amazing job explaining everything in a clear way. Have you in mind to explain also good practice and design pattern regarding asyncio? Thank you for your efforts😀
Dark theme is good. This series is incredible, other explanations refer to he event loop vaguely but seeing exactly what it does definitely grounds things.
Fantastic tutorial.Really appreciate your effort and grace in putting all this valuable and high quality content for free for everyone. I prefer the black theme
Watched on an iPad, the white background seemed clearer and easier on the eyes. Thank you so much for your videos. Fantastic work, enjoyed it tremendously.
Again: amazing, thank you so much! :) As a beginner in python, I am happy to found this video. AsyncIO is so great to use and now, I understand what is happening under the hood.
this was brilliantly articulated and explained... learnt so many OS related things and low level design patterns... thanks for sharing this knowledge...
This is marvelous. It explains things in a clear and precise way. It should be included in the official asyncio documentation. Thanks a lot! -- Dark theme, please.
Thank you again Lukasz. To answer your questions: 1. probably Dark but I don't really mind. Seems like most people prefer dark which is good option. Perhaps it's related to what they are used to programming with day to day. 2. I also want to learn how to use this within a python webserver which is on your list. Specifically around websockets. I'd like to know how to read from some external sources and pipe them to clients connected to my own server.
Black background for the win. I think you'll be covering how asyncio web servers work, I would be interested in that. Also some insight on how the asynchronous interaction with the database is handled would be very nice. Thank you so much for the precious work!
What a high quality series! I have watched a lot of videos about asyncio and none of them discussed in such detail like you did. They just showed how to use the APIs... Will you publish new series about other topics? I'm looking forward to it!
00:00 Introduction. What this video is about. 03:29 Callback call diagram. 03:51 Examples of how the event loop works. 09:42 select 11:09 IOCP - Input/output completion port 12:24 Abstraction levels 15:41 unix_events.py kqueue, epoll, poll, select 18:08 How to define a selector yourself 20:47 Library code analysis 27:04 Analysis through set_debug 29:03 uvloop 32:55 Briefly about the above 34:03 About the next part of the video: Using Coroutines
For batteries included module, would love to see hundreds of async calls to databases or webservers... Thanks for your help and work in explaining this ever evolving topic in Python!
Thank you for this amazing series! Can’t wait for the next episodes. Please, use light theme - it’s easier on eyes when watching in a well-lit environment
Whatever color scheme you choose, I believe it's better to be consistent through the video. It would be easier on people eyes for it to be applied to also docs, bpython, browser, charts etc. I guess most of coders don't respect their circadian rhythm working at night, so unfortunately I'd vote for the dark one :) I love the way you explain things, isn't it take too much effort to make video of a such quality though? I would certainly watch more from you in a simple steam format, just looking at the code and listening to your thoughts.
These series are priceless, thank you so much to putting this out!. Dark theme for the IDE looks a bit better, but no harm on the white one thou. By the way, does asyncio supports io_uring on linux (I've read is something kind of analogous to iocp, not sure if that's true)?
Thank you for your work! You were able to keep great quality :) Thou I use light color schema for everyday work I think dark one is better for YT watching (especially at big TV screen).
Wow, I've been working with Python for almost 4 years and had no idea that 'underscore variable' is actually storing the latest created object or something like this :D
I have a preference for dark theme. It would be good to go into further details about the differences between asyncio and curio/trio, and to talk about their influence on asyncio.
Thanks for sharing your really deep knowledge. As an real-life example application I would like to see the application like GoogleAnalytics. It should gather data from millions of sites (or at least thousands) simultaneously (clicks, events, mouse moves etc). And to be even better than GA. GA displays all the statistics with an about 24-hour delay. I want to be able to see that information in live mode. Theme - not important. This is strange that while coding I usually use the white one, but I really like your black :) By the way - what code editor do you use for everyday codding?
I was trying call_later and even in your video I noticed that it is not delaying for the specified number of seconds. It actually waits for lesser seconds than specified. Why is that?
This is awesome! Thanks for creating this video series/course! I giggled with the "some random dude use it in his video", yeah, sure, "some random dude"... 😂 I like the dark color scheme better 😎 I have a couple of requests 😬: It's probably already planned, but I would like to know what would be the best approach to handle CPU-bound code that doesn't really block on I/O but hogs the event loop, like Numpy and Cryptography stuff. run_in_executor? Are there some guidelines to decide when to specifically send stuff to run_in_executor even if it's not blocking "waiting" but just fully using the CPU? Also, when dealing with code that uses extensions or Cython, if it uses threads, I'm not really sure if that code has access to the same thread locals from threading.local, would it be dangerous to use it and share objects with different functions passed to run_in_executor for the same task/context? Or when dealing with compiled code is there some additional level of abstractions to handle that that doesn't interfere with thread locals? I'm not even sure I'm asking the question correctly, but any insight you could share about that and how to interact with contextvars and compiled/Cython code would be much appreciated!
If asyncio fits the functional programming paradigm (no advanced stuff like functors, monads etc) it would be nice to see some basic examples like function composition, iterators etc
06:08 - I checked the docs for "loop.call_later" - why did the loop only run for 4 seconds and not 8 - as you've specified as parameter to the loop.call_later" call? I noticed the loop terminates at around 06:12 and thought that maybe you did something at 06:12 - 00:08 = 06:04 and you did... namely the "loop.call_later" call... but that call is not supposed to start a new loop, right? Is the call at 06:04 related to why the loop does not last for 8 seconds as you've specified?
how can one see what is scheduled to run on an event loop? I did "dir(asyncio.get_event_loop())" but I cannot see anything useful... is it a static method, maybe?
hello, thank you for this video. I learned a lot, and will re watch your videos several times. I subscribed to EdgeDB bc of your content. 1. dark theme is easier on the eyes. could you just make parentheses and dots etc more visible? 2. everything you did here with the “event loop”, you can just use a while loop. What’s the difference between event loop and a regular (while) loop? In other words, to do what you showed us, I can just use something like: while (until 20 seconds from now) First() Second() Third() Hog() and the result is the same. Both “event loop” and “regular/nonevent loop” blocks while Hog() takes it time. What different thing will learning asyncio allow me to do? 3. why is the trampoline in loop.call_soon(trampoline) called a “callback”? It (at least the first time it’s called) is not being called after anything. It’s just being called like any regular function. 4. conceptually, is loop.run_until_complete(task1) different from loop_call_soon(task1) different? is task 1 a “callback” in both situations? Is task1 processed any differently? Thank you again and I am waiting for your next videos.
Thanks alot for the video. I love this so much. I think either backgrounds works for me. I do prefer the dark one. Personally. And regarding the batteries included. I would like to see how would a worker, a loop maybe could work along side a web app that can consume events from a stream of events asynchronously. Something like Kafka or Redis. I have done this using the help of a library called Faust. But it would be nice to see how would that work along side a webapp. Thank you so much for the videos again.
9:07 "we avoid doing very long operations at one given moment". "making sure you are doing our work in the smallest chunk possible". Is there no simple way to force big chunks of work to pause at regular intervals ? I feel awkward having to manually divide big chunks in smaller chunks, particularly if the smaller chunks are hidden under several layers of function calls.
In some Python environments, such as the standard Python interpreter or Jupyter notebooks, the underscore _ is automatically assigned to the result of the last executed statement. This allows for easy access to the result of the previous operation. However, in other environments, like certain Python shells or IDEs, the underscore _ might not be automatically assigned in the same way. This could explain why you're encountering a NameError in the Python interpreter. It's worth noting that the behavior of the underscore _ can be influenced by various factors, including the version of Python you're using and any customizations or settings in your environment. Therefore, it's not guaranteed to behave consistently across all Python environments.
Amazing job. Best part of the video is seeing your enthusiasm! How is it that the event loop doesn't run "hot" while inside a `while True` loop? github.com/python/cpython/blob/4454057269b995341b04d13f0bf97f96080f27d0/Lib/asyncio/base_events.py#L595-L598 My guess is it's the timeout value inside _run_once, which is then passed into the selector or maybe it's also handled by the selector itself? I guess if you could talk about that a little more it might clear things up for me? As for what could be discussed in the batteries included episode, I'm sure you'd likely get to it, but I think discussing how running synchronous code inside the run_in_executor works, and what, if any, practical limits you get by doing this (for instance, if you have too many of those calls, do you end up back with all downsides of threading or if by doing them inside an event loop it's somehow not as bad?) Also, and now this is getting pretty specific, but if i had a subprocess call (let's say zip a file), and I wanted to stream (PIPE) the standard output of that subprocess back to a user, can you do this using asyncio? Might be a fun example to show? As for theme - very slight preference for dark theme.
Dark theme for the IDE is fine.
Thanks for this amazing series!
David Beazley's ground up work on callbacks and coroutines were phenomenal. Your work on asyncio history concludes the whole story and connects all the dots. This is amazing and can't thank you enough for the gap you have filled. Perfect closure for me.
BTW, I think personally dark theme (easy on eyes), professionally while presenting white background theme.
You're doing an amazing job explaining everything in a clear way. Have you in mind to explain also good practice and design pattern regarding asyncio? Thank you for your efforts😀
Thank you! We'll make sure to cover some good practices and design patterns in Episode 6 while developing the example Web application.
I prefer to see a dark theme in this series. :)
The light one is too light.
Dark theme is good. This series is incredible, other explanations refer to he event loop vaguely but seeing exactly what it does definitely grounds things.
event_list = self._selector.select(timeout)
Let's take a short moment to appreciate this wonder ✨🙏🏽🙏🏽
Fantastic tutorial.Really appreciate your effort and grace in putting all this valuable and high quality content for free for everyone. I prefer the black theme
it's rare to find videos like this, detailed and deep. amazing work, thank you
Watched on an iPad, the white background seemed clearer and easier on the eyes. Thank you so much for your videos. Fantastic work, enjoyed it tremendously.
Thanks for the high-quality content. I also prefer to see the codes in a dark theme.
Wow! Amazed to have found this Thank you so much! Also, Dark! I am hooked.
Again: amazing, thank you so much! :)
As a beginner in python, I am happy to found this video. AsyncIO is so great to use and now, I understand what is happening under the hood.
Informative series! I prefer the dark theme. It's easy on the eyes when watching on my phone or 💻
+1 for white theme. Thanks for this series!
Excellent videos. Thank you for making these so clear and not making huge assumptions. Dark theme.
Amazing! thank you so much for this :) I like the dark theme more.
Amazing. Best description of how the internals of asyncio works. Thanks
Awesome series, 1st time understood how eventloop, multiplexer, selector, etc work under the hood in python...
Thank you so much for this video series. I am okay with the dark background. Looking forward to Co-routines.
Best asyncio series in the Internet. And the Internet is a big place.
this was brilliantly articulated and explained... learnt so many OS related things and low level design patterns... thanks for sharing this knowledge...
This is marvelous. It explains things in a clear and precise way. It should be included in the official asyncio documentation. Thanks a lot! -- Dark theme, please.
Thank you again Lukasz. To answer your questions: 1. probably Dark but I don't really mind. Seems like most people prefer dark which is good option. Perhaps it's related to what they are used to programming with day to day. 2. I also want to learn how to use this within a python webserver which is on your list. Specifically around websockets. I'd like to know how to read from some external sources and pipe them to clients connected to my own server.
Thanks for this videos, they are very useful. I prefer the dark theme :)
still the best video series explaining asyncio. excellent job well done Łukasz.
Black background for the win. I think you'll be covering how asyncio web servers work, I would be interested in that. Also some insight on how the asynchronous interaction with the database is handled would be very nice. Thank you so much for the precious work!
What a high quality series!
I have watched a lot of videos about asyncio and none of them discussed in such detail like you did.
They just showed how to use the APIs...
Will you publish new series about other topics?
I'm looking forward to it!
This is great, super informative. Love the dark theme. Thank you for making this series.
This is an amazing explanation about event loop. Thank you so much!
I think that dark theme is better.
thx for this, mate !
btw. vote 4 dark theme
00:00 Introduction. What this video is about.
03:29 Callback call diagram.
03:51 Examples of how the event loop works.
09:42 select
11:09 IOCP - Input/output completion port
12:24 Abstraction levels
15:41 unix_events.py kqueue, epoll, poll, select
18:08 How to define a selector yourself
20:47 Library code analysis
27:04 Analysis through set_debug
29:03 uvloop
32:55 Briefly about the above
34:03 About the next part of the video: Using Coroutines
Thanks so much for your effort! This is by far the best material on this topic.
Your series mets more than expected. Thank you
For batteries included module, would love to see hundreds of async calls to databases or webservers... Thanks for your help and work in explaining this ever evolving topic in Python!
This is a great series so far. I vote for dark color scheme.
Good video.
I point out at 3:18, you write UNIXes; the plural form of Unix is Unices.
Thank you for this amazing series! Can’t wait for the next episodes.
Please, use light theme - it’s easier on eyes when watching in a well-lit environment
well most programmers hides in not-well-lit env
Great talk, I'm waiting for next episodes! Thanks! I would like to know what is best way to handle byte streams from sockets using aysncio.
Whatever color scheme you choose, I believe it's better to be consistent through the video. It would be easier on people eyes for it to be applied to also docs, bpython, browser, charts etc. I guess most of coders don't respect their circadian rhythm working at night, so unfortunately I'd vote for the dark one :)
I love the way you explain things, isn't it take too much effort to make video of a such quality though? I would certainly watch more from you in a simple steam format, just looking at the code and listening to your thoughts.
These series are priceless, thank you so much to putting this out!. Dark theme for the IDE looks a bit better, but no harm on the white one thou.
By the way, does asyncio supports io_uring on linux (I've read is something kind of analogous to iocp, not sure if that's true)?
why is audience small? This is the best series of explaining ayncIO
Congratulations for the amazing content! BTW, I would prefer the dark theme
Thank you for your work! You were able to keep great quality :)
Thou I use light color schema for everyday work I think dark one is better for YT watching (especially at big TV screen).
Incredible stuff, I am so glad I found your channel. Going to watch all of these for sure.
Amazing content !! Thank you for such deep coverage of the topic !
Wow, I've been working with Python for almost 4 years and had no idea that 'underscore variable' is actually storing the latest created object or something like this :D
I have a preference for dark theme. It would be good to go into further details about the differences between asyncio and curio/trio, and to talk about their influence on asyncio.
Jesteś absolutnie świetnym nauczycielem :D
Thank you very much for this. Looking forward to the next tutorial. Also, I'm OK with the background
Thank you very much for the series, Lukasz! One qeustion: what's with the underscore assignment at 04:00?
Thanks for sharing your really deep knowledge.
As an real-life example application I would like to see the application like GoogleAnalytics. It should gather data from millions of sites (or at least thousands) simultaneously (clicks, events, mouse moves etc). And to be even better than GA. GA displays all the statistics with an about 24-hour delay. I want to be able to see that information in live mode.
Theme - not important. This is strange that while coding I usually use the white one, but I really like your black :)
By the way - what code editor do you use for everyday codding?
I was trying call_later and even in your video I noticed that it is not delaying for the specified number of seconds. It actually waits for lesser seconds than specified. Why is that?
did you manage to find an answer to that?
Best tutorial playlist!
This is an *amazing* series
This is awesome! Thanks for creating this video series/course!
I giggled with the "some random dude use it in his video", yeah, sure, "some random dude"... 😂
I like the dark color scheme better 😎
I have a couple of requests 😬:
It's probably already planned, but I would like to know what would be the best approach to handle CPU-bound code that doesn't really block on I/O but hogs the event loop, like Numpy and Cryptography stuff. run_in_executor? Are there some guidelines to decide when to specifically send stuff to run_in_executor even if it's not blocking "waiting" but just fully using the CPU?
Also, when dealing with code that uses extensions or Cython, if it uses threads, I'm not really sure if that code has access to the same thread locals from threading.local, would it be dangerous to use it and share objects with different functions passed to run_in_executor for the same task/context? Or when dealing with compiled code is there some additional level of abstractions to handle that that doesn't interfere with thread locals? I'm not even sure I'm asking the question correctly, but any insight you could share about that and how to interact with contextvars and compiled/Cython code would be much appreciated!
Yo..big fan!!
bravo, excellent job!!! exactly what I was looking for
i fucking love this series already, sorry for the language but I really needed to emphasise my feeling ;)
💯AsyncIO from scratch better explained ✌🏾🙏🏾
background theme: ⚫️
best resource on the asyncio, thank you
Your explication is fantastic. Good Job.
If asyncio fits the functional programming paradigm (no advanced stuff like functors, monads etc) it would be nice to see some basic examples like function composition, iterators etc
Of course thank you a lot for this amazing material
We'll be getting to that as well, good point!
I took a small moment to appreciate it. :D
what causes this on the Spyder? RuntimeError: asyncio.run() cannot be called from a running event loop
06:08 - I checked the docs for "loop.call_later" - why did the loop only run for 4 seconds and not 8 - as you've specified as parameter to the loop.call_later" call? I noticed the loop terminates at around 06:12 and thought that maybe you did something at 06:12 - 00:08 = 06:04 and you did... namely the "loop.call_later" call... but that call is not supposed to start a new loop, right? Is the call at 06:04 related to why the loop does not last for 8 seconds as you've specified?
Great explanation! 👏
I prefer to see a dark theme in this series
how can one see what is scheduled to run on an event loop? I did "dir(asyncio.get_event_loop())" but I cannot see anything useful... is it a static method, maybe?
Thanks for the video. Really great. I prefer the dark theme.
Excellent video, thank you!
My favourite moment of the whole series: th-cam.com/video/E7Yn5biBZ58/w-d-xo.html 😀
You rock Łukasz!
Thanks so much. Black background works for me, I'm on mac & use youtube dark mode.
Thanks so much make me more clearly from ground-up and why.
8:16 Why after second 11 the hog function kicked in when it was scheduled to start after 15 secs?
Thanks for this amazing series. I prefer to see a white theme.
great detailed explanation
Amazing, thank you so much for this
Thank you for your beautiful video
+1 for white theme
and thanks for your work!
You really do something great, Thanks for your video. BTW, I like black theme.
Thanks for that, I'm waiting for the next Video.
Please use the Dark Theme background.
hello, thank you for this video. I learned a lot, and will re watch your videos several times. I subscribed to EdgeDB bc of your content.
1. dark theme is easier on the eyes. could you just make parentheses and dots etc more visible?
2. everything you did here with the “event loop”, you can just use a while loop. What’s the difference between event loop and a regular (while) loop? In other words, to do what you showed us, I can just use something like:
while (until 20 seconds from now)
First()
Second()
Third()
Hog()
and the result is the same. Both “event loop” and “regular/nonevent loop” blocks while Hog() takes it time. What different thing will learning asyncio allow me to do?
3. why is the trampoline in loop.call_soon(trampoline) called a “callback”? It (at least the first time it’s called) is not being called after anything. It’s just being called like any regular function.
4. conceptually, is loop.run_until_complete(task1) different from loop_call_soon(task1) different? is task 1 a “callback” in both situations? Is task1 processed any differently?
Thank you again and I am waiting for your next videos.
I like dark background (My girl is making me say this. SH(W)E likes dark background. I actually like the light one more)
Thanks alot for the video. I love this so much.
I think either backgrounds works for me. I do prefer the dark one. Personally.
And regarding the batteries included. I would like to see how would a worker, a loop maybe could work along side a web app that can consume events from a stream of events asynchronously. Something like Kafka or Redis.
I have done this using the help of a library called Faust.
But it would be nice to see how would that work along side a webapp.
Thank you so much for the videos again.
This video is very helpful!
The white theme is preferable!
9:07 "we avoid doing very long operations at one given moment". "making sure you are doing our work in the smallest chunk possible". Is there no simple way to force big chunks of work to pause at regular intervals ? I feel awkward having to manually divide big chunks in smaller chunks, particularly if the smaller chunks are hidden under several layers of function calls.
excellent video, is there git examples for this video?
What kind of app are you using at 19:00?
excellent video, I can't wait for the next.
How can I send you what I'm working on to see how to apply it. It is about algorithmic trading
i dont get how you were calling the async functions without declaring async first in the examples ? (for example trampoline).
My brain exploded at select.
Why is it that only the python IDLE accepts loop = _ while the python interpreter returns a NameError: name '_' is not defined error?
In some Python environments, such as the standard Python interpreter or Jupyter notebooks, the underscore _ is automatically assigned to the result of the last executed statement. This allows for easy access to the result of the previous operation.
However, in other environments, like certain Python shells or IDEs, the underscore _ might not be automatically assigned in the same way. This could explain why you're encountering a NameError in the Python interpreter.
It's worth noting that the behavior of the underscore _ can be influenced by various factors, including the version of Python you're using and any customizations or settings in your environment. Therefore, it's not guaranteed to behave consistently across all Python environments.
Amazing job. Best part of the video is seeing your enthusiasm!
How is it that the event loop doesn't run "hot" while inside a `while True` loop? github.com/python/cpython/blob/4454057269b995341b04d13f0bf97f96080f27d0/Lib/asyncio/base_events.py#L595-L598 My guess is it's the timeout value inside _run_once, which is then passed into the selector or maybe it's also handled by the selector itself? I guess if you could talk about that a little more it might clear things up for me?
As for what could be discussed in the batteries included episode, I'm sure you'd likely get to it, but I think discussing how running synchronous code inside the run_in_executor works, and what, if any, practical limits you get by doing this (for instance, if you have too many of those calls, do you end up back with all downsides of threading or if by doing them inside an event loop it's somehow not as bad?)
Also, and now this is getting pretty specific, but if i had a subprocess call (let's say zip a file), and I wanted to stream (PIPE) the standard output of that subprocess back to a user, can you do this using asyncio? Might be a fun example to show?
As for theme - very slight preference for dark theme.
which code editor is he using?
bpython, running in a terminal
hi if the description is very much i confuse but other i do not know .god willing
I prefer the dark side! :)
brilliant!
The Dark Theme Team!
Huge! New subscirber
Crystal clear explanations. Light theme is better !