Understanding Python: Asyncio

In this video, I give an introduction to asyncio and asynchronous concepts.
Included in the lesson are conversions from threading, real-world HTTP requests, and even how to call synchronous code asynchronously.
As always, if you have any questions or suggestions for future videos, please leave a comment down below.
Follow me on Twitter: / jakejcallahan
Source: github.com/Jac...
Timelapse music: 失望した by Eva
Link: • EVA - 失望した [Synthwave]...
Outro music: Elix by Synthness
Link: • Synthness - Elix ★ No ...

[@ruiwang113]

This is the best asyncio introduction video I have watched.

[@dragonkat13]

Great video! once again, you made learning easy and soothing! the Bob Ross of programming!

[@cheesecake_mafia]

Wow!! Another great tutorial Jake. I am really enjoying your teaching style, the choice of content/examples and the difficulty level you choose of your examples.
I do have a query about this, though. I am not sure but guessing that, async operations run on a single process, like threading. So, suppose if I have a piece of program that downloads data from a website using multiple request which is IO intensive, cleans that data using advanced feature engineering techniques which are CPU intensive, and then stores that cleaned data into a DB, which again will be a IO task, then I would be needing asyncio, threading and multiprocessing in one piece of code. Is there some literature like blogs, books, video tutorials or articles you could direct me towards that could help when building such software as there is going to be a great degree of overlap and interaction between all three packages.
Also, is the assumption that async operations run on a single process, correct? Can I make them run in parallel in multiple processes?
I know a google search would do the trick, but I am guessing you will share much better resources than what google would.
But again, excellent tutorial!!
Cheers and Kudos!

[@JakeCallahan]

That is one of the main challenges when dealing with concurrency, determining which method to use based on the situation you're in. One way you could potentially deal with the mix between IO and CPU tasks is to break the CPU bound task into a separate process that would then "consume" the results from the async process as they come in.
However, this does introduce potential challenges around race conditions, so using locking mechanisms (file write locks) are typically key to lessen that risk.

[@cheesecake_mafia]

​@@JakeCallahan Yes, I concur with the arguments made. I did in fact manage to build the thing I was looking for. I'm pretty sure it's not optimised, but it does the task, so decently satisfied.
One great learning was, cause I had to breakup my functions/classes based on the task i.e. IO or CPU, it forced me to reflect and introspect about my architectural design and in the process forced me to think deeply and understand more about the inner workings of the language. This felt like an excruciatingly difficult exercise, but the results do justify the grind. And your tutorials have been more than helpful. Thank you once again for them! Extremely grateful.

[@JakeCallahan]

Perfect! We learn a ton when writing new projects. So much so that it's one of my top recommendations for someone newer to programming or a new language.
Glad I could help

[@PixelMotionHD]

What is the best way, to go around the external API limitations? I want to get all data from the API, yet it yields only 10 data point, and a nextPageToken. So I tried to flood it with multiple calls, but it simply stooped sending data for a while. So I guess, the way to go is to do the initial check for 10-30 data point, and pass the flow of the program two ways, one for interaction with the user, and displaying those first data point, and second, which will continue gathering the data in the background.
For now Im using asyncio and aiohttp, but now I wonder, should I use threading as well, as a way to split those tasks?

[@JakeCallahan]

If the pages need to be requested sequentially, then there aren't many good options, assuming the next page requires the token from the current page response.
Many APIs do let you set a larger size for responses per page, but if this one doesn't, then they are likely trying to limit their resource usage. If that's the case, then the responsible plan would be to follow their design and go slow.
With that in mind, even if all of the above were true, if you have other non-sequential requests that runs against this (or other) API, then you can run those tasks concurrently.
The first challenge is figuring out what can actually be done concurrently.

[@default_youtube_profile]

Do you have a video specifically on aiohttp and uvicorn?

[@JakeCallahan]

Not yet. Those would likely be included in a potential future series on third party libraries

[@cheesecake_mafia]

@@JakeCallahan This is great to know! Excited to learn more. Can you give us a sneak peak of the libraries included in the list?

[@JakeCallahan]

I haven't formally put down a list yet, but it would certainly include aiohttp after requests. Things like pytest might have their own sub-series.
I will give you a hint about the video I'm putting notes down for now, it includes a number of patterns (relatively) newer to Python.