Thank you so so much for this channel. I'm building an app serving fastapi json data from Python parsers I built from tips from this channel. You introduce newbies to so many advanced tools that seem daunting, it's such a huge help
8:09 threads doesnt have limitations because of GIL if we speak about concurrency and compare it with asyncio. GIL just allows only one thread per process to run at a time. But process can still switch between threads, just like asyncio event loop switch between tasks but within single thread. GIL just doesnt allow parallel execution of threads on multiple system threads/cores/CPUs as u like. Its nothing to do with IO, but with CPU time
5 หลายเดือนก่อน
Yes but as you cannot run threads in parallel and use them for only IO a lighter solution in terms of context switching is better and scales better
Thanks for clarifying, exactly what i thought too. It's confusing to say GIL is a disadvantage of threading with IO bound tasks when asyncio is also doing 1 thing at any moment to the same effect of GIL. I wish cooperative vs preemptive multitasking was mentioned so audience can learn for themselves the differences in whether it's the process(asyncio) or system(multithread) that initiates the context switch
Thank you for this video! Really helpful! I would also appreciate if you can deep dive in asyncio library and discuss about canceled tasks and proper exception handling.😁
I'm curious why it was never implemented as a standard from the get-go. Do you know if there were technological limitations or other reasons at the time that prevented async from becoming part of Python's standard library earlier? It would be great to understand the historical context to see how we might push for more comprehensive async support in the future.
@@James-vd3xj because threading is simpler solution and doesn't require special syntax. Main benefit of async is that it is more lightweight, that's it.
Great video Arjan, but one question/request: What is that kind of diagram you show around 2:50 called? I seems great for communicating high-level logic, and you should consider making a video about those. Greetings from Copenhagen, and have a great day!
With async libraries on mysql database. I had to go very low (commit and rollback) instead of simple annotation like in spring boot. It could be cool to have the same annotation with python libs.
Thanks for that video, Arjan. I still do not understand one thing. If I have an IO bound method or library I am using which do not support asyncio is there a way to wrap it to make it async?
@@eyaldahari asgiref standard library module has decorator sync_to_async, but without proper internal implementation it won't give any benefit in concurrency, it just allows to run from async context. For external libs the best way is to use threading, many libs are actually thread-safe, or you can lock critical sections by yourself.
I was surprised the other day, that an mmap to mmap copy (float32->int16 of very large files) was much slower than having async read and write tasks, no matter how I chunked things.
I'm going to soon get some ESP32 development devices and play around with micropython. I'll post code and results on the discord server. Micropython might be a worthwhile topic for some videos, and it looks like a lot of fun.
Arjan-I've seen some of ur videos. Can u explain something/anything without writing 100 lines of code? i mean lets jz say how about u explain something without writing a single line of code. Can u do that?
if I am streaming a decent amount of data from lets say multiple IoT devices to a rpi. I want to serve the data in real time to the server. If I use the concurrent async way, each job will get executed (or literally sent to the server) while blocking each other and at a point won't be displayed in real time? Am I getting this right?
I often write synchronous code, and after a while I realize that I need it to be asynchronous. It’s often painful to redo the entire thing with async in mind, and sometimes I just resort to using threads instead to encapsulate my synchronous code. I always wonder if that’s the right thing to do.
💡 Get my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
Please make more videos on asyncio and asynchronous programming in python, also how newer versions of python dealing with GIL
What's better than Arjan uploading a new video?🤩
Ahah! Thank you for the support :)
you did an excellent job at explaining it in depth, I appreciate the respect fo technical details
Thank you so so much for this channel. I'm building an app serving fastapi json data from Python parsers I built from tips from this channel. You introduce newbies to so many advanced tools that seem daunting, it's such a huge help
Glad to hear the videos have had a positive impact on your journey. Cheers!
Exactly my week's problem. Not a simple library, but very interesting and powefull.Thank you Arjan
Glad it was helpful!
I'd love going more in depth about advanced features and functionalities like Semaphore etc
8:09 threads doesnt have limitations because of GIL if we speak about concurrency and compare it with asyncio. GIL just allows only one thread per process to run at a time. But process can still switch between threads, just like asyncio event loop switch between tasks but within single thread.
GIL just doesnt allow parallel execution of threads on multiple system threads/cores/CPUs as u like. Its nothing to do with IO, but with CPU time
Yes but as you cannot run threads in parallel and use them for only IO a lighter solution in terms of context switching is better and scales better
With threads we have more serious problems like thread safety, object sharing, race conditions. AsyncIO is a life saver.
@@sinancetinkaya lol, race conditions are also possible with asyncio
@@cheatoffchannel6320 you can have race conditions with a single thread and no concurrency at all for x in loop: loop.append(1)
Thanks for clarifying, exactly what i thought too. It's confusing to say GIL is a disadvantage of threading with IO bound tasks when asyncio is also doing 1 thing at any moment to the same effect of GIL. I wish cooperative vs preemptive multitasking was mentioned so audience can learn for themselves the differences in whether it's the process(asyncio) or system(multithread) that initiates the context switch
Thank you for this video! Really helpful!
I would also appreciate if you can deep dive in asyncio library and discuss about canceled tasks and proper exception handling.😁
Great video. Exactly what I needed for my current task.
Glad it was helpful!
I fully agree that Python's standard library needs more async support.
I'm curious why it was never implemented as a standard from the get-go. Do you know if there were technological limitations or other reasons at the time that prevented async from becoming part of Python's standard library earlier? It would be great to understand the historical context to see how we might push for more comprehensive async support in the future.
@@James-vd3xj because threading is simpler solution and doesn't require special syntax. Main benefit of async is that it is more lightweight, that's it.
@@cheatoffchannel6320 Thanks for the brief explanation.
I agree as well
Because javascript people bullied python people into adding async/await syntax. Even Guido himself couldn't defend it, so he left. Sad!
Javascript we workers with fast api is an awesome combo. Promises kick ass.
Great video Arjan, but one question/request: What is that kind of diagram you show around 2:50 called? I seems great for communicating high-level logic, and you should consider making a video about those. Greetings from Copenhagen, and have a great day!
en.wikipedia.org/wiki/Sequence_diagram
Sequence diagram
@ArjanCodes Will you make an async video in python using Django Rest Framework? I will be happy if you make one.
Would be great if you'd make a video on new Python 3.13 features. Especially on --disable-gil option and it's impact on performance.
With async libraries on mysql database. I had to go very low (commit and rollback) instead of simple annotation like in spring boot. It could be cool to have the same annotation with python libs.
Thanks for that video, Arjan. I still do not understand one thing. If I have an IO bound method or library I am using which do not support asyncio is there a way to wrap it to make it async?
@@eyaldahari asgiref standard library module has decorator sync_to_async, but without proper internal implementation it won't give any benefit in concurrency, it just allows to run from async context.
For external libs the best way is to use threading, many libs are actually thread-safe, or you can lock critical sections by yourself.
I was surprised the other day, that an mmap to mmap copy (float32->int16 of very large files) was much slower than having async read and write tasks, no matter how I chunked things.
I'm going to soon get some ESP32 development devices and play around with micropython. I'll post code and results on the discord server. Micropython might be a worthwhile topic for some videos, and it looks like a lot of fun.
Cool! Yes, please post your findings. 😎
Where do i find the course that teach me how to make this own implementation of web server using asyncio??
Arjan-I've seen some of ur videos. Can u explain something/anything without writing 100 lines of code? i mean lets jz say how about u explain something without writing a single line of code. Can u do that?
With the drive towards API consumption all around us i think having concurrency in the stdlib would be a smart move for the Python folks
can you show how to do async calls with AWS resources?
if I am streaming a decent amount of data from lets say multiple IoT devices to a rpi.
I want to serve the data in real time to the server. If I use the concurrent async way, each job will get executed (or literally sent to the server) while blocking each other and at a point won't be displayed in real time? Am I getting this right?
Actually Fastapi use starlette that use anyo.
anyio*
gevent for the win. Same technique under the hood, but all async/await stuff just magically happens.
I wish there were good async desktop app frameworks.
Tornado is another great async web framework
Tornado did async before it was cool!
Cool
cool
자바 코틀린에 비해서 동시성이 떨어질 수 있는 원인이 GIL에 있었구나
almost first 😛
can u support Chinese cc.i think Chinese software developer quantities are in top 3. thanks
I often write synchronous code, and after a while I realize that I need it to be asynchronous. It’s often painful to redo the entire thing with async in mind, and sometimes I just resort to using threads instead to encapsulate my synchronous code. I always wonder if that’s the right thing to do.