Great overview of both Scrapli and async. Been talking with colleagues about async operations and for some reason was fuzzy on the terminology until this vid helped hammer it home. Thanks for taking the time to put this together!
Hey dude, I've been listening to the Hangouts on the train. Very interesting. I'm so chuffed that you got a 1st at Uni. That's massive. Well done brother!
Hey Qman. Nornir with Scrapli is also a very good option. However, Nornir does use threading and as you scale out your network, particularly when dealing with thousands of devices, threads can become resource heavy and costly. At the scale of a few hundred devices I cant imagine you'd notice any great difference, if at all. Dealing with a massive number of devices, though, asyncio will produce faster and more scalable results. To quote from a Real Python article: "For I/O-bound problems, there’s a general rule of thumb in the Python community: 'Use asyncio when you can, threading when you must.'” Hope that helps :)
I came across this video and implemented everything as suggested and even though I have installed the scrapli[asyncssh] library I get this error: ModuleNotFoundError: No module named 'scrapli_asyncssh:'. Then after the error it offers he following for a solution 'pip install scrapli[asyncssh]. After searching the web for a solution and not finding any, I thought perhaps someone on this thread might have a fix. Any ideas, anyone? Running Python 3-11
Hey, Ed. You could use something like a semaphore within Asyncio which will manage an internal counter to break up the concurrency. So you could do it in blocks of 100. That said, if you plan to do only 100 connections at a time I can't imagine youd see much performance difference of using asyncio over threading. In that case, it's probably way easier to use the Nornir framework (and the nornir_scrapli plugin for the connections). Your concurrency will be implemented automatically with threading and you will get the benefit of built in inventory management and cool filtering features. This makes it way easier. I have a couple of videos on how to do such a thing: Simple filtering with Nornir: th-cam.com/video/dnvcg5Hi4S0/w-d-xo.html Advanced filtering with Nornir: th-cam.com/video/mlPdiM_jmUA/w-d-xo.html Or if you wanna stick with Asyncio - look towards using semaphores. Although i havent written any such examples for that as of yet :) Hope that helps! -John
Great overview of both Scrapli and async. Been talking with colleagues about async operations and for some reason was fuzzy on the terminology until this vid helped hammer it home. Thanks for taking the time to put this together!
Thanks so much, Blaine. I really appreciate your kind words and hearing your feedback :)
-John
Hey dude, I've been listening to the Hangouts on the train. Very interesting. I'm so chuffed that you got a 1st at Uni. That's massive. Well done brother!
Thanks, brother! Was a lot of work but worked it in the end :)
Thank you! I have tried it too and it works really fast!
Congratulations on 1st Class Honors!
Thanks, Gezahegn! I really appreciate it :)
-John
Awesome demo
Are you John McGovern ? the trainer at cbt nuggets ?
you sounds familiar
Congratulations John!
Thanks, Ryan. Really appreciate it man :)
What are the advantages of async with Scrapli vs nornir with Scrapli?
Hey Qman. Nornir with Scrapli is also a very good option. However, Nornir does use threading and as you scale out your network, particularly when dealing with thousands of devices, threads can become resource heavy and costly. At the scale of a few hundred devices I cant imagine you'd notice any great difference, if at all. Dealing with a massive number of devices, though, asyncio will produce faster and more scalable results. To quote from a Real Python article:
"For I/O-bound problems, there’s a general rule of thumb in the Python community: 'Use asyncio when you can, threading when you must.'”
Hope that helps :)
I came across this video and implemented everything as suggested and even though I have installed the scrapli[asyncssh] library I get this error: ModuleNotFoundError: No module named 'scrapli_asyncssh:'. Then after the error it offers he following for a solution 'pip install scrapli[asyncssh]. After searching the web for a solution and not finding any, I thought perhaps someone on this thread might have a fix. Any ideas, anyone? Running Python 3-11
This is great! Thanks for sharing
Thanks Carlos!!
Hi John, can you limit on how many devices you can access at a time, I have about 10k and was thinking maybe doing 100 at a time?
Hey, Ed. You could use something like a semaphore within Asyncio which will manage an internal counter to break up the concurrency. So you could do it in blocks of 100. That said, if you plan to do only 100 connections at a time I can't imagine youd see much performance difference of using asyncio over threading. In that case, it's probably way easier to use the Nornir framework (and the nornir_scrapli plugin for the connections). Your concurrency will be implemented automatically with threading and you will get the benefit of built in inventory management and cool filtering features. This makes it way easier. I have a couple of videos on how to do such a thing:
Simple filtering with Nornir: th-cam.com/video/dnvcg5Hi4S0/w-d-xo.html
Advanced filtering with Nornir:
th-cam.com/video/mlPdiM_jmUA/w-d-xo.html
Or if you wanna stick with Asyncio - look towards using semaphores. Although i havent written any such examples for that as of yet :)
Hope that helps!
-John
@@IPvZero thank you. I probably need to start looking into nornir. Love your videos man.
@@eddurguti Thanks, brother. Genuinely appreciate the kind words :)
Congratulations to you.
Thanks, Damian!!
Awesome!
Thanks Jaime! :)
ahh John, thanks for remembering back to your TH-cam password :P
Haha thanks Muhammad! Hopefully be a lot more active now :)