I optimised my code (focused on HTTP requesting), from 70s to 40s by ensuring no data was requested twice, and then by implementing multithreading I reduced the total time to 0.6s. I'll give a 100€ to anyone who convinces me that it wasn't a great idea xD.
The challenge is to know what tool to use for the job. For example, if you need to dig a small hole a shovel will do but if you want to dig a hole the size of a pool you'll need an excavator! Sure, you can use whichever tool, but it will either be an overkill or it will take forever.
I find it very valuable to point out that: *Optimised Code Structure is more important than Optimised Code Solutions*. Things like list comprehensions can be double-edged sword - even if they give you bonus few seconds sometimes, usually they make the code look less readable, especially if you overuse it. In the meantime, there's extremely big benefit of restructuring your code to optimised form - especially for beginners who just start learning their workflow. I just recently finally realised how to structure my program to make it both readable & not initialise new thousand of objects every program loop/frame (tip: instead of initialising all objects in main(.py) before loop, bind them to object in external module, so on main only this object will initialise, altogether with all others you can then refer). Previous concept was extremely readable, but laggy as hell, especially with mods. And although I haven't yet finished "migration" to this system, so can't really say how much profit I will earn by that, I'm more than sure this will eventually be great solution - and done not by changing some vital readable parts of the code, but by rethinking how to place modules/classes/functions in a way that is both good for my mind and effective.
I'd say it's more important to first learn refactoring and actually write clean code. However many programming problems on for example leetcode, codewars and checkio requires optimized solutions to pass all tests. The difference in execution time can be huge between fast and slow solutions. Since Python is slow as it is, improving performance can sometimes be extra important. Fast solutions doesn't automatically mean they are harder to understand. One has to find a balance between optimization and readability for the project in question.
Those are really valid thoughts in my opinion 👍! 📌First of all: Python is about convenience and speed of writing not execution of code. 📌Second: translation helps. Optimus means the best in Latin not the fastest. 📌Third: practice matters. How are newbies supposed to learn how to code, if they are thinking about some kind of optimization all the time 😁? Making mistakes is a huge part of learning. P. S. I don't mind your hoodie. I know that filming isn't the easiest, especially for introverts. Besides, I was born in an evil empire, which was keen on the world domination at the expense of thier own citizens. The worst part was they couldn't properly feed the population or EVEN produce toilet paper. So trust me, I wouldn't judge you, even it was your only hoodie. Thanks for the video 😉
Wow , your channel is really good! I started programming about a year and a half ago (started with ECMA8+ and react and web development) and then I moved heavily into python after i was inspired by Lex Fridman and his podcast. I love python and your videos inspire a lot of improvements to my scripts
Guess it just depends on how important a script is... If it's only gonna be used once and then forgotten... No need for crazy optimization... But if it'll be a lambda function for example and executes very often, performing an important task... Optimization is necessary to reduce costs... It's all about use case and scenario which helps us make a decision
Thanks for the video I just know using List comprehension is a kind of optimization as you mentioned that , if there is other ways for optimizing our code please share those with us.
@@TehIdiotOne Faster than for loops for sure, and I imagine there's very little that can compete with list comprehensions in vanilla Python from my understanding.
Cython. Static typing will literally reduce time exponentially. Dynamic typing was a ridiculous idea. I really don't think the creators thought the language would take off, meant for beginners, a better than shell script, or plugin architecture. Python gained traction however. Personally, I'm not into memory control. If you are really into functional programming or state-based process, service daemons, I would be thinking C or Rust. Something like that. I'm trying to have fun, and C got really tedious trying to build a complete anything. Why kill myself on malloc? Life is short.
I think it wasn't ridiculous for Python, a hobby project. I have a lot of friends that don't know anything about programming, and they still use Python for work, which is f**king awesome in my opinion. Because even if their scripts run slightly slower than ours due to lack of programming experience, it still helped them to progress in their field of work. That's how I love seeing Python being used. Mostly to get the job done, and if you're experienced, to get the job done nice and fast by mixing in some C implementations 😉
I optimised my code (focused on HTTP requesting), from 70s to 40s by ensuring no data was requested twice, and then by implementing multithreading I reduced the total time to 0.6s. I'll give a 100€ to anyone who convinces me that it wasn't a great idea xD.
I'm curious to hear what people have to say for 100€!
What did you use to implement multithreading I imagine you just made all the HTTP requests asynchronously.
The challenge is to know what tool to use for the job. For example, if you need to dig a small hole a shovel will do but if you want to dig a hole the size of a pool you'll need an excavator! Sure, you can use whichever tool, but it will either be an overkill or it will take forever.
I find it very valuable to point out that: *Optimised Code Structure is more important than Optimised Code Solutions*. Things like list comprehensions can be double-edged sword - even if they give you bonus few seconds sometimes, usually they make the code look less readable, especially if you overuse it.
In the meantime, there's extremely big benefit of restructuring your code to optimised form - especially for beginners who just start learning their workflow. I just recently finally realised how to structure my program to make it both readable & not initialise new thousand of objects every program loop/frame (tip: instead of initialising all objects in main(.py) before loop, bind them to object in external module, so on main only this object will initialise, altogether with all others you can then refer).
Previous concept was extremely readable, but laggy as hell, especially with mods.
And although I haven't yet finished "migration" to this system, so can't really say how much profit I will earn by that, I'm more than sure this will eventually be great solution - and done not by changing some vital readable parts of the code, but by rethinking how to place modules/classes/functions in a way that is both good for my mind and effective.
it is kinda funny optimizing code to the point its unreadable tho
I'd say it's more important to first learn refactoring and actually write clean code. However many programming problems on for example leetcode, codewars and checkio requires optimized solutions to pass all tests. The difference in execution time can be huge between fast and slow solutions. Since Python is slow as it is, improving performance can sometimes be extra important.
Fast solutions doesn't automatically mean they are harder to understand. One has to find a balance between optimization and readability for the project in question.
Very well said
Those are really valid thoughts in my opinion 👍!
📌First of all: Python is about convenience and speed of writing not execution of code.
📌Second: translation helps. Optimus means the best in Latin not the fastest.
📌Third: practice matters. How are newbies supposed to learn how to code, if they are thinking about some kind of optimization all the time 😁? Making mistakes is a huge part of learning.
P. S. I don't mind your hoodie. I know that filming isn't the easiest, especially for introverts. Besides, I was born in an evil empire, which was keen on the world domination at the expense of thier own citizens. The worst part was they couldn't properly feed the population or EVEN produce toilet paper. So trust me, I wouldn't judge you, even it was your only hoodie.
Thanks for the video 😉
Wow , your channel is really good! I started programming about a year and a half ago (started with ECMA8+ and react and web development) and then I moved heavily into python after i was inspired by Lex Fridman and his podcast. I love python and your videos inspire a lot of improvements to my scripts
Thanks for the kind words! Happy to have you here :)
Me thought you will show us sample code and test them and show when and where it makes sense to optimize python code. Sydd :(
I can make another video with examples in the near future.
Guess it just depends on how important a script is... If it's only gonna be used once and then forgotten... No need for crazy optimization... But if it'll be a lambda function for example and executes very often, performing an important task... Optimization is necessary to reduce costs... It's all about use case and scenario which helps us make a decision
Thanks for the video
I just know using List comprehension is a kind of optimization as you mentioned that , if there is other ways for optimizing our code please share those with us.
I will continue sharing cool tips & tricks about optimizations in Python :)
@@Indently Thanks bro
keep it up 👍👍👌👌
Are list comprehensions actually faster than other methods?
@@TehIdiotOne Faster than for loops for sure, and I imagine there's very little that can compete with list comprehensions in vanilla Python from my understanding.
Cython. Static typing will literally reduce time exponentially. Dynamic typing was a ridiculous idea. I really don't think the creators thought the language would take off, meant for beginners, a better than shell script, or plugin architecture. Python gained traction however. Personally, I'm not into memory control. If you are really into functional programming or state-based process, service daemons, I would be thinking C or Rust. Something like that. I'm trying to have fun, and C got really tedious trying to build a complete anything. Why kill myself on malloc? Life is short.
I think it wasn't ridiculous for Python, a hobby project. I have a lot of friends that don't know anything about programming, and they still use Python for work, which is f**king awesome in my opinion. Because even if their scripts run slightly slower than ours due to lack of programming experience, it still helped them to progress in their field of work.
That's how I love seeing Python being used. Mostly to get the job done, and if you're experienced, to get the job done nice and fast by mixing in some C implementations 😉
Thanks for the video!
I have spent 5 decades doing evil with my code by optimizing as I write my code. Suck it, all those who think optimization is useless.
First get the code working before trying to optimize too much
Just use Perl.