It's such a short and simple video, yet listening to a professional use the correct terminology to describe these simple things and dropping some nice more complex facts along the way, is a really great way to learn. Thank you Anthony!
I had to deal with a lot of this when using 2to3 on a fairly large code base. The conversion tool doesn't know whether the return from a .keys() call is going to be stored in a variable or immediately iterated over; it just knows that in Python 2, calling keys would give you a list so it preserves that for your new Python 3 code by wrapping any call to keys in a call to list. Same for a whole ton of stuff in Python 3 that does lazy iteration using views or range objects... It's a bit silly because the whole reason Python 3 switched to this behavior is for speed and performance when iterating or chaining calls to zip, map, filter, etc. And by wrapping everything in list calls you lose that performance gain.
i converted a fairly large python 2 program to python 3 by hand and oh boy the old behavior of returning lists vs the new behavior of iterators was annoying to debug. I understand why theyre doing that there since this thing was relying on the returned values explicitly being lists in a lot of places i just wound up wrapping a bunch of things in list() to preserve behavior any time i tried to fix the program "Properly" it just wouldnt work because it was an ugly spiderweb that was hard to keep track of so i just did the lazy thing :/ "It Works I Guess"
The optimized code is so much easier to read! Now I also have an explanation why I should not use .keys() even though people told me it makes the code easier... Would be great to see more advanced (performance) optimizations, maybe even using a profiler.
I've actually done the dct.keys() more times that I want to admit. I always forget that it iterates over the keys by default. Anyway great video Anthony!
I used to find it hard to remember that sorted and reversed, despite their parallel names, don't return the same type. Maybe that's why we see people add redundant list conversions.
the way I think about it is that in order to sort you need some concrete structure whereas reversing many datastructures can be done straight from iterating!
I just made a PR de-2to3fying exactly this a few weeks back in pywin32 ! Almost done passing pyupgrade on it too (only got some f-string improvements left)
I don't know why I thought sorted() returned a generator like everything else in python but it's good to know that it does not. I am definitely guilty of "definitive list()-ing" when I don't wanna check and know I need to use indexing
@@anthonywritescode that makes sense which is why it seems obvious now lol I just always think in my head pythons gonna return a generator but yeah definitely need access to all elements to sort
they are maybe useful for extremely simple tasks but often produce completely meaningless garbage and ime waste massive amounts of time and energy for little benefit
It's so counterintuitive that calling sorted on a dict not only returns an object of a different type (I could understand if it returned something like a sorteddict if that's a thing), but that type isn't even a mapping Weird stuff
In my opinion calling keys here makes it simpler, since `sorted(dct)` reads as if you are sorting the dict itself, while `sorted(dct.keys())` is quicker to read and comprehend
To make it extra confusing, Python 3.6+ dictionaries have their own concept of order (they remember insertion order). Personally, I'd write sorted(dct) in an experienced pythonists team vs. sorted(dct.keys()) in a polyglot team for clarity.
The default of iterating a dictionary being the keys IMHO is not intuitive. As you mentioned, other languages behave differently. I always prefer doing calling `.keys()` in favor of being explicit.
The code at the end is certainly the most optimized code possible for that task. However, I would probably still recommend to leave the dct.keys() just to make it more explicit. Sometimes a little redundancy does wonders for the readability of the code (unless the task demands for maximum optimization, in that case by all means remove it).
Thank you for yet another great vid :) Though I really liked the way the example snippet was optimized, one who isn’t familiar enough with the sorted() func could easily be confused by the sorted(dict_instance) syntax. It’s easy to assume that it sorts the dictionary
I think calling dct.keys() is better as is more explicit and you don't have to remember what dct iterates. If you're that worried about performance, you're better off writing all or some of the code in c/c++/rust.
It's such a short and simple video, yet listening to a professional use the correct terminology to describe these simple things and dropping some nice more complex facts along the way, is a really great way to learn. Thank you Anthony!
I had to deal with a lot of this when using 2to3 on a fairly large code base. The conversion tool doesn't know whether the return from a .keys() call is going to be stored in a variable or immediately iterated over; it just knows that in Python 2, calling keys would give you a list so it preserves that for your new Python 3 code by wrapping any call to keys in a call to list. Same for a whole ton of stuff in Python 3 that does lazy iteration using views or range objects...
It's a bit silly because the whole reason Python 3 switched to this behavior is for speed and performance when iterating or chaining calls to zip, map, filter, etc. And by wrapping everything in list calls you lose that performance gain.
yep! I mentioned that via modernize (an improved 2to3 which uses six when possible)
i converted a fairly large python 2 program to python 3 by hand and oh boy the old behavior of returning lists vs the new behavior of iterators was annoying to debug. I understand why theyre doing that there
since this thing was relying on the returned values explicitly being lists in a lot of places i just wound up wrapping a bunch of things in list() to preserve behavior
any time i tried to fix the program "Properly" it just wouldnt work because it was an ugly spiderweb that was hard to keep track of so i just did the lazy thing :/
"It Works I Guess"
Simple and efficient solution. For huge dictionaries this will probably increase the performance of the app quite a lot.
The optimized code is so much easier to read!
Now I also have an explanation why I should not use .keys() even though people told me it makes the code easier...
Would be great to see more advanced (performance) optimizations, maybe even using a profiler.
I have quite a few videos on that topic actually!
I've actually done the dct.keys() more times that I want to admit. I always forget that it iterates over the keys by default. Anyway great video Anthony!
I used to find it hard to remember that sorted and reversed, despite their parallel names, don't return the same type. Maybe that's why we see people add redundant list conversions.
the way I think about it is that in order to sort you need some concrete structure whereas reversing many datastructures can be done straight from iterating!
When you don't have any sort of editor hint and are left guessing, I can see people just constantly coercing stuff into lists just to be safe.
boy, it's some kindergarden optimizations
Would like to see more content under this theme “all code sucks”
fortunately there's a playlist in the description!
I just made a PR de-2to3fying exactly this a few weeks back in pywin32 !
Almost done passing pyupgrade on it too (only got some f-string improvements left)
I don't know why I thought sorted() returned a generator like everything else in python but it's good to know that it does not. I am definitely guilty of "definitive list()-ing" when I don't wanna check and know I need to use indexing
if you think about it there's not really any way you can return a sort iteratively. you need the whole structure to manipulate in order to sort
@@anthonywritescode that makes sense which is why it seems obvious now lol I just always think in my head pythons gonna return a generator but yeah definitely need access to all elements to sort
my only fear is you'll feature my code in this series someday.
The amount of no-ops I find in Python code is substantial. Makes it less readable. I think LLMs are making this even worse unfortunately.
LLMs are definitely making this worse -- the 2 to 3 migration didn't help either (where keys / items / values returned concrete lists!)
wait me a moment, i have some refactor to do
Hi,
I want to hear from you about your opinion about AI and AI tools for programming.
they are maybe useful for extremely simple tasks but often produce completely meaningless garbage and ime waste massive amounts of time and energy for little benefit
It's so counterintuitive that calling sorted on a dict not only returns an object of a different type (I could understand if it returned something like a sorteddict if that's a thing), but that type isn't even a mapping
Weird stuff
it's not really though. sorting only can happen on a list. period. sorted takes an iterable and returns a sorted list
In my opinion calling keys here makes it simpler, since `sorted(dct)` reads as if you are sorting the dict itself, while `sorted(dct.keys())` is quicker to read and comprehend
you could make the same argument for `list(sorted(dct))` now I don't need to know if it's a list -- but it's wasteful (and ignorant) :)
To make it extra confusing, Python 3.6+ dictionaries have their own concept of order (they remember insertion order). Personally, I'd write sorted(dct) in an experienced pythonists team vs. sorted(dct.keys()) in a polyglot team for clarity.
you can just say python now. 3.8 (not a typo) is end is life
Love these bite sized videos! Super informative and quick to watch
Pls .make more videos on this topic
fortunately there's a playlist with a bunch more in the description!
The default of iterating a dictionary being the keys IMHO is not intuitive. As you mentioned, other languages behave differently. I always prefer doing calling `.keys()` in favor of being explicit.
Did you make this code yourself for educational purposes? I just cannot imagine a real human being writing anything like that for production :)
like all examples in this series this is very real code from production (though anonymized and simplified to avoid calling out the actual authors)
🥹
The code at the end is certainly the most optimized code possible for that task. However, I would probably still recommend to leave the dct.keys() just to make it more explicit. Sometimes a little redundancy does wonders for the readability of the code (unless the task demands for maximum optimization, in that case by all means remove it).
ACSSSSSS!
Thank you for yet another great vid :)
Though I really liked the way the example snippet was optimized, one who isn’t familiar enough with the sorted() func could easily be confused by the sorted(dict_instance) syntax. It’s easy to assume that it sorts the dictionary
idk how you would be confused though -- what else would it possibly do?
I think calling dct.keys() is better as is more explicit and you don't have to remember what dct iterates. If you're that worried about performance, you're better off writing all or some of the code in c/c++/rust.
you're absolutely better to write performance critical code in not-python, but it's just as easy to use the not-wasteful syntax
that said, I would not recommend C in the modern age
This is great....
I don't mind .keys(), I find it more readable.