The fact that you need to sort the list before using groupby is kind of a deal breaker. It's the sorting that's doing the grouping. All the groupby function does is split a sorted list on the appropriate places. Now that I think of it, splitby would've been a better naming choice.
Yeah, it's kind of annoying that you have to sort it and then call groupby separately. And IMO, you almost always want to have the group items as a list too, so you have to do that conversion as well. I also prefer to have the groups as a dictionary from key value to list group, so that's yet another thing to do. In the end, I gave up on itertools.groupby and wrote my own version: groupby[T, K](it: Iterable[T], key: Callable[[T], K]) -> dict[K, list[T]].
the idea of grouping shouldn't be coupled with ordering. If I want to group words by their first alphabet, why do i need to do ordering first? Doesn't make sense at all. the reason of keeping it as a iteratable rather than list is okay, as we could be doing further transformation after grouping. C# does this with explicit ToArray() or ToList() to let caller indicate that the transformation has been completed and we can iterate into actual objects now, and the real grouping (along with other steps) occurs there, I think it's similar optimization idea here.
itertools is written by pystro: Raymond Hettinger, and it is optimized af. Moreover: it reduces your code's cyclomatic complexity. Many for loops, with indented if-blocks disappear. IT doesn't matter that they occur in functions, they are NOT in your code, and your code is less complex by definition of complexity.
@@DrDeuteron oh I mean sure, I've been reading documentation all my life. By just because there's something written about how to do X, doesn't negate the fact that using that obfuscates rather simple stuff unnecessarily. This reminds me of javascript, and that irks me
This is a high quality episode. Itertools is full of very dense goodness, you should make videos for takewhile, starmap, chain, and product/combinations/permutations.
I have a video that covers all of them, but maybe making separate videos such as this one filled with examples could be a another approach to making itertools more understandable for a lot of people :)
@@Indentlyanother thing creators often seem to do is scrape parts of longer detailed videos, and make shorts (sometimes incomplete, with tantalizing hints) highlighting specific points. 👍
What sorts of extensions do you use in vscode to make it look like that? I particularly liked when you had that nice blue bar on the top with a separate play and debug button
This is a great tutorial, thank you. One question: for the last example, how could you sort by the city and then for each city, sort the people by age or by name?
You've mentioned in a previous video that writing out a variable with a colon followed by some statements/operations (ex: "numbers: *list[list[int]]* = ") ensures that the variable _only_ accepts data of that specific type (string, integer, etc) to prevent errors or incorrect execution. While this is good practice, as you've explained in the past, it does present a _lot_ of extra, _optional_ information on screen that may be somewhat confusing to beginners or anyone who's only familiar with the typical "variable = " style of formatting It would obviously be very tedious to give the full explanation in every single video, but if you could perhaps highlight these segments of code at the start of a video just to quickly say something like "this portion is optional, but it ensures that your code only accepts the intended input", I think it would serve as a helpful summary/reminder of what those segments of code actually do Just wanted to mention this as someone who's only been coding off and on for about a year now and still has some trouble understanding the functionality of example code when the formatting isn't entirely clear
That's a fair point, but I don't think that's the target audience. These types of videos are generally for intermediate/advanced users that like consuming niche information. Also, the syntax is called type hinting, not type coercion. Python doesn't throw errors if you say numbers: int = "hello".
@@hidude1354 Depends on how you measure skill levels, I suppose. To me, this video takes a fairly simple form of data analysis (sorting peoples' credentials) that could likely be performed with Python's native functionality and demonstrates how it can be performed in a more efficient way with just a few lines of code by using a specific library. So, I suppose you could say it's closer to intermediate because it's more advanced than covering the absolute basic building blocks of writing code in Python, but I'd still personally consider it a more basic concept and execution because the complexity primarily comes from being familiar with the library itself (you could say that's what all coding boils down to, but you get my point). I still consider myself a beginner and sometimes I watch these videos not even necessarily expecting to understand everything that's shown, but just to get an idea of what sort of stuff is possible in Python and I was surprised at how straightforward this example was, tbh As for syntax, that's one of those things where I'm still not fully familiar with the terminology and forget things, so I just explained it as best as I could. To clarify, I misspoke a little and didn't mean that using a string in place of an integer would cause an error on its own, but was rather speaking more broadly and referring to Indently's video on the topic where he basically said that adding hinting makes you less prone to producing errors in general, such as accidentally using a string variable where an integer variable is expected
@@Loop_KatWell it seems you mistook the concept of the video which isn't about sorting. This is a niche case of needing to iterate over an iterable that is grouped somehow in a memory-efficient way. A lot of his videos are on the more niche side of things, which is why I point these to intermediate/advanced levels bc it's rare you would use some of these concepts. Though this does help your point about not always understanding syntax, the thing is that it'd serve you better to Google this on your own than expect him to explain it bc that's the style you should take to these things. It's just a learning style akin to reading a textbook not in your main subject and doing any digging required on your own, bc if you've forgotten things then it falls under your responsibility to refresh yourself. It's a beneficial style that also helps people that make videos like these keep these short and to the point.
@@hidude1354 I mean, I watched the video, I understood how the iterable over an iterable concept was presented in the examples, and the examples got me thinking about how this method could be applied to other situations or needs. Niche or not, it's still possible to understand the general concept of things like this even if you don't yet have the more advanced knowledge (or practice) to implement them to the fullest extent. Seems a bit silly to suggest that these kinds of videos aren't really useful or informative beyond those very specific needs, imo in any case, the end of the video said to leave a comment if something wasn't entirely clear or explained well enough and I offered my piece from the perspective of someone who took a break from coding for a couple months and had to remember what the hinting syntax actually was. This channel does take the time and care to explain things in a very thoughtful and concise way that feels very approachable regardless of skill level, so it was a small thing that came to mind, as I explained in my original comment. Not much more to say beyond that
I just read them on Python.org, and then try to make the examples more applicable and readable because half the time I read the docs I have no idea what the example means.
Every serious programmer is similar to a mathematician in that they love the exploration of _new_ logic. The first time you implement a group-by function, sure! Not so much the hundredth time 😂
The fact that you need to sort the list before using groupby is kind of a deal breaker. It's the sorting that's doing the grouping. All the groupby function does is split a sorted list on the appropriate places. Now that I think of it, splitby would've been a better naming choice.
This is an interesting one. Though I wish groupby had a flag to sort for you, which would be more efficient.
Yeah, it's kind of annoying that you have to sort it and then call groupby separately. And IMO, you almost always want to have the group items as a list too, so you have to do that conversion as well. I also prefer to have the groups as a dictionary from key value to list group, so that's yet another thing to do.
In the end, I gave up on itertools.groupby and wrote my own version: groupby[T, K](it: Iterable[T], key: Callable[[T], K]) -> dict[K, list[T]].
the idea of grouping shouldn't be coupled with ordering. If I want to group words by their first alphabet, why do i need to do ordering first? Doesn't make sense at all.
the reason of keeping it as a iteratable rather than list is okay, as we could be doing further transformation after grouping. C# does this with explicit ToArray() or ToList() to let caller indicate that the transformation has been completed and we can iterate into actual objects now, and the real grouping (along with other steps) occurs there, I think it's similar optimization idea here.
itertools is written by pystro: Raymond Hettinger, and it is optimized af. Moreover: it reduces your code's cyclomatic complexity. Many for loops, with indented if-blocks disappear. IT doesn't matter that they occur in functions, they are NOT in your code, and your code is less complex by definition of complexity.
But it does confuse people not familiar with this specific syntax
@@TragicGFuel
RTFM
@@DrDeuteron ?
@@TragicGFuel read the manual
@@DrDeuteron oh I mean sure, I've been reading documentation all my life. By just because there's something written about how to do X, doesn't negate the fact that using that obfuscates rather simple stuff unnecessarily.
This reminds me of javascript, and that irks me
Why don't you make a tutorial on complex project for intermediate and advance users then we can apply theory to practice
Perhaps in the future I can try to bring bigger projects to the channel, but for the time being Indently concentrates mostly on the language itself.
Sounds great!
there are other channels for that in my opinion this channel is great because its unique and shows good tips
I don't like that it is named groupby. A groupby shouldn't care about the order of the input, but this one does.
This is a high quality episode. Itertools is full of very dense goodness, you should make videos for takewhile, starmap, chain, and product/combinations/permutations.
I have a video that covers all of them, but maybe making separate videos such as this one filled with examples could be a another approach to making itertools more understandable for a lot of people :)
Could also put them all in an itertools playlist
Not a bad idea at all!
@@Indentlyanother thing creators often seem to do is scrape parts of longer detailed videos, and make shorts (sometimes incomplete, with tantalizing hints) highlighting specific points. 👍
Mostly of what you show us, I think its amazing but I never know when use it. Would be even better if you give us exercices. :)
Exercise: make up an exercise that uses groupby.
Do you have any videos on globals()?
What sorts of extensions do you use in vscode to make it look like that? I particularly liked when you had that nice blue bar on the top with a separate play and debug button
it looks like he's using pycharm (jetbrains), not vscode
Great video, thank you!
This is a great tutorial, thank you. One question: for the last example, how could you sort by the city and then for each city, sort the people by age or by name?
Youre the 👑 love the content!
You've mentioned in a previous video that writing out a variable with a colon followed by some statements/operations (ex: "numbers: *list[list[int]]* = ") ensures that the variable _only_ accepts data of that specific type (string, integer, etc) to prevent errors or incorrect execution. While this is good practice, as you've explained in the past, it does present a _lot_ of extra, _optional_ information on screen that may be somewhat confusing to beginners or anyone who's only familiar with the typical "variable = " style of formatting
It would obviously be very tedious to give the full explanation in every single video, but if you could perhaps highlight these segments of code at the start of a video just to quickly say something like "this portion is optional, but it ensures that your code only accepts the intended input", I think it would serve as a helpful summary/reminder of what those segments of code actually do
Just wanted to mention this as someone who's only been coding off and on for about a year now and still has some trouble understanding the functionality of example code when the formatting isn't entirely clear
That's a fair point, but I don't think that's the target audience. These types of videos are generally for intermediate/advanced users that like consuming niche information. Also, the syntax is called type hinting, not type coercion. Python doesn't throw errors if you say numbers: int = "hello".
@@hidude1354 Depends on how you measure skill levels, I suppose. To me, this video takes a fairly simple form of data analysis (sorting peoples' credentials) that could likely be performed with Python's native functionality and demonstrates how it can be performed in a more efficient way with just a few lines of code by using a specific library. So, I suppose you could say it's closer to intermediate because it's more advanced than covering the absolute basic building blocks of writing code in Python, but I'd still personally consider it a more basic concept and execution because the complexity primarily comes from being familiar with the library itself (you could say that's what all coding boils down to, but you get my point). I still consider myself a beginner and sometimes I watch these videos not even necessarily expecting to understand everything that's shown, but just to get an idea of what sort of stuff is possible in Python and I was surprised at how straightforward this example was, tbh
As for syntax, that's one of those things where I'm still not fully familiar with the terminology and forget things, so I just explained it as best as I could. To clarify, I misspoke a little and didn't mean that using a string in place of an integer would cause an error on its own, but was rather speaking more broadly and referring to Indently's video on the topic where he basically said that adding hinting makes you less prone to producing errors in general, such as accidentally using a string variable where an integer variable is expected
@@Loop_KatWell it seems you mistook the concept of the video which isn't about sorting. This is a niche case of needing to iterate over an iterable that is grouped somehow in a memory-efficient way. A lot of his videos are on the more niche side of things, which is why I point these to intermediate/advanced levels bc it's rare you would use some of these concepts. Though this does help your point about not always understanding syntax, the thing is that it'd serve you better to Google this on your own than expect him to explain it bc that's the style you should take to these things.
It's just a learning style akin to reading a textbook not in your main subject and doing any digging required on your own, bc if you've forgotten things then it falls under your responsibility to refresh yourself. It's a beneficial style that also helps people that make videos like these keep these short and to the point.
@@hidude1354 I mean, I watched the video, I understood how the iterable over an iterable concept was presented in the examples, and the examples got me thinking about how this method could be applied to other situations or needs. Niche or not, it's still possible to understand the general concept of things like this even if you don't yet have the more advanced knowledge (or practice) to implement them to the fullest extent. Seems a bit silly to suggest that these kinds of videos aren't really useful or informative beyond those very specific needs, imo
in any case, the end of the video said to leave a comment if something wasn't entirely clear or explained well enough and I offered my piece from the perspective of someone who took a break from coding for a couple months and had to remember what the hinting syntax actually was. This channel does take the time and care to explain things in a very thoughtful and concise way that feels very approachable regardless of skill level, so it was a small thing that came to mind, as I explained in my original comment. Not much more to say beyond that
Greetings 🤗 Hunky
Respect and gratitude ❤️🙏 for your satisfactory Work 💪🚀🌱🌟
Awesome... Where do you find out about these libraries and techniques?
Cause I find out from you and other TH-camrs.
I just read them on Python.org, and then try to make the examples more applicable and readable because half the time I read the docs I have no idea what the example means.
feels like linq in c# but without as extension methods
That's just pandas with few extra steps
Thank you 😊
i will go into coding school so now these will be useful
Writing manual logic is satisfactory for me personally 😂.. also getting used to this syntax and handing enumerated objects is a bit tricky though😢
Every serious programmer is similar to a mathematician in that they love the exploration of _new_ logic. The first time you implement a group-by function, sure! Not so much the hundredth time 😂
Why not just use pandas?
Why include all of pandas as a dependency if you only need a tiny subset of functionality?
@leegeorg08 You are right, I didn't think enough about it, thank you.