My top 25 pandas tricks
ฝัง
- เผยแพร่เมื่อ 8 ก.ค. 2024
- You're about to learn 25 tricks that will help you to work faster, write better pandas code, and impress your friends. These are the BEST tricks I've learned from 5 years of teaching Python's pandas library.
Don't miss the BONUS at the end of this video!
TRICKS:
0:00 Introduction
0:43 1. Show installed versions
1:20 2. Create an example DataFrame
2:22 3. Rename columns
3:47 4. Reverse row order
4:36 5. Reverse column order
5:01 6. Select columns by data type
5:40 7. Convert strings to numbers
6:55 8. Reduce DataFrame size
8:15 9. Build a DataFrame from multiple files (row-wise)
10:00 10. Build a DataFrame from multiple files (column-wise)
10:45 11. Create a DataFrame from the clipboard
11:50 12. Split a DataFrame into two random subsets
12:57 13. Filter a DataFrame by multiple categories
13:52 14. Filter a DataFrame by largest categories
14:42 15. Handle missing values
15:57 16. Split a string into multiple columns
16:59 17. Expand a Series of lists into a DataFrame
17:39 18. Aggregate by multiple functions
18:41 19. Combine the output of an aggregation with a DataFrame
19:56 20. Select a slice of rows and columns
20:52 21. Reshape a MultiIndexed Series
22:04 22. Create a pivot table
23:01 23. Convert continuous data into categorical data
23:56 24. Change display options
24:47 25. Style a DataFrame
26:14 Bonus. Profile a DataFrame
DOWNLOAD the Jupyter notebook:
github.com/justmarkham/pandas...
WATCH my introductory series, Data Analysis with pandas:
• Data analysis in Pytho...
JOIN the "Data School Insiders" community:
/ dataschool
LET'S CONNECT!
- Email Newsletter: www.dataschool.io/subscribe/
- LinkedIn: / justmarkham
- Twitter: / justmarkham
- Facebook: / datascienceschool
- TH-cam: th-cam.com/users/dataschool?su...
THANKS for watching! 🙌 Which trick are you most excited to start using? CLICK REPLY and let me know!
Thanks. Trick nr 9- building a dataframe from multiple files is the one i've been searching for.
Comment to trick nr. 3. Sometimes modifying column names works by using Python's list comprehension. Say we want to capitalize all column names. Just write:
df.columns = [my_col.capitalize() for my_col in df.columns]
Glad you like trick 9, that is one of my favorites! As for capitalizing column names, a list comprehension works, but a string method is even simpler: df.columns = df.columns.str.capitalize()
Thanks, those tricks are really amazing, I wonder if I can take this magic to the web, is it a good choice to combine pandas power with django?
The last trick will significantly reduce the amount of work we do understand the data through eda.that trick was amazing.thanks kevin
Thanks for a great video. Bonus trick is the most useful.
Video should be called 'Magic with Pandas'. Thanks for this really useful resource!
Thank you! 🙏
The long awited video is out now. Thank you so much mate. Great learning with you.
You're very welcome!
Kevin,
love your content and it is amplified by your speech cadence, enunciation, and tone.
My favorite tricks are 10 and 11 (Building a dataframe from multiple files rows/columns).
Thank you for sharing your knowledge, mad respect!
Thank you so much! 🙏
The bonus content totally blew my mind off! Thank you very much Kevin! You're simply the best! Looking forward to purchase your ML series!
Thank you so much for your kind words!
One of my favorite videos on TH-cam, thank you so much!
Wow! Thank you so much for your kind words! :)
Kevin Markham, you by far one of the best Data Science Instructor, I came across. Thank you for providing us with such high standard content at no cost.
Thank you so much! 🙏
Fantastic! Been using pandas since a year now, You have opened new possibilities for me to keep looking and learning.!! Please keep doing the great work!
Thank you!
Man, you are just so awesome! Giving all these valuable tricks for free on your channel.
God bless you!!!
Thank you so much! Glad you like them 👍
you just saved my life, i was collecting these things over google wasted my good time but now i know where to look for.
keep sharing , God bless you.
Thanks
You are welcome! 😄
Wow, that last tip! What a phenomenal overview and diagnostic tool. I've learned so much from your videos. So grateful for your content.
Thanks very much for your kind words!
Brilliant as usual! From beginning to end! The bonus was also mind blowing :)
Thank you so much, Roberto!
Excited to try ALL of these! Thanks!
You're welcome!
No other trick mentioned in the video is comparable to the Bonus Trick (Profiling).. This is simply outstanding !!!!
Thanks!
This is one of the most useful coolest videos about Pandas I have ever watched. Thank you sooooo much for compiling all these tips and tricks! Absolutely amazing!!!!!!
You're very welcome! 🙌
Thank you Kevin, You are really a good instructor. Also, I really loved how you shout out people your learnt tricks from, that's very awesome!
Thank you!
Fantastic content as always. Thank you Kevin!
You're very welcome Jerome! Glad you liked it 😊
Superb ! Excellent ! introduction for Beginner (myself) to get introduced and master these Pandas at the same time thereby saving me a lot of time. Thank you so much Kevin !!! You Rock !!!
That's great to hear! Thanks for watching 👍
Awesome!! TQVM!
Thanks!
Fantastic tips, thank you!! Super valuable and with clear, practical examples. Once again, thank you!
Thank you so much! 🙏
Awesome pandas tricks!!! Thanks for this great resource
You're very welcome!
Wow the last one was fantastic. styling the dataframe was also pretty neat. Thanks for this 👍
You're very welcome!
Best Pandas trips ever. I’m going to make myself some flash cards for each of these. Thank you.
You're very welcome! Glad you like them!
I had some trouble with my dataframe.
So I was stumbling over the internet to find solution,and your video came.
Although I didn't find solution in your video, but one of the trick gave me a logical idea of how to resolve the issue.
It told me where I was wrong.
And as far as tricks are concerned, they are amazing. Some went over my head but that's okay.
This whole panda python thing is new to me.
I'd prefer to bite slow instead of gulping down entire meal in one go
Thanks for your kind words, and glad I could be helpful!
Best pandas tutorial !! Thanks
You're welcome!
Yeah, 25 tricks!! I would like to thank you a lot for sharing these tips.
You're very welcome!
This is a jewel video! All tricks left me without words! thanks
You're very welcome! 💎
The last 5 tricks were fantastic.!!
Thank you!
Kevin, you are simply amazing. Thanks for sharing these wonderful tricks!
Glad you like them!
Great stuff. Thanks for the tips.
You're welcome!
Last Bonus tip, Best in 2020 till date. Thank you so much. You have made my life easy!!!
Thank you!
Well done Kevin!
Thanks for sharing this great material.
My best regards!
You're very welcome, Hector!
These are really great tricks.
Thank you, for giving me shout out, and thank you for including my trick in your video (pd.profiling) :)
It is one of the best videos on Pandas. I really love some of the tricks e.g., the Clipboard', and 'The Excel copy & paste ones.
Glad it was helpful! Thanks for your kind words!
Thank you for all these tricks.
You're welcome!
All the tricks were mind blowing. But the bonus one was out of the world. It is surely going to help those who struggles Exploratory Data Analysis. Many thanks for sharing this.
Awesome! So glad it was helpful to you! 🙌
I can’t have enough of this!!
Thank you!
Super duper useful video. Amazing!!
Thank you!
The bonus trick is so amazing
Thank you!
Wow... Veryyyy cool tricks. Saves a ton of time.
Thanks Sanjay! 🙌
Thanks for this consolidated and a very crisp video on Pandas....
You're welcome!
Great video. The bonus tip is very useful.
Thanks!
Best pandas video ever!
Thank you! 🙌
Very helpful,
Thanks !
You're welcome!
Great video. I didn't know about the cut function and it will make a portion of my daily data processing tasks a lot easier. Thanks!
That's awesome to hear!
The bonus is cherry on the cake!
🍒
Amazing! Thank you!
You're welcome!
Thank you for the video, really useful stuff!
Glad it was helpful!
Holy Moly What a master piece this is. I liked the clipboard trick a lot man like that literally saves a lot of code and time. And the bonus blew me off. It looks literally like a final report LOL. Can't wait to implement these tricks in my analytics work. Thanks a lot Data School
You're very welcome! Thanks for your kind words!
Useful is the word! Thank you
You're welcome!
Pure GOLD!
Thanks!
Great job Kevin. Learned a lot.
Thanks! Great to hear 👍
It helps me a lot in Data Wrangling. I really love your videos. Keep doing!
Great to hear!
Tnx. The .style.format(dict) is a game changer. I used to convert my parameters to ensure a fixed number of parameters would show up well, i.e convert a fraction to a percentage. Not sure why this had never been mentioned in the many videos I have watched before.
You're very welcome!
Absolutely brilliant!
Thank you!
Kevin, you are a fantastic instructor. I'll sign up for anything you offer. My 2 favorites: clipboards, and, what made my mouth drop - the dataframe profiling.
Thank you so much for your kind words Valerie! I have a lot of resources that might interest you here: www.dataschool.io/start/ And here is my email newsletter: www.dataschool.io/subscribe/ Thanks again!
I love you! KEVIN!
Pandas Profiling! I'm so excited to use it!
Nice! Let me know how it goes 🙌
The best! Thanks so much!
You're welcome!
This was amazing video to sum up basic Pandas usage. Thanks for this useful resources!
Thank you for the kind comment!
You are amazing!!!
Really informative Kevin, appreciate your effort in creating these video's. Such a valuable resource. Trick 19 was probably my favourite.
Thanks for your kind words, and thanks for sharing! 🙌
Very cool! Thank you!
You're welcome!
I was revisiting dataschool video's. In my view these are still one of the best resources out there. Thanks a lot ...
Thank you!
Great as usual, thanks Kevin!
Thank you so much!
Fantastic! It really helped me escaping from a lot of itchy situations!
Great to hear!
the best guy in this business!!!
Thank you so much! 🙏
Needless to say. Thank you once again for the excellent video, like always very neat explanation and clean deployment of ideas.
Thank you so much! I'm glad the video was helpful to you!
thank you this is amazing. will be saving this for future reference!
Awesome! Glad it was helpful to you 👍
First time ever I honestly wish I could upvote this more than once! Great tips, great explanation!
One short addition to trick 19: I enjoyed getting to know the transform method, but as most of the times in pandas, there is another way with (for me at least) more often used functions: Just take the aggregated data-frame and merge it with the old one (left merge) instead of concatenating. I don't know if there are any runtime benefits of any of these methods, but to me it's one less command to remember - even though now I will look into uses of the transform method! :-)
Thank you so much! 🙏 As for trick 19, you are totally right that merge could accomplish the same thing. I like the transform code much better, but I also encourage people to learn fewer functions overall, so there's nothing wrong with just using merge! Anyway, thanks again for your kind words 😊
Great Pandas resource! Difficult to say which trick most 'excites' me, but will go with using transform to add aggregation functions to the data frame. Thanks!!!
I like that trick as well... I only learned about it recently!
Totally awesome!!! Thanks!!!
You're very welcome!
This is a nice vid! When I was learning clojure someone had compiled a '100 most used functions' and boy would I love that for pandas functions or methods: the docs are too extensive but tutorials are typically too brief.
That's an excellent idea!
Thank you so much. I learned a lot of good things. Thank you.
You are so welcome!
Awesome video! Thanks for reminding me of the transform function to combine aggregates back onto a df. Also, the df profiling is a great idea that I've never heard of.
Thank you!
what the awesome tricks!! thank you!!
You're very welcome!
This is incredibly useful! And explained perfectly! Thank you!
It would be awsome to see from you some content on advanced plotting, data plotting tricks :)
Agree, if we can see the plotting ticks will be fan jolly tastic. :p
You're very welcome, and thanks for your suggestions!
Pandas profiling is a new thing for me. It is good to learn new thing everyday. Subbed !!
Thanks!
Excellent content!!!
Much appreciated!
This is great !!!
Thanks!
Thank you so much. This is very helpful.
Great to hear!
Pandas_profiling is BEST Can't wait to use it! Thanks Kevin for doing these
You're very welcome! 😄
Thanks for compiling it at one place
You're welcome!
Amazing tips!!!......keep sharing more such videos
Glad you like them! 😄
AAAAaaaaa sweet tips!
Thank you so much!
You're very welcome!
Many thanks!! Also, I tried another way using listdir as below and it works amazingly.
import os
res=os.listdir(r"C:\Users\blablabla")
pd.concat(pd.read_excel(path) for path in res)
it's actually quite helpful, thx a lot
Great to hear!
25 and Bonus tips are the best.
I like those ones as well!
Don't know how to thank but you are a Nobel man.
Thank you!
I'm so glad I chose to click on this video. Thank you for this amazing video!
You're so welcome!
saved my life!! Thanks!
That's awesome to hear! 🙌
it is amazing because every time I watch this video i can learn sth new! thank u!
That's awesome to hear!
Thanks Kevin! I actually put tip #23 into use immediately after watching.
Awesome! Glad you are putting it into use Adam! 🙌
Thanks for sharing this work.
You're welcome!
Thanks for the tricks, specially for the pivot_table() function and pandas_profiling . For me this is a nice summary of pandas. Thanks
Awesome!
Awesome video!
Thank you!
Thanks for your help. It's very clear and easy to understand.
You're very welcome!
thank you so much for your effort and these videos!
You're very welcome!
Большое благодарю!
You're very welcome!
Such a great video and contents, I do really appreciate your contributions
Thank you so much! 🙌