Comprehensive Python Beautiful Soup Web Scraping Tutorial! (find/find_all, css select, scrape table)

แชร์
ฝัง
  • เผยแพร่เมื่อ 3 ธ.ค. 2024

ความคิดเห็น • 455

  • @KeithGalli
    @KeithGalli  5 หลายเดือนก่อน +4

    I made a new tutorial building off of the knowledge learned in this video! Check it out!
    th-cam.com/video/DcI_AZqfZVc/w-d-xo.html

  • @BennyHarassi
    @BennyHarassi 4 ปีที่แล้ว +435

    Shouts to Keith for giving us all an MIT education without the MIT debt

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +131

      Haha I took one for the team xD

    • @Viralvlogvideos
      @Viralvlogvideos 4 ปีที่แล้ว

      haha

    • @hkemal2743
      @hkemal2743 4 ปีที่แล้ว

      Haha. That was a good one.

    • @krishnahare3638
      @krishnahare3638 3 ปีที่แล้ว

      @@KeithGalli how to start never been good in math 50 years old sitting at home? thnx;-)

    • @tanmaytiwari2450
      @tanmaytiwari2450 3 ปีที่แล้ว

      @@KeithGalli since breaking bad minivans are you know swag 😉

  • @apsilal
    @apsilal 3 ปีที่แล้ว +39

    I paid a bootcamp for learning. But Keith you are way above all that. I understood the concepts from your video only. I owe you man!! Keep going and please don't stop putting up such videos.

    • @KeithGalli
      @KeithGalli  3 ปีที่แล้ว +5

      I appreciate the support! Happy that the videos are helpful

  • @rogueknight2414
    @rogueknight2414 5 หลายเดือนก่อน +1

    I wanted to attempt your recent Advanced web scraping tutorial where I then stumbled upon this amazing tutorial and I'm so glad I did! Thank you very much

  • @ajaykushwaha-je6mw
    @ajaykushwaha-je6mw 3 ปีที่แล้ว +1

    The Best thing about your tutorial are that you start from scratch and teach basic and explain each fragment of code with concept. Love from India.

  • @adrianapetrova196
    @adrianapetrova196 4 ปีที่แล้ว +8

    The last time I tried to understand BeatifulSoup I gave up. You explain it so easy to understand. Thanks for the hard work and the time you spend on teaching us :)

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +2

      Love to hear it! You are very welcome :)

    • @rahuldavid4831
      @rahuldavid4831 4 ปีที่แล้ว +1

      Me too! It's almost like Keith is a godsend

    • @andyn6053
      @andyn6053 4 ปีที่แล้ว

      @@rahuldavid4831 he sure is :)

  • @symnshah
    @symnshah 4 ปีที่แล้ว +5

    I have watched a couple of other videos on BeautifulSoup but believe me this one from Keith is the best one. Keith will take you from scratch to a decent level. Thank you so much.

  • @kinwong6383
    @kinwong6383 4 ปีที่แล้ว +4

    CV Update: Web Scraping expert.
    Joke aside what an awesome tutorial. Felt so satisfying to get the secret message with what you taught!!
    Brilliant work!

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +3

      Haha love to hear it! I had a lot of fun putting this one together, so I'm happy to hear that you enjoyed it :)

    • @andyn6053
      @andyn6053 4 ปีที่แล้ว

      @@KeithGalli your tutorials are really appreciated! thanks man :)

  • @lefu7812
    @lefu7812 4 ปีที่แล้ว +63

    Your tutorials are the best, honestly. Thank you so much for doing this.

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +7

      Glad you enjoy them!! You're very welcome :)

  • @doomimic315
    @doomimic315 3 ปีที่แล้ว +16

    This tutorial was incredibly helpful! Web scraping is something I've always found interesting but just hadn't been bothered to start learning, yet this video made it easy to understand and covered a huge range of ways to deal with potential problems. Seriously can't thank you enough for this video and will certainly be sticking around for any new tutorials you upload.

  • @ranveersharma1666
    @ranveersharma1666 4 ปีที่แล้ว +9

    i am from india . we really dont get this quality stuff here.. so thanks to youtue and you.. for spreading wonderful knowledge.. keep rocking !

  • @bobharford5643
    @bobharford5643 2 หลายเดือนก่อน

    The exercises at the end were great -- a big difference between typing along with the screen and actually having to think through on your own how to solve a problem. I will be off to the new video on web scraping in a few days. Many thanks.

  • @modernmistyk4341
    @modernmistyk4341 3 ปีที่แล้ว

    You saved my life, I hope you're getting all the beautiful things in life you deserve

  • @dhruvrathore2022
    @dhruvrathore2022 4 ปีที่แล้ว +33

    Please do a Seaborn Tutorial ! like you did with Pandas, Matplotlib etc. I watched all of them, really glad i found your channel. Simple, informative & on point.

    • @andyn6053
      @andyn6053 3 ปีที่แล้ว

      @Lucas agree, Derek Banas has a great Seaborn tutorial at his channel!

    • @fardinahsan2069
      @fardinahsan2069 3 ปีที่แล้ว

      If you know matplotlib you know most of seaborn, its a matplotlib wrapper. all matplotlib methods work in seaborn too

  • @santoshvaidya3752
    @santoshvaidya3752 3 ปีที่แล้ว +3

    This is one of the finest videos i have ever seen on training. You are an amazing trainer and most importantly you are explaining things in very simple english, also with examples or exercises that would give an hands on experience for viewers......thanks.

  • @LoganNinefingers
    @LoganNinefingers 4 ปีที่แล้ว +3

    Keith you'll be the first one I cite when I write my nobel prize winning book or whatever it is nobel prize winners write. Golden content. Gracias!

  • @ClaireCodesStuff
    @ClaireCodesStuff 4 ปีที่แล้ว +6

    This is a fantastic tutorial. When I last tried to learn beautiful soup, we were in the awkward transition phase between python 2 and 3 and every tuturial was in python 2 because they hadn't released code for 3 yet. I learned 3 because it was "the future". Of course, I then wanted to use BS so I had try and figure out what I wanted to do in python 2. I gave up in total frustration. This is a crystal clear guide and now I actually understand how it works and how to use it. Thanks Keith!

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +1

      Happy that this tutorial could clarify the details and remove some of that frustration! :)

  • @rahuldavid4831
    @rahuldavid4831 4 ปีที่แล้ว +8

    Thank you so much for this wonderful tutorial Keith! Words cannot describe how much I am grateful to you for making this gem of a video that covers everything you need to successfully scrape a webpage! Trust me when I tell you that NOBODY HAS MADE A BETTER VIDEO ON BEAUTIFULSOUP than you!!! If I could have the liberty of suggesting future videos, I would love if you made a video about "Regular Expressions". Keep up the good work and God bless!!!

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว

      Very happy to hear you enjoyed!! A regex video is a great idea :)

  • @wahaha108
    @wahaha108 2 ปีที่แล้ว

    The best python video i have ever seen. No wasted words, dive into the important topic. Lol, great!

  • @mohitupadhayay1439
    @mohitupadhayay1439 2 ปีที่แล้ว

    As someone earlier said, Big SHOUT OUT to Keith for getting the community such amazing content!

  • @kaustubhgupta12
    @kaustubhgupta12 4 ปีที่แล้ว +18

    When keith do it, its perfect 🤩

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +1

      Aww I appreciate that 😊

  • @manu93ize
    @manu93ize 4 ปีที่แล้ว +1

    by far the best tutorial on youtube for web scraping. you are very good at dumming it down, even total beginner can even understand.
    waiting for NLTK tutorial.
    thank you

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว

      Glad you enjoyed it!

  • @TheFearlessGoat
    @TheFearlessGoat 4 ปีที่แล้ว +6

    I love that you have exercises for us to do in the videos! Learned so much from this.

  • @dusty6193
    @dusty6193 8 หลายเดือนก่อน

    Only a third of the way through this video and I already feel like I understand this better. Thank you, brand new at this

  • @esspi9
    @esspi9 4 ปีที่แล้ว +12

    Amazing.
    Thanks Keith!
    Looking forward to the Selenium and scrapy series.

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +3

      You're welcome!

    • @esspi9
      @esspi9 3 ปีที่แล้ว

      @@theduck3126 Try John watson Rooney channel.
      He's got everything covered.

  • @iklintsov
    @iklintsov 4 ปีที่แล้ว

    best most concise and detailed tutorial on bs

  • @fabianrestrepo82
    @fabianrestrepo82 3 ปีที่แล้ว +2

    Man watching that ending was almost like watching Jack sink, beautiful ending!! keep it up man, great content

  • @investandcyclecheap4890
    @investandcyclecheap4890 4 ปีที่แล้ว +10

    This is such a great tutorial ! I loved being able to pause and figure out the problems on my own. I really learned a lot! Thanks Keith, you rock!

  • @ranveersharma1666
    @ranveersharma1666 4 ปีที่แล้ว +2

    we just love your content.. u taught me pandas very well....!!!

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว

      Love to hear that! :)

  • @soesevenonesix
    @soesevenonesix ปีที่แล้ว

    Keith, your videos are excellent. You are totally getting me through grad school just watching your tutorials. Keep it up!

  • @jamesdavies5386
    @jamesdavies5386 2 ปีที่แล้ว

    Hey this tutorial is great! I've been looking for a decent one like it for some time now and I can't believe it took the algorithm this long to show this on my recommended page

  • @panakitikos
    @panakitikos 4 ปีที่แล้ว

    man when i accomplished that secret word challenge on my own i swear to god i just clapped for u and wished i hadn't already liked the video and being a subscriber so i can like and subscribe. thank you very much keith. you're like my geeky genius best friend that im learning from

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +1

      Haha this comment put a big smile on my face :). I appreciate the support!

  • @chiranthchangappa6231
    @chiranthchangappa6231 4 ปีที่แล้ว +1

    One of the best web scraping contents I've seen to date. But the ending was hilarious!

  • @hemanthkumaar3681
    @hemanthkumaar3681 4 ปีที่แล้ว +1

    i learned numpy ,pandas and other things from ur play list. i was strucked for the past 3 days in webscraping i watched a lot of yt videos bt i coudnt understand as ur content...Thank you so much brother :D . Now i hit(smashed) the bell icon too...

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +1

      Awesome glad this video could help clarify some of the confusion you had. Thanks for smashing the bell icon! xD

  • @nallym82
    @nallym82 4 ปีที่แล้ว +1

    I am very glad that I found your videos. I learnt more from you than all other tutorials combined. Please do a tutorial on xlwings. Thank you

  • @ikki411
    @ikki411 3 ปีที่แล้ว +1

    This tutorial was incredible. I've done 2 Python courses that touched the 'Web Scraping' subject, but I wasn't able to fully understand it. This video was one of the two videos that made me fully understand it, and I couldn't be more happy about it. And finding out the secret message was amazing too :D

    • @h4zmeister
      @h4zmeister ปีที่แล้ว

      wanna share the other video you found helpful ? :)

  • @andyn6053
    @andyn6053 4 ปีที่แล้ว

    At 27:45 "div.text " works just as good as "div.get_text()" to print out the strings of the element. You can even use "header.text" instead of "header.string". Seems like ".text" is universal for getting all kind of strings.

  • @MarceloSantos-nc9wq
    @MarceloSantos-nc9wq 4 ปีที่แล้ว +3

    Keith, many thanks for giving us too many excellent information about hard topics. You do the things seem totally simple to do. Sincerely, your tutorials are the best. Again, thank you so much for sharing all of this with us.

  • @nasser_omar
    @nasser_omar 2 ปีที่แล้ว +1

    Hi Keith,
    I'm really excited to watch this video. Actually, I used to watch your all Python-related videos, especially the Pandas one.
    Keep going, and I hope to meet you one day.
    THANKS, A LOT

  • @kallenmulilonalyanya4181
    @kallenmulilonalyanya4181 2 ปีที่แล้ว

    I like how you make simple stuffs that were really scary. Bravo man.

  • @rogerwprice
    @rogerwprice 4 ปีที่แล้ว +1

    Another fabulous real-wold tutorial. Thanks for the google and stack overflow searches and the errors with recovery!

  • @andrewp319
    @andrewp319 4 ปีที่แล้ว +1

    This is by far the best tutorial I have found after searching through the internet for hours. I subscribed just because of this one great video. Please keep doing videos of practical applications of Python. Project tutorials are the best.

  • @sagebaram5951
    @sagebaram5951 3 ปีที่แล้ว

    How do you know you’ve learned something ?
    Completing the challenge within 1 minute no hints. Thank you so much for all your efforts :)!

  • @adrianobavaresco76
    @adrianobavaresco76 ปีที่แล้ว

    Thank you Keith! This is the best video that i watch about bs4. 👏👏

  • @sreeragmsudheesh
    @sreeragmsudheesh 2 ปีที่แล้ว

    52:30 Not sure if this was posted before but this works for the duplicates. Thanks for all the help Keith!!!
    import pandas as pd
    table = soup.select("table.hockey-stats")
    df = pd.read_html(str(table))[0]
    df

  • @Some_random_guy_16
    @Some_random_guy_16 4 ปีที่แล้ว

    Oh man.. your tasks are excellent. It helped me to get a better confidence in working with soup..

  • @XuX-l8q
    @XuX-l8q ปีที่แล้ว

    one of the best beautiful soup videos, and really want to say thanks! Keith

  • @schoolstudentarea4199
    @schoolstudentarea4199 4 ปีที่แล้ว

    i wish i had a cool teacher like you

  • @fahad203
    @fahad203 4 ปีที่แล้ว

    Are you god? I have a simple approach to your videos. I like them first, then I watch the video. Thanks a lot man, you and few others youtubers are going to put universities out of business

  • @amranazad4540
    @amranazad4540 3 ปีที่แล้ว

    This guy deserves the world

  • @zakriajanjua9170
    @zakriajanjua9170 3 ปีที่แล้ว

    I literally hit like because of that secret message lol. Cheers man really grateful for all your content

  • @bhupindersingh4347
    @bhupindersingh4347 2 ปีที่แล้ว +1

    This is a very will organized web scrapping tutorial. Thanks for sharing.

  • @khinekhinezaw27
    @khinekhinezaw27 4 ปีที่แล้ว +4

    This is the best web scraping tutorial. Thank you so much!

  • @shin-mg7hn
    @shin-mg7hn ปีที่แล้ว

    Your video really help a lot to understanding the Beautiful Soup, thank you, Keith!

  • @nasser_omar
    @nasser_omar 2 ปีที่แล้ว +1

    LOL. Nice secret message.
    I appreciate your effort :)

  • @futuregootecks
    @futuregootecks 2 ปีที่แล้ว

    Wow path navigation is so powerful! Thanks for this!

  • @muhammadkazimraza3456
    @muhammadkazimraza3456 2 ปีที่แล้ว +1

    Very very good video and great exercises specially last one.
    Thanks for such videos

  • @alic
    @alic 2 ปีที่แล้ว

    Brilliant, amazing channel. Major kudos to you Keith!!

  • @rodrigomonteiro8780
    @rodrigomonteiro8780 4 ปีที่แล้ว

    Man you save my life. your tutorials are amazing.

  • @victordias8899
    @victordias8899 4 ปีที่แล้ว

    Bro you're great at these videos. Keep it up. I'm very glad I found your channel and I'm learning a lot from you.
    Regarding the task of getting the "is" from fun-facts, you can get them by this simple one liner:
    [li.get_text() for li in webpage.select('ul.fun-facts li') if 'is' in li.get_text()]
    no regex, no extra loops... just plain string methods with list comprehension!

  • @danniliu2544
    @danniliu2544 3 ปีที่แล้ว +1

    Hi Keith, i second many of the viewers comments. Your tutorial on web scraping is by far one of the best ones out there. Thank you so much for producing this. I do have a question though. Hope you can help clarify, I've not had much success googling. Can you clarify what the difference between select function vs. find_all function? when would you use one over another?

  • @ulugbek385
    @ulugbek385 ปีที่แล้ว

    What wholesome way to end

  • @lalitsharma-gl4kr
    @lalitsharma-gl4kr 4 ปีที่แล้ว

    Value for time invested in watching your videos. Along with the subject knowledge, we understand how to practically approach a problem. Thanks a ton for sharing your knowledge.

  • @irfanshaikh262
    @irfanshaikh262 3 ปีที่แล้ว +1

    Subscribing, coz I loved it.
    Glad I found you @keith.
    Exploring your channel now.
    Appreciate the way you did it so perfectly making it simpler to understand for me.

  • @pablomora7880
    @pablomora7880 4 ปีที่แล้ว

    Well done! First class of Web Scrapping! Awesome

  • @santiagovera7488
    @santiagovera7488 4 ปีที่แล้ว +8

    Can you do a step by step of the whole process of doing a data science-related project? your video explanations or the best

  • @ElvisSCL
    @ElvisSCL 4 ปีที่แล้ว

    Always hit LIKE!!! before I watch

  • @PrielCohen1
    @PrielCohen1 ปีที่แล้ว

    Thank you for the video!
    You explain things so clearly

  • @Beyond..Horizon
    @Beyond..Horizon 2 ปีที่แล้ว

    That's the data scientist's way to tell "Like and Subscribe ". Thanks for sharing knowledge!!

  • @zainabkhan5859
    @zainabkhan5859 3 ปีที่แล้ว

    This is exactly what I was looking for. Thumbs up Keith for this awesom video :-)

  • @gavreleric3493
    @gavreleric3493 3 ปีที่แล้ว

    Wow, really impressive. One of the best channel ! Keith you are very clear with your explanations.
    Thank you for sharing your knowledge :)

  • @tralfazy
    @tralfazy ปีที่แล้ว

    Great video and well done. I learned a lot from it. Thanks Keith!

  • @sunnywen9483
    @sunnywen9483 4 ปีที่แล้ว +1

    so surprised to find treasure youtuber here, will go through all your perfect tut in my summer holiday, hope that u will gain more and more subscribers~

  • @Dee-bk3gk
    @Dee-bk3gk 4 ปีที่แล้ว

    You have a lifetime sub from me. Been looking for videos like this for a long time. Keep up with the great content!

  • @OgoidRei
    @OgoidRei 4 ปีที่แล้ว +1

    Thank you Keith, amazing content, easy to follow, clear explanition, great exercices (with walkthrough) and love the funny breaks/comments during the video. Followed and like

  • @AndyRhye
    @AndyRhye 4 ปีที่แล้ว

    The idea with the secret message was super cool!) You've got that like! Well deserved.

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว

      Glad you enjoyed it! I had fun setting that up :)

  • @Inksaver
    @Inksaver 4 ปีที่แล้ว

    Thanks for this excellent video. I usually teach C# and am familiar with the HTMLAgilityPack, but I was asked by a GCSE pupil to teach him webscraping using Python. We have been following this together and found it very easy to use. Subscribed.

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว

      Awesome! Glad to hear that and thanks for subscribing :)

  • @VarunKumar-pz5si
    @VarunKumar-pz5si 4 ปีที่แล้ว +1

    The wait is over....👍👍

    • @KeithGalli
      @KeithGalli  4 ปีที่แล้ว +1

      Took me a bit to make, but I think it will be worth the wait 😊

  • @sarahburkhardt2037
    @sarahburkhardt2037 3 ปีที่แล้ว

    Thanks for sharing this! I am mostly just popping in to learn, but this is helping me know how to think about data & see that there are a lot of options.

  • @shrutipancholi3544
    @shrutipancholi3544 3 ปีที่แล้ว

    One spot for all my Python needs. Thanks Keith! ; )

  • @ajayrana4296
    @ajayrana4296 3 ปีที่แล้ว

    we need more analyzing videos from you

  • @armandaske
    @armandaske 4 ปีที่แล้ว

    Yes! Awesome tutorial dude. Looking forward to your next web scraping video. Cheers!

  • @vegetable9829
    @vegetable9829 3 ปีที่แล้ว

    finally found a channel where someone is not shouting "subscribe to this channel "

  • @bernardobritto8352
    @bernardobritto8352 3 ปีที่แล้ว

    LOL, loved the secret message. Great work, thanks for the video

  • @adjoudjyoucef7782
    @adjoudjyoucef7782 2 ปีที่แล้ว

    One Word For You Bro: Thank You Very Much

  • @soumyaranjandash3597
    @soumyaranjandash3597 2 ปีที่แล้ว +1

    Amazing Lecture. Here we understood Everything. Thanks a lot Broo 🔥👍🙂

  • @lokotock
    @lokotock 4 ปีที่แล้ว

    Thanks a lot! Your video are clear and pretty useful! And it’s a joy watching them! I’m glad that I found your channel ✨

  • @pratiksarani4947
    @pratiksarani4947 ปีที่แล้ว

    wow a fun exercise !! Have a great fun , Next one is the Pandas One

  • @abdoooooo8583
    @abdoooooo8583 4 ปีที่แล้ว

    Great video .. and I watched A LOT videos about beautiful soup. Keep going with the series

  • @vatsdimri3675
    @vatsdimri3675 2 ปีที่แล้ว +1

    Really learned a lot. Loved the exercises.

  • @muthonigathage263
    @muthonigathage263 2 ปีที่แล้ว

    This was a fun video! Thank you Keith Galli.

  • @Amulya7
    @Amulya7 ปีที่แล้ว

    Beautiful video Kieth. Loved it.

  • @benlucke7763
    @benlucke7763 3 ปีที่แล้ว

    Thanks for the tutorial Keith! Keep up the great work

  • @chineduezeofor2481
    @chineduezeofor2481 3 ปีที่แล้ว

    Wow! This is just too good. Thanks for the video Keith

  • @ukaszkopyszko9818
    @ukaszkopyszko9818 4 ปีที่แล้ว

    Thanks Keith. I'm moving on to next web scraping tutorial rn

  • @matthewfeeley6226
    @matthewfeeley6226 3 ปีที่แล้ว

    Excellent tutorial thankyou

  • @TheAwesomejay
    @TheAwesomejay 3 ปีที่แล้ว +1

    I did the secret mystery challenge by myself! i feel incredible, thank you for this tutorial, keith!

    • @KeithGalli
      @KeithGalli  3 ปีที่แล้ว

      Awesome, love to hear that :). Nice work and glad you enjoyed the tutorial!

  • @unsignedperson476
    @unsignedperson476 4 ปีที่แล้ว

    You are perfect ! You know how to teach. Thank you so much man. Liked your style, and got the subject i have been struggling. Liked and subbed.

  • @nikkidao3322
    @nikkidao3322 2 ปีที่แล้ว

    came for the knowledge, subscribed for the humor

    • @KeithGalli
      @KeithGalli  2 ปีที่แล้ว

      You da best! 😊

  • @MrBeezy514
    @MrBeezy514 3 ปีที่แล้ว

    Dang! that was a good tutorial. I love you Keith, sincerely.