You Should Use CSS Selectors for Web Scraping.
ฝัง
- เผยแพร่เมื่อ 19 ต.ค. 2024
- Lets talk CSS Selectors. I demo the most common, and most useful ones to help you extract data using your web scrapers as easily as possible. I'm using requests_html in Python, which can be pip installed if you do not have it already. It's a great libary and well worth learning
-------------------------------------
Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases
-------------------------------------
Digital Ocean (Cloud Servers, Affiliate Link) - m.do.co/c/c7c9...
Sound like me:
microphone amzn.to/36TbaAW
mic arm amzn.to/33NJI5v
audio interface amzn.to/2FlnfU0
-------------------------------------
Video like me:
webcam amzn.to/2SJHopS
camera amzn.to/3iVIJol
studio lights amzn.to/3aBpKik
small lights amzn.to/2GN7INg
-------------------------------------
PC Stuff:
case: amzn.to/3dEz6Jw
psu: amzn.to/3kc7SfB
cpu: amzn.to/2ILxGSh
mobo: amzn.to/3lWmxw4
ram: amzn.to/31muxPc
gfx card amzn.to/2SKYraW
27" monitor amzn.to/2GAH4r9
24" monitor (vertical) amzn.to/3jIFamt
dual monitor arm amzn.to/3lyFS6s
mouse amzn.to/2SH1ssK
keyboard amzn.to/2SKrjQA
You make one of the best tutorials mate. you simplify it and explain it so well, can't thank enough. Keep it up buddy :)
This is gold. I’ve watched so many of your vids but missed the foundational stuff like this. If I’m ever in the UK I’ll have to buy you a beer!!
I don’t yet work professionally as a developer, and while I really enjoy scraping I wonder if I’m pursuing a route that has a low pay ceiling.
If anyone has any opinions on that I’d love to hear.
Learning web scraping is a good way to teach you skills that will transfer over to any web related job - keep learning until you find what it is you want to do, even if that’s not a tradition dev job. There is money to be made with scraping data but taking those skills and building web apps that connect other software via an API is a good place to look
Great video! Would be great to have another video comparing CSS and Xpath. Differences in their time performance, generality, compatibility with browsers and etc.
The most informative and concise video on this topic! Thanks again John.
Fantastic overview. Thank you much!
Best video on css selector
Thank you for this very useful video!
so easy to follow thank you
Amazing! A cheatsheet would be appreciated too
This is a dilemma for me. CSS selectors seem much easier to read and are more straightforward, but XPath seems a lot more powerful for selecting exactly what you need within complex nested tags. I guess it works to mix and match though, like use CSS by default and switch to XPath when I can't get what I need with CSS?
I’ve always got by with css selectors. I learned them well when I first looked into scrapy as they were in most of the documentation. It’s absolutely fine to use whichever whenever as long as it works for you and you get the results you want!
@@JohnWatsonRooney Thanks John! Between your channel and a Udemy course I got on sale, I'm getting the hang of this stuff quicker than I thought. You're definitely the go-to resource on TH-cam. Kudos and keep it up!
This is huge. I spend the most time trying to grab the correct elements, rather than the actual logic of the code, often coming up with None types. This is going to be my new reference guide
@Shepard Kingston this reminds me of the time I went to the grocery store and one of the aisles had Spam in it
thanks for making simpler
I like your video 👍 xpath seems easy than css. Which is the best option?
I prefer css as I am more used to it. However once you learn xpath properly you can do some cool things
xpath is more powerfull
thank for the best tutorial
Hello John, Could you make a video on Scraping Tweets form Twitter?
wonderful video
just wanted to ask how to select a text from an element?
With requests-html we just add “.full_text” at the end - you will also need “first = True” in there too
@@JohnWatsonRooney Thanks John. Also wanted to know is there any forum (like discord) where I can post issues or share my code?
Like I want to scrape the country name, cases, death etc. for eg. how to scrape the country name?
covid19.who.int/table . Can you please assit?
Please tell me the difference between select and find all method and which one to use when
thanks
Can you make one video on login in amazon by using requests
That was amazing tutorial but I have a problem. I've got a class that has many values however a specificl value is changing in diffrent websites I mean that for example in page one I have my value in name = r.html.find('.sc-1bker4h-4.llfiOB')[0].text but in other I have that value in name = r.html.find('.sc-1bker4h-4.llfiOB')[5].text. Could you help me? Thanks in advance
Can i use it with bs4 instead of requests_html
Yes! I believe you need to use soup.select() instead of soup.find()
@@JohnWatsonRooney Thanks bro...
Not working for scrape tiktok