Basic Webscraper : Get info from the web with Python
ฝัง
- เผยแพร่เมื่อ 21 ก.ค. 2024
- A beginners tutorial for learning to scrape websites with Python.
Test website: toscrape.com/
-------------------------------------
twitter / jhnwr
code editor code.visualstudio.com/
WSL2 (linux on windows) docs.microsoft.com/en-us/wind...
-------------------------------------
Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases
mouse amzn.to/2SH1ssK
27" monitor amzn.to/2GAH4r9
24" monitor (vertical) amzn.to/3jIFamt
dual monitor arm amzn.to/3lyFS6s
microphone amzn.to/36TbaAW
mic arm amzn.to/33NJI5v
audio interface amzn.to/2FlnfU0
keyboard amzn.to/2SKrjQA
lights amzn.to/2GN7INg
webcam amzn.to/2SJHopS
camera amzn.to/3iVIJol
gfx card amzn.to/2SKYraW
ssd amzn.to/3lAjMAy - วิทยาศาสตร์และเทคโนโลยี
Questions about basic web scraping always pop up so I wanted to answer them and help out with this video.
thanks for the video, amazing introduction to webscraping
Worked all the way through this. Great tutorial, many thanks!!
A great pragmatic intro to webscraping. Thanks!
Hi John. I have two challenges in my scraping project: 1) the products must first have a search term entered (e.g. the product name or category, etc.), 2) the results are spread over multiple pages (which your video did cover), but the results I need are divided over several tabs.
Hi John, great tutorial as always. Only addition I would recommend is showing how to take it past the finish line and export to a CSV
Yes of course, I have since covered this in my later videos! Thanks!
John, your channel is amazing! Exactly what I was looking for. I'm gonna study all your videos and cancel my Udemy course too, as you have better content and for free.
Thanks alot John! This video made things clear.
Glad I could help!
Sir I have a question please, sir for title you used find_all and for price just find, please clear this point. thanks
Super useful video, thanks John! New subscriber here
Awesome, thank you!
A* tutorial - and besides the scraping, useful for learning about python more generally. For instance, before today I probably would have created three or four empty lists and appended to each of them individually... rather than simply appending a dictionary, which is much cleaner!
How to save data that have scraped and organize and transfer to csv file?
Hi John. Total noob here. This is the first of your videos I have watched. It is super cool! Question: Target sites will often change the product list (adding or removing), and I may want to keep the data updated on my end at same time. Is there a way to inspect the landing page to see how many pages need to be scraped, and put that into the code as a reference to the last page number, as opposed to a hardcoded number?
how to fix this error: GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.
i learned in 19 mins here what i learned in a 9-hour $89.99 udemy course lol!
Excellent..your explanation is awesome.... If possible can you please make an another video, which includes to get data after clicking some buttons in web page(like Radio buttons, list box, button).
Hi! Thanks for the comment. In my other video I do this using browser automation - How I use SELENIUM to AUTOMATE the Web with PYTHON. Pt1 th-cam.com/video/pUUhvJvs-R4/w-d-xo.html - around the 10 min mark. I will have more webscraping videos coming up too.
how can I scrab book rating?
Awesome video John. Thanks! I will now try to learn how to add the data to a .CSV. Would I be on the right path by using a Pandas data frame to do this?
It’s definitely worth knowing how to do it with the CSV module, but yes use pandas - I do all the time
Just add these lines to save everything to .CSV. It worked for me:
import pandas as pd
table = pd.DataFrame(book_list)
table.to_csv('name_your_file.csv')
Hi mate, here:
for x in range(1,50):
url = f'books.toscrape.com/catalogue/page-{x}.html'
I'm getting: line 7
url = f'books.toscrape.com/catalogue/page- {x}.html'
^
IndentationError: expected an indented block
What am I doing wrong?
Thanks :)
Looks like you missed indenting the lines of code after the “for x in range” part. It needs to be indented to work (4 spaces or tab)