- Save the webpage you want to scrape data from. 0:41 - Upload the saved webpage to Chat GPT using the upload button. 0:55 - Frame a clear and direct prompt for Chat GPT to extract the desired data. 1:08 - Download the CSV file provided by Chat GPT containing the extracted data. 1:59 - Modify your request to Chat GPT if additional data extraction is needed (e.g., product links and ratings). 2:19 - Provide Chat GPT with examples if specific data is not being retrieved correctly. 2:58 - Instruct Chat GPT to correct any errors in the data extraction process (e.g., incorrect segments in product links). 3:34 - Execute a bit of code to automate the process for scraping data from multiple pages if needed. 4:19 - Investigate the structure of URLs for websites with multiple pages to facilitate automated data extraction across all pages. 5:47 - Utilize a programming environment like Visual Studio Code to execute the code provided by Chat GPT for automated data scraping. 6:29 - Adjust the code as necessary to scrape data for the correct number of pages/sites you're targeting. 7:18
Can we do the same using OpenAI, like we provide the prompts, let's say "Which phone having best rating" after providing the webpage url, it will crawl the data from website and produce the output?
Hi, very interesting. Am looking for an option to scrape profile information from approx. 90k profiles from Linkedin. Any thoughts I could "transfer" your example to a Linkedin scraping?
- Save the webpage you want to scrape data from. 0:41
- Upload the saved webpage to Chat GPT using the upload button. 0:55
- Frame a clear and direct prompt for Chat GPT to extract the desired data. 1:08
- Download the CSV file provided by Chat GPT containing the extracted data. 1:59
- Modify your request to Chat GPT if additional data extraction is needed (e.g., product links and ratings). 2:19
- Provide Chat GPT with examples if specific data is not being retrieved correctly. 2:58
- Instruct Chat GPT to correct any errors in the data extraction process (e.g., incorrect segments in product links). 3:34
- Execute a bit of code to automate the process for scraping data from multiple pages if needed. 4:19
- Investigate the structure of URLs for websites with multiple pages to facilitate automated data extraction across all pages. 5:47
- Utilize a programming environment like Visual Studio Code to execute the code provided by Chat GPT for automated data scraping. 6:29
- Adjust the code as necessary to scrape data for the correct number of pages/sites you're targeting. 7:18
Legend
What if you want to scrape all products in a certain search term, instead of on a certain page.
How would you handle scraping dynamically generated websites?
this is incredible! more scraping videos with ChatGPT please!
A wonderful video that we've used as a reference for our recent additions. Your sharing is highly appreciated!
Looking to scrape a website with infinite scroll. Any help ?
Amazing, more please
Very interesting, thank you. Why is it neccessary to first download the web page? Is ChatGPT not able to retrieve the data directly from the web page?
Web scrapping while saving page to your hard drive is kind of misleading, don't you think?
Bless it digital heart ❤😂
Very interesting! In your quote example, what would you do with that info?
Great info & useful !
Can we do the same using OpenAI, like we provide the prompts, let's say "Which phone having best rating" after providing the webpage url, it will crawl the data from website and produce the output?
can you help me to scap the image down as well??
Hi, very interesting. Am looking for an option to scrape profile information from approx. 90k profiles from Linkedin.
Any thoughts I could "transfer" your example to a Linkedin scraping?
Amazing 🎉thank plz more
Awesome. Thanks for sharing...Pls share an example to create data science project using Chatgpt to find health insurance fraud
why did you start with amazon and didnt finish the whole process?
I think you need to use some service to change your IP address as Amazon probably blocks extensive web scraping.
this is very useful straight to point
is it possible to scraping other language website like hindi , Japanese, or other language ?
"but bless his digital heart"
Better than bright data ❤
Sweeeeet! Thanks for real!
Does this data update itself?
The bear is strong on this one...good job
this is interesting
I don't believe in it's accuracy. I won't go for that
Scrape skool community group members data with name and LinkedIn