Web Scraping with ChatGPT Mentions is Mind Blowing!
ฝัง
- เผยแพร่เมื่อ 30 มิ.ย. 2024
- To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/ThePyCoach . You’ll also get 20% off an annual premium subscription.
In this video, I'll show you how to do web scraping with GPT mentions (Scraper + Data Analyst)
- Link 1: www.audible.com/adblbestselle...
- Link 2: www.soccerphile.com/world-cup...
My Data Analysis Course
====================
🔥 SQL + Python + ChatGPT for Data Analysis: www.udemy.com/course/modern-d...
🔥 My FREE ChatGPT Cheat Sheet: artificialcorner.substack.com...
Other Courses
============
🔥 Join My Automation Course in Python: www.udemy.com/course/automate...
🔥 Join My Python for Data Science Bootcamp: www.udemy.com/course/python-f...
🔥 8-hour Web Scraping Course in Python: www.udemy.com/course/web-scra...
💰 Make money by writing about AI, programming, data science or tech: thepycoach.teachable.com/p/me...
Support My Work
==============
💵 PayPal: www.paypal.com/donate/?hosted...
Content
0:00 Intro
0:25 Example 1
6:53 Example 2
Disclaimer: This video was sponsored by Brilliant
To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/ThePyCoach . You’ll also get 20% off an annual premium subscription.
Thank you! I’ll try it out 👌🏻
Wow... Thats awesome! Thanks!
Just watching your videos made me realize how complicated im making my life. Such a brilliant way to scrape.
Thank you can you please create a video detailing how to update the GPTs system instructions you plan to use to account for the new @mentions feature for seamless teamwork and collaboration between your team of GPTs
Youre a legend
Awesome!
The recommended scraper is unable to scrape pages requiring a log in and offers no way to log in prior to attempting a page scrape. It also has a relatively short timeout so slow responding or large pages can't be scraped.
And also these features could be work with paid Gpt's
Any plugin to overcome website that has security blocking ?
Interesting. I have done some scraping before using Python to download date both in Excel and CSV format and insert to a MySQL database. In some of these examples I needed to use my username and password to access the data. Is this possible using Scraper?
I've recently learn about ics. format file but I couldn't understand how to use it could you teach us how to use such file?
hm sadly did not work as the scraper triggered anti-scraping policy of the websites I tried. This seems good for some minor quick scraping jobs but I wouldn't rely on it for actual production scraping.
Yes you may need to utilize proxies when scraping
Is there any method with open-source llms
Use gpt vision, more adoptive to html change
is there a way to do that with Gemini?
I went to Perplexity as to get some data on web scraping. It used your YT channel as one of its references. Welcome to the AI big time brother 😂.
The thing is that Perplexity sometimes is kind of “lazy” and that is due to the short token window it has. So for example if you ask for the updated market cap of 20 companies it will give you info that is not up to date for some of them.
Is there anything like these in hugging face or free as open source app?
TIA.
Can we do this without having the paid version of gpt?
Hi, I can't see the plugins option under model dropdown like you have @3:36 . I have a chat gpt plus subscription. I have also activated the plugins under settings.
Plugins have just been removed. You can either use the ScraperGPT or use an old chat where the Scraper plugin was activated (plugins still work in old chats)
Hi, do we have to give all the pages that we want to scrap ? Is it possible to give only the root page? Great video by the way :)
You'd need to give all the links, unless there's a pattern in the links that you can describe to ChatGPT
@@ThePyCoach So how do you scrape all business locations with their name and latitude and longitude values from Google Maps when it shows different number of business with each zoom levels and hides the rest at another zoom levels? Conceptually the problem is similar to providing ChatGPT with all page numbers but here we have to deal with different zoom levels. Any thoughts?
Awesome, but i rather manual way combine http header knowledge and linux, all command line
Good. You do that
What about scraping data from private sites like LinkedIn? Is that also possible with a version of this method?
If it requires a login then it’ll be impossible with this method. You can try with my other method that involves the code interpreter and an HTML file (the video is on the channel)
is this paid version of chat gpt?I dont see " explore gpt"?
Sad part is most of the people don't have got 4 🙂
Yeah 😢
Is there anything like these open source/ hugging face?
Not sad
I added the very same comment to one of your replies below >>. @ThePyCoach So how do you scrape all business locations with their name and latitude and longitude values from Google Maps when it shows different number of business with each zoom levels and hides the rest at another zoom levels? Conceptually the problem is similar to providing ChatGPT with all page numbers but here we have to deal with different zoom levels. Any thoughts?
Thanks for sharing! when I try to download I get this error message "Failed to get upload status for /mnt/data/worood_product_list.csv" I tried different browser and nothing change. any thoughts on what could be the issue?