Deep Scraping Using Browse AI's Bulk Run Feature
ฝัง
- เผยแพร่เมื่อ 11 ธ.ค. 2023
- In this quick video, we'll show you how to use two robots and Browse AI's Bulk Run feature to extract data from a list of items, then do some deep scraping to get information from the detail pages of those list items. Finally, we'll get the data into Airtable.
- วิทยาศาสตร์และเทคโนโลยี
Can you input a google spreadhseet instead of a csv file while setting up a bulk run?
great looking product. Does your solution allow for the scraping of all websites? Or only websites that allow for scraping?
If by "allow for scraping" you mean websites that give permission to scrape data, we have a help doc that addresses that: help.browse.ai/article/163-scraped-data-ownership-usage. But, from a technical standpoint, most websites *can* be scraped using Browse AI (or other similar tools). The best way to see if it's possible to extract your desired data is test it out :) And if you run into any challenges, our Customer Success team is happy to help!
Anyway to interact with the page? Like clicking elements?
Yes! Once you start training your robot, it's watching all the things you're clicking and should repeat those when it runs.
hello! is there a way to have robot B automatically append the details onto the rows created by robot A?
This is not currently possible using Browse AI alone, but we have some major new updates (including this) on the roadmap for this quarter :) You *could* use something like Zapier if you really wanted to do it. In terms of Airtable, as I alluded to at the end of the video, you can link tables such that the columns can appear within the data created by Robot A. Would you be interested in seeing how that would be done with Airtable?
@@BrowseAI love that this is going to be a new update! I was actually able to set up a workaround with Zapier + google sheets + Zapier tables. Would definitely simplify things a lot for a user if they were able to do that within the product. It took me 15-20+ hours of testing and about 1k credits to get to this point (just an FYI to help you quantify the value of the feature you're adding!).
Is there any feature in which be can extract contact details of each website url that it extracted means each different company have different design is it possible to extract contact details of each website url ?
Hmmm...this one's tricky to answer without seeing specifics. If you haven't already, I'd suggest reaching out to our support team and providing them with some more details about what you're trying to accomplish :)
How did you select all of the other info at the same time? On my bot, it only does 1 at a time. Then after I put in all the info, I end up with just 1 highlighted text, not all the other texts I expected it to copy. This is a bit frustrating.
Not sure if you're referring to the selection of each piece of text while training the robot? It should simply be a matter of hovering over what you want to capture until it's highlighted, then clicking on it. When you've highlighted all of them, you go through and name them. Hopefully you've reached out to our support team if you weren't able to get things working, as they would be happy to help you out with this!
Is there any way to automate the connection?
You could use our Workflows feature, which connects multiple robots together. Robot A gets a list of items and Robot B extracts data from each of those items (and integrates with apps like Google Sheets, Airtable, etc).
Does Browse AI have a user forum, running into issues with setting up robot scraping and I reckon other users have too.
Sorry to hear you're running into issues! We don't currently have a user forum, though we have a super responsive and helpful support team that can work with you to resolve issues.
How do i do this with a site that needs a password? 'login' button is selected but it always fails.
If you're not able to get it working yourself, our wonderful support team has some magical powers up their sleeves! You can reach them via the help button on the bottom right of our marketing site and help center.
Hey @BrowseAI can we share our robots via a link or just export it and give the file to someone so they can import the robot and use it easily?
Thanks for the awesome feature request! This is on our roadmap, but in the meantime, you can contact our support team and they can help copy your robot to another account :)
@BrowseAI No worries! Glad a company is actually innovating
Please forgive me, but in terms of trying to provide constructive criticism...I find it really hard to follow your videos. From my point of view you keep zooming the mouse around the screen, mumbling kind of fast, and correcting mistakes on the fly. It would be helpful, please, if you were to slow down a bit, coordinate your mouse movements with your speech, and edit out and re-do the mistakes you made recording the video originally. This product looks really great, and you certainly have command of it, no doubt. But, you know it so well that you zoom at a speed that your typical audience can't follow...and thus adopt your product. That's the whole point, I think. Sorry for the negative comment, really trying to be helpful from my point of view. Thank you.
Thanks for the thoughtful feedback. Truly. This is very helpful :) We could definitely make it more clear what the intended audience/experience level is for certain videos, or maybe include sections that can be skipped if a person understands those particular concepts. It's a tricky task to make videos accessible to those who know our product well and those are new to Browse AI, all the while nailing the length and pace. We're still pretty new to this, and constructive criticism like yours really is valuable. Thanks for taking the time to provide it! (note from the guy who made this video: "mumbling kind of fast" is the same complaint my wife has 😂)
@@BrowseAI might be helpful if you explain the general framework or concept, say this is the list of the url i will need for robot B to perform XYZ tasks etc. It's a bit hard follow to be honest
Can you tell me if there is Dropdown list like FAQ section in details page , How can i use Browse ai to scrape that data also?
Thanks!
That might depend on how those FAQs are created, code-wise. You can try toggling several of them open, then capturing a list. That's a great idea for a video and/or help article - thanks!
Even if you rush the video, i can do this in less than 10 min without an AI, i din't find this very practical at all
If you can truly do that yourself, that's awesome :) Can you do it on auto-pilot, scale it across multiple websites, handle captchas, adapt to changing website designs, search keywords, integrate with other apps, handle pagination, take screenshots, etc? Those are the obstacles we're helping to overcome, and the entire deep scraping process will be improved beyond what this video shows (reminder that we need to make an updated video 💪).
Bro we trying to watch these videos to see hows it done not get rushed thru it. If you want to show quickly something can get done turn it into shorts. ffs
Noted! We sometimes err on the side of providing an overview, but we're hearing from users that detailed and slower-paced videos are what would be helpful. Thanks for the feedback!