Scrape ANY Website with AI!!! 💥 AI powered Web Scraping 💥

แชร์
ฝัง
  • เผยแพร่เมื่อ 23 ธ.ค. 2024

ความคิดเห็น • 24

  • @vaibhavgeek
    @vaibhavgeek หลายเดือนก่อน +6

    I wonder whats' untitled 1 to 34? do you have access to AGI???

  • @romanemul1
    @romanemul1 หลายเดือนก่อน +1

    The problem at the end was that it was array of objects and it appeared as multiple objects without enclosing square brackets

  • @aadeshabhang2657
    @aadeshabhang2657 หลายเดือนก่อน

    man i cant figure this out always some IndentationError or SyntaxError: incomplete input

  • @softmangaming4251
    @softmangaming4251 หลายเดือนก่อน

    Can it scrape all pages from the website?

  • @aditya_sahani
    @aditya_sahani หลายเดือนก่อน

    Is llama 3.2 eb model running locally enough to do the scrapping? How to do it?

    • @unclecode
      @unclecode หลายเดือนก่อน

      In my experience it does a pretty job, specially if you pick up some good fine-tuned model to generate structured data, like what Groq fine tuned and its available in HF,

  • @vivekchandan6255
    @vivekchandan6255 หลายเดือนก่อน +2

    Hi bro could you please upload it using local models or if you can provide any links also helpful, Thanks.

    • @1littlecoder
      @1littlecoder  หลายเดือนก่อน

      I'll try to put together something bro!

    • @vivekchandan6255
      @vivekchandan6255 หลายเดือนก่อน +1

      @1littlecoder thanks bro once uploaded that would be helpful for many students 😁

  • @abubakkarsiddique13
    @abubakkarsiddique13 หลายเดือนก่อน

    Nice, for ollama llm, how can we setup on craw4ai, any tutorial?

    • @1littlecoder
      @1littlecoder  หลายเดือนก่อน +1

      Will work on it soon!

    • @abubakkarsiddique13
      @abubakkarsiddique13 หลายเดือนก่อน

      @@1littlecoder Thanks, looking forward

    • @unclecode
      @unclecode หลายเดือนก่อน

      It simply support ollama, in the `provider` property you just pass `ollama/MODEL_NAME`

  • @minglv
    @minglv หลายเดือนก่อน +1

    Can this method crawl all the information on the website?

    • @d.d.z.
      @d.d.z. หลายเดือนก่อน +1

      I was thinking the same but it seems is mainly to build datasets.

    • @Tanvir1337
      @Tanvir1337 หลายเดือนก่อน

      @@d.d.z. yup

    • @harisjaved1379
      @harisjaved1379 หลายเดือนก่อน

      Yes you can do it! Just pass in the field names and how you want them back

  • @st-hs2ve
    @st-hs2ve หลายเดือนก่อน +1

    Is it legal to crawl?

    • @unclecode
      @unclecode หลายเดือนก่อน

      In many cases, scraping data that is publicly accessible on the internet is considered legal. However, the purpose of using a crawler is not just to scrape other people's websites. Now, many website owners, online magazines, bloggers, TH-camrs, and even individuals with an Instagram account want to crawl their own information. Enterprise companies, having many private or public websites, forum or similar sources that they use the help of crawlers that can extract all the data. For instance, they use their own fine-tuned large language models internally, and they allow other companies to use their trawlers. As a result, they need an open-source engine to use for data enrichment and running their own servers. I believe that everybody should be able to have their own language model fine-tuned by their own personal data to democratize AI. That means everybody should be able to extract the information they have in a proper way. In the not-so-distant future, I envision everyone being able to crawl through their own messages from social media accounts and emails, and have control over them. What they do with that information is up to their own units at their own stages. This is one of the most interesting uses of trawlers.

  • @d.d.z.
    @d.d.z. หลายเดือนก่อน

    Nice and easy.

  • @MichealScott24
    @MichealScott24 หลายเดือนก่อน +1