How To Get High Quality Data For Your Website Directory

แชร์
ฝัง
  • เผยแพร่เมื่อ 31 ธ.ค. 2024

ความคิดเห็น • 111

  • @FreyChu
    @FreyChu  3 วันที่ผ่านมา +8

    Edit: Ok so it appears that a lot of you have smarter ways to automate data enrichment, which is awesome. My email and twitter/X DMs are open for anyone who is willing to chat about automating this step of the process! Would love to learn from you.
    Before you start building your directory website, making a logo, choosing fonts in Wordpress...here's the most important part of the whole directory building process.
    If anyone discovers a better way to clean data, please share them below!
    Subscribe to the directory newsletter: shipyourdirectory.kit.com/
    P.S. sorry for the bird that started chirping mid-video.

    • @joshuadav10
      @joshuadav10 2 วันที่ผ่านมา

      Please make an updated video if you get some good info! This was amazing, and I'd love to learn more!

  • @MinhNguyen-wd2wc
    @MinhNguyen-wd2wc 2 วันที่ผ่านมา +8

    YOOOO WHATEVER YOU DO PLEASE PLEASE PLEASE FINISH OUT THIS SERIES from validating to building website to pgrammatic SEO to taking the #1 Google Search result. YOU ARE THE MAN

  • @camiloaquino4466
    @camiloaquino4466 3 วันที่ผ่านมา +4

    Good job! Thanks takinfg the time to share and teach what us what you know. Looking forward to see how your directory turns out.

  • @Debblogging1111
    @Debblogging1111 2 วันที่ผ่านมา

    Enjoying your work! Trying to build an area directory

  • @marcusedvalson
    @marcusedvalson 3 วันที่ผ่านมา +4

    Thanks man, great video.
    A couple thoughts:
    1. The more data you have in your directory, the bigger the moat. While parsing through a massive number of listings is a lot of work, it also makes the data more valuable.
    2. For the handful of directories I have, APIs have been incredible. Sometimes people have already collected the data and offered it publicly. Your job is then to present the data through a new lens. Yesterday I scraped data for 615 movies, and am building a directory in a movie niche. I used Claude to build a script that scraped from multiple data sources and combined it. I just told Claude in plain language what I wanted. Worked well.
    3. My hypothesis is part of the added value of a directory is someone knowing that a real person created it. It isn’t some programmatic BOT spinning up a website. This perception adds value to the data.
    I am looking forward to your video on getting traffic

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +1

      Solid points.
      1. Yes, I agree but I noticed a larger data set leads me down to the programmatic directory path where data is dynamically pulled via a Wordpress plugin using shortcodes. It’s way easier to publish (and arguably offers better UX) but noticed it doesn’t rank as well on Google compared to fully static pages.
      2. That’s dope! I need to play around with Claude more. Did you need to clean the data still after this? Or was the data from Claude already looking cleaned?
      3. 100% agree, there are some things I do to generate human sounding descriptions for each directory listing so it’s valuable + avoids AI sound text and plagiarism. Will cover this in future vids.

    • @nathann6482
      @nathann6482 3 วันที่ผ่านมา

      Your number 1 response is cap bro. Purely your experience - no data to back it up. Others actually have the opposite view.

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +1

      @@nathann6482 oh yeah i'm 100% just talking from my personal experience. I'm not saying pSEO doesn't work at all. Its worked for me and I'm a fan. Just pointing out that those dynamic pages, compared to my static pillar page content, hasn't ranked as well for my target keywords.
      honestly just depends on the niche. this case study is aiming for high traffic/display ad monetization. Open to hearing your experience with pSEO projects!

    • @marcusedvalson
      @marcusedvalson 3 วันที่ผ่านมา

      @@FreyChu yeah I had to clean the data up, especially since it was from multiple sources. It is a non linear process. I refined some of it myself, passed it to chat gpt, etc. I went through a couple dozen iterations of my CSV before I was happy. Now I will pay my daughter to generate 600 Amazon affiliate links to fill in one of the columns. Tedious work, but tedious is another moat 😎

    • @chrisb.9856
      @chrisb.9856 2 วันที่ผ่านมา

      Thank you for sharing your process!

  • @BWBGarage
    @BWBGarage 3 วันที่ผ่านมา +6

    I just did a directory that started with 25,000 rows. Refined to roughly 1800. I was able to automate the data enrichment with google sheets addons. I think you could automate your review scraping pretty easily and clean the data with clever AI prompting. If you want some help I’d be happy to give some pointers.

    • @yoyoma4424
      @yoyoma4424 3 วันที่ผ่านมา +4

      I would love this as well!

    • @AwesomeCameras
      @AwesomeCameras 3 วันที่ผ่านมา +4

      I did something similar as well, just loading the csv into Claude AI, and having it clean the data up for me. Much better for large datasets

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +5

      Would love pointers on this! I tried to do this with the ChatGPT plugin for google sheets and failed lol
      Do you mind messaging me on X or via email at shipyourdirectory@gmail.com?

    • @hans6304
      @hans6304 3 วันที่ผ่านมา +1

      Please make a video and share! You'll be a great tutor 💪💪

    • @AwesomeCameras
      @AwesomeCameras 3 วันที่ผ่านมา

      @@hans6304 will do! I've actually been planning on doing some videos around some of my projects involving this stuff on my other channel: www.youtube.com/@joeyready
      Will let you know when I get something up on the matter :)

  • @hashim.pakara
    @hashim.pakara 2 วันที่ผ่านมา

    Man, you came again with gold mine 🎉 thanks ❤

  • @jddude9
    @jddude9 วันที่ผ่านมา

    Another banger video. Now i am getting greedy to see you build whole WordPress website with this big data. I would love to see how you monetize and display ads on the website. Thank you for doing this for us.😃

    • @FreyChu
      @FreyChu  41 นาทีที่ผ่านมา

      @@jddude9 thanks man! that’s the plan, but I think it’s gonna take a couple months for the full build out.
      Also have some concerns with building this in public will affect how it would actually rank organically, if that makes sense.
      We’ll see how it turns out

  • @luxurycardstore
    @luxurycardstore 3 วันที่ผ่านมา +1

    Fascinating video. I know even less about spreadsheets than you do, but I still want to learn how to make a directory with recent technology like AI. Approx 20 something years ago I made a local directory for doctors by manually copying and pasting the data. I enjoyed building it, but I eventually let it go because I knew nothing about getting traffic to it. Without traffic, it was a useless project.

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +1

      It's the best time to start with all this cool new technology. I'm still learning too as you can see. Also, so many great (and lucrative) directory opportunities lie in the healthcare field still.

  • @fiftyseventh
    @fiftyseventh 3 วันที่ผ่านมา

    Frey with another classic 🔥

  • @triwebdigital8436
    @triwebdigital8436 3 วันที่ผ่านมา +1

    Hey @FreyChu, as you mentioned to use the copy pasting the data from Google Maps Listings to the directory, is that allowed to copy that data like reviews and images and store them on the site as that against to google terms. there is no clear answer for this anywhere.

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา

      I haven't had any issues on my directory websites using google maps photos and reviews. Probably in the gray area, but not completely sure to be honest

  • @einthwil
    @einthwil 2 วันที่ผ่านมา

    can you please share the website template design for the dog park niche?

    • @FreyChu
      @FreyChu  วันที่ผ่านมา

      @@einthwil I’m gonna create a custom template on Wordpress for this because of how much data there is, this is the next video :) formatting data on wp, etc

  • @Newsinrealestate
    @Newsinrealestate 2 วันที่ผ่านมา +1

    I think your videos are great!!!

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา

      What's it called?

  • @myjamal89
    @myjamal89 3 วันที่ผ่านมา +2

    thanks a lot for the video.
    i am a web dev, i don't know much about wordpress but there is a way to fill the data into you website easly using Puppeteer or cypress.
    basicly it will act as a robot that control your browser, reads your csv file and fill the form then submit the data row by row.

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา

      Dude thanks! puppeteer looks super interesting. Id love to save the extra costs when it comes to creating the static directory on Wordpress.
      Would you say it’s relatively easy to learn as a non coder?
      Started learning a little JavaScript recently but literally just started lol

    • @myjamal89
      @myjamal89 3 วันที่ผ่านมา

      ⁠it’s not hard, specially if you use claud.ia or chat gpt to write the code for you.

    • @myjamal89
      @myjamal89 2 วันที่ผ่านมา

      @@FreyChu Yesi think you can succed to do it with the help of claude or chatgpt.
      note: learn about selectors in chrome dev tool , you will need it

    • @myjamal89
      @myjamal89 2 วันที่ผ่านมา

      yes you can do it if you use some ia to help your write the code. you need to learn about selector in chrome dev tools

    • @PinkKoala-k4s
      @PinkKoala-k4s 2 วันที่ผ่านมา

      @@myjamal89 Do you know any good guides/video examples when doing this with Puppeteer or Cypress? Thanks

  • @eNVy100
    @eNVy100 3 วันที่ผ่านมา +1

    Is 122,916 the max it can spit out? I get the same number for every pull

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา

      Not sure actually. This is the biggest data set I've ever scraped

    • @eNVy100
      @eNVy100 2 วันที่ผ่านมา

      @@FreyChu Then do you reccomend not doing nationwide and focusing on certain states first. To ensure you're not missing data from the scrape

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา +1

      @@eNVy100 Yeah you can go that route, check which locations have highest keyword volume and lowest diffculty and nail those pages first

  • @DannyChowmrchow
    @DannyChowmrchow 32 นาทีที่ผ่านมา

    Hey Frey, i wanna ask if im starting out to learn SEO - are they any course you recommend me taking?

    • @FreyChu
      @FreyChu  16 นาทีที่ผ่านมา

      @@DannyChowmrchow I’ve never taken an SEO course so I wouldn’t be able to recommend any.
      What I did was just spent like 100 hours learning through TH-cam back in 2022 and started a directory site as a knowledge playground to understand which SEO principles worked and which didn’t.
      It sucks to say, but SEO is changing really rapidly these days.
      Many basic principles still work but most SEO advice I read online is just so wrong. Lots of people who share advice don’t actually own and operate their own website, experimenting with their own SEO tactics.
      My rec is to use TH-cam, go down the rabbit hole of on page optimization, backlinks, internal linking, and keyword research. I will also be making vids on directory SEO in the future

  • @zakhir123
    @zakhir123 2 วันที่ผ่านมา

    Hi Frey, what do you think about using a CMS like Wordpress compared to using dedicated platforms like edirectory, brilliant directories, etc for building a directory?

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา

      I use wordpress for all my directories! Never tried any dedicate platform, just tried Geodirectory which is a WP plugin for programmatic directory builds

  • @jwalkoviak
    @jwalkoviak 3 วันที่ผ่านมา +1

    Great video thanks for sharing all this behind the scenes details. This was very helpful. Question: Do you also get the photos when you scrape the data? If you do, how do you go about adding that to the directory? is that a manual or automated process?

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา

      Yeah I get them from scraping so I don't have to manually get them. adding to the directory can be automated forsure! For a non-coder like me, I hire a wordpress dev to resize it, compress image size and then add it to my static directory pages

  • @Christopher-N7
    @Christopher-N7 3 วันที่ผ่านมา

    Thanks for the video. How many listings do you recommend launching with to validate the niche? Thanks in advance.

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +2

      There's not really a set rule for me when it comes to number of listings. It's all based on the location your directory is targeting and the keyword search volume/difficulty.
      Like, if your keyword research within your niche shows 1000 monthly searches for "[keyword] los angeles" then I'm going to make sure to add as many quality listings for los angeles to avoid thin content and increase my chances of ranking for that keyword.
      Alternatively, if "[keyword] Long Beach" is only getting 50 monthly searches, then I'd probably not spend as much time creating listings for that city.
      Hope that makes sense!

    • @Christopher-N7
      @Christopher-N7 วันที่ผ่านมา

      @ makes sense, thanks!

  • @byziad
    @byziad 3 วันที่ผ่านมา +1

    Very nice video! Would be great to have some information regarding ahrefs alternatives (free or lowcost) since it's 130$/month

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา

      There's a ahrefs lite that came out kinda recently. I probably would recommend that if anything.
      I've tried an alternative called keysearch but it wasn't nearly as good in my opinion. Data was also completely off compared to ahrefs (i used it to lookup my own videos)
      Unfortunately, you get what you pay for when it comes to these research tools imo.

  • @yoyoma4424
    @yoyoma4424 3 วันที่ผ่านมา +1

    Hi there! Question, is it legal to scrape? I wanna make sure it is. Ty and thanks for your help!

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +1

      I believe it's technically a gray area thing (depends on what data you're scraping right), but all this data is public so I lean towards it being totally okay :)

  • @bobbyw1074
    @bobbyw1074 2 ชั่วโมงที่ผ่านมา

    Thanks for sharing. Question: since Wordpress has no code templates and since you did all the work of scraping and cleaning and enriching the data, why did you hire a word press dev instead of just pasting the data in? I assume you can just paste the data in right?

    • @FreyChu
      @FreyChu  33 นาทีที่ผ่านมา

      @@bobbyw1074 yeah totally, you can just paste the data yourself for no cost. But as you can imagine for this niche, pasting 5000 listings worth of data would take a very long time.
      Even at 100 listings, it’s a lot lol
      I pay for speed so I can start another website directory build immediately after

  • @mmxcrono
    @mmxcrono 2 วันที่ผ่านมา

    Hey Frey thanks for the video! I think outscraper might be skirting around the Places API TOS by not "caching or storing" the data they send to you, and not selling the data, instead selling the service of pulling the data. However if you use the data you might violate Google's TOS, should probably be careful. Anyways GL!

  • @mohamadmj8790
    @mohamadmj8790 13 ชั่วโมงที่ผ่านมา

    What do you think is the best WordPress directory template and plugin?

    • @FreyChu
      @FreyChu  45 นาทีที่ผ่านมา

      @@mohamadmj8790 haven’t tried many of them tbh. I mainly like to build out my own templates on Wordpress rather than using a plugin.
      Next video is on formatting a static directory on Wordpress

  • @jamestucker4800
    @jamestucker4800 3 วันที่ผ่านมา +1

    There is a scraper I bought for about $30USD which also scrapes all the review keywords also

    • @julianm4500
      @julianm4500 3 วันที่ผ่านมา +1

      Drop the link mate!

    • @DougieW
      @DougieW 2 วันที่ผ่านมา

      I too would love the link

  • @imwilliamzhang2
    @imwilliamzhang2 3 วันที่ผ่านมา

    How do you upload the CSV to the website?

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา +1

      You can use a directory plugin like Geodirectory (there's a lot out there, this is just the one i use). Or you can hire a wordpress dev to write a custom script and automate publishing static pages for your directory

    • @Mackancheesey
      @Mackancheesey 2 วันที่ผ่านมา

      @@FreyChu Why not use a plugin like WP All Import? Or something similar instead of custom script?

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา

      @@Mackancheesey that can work too :) I’ve just never used wp all import, but from a quick look it looks solid

  • @PinkKoala-k4s
    @PinkKoala-k4s 3 วันที่ผ่านมา

    amazing video! :) looking forward to more videos like this! Very interesting.
    If you manage to find a good way to automate this step more, please share it here on youtube! Would be nice to see! Thanks man for the value.

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา

      Definitely need to level up my automation game, will do. Thank you!

  • @Hobnockers
    @Hobnockers 3 วันที่ผ่านมา +1

    Hey Frey, that’s cool. But I wanted to make sure I do understand what you are trying to do here.
    When you build this directory, you basically compete against the google maps directory?
    And you try to build your directory as a Wordpress website better and more user friendly as Google maps by providing a more refined or accurate search?
    Is this the idea and your business model?

    • @jrgzz
      @jrgzz 3 วันที่ผ่านมา +1

      I think you’re right on the money, and this is likely a doomed model. He mentions wanting to list the whole countries dog parks…. But Google is already doing that better than a one man team with limited tech ability could ever do. Niching down is the way to provide actual value

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา

      Yeah you've got it, that's the big picture. Identify where google maps' shortcomings are for high search volume keywords, then build out a directory where the core strength is filling that need.
      A lot of it is also formatting the information on a website better than Google maps can too.
      It's a simple idea, but not always easy. The success hinges on the competitive of the space. In the last video, the #1 rank directory for dog park near me was getting 20k/monthly searches. that's a good amount of people looking beyond google maps for what they're looking for!

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +1

      @@jrgzz Niching down is always solid. The issue is how you plan to monetize it! If you niche down too much, and your game plan is monetizing through display ads, that can be rough.
      I agree that dog parks is a really big niche...may be not niched down enough too.
      But I disagree that google is doing a better job mapping out dog parks. Based on the search volume other dog park directories are getting + social validation from reddit, I still think there's an opportunity here

    • @HammerBanger
      @HammerBanger 2 วันที่ผ่านมา

      @@FreyChu I think you are onto something here Frey. But when you are talking about $$$, what's your plan to monetize the directory? Also, really curious how you actually build your WordPress site with the directory, in terms of your structure, organisation, categories, blog posts, keywords, etc.

  • @irhapsody2010
    @irhapsody2010 3 วันที่ผ่านมา

    I came here after you did a reaction video of a directory video that gregg isenberg did withe another person and he stressed not spending ao much time on it until you justify the traffic. Do you put all this time and money just to test it?

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา

      Yeah, for the most part. I'll spend a lot of time researching and coming up with an SEO strategy based on competition and keyword research, then spend like 1-2 months building out the directory.

  • @qasimux
    @qasimux 3 วันที่ผ่านมา

    Thanks, Frey

  • @einthwil
    @einthwil 5 ชั่วโมงที่ผ่านมา

    did you do any link building to your directory sites?

    • @FreyChu
      @FreyChu  47 นาทีที่ผ่านมา

      @@einthwil yeah most def! I let it brew for a few months first to see how far on page optimization takes it
      If it naturally ranks on the 3rd page for example and stays there for a month or so, that’s when I know it’s backlink building time

  • @wacalu
    @wacalu 3 วันที่ผ่านมา

    Great video man! Is it okay to use those photos from google maps?

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา +1

      From a quick search it looks okay to use. So far haven't had issues on any of my sites using them!

    • @wacalu
      @wacalu วันที่ผ่านมา

      @@FreyChu thx =)

  • @SimonStJohn
    @SimonStJohn 3 วันที่ผ่านมา

    Frey.. great video 😊 Cant the yags be scraped? Couldn't you scrape comments and then use AI to read both and suggest tags and even write a summary based on the comments?

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +1

      Yeah definitely! Originally I was going to do it this way. I chose “review tags” under parameters when scraping and out scraper failed to give me the tags ☹️

    • @SimonStJohn
      @SimonStJohn 3 วันที่ผ่านมา

      @FreyChu oh, cool. Also, have you tried Apify?

  • @EnergiesLives
    @EnergiesLives 3 วันที่ผ่านมา +1

    So when building a directory, I will need permission from the businesses to put on the directory?

    • @AwesomeCameras
      @AwesomeCameras 3 วันที่ผ่านมา +1

      There shouldn't be any legal issues with including businesses on your directory

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา +1

      Nope, you don't need permission. They're only benefitting from an seo perspective because you're creating a citation for these local businesses

    • @EnergiesLives
      @EnergiesLives 3 วันที่ผ่านมา +1

      @@FreyChu looking forward to your course! ty

  • @fiftyseventh
    @fiftyseventh 3 วันที่ผ่านมา +1

    why not focus on the major cities first? And then expand… or maybe 10-20 parks in a city

    • @FreyChu
      @FreyChu  2 วันที่ผ่านมา +2

      Not a bad idea, but major cities are generally harder to rank for on google. Google does not like traffic-less pages, so if you target small cities it's like super easy to rank on the first page quickly and start getting some traffic. More traffic, more trust, easier it is to rank for larger pages imo

  • @gameplay6751
    @gameplay6751 3 วันที่ผ่านมา +1

    Thanks a lot for sharing all this information. i do the same thing you do for building directories with automated wordpress website with Python and scrap the data. If you want to collaborate, i will help, with Payton you can make the process more easier and faster ( also you can clean the data with your requirements and upload it directly to wordpress, Full automated site 🤝

    • @jwalkoviak
      @jwalkoviak 3 วันที่ผ่านมา

      do you use geodirectories for you sites?

    • @FreyChu
      @FreyChu  3 วันที่ผ่านมา

      I'd love this, can you send me a message on X or email me at shipyourdirectory@gmail.com? Would love to learn how you're automating this step

  • @shivanshdubey1189
    @shivanshdubey1189 3 วันที่ผ่านมา

    Bro please teach from scratch cause I can't understand which panel are you using like it doesn't look like c panel

  • @nickm1049
    @nickm1049 17 ชั่วโมงที่ผ่านมา

    So basically ChatGPT pulling stuff out their ass again 😂 I don't understand how people especially would trust numbers coming out of that thing

  • @problemsolvers7085
    @problemsolvers7085 11 ชั่วโมงที่ผ่านมา

    Hi Frey,
    I hope you're doing well! I wanted to check if you’re looking for a video editor and thumbnail designer for your TH-cam channel. I'd love to help enhance your content and visuals.
    Let me know if you're interested!
    Best regards,

    • @FreyChu
      @FreyChu  46 นาทีที่ผ่านมา

      @@problemsolvers7085 not at the moment :) thanks tho and good luck

  • @xnegusx
    @xnegusx 3 วันที่ผ่านมา +2

    waste of money apify is free

    • @ChetanRao
      @ChetanRao 2 วันที่ผ่านมา

      ... for $10 worth of scrapes, yeah, it's free. Try getting all the rows of data you want with that budget.

  • @Newsinrealestate
    @Newsinrealestate 2 วันที่ผ่านมา

    Dumb question… when you are getting info.. why are you paying?

    • @ChetanRao
      @ChetanRao 2 วันที่ผ่านมา

      It costs money for the scraper to run. Someone has to set up the software, pay for the servers, APIs, etc etc. You can dramatically reduce those costs if you know how to code and work with the Google Maps API yourself.

  • @SharkSwayze
    @SharkSwayze 44 นาทีที่ผ่านมา

    boring business?