Thank you so much brother. Your video saved lots of my time and helped. Infect I have zero coding knowledge. Just followed your instructions its worked perfectly. Keep it up these good works. All the very best.
@@AminBoutarfi Worked awesome the other day but today I'm getting this error: result = next(iter(done)).result() playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("xpath=(//div[@role=\"article\"])[1]") ============================================================ any solution?
Hi Amin, firstly, thank you for this lesson, I feel more confident with pursuing the language. I do have an issue though , So I noticed the code is different on the github, around the listings area, I managed to get python to start chrome, open maps, input search box, it scrolls once then closes. I don't have any errors that i can see, I may just be missing a value.
Thank you for the video. By the way, on my computer it has different xpath because I use the exact same code with your video, the listings result 0 lists, is it possible?
It just worked with the update you made in Github. Out of all the web scrapping videos I watched on TH-cam, you have been the fastest and easiest. One other question, do you have a script to located emails too? @@AminBoutarfi
Hi admin, thanks for you video. It help me so much. But I have a issue and I can't fix it. Can u help me? When I search for a specific address (Currently Scraped: 1), it won't continue and has an error in page.hover. Is there any way to fix it?
This series will be more useful if you teach how to crawl the comments for each business from Google maps, as well as the ratings, or filter by rating stars
Yes that's a Google maps thing! Instead of searching in an entire city, you should search in multiple regions inside it for example since Google Maps will give the first 120 either ways.
i don't know how to get the X-path, can you guide me, i tried but the output is like this //*[@id="QA0Szd"]/div/div/div[1]/div [3]/div/div[1]/div/div/div[2]/div[2]/div/div[1]/div[1]/h1/text() doesn't seem right. I tried pointing to the element then choosing copy path
thanks habibi this works, but how can i search multiple cities at once and leave it on over night? also can i have multiple chrome browsers running? thanks
I am getting this error: Looks like you launched a headed browser without having a XServer running. Set either 'headless: true' or use 'xvfb-run ' before running Playwright. Do you know what's wrong? Your help would be much appreciated
I believe that you are running the script on a server without a UI. 1- install xvfb 2- run script as follows: xvfb-run python3 main.py -s= -t= Hope it helps!
Hi Amin Thanks a lot for the video i am getting error while installing Playwright which mention couldn't build greenlet and need to install "pyproject.toml-based" Could you help me how to fix this? Thanks never mind, i have downgraded the python version and it works Thanks
By default playwright throws a timeout error after waiting 30000ms (30 seconds) in either: 1- Internet was slow and didn't load page. In this case increase time out time: for example: page.goto('www.google.com/maps', timeout=60000). In here I'm playwright waits for 60 seconds. 2- Or you asked for an elements that wasn't found. In this case for check if element is present on the page before scraping as follows: if page.locator('your xpath'). count>0: Element = page.locator('your xpath'). Hope it helps!
Hello @AminBoutarfi after 2 problem (if page.locator('your xpath'). count() > 0: ) i have problem below . Could you please help me with it? Many Thanks. result = next(iter(done)).result() playwright._impl._api_types.Error: Error: strict mode violation: locator("xpath=//a[contains(@href, \"www.google.com/maps/place\")]").nth(9).locator("xpath=..").locator("xpath=//span[@role=\"img\"]") resolved to 2 elements:
Hello Bro this is a good tuto but still there are some erorrs , the name coms blank, scrolling not working always shows the first result even the script shows that scrolling ... please check it more more time and make a new video or update your repo. Thank u
Thanks for the feedback. Currently working on an Updated video/code. This is always the case with Web Scraping. If a website changes anything, we need to update the code so
Feature still not implemented. You need to extend the project! You would need to be able to read all input (for different cities..etc) from a file, and use something like multiprocessing or just remove the sleep time (use proxy to not be blocked)
@@AminBoutarfi Thanks for the reply. Still getting this error - playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("xpath=//button[contains(@data-item-id, \"phone:tel:\")]//div[contains(@class, \"fontBodyMedium\")]") Could me please tell me how to use the if condition so that if any of the xpaths are not available, the script won't stop and scrape whatever is available.
@@AminBoutarfi Just an update: I used the IF & ELSE statements before each xpath. However, the generated xlsx file and csv files have blank values. Can you please help me out with this?
@@AminBoutarfi An example is - if page.locator('name_xpath').count('name_xpath') > 0: business.name = page.locator(name_xpath).inner_text() else: business.name = ""
@@kunalsarmagermin8 Hi bro, did you can used if else statement for the case have no phonenumber ? could you share with me, I have a trouble with this. Tks
Unfortunately no! the reason is simple, There are no emails on Google Maps. You would need to get the websites, the go through them and search for emails
Thank you so much brother. Your video saved lots of my time and helped. Infect I have zero coding knowledge. Just followed your instructions its worked perfectly. Keep it up these good works. All the very best.
thnx for the video, but do u have a tutorial about scrolling down to the end of page in order to get all what we can ?
This is too amazing! Thank you so much for sharing. it's too helpful in my startup business :)
Thank you! Glad it helped!
@@AminBoutarfi Worked awesome the other day but today I'm getting this error:
result = next(iter(done)).result()
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator("xpath=(//div[@role=\"article\"])[1]")
============================================================
any solution?
@@user-dr6ij6xm7w Yes same here. The role atricle has been removed :(
This is amazing! Thank you so much for sharing! :)
Thank you 🙏 more to come!
Thank you so much for this video!
Thank you for your support!
Everything works fine. But "name" column is blank. Something wrong in XPATH I think
Hi Amin, firstly, thank you for this lesson, I feel more confident with pursuing the language. I do have an issue though , So I noticed the code is different on the github, around the listings area, I managed to get python to start chrome, open maps, input search box, it scrolls once then closes. I don't have any errors that i can see, I may just be missing a value.
Thanks for the support! This video is part 1, check the lastest part
how to implement all results, not just 5?
Can you please show how to include the reviews as well?
Good Idea! Will add that in the next Google Maps Scraping tutorial!
@@AminBoutarfi I'm also waiting for this video.
bon courage 5oya kmal
Hmm.. for some reason the name is coming up blank? I am looking for Courthouses in NC
Working on an updated video/code!
Do I simply need to download Python (and follow the code as per video) to scrape data from Google Maps? I am not a developer.
How to update the xpaths?? please
Thank you for the video. By the way, on my computer it has different xpath because I use the exact same code with your video, the listings result 0 lists, is it possible?
Could you show the code for using a Proxy service?
Will do that in future video!
Sir, it does not open the chrome but it prints zero as I change -l and -s values. what should I do?
I see when I make a search I can’t be specific. For example Latino restaurants in Dallas Texas. Only if I do restaurants in Texas USA.
I believe "Latino restaurants in Dallas Texas" works too, I just tried it
It just worked with the update you made in Github. Out of all the web scrapping videos I watched on TH-cam, you have been the fastest and easiest. One other question, do you have a script to located emails too? @@AminBoutarfi
Hi admin, thanks for you video. It help me so much. But I have a issue and I can't fix it. Can u help me?
When I search for a specific address (Currently Scraped: 1), it won't continue and has an error in page.hover. Is there any way to fix it?
This series will be more useful if you teach how to crawl the comments for each business from Google maps, as well as the ratings, or filter by rating stars
Arrived at all available
Total Scraped: 120
always stops at 120, but there are lots of businesses. But not continuing. Why? Do you have any solution?
Yes that's a Google maps thing! Instead of searching in an entire city, you should search in multiple regions inside it for example since Google Maps will give the first 120 either ways.
but still, 120 is too low for my search of interest. is there any way to do this thing properly? without limitation?
Did you find a way to scrape more data
how to add scrolling feature and I want to scrap data from multiple pages
I added the scrolling feature to the Github repo. hope it helps!
i don't know how to get the X-path, can you guide me, i tried but the output is like this //*[@id="QA0Szd"]/div/div/div[1]/div [3]/div/div[1]/div/div/div[2]/div[2]/div/div[1]/div[1]/h1/text()
doesn't seem right. I tried pointing to the element then choosing copy path
thanks habibi this works, but how can i search multiple cities at once and leave it on over night? also can i have multiple chrome browsers running? thanks
I'm including that feature soon! Stay tuned
Hi, data is not being save in csv and excel
I am getting this error: Looks like you launched a headed browser without having a XServer running. Set either 'headless: true' or use 'xvfb-run ' before running Playwright.
Do you know what's wrong? Your help would be much appreciated
I believe that you are running the script on a server without a UI.
1- install xvfb
2- run script as follows: xvfb-run python3 main.py -s= -t=
Hope it helps!
This video may have been released one year ago, but it’s still no longer relevant
but his github code is still relevant
thank you
Data not saving in excel and csv
Hi Amin Thanks a lot for the video
i am getting error while installing Playwright which mention couldn't build greenlet and need to install "pyproject.toml-based"
Could you help me how to fix this?
Thanks
never mind, i have downgraded the python version and it works
Thanks
Glad you fixed it. Whenever you come across that error, note that it's a version thing
"playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded" facing this error. Can you please help me with it. Thanks in Advance :)
By default playwright throws a timeout error after waiting 30000ms (30 seconds) in either:
1- Internet was slow and didn't load page. In this case increase time out time: for example:
page.goto('www.google.com/maps', timeout=60000). In here I'm playwright waits for 60 seconds.
2- Or you asked for an elements that wasn't found. In this case for check if element is present on the page before scraping as follows:
if page.locator('your xpath'). count>0:
Element = page.locator('your xpath').
Hope it helps!
@@AminBoutarfi It worked. Thanks a ton.
Hello @AminBoutarfi after 2 problem (if page.locator('your xpath'). count() > 0: ) i have problem below . Could you please help me with it? Many Thanks.
result = next(iter(done)).result()
playwright._impl._api_types.Error: Error: strict mode violation: locator("xpath=//a[contains(@href, \"www.google.com/maps/place\")]").nth(9).locator("xpath=..").locator("xpath=//span[@role=\"img\"]") resolved to 2 elements:
=========================== logs ===========================
waiting for locator("xpath=//a[contains(@href, \"www.google.com/maps/place\")]").nth(2).locator("xpath=..").locator("xpath=//span[@role=\"img\"]")
Hello Bro
this is a good tuto but still there are some erorrs , the name coms blank, scrolling not working always shows the first result even the script shows that scrolling ...
please check it more more time and make a new video or update your repo.
Thank u
Thanks for the feedback. Currently working on an Updated video/code. This is always the case with Web Scraping. If a website changes anything, we need to update the code so
When I run program it only scrape 120 and I need 1k data is it possible to scrape huge Data
You need setup user-agent and proxy aggregator for scrape huge data
@@Faybmi how to set up do you have a tutorial
How can we do For Multi Cities or Multiple Business at one time 😢😢
Feature still not implemented. You need to extend the project! You would need to be able to read all input (for different cities..etc) from a file, and use something like multiprocessing or just remove the sleep time (use proxy to not be blocked)
if page.locator('website_xpath').count>0:
TypeError: '>' not supported between instances of 'method' and 'int'. Getting this error. Can you help me?
Count is a method that you are calling! you should use count() "with parenthesis". Hope it helps!
@@AminBoutarfi Thanks for the reply. Still getting this error - playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator("xpath=//button[contains(@data-item-id, \"phone:tel:\")]//div[contains(@class, \"fontBodyMedium\")]")
Could me please tell me how to use the if condition so that if any of the xpaths are not available, the script won't stop and scrape whatever is available.
@@AminBoutarfi Just an update: I used the IF & ELSE statements before each xpath. However, the generated xlsx file and csv files have blank values. Can you please help me out with this?
@@AminBoutarfi An example is -
if page.locator('name_xpath').count('name_xpath') > 0:
business.name = page.locator(name_xpath).inner_text()
else:
business.name = ""
@@kunalsarmagermin8 Hi bro, did you can used if else statement for the case have no phonenumber ? could you share with me, I have a trouble with this. Tks
How we can earn money from this data?
how do i package it using Nuitka i m getting errors
I'm not familiar with Nuitka!
will this work on windows
Yes! On any computer OS I believe
Please could I search email address also ?
Unfortunately no! the reason is simple, There are no emails on Google Maps. You would need to get the websites, the go through them and search for emails
please reply soon
Hi, how can i contact you privately ? I would like to request a quote for a project
Hey, here's my email: mr.amineboutarfi@gmail.com
Hi, data is not being save in csv and excel
Hi, data is not being save in csv and excel
Should be fixed!