Hello, Sir, I found this video very useful! Kindly tell me guide us that if we have several pages of this kind of listing, how we get data from second page, third page and so on? Hope the answer will be hear from you soon. Thanks!!!!!
Hi Sir I am facing a similar query in my Vba code. For eg. in span if i want address of a person, the innertext of span class gives me ‘address’ as the output. But the actual address is after an element
Hi Dinesh, I have a question. In this video we can scrape data from website but only few results, only shown 10 of 17. How can we scrape all search result 17 of 17 (or 14433 of 14433)? I mean, all data. If result is on 5 pages, we can get all results, not only shown . The same situation we have in your another video ("Scraping Webpage Tables Data Using VBA") where search result took 5 pages (200 shown of 605). How to scrape 605 of 605?
Hi Dinesh, I appreciate the work you do. It is exceptionally helpful. I am, however, having some trouble with the code for this module. I have both adapted it for my own purposes as well as applied it directly as is as a test, but neither of which will actually write any data into excel. Do you have any insight into why this might be and what I can do to fix this? Thanks
Thanks for the video. What about if you need to scrape a website without knowing the layout of the site and the specific class name. How do you loop through the classnames to find out if there is one called say 'Result'?
Using a loop after the ready-state is reached and search for 'Result' in the IE document. In fact, you can get the complete HTML document into your Excel worksheet. Do a search at www.exceltrainingvideos.com
Dear Andrew, thanks for this video. You really make our lives so easier. I'm an ardent follower of your channel and have learned a lot by just tuning into your channel. I see that the video on automating web works wonderfully well. I have tried many websites and found it working without any hassles. However the same code isn't working when I try to implement in my office. I mean these codes don't work for internal websites of my organization. I'm getting automation error(unspecified error)/ disconnection error (the object invoked has disconnected from its clients) when I try to run the same code. Is there a special reason or if there is any other way round?
Dinesh Kumar Takyar I need your help in deleting blank lines in a cell. As in above video in your excel sheet after extracting data from web you are having blank lines in each cell in the sheet. so can you please suggest me how can i remove the only blank lines in cells. Thanks in advance and waiting for your reply.
+satish hurjal This link might be useful: www.exceltrainingvideos.com/tag/delete-multiple-blank-rows-between-excel-worksheet-data-using-reverse-for-next-loop/
Hello sir, getElementsByClassName method is not available in HtmlDocument class. I have checked in view browser window also and if I use it then showing invalid method or property. Please help
Dinesh Kumar Takyar I tried all possible ways with no success. Could you send me your email so I send you the code or should I send it to you here. Thank you.
while doing continuos web scraping using ie on webpage ether it timeout from session automatically or session stops doing tasks automatically. Any solution to fix this
+Dinesh Kumar Takyar Thats great thanks, I have seen this before. The following lines are the problem HTMLDoc.all.Email.Value = “dinesh.takyar@gmail.com” HTMLDoc.all.passwd.Value = “abc+123” I have to use "username" and "password" on my website. It doesnt like the two lines above and I'm not sure what tag or id to use. I have looked around and found nothing that helps.
Hi Dinesh Kumar Takyar, Thank you so much for this lovely knowledgeable video tutorial, though I have one question for you, for wait until to load a page we use below code. Do DoEvents Loop until ie.Readyestate=4 However my question is, how can we wait until some part of that page refresh on the selection of some combo box value, let me explain I want to scrap this web site uk.investing.com/indices/us-spx-500-historical-data, and when you scroll down, there is a combo box called (Time Frame), and when you will change the value from that combo box, the data below refreshed, my question is what is the code to wait until load a fresh data on that table, so I can scrap updated data as per combo box selection value, I hope you got my point. Thank you so much, God bless you.
![🔴 ฟุตบอลแชมป์กีฬา 7HD แชมเปียน คัพ 2024 [รอบ 8 ทีมสุดท้าย] - ช่วงบ่าย](http://i.ytimg.com/vi/M0S_ReCIkxg/mqdefault.jpg)
Very well explained video on web-scrapping with an easily understandable example. Excellent work.
Hello,
Sir,
I found this video very useful!
Kindly tell me guide us that if we have several pages of this kind of listing, how we get data from second page, third page and so on?
Hope the answer will be hear from you soon.
Thanks!!!!!
Hi Sir
I am facing a similar query in my Vba code. For eg. in span if i want address of a person, the innertext of span class gives me ‘address’ as the output. But the actual address is after an element
Hi Dinesh,
I have a question. In this video we can scrape data from website but only few results, only shown 10 of 17.
How can we scrape all search result 17 of 17 (or 14433 of 14433)?
I mean, all data. If result is on 5 pages, we can get all results, not only shown . The same situation we have in your another video ("Scraping Webpage Tables Data Using VBA") where search result took 5 pages (200 shown of 605). How to scrape 605 of 605?
My question exactly!
My question too. :(
Hi Dinesh,
I appreciate the work you do. It is exceptionally helpful. I am, however, having some trouble with the code for this module. I have both adapted it for my own purposes as well as applied it directly as is as a test, but neither of which will actually write any data into excel. Do you have any insight into why this might be and what I can do to fix this?
Thanks
EXCELLENT PRESENTATION
Thanks. Kindly share with your friends also.
Thanks for the video. What about if you need to scrape a website without knowing the layout of the site and the specific class name. How do you loop through the classnames to find out if there is one called say 'Result'?
Using a loop after the ready-state is reached and search for 'Result' in the IE document. In fact, you can get the complete HTML document into your Excel worksheet.
Do a search at www.exceltrainingvideos.com
Dear Andrew, thanks for this video. You really make our lives so easier.
I'm an ardent follower of your channel and have learned a lot by just tuning into your channel.
I see that the video on automating web works wonderfully well. I have tried many websites and found it working without any hassles. However the same code isn't working when I try to implement in my office. I mean these codes don't work for internal websites of my organization. I'm getting automation error(unspecified error)/ disconnection error (the object invoked has disconnected from its clients) when I try to run the same code.
Is there a special reason or if there is any other way round?
Thanks in advance
I have one doubt if class name changing every time how to take dynamic class name in ie automation
I don't know.
hello sir,
can we use excel VBA to run SAP also??
Hello Sir,
I am getting an error where the debugger is highlighting
Set html = ie.document
can you please help?
Check your code again.
Dinesh Kumar Takyar
I need your help in deleting blank lines in a cell. As in above video in your excel sheet after extracting data from web you are having blank lines in each cell in the sheet. so can you please suggest me how can i remove the only blank lines in cells.
Thanks in advance and waiting for your reply.
+satish hurjal This link might be useful: www.exceltrainingvideos.com/tag/delete-multiple-blank-rows-between-excel-worksheet-data-using-reverse-for-next-loop/
I am getting runtime error 462 'the remote server machine dose not exist or is unavailable' at 'Do While ie.readyState READYSTATE_COMPLETE'
Most probably many instances of IE are running at the same time.
Very nice, thank you :)
Hello sir, getElementsByClassName method is not available in HtmlDocument class. I have checked in view browser window also and if I use it then showing invalid method or property. Please help
Check your code or just google to get info about getElementsByClassName.
go into tools>references> then select - microsoft html objects & - Microsoft internet controls
Sir, I am still getting an error of type mismatch on that line. Can you please help
I had a small problem with this code. Could you help me on that plz ?
Check your code line by line. Or search my website www.exceltrainingvideos.com
Dinesh Kumar Takyar I tried all possible ways with no success. Could you send me your email so I send you the code or should I send it to you here. Thank you.
while doing continuos web scraping using ie on webpage ether it timeout from session automatically or session stops doing tasks automatically. Any solution to fix this
This happens when the internet connectivity is poor.
Sometimes also when you run the macro multiple times without refreshing.
Can you do this to a website that has a username and password you must login to first. I could never get past the login.
+dukestt Have a look at this link:
www.exceltrainingvideos.com/how-to-login-automatically-into-website-using-excel-vba/
+Dinesh Kumar Takyar Thats great thanks, I have seen this before. The following lines are the problem
HTMLDoc.all.Email.Value = “dinesh.takyar@gmail.com”
HTMLDoc.all.passwd.Value = “abc+123”
I have to use "username" and "password" on my website. It doesnt like the two lines above and I'm not sure what tag or id to use. I have looked around and found nothing that helps.
+dukestt You have to use your own ID and password :)
Yes, I was. It still wasn't working. I'll work on it a bit more. See what I can come up with.
error occured "user-defined type not defined"
Check your variable definitions.
Hi Dinesh Kumar Takyar,
Thank you so much for this lovely knowledgeable video tutorial, though I have one question for you, for wait until to load a page we use below code.
Do
DoEvents
Loop until ie.Readyestate=4
However my question is, how can we wait until some part of that page refresh on the selection of some combo box value, let me explain I want to scrap this web site uk.investing.com/indices/us-spx-500-historical-data, and when you scroll down, there is a combo box called (Time Frame), and when you will change the value from that combo box, the data below refreshed, my question is what is the code to wait until load a fresh data on that table, so I can scrap updated data as per combo box selection value, I hope you got my point.
Thank you so much, God bless you.
the below put a 4-second delay on the website - you can increase if you have a slower internet connection.
Application.Wait Now + TimeValue("00:00:4")