Is there a video about automate scrapping in Python? The first one about scrapping was about python and was really useful and awesome. Thanks for being so clear and informative. Keep rocking!
Thanks! That is something we might put together in future! Our free Python web scraping tutorial is here if you need: th-cam.com/video/XQgXKtPSzUI/w-d-xo.html Rebecca
Hi, I have 52,000 urls and I need to create a search engine so that when they search for their question they get it . How do I do that ? I even have a json file . This should be done using R . If yes I can be in touch with you based on this .
Hi, like your video. Copied the code from your code repository, but I get this error: > # Create a dataframe containing the urls of the web > # pages and their converted datetimes > marketwatch_webpgs_datetimes
I when back through your video and at 5:01 there are some lines that creates urls on the screen: urls % html_nodes("div.searchresult a") %>% #See HTML source code for data within this tag html_attr("href") however, these are not in the current version of r_web_scraping_coded_example_share.R on your code repository.
Hey there! This could likely be due to datetimes being tagged under "div.deemphasized span.invisible" during certain times of the day. I briefly went over this in the video, but to help simplify the this, it is in the full script link below the video (see code.datasciencedojo.com): # Grab all datetimes on the page datetime % html_nodes("div.deemphasized span") %>% html_text() datetime # Filter datetimes that do not follow a consistent format datetime2
Hi, thank you so much for the nice video, I am new in this field and this video absolutely helpful for a beginner like me. However, when I run your coding in the part of looping titles and bodies, I got an error message which mentioned that the article didn’t exist. Can you help me to figure out, what could be the cause?
When I run the code down below new columns named Title and Body are not added to marketwatch_latest_data. Even when I copy your code and run it, it still does not work. What could be the problem? marketwatch_latest_data$Title
Thanks for such a clear step by step tutorial. I've gotten quite far in, but have faced the same issue as Mickey, where names(marketwatch_latest_data) results in [1] "webPg" "DateTime" "DiffHours" only. Would you be able to help? Thanking you in advance.
Hey folks! Glad you are following along :) Here's what could be happening in regards to your problem. It could be that it is not able to collect data that was published within an hour of whatever timeframe you have specified here: # Filter rows of the dataframe that contain # DiffHours of less than an hour marketwatch_latest_data
@@michellelai6529 Glad you are following along :) Here's what could be happening in regards to your problem. It could be that it is not able to collect data that was published within an hour of whatever timeframe you have specified here: # Filter rows of the dataframe that contain # DiffHours of less than an hour marketwatch_latest_data
Mind sharing the error message? Just checking, are you using gmail? Sometimes gmail blocks from less secure apps. Enable 'Allow less secure apps' in your gmail account. You might want to set up a separate email account for this so you don't compromise security on your personal gmail account. Or, you could try setting smtp in your gmail account settings.
your voice makes me feel like I'm on board an airplane hostess of the area
honestly man
Thank you this was a really good video, easy to follow and well paced.
Thank you for your feedback!
This is a great video. I now know how to get data from one wikipedia page. Is there a way to extract all text from all wikipedia pages?
You make it look very easy 😅
Glad to help you, Aya!
Is there a video about automate scrapping in Python? The first one about scrapping was about python and was really useful and awesome. Thanks for being so clear and informative. Keep rocking!
Thanks! That is something we might put together in future! Our free Python web scraping tutorial is here if you need: th-cam.com/video/XQgXKtPSzUI/w-d-xo.html
Rebecca
Thank you very much for the nice video. Is there a package that can beautify the email contents so that it looks in a formatted structure?
If you are joining the datetime; say when not all are visible, what are you supposed to join on?
Please make a video on clicking a button programmatically on a website using R for data extraction/scraping purposes.
What an awesome video! Congrats and keep the hard work. Hope to see more web scraping videos from you. Great Great video. Thanks a lot.
Thanks, Giuliko! Glad you found it useful. Part 2 is yet to come! Soon!
@@rebeccamerrett6536 I'm looking forward to watch it. You are by far my favorite R channel on TH-cam. Thanks a lot once again.
@@giuliko Thank you! It means a lot, and encourages to keep going :)
Hi, I have 52,000 urls and I need to create a search engine so that when they search for their question they get it . How do I do that ? I even have a json file . This should be done using R . If yes I can be in touch with you based on this .
Amazing job.
Hi, like your video.
Copied the code from your code repository, but I get this error:
> # Create a dataframe containing the urls of the web
> # pages and their converted datetimes
> marketwatch_webpgs_datetimes
I when back through your video and at 5:01 there are some lines that creates urls on the screen:
urls %
html_nodes("div.searchresult a") %>% #See HTML source code for data within this tag
html_attr("href")
however, these are not in the current version of r_web_scraping_coded_example_share.R on your code repository.
Hi so when I run this marketwatch_webpgs_datetimes
Hey there! This could likely be due to datetimes being tagged under "div.deemphasized span.invisible" during certain times of the day. I briefly went over this in the video, but to help simplify the this, it is in the full script link below the video (see code.datasciencedojo.com):
# Grab all datetimes on the page
datetime %
html_nodes("div.deemphasized span") %>%
html_text()
datetime
# Filter datetimes that do not follow a consistent format
datetime2
@@Datasciencedojo thanks for the prompt and detailed answer.
Hi, thank you so much for the nice video, I am new in this field and this video absolutely helpful for a beginner like me. However, when I run your coding in the part of looping titles and bodies, I got an error message which mentioned that the article didn’t exist. Can you help me to figure out, what could be the cause?
Im having the same problem:( have you solved this problem by any chance? Thanks in advance.
When I run the code down below new columns named Title and Body are not added to marketwatch_latest_data. Even when I copy your code and run it, it still does not work. What could be the problem?
marketwatch_latest_data$Title
Thanks for such a clear step by step tutorial. I've gotten quite far in, but have faced the same issue as Mickey, where
names(marketwatch_latest_data) results in [1] "webPg" "DateTime" "DiffHours" only.
Would you be able to help? Thanking you in advance.
Hey folks! Glad you are following along :)
Here's what could be happening in regards to your problem.
It could be that it is not able to collect data that was published within an hour of whatever timeframe you have specified here:
# Filter rows of the dataframe that contain
# DiffHours of less than an hour
marketwatch_latest_data
@@michellelai6529 Glad you are following along :)
Here's what could be happening in regards to your problem.
It could be that it is not able to collect data that was published within an hour of whatever timeframe you have specified here:
# Filter rows of the dataframe that contain
# DiffHours of less than an hour
marketwatch_latest_data
i have tried to send the mail but it is showing authentication error.. any help?
Mind sharing the error message? Just checking, are you using gmail?
Sometimes gmail blocks from less secure apps. Enable 'Allow less secure apps' in your gmail account. You might want to set up a separate email account for this so you don't compromise security on your personal gmail account.
Or, you could try setting smtp in your gmail account settings.
Her voice is soooooooooooooooooooooooooooooooooo pleasing..
**cough cough
I meant nice tutorial ❤️
thanks !!!!!!!
Welcome!
Rebecca
awesome
your way of speaking is very stressful