Darshil, very enthusiastic and excited I feel while listening your beautiful classes. Please explain different Hadoop and spark projects. That immensely help us to clear the interviews and should be good performer in job role. As You said , your data engineer course will lauch , please explain concept and how concept will be used in project. Very excited for your course. Your course and projects should meet all our needs. Thankyou darshiil. All the best.
@@rajeshramisetti83 Yes, that's the goal! I will try to make everything as much as easier as I do it on TH-cam Not only Data Engineering but I will try to cover everything I can from multiple domain Cloud/Devops/ and many more
Thank you Darshil for this amazing video. With the help of this I made Scraping from stock market data based on weekly or monthly gainer and exported it as CSV. it is helpful for swing trades or investment. Thank you for teaching us great things.
Hi, I was working on something similar at the moment. Could you tell me where to get live data from and what kind of data do I need exactly to begin with. This would really help me alot.
hey did your code work? because seems like i am getting blocked by amazons anti bot measure (which stops any bot from scraping their data ) could you help me out ?
It was a perfect learning example of the data mining technique of web usage recently learned about it and saw it practically in your video . Your videos are fantastic for anyone looking for a career in data. Thanks for all things that you provide
Hi bro, You are dropping very useful and productive tutorials on yt. Generally i watch python videos specially web scraping. But i didn't find any playlist on this topic,please drop more videos on this topic and make a playlist.
Bro thanks for sharing the content as alwys it's amazing ... Can you please post a video about a sample real time data engineer projects you came across and how infrastructure decided like memory etc and what steps involve from development to production. To help us to understand a full picture of actual things happen in the organisation. Thanks much
Thanks for walking through the project. Curious to know how can you automatate this process? And what needs to be done for user-agent if we want to automate this process.
Hello, thank you very much for this video. In my case the line [ links = soup.find_all("a", attrs={'class':'a-link-normal s-underline-text s-underline-link-text s-link-style a-text-normal'})] it gives me emptiness and i don't understand the reason. Can you help me? please
Hi Darshil, Good day ! Here you have shown us without logging in to the Amazon page. There is requirements like, after your login only you need to extract some info from the page. How to do that. For Example - I have to send daily a scheduled mail for VOIP Recharge amount to our team. I have to login each time to the page and paste 'your available amount: 20000 INR" to mail sub and body. How can I make it automated. (I have login details of the page but I do not have any API Key or link.) Hope I will get my answer.
I think many people, including me are getting 503 error when trying to send request to Amazon. I tried different domains but it does not work. Any other suggestion to overcome this error?
hello i love the video it so informative , i just have a little request , how to go from page to another page , like if scraped the page one , how to go to the next page for scrapping
The video was too good suggestion- If you could have written the code in the video instead of reading it in second half, it could be cherry on the cake
Thanks Darshil for such an amazing content. Just a query, since Twitter API is not free these days, is there an alternative to complete the Twitter Data Pipeline Project in that case ?
I am trying to extract the text in for loops but it shows that de find object is a nonetype object that has not the attribute text. But if I choose just one element of the list, i can obtain the text. It just does not work in a loop
Thank you Darshil. How to prevent from being blocked while scraping the data as I think I have been blocked by one of the websites that I have been scraping. Any help?
Hi, this tutorial is very easy to understand, thanks for making this tutorial. I want to ask whether the use of single (' ') or double quotation (" ") needs to be considered here?
Hmm, anyone having an issue with the data retrieved from the FIND_ALL() function? It retrieves the HREF links from A tags, but it doesn't do it in order & it skips the first A tagged, HREF element. Hmm
Respones is the output I got and after I tried to access amazon content and it is not allowing to get the contents, eventhough I used user agent. How much time we need to wait for amazon webpage to give access?
When I try to use this program to get an update on the price of the saved items in my cart, the scraper i built using the code above doesn't work. It always gives an empty list. What could I be doing wrong?
Hi Darshil, This one was very informative like your other videos. I like them because they are accurate and to the point and also project-based. I've one doubt related to web - scraping, like some of the modern web pages don't use visual pagination with the page numbers, instead they have like a show more button which renders the new results. In this case it doesn't change the webpage so how can we scrape the new results.
Similar, I have a requirement to get the cost of ola and uber for certain list of lat lon ,I am facing issue to connect to the ola and uber website due to authentication and how can I pass the lat lon to the from and to location of website to extract cost. Help me, if possible
Hey darshil, while executing getting an error ' 403 forbidden ' so pls tell me what to do in that case. Or anyone who encountered this issue how to solve it pls tell me guys..
Basic concepts and steps are ok, but you talk way too fast without pausing and you never address errors that may come up and how to fix them. This was neither easy nor comfortable, but thanks for the effort.
Hi, Thanks for the feedback. 1. My natural talking pace is fast so if needed try slowing down the video from TH-cam option 2. I can’t predict error before they come, I did flag few things such as they might change HTML structure and there are few things you need to figure out always Making slow and long videos don’t work for my channel, people watch and leave quickly that affects overall reach I need to compress many things in short video as I had to cover many thing
Don’t forget to hit the like button and tell me what you want see more?
Darshil, very enthusiastic and excited I feel while listening your beautiful classes. Please explain different Hadoop and spark projects. That immensely help us to clear the interviews and should be good performer in job role. As You said , your data engineer course will lauch , please explain concept and how concept will be used in project. Very excited for your course. Your course and projects should meet all our needs. Thankyou darshiil. All the best.
Please don't forget to relase projects.
@@rajeshramisetti83 Yes, that's the goal!
I will try to make everything as much as easier as I do it on TH-cam
Not only Data Engineering but I will try to cover everything I can from multiple domain Cloud/Devops/ and many more
one real time End To End Project please
awesome tutorial...
but how we can run to scrape one entire category or 1000 pages
Amazon block our bot
what to do at that time!!!
thank you
I always feel difficulty in doing web scraping, but you man makes it so easy and comfortable
Just loved it!!
Thank you Darshil for this amazing video. With the help of this I made Scraping from stock market data based on weekly or monthly gainer and exported it as CSV. it is helpful for swing trades or investment. Thank you for teaching us great things.
Hi, I was working on something similar at the moment. Could you tell me where to get live data from and what kind of data do I need exactly to begin with. This would really help me alot.
Not the 50 videos in a playlist but more data than the whole playlist appreciated man ♥
hey did your code work? because seems like i am getting blocked by amazons anti bot measure (which stops any bot from scraping their data ) could you help me out ?
It was a perfect learning example of the data mining technique of web usage recently learned about it and saw it practically in your video
. Your videos are fantastic for anyone looking for a career in data. Thanks for all things that you provide
Hey I need help while scraping do you mind sharing email id can we have google meet?
@@vishaljeshnani1352 Is it done or you still need help
Please don't quite any content. Because, i am learning from your content. Upload the content like learning with projects.
I feel some skipping in a blink in the video frames. for example of getting agent details.
That is really tutorial. We need such kind of practical Python tutorial.
Thank for this amazing tutorial. You made it so simple and easy to apply.
Your content and the way you are teaching, it's awesome brother.
Keep teaching us 😊...
bro u r a LIFESAVERRRR i am SO THANKFUL for this video and code
Thank you soo soo much bro, loved the way you explained, I've made a web scrapping for flipkart by just watching your video, thanks alot!!!!!
Great Zahid, I am also going to do the same 😉
Hi bro,
You are dropping very useful and productive tutorials on yt. Generally i watch python videos specially web scraping. But i didn't find any playlist on this topic,please drop more videos on this topic and make a playlist.
Thumbnail madhi tar लइच खुश झाला भो 😂😂..
Bro thanks for sharing the content as alwys it's amazing ... Can you please post a video about a sample real time data engineer projects you came across and how infrastructure decided like memory etc and what steps involve from development to production. To help us to understand a full picture of actual things happen in the organisation. Thanks much
Thanks man for your help, I could understand everything right from the start. You explained it very easily 😃
Thanks for your effort.. it is such a great learning watching your videos😊
Thank you for this video bro make on how data engineer work on daily basis
I can easily understand from your explanation . thanks
Useful. Pls, Continue like this. Thank you.. Really Appreciated.
Thanks for walking through the project. Curious to know how can you automatate this process? And what needs to be done for user-agent if we want to automate this process.
Amazing tutorial on webscraping and data preperation!
This one video is better a lot of courses
i think now amazon has changed api or something that does not accepting scraping requests as it detects scarling
Best ever and easiest tutorial.
I’m so glad I found you omggg
thank you so much bhai for giving me needed content
this video really helped me , thank you so much 😍
I found exactly what i needed! Thank you!
Really helpful keep making more videos like this ✨
Good job , amazing Tuto !!
Thx Teacher
You are a chad. Have someone ever told you that? 💯
This tutorial was amazing, can you do a tutorial to extract data from multiple pages, say 20 pages in amazon?
I'm not able to do this, when I'm extracting the HTML code from amazon, it is not having any anchor tag, instead it is saying "something went wrong"
thank you!
one more vedio for selenium plz
Hello, thank you very much for this video. In my case the line [ links = soup.find_all("a", attrs={'class':'a-link-normal s-underline-text s-underline-link-text s-link-style a-text-normal'})] it gives me emptiness and i don't understand the reason. Can you help me? please
Same problem
Bro same problem faced
Hi Darshil, i tried this but for some reason, I am unable to pull in the product title? Any tips. Thank you.
same problem
Thankyou Darshil
Thankyou bhaiya ❣️
You're a blessing.
this is awesome and hopefully you do the pagination also and this will be the best tutorial for web scraping.
bro your jawline ☠
Dada ek number
Hi Darshil, Good day ! Here you have shown us without logging in to the Amazon page. There is requirements like, after your login only you need to extract some info from the page. How to do that. For Example - I have to send daily a scheduled mail for VOIP Recharge amount to our team. I have to login each time to the page and paste 'your available amount: 20000 INR" to mail sub and body. How can I make it automated. (I have login details of the page but I do not have any API Key or link.) Hope I will get my answer.
I think many people, including me are getting 503 error when trying to send request to Amazon. I tried different domains but it does not work. Any other suggestion to overcome this error?
got any solution
@@onkarHINDU Did you got any solution for the problem?
Hi bro , try using retry mechanism I have done the same and it is returning me the data.
@@omkarpatil9717 How to use retry mechanism? can you show how you have done
Awesome, thanks 👍
hello i love the video it so informative , i just have a little request , how to go from page to another page , like if scraped the page one , how to go to the next page for scrapping
Great tutorial.. for future tutorials, can you reduce the size of your face when writing code, it is tooo big!
The video was too good
suggestion- If you could have written the code in the video instead of reading it in second half, it could be cherry on the cake
It becomes repeatable that's why I did not do it
@@DarshilParmar Ohk Understood. Thank you. Loved your content. looking forward to more videos like this
Thanks Darshil for such an amazing content.
Just a query, since Twitter API is not free these days, is there an alternative to complete the Twitter Data Pipeline Project in that case ?
I am trying to extract the text in for loops but it shows that de find object is a nonetype object that has not the attribute text. But if I choose just one element of the list, i can obtain the text.
It just does not work in a loop
Hey, I'm getting the response 503. What should I do?
same
@@aakif6182 Did you got any solution for the problem?
Yeah because the url response might not be providing details at that time due to load on webpage. You can try retry mechanism there and check
same problem
have u use any proxy and ip fingerprinting use for real human beahivour for scraping large number of data?
you're the best
Thank you Darshil. How to prevent from being blocked while scraping the data as I think I have been blocked by one of the websites that I have been scraping. Any help?
How can it be done for pages that need to scroll to load more products?
Thank you
Hi, this tutorial is very easy to understand, thanks for making this tutorial. I want to ask whether the use of single (' ') or double quotation (" ") needs to be considered here?
doesn't matter in python
Thanks
Hmm, anyone having an issue with the data retrieved from the FIND_ALL() function? It retrieves the HREF links from A tags, but it doesn't do it in order & it skips the first A tagged, HREF element. Hmm
Yes I've been having this issue also
Sir, what is the purpose of adding an Ads campaign in the storefront in Amazon and how is it done? please reply
Not working, can't pull the data out, ig amazon has changed something, though we can see html tags but can't pull the data
Hi , you can pull the data
@@omkarpatil9717 No they have applied preventive measures, you either need to use scrapy playwright or selenium to do it
Respones is the output I got and after I tried to access amazon content and it is not allowing to get the contents, eventhough I used user agent. How much time we need to wait for amazon webpage to give access?
I am also having same issue can anyone help?
@@itsdevilish9856 Did you got any solution for the problem?
please did you get any solution?????????? please please give meeeterad
@@juanignacio4353 Yes
Try retry mechanism here you would get the response
I am getting error in if_name_=_main_
NameError: name is not refined
Can anyone pls help me to resolve this error
im not getting anything. what could be the problem?
You didn't get html contents or you go 503 error then try aiohttp library
i am not getting where is anchor tag. when i check the code their only class a code. how to get a
awesome
How would you go about using proxies to avoid captchas and rotating proxies as well?
Is Amazon not letting you to scrape data.
Great
When I try to use this program to get an update on the price of the saved items in my cart, the scraper i built using the code above doesn't work. It always gives an empty list. What could I be doing wrong?
Thanks a lot brother! Very helpful!!
Hi Darshil,
This one was very informative like your other videos. I like them because they are accurate and to the point and also project-based.
I've one doubt related to web - scraping, like some of the modern web pages don't use visual pagination with the page numbers, instead they have like a show more button which renders the new results.
In this case it doesn't change the webpage so how can we scrape the new results.
You will have to use Selenium
@@DarshilParmar Ok thanks.. I'll try once
Similar, I have a requirement to get the cost of ola and uber for certain list of lat lon ,I am facing issue to connect to the ola and uber website due to authentication and how can I pass the lat lon to the from and to location of website to extract cost.
Help me, if possible
Amazon is prompting us to login when we click see more reviews.Is there anyway to handle this
i am encountering some error when i try to get text i.e. the productTitle is it because of captcha, please help
I am getting error on accept language.. telling invalid syntax
How can I scrap product price details of all products in amazon? is it possible?
😅 Nice explanation I'm using scrapy actually but thinking about how to get data everything from the link and is it gonna block my ip address and ..
i have a task that i need to get the prices of 3000 products , but the price div is different for differnt link so i am not getting the price
cant find the CLASS in ANCHOR tag, ID is there instead of class, where you are extracting the liinks ?
PLEASE HELP !!!
They keep changing structure
@@DarshilParmar so how am i going to extract the links now ?
Bro is it possible to scrap purchase data of these websites ? , i mean purchase count of a particular product.
I am getting response 503 after so many tries, can anyone show/tell me that where em I doing wrong.
Did you got any solution for the problem?
You can try retry mechanism here by sending request again and again
bro could please tell me how to get product href link and feature image href links in directory
header --- user agent
Hi Darshil can you make a video on data pipeline and how to dockeriz the entire pipeline
I will add this in my list, thanks for suggestion
Hello, newbie here..every element doesn't contain anchor tags and 'href' in the page. How to access such data in that case?
Check the tag name and you can find it by Class or ID
How much time is required to scrape atleast 5 pages or how quick is scarpping
At first you went Slow and Explained Everything , But when converting your Data to Dataframe , it was too Fast
how to to get reports from amazon seller central and amazon ads using API or we can scrape them.
❤
How do you scrape from the other next pages also
Captcha if I use your header
I am unable to get data using .find method
same here
Hey darshil, while executing getting an error ' 403 forbidden ' so pls tell me what to do in that case.
Or anyone who encountered this issue how to solve it pls tell me guys..
IP might be blocked, maybe try diffrent internet or IP
I am getting few errors here can you pleaese help me through google meet?
How to manage college study with coding
it gives me empty list in link
me too
did u find the solution
yea.. Me too. Pl post the solution
I'm getting response 503 😥 . Please help !
Are you still getting 503 error?
Did you got any solution for the problem?
Please have a look at discord link I guess its expired I am not able to join
Basic concepts and steps are ok, but you talk way too fast without pausing and you never address errors that may come up and how to fix them. This was neither easy nor comfortable, but thanks for the effort.
Hi,
Thanks for the feedback.
1. My natural talking pace is fast so if needed try slowing down the video from TH-cam option
2. I can’t predict error before they come, I did flag few things such as they might change HTML structure and there are few things you need to figure out always
Making slow and long videos don’t work for my channel, people watch and leave quickly that affects overall reach
I need to compress many things in short video as I had to cover many thing
How dumb someone can be like you , reduce the speed to .75 or less than that 😂