How to Fetch data from any website using ESP8266 | IoT Projects
ฝัง
- เผยแพร่เมื่อ 1 มิ.ย. 2024
- This video shows how to fetch data from any non secured website. You can fetch the data like live time, Temperature, Cricket score, Stock prices, TH-cam video views etc. You can capture that data and use it any where according to your projects.
For eg. you can make a wifi clock in which live time, temperature and weather forecast will be displayed.
You can even make a pollution monitoring display as there are many websites giving live data of pollution in different cities so you can use that data.
_________________________________________________________________
if you love watching my videos and interested in learning new new projects,then donate me at PayPal so that I can do it forever and better then before.
www.paypal.me/techiesms
_________________________________________________________________
For more details regarding this project and lot more other, visit my website
www.techiesms.com
Arduino Code - drive.google.com/file/d/0B4mv...
Like on Facebook for regular updates
www. techiesms
Follow on twitter for tech tweets,
/ imtechiesms
WhatsApp no :- +91 82000 79034
#techiesms
![เกิดใหม่ทั้งทีก็เป็นสไลม์ไปซะแล้ว ซีซั่น 3 - ตอนที่ 57 [ซับไทย]](http://i.ytimg.com/vi/pNhft3YZFRE/mqdefault.jpg)
Man! We'd be lucky to get 22deg in the summer!
You lucky clever git.
Thanks for the video. This area has been intriguing me for a while, and it's always nice to see videos that are clear and concise as yours.
All the best, and wish me luck!
Wow, Great job
Only you are the one on the whole youtube who explained in detail but yet simple to understand. Keep up the good job.
Awesome work.
Also, you didnt copy the whole code from some 3rd party website or github, but u wrote the whole code by yourself, Love it about you.
Keep up this good job on youtube man, just awesome.
This is the best explanation of esp8266 thingspeak integration I have seen on TH-cam - even with your sniffing! - thanks for posting it.
You're awesome :)
I never thought it was such simple to parse specific content from webpage.
Thanks Sachin Soni :)
Thank you :-)
The only thing i'll like to tell you is
Share it with others.
I just wanted to say thanks for taking the time to post this! Though I am using another platform as my embedded device, this video was exactly what I was wanting to learn and was able to get it working quickly thanks to your help. Great job & thank you!
Thank you for taking time to appreciate my work.
Just share it if you feel this to be worth sharing with others.
And let me know which platform have you used for your work?
Great video. You are really making a difference. Keep up the good work!
I am getting an error: "Error Parsing Document...". Please respond on how to fox or make an updated video. Everything else in the video is very easy to understand. Great Job!
Same problem
Bro, THANK YOU for this video. Really helpful.
Lots of information in a great video, well done!
hi, thank you very much for your tutorial, i could parse the time of the website as shown on the video but , the thing is im trying now to parse the wind direction of my city, and didnt work on any of all the websites that offers that information , it says "Error parsing document, try a different parse string." what can i do? thank you
Great video, well explained step by step! Thanks, very useful for me.
+sam peace feels great to hear these words.
Thanks
And share it if you liked it
that was simply more than amazing. I should give you 11 likes.
worth 20 minutes.
Hi, that was an amazing video, and I'm trying one myself.
However do you know why some of the xpath results in "Error parsing document" in thingspeak? What are the limitation of parsing text from the website?
Thankyou so much.. i was looking for ages!
great video man TQ very much for sharing! :) keep it up man!
visit my website for more such amazing projects
www.techiesms.com
Great video. Thank you so much.Very clear.
Great video! Thanks
Seeing you`re from Delhi, hope you doin well, cause of the Pandemic!
Thanks man! You helped me a lot.
Amazing video, very much appreciated!! thank you..
Hi.
A terribly complicated code for novices. Would you find some time to present the program code where it displays this time in the serial monitor. I just want to know how to easily load this data into arduino. I want to understand the mechanics you use, just the simplest code that logs into the wi-fi network and loads (API) into the serial monitor. I use wemos d1 mini. Best Regards
hey, nice video!. i'm wondering how long is the minimum sampling time to fetching the data through tcp connection(internet), can i make it like collecting the data every 500ms or smaller?
Hello, thanks for the nice video. I tried to make the same setup as you with Arduino mega. But it generates lots of compiling errors because of the library SWTFT.h. would you please upload the libraries you used so we can use them? (I looked for the libraries in internet, but they seem to be not the right ones.) thanks again.
Thank you for a nice tutorial
Wow, Great job. Thanks.
my webpage from arduino wants to send a text data to 8266wifi. what configuration I should make in 8266 sketch to receive it. Now I have to copy the text from webform and paste it at the webclient of nodemcu webserver
thanks
Thank you for showing ThingSpeak
how can i get header of any url and read its end containing "?anyvariable" and switch or work according to it.example switching led on & off from website on webhost?
Excellent work.
+Ho Brian thank you
Thanks!!...really very nice tutorial...keep sharing more videos.
+Raj Singh Parihar
sure
And keep sharing my stuffs
bhaiya i have very much feared about the coding ...so which material i can use to solve my coding related problem?
Sachin Soni thank you very much for your detailed sketch!
I have 2 questions:
- what is doing button "Views"?
- How esp8266 connected to the Arduino?
Views button fetches number of views of my youtube video.
And yes,I have connected esp8266 to arduino. And communicating by Serial1 pins of MEGA
Sir i am working on a project.. Which is.. Controlling esp32 built-in led through whatsapp.... Its done by twilio Whatsapp service... Its all working perfectly.... But i am also adding another feature that.. When we type time in the WhatsApp.. Then the esp32 will give the time from a website... And its done by thingspeak... Ya is working in the serial monitor... Its showing the time... But when i pass that value to my WhatsApp its giving garbage value like HTTP/1.1 200 OK... But not the time.... Pls help...
Thank you so much!
Hi ! How to create an app where there will be input that will be sent to the website after that out put will be given back from the website that should be displayed on our app
Hey! I have issues with fetching data from api on web on my arduino. Can you tell me what to program on arduino and what to program on esp8266? Can i upload the code just to arduino and then communicate with esp?
Thank you.
M
Checked the thingspeak error 'Error parsing document...' and found that it happens when a proper ID is not found. Tried with a weather website to get the temperature and copying the XPath gave me the complete id of the DIV which then returned the complete HTML content showing all parameters like humidity, windspeed, wind direction and many others.
fantastic! thank you!!!
hello bro i have a small doubt, im trying to fetch gold rate from a webpage but the problem is its span id has been left blank so when i try to fetch the whole span tag is what im getting as response so how to filter the data alone in that response
GOOD WORK MAN
amazing video
you rock
Amazing as always
Thank you.
Great video. thanks.
visit my website for more such amazing projects
www.techiesms.com
Does this work with local server, when the sensor shows data on its own webserver, can we provide the link of that server on thingspeak..??
Have you revised this to maybe use a ESP8266 or 32 ?
Awesome video. Keep it up.
+Alex walker Thank you for this. Just keep sharing knowledge with everyone as I do.
How can i get data from my api which is hosted on cloud every second.
Is it put load on cloud server when i hit every second from nodemcu.
Any solution?
Hello, congratulations for the excellent work... Can I access the Arduino code?
Great, thank you.
visit my website for more such amazing projects
www.techiesms.com
you are the best
Awesome
Bro.... What display you have used
Can i use this type of things to create an application???
TATA TUTU DADA DUDU... Funny English ac accent LOL 😂 😂
can we fetch data form both HTTPS secure website also
Great...Thanku
i am getting the time thats part is ok i just want the print time on serial monitor i am using node mcu
hallo, can you make another tutorial about finding a name of product by the barcode and save it. So if the next time that barcode appear/show, arduino/esp8266 already have it.
hello sachin ,Thingspeak platform which u used what are the interface process wth our hardware any documented steps u can provide ???
yuvraj upadhyay their is everything in the code
Thanks bud
hey i am stuck on this i made api of thingspeak i need help in filtering data
gooooood thanks pro
which language and which software use for run the code?
bhai title ke accordung btao ye kya hoch poch kr ke code btaya hai simple ye btao serial print kaise kre api dat ko
Nice tutorial. I have a similar Project but i'm stuck at a problem to which i cant seem to find a good solution. Basicly i send humidity and temperature to a website where it is stored in a mysql Database and then displyed on a website created by the server. What i want now, is to be able to put in setting for the arduino and send them back to it trough the web. The Arduino should then receive the data, and add it to its variables. This would enable me to basicly configure the measurement freequency and other things using the webpage. Any ideas on how to solve this?
+SunWoodMC I didn't understand this properly
Mail me with the proper explanation
I'll try to help you if possible
Can we get multiple data from one website in a single api ?
you are my inspiration. How did you learn programming so good?
By working on different projects everyday ✌🏻
the link you've shared is not working. for it to work you have to make the file unrestricted in the google drive so that anyone with the link can get the code.
Hope you do this and help me and my fello coders.
Cheers.
It is awesome ,admirable work .......
I have the same parse string as you and when I copy the url and click it says "Error parsing document, try a different parse string."
wonder what could have gone wrong ...please advice.
Once again a very admirable video and I am sharing it with my students.
As I said it won't work with all the websites.
Can I use Arduino Uno for fetching the data?
Bro I can receive data from razorpay payment captured or not
At around 5:30 just make sure to update the url from http to https.
Thanx.
I am working on a project where I interfaced esp8266-01 with arduino uno and LCD(16*2) .
The communication between Esp8266 and webpage is working fine with AT commands. I used C programming.
I tried toggling LED from webpage using ESP8266 wifi module and Arduino uno. It is working fine.
Now I have a web page where I have two text box and submit button. When I type some text on text box and click on submit button, the text should appear on LCD. I am new to web and server. I tried using get method from internet resource but could not achieve. Any idea Sir.
can you just send me the code and other relevant data so that I can study that
sachinsms1990@gmail.com
Great tutorial sachin.
how can i get the temperature details on my android phone.
you can make an app for this if you know how to make one.
Hi, As I see that the time displayed are not moving (not counting), Im thinking of delay in which on every minute it fetches the data on the host. (I see that it takes time on fetching data so updating every second would be non sense at all.) Would that affect the performance of the ESP if I do it? Im planning on esp based time, date and weather station that, ofcourse, will be powered 24/7. Any reply would be appreciated.
A very late response but pinging the API many times would not be optimal. Also there are some restrictions for APIs about how frequently they can be pinged. You could fetch the time every hour or something and then keep checking the millis difference to refresh it.
@@jacksparrow2000 thank you for the response. I guess it would be more optimized if I added RTC on the project.
@@christianacar2745 Yes Christian, that's the best option. I thought you didn't have enough space on your board ;). I use the PFC8523 for my projects but it gains about 3 seconds in a couple of days. It's a bit temperature sensitive. You could try the more accurate DS3231.
wooow it is great, but how to add ESP8266 in to proteus???
where the sound i heard in your video just like seperate from my room. it is different with other videos . had you used some special device to make this video? it sound wonderful in this way, can i have it ?
Meadow Van I'm not getting what you are trying to say.
if you want to know anything mail me at sachinsms1990@gmail.com
I want to access jiofi sd card data in nodemcu how to do it please reply .
hey is there a way to do this without using the api
Can you tell what are the connections
sir i'm working on arduino uno and esp8266 the thing is the user need
to be notified via mail about the light on and off and user sends
data on a webpage which allows for light control so regarding that
connections between arduino and wifi module im confused can u just guide
me about this
+Brundha Sholaganga mail me you doubt at
sachinsms1990@gmail.com
Still a bit dark to me how you get from "div id= ct" to
//*[@id="ct"]
You say you 'copy extag' whatever that may be.
What browser/editor are you using that actually has a rightclick copy extag?
+Ed19601 I'm using google chrome
ah ok, thanks. in the mean time i recognised the icon :-)
Very nifty way of doing this. I learned something new about thingspeak, thanks
Grt
i am not able find the x path ...please help me
How can we connect the google sheet data with esp8266
Keep up the good works.
I have been trying to get the code to hack it for my use but I just couldn't get.
Could you please help me out?
👏👍
nice tutorial thanks for these precious informations. I tried to see indian railways running status. so i generated an api using thinkspeak, the generated api key is constant for any train any station and any day. so if i have to see train running status of a different train or station then i have to edit my ThingHTTP in thingspeak.com, I noticed the resultant api key is same.
kindly suggest if there is any way to acess the content of a website based on some contents( train no, station and date) feed by user. or if the node mcu can change the ThingsHTTP contents using AT commands.
+ashish ranjan Prasad thanks for the appreciation.
And let me get a clear view of what you want.
So you want to make a program to monitor the status of the train continuously
Is it ??
Yes i want to check the status of a running train at a station of my intrest. And want to give some flexibility to user to feed train no, station and date of travel. I am using a website www.railyatri.in if i change train no or station or date of travel only 3 feilds in the complete url are changing, so the complete url can be changed from data fed by user and a lookup table. But the thinkspeak generated api key is same even if i change any feild.
+ashish ranjan Prasad I found out your solution and written a blog on that so everyone can be benefited. visit my website and do let me know if that was helpful or not.
www.techiesms.com
Hi, great video. Do you have some project showing how to fetch mp3 music using API? Thank you
fetching mp3 means what,
do you want to just fetch the URL of that song or you want to download that mp3 file.
Downlod the mp3 data from a live radio and play it using the vs1003 IC. Is it possible to do that using the AT Comands?? Tks
th-cam.com/video/Rc8va3NSI8A/w-d-xo.html
this video might help you
I already have seen this video. But they are using the Espressif SDK to compile all the code. I would like to use AT comands because I will use PIC18F2550 to comunicate with ESP8266.
+Anderson Lima I don't have idea of this project
I haven't gone through this
But I personally found this project interesting
I would like to make one for me and my subscribers. But for this I want to do research and it will take time.
Very nice. :-)
+Mads Thorup thanks.
What is the connection between ESP8266-01 with arduino?
Is it
ESP TX Arduino TX
ESP RX Arduino RX
VCC & CHPD --> 3.3 Volt
ESP GND Arduino GND ??
or something else / SoftwareSerial.
+IMPK you can do it like this also.
But in this particular project,I have used Serial 1 pins of Arduino Mega.
You can visit my website for connection diagram
Goto www.techiesms.com
And just search for "Fetching Data", you'll find the article
tx goes with rx, rx goes with tx
i got every thing correct but at the last when i hit on time button ,i am continuously get only one thing that is fetching data that it but i cant proceed ahead .
1.My ESP get connected to my network
2.APi is correct ,i have verified it
3.But still i got only one thing on screen that its fetching data......
can u please help me out
same issue bro
hello sir,Great tutorial ! I wanted to know that can we do the same thing without arduino i.e standalone esp8266 connecting to website and sending receiving data ??any code or reference ??
for that you need to use the 12th model of ESP i.e ESP8266 12e module
I have made a tutorial video for the same project using that board also.
th-cam.com/video/I6qo_2USF6o/w-d-xo.html
Thanks alot sachin :))) can i use it to get data from one of my website without using thingsspeak???
also that website needs to be logged in. And i want to GET/POST data as that example shows only for GET request. please help !
Is it possible to use a regular expression to filter the data?
Means?
Regular Expressions also called regex are a method to filter a string out of a larger string witch fits to certain pattern. This pattern is called the regex.
can i get data from a site where it requires a login? watching for your next video as a response :) (great video btw)
+everyday you won't be able to do that frm a site which requires login
that's a shame... thanks for your kind reply!
Hi,
This program won't compile for me. The error is: Error compiling for board Node MCU 1.0 (ESP-12E Module)
What am I doing wrong?
+Barry Black Actually the code which I have uploaded is not written for Node MCU board.
It is written for Arduino board in which ESP-01 module is attached
Well no wonder I am having problems!
Thanks for the response.
It's a great video anyway :)
but you don't initialise the connection (rx,tx) on the code ?
wat firmware should be used to flash it so tht wifi module can be used for this project
+PRASHANT NATH in AT command firmware
+PRASHANT NATH you can download that firmware from here
github.com/techiesms/ESP8266-Firmware-Upgrading-Files/tree/master/Flasher%20For%20ESP8266
techiesms for every project that you sent does only this firmware can be used or for diffent project different firmwares are used??? please reply... m new to this and want to do mre on it
+PRASHANT NATH for the project made using AT command I have used this firmware only
And for the project made using Arduino ide, I have used Arduino firmware
techiesms plz can u send the arduino id firmware
Hi bro.... could you please tell me in which city you lives in??
Ahmedabad
Thank you friend for great videos , but i am facing problems to create server and i am using a proxy internet connection . please solve my problem.
This is .... I need your help with the following problem:the device(esp8266) which needs to connect via a proxy to the internet. I can't set a proxy at the device directly.
I'm working in that and I'll definitely reply you if the stuff works.
Give me your mail I'd.
+Sachin Soni thanks , actually the internet connection in my institute is not proxy free ,to access net in any device we need to enter the proxy settings .
.
.
.
.
.
.
ishuj642@gmail.com
How can i download "swift.h" hardware specific library, and library "touchscreen.h"
touchscreen.h - github.com/adafruit/Touch-Screen-Library
SWTFT.h - github.com/Smoke-And-Wires/TFT-Shield-Example-Code/tree/master/SWTFT-Shield