Another good tip is add the version of the firmware on the page. Sometimes you cant remember or want to know which version is. I prefer to use the tradicional OTA included.
Yes, indeed, as I do for my web radio, so I know what the version of the code is. In PlatformIO you can do this automatically when you (for example) code for release, it will increment the version number.
Great video Ralph. Solves loads of issues I was quandering, such as putting esp32 boxes in areas that I really did not want to have to root around in after they were put into service
Indeed, just remember what I said about a test rig. It's no fun having to root around in such places because of a coding boo-boo so you can reconnect a USB cable!
Every time I watch any video of yours, I say to how much that man is amazing. Now I say it to you. Thanks for distribute that kongle throw ota with nice explanation you made.
I did OTA a bit differently, I have ESP32 LoRa to wifi gateways that I designed, as they are on a internet connected network I just have an “update” menu on the ESP which fetches the update bin from my website. All I have to do compile the sketch as a bin and FTP it to the site. Updates could be made automatic but I preferred to make it a manual process.
I do the same. Also works for devices 100 miles away. Cannot accidently update the wrong sketch and can use my own wifi-routines (list of AP's manageable via webpage and revert to AP mode when no network is found). Just simply upload updated sketch via ftp to my server. Devices check every time they come out of sleep-mode or with a reset. Multiple devices can also share the same sketch. The solution Ralph points to does not seem very secure either...
@@mdijkens Yes, security seems to often take the backseat in IoT solutions. I like the idea of converting the update process into one involving polling. Maybe there are alternative methods which build on the ‘elegant” and maybe use certificate exchanges to limit the source of updates?
Welcome aboard! I'm a bit worried about the "relatively" simple explanation but I guess there is a limit on how simple it can be. As Einstein said, Make it as simple as possible but no simpler! The good news is that I've been using this method for several weeks and it's very stable.
Yay a new video. Just about to settle in and watch it and a giant storm has knocked out our internet (Fixed wireless) connection. It’s hard to believe we’re only 150 klms away from Australia’s second largest city and only 30 klms away from Ballarat, a regional city of 110,000 people. Don’t you love the modern world of hi-tech. I’ll probably watch it over the weekend.
I don't need a storm to knock out my Internet, just a cavalier British Telecoms' engineer pulling plugs out of the local exchange cabinet with scant regard who might be affected. Happened umpteen times last week.
Many thx Ralph... hope your finger heals up quickly. This video has reminded me ... have you thought about a hints/tips on security hardening IoT projects? ... I think there are a lot of naive folk out there who assume nothing malicious ever gets onto their network and can’t understand how swarms of zombified IoT devices could be used to launch DDoS attacks or amplify other vectors such as ransomware attacks.
Working on it, Chris! Although I reckon that if someone did get something past my Wi-Fi the last thing they would be interested in is my ESP32 projects; more likely trying to encrypt my disks. But it's something I'm actively thinking about.
Thanks for sharing this here on TH-cam, I think this will be useful for me too when I'm going to play around with my Arduino Uno with the ESP8266 chip integrated into one board.
Yes, I have one of those! But I never understood why we would need _both_ chips on the one board - other than that we can use the ESP chip for Wi-Fi and the Arduino for other stuff? But the ESP chip is so much faster... less pins, though.... when I bought it, it sounded like a good idea, but now I'm not so sure.
Many thanks for this Ralph. After a little tinkering I have managed to get this running alongside a fauxmoesp server by changing the port number. I have an automated blind opener system using ESP8266 with amazon echo and I can now update the software without plugging in a laptop or having to take the units off the wall.
@ Mr.BianchiRider LOL, I often use the computer and magnetic core memory that was used to send man to the moon when expressing my feelings for modern day electronics. Usually, I say " We sent a man to the moon using magnetic core memory and room full of computer but we can't seem to blink a stinking light on a tower!"
I suspect the ESP32 has umpteen times the computing power of the moon launch in 1969. Compare: ESP32: 240MHz clock speed, dual core, 4/16/32 Mb of program storage, 300Kb runtime memory, 600 DMIPS. A couple of years after the moon launch, in 1971, Intel released the 4004, the "first commercially available microprocessor... It contains the equivalent of 2,300 transistors and was a 4-bit processor. It is capable of around 60,000 instructions per second (0.06 MIPS), running at a maximum clock speed of 740 kHz." Source: en.wikipedia.org/wiki/Timeline_of_computing_1950%E2%80%931979
I've not had an issue in this respect; in fact, the original code keeps running right through the upload process (which happens in parallel) and only stops running when the system reboots and uses the new code.
True. My thumb is _nearly_ recovered now (it's taken weeks). Still a bit tender but the bacon slice I took off with the craft knife has finally regrown. It even has fingerprints! Another couple of weeks should do it, I reckon.
Hi Ralph Thanks for the videos. You mentioned in passing about some updates for the Web Radio project. My Web Radio is used daily and working well. Always interested if you have released any updates John
The latest update, John, was a combination of hardware and software, so not for the faint-hearted! I found the ESP32 Web Radio would sometimes hang on switch on when it tried to initialise the VS1053B MP3 decoder module. I discovered that the RESET of that module and the ESP32s were wired together. This was improved by taking the RESET of the VS1053 to a separate, spare GPIO pin (25) of the ESP32. When the ESP32 is ready for the VS1053 it would bring that module's RESET pin LOW for a hundred milliseconds then continue as it always did. Since then I've not had one "hang". But it does require a (relatively simple) bit of surgery on the PCB. Additionally, I've found a simple BT transmitter module to send the audio to a BT speaker without having to press any "connect" buttons or anything; I had to create a PCB for that (it was 1.27mm pin spacing!) and it works well enough except I find the output signal very noisy even on battery. As it's early days for that project I have yet to experiment to make it quieter. I've also made some improvements in its Internet failure handling. I get cut off from the Internet on some days at least half a dozen times (it's what happens when you live in a field). The ESP32 radio does not always recover from that as it thinks it is still "half connected" (it can read the header, or gets a redirection) but can't read the stream. I want to tackle that some more before releasing that code. I'll be still tweaking this code when I'm 90, I'm sure.
@@RalphBacon Thanks Ralph for the very quick reply. I shall reread this so i understand. No internet problems but a very occasional hang when starting, a press of the reset fixes it. Regards John
If your Internet is stable (doesn't disconnect) like mine was in Milton Keynes, it will keep playing 24x7. It's the start-up issue you mention that I've fixed with that hardware/software mod. But, as you say, pressing reset also fixes it!
I've sorted the "noise" issue; if you recall I send the audio output via isolating transformers, not least because the COM of the VS1053b output should *not* be connected to ground (datasheet specifies). I forgot and connected my BT transmitter directly to the output jack. When I connected it to my built-in isolation transformers it was blissful silence (well, apart from the program content).
Check my GitHub page for this project; it mentions that there are two different AsyncTCP files; one for the ESP32, the other for the ESP8266. (Links are provided). Obviously I had to use the ESP32 version for my demo, but just change that in your setup and it will be fine, I'm sure. github.com/RalphBacon/228-Wireless-Over-the-Air-OTA-ESP32-updates
@@RalphBacon yes it seems to be the problem. for anyone trying on esp8266, you must edit instructions.h aldo change WIFI.h to esp8266wifi.h in the include line on demo.ino. and last idhave to comment all relative to debuging: // WiFi debugging //#include //#include //#include
Great video Ralph. I have used ota many times, and prefer using a static ip for the target board. With your example you only get to see the dhcp ip if you view it via serial, but if you aren't usb connected and that dhcp ip changes you won't know where to /update to (although I know there is a nameable dns route to overcome that)
Yes, you can specify a static IP address in the ESP32 Wi-Fi setup, of course, but I have set my DHCP part of my router to always give out the same IP address (if it can) so I never really did this. I think your way of a static IP address is safer (as long as it's carefully controlled to avoid clashes). Ideally, static IPs should not be part of the DHCP pool of available addresses. My DHCP can have a restricted range so I can do this. I might change my Smart Heater Controller to use a static IP address and show others how to do this! Thanks for the idea!
@@RalphBacon Thanks for the Reply Ralph, and thanks for the Videos, you have really taught me alot that I never knew, and some great tips and tricks. Regarding DHCP Pool, I usually leave it set for 1-199 for DHCP, so I know that I can statically assign from 200 onwards and know what is what, but even with that I sometimes come a-cropper when I forget that something on the same IP is still powered up :))))
I tried your code and got an error with the this line, #include . It said it could not find it. I did a quick search and found a hit from March of this year that this error was resolved by using this statement instead, #include "esp_private/wifi.h". And it works. Not sure where the dependency would be. Maybe a newer or older version of ESP32 Board Library on my end or yours. I am running ver 2.0.0 under board manager for ESP32. Just wanted to let you know.
That is left-over from my personal debugging. Oops. Sorry Mike. You should be able to just comment that line out. If you look into the "Identification.h" file you will spot some commented out lines that allow debugging of the Wi-Fi (all part of the Espressif ESP32 framework). Useful for when things go wrong but not required.
Can I call this push OTA method ? What about pull OTA. So the device(s) has to check wether any new firmware file has been added to the webserver or cloud storage. We have deploy some ARR in rural area, and build an early warning system (river flood) for the local community. And now it would cost alot when we need to edit just 1 line of the firmware code. Thank you for your kindly reply.
I briefly looked into this at the time but didn't pursue it to completion. It's something I intend to do when I have time. One way is for a Node-red/MQTT set up to publish a new update and when the ESP device (that has subscribed to that message) is ready it will go and get it. I've also looked at the Espressif documentation and there is a command to get the update; but how it does it I haven't yet determined. As I say, I haven't really delved deeply enough into it. A final method is for the ESP device to regularly poll (via a GET request) a web page (or file). If it gets a 200 (not a 404 or 500) then it knows that it can go and get the update from a known location (cloud storage) and do the biz. This allows the controller (you) to upload the new code to the cloud storage and then create the web page that the ESP device will be looking for (once an hour, or whatever) as a flag. I'm guessing and hoping someone has trod this path before me as I do want to find a way of doing this. Scour Google or stay tuned!
@@RalphBacon wow.... I really thank you for the quick and detailed reply. will stay tuned, and keep looking from forums that may have implemented the OTA update function.
Mr Pathub: If you are useng ESP32 .The easy way is to use ESP.getSketchMD5(); That will give you the current MD5 of the sketch. Then you can just compare that MD5 with a MD5 you have on an online server. If the two MD5 differ, then do something, if they match then do nothing.
@Ralph I love your videos but on this one I noticed that awesome breadboard 😮!! Do you remember where you purchased it?? Looks perfect for those funky sized dev boards
That awesome breadboard has jumped in price by about 50% since I started using them (about 1 year ago) but they are good quality AND a bit wider which makes it better for ESP32 dev boards and the like. Link: bit.ly/breadboardUK but others are available in the range.
What happens if the upload is disrupted? For example only 50% uploaded and the connection terminates? Does the lib discard the update or does it flash the incomplete firmware?
Hi Ralph, Late reply I know but thanks for this video. I’ve just implemented it and it works well. I'll now add it to my DCC throttle project to see if it works with the WiFi already in use. I thought I'd comment here as my comments on your most recent video seem to be disappearing again. Ho hum…… Regards, Colin Murdoch.
You must have _really_ upset YT at some stage, Colin, that's all I can say! Or, you are still trying to put links into your comments which will definitely get them zapped. Anyway, I'm glad you got this working, and I'm still using mine for the Smart Heater Controller although I did have to revert to USB last week as I upload a bit of rogue software that caused a continual reboot. I should take my own advice and always test upload to a test device first!
Hi Ralph, Well, it worked on the test blink program, but I’ve now added the code into my railway throttle project and have hit problems. The OTA code runs, but the program fails to run the rest of my setup code - or at least nothing shows on the serial monitor. Perhaps it conflicts with the standard WiFi library? I think I will be back to pressing the Boot button. Regarding upsetting YT - I uploaded a small video about my Arduino controlled railway traverser and stupidly said it was suitable for children. That seems to be what started the problems. Now if I respond to one of your videos and mention the name of the site where you store your code, it seems to block it. Weird or what? Regards, Colin.
Hmm, most TH-camr's avoid ticking that "Made for kids" because that really does restrict you (unless you have a channel for kids, of course). Additionally, creating _any_ link (except to another TH-cam video) seems to invoke the wrath of the YT-bots and your comment will be summarily executed with the delete key. Sledgehammer to crack a spam nut. Back to your issue of OTA updates. If the ESP32 has accepted the update (you get a one line serial output line that one has started on the receiving device) when it has completed the device will reboot and "just run" as normal (but on the new partition). Just like as if you had pressed the RESET button. So what happens in your case?
@@RalphBacon well, I’m not really sure. The update process seems to work, in that I can connect to the web server page, find the stored binary and the update runs. The ESP32 then restarts as you say, runs the OTAinit() function (I have a Serial.print to confirm this) then outputs some junk characters and does not seem to proceed any further. Now with hindsight the last upload I did was with a binary from Plarformio, so I am wondering if there was something wrong with that. I think perhaps I should try it again with a binary from the Arduino as it is easier to see the correct file to upload. Have you tried it from PlatformIO? Regards, Colin.
more like most middle east languages where all you hear is: skr sjah Coca-Cola fjio kfhj etkge Xerox fjd faslij Vegas iroweope and then you're confusedly wonder %)
If you upload code that goes awry (constant rebooting) there is little you can do, quite frankly. Well, revert to USB uploads again. The actual OTA upload either works or doesn't. Partially updated bin files just get abandoned and overwritten on the next attempt.
Yes, I thought that might be a problem (my project never sleeps). You would almost certainly have to set a flag so that on the next wake-up it would detect the flag and stay awake ready for the manual OTA. Using MQTT might be easier as you could publish a message that would be received upon wake-up and either update itself or just stay awake. Or get the ESP32 to get a webpage so see if an update was pending. Described in this thread with a link to the full code: community.platformio.org/t/ota-and-deepsleep-mode/7235/15
When the OTA start is detected can you not disable the use of the hardware timer? I have to do this in my project for an interrupt that is triggered by movement. If I do not disable the interrupt (and I move about) the whole OTA crashes with a PANIC. As I now disable the interrupt on start of OTA (programmatically) it works just fine. Detect the OTA start by this line at the top of the loop: if (Update.progress( ) > 0)
The beehive I mention in my videos is a virtual beehive; it embodies the idea that it's not somewhere you can just dive into to (re-)connect some equipment (or code). Is your first question a riddle? Like, what's the difference between a raven and a writing desk?
Hi Ralph. Elegant OTA might be good for beginners, but it has some secret code and are not so configurable. And when you upload something with any OTA, you can actually upload anyting, not only bin files. So i made my own OTA that checks the MD5 checksum of the file that is being uploaded to the ESP32. (Using cryptoJS). It seams to work perfectly, but the code is in alpha state for now. But i would love to let you give it a try if you are interested. (The MD5 checking is optional). The code even have some sort of version checking for the spiffs and sketch. It works in AP and STA mode to. PS:If you can do a video about the partions that you mentioned, that would be great!
@@RalphBacon You mean that in either case you select Firmware(Sketch) or Filesystem(Spiffs) In the ElegantOTA it allways uploads to the Firmware(Sketch)? BTW: I would like to have some kind of cantact with you outside of youtube. Is there any other way i can have a contact with you? (Email, whatsapp?)
I don't really use Facebook, far too much spam. Send me an email (look up the About page on my channel). Yes, I mean that when I upload the SPIFFs partition on my ESP32 it also zaps the program partition which can't be right! I just know I'm doing it wrong but I haven't had the time to investigate further.
It looks like there is an option for "upload_flags = --auth=password" type function, but I'm just starting to explore this. But leaving your IOT network wide open invites many more issues. Outdated firmware, default passwords, there's just so much to go wrong.
I really should pin this reply as everyone seems worried about security but not about the actual Wi-Fi this is running on! Anyway, just amend the following line: AsyncElegantOTA.begin(&server, “username”, “password”);
hi, a favor, maybe you can help me, I have been reading about the ota data partition, it has a size of 0x2000 bytes, I wanted to understand how these bytes are filled?. I thank you in advance.
In your sketch you must create a "data" folder. The contents (files) are uploaded separately from your code. I only have experience in how to do this with Visual Studio PlatformIO. Are you using the Arduino IDE?
Thank you for another great video, Ralph! I'm curious...could you briefly explain how this manages to work on a device that's already in the loop()? The only declaration I see is in setup(), nothing about OTA in the loop().
Yes I can! The OTA code (if the standard Arduino version is used) has to check at the top of the loop whether an OTA is trying to start. This (Elegant OTA version by Ayush Sharma) is more sophisticated and now does not need that check (it is monitoring in the background). If you want to know in the loop( ) whether an OTA is starting, you can use the following "if" statement: if (Update.progress() > 0) { I do this in my workshop project to turn off an interrupt that can cause a PANIC if an OTA is in progress and the interrupt is triggered. Once the OTA has completed, upon reboot it all goes back to normal. Simples. 😉
So... If my thingy already has a server.on function call this code now prevents my web server stuff from running when type in my IP address, right? How can we have both? OTA on a web server thingy???
I have both OTA and a Web Server on all my ESP32s. The libraries I am using you'll find in the code examples in my GitHub for any of my recent-ish projects, but in summary: 1. ayushsharma82/AsyncElegantOTA@^2.2.6 2. me-no-dev/AsyncTCP@^1.1.1 3. me-no-dev/ESP Async WebServer@^1.2.3 If you have "issues" using the library in #3 above (because someone hasn't updated the version number) then use github.com/me-no-dev/ESPAsyncWebServer.git which is the same but has been "fixed". Above, the first library is the OTA library and the second and third provide the web server functionality. Look at my projects (eg World Clock, Barrel Pond) for full examples. If you use interrupts, ensure you turn them off as soon as the OTA starts otherwise it will fail (best not to ask me how I know this... 3 hours of my life I'll never get back).
Thanks for the video. For me, it works using your demo program. HOwever, when I applied the OTA " modification/ add on" to one of my existing sketch which uses ESP-NOW, it looked like the OTA portion connected to wifi and then my own sketch started up. HOwever, the wifi portion of my own sketch doesn't work anymore. perhaps ESP-NOW is one of the limitations. ??
I use the following libraries, and in this order: #include #include #include Allow OTA software updates #include Then I create the AsyncWeb server first (allows web pages to be served): AsyncWebServer server(80); Here go all the "server.on" definitions (if you have any). Then start the OTA server: AsyncElegantOTA.begin(&server); Finally get the server running: server.begin(); Finally I log in as a client (station) to the Wi-Fi: WiFi.begin(SSID, PASSWORD); ESP-NOW does not require you to login to the Wi-Fi. Set the mode: WiFi.mode(WIFI_MODE_STA); Then find (and register) the peer device using the MAC address and send the data packet: esp_now_send(broadcastAddress, (uint8_t *)&sensorData, sizeof(sensorData)); Phew! How does this compare with your setup?
I have these in my main tab #include #include #include #include #include #include "Identification.h" then this right after void setup() { otaInit(); I left Identification.h unchanged. not sure why it isn't working.
When you say "isn't working", what _exactly_ is not working? Does your sketch connect to your Wi-Fi successfully? If not, post that bit of the debugging lines your sketch will doubtless send out.
@@RalphBacon thanks Ralph for following up with me. the part of the sketch that is not working relates to ESP-NOW's onDataRecv() callback. The OTA version of the sketch doesn't invoke the OnDataRecv callback when ESP-NOW messages are sent by other ESP units. Hence, I wonder if where there's already an active wifi.begin() running, ESP-NOW traffic is somehow ignored ? here are code segments.. WiFi.mode(WIFI_STA); ... esp_now_register_send_cb(OnDataSent); esp_now_register_recv_cb(OnDataRecv); memcpy(peerInfo.peer_addr, broadcastAddress, 6); peerInfo.channel = 0; peerInfo.encrypt = false;
// Add peer if (esp_now_add_peer(&peerInfo) != ESP_OK){ Serial.println("Failed to add peer"); return; } ... void OnDataRecv(const uint8_t * mac, const uint8_t *incomingData, int len) { memcpy(&myData, incomingData, sizeof(myData)); Serial.print(" Repeater OnDataRecv --- > "); Serial.print(" Received Msg : "); Serial.print(myData.wifi_msg); Serial.print(" Msg Count: "); Serial.print(myData.wifi_msg_id); Serial.print (" MAC: "); tmp_group = String(myData.wifi_msg); command = myData.wifi_command; for (int i = 0; i < 6; i++) { Serial.printf("%02X", mac[i]); tmp_mac_address[i]=mac[i]; if (i < 5)Serial.print(":"); } again, thanks for following up. I was only asking a general question . I really appreciate that you're spending time to help.
I would check that the _sender_ is actually using the correct MAC address of the _receiving_ ESP32. If so, check the order of adding the WiFi, server, OTA server and so on with the relevant "begin" statements. RandomNerds website has examples of this working.
The good thing about the ElegantOTA is that it works well with the ESPAsyncWebServer (used here). I've had no problems in several weeks' of use in my Smart Workshop Heater Controller and I include all sorts of libraries.
Hi Ralph. Great explanation, thanks. But I have an ESP32 project which already has a webserver, and if I add the required libraries, and your Identification.h I get a whole load of compilation errors, such as: error: conflicting declaration 'WebServer server' WebServer server(80); Any idea how to get this working with an existing webserver?
Oh dear; yes you will get errors as my "Identification.h" contains _everything_ including a webserver. If you edit that file you might be able to remove the AsyncServer entry and substitute your own preferred on. That would fix it, but whether the OTA works well with your particular server I cannot say.
Thanks for this great video, Ralph. 🙂 I could not compile your ESP32 code for my ESP8266, but I assume this is merely due to my lack of knowledge regarding VS Code and PlatformIO (I never used these tools until 1..2 weeks ago, and I am completely new to microcontrollers in general). Anyway - I could combine what you shared here with your recommendations regarding namespaces (in another video) and by taking code snippets from both videos I now have kind of a base for creating clean code and using OTA. It is running on 1 of my 3 devices ... and I need to remember changing project title and project description when loading it to the other 2. 😉
At least you got it running in a very short time (congrats!). If you get the chance, try and get an ESP32 module (such as the DEVKIT-C v4 that I use) and you might find "it just works" with that hardware.
Yes, absolutely. We'll assume you cannot force your device to wake up (eg by an external stimulus on a pin) and it only wakes up every X minutes/hours to do its thing. Unlike the method I show here, which is effectively a "push" OTA update, you will need to initiate an update from within the client when the device wakes up (not every time, perhaps once a day). The new image is stored in a suitable place such as a server (could be as simple as another, always-on ESP32 with sufficient filestore, whether that is LittleFS, an SD card or something else). Your device then attaches to the server, checks for an update and, if one is there, downloads it and reboots itself on the new image. I hope to cover this in a future video as it is not as straightforward as this method. Stay tuned (and subscribed!).
@@RalphBacon Yes, this is exactly it, just as you described. I've seen comments on this approach you describe but I don't know how to get there from here (i.e. I'm too inexperienced). My application is an outdoor ESP32 with an integrated BME280. As you know, this wonderful sensor will self-heat if you read it too often. Also, since they are on the same board, it is imperative that the processor not run any more than necessary to prevent heating the board, and the sensor which are right next to each other - ugh! (Adafruit Feather ESP32-S2 with BME280). I sleep it for 52 seconds, it wakes for about 8 seconds, takes a reading and sends it over ESPNOW to a receiver at my desk. Anyway, OTA updates would be SO much more convenient than having to go get the sensor board from outside, bring it to the shop, etc. But as you say, it's not so straightforward. Thanks again for all you do for the beginner Arduino community. My skills have improved a lot thanks to you, Rui & Sara Santos, and many others in forums. Slowly, I am getting to the point where I *almost* know what I'm doing!!!
Great tutorial but I was having trouble getting OTA to work on my project when it was powered through the VIN pin. Eventually got it to work when powered via USB port and battery bank. Also important to link the IP address to the MAC address on your router.
Sorry to hear that. It should, however, make no difference whether powered via the USB port or directly via the 5V pin - assuming your 5V power supply was clean and stable and capable of delivering at least 500mA (Wi-Fi takes a lot of power). I use the Vin pin most times; the only downside is that you sometimes don't get the LED on the module lit to show power is present, which could be desirable if you are trying to run from a battery and/or using Deep Sleep (as I am with my most recent project!)
Thanks for the video Ralph! You gave me an idea that I could use for an old Wemos D1 mini project I needed to update. For some reason, I can not get the Arduino IDE to upload to it any more via the usb cable. (Probably not using the right swear words.) However, the previously installed code had an "Update Firmware" option built in. I recompiled the code in the IDE and exported to a binary. I then used the web page update and sent the binary file to my mini and it worked like a charm. Now I have more time to figure out how to get the cabled connection upload to work...
yes it could be the usb cable, but it could also be the usb connector on your Wemos. Pushing in the USB cable can dislodge that micro-USB connector quite easily. If you have one of those 3v3 USB to TTL connectors....try attaching those directly to Tx and Rx of your Wemos and see if that works (make sure it is a 3V3 one)
This is Line 10 of the file "Identification.h"? I suspect you are using the Arduino IDE (not Visual Code Studio with PlatformIO) and therefore the Arduino implementation doesn't include this feature. Just comment out that line and also lines 56-57 (unless you don't get an error, somehow, if you leave them in). What this all does is tell the underlying Espressif Wi-Fi code to issue lots of diagnostic messages that can help when you can't connect to your Wi-Fi, but other than that not needed.
Hi Ralph, thanks for the info. I have an ESP32 application that is already utilizing WiFi and connects to the internet and an MQTT server, along with using the WiFi Manager library to configure the application without hard coding the SSID and Password. I am wondering if this OTA library would work with all that other stuff going on?
It should do, but whether the web server code you use works well with this particular Elegant OTA I cannot say. You will have to edit the "Identification.h" file and substitute a call to your own webserver. If you already have that up and running I'm sure you could work out what needs to be done.
Hi Ralph, it's me again !!! While OTA works for some of my sketches after I changed the wifi.mode to ap_sta. However, I have a sketch that uses , with OTA installed, the sketch would fail on a "httpGETRequest". (the OTA portion works flawlessly though... at least something is working !) 1:24:25.110 -> I (29876) wifi:ap_probe_send over, resett wifi status to disassoc 11:24:25.206 -> I (29876) wifi:state: run -> init (c800) 11:24:25.238 -> I (29876) wifi:pm stop, total sleep time: 0 us / 22804210 us 11:24:25.301 -> 11:24:25.301 -> I (29922) wifi:new:, old:, ap:, sta:, prof:1 11:24:25.529 -> Error code: -1 Just an fyi .... this just happened .... I'll go and see if I could find a solution. thanks for listening.
I think you have to be careful which libraries you use with which. Espressif (and me-no-dev) have various examples of OTA working alongside other Wi-Fi requirements, so do check them out!
Hi Ralph Could this be done over the internet, say if the bin file was on a google drive and a web page on the same drive and people go to the address of a google link then people can update their ESP32 if the ESP32 in on the internet
I don't see why not. If you can address the /update page of the ESP32 then the PC simply asks for the file, regardless of where it is. Should work, but I've not tried that(yet).
No, not really. Well, not as a non-professional. You might want to do it if you run a collection of sensors for a water company, for example. But not for just the couple I have in range of my own wi-fi!
Hello! If the esp32 operates in Ap mode and creates a html page and displays data using websocket, can this ota update still work in some form? How can I somehow bring the two together?
If you operate in AP mode (instead of STA mode) then you are creating a brand new network, not linked to your current Wi-Fi. So if you connect to that network (in this case, the ESP32) you can still use the OTA feature. From your phone, for example, assuming that you have the .bin file on your phone. But then your phone won't be connected to your home Wi-Fi so you can browse elsewhere for that file. On a PC you could have two network cards, connected to two different networks.
@@RalphBacon I tried this earlier, but something didn't work, so I left it, but I'll try again. All the demo programs I found are in STA mode, so since I haven't used the wifi functions so far, it wasn't easy to start.
@@RalphBacon Actually, there isn't. What I'm using is a 3-axis camera moving system I made at home, I can see its parameters (position, speed, endpoints) by connecting to the WiFi of the ESP32 and displaying the data on a laptop or a tablet. This works even if there is no accessible network nearby. I only had time for a short test so far, if I delve deeper into it I think it will come together and then I will try STA mode. By the way, when I reach the final version, it won't be so important to upload the program to the ESP, but now if I'm working on the code, this method would be easier than always connecting a cable to the ESP. By the way, your videos are extremely informative and detailed!
Hi Ralph. Once again a very useful video. I'm writing an ESP32 project at present so I’ll give this a try. My project includes a WiFi connection to an esp8266 using the client.connect(address, port) construct. Is there likely to be any confusion between two possible Wi-fi connections or should they be able to exists side by side so to speak? Regards, Colin.
I'm using OTA, a web server and ESP-NOW all at the same time and they work just fine. On that basis, you _should_ be OK but it does depend, I suspect, on what libraries you are using. The only way to know is to try it! You can do that on the desktop with a spare device.
@@RalphBacon My code at present (untested yet) is using the standard WiFi library which is quoted in your header file, so hopefully no conflict. As you say, try it out and see.
No, I have not progressed that (yet) but it is becoming more pressing as time goes by. Still very much on my list, Duncan, you have a need too? One issue for me is that I'd like to upload the data partition without zapping the program (sketch) which happens when doing it via USB, for some reason. I need to investigate more. Well, start to investigate. 😮
Oh yes. The Americans left a solar-powered Wi-Fi unit up there (in fact, more than one) and the coverage is good, no interference. They did it so they can communicate with the moon-based modules up there, when they are in orbit. Which could be in 2024, next.
Hi Ralph, thanks for the great video. I have a question. In my project, in the very beginning I tried to use OTA drive and it worked. So I said yea it works, perfect! and moved on. Time passed and now I have like 2000 lines of code. I have tried OTA again and it didn't work. After some checking I found out in the Serial monitor it said :" handleUpdate(): FreeSketchSpace to low (1310720) needed: 1544640, flush(): fail on fd 48, errno: 11 " no more processes" bla bla. So when I checked my bin file it was like 1.47mb. I realize it is due to the size of my sketch file. So how can I use OTA under these circumstances, because I really do need OTA updates in this project. Would this alternative ElegantOTa work for me? Or what would you suggest?
It's all down to your partition size(s), Okan. Assuming you have a 4MB flash ESP32, how do you split up that memory? Some of it is required by the OTA system but I would expect a 1.5MB program to fit (it needs twice that, so 3MB on the ESP32). I will be doing a video on how partitions are created and managed on the ESP32 "soon", can you wait that long. If not, email me and I'll send you a suggested partition .csv file.
@@RalphBacon Thank you Ralph, you are so kind. I thought the same, I tried everything I could but no. It just gives an error there in my code for some reason. I have trried different partitions as well, for instance in my platformio.ini file, I added this partition "board_build.partitions = min_spiffs.csv" to keep spiffs as little as possible, leaving more space for OTA and code. No, didn't work. I tried huge_app.csv the similar way, again no. Here is the error message: [ 23646][E][tinyHTTP.cpp:99] get(): HTTP error: 304,HTTP/1.1 304 Not Modified. On OTA drive I see the number of requests number tick one up everytime I try an OTA update but nope..doesn't download anything..
@@RalphBacon ok it worked. I feel like a true idiot. It didn't work because the version I have on my device and the version I uploaded on the OTA drive are the same. So I suppose it checks the versions in advance, if the versions are the same, why bother download! Damn! 😂😂😂😂Thanks anyway Ralph!
Hi Rodney, I got it loaded with all the libraries, but when i type in the address it can't be found. The script i am using already connects to the wifi, could there be a conflict? ie, i was posting info to my webserver and added this to the script.
I just flashed your demo code, work perfectly. I wonder if its an issue since i am connected to my network with my own code, not the code in the "identification.h" file.
Check the IP address of your Esp32 device on start up. If that's not the problem I'll look at it tomorrow if you send me the output from your serial output.
@@RalphBacon Well, i get the IP address and other info as i should with the demo.ini file, but in my code, i just get my serial.print stuff. It acts like it isn't loading the .h file. but i get no error saying it isn't loading. weird..
@@RalphBacon Ok, got it working.. all my fault, works perfectly as you documented. I would explain the issue, but its to embarrassing. Can I send you some drinking money? do you have a paypal account? This really rocks..
Don't be embarrassed, Jim, we all do silly things. Glad you got it working. If you feel that pleased you can always buy me a coffee! buymeacoffee.com/ralphbacon
without oTA, like arduinos & such....when I'm done, I package a usb cable with , or port it to the side of the box. Usually epoxied in place. I'm excited to try this OTA stuff, but I think I'll continue my practice of having an accessible port, as a fail safe.
Even with OTA, it's all too easy to forget something and have a "bricked" device that is probably constantly rebooting itself. Don't ask me how I know this! A separate USB port as you suggest is a very sensibly suggestion.
@@RalphBaconI finally got it all working, few kinks to work out in the new versions of what you showcased. I made it harder than it should have been (PIO was being prickly). Many thanks for laying out a great template. Oh and on about my 3rd upload, I got turned around in my projects folder and sent some random firmware.bin file to one. Fortunately, they are still on the bench.😁 You happen to have any videos covering switching between espNow & OTA ? I need espNow for normal operation, I've set up an upload mode where espNow stops for OTA. al fine. I'm trying to deal with a case where I accidentally enter OTA mode, but dont want/cant sent an upload, also failed uploads(not seen 1 yet). Gettin the espNow running again, without restarting the board. last question, any good forums you'de recommend for this sort of thing?
@@alexrossouw7702 Yes, thats true. I use it for controllers without a serial interface. Sometimes I use the syslog option or even XMPP. But you're right. print("bla") is easier.
@Fil lempie Yes, we can do this using the WebSerial library but I'm still in the early stages of testing - It's something I hope to cover in a future video as I missed monitoring my ESP32s!
I saw his video of comor upload sketch to sp32 wirelessly everything has worked well until I decided to upload one that consists of sending by Bluetooth messages to a matrix of LEDs, that if I did not want to upload it wirelessly tells me it does not compile for board sp32 but it happens only with that type of program that have Bluetooth then that via I can use to load the sp32 these programs wirelessly.
@@RalphBacon It will be that programs that carry Bluetooth can not be charged wirelessly with OTA because the libraries of both communications are not compatible I think
If that is true then you could try a different library for either BT or Wi-Fi - preferably two that have been proven to run side-by-side without issues.
Well, as I said, I would try finding two libraries (Wi-Fi & BT) that work together well and don't stop BT from working. Many others must have done this successfully. Buena suerte!
There are many options for shipping, Bill. I even tried the UK Post Office method for $8. It took a couple of weeks but turned up. There must be cheaper options for the States and elsewhere too, I would have thought.
Cool. I suspect lots of advanced users roll their own, so to speak. Well, in some ways, this is the beginning of my own method (that I've now updated to include the software version number).
There is indeed. Change the line in the "Identification.h" file from: _AsyncElegantOTA.begin(&server)_ to: _AsyncElegantOTA.begin(&server, “username”, “password”);_ There's a lot of comments on security but, frankly, if a hacker can get to our Wi-Fi I think we'll have bigger problems than wondering whether he hacked our ESP32 devices! Like where to send the Bitcoin ransom payment to unencrypt our disks!
@@RalphBacon Thanks Ralph. I was not thinking about security, more about someone accidentally overwriting the wrong device software in a facility where multiple users and devices exist.
When I try to 'Export compiled binary' I get this error In file included from C:\Users\my profile folder \Documents\Arduino\demo\demo.ino:12: C:\Users\my profile folder\Documents\Arduino\demo\Identification.h:10:10: fatal error: esp_wifi_internal.h: No such file or directory #include ^~~~~~~~~~~~~~~~~~~~~ compilation terminated. exit status 1 Compilation error: esp_wifi_internal.h: No such file or directory This is trying your demo file.
Does it compile and upload normally, that is, via the USB cable. Always get that bit working first. Have you actually got a path to the "missing" library? It should be in the libraries folder (which should be in the sketch folder)?
First of all, thanks for your effort to share this tutorial. I would like to use an esp32 or esp8266 so that, for example, if I wanted to change the SSID and Password remotely on the same network, would this be possible? If not, which way to do it? I note here that I don't understand English correctly. Greetings and my thanks
Do you have multiple Wi-Fi networks? If so, you can programmatically change the SSID and network from within the sketch and reconnect to a different network, no problem. Ensure the SSID and Password variables are not constants as you cannot change a constant variable. Is that what you mean?
@@RalphBacon First, thanks for your reply. No friend! I gave the example of the SSID and Password but I explain it in another way: I have a LED Matrix Clock with ESP8266 mounted on the wall, away from the computer. Once a year it changes daylight saving time and I don't want that, so when it does (change daylight saving time), I want to use the OTA function to remotely fix daylight saving time so I don't need to take the ESP from the wall and bring it to the computer. Did you understand? Thanks
@@RalphBacon Thanks Ralph but I have another question please: As I said, I have this 'LED Matrix Clock with ESP8266' so let's say I want to make several of this and take/install in different places (houses, apartments) so on different networks; So, in principle, in each house or apartment that I bring the 'Matrix Clock LED', I must change the SSID and Password ok? But, wouldn't there be a better way not to have to do this every time? You understood me? I am grateful to you and waiting for an answer.
quite comprehensive video - thank you however, few things: 1- you exposed your IoT without password - anybody can now change your code on the remote device - this library gives also possibility to at least provide user/password however no https 2- if your IoT goes to sleep you need to add code to stop sleeping when OTA is in progress (i.e. by MQTT or so)
By changing one line you can have user/password challenge but if someone has hacked your Wi-Fi will that stop them? Anyway, just amend the following line: AsyncElegantOTA.begin(&server, “username”, “password”);
I don't know the definitive answer to that as I use it with standard ESP32 WiFi and the AsyncWebServer library. Try the GitHub readme that has developed this to see whether he mentions WiFiManager: github.com/me-no-dev/ESPAsyncWebServer
Also, this implementation is very insecure even for local network. Very few WiFI APs can properly isolate such insecure devices and even in case of private VLAN if one of them is compromised then others are easy targets. If your IoT device is already connected to Internet, I'd better hardcode some 99.(9)% uptime SLA (2FA secured, ofc) URL like share from Dropbox or the like which is just a text file where you put an actual link (and ver. num. + hash) to new firmware for OTA routine. Do not hardcode direct link to binary though.
My device, Leeroy, is not connected to the Internet _per se_ , only internally. That is, I have no port forwarding to allow it. Should someone get in past my router it's curtains for me. I guess we could use a different port other than the standard 80; we could use a username/password (one-line change mentioned in my other replies); we could use 2FA via a mobile phone - but this is a Smart Workshop Heater not my bank account so where do we stop?
@@RalphBacon All these are subjective choices for your one particular use case. You may be comfortable with them, that's beside the point. But keep in mind that you are not only documenting your own creations but also teaching others (bad practice in this case) who would most likely copy-paste first, then maybe but not necessarily properly evaluate and complete design.
Harsh words, Leeroy. Ouch. I'm not sure the millions of IOT devices installed in consumers' homes are protected any further than their router. Sonoff devices, for example. Any number of LED strips. Even amazon Echo. I once saw an episode of Big Bang Theory where some guy in China started dimming the lights of the team's apartment! Anyway, I'd love to get some constructive feedback in what are Best Practices in securing IoT devices, if we don't do it at the router?
@@RalphBacon by router I assume you mean firewall portion of it, in that case it only filters traffic between its network interfaces. So, e.g. in your network you separated IoT from PCs, good firewall can do that. What it can not on the other hand is to control traffic between ethernet cards on one segment of net (connected to one switch and corresponding to the same net interface of firewall in other words). So even though you separated IoT devices, they still may bypass firewall while communicating to each other and you have a lot more work to do to be able to control this relations. It is a bit more complicated than I described with WiFi, but then again it is less secure in the first place. And you don't have to forward ports to device to enable it to act maliciously. Often, its enough if this device is able to reach anything other than RFC1918. Yes, current state of IoT security is awful and it won't change unless we force hardware encryption accelerators adoption and supporting software as a standard peripheral for our uC of choice. Right now it is expensive. And all 'smart' devices simply skip this part for keeping costs down.
And keeping the costs down is probably fundamental to the manufacturers of any of these devices. But I still maintain that if someone gains access to your network (Wi-Fi or hard wired) then everything else will probably fall like dominoes anyway. The thing is, unless someone really, really wants to hack _your_ system I think we are safe by virtue of obscurity. I suppose we could change the port on the webserver here but any hacker will probably try the obvious ones (81, 8080, 8181). The best I'm hoping for is that no-one is interested in the slightest in my set up. But you have worried me a bit though. 🤦♂️
Ralph, I have been trying to get this to work for a while this morning. I keep getting these errors when trying to compile. exit status 1 Error compiling for board ESP32 Wrover Module I deleted all the libraries that I already had loaded and installed the ones you listed thinking that would fix it but no joy. I also tried other esp32 and esp8266 boards to no avail. I then commented out all the include library statements and found that the include AsyncTCP.h was the one causing the errors. So I deleted the libraries I had installed and downloaded the library again, making sure to get the one for the ESP32. Do you have any clue?
Note that there are two libraries for the AsyncTCP; one for the ESP32, one for the ESP8266. Ensure you have the right one installed for your board. If you still get errors, email the output so I can investigate it for you.
@@RalphBacon I did verify that I had installed the correct version for the esp32. Here is the messages it gives when trying to compile. Using library AsyncTCP-master at version 1.1.1 in folder: C:\Users\Documents\Arduino\libraries\AsyncTCP-master Using library AsyncElegantOTA-master at version 2.2.6 in folder: C:\Users\Documents\Arduino\libraries\AsyncElegantOTA-master Using library WiFi at version 1.0 in folder: C:\Users\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\libraries\WiFi Using library Update at version 1.0 in folder: C:\Users\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\libraries\Update Using library ESPAsyncWebServer-master at version 1.2.3 in folder: C:\Users\Documents\Arduino\libraries\ESPAsyncWebServer-master Using library FS at version 1.0 in folder: C:\Users\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\libraries\FS Using library Credentials in folder: C:\Users\Documents\Arduino\libraries\Credentials (legacy) exit status 1 Error compiling for board ESP32 Dev Module.
There's no error I can see in the bit you pasted. You need to send me the whole thing from the time you press Verify to when it completes. Ideally in an email, not here, but...
No, Bernard, I use it all the time here, very reliable. However, I did discover that if you have ANY kind of interrupt routine running (or triggered) whilst the OTA is in progress then the OTA update will fail. I have to now disable the interrupt when I detect an OTA is in progress in the main code. It gets re-enabled automatically when the device reboots after the OTA.
As part of the OTA process there will have been declared (not by you) the following: UpdateClass Update; This means you now have access to the Update object. In my loop, I test for the update running and disconnect the interrupt like this: if (Update.progress() > 0) { // The interrupt will cause a PANIC if triggered during OTA so detach detachInterrupt(digitalPinToInterrupt(pirPin)); log_v("Interrupt for PIR detached during OTA"); }
I think you've been watching my previous videos at x1.5 or something like that - this is my normal voice! Really. No cold. No corona (apart from the beer). 🤣
Hang on, as soon as you hear "Arduino" on an Arduino-based YT channel you stop watching? Have I got that right? (This video is about the ESP32 though).
Ralph, you uploaded this a year and a half ago and it's still helping folks...like me. Thank you.
Glad it's helping Arduinites around the world, Hugh!
too many silent "L"
Another good tip is add the version of the firmware on the page.
Sometimes you cant remember or want to know which version is.
I prefer to use the tradicional OTA included.
Yes, indeed, as I do for my web radio, so I know what the version of the code is. In PlatformIO you can do this automatically when you (for example) code for release, it will increment the version number.
Great video Ralph. Solves loads of issues I was quandering, such as putting esp32 boxes in areas that I really did not want to have to root around in after they were put into service
Indeed, just remember what I said about a test rig. It's no fun having to root around in such places because of a coding boo-boo so you can reconnect a USB cable!
Every time I watch any video of yours, I say to how much that man is amazing. Now I say it to you.
Thanks for distribute that kongle throw ota with nice explanation you made.
Wow, thank you! 👍
I did OTA a bit differently, I have ESP32 LoRa to wifi gateways that I designed, as they are on a internet connected network I just have an “update” menu on the ESP which fetches the update bin from my website. All I have to do compile the sketch as a bin and FTP it to the site. Updates could be made automatic but I preferred to make it a manual process.
Could you share your code?
I do the same. Also works for devices 100 miles away. Cannot accidently update the wrong sketch and can use my own wifi-routines (list of AP's manageable via webpage and revert to AP mode when no network is found).
Just simply upload updated sketch via ftp to my server. Devices check every time they come out of sleep-mode or with a reset. Multiple devices can also share the same sketch.
The solution Ralph points to does not seem very secure either...
@@mdijkens Yes, security seems to often take the backseat in IoT solutions. I like the idea of converting the update process into one involving polling. Maybe there are alternative methods which build on the ‘elegant” and maybe use certificate exchanges to limit the source of updates?
It's great that some of you are well ahead of the curve. Let's hope this video is the starting point for others to follow in your footsteps. 👍
@@RalphBacon you are absolutely right about that Ralph. Keep up the good work. Really appreciated
Finally a straightforward and relatively simple explanation, you just earned a like, subscribe and I even rang the bell...
Welcome aboard! I'm a bit worried about the "relatively" simple explanation but I guess there is a limit on how simple it can be. As Einstein said, Make it as simple as possible but no simpler!
The good news is that I've been using this method for several weeks and it's very stable.
"We don't need the elf" - Sage words given the time of year ;-)
Drat! No presents for me this year then 🎅🏽
Yay a new video. Just about to settle in and watch it and a giant storm has knocked out our internet (Fixed wireless) connection. It’s hard to believe we’re only 150 klms away from Australia’s second largest city and only 30 klms away from Ballarat, a regional city of 110,000 people. Don’t you love the modern world of hi-tech. I’ll probably watch it over the weekend.
strange that you use klms not Km, only 150 Km ? in some countries that is half the length or more than,
was that wireless internet (wisp) or wired ?
I don't need a storm to knock out my Internet, just a cavalier British Telecoms' engineer pulling plugs out of the local exchange cabinet with scant regard who might be affected. Happened umpteen times last week.
Many thx Ralph... hope your finger heals up quickly.
This video has reminded me ... have you thought about a hints/tips on security hardening IoT projects? ... I think there are a lot of naive folk out there who assume nothing malicious ever gets onto their network and can’t understand how swarms of zombified IoT devices could be used to launch DDoS attacks or amplify other vectors such as ransomware attacks.
Working on it, Chris!
Although I reckon that if someone did get something past my Wi-Fi the last thing they would be interested in is my ESP32 projects; more likely trying to encrypt my disks. But it's something I'm actively thinking about.
Hi Ralph, is nice to see you. Have a great day.
You too! Nice to hear from you Steven!
Thanks for sharing this here on TH-cam, I think this will be useful for me too when I'm going to play around with my Arduino Uno with the ESP8266 chip integrated into one board.
Yes, I have one of those! But I never understood why we would need _both_ chips on the one board - other than that we can use the ESP chip for Wi-Fi and the Arduino for other stuff? But the ESP chip is so much faster... less pins, though.... when I bought it, it sounded like a good idea, but now I'm not so sure.
Many thanks for this Ralph. After a little tinkering I have managed to get this running alongside a fauxmoesp server by changing the port number. I have an automated blind opener system using ESP8266 with amazon echo and I can now update the software without plugging in a laptop or having to take the units off the wall.
It's always the "little tinkering" that is required to get things running smoothly, glad it's all working now.
Fantastic video as always.
Thanks for sharing your expirences with all of us :-)
Glad you enjoyed it!
00:30 Sending a rocket to the moon. I posit that your ESP32 has as much, if not more, computing power as what was on the Apollo 11.
@ Mr.BianchiRider LOL, I often use the computer and magnetic core memory that was used to send man to the moon when expressing my feelings for modern day electronics. Usually, I say " We sent a man to the moon using magnetic core memory and room full of computer but we can't seem to blink a stinking light on a tower!"
@@TheNCGardener I usually whip out one of my two slide rules. One is round.
I suspect the ESP32 has umpteen times the computing power of the moon launch in 1969. Compare:
ESP32: 240MHz clock speed, dual core, 4/16/32 Mb of program storage, 300Kb runtime memory, 600 DMIPS.
A couple of years after the moon launch, in 1971, Intel released the 4004, the "first commercially available microprocessor... It contains the equivalent of 2,300 transistors and was a 4-bit processor. It is capable of around 60,000 instructions per second (0.06 MIPS), running at a maximum clock speed of 740 kHz."
Source: en.wikipedia.org/wiki/Timeline_of_computing_1950%E2%80%931979
One tip: if you use a watchdog with a timeout with with 5sec, disable the watchdog before the upload starts.
I've not had an issue in this respect; in fact, the original code keeps running right through the upload process (which happens in parallel) and only stops running when the system reboots and uses the new code.
@@RalphBacon oh, nice. The standard OTA lib does't do this, it's blocking. Thanks for pointing out this ota lib
Thanks for all the knowledge and support.
You bet! Any time!
There's three things we can count on, sun rise and set, the four seasons, and Ralph injuring a digit. 😀
True. My thumb is _nearly_ recovered now (it's taken weeks). Still a bit tender but the bacon slice I took off with the craft knife has finally regrown. It even has fingerprints! Another couple of weeks should do it, I reckon.
Good Tutorial about some Very handy features, yet again. Thank You!
Glad you enjoyed it!
Hi Ralph
Thanks for the videos. You mentioned in passing about some updates for the Web Radio project.
My Web Radio is used daily and working well.
Always interested if you have released any updates
John
The latest update, John, was a combination of hardware and software, so not for the faint-hearted!
I found the ESP32 Web Radio would sometimes hang on switch on when it tried to initialise the VS1053B MP3 decoder module. I discovered that the RESET of that module and the ESP32s were wired together. This was improved by taking the RESET of the VS1053 to a separate, spare GPIO pin (25) of the ESP32. When the ESP32 is ready for the VS1053 it would bring that module's RESET pin LOW for a hundred milliseconds then continue as it always did. Since then I've not had one "hang". But it does require a (relatively simple) bit of surgery on the PCB.
Additionally, I've found a simple BT transmitter module to send the audio to a BT speaker without having to press any "connect" buttons or anything; I had to create a PCB for that (it was 1.27mm pin spacing!) and it works well enough except I find the output signal very noisy even on battery. As it's early days for that project I have yet to experiment to make it quieter.
I've also made some improvements in its Internet failure handling. I get cut off from the Internet on some days at least half a dozen times (it's what happens when you live in a field). The ESP32 radio does not always recover from that as it thinks it is still "half connected" (it can read the header, or gets a redirection) but can't read the stream. I want to tackle that some more before releasing that code.
I'll be still tweaking this code when I'm 90, I'm sure.
@@RalphBacon Thanks Ralph for the very quick reply. I shall reread this so i understand. No internet problems but a very occasional hang when starting, a press of the reset fixes it.
Regards John
If your Internet is stable (doesn't disconnect) like mine was in Milton Keynes, it will keep playing 24x7. It's the start-up issue you mention that I've fixed with that hardware/software mod. But, as you say, pressing reset also fixes it!
@@RalphBacon Thanks Ralph
I've sorted the "noise" issue; if you recall I send the audio output via isolating transformers, not least because the COM of the VS1053b output should *not* be connected to ground (datasheet specifies). I forgot and connected my BT transmitter directly to the output jack. When I connected it to my built-in isolation transformers it was blissful silence (well, apart from the program content).
it looks that the file identifications.h is prepared only to deal with esp32 and the example isn't working with esp8266
Iam having the same issue
Check my GitHub page for this project; it mentions that there are two different AsyncTCP files; one for the ESP32, the other for the ESP8266. (Links are provided).
Obviously I had to use the ESP32 version for my demo, but just change that in your setup and it will be fine, I'm sure.
github.com/RalphBacon/228-Wireless-Over-the-Air-OTA-ESP32-updates
@@RalphBacon yes it seems to be the problem. for anyone trying on esp8266, you must edit instructions.h aldo change WIFI.h to esp8266wifi.h in the include line on demo.ino. and last idhave to comment all relative to debuging:
// WiFi debugging
//#include
//#include
//#include
Yes, it seems I have ignored the ESP8266 somewhat. My bad. I will try to be more inclusive in the future.
Great video Ralph. I have used ota many times, and prefer using a static ip for the target board. With your example you only get to see the dhcp ip if you view it via serial, but if you aren't usb connected and that dhcp ip changes you won't know where to /update to (although I know there is a nameable dns route to overcome that)
Yes, you can specify a static IP address in the ESP32 Wi-Fi setup, of course, but I have set my DHCP part of my router to always give out the same IP address (if it can) so I never really did this.
I think your way of a static IP address is safer (as long as it's carefully controlled to avoid clashes). Ideally, static IPs should not be part of the DHCP pool of available addresses. My DHCP can have a restricted range so I can do this.
I might change my Smart Heater Controller to use a static IP address and show others how to do this! Thanks for the idea!
@@RalphBacon Thanks for the Reply Ralph, and thanks for the Videos, you have really taught me alot that I never knew, and some great tips and tricks.
Regarding DHCP Pool, I usually leave it set for 1-199 for DHCP, so I know that I can statically assign from 200 onwards and know what is what, but even with that I sometimes come a-cropper when I forget that something on the same IP is still powered up :))))
I tried your code and got an error with the this line, #include . It said it could not find it. I did a quick search and found a hit from March of this year that this error was resolved by using this statement instead, #include "esp_private/wifi.h". And it works. Not sure where the dependency would be. Maybe a newer or older version of ESP32 Board Library on my end or yours. I am running ver 2.0.0 under board manager for ESP32.
Just wanted to let you know.
That is left-over from my personal debugging. Oops. Sorry Mike.
You should be able to just comment that line out. If you look into the "Identification.h" file you will spot some commented out lines that allow debugging of the Wi-Fi (all part of the Espressif ESP32 framework). Useful for when things go wrong but not required.
Can I call this push OTA method ?
What about pull OTA. So the device(s) has to check wether any new firmware file has been added to the webserver or cloud storage.
We have deploy some ARR in rural area, and build an early warning system (river flood) for the local community. And now it would cost alot when we need to edit just 1 line of the firmware code.
Thank you for your kindly reply.
I briefly looked into this at the time but didn't pursue it to completion. It's something I intend to do when I have time.
One way is for a Node-red/MQTT set up to publish a new update and when the ESP device (that has subscribed to that message) is ready it will go and get it.
I've also looked at the Espressif documentation and there is a command to get the update; but how it does it I haven't yet determined. As I say, I haven't really delved deeply enough into it.
A final method is for the ESP device to regularly poll (via a GET request) a web page (or file). If it gets a 200 (not a 404 or 500) then it knows that it can go and get the update from a known location (cloud storage) and do the biz. This allows the controller (you) to upload the new code to the cloud storage and then create the web page that the ESP device will be looking for (once an hour, or whatever) as a flag.
I'm guessing and hoping someone has trod this path before me as I do want to find a way of doing this. Scour Google or stay tuned!
@@RalphBacon wow.... I really thank you for the quick and detailed reply. will stay tuned, and keep looking from forums that may have implemented the OTA update function.
Mr Pathub:
If you are useng ESP32 .The easy way is to use ESP.getSketchMD5();
That will give you the current MD5 of the sketch. Then you can just compare that MD5 with a MD5 you have on an online server.
If the two MD5 differ, then do something, if they match then do nothing.
@Ralph I love your videos but on this one I noticed that awesome breadboard 😮!! Do you remember where you purchased it?? Looks perfect for those funky sized dev boards
That awesome breadboard has jumped in price by about 50% since I started using them (about 1 year ago) but they are good quality AND a bit wider which makes it better for ESP32 dev boards and the like.
Link: bit.ly/breadboardUK but others are available in the range.
What happens if the upload is disrupted? For example only 50% uploaded and the connection terminates? Does the lib discard the update or does it flash the incomplete firmware?
The upload just gets aborted. The original code continues to run as if the upload never happened, no reboot, nothing.
Hi Ralph, Late reply I know but thanks for this video. I’ve just implemented it and it works well. I'll now add it to my DCC throttle project to see if it works with the WiFi already in use.
I thought I'd comment here as my comments on your most recent video seem to be disappearing again. Ho hum……
Regards,
Colin Murdoch.
You must have _really_ upset YT at some stage, Colin, that's all I can say! Or, you are still trying to put links into your comments which will definitely get them zapped.
Anyway, I'm glad you got this working, and I'm still using mine for the Smart Heater Controller although I did have to revert to USB last week as I upload a bit of rogue software that caused a continual reboot.
I should take my own advice and always test upload to a test device first!
Hi Ralph,
Well, it worked on the test blink program, but I’ve now added the code into my railway throttle project and have hit problems. The OTA code runs, but the program fails to run the rest of my setup code - or at least nothing shows on the serial monitor. Perhaps it conflicts with the standard WiFi library? I think I will be back to pressing the Boot button.
Regarding upsetting YT - I uploaded a small video about my Arduino controlled railway traverser and stupidly said it was suitable for children. That seems to be what started the problems. Now if I respond to one of your videos and mention the name of the site where you store your code, it seems to block it.
Weird or what?
Regards,
Colin.
Hmm, most TH-camr's avoid ticking that "Made for kids" because that really does restrict you (unless you have a channel for kids, of course).
Additionally, creating _any_ link (except to another TH-cam video) seems to invoke the wrath of the YT-bots and your comment will be summarily executed with the delete key. Sledgehammer to crack a spam nut.
Back to your issue of OTA updates. If the ESP32 has accepted the update (you get a one line serial output line that one has started on the receiving device) when it has completed the device will reboot and "just run" as normal (but on the new partition). Just like as if you had pressed the RESET button. So what happens in your case?
@@RalphBacon well, I’m not really sure. The update process seems to work, in that I can connect to the web server page, find the stored binary and the update runs. The ESP32 then restarts as you say, runs the OTAinit() function (I have a Serial.print to confirm this) then outputs some junk characters and does not seem to proceed any further. Now with hindsight the last upload I did was with a binary from Plarformio, so I am wondering if there was something wrong with that. I think perhaps I should try it again with a binary from the Arduino as it is easier to see the correct file to upload. Have you tried it from PlatformIO?
Regards,
Colin.
Is HTML like Greek if you're Greek?
In this case is like Chinese ... ! (I am Greek).
@@MikeKranidis Hahahah me too.
more like most middle east languages where all you hear is:
skr sjah Coca-Cola fjio kfhj etkge Xerox fjd faslij Vegas iroweope
and then you're confusedly wonder %)
Perhaps Klingon would have been better?
@@RalphBacon Darn good comment Ralph. It really made me laugh :)
You could implement a watchdog to the OTA function in case of something goes wrong.
If you upload code that goes awry (constant rebooting) there is little you can do, quite frankly. Well, revert to USB uploads again.
The actual OTA upload either works or doesn't. Partially updated bin files just get abandoned and overwritten on the next attempt.
Nice tutorial. How to do this when you use deepsleep on the esp?
Yes, I thought that might be a problem (my project never sleeps). You would almost certainly have to set a flag so that on the next wake-up it would detect the flag and stay awake ready for the manual OTA.
Using MQTT might be easier as you could publish a message that would be received upon wake-up and either update itself or just stay awake.
Or get the ESP32 to get a webpage so see if an update was pending. Described in this thread with a link to the full code:
community.platformio.org/t/ota-and-deepsleep-mode/7235/15
ElegantOTA does not work if your program is using a hardware timer on the ESP32. It’s a real bummer!
When the OTA start is detected can you not disable the use of the hardware timer?
I have to do this in my project for an interrupt that is triggered by movement. If I do not disable the interrupt (and I move about) the whole OTA crashes with a PANIC. As I now disable the interrupt on start of OTA (programmatically) it works just fine.
Detect the OTA start by this line at the top of the loop:
if (Update.progress( ) > 0)
Great another nice informative video where you do the work so I don't have too:-)
Glad you enjoyed it!
what's the differance between "sketch upload" and "upload"?
does the behive counter count the bees that enter/exit the beehive?
The beehive I mention in my videos is a virtual beehive; it embodies the idea that it's not somewhere you can just dive into to (re-)connect some equipment (or code).
Is your first question a riddle? Like, what's the difference between a raven and a writing desk?
Hi Ralph.
Elegant OTA might be good for beginners, but it has some secret code and are not so configurable.
And when you upload something with any OTA, you can actually upload anyting, not only bin files.
So i made my own OTA that checks the MD5 checksum of the file that is being uploaded to the ESP32. (Using cryptoJS).
It seams to work perfectly, but the code is in alpha state for now. But i would love to let you give it a try if you are interested.
(The MD5 checking is optional). The code even have some sort of version checking for the spiffs and sketch.
It works in AP and STA mode to.
PS:If you can do a video about the partions that you mentioned, that would be great!
Yes, the next thing I want to crack is uploading the DATA partition without zapping everything else (like the sketch).
@@RalphBacon You mean that in either case you select Firmware(Sketch) or Filesystem(Spiffs) In the ElegantOTA it allways uploads to the Firmware(Sketch)?
BTW: I would like to have some kind of cantact with you outside of youtube. Is there any other way i can have a contact with you? (Email, whatsapp?)
I sent you a friend's request on Facebook.
I don't really use Facebook, far too much spam. Send me an email (look up the About page on my channel).
Yes, I mean that when I upload the SPIFFs partition on my ESP32 it also zaps the program partition which can't be right! I just know I'm doing it wrong but I haven't had the time to investigate further.
@@RalphBaconI not see your email,and TH-cam refuse me from sending mine in a comment.
What to do?
Good Tutorial about some Very handy features, yet again. Thank You![
Glad it was helpful!
Yay 🖖👍🏻Thanks, cool 😁
It looks like there is an option for "upload_flags = --auth=password" type function, but I'm just starting to explore this. But leaving your IOT network wide open invites many more issues. Outdated firmware, default passwords, there's just so much to go wrong.
I really should pin this reply as everyone seems worried about security but not about the actual Wi-Fi this is running on!
Anyway, just amend the following line:
AsyncElegantOTA.begin(&server, “username”, “password”);
hi, a favor, maybe you can help me, I have been reading about the ota data partition, it has a size of 0x2000 bytes, I wanted to understand how these bytes are filled?.
I thank you in advance.
In your sketch you must create a "data" folder. The contents (files) are uploaded separately from your code. I only have experience in how to do this with Visual Studio PlatformIO. Are you using the Arduino IDE?
Thank you for another great video, Ralph! I'm curious...could you briefly explain how this manages to work on a device that's already in the loop()? The only declaration I see is in setup(), nothing about OTA in the loop().
Yes I can! The OTA code (if the standard Arduino version is used) has to check at the top of the loop whether an OTA is trying to start. This (Elegant OTA version by Ayush Sharma) is more sophisticated and now does not need that check (it is monitoring in the background).
If you want to know in the loop( ) whether an OTA is starting, you can use the following "if" statement:
if (Update.progress() > 0) {
I do this in my workshop project to turn off an interrupt that can cause a PANIC if an OTA is in progress and the interrupt is triggered. Once the OTA has completed, upon reboot it all goes back to normal. Simples. 😉
So... If my thingy already has a server.on function call this code now prevents my web server stuff from running when type in my IP address, right? How can we have both? OTA on a web server thingy???
I have both OTA and a Web Server on all my ESP32s. The libraries I am using you'll find in the code examples in my GitHub for any of my recent-ish projects, but in summary:
1. ayushsharma82/AsyncElegantOTA@^2.2.6
2. me-no-dev/AsyncTCP@^1.1.1
3. me-no-dev/ESP Async WebServer@^1.2.3
If you have "issues" using the library in #3 above (because someone hasn't updated the version number) then use github.com/me-no-dev/ESPAsyncWebServer.git which is the same but has been "fixed".
Above, the first library is the OTA library and the second and third provide the web server functionality. Look at my projects (eg World Clock, Barrel Pond) for full examples.
If you use interrupts, ensure you turn them off as soon as the OTA starts otherwise it will fail (best not to ask me how I know this... 3 hours of my life I'll never get back).
Thanks for the video. For me, it works using your demo program. HOwever, when I applied the OTA " modification/ add on" to one of my existing sketch which uses ESP-NOW, it looked like the OTA portion connected to wifi and then my own sketch started up. HOwever, the wifi portion of my own sketch doesn't work anymore. perhaps ESP-NOW is one of the limitations. ??
I use the following libraries, and in this order:
#include
#include
#include
Allow OTA software updates
#include
Then I create the AsyncWeb server first (allows web pages to be served):
AsyncWebServer server(80);
Here go all the "server.on" definitions (if you have any).
Then start the OTA server:
AsyncElegantOTA.begin(&server);
Finally get the server running:
server.begin();
Finally I log in as a client (station) to the Wi-Fi:
WiFi.begin(SSID, PASSWORD);
ESP-NOW does not require you to login to the Wi-Fi. Set the mode: WiFi.mode(WIFI_MODE_STA);
Then find (and register) the peer device using the MAC address and send the data packet:
esp_now_send(broadcastAddress, (uint8_t *)&sensorData, sizeof(sensorData));
Phew! How does this compare with your setup?
I have these in my main tab
#include
#include
#include
#include
#include
#include "Identification.h"
then this right after void setup() {
otaInit();
I left Identification.h unchanged.
not sure why it isn't working.
When you say "isn't working", what _exactly_ is not working? Does your sketch connect to your Wi-Fi successfully? If not, post that bit of the debugging lines your sketch will doubtless send out.
@@RalphBacon thanks Ralph for following up with me.
the part of the sketch that is not working relates to ESP-NOW's onDataRecv() callback. The OTA version of the sketch doesn't invoke the OnDataRecv callback when ESP-NOW messages are sent by other ESP units. Hence, I wonder if where there's already an active wifi.begin() running, ESP-NOW traffic is somehow ignored ?
here are code segments..
WiFi.mode(WIFI_STA);
...
esp_now_register_send_cb(OnDataSent);
esp_now_register_recv_cb(OnDataRecv);
memcpy(peerInfo.peer_addr, broadcastAddress, 6);
peerInfo.channel = 0;
peerInfo.encrypt = false;
// Add peer
if (esp_now_add_peer(&peerInfo) != ESP_OK){
Serial.println("Failed to add peer");
return;
}
...
void OnDataRecv(const uint8_t * mac, const uint8_t *incomingData, int len) {
memcpy(&myData, incomingData, sizeof(myData));
Serial.print("
Repeater OnDataRecv --- > ");
Serial.print(" Received Msg : ");
Serial.print(myData.wifi_msg);
Serial.print(" Msg Count: ");
Serial.print(myData.wifi_msg_id);
Serial.print (" MAC: ");
tmp_group = String(myData.wifi_msg);
command = myData.wifi_command;
for (int i = 0; i < 6; i++)
{
Serial.printf("%02X", mac[i]);
tmp_mac_address[i]=mac[i];
if (i < 5)Serial.print(":");
}
again, thanks for following up. I was only asking a general question . I really appreciate that you're spending time to help.
I would check that the _sender_ is actually using the correct MAC address of the _receiving_ ESP32. If so, check the order of adding the WiFi, server, OTA server and so on with the relevant "begin" statements. RandomNerds website has examples of this working.
can i put (include) any other library like dht, ArduinoJson, PubSubMQTT ... etc ? because in Blynk Library I had a problem to get web server work
The good thing about the ElegantOTA is that it works well with the ESPAsyncWebServer (used here). I've had no problems in several weeks' of use in my Smart Workshop Heater Controller and I include all sorts of libraries.
Hi Ralph. Great explanation, thanks. But I have an ESP32 project which already has a webserver, and if I add the required libraries, and your Identification.h I get a whole load of compilation errors, such as:
error: conflicting declaration 'WebServer server'
WebServer server(80);
Any idea how to get this working with an existing webserver?
Oh dear; yes you will get errors as my "Identification.h" contains _everything_ including a webserver. If you edit that file you might be able to remove the AsyncServer entry and substitute your own preferred on. That would fix it, but whether the OTA works well with your particular server I cannot say.
Thanks for this great video, Ralph. 🙂 I could not compile your ESP32 code for my ESP8266, but I assume this is merely due to my lack of knowledge regarding VS Code and PlatformIO (I never used these tools until 1..2 weeks ago, and I am completely new to microcontrollers in general).
Anyway - I could combine what you shared here with your recommendations regarding namespaces (in another video) and by taking code snippets from both videos I now have kind of a base for creating clean code and using OTA.
It is running on 1 of my 3 devices ... and I need to remember changing project title and project description when loading it to the other 2. 😉
At least you got it running in a very short time (congrats!).
If you get the chance, try and get an ESP32 module (such as the DEVKIT-C v4 that I use) and you might find "it just works" with that hardware.
Great video and much appreciated, Ralph. Can you make any suggestions for using this method in a ESP32 that spends most of its time in deep sleep?
Yes, absolutely.
We'll assume you cannot force your device to wake up (eg by an external stimulus on a pin) and it only wakes up every X minutes/hours to do its thing.
Unlike the method I show here, which is effectively a "push" OTA update, you will need to initiate an update from within the client when the device wakes up (not every time, perhaps once a day).
The new image is stored in a suitable place such as a server (could be as simple as another, always-on ESP32 with sufficient filestore, whether that is LittleFS, an SD card or something else). Your device then attaches to the server, checks for an update and, if one is there, downloads it and reboots itself on the new image.
I hope to cover this in a future video as it is not as straightforward as this method. Stay tuned (and subscribed!).
@@RalphBacon Yes, this is exactly it, just as you described. I've seen comments on this approach you describe but I don't know how to get there from here (i.e. I'm too inexperienced). My application is an outdoor ESP32 with an integrated BME280. As you know, this wonderful sensor will self-heat if you read it too often. Also, since they are on the same board, it is imperative that the processor not run any more than necessary to prevent heating the board, and the sensor which are right next to each other - ugh! (Adafruit Feather ESP32-S2 with BME280). I sleep it for 52 seconds, it wakes for about 8 seconds, takes a reading and sends it over ESPNOW to a receiver at my desk. Anyway, OTA updates would be SO much more convenient than having to go get the sensor board from outside, bring it to the shop, etc. But as you say, it's not so straightforward.
Thanks again for all you do for the beginner Arduino community. My skills have improved a lot thanks to you, Rui & Sara Santos, and many others in forums. Slowly, I am getting to the point where I *almost* know what I'm doing!!!
Great tutorial but I was having trouble getting OTA to work on my project when it was powered through the VIN pin. Eventually got it to work when powered via USB port and battery bank. Also important to link the IP address to the MAC address on your router.
Sorry to hear that. It should, however, make no difference whether powered via the USB port or directly via the 5V pin - assuming your 5V power supply was clean and stable and capable of delivering at least 500mA (Wi-Fi takes a lot of power).
I use the Vin pin most times; the only downside is that you sometimes don't get the LED on the module lit to show power is present, which could be desirable if you are trying to run from a battery and/or using Deep Sleep (as I am with my most recent project!)
Thanks for the video Ralph! You gave me an idea that I could use for an old Wemos D1 mini project I needed to update. For some reason, I can not get the Arduino IDE to upload to it any more via the usb cable. (Probably not using the right swear words.) However, the previously installed code had an "Update Firmware" option built in. I recompiled the code in the IDE and exported to a binary. I then used the web page update and sent the binary file to my mini and it worked like a charm. Now I have more time to figure out how to get the cabled connection upload to work...
I'm very glad it made you think about such things, David - my work here is done!
yes it could be the usb cable, but it could also be the usb connector on your Wemos. Pushing in the USB cable can dislodge that micro-USB connector quite easily. If you have one of those 3v3 USB to TTL connectors....try attaching those directly to Tx and Rx of your Wemos and see if that works (make sure it is a 3V3 one)
fatal error: esp_wifi_internal.h: No such file or directory (I'm using esp32s3 devkit)
This is Line 10 of the file "Identification.h"?
I suspect you are using the Arduino IDE (not Visual Code Studio with PlatformIO) and therefore the Arduino implementation doesn't include this feature. Just comment out that line and also lines 56-57 (unless you don't get an error, somehow, if you leave them in).
What this all does is tell the underlying Espressif Wi-Fi code to issue lots of diagnostic messages that can help when you can't connect to your Wi-Fi, but other than that not needed.
Elegant OTA-tutorial :)
Glad you think so, Egon!
Thank You Ralph
You're most welcome, Mark!
Hi Ralph, thanks for the info. I have an ESP32 application that is already utilizing WiFi and connects to the internet and an MQTT server, along with using the WiFi Manager library to configure the application without hard coding the SSID and Password. I am wondering if this OTA library would work with all that other stuff going on?
It should do, but whether the web server code you use works well with this particular Elegant OTA I cannot say. You will have to edit the "Identification.h" file and substitute a call to your own webserver. If you already have that up and running I'm sure you could work out what needs to be done.
Hi Ralph, it's me again !!! While OTA works for some of my sketches after I changed the wifi.mode to ap_sta. However, I have a sketch that uses , with OTA installed, the sketch would fail on a "httpGETRequest". (the OTA portion works flawlessly though... at least something is working !)
1:24:25.110 -> I (29876) wifi:ap_probe_send over, resett wifi status to disassoc
11:24:25.206 -> I (29876) wifi:state: run -> init (c800)
11:24:25.238 -> I (29876) wifi:pm stop, total sleep time: 0 us / 22804210 us
11:24:25.301 ->
11:24:25.301 -> I (29922) wifi:new:, old:, ap:, sta:, prof:1
11:24:25.529 -> Error code: -1
Just an fyi .... this just happened .... I'll go and see if I could find a solution. thanks for listening.
I think you have to be careful which libraries you use with which. Espressif (and me-no-dev) have various examples of OTA working alongside other Wi-Fi requirements, so do check them out!
Hi Ralph great video, I wanted to find out if I can do this for Arduino with an Ethernet shield for the server
You can, but not this way. Arduino have their own "over-the-air" method of updating micros so you will have to read up on that, which I have not used.
I'm surprised that a .BIN file is sent to VLC as the default program. :)
Yes, Win-doze strikes again!
Hi Ralph
Could this be done over the internet, say if the bin file was on a google drive and a web page on the same drive and people go to the address of a google link
then people can update their ESP32 if the ESP32 in on the internet
I don't see why not. If you can address the /update page of the ESP32 then the PC simply asks for the file, regardless of where it is. Should work, but I've not tried that(yet).
@@RalphBacon I like what you said YET, do you think you would be doing something like that so people can update their Arduino device from a long way
No, not really. Well, not as a non-professional. You might want to do it if you run a collection of sensors for a water company, for example. But not for just the couple I have in range of my own wi-fi!
Hello! If the esp32 operates in Ap mode and creates a html page and displays data using websocket, can this ota update still work in some form? How can I somehow bring the two together?
If you operate in AP mode (instead of STA mode) then you are creating a brand new network, not linked to your current Wi-Fi.
So if you connect to that network (in this case, the ESP32) you can still use the OTA feature. From your phone, for example, assuming that you have the .bin file on your phone. But then your phone won't be connected to your home Wi-Fi so you can browse elsewhere for that file.
On a PC you could have two network cards, connected to two different networks.
@@RalphBacon I tried this earlier, but something didn't work, so I left it, but I'll try again. All the demo programs I found are in STA mode, so since I haven't used the wifi functions so far, it wasn't easy to start.
I'd suggest get it all working in STA mode, then switch to AP mode. Any particular reason for using AP?
@@RalphBacon Actually, there isn't. What I'm using is a 3-axis camera moving system I made at home, I can see its parameters (position, speed, endpoints) by connecting to the WiFi of the ESP32 and displaying the data on a laptop or a tablet. This works even if there is no accessible network nearby. I only had time for a short test so far, if I delve deeper into it I think it will come together and then I will try STA mode. By the way, when I reach the final version, it won't be so important to upload the program to the ESP, but now if I'm working on the code, this method would be easier than always connecting a cable to the ESP. By the way, your videos are extremely informative and detailed!
Hi Ralph. Once again a very useful video. I'm writing an ESP32 project at present so I’ll give this a try.
My project includes a WiFi connection to an esp8266 using the client.connect(address, port) construct. Is there likely to be any confusion between two possible Wi-fi connections or should they be able to exists side by side so to speak?
Regards,
Colin.
I'm using OTA, a web server and ESP-NOW all at the same time and they work just fine. On that basis, you _should_ be OK but it does depend, I suspect, on what libraries you are using. The only way to know is to try it! You can do that on the desktop with a spare device.
@@RalphBacon My code at present (untested yet) is using the standard WiFi library which is quoted in your header file, so hopefully no conflict. As you say, try it out and see.
Have you managed littlefs OTA?
No, I have not progressed that (yet) but it is becoming more pressing as time goes by. Still very much on my list, Duncan, you have a need too?
One issue for me is that I'd like to upload the data partition without zapping the program (sketch) which happens when doing it via USB, for some reason. I need to investigate more. Well, start to investigate. 😮
nice practical info Raplh
Glad you think so!
So there's wifi on moon?
Oh yes. The Americans left a solar-powered Wi-Fi unit up there (in fact, more than one) and the coverage is good, no interference. They did it so they can communicate with the moon-based modules up there, when they are in orbit. Which could be in 2024, next.
Hi Ralph, thanks for the great video. I have a question. In my project, in the very beginning I tried to use OTA drive and it worked. So I said yea it works, perfect! and moved on. Time passed and now I have like 2000 lines of code. I have tried OTA again and it didn't work. After some checking I found out in the Serial monitor it said :" handleUpdate(): FreeSketchSpace to low (1310720) needed: 1544640, flush(): fail on fd 48, errno: 11 " no more processes" bla bla. So when I checked my bin file it was like 1.47mb. I realize it is due to the size of my sketch file. So how can I use OTA under these circumstances, because I really do need OTA updates in this project. Would this alternative ElegantOTa work for me? Or what would you suggest?
It's all down to your partition size(s), Okan. Assuming you have a 4MB flash ESP32, how do you split up that memory? Some of it is required by the OTA system but I would expect a 1.5MB program to fit (it needs twice that, so 3MB on the ESP32).
I will be doing a video on how partitions are created and managed on the ESP32 "soon", can you wait that long. If not, email me and I'll send you a suggested partition .csv file.
@@RalphBacon Thank you Ralph, you are so kind. I thought the same, I tried everything I could but no. It just gives an error there in my code for some reason. I have trried different partitions as well, for instance in my platformio.ini file, I added this partition "board_build.partitions = min_spiffs.csv" to keep spiffs as little as possible, leaving more space for OTA and code. No, didn't work. I tried huge_app.csv the similar way, again no. Here is the error message: [ 23646][E][tinyHTTP.cpp:99] get(): HTTP error: 304,HTTP/1.1 304 Not Modified. On OTA drive I see the number of requests number tick one up everytime I try an OTA update but nope..doesn't download anything..
@@RalphBacon ok it worked. I feel like a true idiot. It didn't work because the version I have on my device and the version I uploaded on the OTA drive are the same. So I suppose it checks the versions in advance, if the versions are the same, why bother download! Damn! 😂😂😂😂Thanks anyway Ralph!
Hi Rodney, I got it loaded with all the libraries, but when i type in the address it can't be found. The script i am using already connects to the wifi, could there be a conflict? ie, i was posting info to my webserver and added this to the script.
I just flashed your demo code, work perfectly. I wonder if its an issue since i am connected to my network with my own code, not the code in the "identification.h" file.
Check the IP address of your Esp32 device on start up. If that's not the problem I'll look at it tomorrow if you send me the output from your serial output.
@@RalphBacon Well, i get the IP address and other info as i should with the demo.ini file, but in my code, i just get my serial.print stuff. It acts like it isn't loading the .h file. but i get no error saying it isn't loading. weird..
@@RalphBacon Ok, got it working.. all my fault, works perfectly as you documented. I would explain the issue, but its to embarrassing. Can I send you some drinking money? do you have a paypal account? This really rocks..
Don't be embarrassed, Jim, we all do silly things. Glad you got it working. If you feel that pleased you can always buy me a coffee!
buymeacoffee.com/ralphbacon
Thank you!
You're welcome!
without oTA, like arduinos & such....when I'm done, I package a usb cable with , or port it to the side of the box. Usually epoxied in place.
I'm excited to try this OTA stuff, but I think I'll continue my practice of having an accessible port, as a fail safe.
Even with OTA, it's all too easy to forget something and have a "bricked" device that is probably constantly rebooting itself. Don't ask me how I know this! A separate USB port as you suggest is a very sensibly suggestion.
@@RalphBaconI finally got it all working, few kinks to work out in the new versions of what you showcased. I made it harder than it should have been (PIO was being prickly). Many thanks for laying out a great template.
Oh and on about my 3rd upload, I got turned around in my projects folder and sent some random firmware.bin file to one. Fortunately, they are still on the bench.😁
You happen to have any videos covering switching between espNow & OTA ?
I need espNow for normal operation, I've set up an upload mode where espNow stops for OTA. al fine. I'm trying to deal with a case where I accidentally enter OTA mode, but dont want/cant sent an upload, also failed uploads(not seen 1 yet). Gettin the espNow running again, without restarting the board.
last question, any good forums you'de recommend for this sort of thing?
Can ESP OTA send my serial monitor statements? Basically like "Console.print" with the Arduino Yun
You can output debug information to a html page and connect to the webserver on the microcontroller.
there are other ways to do this
@@fillempie1501 I do that but that's more of a hassle than Console.print("foo"); is
@@alexrossouw7702 Yes, thats true. I use it for controllers without a serial interface. Sometimes I use the syslog option or even XMPP. But you're right. print("bla") is easier.
@Fil lempie Yes, we can do this using the WebSerial library but I'm still in the early stages of testing - It's something I hope to cover in a future video as I missed monitoring my ESP32s!
I saw his video of comor upload sketch to sp32 wirelessly everything has worked well until I decided to upload one that consists of sending by Bluetooth messages to a matrix of LEDs, that if I did not want to upload it wirelessly tells me it does not compile for board sp32 but it happens only with that type of program that have Bluetooth then that via I can use to load the sp32 these programs wirelessly.
A bit strange that using BT in your code should stop OTA updates.
@@RalphBacon It will be that programs that carry Bluetooth can not be charged wirelessly with OTA because the libraries of both communications are not compatible I think
If that is true then you could try a different library for either BT or Wi-Fi - preferably two that have been proven to run side-by-side without issues.
@@RalphBacon el problema es que yo siempre tengo wifi pero si tengo Bluetooth,tendré que conformarme seguirlo cargando con cable
Well, as I said, I would try finding two libraries (Wi-Fi & BT) that work together well and don't stop BT from working. Many others must have done this successfully. Buena suerte!
two (2) Dollar for a PCB? Yes! But 48$ additionally for handling and sending!
There are many options for shipping, Bill. I even tried the UK Post Office method for $8. It took a couple of weeks but turned up. There must be cheaper options for the States and elsewhere too, I would have thought.
@@RalphBacon this is what I think too. But it wasn't offered by the producer (in China I guess).
Hi
Do the codes still work?
The OTA updates are very easy to use and, yes, still work just fine!
uff 🤙🏻
Oy vey, it wasn't _that_ difficult was it?
@@RalphBacon My way is OTA from my own WEB-Server - it's also not easy but it runs and i know what happens...
Cool. I suspect lots of advanced users roll their own, so to speak. Well, in some ways, this is the beginning of my own method (that I've now updated to include the software version number).
Is there a way to set this up to require a password before updating?
There is indeed. Change the line in the "Identification.h" file from:
_AsyncElegantOTA.begin(&server)_
to:
_AsyncElegantOTA.begin(&server, “username”, “password”);_
There's a lot of comments on security but, frankly, if a hacker can get to our Wi-Fi I think we'll have bigger problems than wondering whether he hacked our ESP32 devices! Like where to send the Bitcoin ransom payment to unencrypt our disks!
@@RalphBacon Thanks Ralph. I was not thinking about security, more about someone accidentally overwriting the wrong device software in a facility where multiple users and devices exist.
When I try to 'Export compiled binary' I get this error In file included from C:\Users\my profile folder \Documents\Arduino\demo\demo.ino:12:
C:\Users\my profile folder\Documents\Arduino\demo\Identification.h:10:10: fatal error: esp_wifi_internal.h: No such file or directory
#include
^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
exit status 1
Compilation error: esp_wifi_internal.h: No such file or directory
This is trying your demo file.
Does it compile and upload normally, that is, via the USB cable. Always get that bit working first.
Have you actually got a path to the "missing" library? It should be in the libraries folder (which should be in the sketch folder)?
Very informative video Sir
So nice of you! Nice to see you here again, Muhammad.
First of all, thanks for your effort to share this tutorial.
I would like to use an esp32 or esp8266 so that, for example, if I wanted to change the SSID and Password remotely on the same network, would this be possible? If not, which way to do it?
I note here that I don't understand English correctly.
Greetings and my thanks
Do you have multiple Wi-Fi networks? If so, you can programmatically change the SSID and network from within the sketch and reconnect to a different network, no problem. Ensure the SSID and Password variables are not constants as you cannot change a constant variable.
Is that what you mean?
@@RalphBacon First, thanks for your reply.
No friend! I gave the example of the SSID and Password but I explain it in another way:
I have a LED Matrix Clock with ESP8266 mounted on the wall, away from the computer. Once a year it changes daylight saving time and I don't want that, so when it does (change daylight saving time), I want to use the OTA function to remotely fix daylight saving time so I don't need to take the ESP from the wall and bring it to the computer. Did you understand? Thanks
@@RalphBacon Hi Ralph, I managed to solve it with ArduinoOTA; Thank you
I'm glad you got it sorted Daniel!
@@RalphBacon Thanks Ralph but I have another question please: As I said, I have this 'LED Matrix Clock with ESP8266' so let's say I want to make several of this and take/install in different places (houses, apartments) so on different networks; So, in principle, in each house or apartment that I bring the 'Matrix Clock LED', I must change the SSID and Password ok? But, wouldn't there be a better way not to have to do this every time? You understood me? I am grateful to you and waiting for an answer.
Thanks, Well done
Thank you too!
Thank You
Your welcome!
quite comprehensive video - thank you
however, few things:
1- you exposed your IoT without password - anybody can now change your code on the remote device - this library gives also possibility to at least provide user/password however no https
2- if your IoT goes to sleep you need to add code to stop sleeping when OTA is in progress (i.e. by MQTT or so)
1 - not anybody. Only people who has access to your wifi-network, so almost nobody.
@@cokmethorst9961 lol ;-)
By changing one line you can have user/password challenge but if someone has hacked your Wi-Fi will that stop them?
Anyway, just amend the following line:
AsyncElegantOTA.begin(&server, “username”, “password”);
Will this work with WiFiManager?
I don't know the definitive answer to that as I use it with standard ESP32 WiFi and the AsyncWebServer library.
Try the GitHub readme that has developed this to see whether he mentions WiFiManager: github.com/me-no-dev/ESPAsyncWebServer
Good Tutorial
Glad you think so!
Also, this implementation is very insecure even for local network. Very few WiFI APs can properly isolate such insecure devices and even in case of private VLAN if one of them is compromised then others are easy targets.
If your IoT device is already connected to Internet, I'd better hardcode some 99.(9)% uptime SLA (2FA secured, ofc) URL like share from Dropbox or the like which is just a text file where you put an actual link (and ver. num. + hash) to new firmware for OTA routine. Do not hardcode direct link to binary though.
My device, Leeroy, is not connected to the Internet _per se_ , only internally. That is, I have no port forwarding to allow it. Should someone get in past my router it's curtains for me. I guess we could use a different port other than the standard 80; we could use a username/password (one-line change mentioned in my other replies); we could use 2FA via a mobile phone - but this is a Smart Workshop Heater not my bank account so where do we stop?
@@RalphBacon All these are subjective choices for your one particular use case. You may be comfortable with them, that's beside the point. But keep in mind that you are not only documenting your own creations but also teaching others (bad practice in this case) who would most likely copy-paste first, then maybe but not necessarily properly evaluate and complete design.
Harsh words, Leeroy. Ouch. I'm not sure the millions of IOT devices installed in consumers' homes are protected any further than their router. Sonoff devices, for example. Any number of LED strips. Even amazon Echo. I once saw an episode of Big Bang Theory where some guy in China started dimming the lights of the team's apartment!
Anyway, I'd love to get some constructive feedback in what are Best Practices in securing IoT devices, if we don't do it at the router?
@@RalphBacon by router I assume you mean firewall portion of it, in that case it only filters traffic between its network interfaces. So, e.g. in your network you separated IoT from PCs, good firewall can do that. What it can not on the other hand is to control traffic between ethernet cards on one segment of net (connected to one switch and corresponding to the same net interface of firewall in other words). So even though you separated IoT devices, they still may bypass firewall while communicating to each other and you have a lot more work to do to be able to control this relations. It is a bit more complicated than I described with WiFi, but then again it is less secure in the first place.
And you don't have to forward ports to device to enable it to act maliciously. Often, its enough if this device is able to reach anything other than RFC1918.
Yes, current state of IoT security is awful and it won't change unless we force hardware encryption accelerators adoption and supporting software as a standard peripheral for our uC of choice. Right now it is expensive. And all 'smart' devices simply skip this part for keeping costs down.
And keeping the costs down is probably fundamental to the manufacturers of any of these devices.
But I still maintain that if someone gains access to your network (Wi-Fi or hard wired) then everything else will probably fall like dominoes anyway.
The thing is, unless someone really, really wants to hack _your_ system I think we are safe by virtue of obscurity.
I suppose we could change the port on the webserver here but any hacker will probably try the obvious ones (81, 8080, 8181). The best I'm hoping for is that no-one is interested in the slightest in my set up.
But you have worried me a bit though. 🤦♂️
Ralph, I have been trying to get this to work for a while this morning. I keep getting these errors when trying to compile.
exit status 1
Error compiling for board ESP32 Wrover Module
I deleted all the libraries that I already had loaded and installed the ones you listed thinking that would fix it but no joy.
I also tried other esp32 and esp8266 boards to no avail. I then commented out all the include library statements and found that the include AsyncTCP.h was the one causing the errors. So I deleted the libraries I had installed and downloaded the library again, making sure to get the one for the ESP32.
Do you have any clue?
Note that there are two libraries for the AsyncTCP; one for the ESP32, one for the ESP8266. Ensure you have the right one installed for your board.
If you still get errors, email the output so I can investigate it for you.
@@RalphBacon I did verify that I had installed the correct version for the esp32.
Here is the messages it gives when trying to compile.
Using library AsyncTCP-master at version 1.1.1 in folder: C:\Users\Documents\Arduino\libraries\AsyncTCP-master
Using library AsyncElegantOTA-master at version 2.2.6 in folder: C:\Users\Documents\Arduino\libraries\AsyncElegantOTA-master
Using library WiFi at version 1.0 in folder: C:\Users\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\libraries\WiFi
Using library Update at version 1.0 in folder: C:\Users\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\libraries\Update
Using library ESPAsyncWebServer-master at version 1.2.3 in folder: C:\Users\Documents\Arduino\libraries\ESPAsyncWebServer-master
Using library FS at version 1.0 in folder: C:\Users\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\libraries\FS
Using library Credentials in folder: C:\Users\Documents\Arduino\libraries\Credentials (legacy)
exit status 1
Error compiling for board ESP32 Dev Module.
There's no error I can see in the bit you pasted. You need to send me the whole thing from the time you press Verify to when it completes. Ideally in an email, not here, but...
wow superfantabulous😊😊😊😊😊😊😊😊😊😊
Thank you so much 😀
HTTP Error
Stops on 70%
Buggy and unreliable.
Elegant OTA is a waste of time
No, Bernard, I use it all the time here, very reliable. However, I did discover that if you have ANY kind of interrupt routine running (or triggered) whilst the OTA is in progress then the OTA update will fail.
I have to now disable the interrupt when I detect an OTA is in progress in the main code. It gets re-enabled automatically when the device reboots after the OTA.
@@RalphBacon Ok cool, how do you detect that OTA is in progress?
As part of the OTA process there will have been declared (not by you) the following:
UpdateClass Update;
This means you now have access to the Update object. In my loop, I test for the update running and disconnect the interrupt like this:
if (Update.progress() > 0) {
// The interrupt will cause a PANIC if triggered during OTA so detach
detachInterrupt(digitalPinToInterrupt(pirPin));
log_v("Interrupt for PIR detached during OTA");
}
@@RalphBacon Ok cool thanks. I will try that. Thanks for the help
What happened to your voice? You dropped an octave. Having a cold, corona?
I think you've been watching my previous videos at x1.5 or something like that - this is my normal voice! Really. No cold. No corona (apart from the beer). 🤣
@@RalphBacon well, it must have been my audio settings then :-)
Messaged you on fb ralph cheers
I'll have a look, Paul. Or email me.
As soon as I hear the Arduino I stop watching.
Hang on, as soon as you hear "Arduino" on an Arduino-based YT channel you stop watching? Have I got that right? (This video is about the ESP32 though).
@@RalphBacon i did not search the Arduino. It was recommended to me. Nothing against your channel.