Streaming Video and Audio over WiFi with the ESP32
ฝัง
- เผยแพร่เมื่อ 18 ก.ย. 2023
- Sound and Vision - it works! As always we're sponsored by www.PCBWay.com/?from=atomic.
It's working! We've got Sound and we've got Vision. We've even got a remote control - pretty cool!
The display I'm using is this one - s.click.aliexpress.com/e/_DmS...
But the code should work with any display that the TFT_eSPI library can drive.
The code is available here: github.com/atomic14/esp32-tv
It's pretty rough and ready - use it at your own risk!
---
If you'd like to help support the channel and help me buy more nonsense - I have a Patreon page: / atomic14
We've also got a shop now! shop.atomic14.com
And if you're shopping on AliExpress or Amazon then if you click on these links I will get a little bit of affiliate money:
Aliexpress - s.click.aliexpress.com/e/_Ac05mF
Amazon - www.amazon.com/?tag=atomic14-20
You can also join us on Discord here: / discord
There's also the Atomic14 newsletter here: atomic14.substack.com/
And don't forget about the MakerNews newsletter - makernews.substack.com/ - วิทยาศาสตร์และเทคโนโลยี
If you'd like to help support the channel and help me buy more nonsense - I have a Patreon page: www.patreon.com/atomic14
We've also got a shop now! shop.atomic14.com
And if you're shopping on AliExpress or Amazon then if you click on these links I will get a little bit of affiliate money:
Aliexpress - s.click.aliexpress.com/e/_Ac05mF
Amazon - www.amazon.com/?tag=atomic14-20
You can also join us on Discord here: discord.gg/h2DDnDdjD2
There's also the Atomic14 newsletter here: atomic14.substack.com/
And don't forget about the MakerNews newsletter - makernews.substack.com/
Haven't seen The Young Ones in forever.
Really hard to find much of it online at high quality.
Hello there! I'm absolutely stunned at how you managed to do this. Especialy the sd card hack....well done. Is there any way I could pay you to help me with the same project? I have an esp32 wrover with camera and would love to make a miniature unit.
You sir, are a true asset to the world.
Thanks very much :)
Excellent project! I'm attempting (unsuccessfully so far) to stream camera and audio from an ESP32. Your design has sparked a few ideas, thanks! I'm a big fan of your channel, you do great work.
Thank You! I'm a big fan of your channel - really useful and informative - and fun to watch!
Both of you guys are my “go to” when I need these stuff.
Stay blessed and strong
Wow that's incredible indeed. Thank you so much. We really appreciate your great effort
Thank you. Appreciate it.
Wow. Incredible job. Very motivated to keep pursuing this kind of embedded work.
What a great collection of little elegant hacks - can't wait to try this! Super inspiring, can't wait for more!
Good luck! The code is very “proof of concept” but should work…
Amazing project! Really would love to make this - with The Office US running on loop all day long! Haha
Do it!
@atomic14 probably want to see an SD version... 😎🤣
Check out the latest code on GitHub :)
@@atomic14 ohhhhhhh...... nice!!!
This is a very valuable project with a lot of tricks and knowledge. Thanks for sharing this with us. I will try it out.
Thanks - the code is pretty rough, but hopefully understandable.
THE YOUNG ONES!!!
So few knew about or remember that show...
Yep, bit niche nowadays, but what a great show.
This gives me goosebumps;
Thanks buddy, waiting for the esp32 pcb work through, so that I can order my first pcb from PCB WAY 😊
Think I'm ready to start on the new PCB.
@@atomic14 ,
Already subscribed and
all notifications are turned on for this channel.
I’m in receive mode now :)
This randomly appeared in my feed. Hopefully TH-cam is sending some good algorithm vibes your way as this is super cool. Great work!
Thanks! The algorithm will work it’s mysterious ways. I’ve got used to videos either taking off, or dying in obscurity!
Wow! I'd almost decided it was a lost cause to try to get this level of quality out of the ESP32 but you've done it!
It's amazing what you can do on a module that just costs a couple of dollars.
Nice work on this! Love the JPEG streaming technique and audio stream technique.
Thanks - was nice that “it just worked” one of those projects that just falls into place.
That's why I subscribed your channel. GEM 💎
whoah 59fps raw draw rate was much faster than I was expecting
Yeah, pretty crazy, but that is doing nothing apart from pushing pixels to the display. It's pretty amazing that the SPI can run at 80MHz reliably.
Good work!!!! I was thinking, is that possible to add A/V to connect some old games ou VHS
Was wondering how you'd handle drift/decorrelation: syncing based on the elapsed sound is pretty nifty! Nice work using the 2 cores + DMA, all this was quite fun--thanks!
Thanks! Was a fun project - all fell together quite nicely. Code is a mess :)
I really love your videos! Thanks for your detailed explanations.
By the way, do you have a video or a link about how you set up your editor for easy flashing/programming for the ESP32?
I just use platformio plugin for VSCode. But for small projects the Arduino IDE is perfectly good.
Espectacular!!! 😮 Estoy haciendo un sistema de letreros de propulsión y aplicaré este video para dicho proyecto!! 🎉🎉😊
Good luck with your project!
Well done sir, thats pretty impressive
Thanks!
Wow. That's some impressive work
Thanks very much!
Hi! There is a way to stream audio by socket and reproduce that on the esp32 side? Thanks
Genius, I love it! Have any plans for a ESP TV PCB? Without the screen breakoutboard? You can get it even smaller! Ow and just a suggestion/question. Are you constantly outputting data on the serial bus? If so, if you stop that your code will run faster. But I'm sure you already thought of that. Again, amazing work!
I’m very tempted to find a really small screen and build a custom pcb specifically for that - with an SD card holder. But the code will work on pretty much any ESP32 - I’m going to try it out on my collection of devices (including the touchdown) and see how well it works.
@@atomic14 I'm curious to see what the smallest screen is you can find!
Awesome work !
Thanks!
Wow that was quick! Nice work, I'm surprised you were able to use both cores without jamming up the WiFi. Maybe I need to try more experiments rather than taking the espressif API as gospel.
Most of the time the cores aren't really working very hard - either waiting for network traffic or waiting for DMA requests to finish. So it's probably pretty lightly loaded.
@@atomic14 Aren't there actually three cores?
looking good can't wait for the washing machine.
Ken and Barbie's house is coming along.
Very interesting project 🎉
Thank you! Cheers!
Haha great! So we can built mini retro TVs now!
They'll be everywhere soon!
This is fantastic!!
Thanks!
Legend. Absolutely epic.
Thanks!
Wow. Thanks for putting this genius level stuff within the reach of ordinary hobbyists. Would it be possible to do this on other types of screens (an old tablet screen, for e.g.)?
Probably not on an ESP32 they only really work with SPI screens.
@@atomic14 So it should work on TFT LCDs like ILI9488, ILI9486, ILI9341, etc.? How complicated would it be to integrate touch screen capabilities into your project?
So cool!
Thanks :)
Amazing! Question, could you run the server on a android phone?
No reason why not - you'd need to write the code to do it. At the moment my server is written in Python and uses tools like ffmpeg. But all the mobile platforms have good support for this kind of stuff.
you should add the rc filter for the ir-receiver. i also didn't add one on a board a few years ago since it always worked. then after using it a while it stopped working, after touching the pins of the receiver (or toggle power) it worked again. it somehow started to oscillate or something. quite annoying bug...
Definitely - I’ll build that into the next PCB. The IR control does seem to be a bit flaky at the moment.
its possible to work like a audio video live stream via udp server?
Awesome work, very impressive. 😀
Patrick from Bethesda, Maryland, USA
Thanks! I know Bethesda - I worked for a while at a company near there.
Is it posible to reproduce TH-cam online on esp32 without server?
What do you think about the idea that we need a new media codec that holds both audio and video?
Ever since Periscope became popular it’s felt like the right time: Since we get audio and video from the same place (the device or the streaming media file) we can combine them during encoding and never have out of sync audio again.
Up to a point I'd agree. The issue you end up with is that it can be quite expensive to decode and display the video and you might want to skip the video frames, but still keep playing the audio. There's also the problem of needing multiple audio streams for different languages - it all gets pretty complication. Even simple containers start to get hard to use (I've just been decoding AVI files and my head hurts!).
@@atomic14 I appreciate your insight. It makes sense that various devices could regularly choose to skip video decoding some frames to keep users comfortable. I don't think different languages are that big an issue since we already have the technology for arbitrary numbers of channels (and in the main use-case I'm talking about livestreams that are not realtime-translated, though maybe I'm limiting it there since realtime translation tech could be less than 5 years away).
I wonder: if hypothetically there was a codec that could encode both video and audio with the same process, would the result be a single stream that contains both in-sync?
I understand "how we got here" (silent films > adding sound > a rich history of format-specific tools and workflows) but something is nagging in my brain about the fact that we don't really use cameras and mics that way any more, that the majority of recording, editing, and encoding is done on a single device and that device now needs mitigations to avoid the kind of "professional work" that is inherent in these long-standing workflows (such as keeping audio and video in sync).
i only want to play audio from a link...what should i do???
Do they have these displays with SPI ports?
I2C is a lot slower. Maybe that can speed things up?
It's an SPI display. We're pretty limited by the time it takes to decode the JPEGs, the library is pretty optimised but you could probably get it faster with hand tuned assembly.
@@atomic14 My bad. I saw the SCL and SDA and thought it was I2C.
Didn't look for CS after that. Does it have both I2C and SPI?
I wouldn't have been tricked, if they used SCK and MOSI markings like normal people.
I think it's just SPI - but these displays are a bit of a mystery to me!
You, sir, are like the american Andreas Spiess, so, you are already in the best company... ;-) 6:39 It´d be much easier to solder, if you´d dip the component´s leads in flux and a solder-bath before the actual soldering. I´m very bad and clumsy at soldering, since ever, but the solder-bath solved the problem.
I’d be worried about tipping it over and spilling it everywhere - I’m ridiculously clumsy! But it would be interesting to try out.
@@atomic14 I beg you, I´m surely clumsier !!!! ;-)
Seems like what you really need is a program you can run on a desktop that can take any steaming feed and reduce it to a stream of JPEG's and audio, to then nurse the ESP player.
Definitely doable. My server at the moment is very simple and preprocessed the entire video. But there's no reason someone couldn't write something more sophisticated.
Iss there a schematic for this great project ?
No, but it's basic components that you can connect up on a breadboard with an ESP32 dev board.
As I can understand from the explanation at the sourcecode you are not streaming but polling. That method is pretty slow and has alot of overhead by sending requests, headers etc and open and close the connection on each request. Why not using Websockets instead? With Websockets the connection stays open (real-time conection) and you push and pull data in any format you want. Actually when the connection is open the server can do all of the decoding work and can deliver a format that is directly usable to the ESP. In fact very similar to how analog TV-broadcasting works. Because this is a very low resolution, It could be possible to send an analog RF signal (made digital) over Websockets including FM quality sound. The only thing the ESP have to do is to decode the RF signal (digitally received) and show the updates like an old skool TV does. With Websockets you can also make channels by using a different port number. This is just for info and ideas, not to criticize your creation. Have fun. 🙂👍
Websockets is definitely an interesting idea. To be honest it didn’t even occur to me use them until this message. I have switched on http keep alive which should mean the http session stays open. But there’s always room for improvement!
OMG The Young Ones!
DIAGRAMA DE CONECCIONES ????
Can Esp32 cam send a stream that the little TV can receive
Quite possibly yes. The ESP32 Cam can generate a stream of JPGs. So would be quite easy to do.
@@atomic14 Will be so grateful if you could implement the SD card version and the ESP32 cam example. Thanks indeed for your great effort.
SD Card version is done. I need to find where my ESP32 cam has vanished to! It's somewhere in the collection of stuff...
Which display is this
I'm using this one - s.click.aliexpress.com/e/_DCdxXwX
Awesome. I wonder if any young people will know what that BBC place holder is?
Now you’re just making me feel old :(
@@atomic14 it's called sophisticated
great !!!!
Thanks you!
thanks to you !!! You inspire me !!! @@atomic14
Not bad at all,
Thanks!
👍👍👍
could you please provide your code?
In the video description :) - github.com/atomic14/esp32-tv
Not a TV, a video player.
You need a big freznel lense to watch this like in brazil.
Ah now, that's a great film. Like my Young Ones video, might be lost on quite a few people now :)
next video play video from sd card
It's on the list.
Check out the latest code - serving AVI files from SD Card - new video in the works.
do i look like i know what a jpeg is
How old are you sir?
Well, you know now :)
What about to play youtube?