Who Cares About MCU with Camera?
ฝัง
- เผยแพร่เมื่อ 23 มิ.ย. 2024
- In this video, we are going to have a vision test with two MCUs, Arduino Portenta Vision Shield and ES32-CAM(OV5640).
Unfortunately, acquiring and processing images from the camera module is a resource-intensive task. If you want high-quality images and fast FPS, you'd be better off using Single Board Computers like Raspberry Pi. Nonetheless, I want to try what is possible in the MCU's low power and lack of resource environment.
I hope this video will be helpful to anyone who wants to do a vision project.
*Timestamps
0:00 - Intro: Portenta H7 Vision Shield vs ESP32-CAM with OV5640
0:50 - Comparison: Portenta H7 Vision Shield vs ESP32-CAM with OV5640
2:30 - Test #1: Image on LCD
4:27 - Test #2: Image Quality (320x240)
6:05 - Test #3: Frame Buffer Acquisition Speed
6:33 - Test #4: Power Consumption
7:04 - Test #5: Person Detection (TinyML)
8:04 - Outro: Portenta Vision Shield, TinyML, LVGL
[Arduino Portenta H7 vs. ESP32-S3]
• Who Needs High-Perform...
[OV5640 for ESP32 Camera (Compared to OV2640)]
• OV5640 for ESP32 Camer...
[Auto Focus with OV5640 on ESP32 AI-Thinker Board]
• Auto Focus with OV5640...
[TF Lite Person Detection Model]
github.com/tensorflow/tflite-...
[ESP32-CAM AI-THINKER + FTDI USB to TTL Serial Converter]
amzn.to/3HBjjKM
[Waveshare OV5640 Camera Board(B) Image Sensor Module]
amzn.to/3XGSWKL
[ELECROW Raspberry Pi Monitor 7 Inch Touchscreen IPS 1024x600]
amzn.to/3USCHIu
#Comparison #Vision #Camera #MCU #Portenta #Espressif #ESP32 #STM32H747 #ThatProject - วิทยาศาสตร์และเทคโนโลยี
For more projects - youtube.com/@ThatProject
That Project Github Repository - github.com/0015/ThatProject
Join FB Group - facebook.com/groups/138965931539175
The HM-01B0 camera is focused closer, near focus. The OV2640 is focused for far. If you want to get a better close image for the OV2640, unscrew the lens a little so the lens sits a little further from the sensor. If you look at the ceiling light in your current images you can see a clean edge for the OV2640, the HM-01B0 is blurry.
To close focus the lens needs to move out. For far focus move in.
Thanks for making this
2640 would have done better at lower resolutions, just because how the sensor works and how it bins the image. Lenses make a difference as well of course. ESP32-S3 is much faster at parsing images (especially when OPI PSRAM is used). Maybe something to try next :)
Thanks for this. Actually, I have the plan to do it with ESP32-S3 as well.
@@ThatProject please try with ESP32-S3-CAM, pretty good for object tracking edge application.
Yes, I'm looking for the best device for it. Thank you,
Pretty impressive to get a Cortex-M7 to run slower than an ESP32
something that is just fast enough to transmit a video feed to a server would be a great open source cctv/smart doorbell alternative
Very informative. I am only just started using ESP32. I have an S2 too but not used it yet. I think the ESP32 will ably do every project I will be doing for the forseable future. I must say If I was a better coder I am sure the Arduino Uno would surfice for nearly everything I do even now. One of the main reasons I changed to esp32 was to standardize voltages to 3.3v. 5v + 3.3V caused to many problems and so much duff info out there, like yes you can , no you cant, do such and such. If I was going to splash out that much on a Microcontroller I would spend more and get an Nvidia!
Please let us know what are the smaller TFT displays you used? Where can we get them?
Screen Size: 3.2 inch
Driver IC: ILI9341
Resolution: 320*240 (Pixel)
Module Interface: 4-wire SPI interface
Great! What's your next one?
Well, I can't say now clearly but it's gonna be interesting. 🤨
Thank you !
Please test it ESP32 Cam with Wroover.
Is a difference with more memory?
You mean WROVER?
The biggest difference between the WROOM and the WROVER chips is that the WROVER integrates an 8 MB "SPI PSRAM" chip and we absolutely need PSRAM to run the camera module.
Turn on the PSRAM will slow down the ESP32. The best you can get is around 20fps.
But the PSRAM will not affect the ESP32 S3.
This does not surprise me. Espressif have always been ahead of the game. The chap who runs it is super intelligent. By the way, they do a lot of good as a firm regarding social issues like environmental, educational and looking after their staff well. Developer support is also second to none.
It’s a camera issue not the mcu, ov5640 works great with stm32
Hi! I have ESP32 and ov7670 camera module no fifo. Can i do person recognition amd tensorflow lite with them? Or do i need thes esp32 cam?
If you connect the ESP32 and the OV7650, I think you can use it without problems.
@@ThatProject thank you! I will subscribe
A Cortex-M7 slower than an ESP32, there is something strange. Is the program optimized? The STM32H7 has DCMI interface, and hardware JPEG codec.
I agree with you, but it was definitely slow in my tests with the Portenta Vision Shield. As you can see by looking at the code, there was nothing that could be further optimized.
@@ThatProjectyour channel is great!! About your answer, I have see a video on the TH-cam, that show a stm32h750 running a digital camera to lcd at 480x320, more than 30fps. But when writing a mjpeg video to sd card, it slows to aprox 10fps. It motivated me to buy a weact stm32h750 board to do some research.
It would be interesting to compare with raspberry pi zero 2 w with a raspberry pi camera module 3.
what function did you use to draw the grayscale image to screen on the esp32cam?
You can set the pixel format of the camera configuration.
config.pixel_format = GRAYSCALE
@@ThatProject sure, but do you still use tft.pushimage() for drawing on screen? instead of drawBMP or something like that alocates less memory? i want a grayscale 240x240 with the most fps possible so uint32_t or uint16_t that tft.drawimage() uses is much more than what id need. seems like i could optimize the code by using a more trimmed down version of drawimage(). does this make sense? thank you for your patience and attention. your channel is the golden standard for esp32 content.
hi! I tried the ESP32-CAM module with the OV5640 sensor , like you did . But i think there is a HUGE PROBLEM . The OV5640 sensor is OVERHEATING . (while the most common and cheaper OV2640 is ok) . . The high temperature causes a purple noise to show slowly on the image (starting from the outer part of the image to the center of it) . In low light conditions the image is completely destroyed by this purple noise to the point that ov2640 offers a better image than the ov5640 . You surely must have this problem , too ! It is even visble in the intro of your video around 22 seconds in. on the OV5640 side , you can see the white cloud around the center of the image , that is noise . if you blow some air to the sensor to cool it down it dissapears , but that means a (pretty big) heatsink must be used with the OV5640 . Can you please look into this ? Maybe it is something that can be fixed without a big heatsink . 😵💫😵💫😵💫.
I don't think there is anything wrong with my OV5640. Where is the noise around 22 seconds of my video? It's just reflected light. Please check the other part of my tests. Anyway, the OV5640 I used is from Waveshare.
@@ThatProject Thank you for answering , if you compare the frame on the 5 seconds mark and the frame on the 30 seconds mark , you can see that some detail is missing (for example the corner up and left and some lines of the square tiles) . Maybe it is reflected light and i need a longer video to say for sure, but to me it looks like purple noise slowly creeping from the outter part of the image towards the center . I have tested 2 esp32 modules and 3 different OV5640 sensors so far (also different lenses ), but the same problem is visible .That purple noise ! In low light ( think about night sky light condition , since that is the project i am interested in ) it is very visible . Lastly , I checked the other parts of the video (for example at 3:13) but i think there is more light , as there are maybe two ? light sources visible above the cameras . In ok-ish light conditions ( from what i have tested) i can say that the effect of the noise is less visible . 🤔
K210 can do better than H7 and ESP32 in these regards.
The Maix Nano with camera is much smaller and cheaper than H7. You can also attach ESP-01 to it if you need WiFi connection.
You can use Maixduino or Maix Dock if you need camera, LCD, and WiFi.
Further, you can run dual cores of K210 at 600Mhz and get very nice results.
2 cents~
Thank you for the good information. Let's see how it goes.
@@ThatProject BTW, "H7 + vision shield" has bundle price for $149. But Nicla Vision is only for $119.
@@ThatProject
Just tested 600Mhz K210 using one core only:
For QVGA displaying only is 118~121 FPS.
For QVGA camera video playing on LCD is 40 fps.
For QVGA camera video face detection, marking, and playing on LCD is 38~40 fps.
On ESP32 CAM, the best I can get is 20 fps.
Seems that the H7 is not good at this application. Well....
There is the new sipeed m1s dock as well which is supposedly better for machine vision than the k210 based boards.
@@conorstewart2214 For software consideration, it still has a long way to go....
BTW, there are many other little boards out there with RISC or A7 class devices, the issues are always not hardware but software.
I agree that Human detection was incorrect. You show how portena find in 0.5 second on picture 320*240. And in otherwise stm32 in 5 seconds on 5 Megapixels.
You should check it on same size, or even less megapixel camera to make camera buffer smaller and bit faster
I killed 2 ESP32 Cams due to overheating. They get verry hot and need to be cooled.
OMG. What did you do with them???
@@ThatProject I built a small case and let them stream the image continuously for a day.
The 8266 chips are like that. They overheat if you run the wifi excessively and do not have a heat sink. With the ESP32 the screened can will suffice to dissipate the heat. A tiny stick-on heat sink will not fix the problem.
Who Cares About Marvel Cinematic Universe with Camera?
😂
@@ThatProject Since I didn't know what the acronym MCU meant, the first thing I thought of was Marvel Cinematic Universe!
Esp hori
esp cares