Autofocus on a Raspberry Pi Camera?
ฝัง
- เผยแพร่เมื่อ 1 มิ.ย. 2024
- ArduCam has a new 16 megapixel Autofocus camera for the Raspberry Pi. Is it worth $25? You bet! But it's not all roses and butterflies. The software isn't up to speed yet, and the autofocus might not work quite like you'd expect.
The camera was initially launched via Kickstarter: www.kickstarter.com/projects/...
You should be able to buy the camera soon-check out www.arducam.com for more information.
Support me on Patreon: / geerlingguy
Sponsor me on GitHub: github.com/sponsors/geerlingguy
Merch: redshirtjeff.com
LeMaRiva Tech video on HQ Camera autofocus/aperture control: • Autofocus for the Rasp...
#RaspberryPi #ArduCam
Contents:
00:00 - ArduCam's new camera
00:40 - Camera overview
02:01 - Autofocus
03:29 - Image quality
04:59 - Video quality
05:32 - Software issues
07:02 - Recommendations - วิทยาศาสตร์และเทคโนโลยี
That "Only fans" box, tho...
Yeah, I always knew it .. there must be a box containing only fans!
Jeff, I agree with you 110% when you said ..."we both know THAT'S not happening".
...you're also right about this being an "impulse buy" 😉
awesome! This'll be neat!
Well done as always, Jeff! Once the software matures a little and support gets merged in upstream to libcamera, I imagine this camera will be a real help for motionEyeOS and 3d printer users. Having a camera better focus on the type and size of a 3d print I'm printing on the fly will hopefully mean more accurate results for services like Print Nanny and Spaghetti Detective.
Yes! Monitoring 3D printing on any of the printers with movable beds is a chore with the Pi Camera v2. I think this camera will do nicely for that.
I could use some automated 3D print monitoring.
7:33 outch… that one hit close to home 😵💫
the 1080p 30 fps is (if remember correctly) a limitation in the broadcom VPU firmware/driver for the CSI port. Same as the thoshiba based hdmi to csi module.
That would make sense. Maybe a future Pi generation will break through and get 4K@30 and 1080p@60!
@@JeffGeerling from what i gather over the years , the limitation is purely software hidden in the closed source binary from broadcom used to bootstrap the processor.
Again if memory serve, the arm core is initialize by the GPU/VPU, like in the Nvidia Tegra.
The issue would be licensing related :(
PS : You can call it a bios, but run on the gpu first (or a microcontroller inside).
X86 graphic card have a bios too, but written for the processor to initialize VESA routines and mailbox, semaphore as interface for the driver.
( that's why graphic card driver failed on Arm board, the bios/firmware is for X86 )
@@vincentvoillot6365Nothing in regards to the camera, but about the GPU stuff, as this has been said quite a few times, but isn't really the reason they don't work that easily. The linux GPU drivers don't really rely on the GPU bios, atleast the x86 part. On AMD cards, most of the bios isn't x86 code, but instead ATOMBIOS bytecode. I think there is some x86 code that functions as an interpreter for the bytecode, but the kernel driver just includes an interpreter too, so that's not a problem. The issue that makes it a lot harder to get GPUs to work is that the Pis PCIe bus doesn't support 64 bit transfers, and that there is no cache snooping/cache coherency, which makes it more complicated to ensure the card actually gets the data it's supposed to get.
@Coreforge
I didn't knew about the 64bit transfert issue, wondering about the cause ? Packet handler logic done on the cheap ? Strange that should be resolve during the handsake between devices, PCIe seem extremly flexible on his protocol ( header/transfer size, number/speed of lines )
On X86, VGA BIOS is load at boot time, that's why you can enter your pc bios on your screen. You can see this as an API a small x86 code to interface to wathever is going on GPU side. Your PC BIOS do the same for all onboard periphericals avaible at startup.
Like the usb stack for storage, keyboard and mouse handling or the SATA/PCIe (same tech) for drive discovery , did linux have a complete implementation or just expand over/ interface with it ? some of these are litteral blackbox.
"Stanford Seminar - A Superscalar Out-of-Order x86 Soft Processor for FPGA" on youtube around 9.20 ;)
About cache snooping, it is the invalidation of a line of cache when the corresponding adress is overwrite in memory during a DMA transfert. It is done at the processor level on his local bus (AXI in this case) on wich depend the PCIe root. It's alway the case in architecture with a central 'shared' memory since processor have cache.
And the GPU does the same with his cache and local memory. Yes the current gen surely have more advance feature, but none of these should be mandatory for compatibility sake.
@@vincentvoillot6365 - The problem I've heard is the PCIe root complex on the BCM2711 is broken in some way (though I've never gotten a specific as to what, exactly), but the symptom is that the entire system halts (in a non-recoverable way) when certain memory access patterns happen-most often for me when testing GPU drivers. No kernel panic, serial debug output goes away, nothing.
I still use the Pi webcam project you posted a while back for all of my online classes and family video calls.
Thanks again.
You're welcome! Next time you think about building one again, check out the showmewebcam project-it's even lighter weight, and it looks like they might crack the nut on making it work better on the Pi Zero 2 W (which can handle the 1080p output better, if you want it).
I liked this video off the strength of your transitions into and out of the eye of the ArduCam ... Your videos are still getting better over time, wish you well Jeff.
Biggest upgrade this year is a new motion control slider that makes some fancy camera moves (especially for tiny things like this camera) waaay easier than it used to be. Glad someone noticed the effort :D
Ive been waiting for you to cover this, seemed like it would be up your alley.
Great video, looks very promising and the introduction of an outro, although I miss the bloopers
I backed 4 of these in their kickstarter. Can't wait to get them and set them up.
Very cool! I'm definitely interested. Thanks Jeff!
7:40 that is so true, but maybe this I will finally make comeback because of the functalitty improvement of the camera from arducam, thanks!
Great video Jeff.
Thanks for all you do!!!
Great Video Jeff! I'm looking forward to adding this camera to my cctv project.
The Arducam will find a fantastic home in a 3D print bay, especially if the camera is mounted close to the print bed and extruded. Autofocus that could work automatically for different size models would be insanely useful.
If your print bay is 7 ft from the cam then ya.
@@Hex-Mas I disagree. Printer with movable beds will have varying focus purely because the model could slide both towards and away from the camera, ruining the finer details. Sure, if it's just a monitoring camera it'll be fine, but a nice time apse that remains constantly in focus without the need to stop the bed (Ender and Prusa series machines in particular) will benefit heavily from overall image quality and reduced print times.
@@davidmalkowski7850 ok sure lol. I just use a android and Cam over USB works flawlessly and has all the features that rpi cam's lack.
i was thinking this too. All of the issues are software, which means even if its annoying today it wont be in the future. | also taking into account grid based movement detection could trigger autofocus on-the-fly as-is, solving this problem today. (Motion is the name of the software im referring to. you can trigger events, to request a focus update).
Combined with a wide-angle camera mounted at the edge of the print area, this would make print monitoring great.
1:03 I *NEED* that Raspberry Pi mug
7:34 The only fans crate with fans catch me... 🤣🤣🤣🤣
For the shallow tripod mount, you can just add neoprene washers between the two to compensate so it isn't loose.
This is true-for my testing in this video, I just cut a piece of cardboard and poked a hole in it for the screw.
The "ONLY FANS" box at 7:32 had me cracking xD
cool video and great info...I also liked at the end the bins. I'm getting way to much stuff for to many projects laying around and cluttering things up. Looking for a way to organize things....thanks.
It's a constant battle. This was one of the good days, when everything was in its place :)
i have been using V1 camera so this will be a great upgrade for me
I like the part with boxes full of stuff. Do you plan to go through that one time and make some kind of retrospective video?
That may be a fun thing to do, maybe in a future live stream! Last time I took something of a tour of the office was in spring last year. Could make it annual!
Perfect timing for a new video. Just got my food delivered 👏🏻 Snack & Watch!
Nice. Autofocus, finally. Will be great to see the software catch up.
Very cool. I immediately went to the Kickstarter page, only to find that the campaign has ended. Bummer! But I'll probably still get one. Great video!
They should have some available pretty soon after shipping the campaign backers' units!
I've been considering a little surveillance cam project. I wonder if this cam would be a good fit.
Hey Jeff, thanks for the video! Just yesterday I was looking at Pi cameras. Looks like V1 camera is still a great bang per buck. I was fascinated about it's price, somewhat around $4 delivered from Ali and with 1m cable. At least it worth to have that one just in case, and replace later with better one if needed. V2 is almost 10x more expensive but not that much better. The new ArduCam looks a lot better with the same claimed price as V2.
Yeah, I still have two of the original Pi Camera models, and if you just need to capture an environment, and don't need the best quality, they're great for that.
Thanks Jeff.
Hi Jeff, thank you so much! Mind helping me here pls? Im considering the HQ vs the Arducam for PHOTOGRAMMETRY. Which do you think has better "spacial fidelity"? Also, can Arducam shoot RAW? And a dumb question, which has better resolution?
I have two Pi HQ Cameras set up as webcams using your script. One is a Pi Zero and the other is a Pi 4. I love having them as B and C cameras because they make it easy to put different camera angles on stuff I'm doing on the workbench. I am by no means a programmer, but I would love the ability to adjust things like resolution, brightness, exposure, and shutter speed on the fly inside the windows environment I use to record video. Also, I would love the ability to shoot 1080p in 24fps. Theoretically it should be possible, as the sensor is more than enough to handle it. 4K 24p should also be possible, but I don't want to get greedy.
I have a optical lens with zoom focus variable iris and a lot of cool things I've bought that years ago. My plan has always been to use with a rpi. The issue i was facing was how to get feedback from the ISP. But now there is libcamera I'll take a look on how they did the autofocus...
I should also 3dprint an adapter for that lens and finally understand how the iris works...
That was a project for a drone camera to get zoomed in picture without going to near to walls. (My mother is an architect and would have loved a device to do such thing... Expectantly for roofs)
Man it has been almost 10 years...
Why not add another box full of hopes and dreams? Now you’re speaking my language.
Could be good start for an open electronic glasses design.
I backed this on kickstarter. I want to test it for robotics. For robotic computer vision you only need very low res but you need very low exposure times because if the robot is moving fast higher exposure times blur the image. The RPi HD cam is both more expensive and much more bulky. Exposure times are limited by both the amount of light that can come through the lens(size of lend) and the size of the sensor to absorb the light. The arducam does have a bigger lens and bigger sensor than the RPi standard def cams so am hoping for faster exposure times and less blurring on moving robots from the arducam.
Would be great if you could make a video about all cameras (mostly non-official) which you can use with raspberry pi! I am working on rpi+edgetpu+camera project so it would be very useful!
Now that you're done with the general review video, could you (hopefully non-destructively) tear the Arducam down? I'm interested in knowing if it's possible to use it for PCB inspection tasks by replacing/modifying the lens enclosure :)
Has there been any improvements on the software side of things? I just bought this Arducam and am finding out it does not work with my RasPi 3 B+ out of the box. What do I have to do to make it work?
I to an Arducam for x-mas but not the one with the C mount, instead the one wich is based on the HQ, cam but supports 4 lane CSI-2, hence I can use the sensor to its full 4k60 potential.
Disadvantage is because the RPi doesn't have an h.265 hardware encoder (only a decoder) and I doubt the Pi can encode 4k60+HDR in software without dropping a frame.
So my Idea is to connect the camera to an FPGA which does h.265 (and more) than send the videostream via PCIe to the Pi and than save it to an SSD (again via PCIe)
Certainly a very complicated project and I don't know if I will even start it in 2022.
That Rocky Linux T Shirt looks good!
Hi Jeff, Please tell us how to add audio to an RP high-quality camera and do video streaming
I've been jazzed since I bought the HQ Cam last year, but the complexity of focusing has meant I haven't put it to good use.
I kinda just aasumed there were _many_ other 3rd party camera module options. Your video seems to suggest there are less than I'd thought...
There are a few, but almost all of them are basically clones of the v2 Pi Camera, or they have very little support so good luck getting them working.
ArduCam at least seems to have a good track record in the Pi community, and works on the software and documentation to make it easier to use their products.
Is it worth $25?
Take my money now, now now now.
I think that's a yes in cap's and animated happy GIF
I wish you could make a guide! Im struggling for several hours now to get it to run on my Pi4 b. Tried it in buster, bullseye and the openscan image (from a 3d scanner i want to use it for) . So far im not even sure if the camera is running.
Damn that hits close to home about the box of unbuilt projects.
hello i've question , could my camera work automatically without the necessity to click run in the terminal of my raspberry pi, i've tried already the auto run but it dosen't work !!!
I wonder how much the autofocus delays the pics, i.e., I am using the Pi to take continuously still pics, which brings me to around 1 frame per second using a HQ camera module on an "old" Raspi Zero. What "frame rate" would be possible with this camera module?
May I ask why you substitute for the aperture interms of shutter speed? I feel that going with the best features of each camera is a reflection of it's true abilities.
I used the widest aperture for each camera (and the lens with the HQ camera), since you can't actually control aperture on the two smaller cameras. Then I adjusted shutter speed so the level of light hitting the sensor was about the same between the three cameras.
If I didn't adjust the shutter speed, the cameras with the smaller apertures (higher f-stop) would've had darker images at the same ISO (I believe I did ISO 80 or ISO 100 for each of the cameras).
any progress on the cut up raspberry pi? love that episode
It shall return, don't worry. just have some other things that I've been knocking out and it's currently in a little ESD bag on my bench!
@@JeffGeerling best keep it safe eh :)
do you have a link to the motorized autofocus for the big pi camera?
This is great.
I have been consuming knowledge from your videos like crazy and decided to leave a sub. I see you have this Squier behind you, and now I even noticed you have a guitar pick in it. So you do play on it, and its not just for the looks. I also play guitar, and I was wondering if you could find any use for us guitarists with raspberry pi. And im not talking about DAW and some guitar tuner app that will run like hell. Im thinking about something we can realy use. Like amp sim. IR loader. Multi FX. Single FX pedal. Gate. Im sure most beginner guitarists NEED a freaking gate before someone gets crazy. But it need to run fast and well and without latency. Otherwise nobody will use any of this stuff.
It's something I've pondered before, haven't thought of anything where a Pi would do better than an existing solution (at least not without adding a bunch of extra hardware that actually does the work that needs to be done, in terms of signal processing).
Mostly I've seen people use Pis for recording (sometimes) and light effects, but the Pi isn't the greatest for close to real-time audio processing :(
And yes, I play... in a sense. If learning to play three times in a decade then letting my fingers go soft again counts :D
I'd call myself very beginner level-there's also a nicer Fender acoustic guitar on my other wall, though it's loud enough that I can't play it too often. Only time I get a chance is when kids are sleeping!
Hey, and what about the arducam that can handle 4 lane CSI? With more lanes it´s possible to record even higger resolution!...
Right?
The name of the camera:
Arducam High Quality IR-CUT Camera for Raspberry Pi, 12.3MP 1/2.3 Inch IMX477 HQ Camera Module with 6mm CS Lens for Pi 4B, 3B+, 2B, 3A+, Pi Zero and more
- Sensor: IMX477
- Optical Format: 1/2.3″ (diagonal 7.857mm)
- Resolution: 4056(H) x 3040(V) 12.3MP
- Pixel Size: 1.55um x 1.55um
- IR Sensitivity: IR Sensitivity:onboard motorized IR cut filter, support both day and night vision
- Interface: 4-lane MIPI CSI-2
- Default Lens Mount: CS-Mount
- Frame Rate: (depends on the target platform, maximum fps limit might apply)
Full resolution@60fps(normal), 4K2K@60fps, 1080p@240fps
Full resolution@40fps (12bit normal)
Full resolution@30fps (DOL-HDR, 2 frames)
Is that real?
Can i use it in Jetson nano for computer vision projects
Nice. My coworker sent me an article on this a few weeks ago. Will be using it for my 3D printer. Was also tempted by their “spy cam” just to see how they pull it off.
ArduCam seems to have a lot of fun projects in the works.
Does it give the option to point focus on a specific region of the incoming image? Edit: Nevermind you touched on that later in the video. :)
I noticed that Arducam were working on a 48MP camera module for the Raspberry Pi which would be cool if it were auto focus, for higher quality images.
Does it work in low light? Coz the ArduCam i have you have to turn on 12 lights to get it at 40% brightness.
I use droidcamX over USB works flawlessly.
🙃 .... soooo finally : what is the difference between these cams, and can we simply use any one of them almost with the same picture quality, effects 🤔 ?
Thx
Did ardicam give you any sort of timeline for there work pushing upstream?
Nothing specific. I should hope it'll happen this year.
Given that the autofocus in the OnePlus 6 using the same sensor in its rear camera is really impressive, the software still has a very, very long way to go on the pi...
Sounds like tons of room for exciting future improvements!!
I'll look forward to that
could i use a pico w for the cam, as there are no zero 2w
Seeing as the IMX519 is a pretty standard Sony EXMOR based sensor, it has phase detect autofocus built in to the sensor. So any issues with autofocus are going to be software based and/or csi interface based. As said below the csi implementationis pretty basic and capped realistically at 1080p30 to hit the $35 price point on the majority of Pi's
Facetracking, eye tracking etc. are 99% software features as well, and on a smartphone or camera are built using the ai chipset features hardware wise to track objects. Seeing as there is a distinct lack of any such hardware on a Pi that's why it will take some time - again the software lacks behind the hardware, something that has happened across a lot of products in the last 6 months
Does it provide any feedback? Could it potentially be used for range finding?
You could probably do that but you would have to write a python script and calibrate the distance with a tape measure.
Can you set fixed focus using auto focus settings?
Never mind, that just got answered!
For anyone still wondering-yes ;)
Hi Jeff
I saw your Rocky Linux T-Shirt, so i would like to ask whats the diffrence between Rocky Linux and AlmaLinux, maby you or someone else of your community could give me a clear answer. Cause I now work for a cloud provider and I create the images which we will provide. But we still have no upgrade path to centos 7. Based on my research the only diffrence is the logo and where the package mirrors are located...
Thanks for any answer & carry on doing such great content
Mostly it's the leadership / affiliations. Rocky was started by a few people who were more strongly associated with the historical CentOS leadership, and has some backing from certain corporate sponsors. Alma is similar, though I think they were slightly earlier in getting releases out the door.
Right now you can't go wrong with either, but long-term I think one of the two will probably end up merging into the other.
I was getting ready to click when you said until next time, lol
What is the advantage of those cameras against usb cameras? Real time DSP for things like face recognition?
Mostly for integration into more compact builds, where you don't want to hack apart a USB webcam to try to fit it in a case. Also, as the V4L and libcamera utilities get better / more standardized, I hope to see more projects that just work out of the box with any CSI camera.
Mr Scissors :)
I wouldn't be surprised if Chris/ExplainingComputers takes a look at one of these at some point.
I'd love for him to add it to his lineup and compare it to more of the cameras that he has. He's done a lot more with Pi cameras than I have!
wish there was a cmos connected directly to ram to capture as many as possible fps. for slow mo
Maybe Octolapse usecase after support is added ?
I am having problem with v2 camera and pi4 8gb. The libcamera can’t fined the camera. I hate the new raspbian OS. The change the OS without fixing the main future like the ribin camera slot.
Would be nice to see this cam with cs-mount
Huh I saw the iBook g4 disk replacement I think it was from like 15 years ago and I’m surprised how iBooks compared to new laptop also like iBook g4 looks like but there not too good in 2022
Do these cameras only support jpeg output, or can you get raw images as well?
You can get raw image output as well using the proper libcamera options.
Would be very useful for a very high-quality Wifi camera
Edit: Is that $25 price just the Kickstarter price, or is it the fully released non-kickstarter purchase price?
I'm honestly surprised that nobody has made an autofoxus camera before, given the huge quantity of mobile phones that do autofocus
As long as you can focus manually with digital zoom it will be clear and high-res.
It seems to focus better than my Panasonic G85 and GH5 🤣🤣🤣🤣
lol
We need to see tests that determine if it can detect Red Shirt Jeff's attempts to break in and cause shenanigans.
I like the 'Only Fans' box on your shelf lol.
Let's just hope red shirt jeff doesnt turn into red light jeff
@@gspeed0689 Oh no! Bad visuals lol.
Might make a nice replacement for my security camera.
I backed that camera too.
Guess it is time to start building that birdhouse with built-in surveillance that's been on my todo list for a while 🐦️🐦️🐦️
Hi Jeff, can this do IR?
No, but they sell a version without the IR filter, that can.
Do you know of any 480+ FPS Pi cameras?
0:28 That's how you're supposed to do it! Would you have it any other way?
I'm curious why autofocus doesn't work by default. AFAIK, it's possible to enable/disable autofocus without any extra scripts. v4l2-ctl --set-ctrl=focus auto=1 on most cameras. Does this specific camera require non-standard API to control? How all other cameras work on Linux without patches?
P.S. sorry for my bad English, I'm not from US
This camera does have a slightly non-standard interface, my guess is it has to do with routing the autofocus signaling through CSI on the Pi, which is a little different than the way a typical USB webcam is built.
I'd like to see a LIDAR autofocus camera for the Raspberry Pi.
I can't think of anything clever or related to comment so I'll just post the bee movie scrip to help feed.
Bad news. No script. Sorry. I really hope this gets the software support it needs. Please keep us up to date. Cheers!
The first question:
Did you have to recompile the kernel? 😂
Ha, thankfully not this time, though I did have to uninstall libcamera!
Cool
Wow no outtakes today. #TeamTheVideoEndsAtTheEnd
Heh, I have been experimenting a bit. I still want the ability to highlight and thank everyone who supports this channel, so not 100% sure how I'll continue that if I do end up cutting short the outtakes.
@@JeffGeerling Thanks for the reply. Wasn't knocking the lack of outtakes but rather acknowledging the thank you at the end. Keep up the entertaining work in Mizzori (pronounced misery, right?) #IowaLife
Somehow this cam looks better than what most twitch cams looks like
Part of that is lighting. The other part is having such a handsome subject 🤣
Such a nice video!!! ♥️🌈You have a new subscriber here💐🌈♥️🌈♥️🥰🌈😍
No early drop off at the end here...lol
That only fans box lol. That's where the real redshirtjeff money lies.
Jeff, help RP autofocus on producing more units. Wonderful stuff in RPville, but no bloody hardware.
I'll bet Red Jeff could fix the outstanding issues with only a few power tools or Henry the Hammer. ;~).
And Stanley the Knife? ;)
if only there was Boomer the Blowtorch.. 🤣🤣
@@pandersen7984 Do NOT give Red Shirt Jeff any ideas!
16 mega pickles? I don't think I can handle 1!
🥒