Machine Vision with HuskyLens
ฝัง
- เผยแพร่เมื่อ 12 พ.ค. 2024
- HuskyLens is an AI Machine Vision sensor based upon the powerful Kendryte K210 processor. Today we will take a look at this amazing device, both by itself and with an Arduino.
Article with code samples: dronebotworkshop.com/huskylens
More articles and tutorials: dronebotworkshop.com
Join the conversation on the forum: forum.dronebotworkshop.com
Subscribe to the newsletter and stay in touch: dronebotworkshop.com/subscribe/
The Kendryte K210 processor is a 64-bit RISC microcontroller which has been designed for machine vision and voice recognition applications. I’ve picked up a few sensors based upon this powerful microcontroller, and today we will look at one of them - the HuskyLens.
The DFRobot HuskyLens packages the power of the K210 with a video camera, small TFT screen and a microSD card socket. This powerful, yet inexpensive, device can perform several machine visions tasks including face recognition, object classification, object detection and line following, plus many more.
Today we will take a detailed look at the HuskyLens. After updating the firmware, we will put the device to the test in all of its different modes.
After that, we will hook a simple Arduino Uno to the HuskyLens and observe the data we can get back from it.
We will also see how we can perform more advanced functions with Arduino and HuskyLens, such as updating the text on the screen. And we will also see how to save our trained models using the microSD card.
I’ll even give you a quick peek at a small robot I’m deigning around the HuskyLens!
Here is the Table of Contents for today's video:
00:00 - Introduction
02:32 - Look at HuskyLens
10:03 - Firmware Update
12:32 - Face Recognition demo
14:15 - Object Tracking Demo
15:24 - Object Recognition Demo
18:09 - Line Tracking Demo
19:24 - Color Recognition Demo
20:48 - Tag Recognition Demo
22:13 - Object Classification Demo
24:50 - HuskyLens with Arduino - UART Mode
31:25 - Changing Interface Type Manually
32:51 - HuskyLens with Arduino - I2C Mode
35:30 - Modifying HuskyLens Text
39:14 - Using the microSD Card
41:51 - Conclusion
As I said, I have a few sensors based upon the Kendryte K210, the HuskyLens is only one of them. I’ll be showing you more of these advanced machine learning devices in future episodes. - แนวปฏิบัติและการใช้ชีวิต
Hi everyone - It's come to my attention that there has been someone masquerading as myself, responding to some comments here with a link to a Telegram chat to win a prize from me. THIS IS A SCAM, I am not holding a contest, nor do I have a Telegram account. PLEASE DO NOT RESPOND TO THESE MESSAGES!!
It's happening on a lot of my videos, I'm taking steps to remove them manually, but as I have 162 videos, it will take some time. If you do run across a suspicious comment, I would appreciate you letting me know at info@dronebotworkshop.com.
Thanks!
Bill (The real one!)
Hmmmm, 8 different microphone inputs. Being a Marine there's only one thing I can think of to use this capability for, receiving gunshot sounds and determining the direction they came from for auto return fire to that location instantly, and neutralizing the threat immediately. That would be a great project for you to work on and showcase.
That is actually a VERY excellent point! (can't say the Corps doesn't make a sharp tactician!) I was actually about to dismiss this for my home surveillance/AI system because of how many cameras I would like since it would make more sense to have a central processing server, BUT maybe one of these on top of the house dedicated to that cause w/ a rotating shaft that could use those mics to determine to override the direction it is pointing would be nice? I bet I could calibrate it a bit manually and then update it by asking where hunters were when they blasted and what they used(surrounded by a State forest and know most of the people with hunting camps around me). By knowing the type of gun and distance, one could probably use the average decibels etc for that type of round to calculate the distance of the shooter as well as direction. Then it could scan for movement, track, provide range, and record etc. It has to also be possible to train an AI algorithm to determine the type of round from the sound which could help with automatically determining distance as well if it can be tagged to decibel data for that round. I mean if I can tell roughly from my limited experience I'm sure AI can. Anyway, thanks for the idea! Who knows if it is practical, but it would be an interesting project for sure!
Just like in Fortnite
Sorry I missed the party. You could easily build something to detect the direction of a gunshot. Once direction was determined it would be a simple matter to use trigonometry to calculate a distance. No consumer or industrial components meet the required specifications for military applications according to the Government Accountability Office. Those specifications are outlined in the United States Military test specifications as determined by the Department of Defense. You must use components designed specifically for either military or aerospace applications. Those components meet the required minimum specifications. The cost those components are going to be upwards of 20 times more than their consumer cousins. Thats why military electronics cost so much. A prime example would be a military laptop or radio. Those devices have more than just a robust case.
@@theminer49erz with a PTZ camera that uses the PelcoD protocol, it would be pretty simple to have a computer send a set position command to the PTZ in the direction of the microphone input.
One other thought would be to use remote "beacons" that send out a wireless signal when a attached microphone detects a loud enough sound. All the camera controller has to do is send a "call preset" signal to the ptz camera to look in the direction of the beacon that sent the signal.
Nice video Bill ! I watched it completely ! Looking forward to see the result on the robot build in progress !
Fascinating device. And, as is always the case, extremely well presented and documented. Thank you!.
This guy is a pure legend. All his explanations are so clear; I understood everything!
Keep going!
The only so clear explanation was HuskyLens "Board name" 6000 tims 😅
This is really nice & simple, really simplifies integration with arduino projects when its a separate unit with its own microcontroller and just gives out easily usable coordinates & data back.
Great info . Thank you for investing your time to share this with us . 1 year into arduino and mini pc . My first build was for a sonic alarm for parking pad . Hidden with outdoor speaker material in an old bose outdoor speaker . Saves me from security camera psychosis.
Big help brother !!!!
Thank you
I would like to thanks dronebot teaching. Very clearly and appreciated
Great video, as usual... Amazing to have this technology in our hands...
Thanks for pointing this out! As always a superb video!
Thanks for the detailed video!! Definitely going to buy this soon!
Thank you for your great content. You are a great presenter,and really simplify the whole process and application.
A lot of possibilities here. Really interesting. Thanks for answering all the questions I would have.
Great video, thanks for creating and sharing. Love your videos!
Very excited to see this, I got in on the Kickstarter for it but have not gotten to trying it out.
Well done Bill, another magical video.
Thank you for everything.
fantastic teacher
Thanks for your great tutorial and I hope to see a complete project using this lens as you said, until this thank you so much!
Wow! That's a great presentation!
Pretty neat ! Pretty powerful kind of curious how far away it can recognize.
absolutely the best tutorial on the internet
Very interesting video! Thanks Bill.
This is A Eye opener . Love it
Best tutorial ever
Great tutorial!
Hope you will follow-up with the audio equivalent....
wonderful job!
Thank you for this phantastic video! 👍👍👍
"It's an AI-opener of a lesson today" :)
great explanation, thank you
Allways helpful
Cracking tutorial! Many thanks
Great Video! Could you do an episode about load cells? That would be interesting!
Thanks a lot sir
@DroneBot Workshop: Do I have to train the HuskyLens with live footage or can I also use previously made recordings to do so? Great video btw. really makes me want to buy one of these also.
Thanks 👍
Happy Diwali have good year ahead
Awesome Video .... Thank you :)
This thing does almost everything, but what would you use for object avoidance in a drone ?
Thanks for mentioning HuskyLens 6000 times 😌
Nice video. Thanks. Can I use the Husky for detecting/identify plants or maybe rodant holes in the ground.
Like was clicked at 5.second .Amazing video like others.Thank you
vey nice work sir...
dear sir, I new in your classes.
Have you ever thought about creating a controller for a solar system, where there is a temperature probe on the panel and a probe in a tank, where you can give an order to a circulating pump. Or a control of a three-way valve, where there is a hot water inlet and a return of cold water, where a water temperature is set on the way. The valve control is done by a 24V AC servo motor or other type.
Cool video, keep it up, thank you :)
Is there an alternative to this device that doesn't include the camera interface? I'd like to feed it images/frames programmatically but I really like the features of the SoC in this board.
Right ordered my HuskyLens, it's gonna take like 6ish weeks from China but it's on the way :)
Another first rate instructional video.
This is a cool one.😁👍
'blank function' is known as a 'prototype', typically placed in a header (.h) file.
could you use this to detect movement of metal? it must detect movement, not just presence, but the difference between present and moving, and present and stationary.
Hey, can u do a video on how to get started with iot, electronics and stuff!
Do you think I could use this to monitor selected gear on a dashboard? I'd be looking at picking out P N R D - but of course on some vehicles, they show only the current gear, some others will always show P R N D but with a highlight on the current gear (either light up, coloured, bolded, border drawn round). It seems like this would be a quick route to the ability to do this, but I do wonder if the problem will be lack of distinction between different gears i.e. P chaning to N withouth any other changes or change of view point. I wonder if the "tag" function would work for this.
Question: Can you connect a higher resolution camera to it? Is its source code open source?
I've been working on replicating the Kiva robots that Amazon uses. I saw this and was thinking it was what they used to follow the lines on the floor and read April tags on the ground and under the shelves. I guess this lens can only do one at a time. Do you think they use 3 of them or something else?
could you do a nrf24 vs hc12 comparison .
I had bought a Pixy Cam. The Husky Cam looks improved but I'm not ready to buy. I have 4 cats so I'm interested in cat recognition. Is this function planned for a future release?
Just purchased one.. waiting on it to be delivered. , I noticed you showed baud rates for serial speeds, but when in I2C connection, you didn't mention ( or I missed it? ) I did notice I2C was set to 115200 in the code? ( cuz you said 9600 should be used in serial? ) But can you set the baud rate in I2C? or is leaving it all in AUTO mode to let it decide how fast to go , a better idea?
Do you think the HuskyLens is sensitive enough to track a star or planet? Trying to find a way to have Arduino track stars for my telescope.
Can the HuskyLens somehow store information about the image/object it detected and then bring up notification stating the image is sees now it saw before?
is this better than the Pixy2?
Ca you do rubik solve from this?
this would be an interesting Click BLE PLC automationDirect industrial or home budget project vision addon... 👍👍👍 thanks a lot.. 🤔
Amazing video as always. The price on this thing is insane, I was expecting it to cost hundreds of dollars.
Can we know how much it costs? I don't want to watch an entire video to find out.
@@ziad_jkhan $60-ish us.
@@mr.grieves6662 Not bad but, at this price point, it's probably better to wait for OAK-D Lite coming up at $85 in April with much better specs
Well, I just checked the price and it seems like this was the early bird price which is no longer available so it looks like it's at $125 now :/
Can you make your own computer vision software using HuskyLens that will start upon boot, without the user needing to press anything?
Can the camera be extended away from the board with a longer ribbon cable?
How to Track an Object in the Shape of an Ellipse using Computer Vision (CV) ?
How many different face/object that Huskylens can save?
Just a correction: Object regognition is not same as tracking. The word you are looking for is detecting and recognizing.
So squirrel is not on the list of objects in the Object Recognition mode. I know I can't add new objects but can I use the Object Classification mode instead? If yes, will it work if I press and hold the learn button, then flip through a bunch of photos on the computer screen? How long can I hold the learn button for? Thanks!
Long time you released a video.
HuskyLens is an AI Machine Vision sensor based upon the powerful Kendryte K210 processor. Today we will take a look at this amazing device, both by itself and with an Arduino.
Do you know of any security issues using this? Meaning will it try to transmit data to a server? Granted if I used it, I wpuld have it on an offline network, but a clever "malware" could prob find a way out of a VLAN.
Also have you ever built something like this from "scratch"? Like using dedicated IP cameras on a network that do this sort of AI on a PC? I have a pretty powerful home server PC with multible half decent(for lower end AI) GPUs. I was wanting to set up a VM using those GPUs to do a home AI server. These would be awesome, but for the amount of cameras I would like(around the property, chicken coupes, etc) that would get expensive and seems superfluous. A video on that would be awesome! Unless you have one and I missed it. I've only been watching for maybe a year when I can sit and pay attention. You are an excellent teacher BTW! I have learned a lot from you and gotten MANY ideas! I can't wait to have to time to give them a try! Hope you have a safe and happy new year!
I would love to create something to track pool balls on a table. If it could track each players run of balls in each inning that would be huge. How hard would it be to create something like that and where could I find someone to help create something like this?
ALso, If you mentioned it, I didn't notice.. but.. IS there anyway to tell which version you have? 2M or 5M camera? ( without the box anymore) .. cuz didn't realize it came in two different type cameras when I purchased it??
Can the screen be removed and pointed to face away from the camera?
can we change the mode over arduino code. Example: i choose the face recognition then i want to change over an arduino code into the object recognition, is it possible ?
Please do something like this for the Nicla Vision
Hi Sir, may i know which program include the modify name?
hello, i just bought this item, and i have being trying to recognise texts on a lcd screen but they all apear as ID1, is there any way i can make it work?
What is your os on the computer?
Pretty sure the ESP32 Cam can do a good bit of what the HuskyLens can, at not only a much cheaper price, but the ESP32 Cam is a POWERFUL MCU! But this is MUCH easier to use, Has an IPS Display, and best of all it's a DFRobot Product, so u kno the product support will be BEYOND outstanding with Incredible build quality and the circuit design will be flawless (U have no idea how many shields, breakouts, and even consumer grade computers and electronics I've seen in my life with flawed circuit designs)!!!
I was thinking about the PixyCam, is that device too old?? Or is it still somewhat useful??
I have a PixyCam2. This device does a bit more than the PixyCam, so might be a bit more useful. In the end I only ever used PixyCam in some novelty Lego projects - I never found any real world uses.
@@martinwhitaker5096 I have a Pixycam too, I think the same, thanks!
hot dog/not a hot dog.
What are applications for HuskyLens? I cant think of any tbh
The dfrobot webpage says that huskylens has gesture recognition, but I don´t see it in your video. Did you have the chance to see if it's true?
@ DroneBot Workshop
How does does the HuskyLens compare to the Pixy2 Camera? (besides to the LCD screen of course)
I'm particularly interested in object tracking (of new objects I select) & line following
Geez this thing sounds a lot like the PixyCam (CMUCam) on steroids!! Obviously didn't have the object recognition or display, but it did do colors, it did have a training mode, LEDs, object tracking, etc... I wonder if some of the same people worked on it???
This camera is better than pixy2. Change my mind
Great video! Would love to see this compared to Luxonis OAK (OpenCV AI Kit) line of products and their upcoming OAK Lite that had a successful Kickstarter campaign.
Me two, got the OAK-D Wifi and a Realsense aswell. Thanks for the vid, it was straight forward as always.
It costs $50 for non-pro version. I don't know what other people's criterion for inexpensiveness, but to me, it does not seem inexpensive.
I think it's about right,
For a person who works on it/needs it etc. $50 isn't gonna be expensive, for a person whose messing around with it, it would be expensive.
It's like Arduino Vs raspberry pi
in UK, A Uno could cost £8 or so while the Pi would cost around £60
Both compute but do different things
Dude did a video before about a microcontroller that had a camera and low level AI and cost I think about £20 but did nothing compared to this
Edit: it was the ESP32 which is $10 and with some programming recognised people
👍
But how much does it cost??? I'll only watch if it's worth it.
Is this open source
Why is this so limited? A device like this should have an sdk allowing you to develop your own algorithms for it. I don't know whether that is possible with this device, but at least you did not mention it. As it is now, it is more of a gimmick than an actually powerful and versatile tool. Ex. the hardware should be perfect for running an algorithm to detect 3d printing failures (read: spaghetti detective), but that would require putting your own neural network on it. Something like that would make it worth the price (which is not so super-low if you compare it to a Pi Zero 2 + pi-camera), but what use is detecting car or bikes or faces or lines? The algorithms for that aren't so complex that a Pi Zero 2 wouldn't be able to do it - probably at a lower framerate, but what use is high-framerate if there's no usecase needing it? So really, this thing needs an SDK instead of the standard firmware with this limited functionality.
I got it...
I'm very new in arduino
🌹🌟🌹
first
doesn't really matter but am still gonna say
Hello Sir, I need help with something, I will pay. Thanks in advance
another great video with excellent explanations !
54€ is not a lot for all the functionnality.
don't say "colored objects", it's better to say objects of color ^^
Hanzhen harmonic gear , robot joint gear over 30 years experience