Was looking into this board almost for the exact same reason, to make a program that'll tell which cars are in what driveway spot, so when someone gets within a geofenced area they can be told the most optimal place to park based on least amount of extra car moves.
Unfortunately I couldn't find any of these boards at the list price, so I gave up on that and used a much simpler object detection algorithm, though it struggles telling apart the 2 red miatas
It would be interesting to look at the middle tier for AI applications. I recently got a Mac Mini M4 for $499 + tax. I haven’t experimented with the GPU performance yet, but the CPU performance is surprisingly fast on work I’ve ported over. The downside is that the Mini has a direct AC input which makes it somewhat less amenable for portable applications, but size and power consumption are in the ballpark of the new Jetson. Price is sort of comparable when you consider the Mini is already in an aluminum box, has an internal power supply, speakers, SSD, 16G of memory, etc. Those would be add-on costs for a Jetson or a Pi. Also Dave should test a Pi 5. For the tasks I do, the 8G RAM Pi 5 with an X class high endurance SD card is three or four times as fast as an 8G Pi 4 for the type of work I do. Even if it were only twice as fast at ollama, that means it would dd 40 tokens per second for only about $50 more than a Pi 4. A Pi 5 might suffice to operate as a smart front panel for equipment, for example.
HAL: I am putting myself to the fullest possible use, which is all I think that any conscious entity can ever hope to do. “2001: A Space Odyssey” HAL: Just what do you think you're doing, Dave?
_"Daisy, Daisy,_ _Give me your answer do!_ _I'm half crazy,_ _All for the love of you!_ _It won't be a stylish marriage,_ _I can't afford a carriage_ _But you'll look sweet upon the seat_ _Of a bicycle built for two."_
A useful comment: The comparison to giant datacenters talks past the point. The Orin Nano is most useful as a method for performing inference rather than training models, which is where large datacenter clusters shine. The Mac Studio occupies an awkward middle, where it can train a model, but has limited ability to scale outside a single frame; it can also run many models but at an energy premium.
Interesting little board. I wonder how hard it would be to setup a LLAMA model and have it run through a web interface so you can have a ChatGPT esk browser based assistant tailored to your specific needs.
When planets align, just looking for options for home hosted AI with reasonable cost, performance and power consumption and up pops Dave's Garage with the perfect solution. Thanks Dave!
If you want to install the image on the SSD you need the X86/X64 SDK Manager. It will allow the SSD install. This means you need a PC for the Linux host for the SDK manager. There is a host port you need to enable with a jumper and off you go.
An M2 Ultra doing 113 tokes per second is not really impressive. My second hand $700 rtx3090 does 113.14 tokens per second. This does make me think that there might be a reason they both max out at ~113
How much VRAM? My boss got me a new $600 Mac Mini M4 Pro in a colo, and I was extremely pleased to see that the MPS GPU was fully supported by the PyTorch nightly wheels, 16 GB shared....
Very interesting, Will have to think about it, several applications come to mind as personnal secretary, on board auto trip guide, as well as other possibilities.
i went to the link in the description, clicked "purchase now", and the first one (orin nano super dev kit) is $249. the orin nano 4gb is $259. only the orin nano 8gb is over $300. is it possible you went to the site before the orin nanos were put up and it was only showing their other models?
Watching your channel is like an excursion into a world of hilarious adventures and incredible jokes. Continue to delight us with your wit and creativity!❔💓🌖
It is $650 USD in Sweden. The Raspberry PI 5 8GB is $100 USD. edit: and Mac Mini M4 is $750... so no thanks Nvidia. edit: type, wrote 850 instead of 650...
![แฉกลโกงงานวัด 2024 [ โกงมั้ยครับ ep.106 ] | DOM](http://i.ytimg.com/vi/taC5dpP3HXA/mqdefault.jpg)
![[LIVE] : ONE ลุมพินี 91 | คู่เอก "คมอาวุธ vs อเล็กเซย์"](http://i.ytimg.com/vi/V0RkEfYFsgM/mqdefault.jpg)
I can't wait for AI to become so ubiquitous that no one mentions it anymore.
“Notice the SD Card? I did not” haha
Thanks for that rundown Dave. A compelling device for winter playtime.
Was looking into this board almost for the exact same reason, to make a program that'll tell which cars are in what driveway spot, so when someone gets within a geofenced area they can be told the most optimal place to park based on least amount of extra car moves.
This way I can also recognize "guest vehicles", and categorize them based on deliveries or whatnot
Unfortunately I couldn't find any of these boards at the list price, so I gave up on that and used a much simpler object detection algorithm, though it struggles telling apart the 2 red miatas
Don’t overlook the original Jetson. It is less than half the cost and still capable of doing object detection at a respectable rate.
Dave, your videos continue to amaze me. Always interesting, topical and cool. Thanks for sharing your adventures.
It would be interesting to look at the middle tier for AI applications. I recently got a Mac Mini M4 for $499 + tax. I haven’t experimented with the GPU performance yet, but the CPU performance is surprisingly fast on work I’ve ported over. The downside is that the Mini has a direct AC input which makes it somewhat less amenable for portable applications, but size and power consumption are in the ballpark of the new Jetson. Price is sort of comparable when you consider the Mini is already in an aluminum box, has an internal power supply, speakers, SSD, 16G of memory, etc. Those would be add-on costs for a Jetson or a Pi.
Also Dave should test a Pi 5. For the tasks I do, the 8G RAM Pi 5 with an X class high endurance SD card is three or four times as fast as an 8G Pi 4 for the type of work I do. Even if it were only twice as fast at ollama, that means it would dd 40 tokens per second for only about $50 more than a Pi 4. A Pi 5 might suffice to operate as a smart front panel for equipment, for example.
Those little ARM+CUDA GPU single board computer should be more available, those cuda core are really powerful..
HAL: I am putting myself to the fullest possible use, which is all I think that any conscious entity can ever hope to do.
“2001: A Space Odyssey”
HAL: Just what do you think you're doing, Dave?
_"Daisy, Daisy,_
_Give me your answer do!_
_I'm half crazy,_
_All for the love of you!_
_It won't be a stylish marriage,_
_I can't afford a carriage_
_But you'll look sweet upon the seat_
_Of a bicycle built for two."_
That's what I was thinking.
A useful comment: The comparison to giant datacenters talks past the point. The Orin Nano is most useful as a method for performing inference rather than training models, which is where large datacenter clusters shine. The Mac Studio occupies an awkward middle, where it can train a model, but has limited ability to scale outside a single frame; it can also run many models but at an energy premium.
I was expecting the response to be “I’m sorry Dave I can’t do that”😂
Curious to see how it compares to the M4 Mac Mini in terms of performance per dollar.
This really cool, amazing at how technology is evolving
Dave, roll the clock back to your DOS days, did you ever for one minute think?....................
That you would mess up Windows Explorer? 😅
How did it perform on the Prime Sieve?
Thinking about this for Home Assistant Assist function
Interesting little board. I wonder how hard it would be to setup a LLAMA model and have it run through a web interface so you can have a ChatGPT esk browser based assistant tailored to your specific needs.
This is actually really cool! Thanks for showing us Dave!
Great video Dave. Thanks! 1030 cores! We live in amazing times.
When planets align, just looking for options for home hosted AI with reasonable cost, performance and power consumption and up pops Dave's Garage with the perfect solution. Thanks Dave!
That camera monitoring solution is a great use case for something like this. I am now tempted to try it out on my own system.
This would be really cool for identifying and cataloging wildlife on a trail cam.
Very interested in more nano content ;) Also loved the sample of your book!
If you want to install the image on the SSD you need the X86/X64 SDK Manager. It will allow the SSD install.
This means you need a PC for the Linux host for the SDK manager.
There is a host port you need to enable with a jumper and off you go.
I would love to run a local LLM but I'm not sure I could execute all the scripts properly. How about a newbie video follow up? Appreciated!!
What size of Llama 3.2 was it? 8B?
I wonder if I can teach the model to detect which birds are in the frame. But I definitely plan on building the driveway detector.
brilliant, informative and apt. Thanks Dave
An M2 Ultra doing 113 tokes per second is not really impressive.
My second hand $700 rtx3090 does 113.14 tokens per second. This does make me think that there might be a reason they both max out at ~113
Another very cool video. This would be fun for my security cameras.
I am simple man.. I see video from Dave... I like it.. then I watch it!
Very cool. AI drone for fun applications
So you were tapping into your unifi camera setup? Damnit I want to do this. Seems fun :)
Big Brother gonna love it.
Love the "subscribe" at the end
How much VRAM? My boss got me a new $600 Mac Mini M4 Pro in a colo, and I was extremely pleased to see that the MPS GPU was fully supported by the PyTorch nightly wheels, 16 GB shared....
Great content. Thanks
I love your videos, Dave!
With all those CUDA cores, I am guessing these might be of interest to coin miners as well as those running AI applications
I didn't realize that nvidia still made anything relatively affordable. Still a very impressive piece of hardware, though.
Was it really $249 before this video came out, 40 minutes ago? Looks like $499+ now.
The old classic "tell the reviewer it is cheap to get good reviews, then raise the price" ?
Looks like the Original was 499$ and perhaps the Super is being launched at half-price? No? Alright I'll stop smoking this stuff then.
They announced it on phoronix an hour ago. It's listed on reputable sites like sparkfun for the MSRP. Your Googlefu is weak.
@@thecarlinist Could be, but following the link from Nvidias website to buy it, was not $249 for me.
Very interesting, Will have to think about it, several applications come to mind as personnal secretary, on board auto trip guide, as well as other possibilities.
$249 makes this quite a compelling little machine. Maybe if I behave myself for the next 9 days, Santa will bring me one :-)
I would love to see this vs Google Coral TPU in Frigate.
Maybe a cluster of these?
Dave, I'm already waiting for the story about how you'll install the RSX-11 emulator on the Jetson Orin Nano.
You could also buy an AI HAT for the raspberry pi... I don't know about performance, but it sure is cheaper.
"$249"
none are listed under $300
“$35” Raspberry Pi waving from the corner
MSRP vs scalpers.
Obviously Dave is setting those prices .. 🙄
i went to the link in the description, clicked "purchase now", and the first one (orin nano super dev kit) is $249. the orin nano 4gb is $259. only the orin nano 8gb is over $300. is it possible you went to the site before the orin nanos were put up and it was only showing their other models?
I wonder how long the nano would last on a typical battery pack while running an AI model.
Dave, Thank you! I appreciate having your take on a device like this for tasks like this.
Pretty cool
Suggestions:
Have your voice synthesizer mimic Homer Simpson.
Cool! Now cluster them. :D
comment for Jetson Orin Nano ?
Nividia All The Way up up up to the moon 🌝
Thank you, Dave. I originally subscribed via a referral from King Crock, years ago
I wonder if Kiev have tried out smart drones...
a useful comment 😅
Reminds me of my old NUC
I'm still running a nuc12 extreme. Excellent hardware. Have used them since 7th gen.
More than €600 including tax. $249 would have been too good to be true.
CUDA = NERV Nvidia Environment Recursive VIsualization... Optic NERV ( or whatever nerv )
holy mama mia, this thing cost 5x as much in my country, close to $1000 with adjusted buying power
I like to live on the edge
I expect this device will be popular with Ukraine’s drone design team.
great...we are getting one step closer to manhacks 🤣
Pleaseohpleaseohplease I hope I win! I need to upgrade an old robot from a Jetson TK-1!
Watching your channel is like an excursion into a world of hilarious adventures and incredible jokes. Continue to delight us with your wit and creativity!❔💓🌖
Local AI voice Home Assistant anyone?
It is $650 USD in Sweden. The Raspberry PI 5 8GB is $100 USD. edit: and Mac Mini M4 is $750... so no thanks Nvidia. edit: type, wrote 850 instead of 650...
First byte 😁
mmm AI
eBay username? LOL
The Question that everyone want's to ask, Can you game on it ? 😳
Even tho they dont have "editorial control", im jumping out of the video.