There is a new shield coming with the T239. They've had it ready for a long time. Just waiting on Nintendo to announce the Switch 2 in January since they use the same silicon.
No stats shown in the video, sadly, but that's a quantized 3b model down to 2GB so with ~60GB/sec of memory bandwidth you could hypothetically get up to around 30 tokens per second with it. I'd eyeball that generation at about 20 tok/sec which is pretty good given the unoptimized setup out of the box. At the price this seems like a good deal if you can tolerate being limited to quantized 8bn models and lower, albeit with quite small context. You can get a lot out of sub 8bn models now, it's just been shown Llama 3.1 8bn (a bit old now) can outperform the 70bn model in certain benchmarks with chain of thought approaches (included a link but YT hated it..) so this is certainly a viable unit for low level experimentation like in CS workshops or amateur researchers. I'd buy one.
can you explain what good models are like and which one are good and what kind of hardware should one be running? I see Orin has up to 64GB of VRAM but the bus rate remains the same, so I'm not sure if it is bad idea to max out that device or just go with some other device with higher bandwidth output.
The Switch 2 will feature 8 cores instead of 6, and its GPU will have 1536 shaders instead of 1024. Considering the Switch 1's GPU is about 350-400% weaker yet still manages to run games like Zelda: Breath of the Wild at 960p30fps and Mario Odyssey at 1080p to 960p at 50-60fps, the Switch 2 will be a significantly powerful piece of hardware for Nintendo.
@ no I wasn’t trying to say this in a negative way, it’s just funny to me that this is what Orin is mainly used for but its power efficiency is what makes it a viable option for a handheld. Yeah I know the Switch 2’s Tegra APU will have much better specs than this chip. It’s rumored to have a 4 TFLOP GPU which is a huge upgrade over the 0.5 one on the OG Switch. DLSS and RT will be a game changer for that thing as well. I very much look forward to getting it when it launches!
@@RAUlCODEXD they're basically the same others like Apple used imperfect chipset for their own budget products like A17 pro inside Ipad mini only has 5 GPU core .
LLMs run purely from memory (unless they need to swap to disk, which is not something you want to do in any case, not just with LLMs). So what ETA was describing when he said the first time was slow was the initial load of the model into RAM. Once a model is loaded, it will respond as fast as the memory and processor can spit out the data. Adding an NVMe drive may speed up this initial load time by a small amount, but in general it's not going to make it better at LLMs.
Hide what? Nintendo fan bois will still buy whatever is in the next generation Switch. That is the problem, unless it's completely argreeous you'll buy it. Flipping Reto Gaming is driven by Nintendo fan bois.
@@SmartphoneWars this is NOT the switch 2 hardware. This is running on MAXWELL architecture which is what the first Nintendo switch uses inside its system. The switch 2 is rumored to be using amprere tech inside its system which is much better and supports ray tracing.
Can you get bazzite to work on arm chips? That would be cool to see. I know it would probably work horribly but those specs lead me to believe it might work.
i know, i have 4gb orin nano on my desk. But it still costs almost 400$, so more than the new 8GB board. I want to know if we are getting orin nano 4GB super that will cost around or less than 200$
It would be kind of interesting to actually see games running on it for android I wonder if there’s a way to maybe get android working on the jet and try running some performance test on it
Could you use this to run a Jellyfin server? I use my main PC for mine but I feel like it's a waste of power, can you serve 4k hdr video files from this?
It's weird to hear you talk this positively about AI. Thought you were one of the many people just plainly frustrated by it. But. Yea. Good to see you have fun with it. A good toy to play with, and a good tool for robotics and whatnot. Now, if only the slop would stop coming in from every place. AI in some places is good, AI in everything is just horrible.
Zo its just a stock jetson nano overclocked to 25w. No new CPU or GPU technology no extra core or ram its just 25w instead of just 17w 😢 i can just overclock the nano i already have and ill get the exact same results!! Why doesn't it have a 16 core CPU and RTX 3060ti class GPU.. there has been plenty of time to incorporate newer technologies but nope so don't rush out to buy this SBC because o-pi5 or r-pi5 with a few coral chips will be much better from the start.
Who knows? If I was to guess, Nintendo have probably been sitting on it. Maybe their one is already in the warehouse until next year, so that they can really ramp units up in production, and also polish up a decent launch lineup.
@@Berserkism That's not true. Both chips belong to the same generation and share the Ampere architecture. A few months ago, information leaked and was confirmed that the Switch 2's chip will be the T239 with Ampere architecture, featuring 1,536 CUDA cores, making it more powerful than the chip used in the development kit shown in the video.
Take a look at the games ported to run on Apple's M-series chips-they're all ARM-based. The GPUs in these chips are just as powerful as those used in Nvidia's ARM chips.
No it's not. Intel's current crop of CPU's are just as power efficient. Software is way behind current hardware capability. Consumer hardware/software cannot support all the cores available with Ampere CPU's
ummmmmmm. 1. he made a video with details 2. why not research something for a few mins that you're unaware of? 3. NVIDIA Jetson Orin Nano 4. its purpose is to save the world 5. this device is for polar bears..... and squirrels..... and pokemon
I don't think emulation matters to someone with the money to buy a non cheap mini pc. I imagine those people have a dedicated gaming computer and/or laptop.
meh, another nvidia kit that kills all support for prior units (t1 jetson nano was killed off in jetpack 2, 6 months after release for the t2 it was year) , orin is going the same. you get the device, design software around it then bang it's killed off and the blobs cannot be used in the next version, plus they have thier own kernel port so forget pulling official kernel and just building, you need so many nvidia packages thats it's not worth it. plus the NDA's for anything good, would rather get Quallcom device of the same power + a AI module.
ARM is the superior architecture however xxx86 got first move advantage and it was a solid chess match for white after that due to the fact it was first to do IO
There's nothing really AI about it tbh. 8GB? Can hardly run the cruddiest of image gen models and LLMs. It's fine to put in some robot I guess? But still; the issue is memory, NVIDIA knows it, that's why they refuse to put AI-capable memory in consumer products.
I love seeing my hobbies converging! Didn't think I'd see retro gaming and local LLMs in one video!
They should upgrade their Shield TV lineup based on this SoC. It's long overdue at this moment.
I was wondering when they'll finally update that thing. I'm guessing we'll find out once the Nintendo Switch successor comes out
@@Joe20110530 that would be amazing, but I doubt we will see another one from them unless they can market it with Ai
Shield is overpriced outdated
There is a new shield coming with the T239. They've had it ready for a long time. Just waiting on Nintendo to announce the Switch 2 in January since they use the same silicon.
I second this is would buy
pretty chill Dragon, "you want to transform me into a robot, hrm sure why not"
Don't know why i need to laugh so hard about this.
Nvidia should make arm chip for handheld
This is most likely a switch soc (or at least very similar)
@@drakechapman7476 According to the Switch 2 SoC leak that happened like a year ago or so, this is the same except has a few hundred less CUDA cores.
He does, Nvidia produces the Nintendo switch brain, and probably their successor.
They actually have them. They're called NVIDIA Tegra used in the Nintendo Switch I believe and in some older tablets.
They do
Thank you. Looking forward to more videos about this board.
Good to see some emulation being done by ETA Prime. This tiny unit looks great.
8:45 Duuuube I love Subarus! Good choice.
@5:49 theres like a cold start when starting/launching any models as they are being loaded to vram/ram
I Knew you would be making a video, I checked and saw a video of the old one, now it's here 🙂
I bet it looks crispy in the dark.
Now if someone can shrink this to the size of the CM4 form factor that’d be nice
No stats shown in the video, sadly, but that's a quantized 3b model down to 2GB so with ~60GB/sec of memory bandwidth you could hypothetically get up to around 30 tokens per second with it. I'd eyeball that generation at about 20 tok/sec which is pretty good given the unoptimized setup out of the box. At the price this seems like a good deal if you can tolerate being limited to quantized 8bn models and lower, albeit with quite small context. You can get a lot out of sub 8bn models now, it's just been shown Llama 3.1 8bn (a bit old now) can outperform the 70bn model in certain benchmarks with chain of thought approaches (included a link but YT hated it..) so this is certainly a viable unit for low level experimentation like in CS workshops or amateur researchers. I'd buy one.
How much RAM do I need for those reduced models?
can you explain what good models are like and which one are good and what kind of hardware should one be running? I see Orin has up to 64GB of VRAM but the bus rate remains the same, so I'm not sure if it is bad idea to max out that device or just go with some other device with higher bandwidth output.
Why would i use this instead of a normal pc? Maybe unless it's used in a robot like advertised where the small formfactor has an advantage
Crazy the Switch 2 will be based off of one of these😂
The Switch 2 will feature 8 cores instead of 6, and its GPU will have 1536 shaders instead of 1024. Considering the Switch 1's GPU is about 350-400% weaker yet still manages to run games like Zelda: Breath of the Wild at 960p30fps and Mario Odyssey at 1080p to 960p at 50-60fps, the Switch 2 will be a significantly powerful piece of hardware for Nintendo.
@ no I wasn’t trying to say this in a negative way, it’s just funny to me that this is what Orin is mainly used for but its power efficiency is what makes it a viable option for a handheld. Yeah I know the Switch 2’s Tegra APU will have much better specs than this chip. It’s rumored to have a 4 TFLOP GPU which is a huge upgrade over the 0.5 one on the OG Switch. DLSS and RT will be a game changer for that thing as well. I very much look forward to getting it when it launches!
😮
@@RAUlCODEXD they're basically the same others like Apple used imperfect chipset for their own budget products like A17 pro inside Ipad mini only has 5 GPU core .
LLMs run purely from memory (unless they need to swap to disk, which is not something you want to do in any case, not just with LLMs). So what ETA was describing when he said the first time was slow was the initial load of the model into RAM. Once a model is loaded, it will respond as fast as the memory and processor can spit out the data.
Adding an NVMe drive may speed up this initial load time by a small amount, but in general it's not going to make it better at LLMs.
I can't wait to get
my nails done today.
you should try the jetson that runs 270tops and run windows arm on it
Jetsin Orin made me think he is Miles Dyson: the man behind Skynet.
I so want that nvidia heatsink and cooler!
Thanks for review !
Need proton on arm devices asap
I cant wait for Nvidia ARM CPUs for Windows PC. I think this tiny board is really snappy.
Just ordered 4 of these to play with on the side.
How was the difference in latency if you run with a M2 SSD? A SD-card is not very fast... Maybe something to test in a followup video?
USL cards are very fast.
Thanks for reviewing the Switch 2 early 😅
I smell a Shield TV 2025 model based on this 😍
That is the switch 2 hardware! 7-15W tdp even. Man, Nvidia can't hide anything. These must be slightly higher clocked units (due to the fan).
Hide what? Nintendo fan bois will still buy whatever is in the next generation Switch. That is the problem, unless it's completely argreeous you'll buy it. Flipping Reto Gaming is driven by Nintendo fan bois.
@@SmartphoneWars this is NOT the switch 2 hardware. This is running on MAXWELL architecture which is what the first Nintendo switch uses inside its system. The switch 2 is rumored to be using amprere tech inside its system which is much better and supports ray tracing.
@@Celestial-City 02:00 It's Ampere!
😮
@@Celestial-Citythis is not maxwell
"In this video"?? I shall correct it.
"Hey everyone it's ETA Prime here" ahh much better.
finnaly a jetson ano again
Can you get bazzite to work on arm chips? That would be cool to see. I know it would probably work horribly but those specs lead me to believe it might work.
Why didn't you get the Jetson AGX Orin Variant? Has more power.
Can you run an android image on this? I would love to see some android gameplay on this.
Passively Cooled RTX 3050 in a mini-pc would surely be higher spec than this right? Maybe even cheaper...
And I know it's 6GB VRAM but the main RAM could be higher
How does it compare to mac mini base model in terms of llm speed?
@ETAPRIME can you ask your contacts at nvidia if they will offer 4GB version, like they did with non-supper orin nano?
They have a SKU with less memory
i know, i have 4gb orin nano on my desk. But it still costs almost 400$, so more than the new 8GB board. I want to know if we are getting orin nano 4GB super that will cost around or less than 200$
Did you have to install the OS or is the one you are showing came with the unit?
Get some POP OS on that puppy for steam support
It would be kind of interesting to actually see games running on it for android I wonder if there’s a way to maybe get android working on the jet and try running some performance test on it
Interesting how you used AI voice over models to narrate these. What software did you use?
Could you use this to run a Jellyfin server? I use my main PC for mine but I feel like it's a waste of power, can you serve 4k hdr video files from this?
guys, it's 5:09 am and I can't sleep
3B tokens, I think my watch could run that LLM
@@griftgfx no it couldn't unless you have the 007 watch
Possibly the Switch 2 Specs
Can we install Retropie or something similar on this?
can you use a egpu ? occulink ?
What are the odds the next Nintendo Switch is built on these?
What are the use cases for this? I can see it being useful as a super-capable Arduino or Rasberry Pi replacement
6:02 it's loading the model into the DDR5
the dev kit is 250$ but how much is the compute board itself in bulk?
It's weird to hear you talk this positively about AI. Thought you were one of the many people just plainly frustrated by it. But. Yea. Good to see you have fun with it. A good toy to play with, and a good tool for robotics and whatnot. Now, if only the slop would stop coming in from every place. AI in some places is good, AI in everything is just horrible.
Also. PS3 is amazing. For that size and that TDP. So good.
Zo its just a stock jetson nano overclocked to 25w. No new CPU or GPU technology no extra core or ram its just 25w instead of just 17w 😢 i can just overclock the nano i already have and ill get the exact same results!! Why doesn't it have a 16 core CPU and RTX 3060ti class GPU.. there has been plenty of time to incorporate newer technologies but nope so don't rush out to buy this SBC because o-pi5 or r-pi5 with a few coral chips will be much better from the start.
Still no redmagic 10 pro full review showing off winlator, and still no links to working drivers for snapdragon 8 elite
Your point is?
So that's gonna be the Switch 2?
You are running a 3B parameter model right.. Can it run a 12B model?
it only got 8GB of RAM, 12B model need more than 8GB of RAM
although you can using SWAP memory to load the model
Is the switch 2 gonna be based on this hardware?
is this the switch 2?
Nintendo is using an older and much less sophisticated model. Cheaping out as usual.
Who knows? If I was to guess, Nintendo have probably been sitting on it. Maybe their one is already in the warehouse until next year, so that they can really ramp units up in production, and also polish up a decent launch lineup.
@@Berserkism That's not true. Both chips belong to the same generation and share the Ampere architecture. A few months ago, information leaked and was confirmed that the Switch 2's chip will be the T239 with Ampere architecture, featuring 1,536 CUDA cores, making it more powerful than the chip used in the development kit shown in the video.
Yes
I now wonder if it's possible to run SteamOS or Windows on this...ARM is sadly still undercooked in terms of software, but there's great potential.
How does Windows on ARM run on this vs Snapdragon?
It would be nice if Nvidia released an Arm Laptop/Tablet with this like Samsung’s 14 inch Tablet
Problem with Windows on ARM is its under developed.
@@AnthonyThomas-o4l They have it for Snapdragon. Nvidia has the libraries they need
😮
Does the m.2 support Gen 4? Or just Gen 3?
Can Windows 11 run on it now that it supports ARM? curious to see gaming performance in current games, would this be better than AMD APU?
Take a look at the games ported to run on Apple's M-series chips-they're all ARM-based. The GPUs in these chips are just as powerful as those used in Nvidia's ARM chips.
With 2.4 Tflops GPU ?? 😂
Is there Box86, so you could test Steam?
Can it run stable diffusion too?
Arm is the future
No it's not. Intel's current crop of CPU's are just as power efficient. Software is way behind current hardware capability. Consumer hardware/software cannot support all the cores available with Ampere CPU's
@AnthonyThomas-o4l we'll see
Can it run KODI with lossless audio out?
Lets see some native code execution on the emulator that shall not be named!
$249
Can you use 125% or + display scaling on your videos? I cannot see your content I am vision impaired.
I dont really understand what this is but i am interested, someone know a road map or some entrypoints to look at ?
Nvidia handhelds coming soon
what kind of device is this? what's it called? what's its purpose? who is it for? asking this 'cause I'm interested.
ummmmmmm.
1. he made a video with details
2. why not research something for a few mins that you're unaware of?
3. NVIDIA Jetson Orin Nano
4. its purpose is to save the world
5. this device is for polar bears..... and squirrels..... and pokemon
@@Kevin-vh6nn I meant it generally, dum dum
@@stourn2399
hang in there cupcake
@@stourn2399
go to your room.... close the door.... sit on your little hands, dont you come outside until i feel you're ready for life....
@@stourn2399
they put warning labels on shampoo, bc of you
hang in there kid
can you install Steam OS on this one? and try some low end stuff?
I don’t think SteamOS runs on ARM yet… (there are rumors that theyre gonna support it in the future though!)
That's a weird Switch.
Insane
OG Jetson Nano was $99
I don't think emulation matters to someone with the money to buy a non cheap mini pc. I imagine those people have a dedicated gaming computer and/or laptop.
Can you install Android OS on this?
Could this be the hardware Nintendo Switch 2 is based of?
No it's not what the Switch 2 is based off of
Orin nx is for switch 2
Run all other Jetsons site by site and compare !!!
raspberry pi on steroids
where we can buy that?
in the description,
250$ website called sparkfun
meh, another nvidia kit that kills all support for prior units (t1 jetson nano was killed off in jetpack 2, 6 months after release for the t2 it was year) , orin is going the same. you get the device, design software around it then bang it's killed off and the blobs cannot be used in the next version, plus they have thier own kernel port so forget pulling official kernel and just building, you need so many nvidia packages thats it's not worth it. plus the NDA's for anything good, would rather get Quallcom device of the same power + a AI module.
Were you doing something so important that you still need support? I don't get it.
Llms are more practical and actually useful compared to video games.
T239 baby🎉
Switch 2?
@@orlandoalessandrini2505 Exactly.
@@whipmanx3 good times
This is the same chip?
@@yashveerthakur7265 its based on this chip. So it'll be similar in performance.
switch 2?
Pretty neat
Definitely would be Switch 2 🥹
Someone's gotta ask... Will it run Crysis???
8gb is tough tho
for AI shit nvidia lowers their prices but regular gpus they just double it
AI is more important than playing games.
@ no
Already on back order. 😔
nvidia pc nvidia pc
As always, nVidia can't put more than 8gb ram in anything that cost less than a fortune...
p[i5 seems a lot better nvidia is just spewing out garbage now.
ARM is the superior architecture however xxx86 got first move advantage and it was a solid chess match for white after that due to the fact it was first to do IO
❤
>8GB RAM
>AI
This is useless.
Leatherjacket come back if give us 64GB with 1tb/s bandwidth otherwise 2x RTX 3090 are better.
test switch
switch 2 cpu
😂 Waist Of Money
Hip of money?
@@Clownworld74 Leg of money?
Waist Of Money
I think its the knee of money
It’s wrist of money instead
Waste Of Monkey
I think its chest of money
No it's a handful of money
There's nothing really AI about it tbh. 8GB? Can hardly run the cruddiest of image gen models and LLMs. It's fine to put in some robot I guess? But still; the issue is memory, NVIDIA knows it, that's why they refuse to put AI-capable memory in consumer products.
They have a 16GB SKU. To do any decent AI development this serves just fine and keeps other hardware from having to do it.
😶