Can't wait for a decade from now when they're packing 1024 TOPS into "entry-level" devices claiming "you definitely need all this power for current models"
It's nice to know that you could run multiple neural networks on independent NPUs! Like one for facial expressions, another for voice recognition, and another for text-to-speech!
🤗 Accelerate might be able to manage using all the NPUs at once. It's meant for using multiple GPUs for a single inference or training task but it might support NPUs too
This reminds me of the Friends meme where Chandler gets a fancy expensive computer, and when asked what he is going to use it for he just says "Idk, games and stuff". The infra is still fun even if you don't actually need the compute lol.
seriously tho, 26 TOPS on that small thing is impressive by itself. combining a card with a few of those 26ers with something with PCIE-4x16 would make for impressive AI processing in a small package.
I appreciate that there are so many different TPUs/NPUs on the market, I’m just frustrated that we’re all beholden to nVidia when it comes to actually training models and running a lot of things.
@@Level2Jeff I’ve got a used Tesla P40 with a water cooler in addition to another rtx 2060. Now that Intel works with Tensorflow and PyTorch, I’ve seriously considered just getting a 16 gig Arc 770 and paying for cloud computing if I need to train a model that needs more memory.
Oh, man. That's a visually menace pi XD Yesterday, you tried and SPAGHETTIBLY FAILED chaining NPU and now this. lol Thank you for the attempts otherwise I would have tried myself. I think 2 NPUs can easily work with running 1 neural network on each NPU. This kind of configuration can realize many real world applications I have been dreaming of so many years. Thank you again.
Definitely! I think on the Pi 5 at least, that would probably be the ideal number of NPUs. You could stretch it to 4 okay too, but at that point the cost/build could point you to something a bit beefier like Jetson Orin.
That's kind of like the pi supercomputers where they were hooking like 25 50 pi's together and calling it a super computer, but you have to make custom software to support the thing and take advantage of the whole cluster or w/e they call it.
This would be useful to run Frigate AI detection from multiple cameras (e.g. PoE RTSP streams). Even this many devices is less power hungry than a 4080 to run 24x7, and compared to the cost of a good IP camera its not unreasonable.
The trick to using those NPUs in parallel will be building a pipeline across them, where each NPU is supporting a subset of the layers in a model. This is a common technique in both training and inference, though I'm not sure if Tensorflow Lite supports it. A pipeline would allow you to partition the model weights and compute across all of the NPUs, giving you a chance to run larger models then you could do on a single NPU. Your PCIe setup is very low bandwidth, but that is less of a concern here because pipeline parallel is only sending the activations (relatively small input tensors in inference) between the NPUs, not the larger weights. Based on the limited information that I can find about Hailo and their sample hardware (a few of them have many Hailo chips on a single PCIe card) it looks like their software may support this.
I love the crazy/nuts thing that is this set up! And, even if the "Coral Dual TPU" only shows up as one of them, I'll bet that it is still cheaper and faster than the USB version. Now, where was that AM power meter that your Dad has . . . ? ;-)
Thanks for trying AND explaining the problems! Can't wait for someone to write a "Kitchen Sink Aggregator" patch that'll normalize or make modules generically access -- ahh, I just wanted to use kitchen sink in a sentence.
"And I will see you, in the next one!" AAAAAAAAHHHHH WHO SAYS THAT?! WHO SAYS THAAAAT!??!?! It JUST escaped my head! WAIT, Chris Titus Tech does?! I just read the description! _Well then!_ ...I think _this exact one_ is a bit common, is it not?
Hi Jeff, just wanted to let you know that you CAN use multiple hailo cores together, using the VDevice API - it automatically identifies the cores (granted it will only work with hailo chips).
I’m amazed by the things you are doing with the humble RPi. Not as amazed as that fractal north case for the pi5 they are showing off at computex, but still amazed.
It’s always fun to see you pushing the limits on single board computers. I’m sure it helps the hardware makers consider unusual use cases. Jeff, have you seen the stories on the new research into IBD which suggests a genetic link and that certain cancer drugs might be effective? I saw the story on the BBC news site. Obviously I can’t post the link here, but I thought you might be interested.
Yes, I've been following that research-still 5+ years out from being practical but it's promising. The key is to find drugs that have fewer side effects than current TNF-blockers (which have similar mechanisms but target the immune system more broadly).
Being able to run two Halo8's would be a cool project as in theory would be (2x 26, or up to 54 TOPs in theory). Combined with dual Pi cameras, would offer high frame rate stereo depth of field, or other fun video processing. BTW: the Home Assistant monitoring is fascinating. Would be interesting in hearing more details.
As price drops for NPU's people will use more then software will write the software for it. That is what the raspberry pi was designed for learning in a fun way.
How do GPU bit miners support the core splits? Some setups are looking at spreading a single calculation over thousands of cores. Maybe there's a way to port a miner task divider to NPU tasks. But I also wonder if we are still limited to running each TPUs individual Floating Point rating? Or could we run full 32-bit models?
We need cheap PLX chips that take a newer standard bus in uplink and can provide many downlink at lower speed. Also on consumer mainboard it is starting to be annoing. 20 or plus pcie gen 5 lanes where almost all hardware is gen 3 and 4. Lot of throughput lost.
Couldn't agree more. Though those of us needing all that PCIe goodness are a slightly rare breed... and the answer till now is usually buy a big server CPU that gobbles up like 120W idle :D
@@Level2Jeff Even if we are the minority I think that the approcha that apple did with the MAC pro using the PLX and many lanes/slot is something that other HEDT/Workestantion OEM should start to consider. New AM5 EPYC has been annoucned but lot of bandwidth is lost if you just plug a single HBA card on today mainboards x16 slot.
Hey Jeff! I'm still pretty curious about everything "AI". I'm just not sure how to take advantage of any of it right now. I'd love to see a video going over a bunch of different AI projects that these can be used for, either here or on the main channel. Obviously frigate is one, I've also seen some self hosted AI chat bots, though I'm not sure how well any of them would fare on a pi. I know you're the "pi" TH-camr, but I'm also curious about other applications of such accelerators. I wonder if an AI chat bot would work decently well on a 1L PC (or some similar micro x86 system) using something like the Hailo for processing rather than trying to cram a GPU in a small system like that. If you know of a TH-camr who is doing that sort of thing, I'm happy to check them out, just let me know. Keep up all the cool videos. Cheers!
before the WHAT it's compelling to know the WHY. waited for the payoff but I guess I don't get enough why I'd want to use all these TOPs I'm excited about using new tech but using it is the key. The excitement about the build comes after the excitement about the functional power. So I guess I don't get this video too much.
So like, objectively I know that the Hailo isn't really meant for inference of production grade LLMs or anything. ...But like, I still want to see if a person couldn't do something silly like a bespoke MoE architecture with 128M active parameters and still get okay quality and speed.
There a Qualcomm AI accelerator that use Dual M.2 slots, that board could be perfect for it (I've never been able to find the spec for dual m.2, so the spacing might be off), just one of those could get 200TOPS/25W Good luck finding one though...
It's cool to see that working _technically_, but in terms of inference, that's kinda the equivalent of having a 1000 CPU cores running on a single, shared stick of DDR4 memory. Do you hypothetically have enough cores to run a ton of applications at the same time? Yeah. But how much you'll be ably to do with the system will depend on the throughput of your memory. And it's the same with these little compute sticks. They rely on your Pi's memory so while you - on paper - have a lot of TOPS, running any task that operates on a reasonable amount of data or live streaming data will quickly saturate your onboard memory and pcie bus and prevent you from doing anything useful with multiple NPUs
I wonder how hard it would be for a Pi to orchestrate all these TPUs together. Could it offload some orchestration onto another TPU? I kind of want to write some code for this thing to see how it performs with multiple processors.
Can those neural chips be used for something custom other than video recognition? Hi, do you have an idea on how to utilize 2 TPUs of that double coral chips? I want to make similar board for deskyop PC, how to do that?
I wonder how feasible it is to put an abstraction layer in front of the 25 coral TPUs so that the software only sees one big TPU? That's more-or-less what RAID drivers for ZFS and BTRFS do so clusters of 25+ drives appear as one block device. And what CPUs do to make scalar code think it's still being executed in-order as the only process on a single-threaded CPU in a virtual memory space, despite out-of-order execution, superscalar execution, hyperthreading, multithreading, branch prediction, and more, all happening in the background.
I really like your videos they always bring novel information Do you know when it will support running NVMe SSD and Hailo AI Kit simultaneously? (with the NVMe SSD used as the system boot disk)
so Google TPU might be the LPU arranged in certain way? doesn't seems so, because LPU have huge memory chips... well the tops seems matched I wonder where this experiment will go into this is really interesting. Really interesting! indeed!
I had trouble getting my coral to work with codeproject ai long-term. It would work for a few hours but then stop responding. I don't know where the issue was on that. I mostly just gave up on it.
I had a similar issue in my testing, though didn't take too much time to debug it. One time it seemed to lock up the frontend of the Pi, had to force-poweroff!
You don't need to make software for them to all work together just create something in python and back off it. you know what have you using the python multithreading library to trick it into multithreading ?
think you might be able to look over the nanopi boards since most have built in NPU? I know the Nanopi R5S has one. Not that much of one but still there
can this be used for llm's and stable diffusion or is it only useful for video tracking. one day graphics cards will be replaced by AI accelerator cards, and all you'll need is a low power gpu.
Great video, does this also works with Pi4 ? Please also mention in future if the projects also works with Pi4 possible since not everybody has the latest and greatest :-)
Hey, thanks a lot for your videos. I'm wondering something that i'm sure has been adressed but i can't find a definitive answer on this. Does TPUs accelerate local LLM answer generations (Ollama for example) ? Thanks
you are saying hold back while other people are saying the trillion dollar cluster is on the way - what would be nice is to combine a cluster of pi each with 25 top into one like nvidia does with its cards when they are in a cluster
Crap, I need one of those PCIe expansion boards for my NanoPC-T6 to be able to give it dual WiFi plus 5G Cellular and possibly a couple other nifty adapters. Dang it
![[Full] 4 ต่อ 4 Celebrity EP.910 | 22 ก.ย. 67 | one31](http://i.ytimg.com/vi/4TR3MVpOiw8/mqdefault.jpg)
Can't wait for a decade from now when they're packing 1024 TOPS into "entry-level" devices claiming "you definitely need all this power for current models"
1 BOPS?
640 TOPS ought to be good enough for anyone
@@heblushabusT = trillion, so B would be billion. Next step is Pflops for Petaflops
@@tuqe oh, right. bops sounded funny tho. so, POPS?
@@tuqeisn't T tera in this case?
It's nice to know that you could run multiple neural networks on independent NPUs! Like one for facial expressions, another for voice recognition, and another for text-to-speech!
i would love a video on the home assistant power consumption!
+1 Yes, me too
Well that would be a 1 minute video 🤣
Oh thank god, I’ve been itching ever since you showed that b-roll
My favorite part of this video is definitely when the box gets identified as a cell phone and he holds it up to his ear 😅
🤗 Accelerate might be able to manage using all the NPUs at once. It's meant for using multiple GPUs for a single inference or training task but it might support NPUs too
This reminds me of the Friends meme where Chandler gets a fancy expensive computer, and when asked what he is going to use it for he just says "Idk, games and stuff".
The infra is still fun even if you don't actually need the compute lol.
seriously tho, 26 TOPS on that small thing is impressive by itself. combining a card with a few of those 26ers with something with PCIE-4x16 would make for impressive AI processing in a small package.
Yeah, Hailo makes a 'Century' card that does just that-hopefully as multi-NPU becomes more popular, programming for it also becomes more popular!
OMG!!! Who are you going to be from now until the next video?!?! End of an era...
Ha!
That was fast
I was about to say that
My AI predicted it
that's what she said!
@@Level2Jeffu okey? u look a little red :p
Never stop doing what you do Jeff. Love the content, love the experimentation.
I had no idea this channel existed, this is awesome!
and now you do, ha! this is the channel where things get crazy
jeff sometimes is running low on iq because from time to time he post videos about making a fatality on a pc
Jeff!!! The increase in content per week has been amazing. Don’t over due it but man I’m loving it
This is perfect for having multiple models working on different tasks then feeding all there results to the AI that monitors them 😊
For anyone with these dual Google coral chips, if you would like to use both corals you can with adapters. For example the ones for sale on MBS-Shop.
I appreciate that there are so many different TPUs/NPUs on the market, I’m just frustrated that we’re all beholden to nVidia when it comes to actually training models and running a lot of things.
Ditto. Wish at least AMD could offer something that would take the bottom out on either price or efficiency, but right now it is what it is :(
@@Level2Jeff I’ve got a used Tesla P40 with a water cooler in addition to another rtx 2060. Now that Intel works with Tensorflow and PyTorch, I’ve seriously considered just getting a 16 gig Arc 770 and paying for cloud computing if I need to train a model that needs more memory.
"I've created a monster!
No one wants to see Marshall no more,
They want Jeff.
I'm like chopped liver"
Oh, man. That's a visually menace pi XD
Yesterday, you tried and SPAGHETTIBLY FAILED chaining NPU and now this. lol
Thank you for the attempts otherwise I would have tried myself. I think 2 NPUs can easily work with running 1 neural network on each NPU. This kind of configuration can realize many real world applications I have been dreaming of so many years. Thank you again.
Definitely! I think on the Pi 5 at least, that would probably be the ideal number of NPUs. You could stretch it to 4 okay too, but at that point the cost/build could point you to something a bit beefier like Jetson Orin.
Bravo Jeff! That was alot of work on your part. Again, congrats and thanks for all the hard work you do for us!
Extremely helpful video for beginners
That's kind of like the pi supercomputers where they were hooking like 25 50 pi's together and calling it a super computer, but you have to make custom software to support the thing and take advantage of the whole cluster or w/e they call it.
This would be useful to run Frigate AI detection from multiple cameras (e.g. PoE RTSP streams). Even this many devices is less power hungry than a 4080 to run 24x7, and compared to the cost of a good IP camera its not unreasonable.
@Level2Jeff - Power Monitor Dashboard all the things - YES PLEASE
"Why?" well to quote The Doctor from The Waters of Mars, "Fun"
Jeff, I too would love to see a video on how to setup a Home Assistant Dashboard for Power Monitoring. You sir, are a Wizard!
The trick to using those NPUs in parallel will be building a pipeline across them, where each NPU is supporting a subset of the layers in a model. This is a common technique in both training and inference, though I'm not sure if Tensorflow Lite supports it.
A pipeline would allow you to partition the model weights and compute across all of the NPUs, giving you a chance to run larger models then you could do on a single NPU. Your PCIe setup is very low bandwidth, but that is less of a concern here because pipeline parallel is only sending the activations (relatively small input tensors in inference) between the NPUs, not the larger weights.
Based on the limited information that I can find about Hailo and their sample hardware (a few of them have many Hailo chips on a single PCIe card) it looks like their software may support this.
I love the crazy/nuts thing that is this set up! And, even if the "Coral Dual TPU" only shows up as one of them, I'll bet that it is still cheaper and faster than the USB version. Now, where was that AM power meter that your Dad has . . . ? ;-)
I actually met the alftel guy and he is a GENIUS with RF stuff.
I don't doubt it. The contraptions he makes...
Thanks for trying AND explaining the problems!
Can't wait for someone to write a "Kitchen Sink Aggregator" patch that'll normalize or make modules generically access -- ahh, I just wanted to use kitchen sink in a sentence.
"And I will see you, in the next one!"
AAAAAAAAHHHHH WHO SAYS THAT?! WHO SAYS THAAAAT!??!?!
It JUST escaped my head!
WAIT, Chris Titus Tech does?!
I just read the description!
_Well then!_
...I think _this exact one_ is a bit common, is it not?
Hi Jeff, just wanted to let you know that you CAN use multiple hailo cores together, using the VDevice API - it automatically identifies the cores (granted it will only work with hailo chips).
HailoRT seems to have some multi-core configuration too... definitely some fun to be had here!
hehe a pi mad scientist cobbling together contraptions no one thought of. love it! great scott!
I’m amazed by the things you are doing with the humble RPi.
Not as amazed as that fractal north case for the pi5 they are showing off at computex, but still amazed.
It’s always fun to see you pushing the limits on single board computers. I’m sure it helps the hardware makers consider unusual use cases.
Jeff, have you seen the stories on the new research into IBD which suggests a genetic link and that certain cancer drugs might be effective? I saw the story on the BBC news site. Obviously I can’t post the link here, but I thought you might be interested.
Yes, I've been following that research-still 5+ years out from being practical but it's promising. The key is to find drugs that have fewer side effects than current TNF-blockers (which have similar mechanisms but target the immune system more broadly).
Level 2 Jeff is truly on another level.
The second one!
Please make a power outlet video, I'm just getting into HASS now and power monitoring is next on my list - Also great video cheers!
Being able to run two Halo8's would be a cool project as in theory would be (2x 26, or up to 54 TOPs in theory). Combined with dual Pi cameras, would offer high frame rate stereo depth of field, or other fun video processing.
BTW: the Home Assistant monitoring is fascinating. Would be interesting in hearing more details.
Jeff the mad RPi guy! Love these builds when you push the envelope! 😎🤣
I would love to see a video going over your choice of power metering smart plugs & the integration into HomeAssistant.
Needs a compatible case to keep it all safe or maybe 3d printed case perhaps ,Sound like there needs to be a Part 2 to this video.
As price drops for NPU's people will use more then software will write the software for it. That is what the raspberry pi was designed for learning in a fun way.
How do GPU bit miners support the core splits? Some setups are looking at spreading a single calculation over thousands of cores. Maybe there's a way to port a miner task divider to NPU tasks. But I also wonder if we are still limited to running each TPUs individual Floating Point rating? Or could we run full 32-bit models?
Jeff, you have to write a book with a chapter on each of these types of Pi mods.
Great job Jeff, you achieved 55 flops. Also today, NVIDIA released GB200 NVL72 platform achieving 1.4 exaflops/tops 🎉
Heh, Nvidia costs a wee bit more too :)
But the more you buy, the more you save!
This isn't Level 2 Jeff. This is Level over 9000 Jeff.
We need cheap PLX chips that take a newer standard bus in uplink and can provide many downlink at lower speed.
Also on consumer mainboard it is starting to be annoing. 20 or plus pcie gen 5 lanes where almost all hardware is gen 3 and 4.
Lot of throughput lost.
Couldn't agree more. Though those of us needing all that PCIe goodness are a slightly rare breed... and the answer till now is usually buy a big server CPU that gobbles up like 120W idle :D
@@Level2Jeff Even if we are the minority I think that the approcha that apple did with the MAC pro using the PLX and many lanes/slot is something that other HEDT/Workestantion OEM should start to consider.
New AM5 EPYC has been annoucned but lot of bandwidth is lost if you just plug a single HBA card on today mainboards x16 slot.
Love these kind of videos so keep em coming
0:04 Just flappin' in the breeze
Here before 20k subscribers! Keep up the great work Jeff
Hey Jeff! I'm still pretty curious about everything "AI". I'm just not sure how to take advantage of any of it right now. I'd love to see a video going over a bunch of different AI projects that these can be used for, either here or on the main channel. Obviously frigate is one, I've also seen some self hosted AI chat bots, though I'm not sure how well any of them would fare on a pi.
I know you're the "pi" TH-camr, but I'm also curious about other applications of such accelerators. I wonder if an AI chat bot would work decently well on a 1L PC (or some similar micro x86 system) using something like the Hailo for processing rather than trying to cram a GPU in a small system like that.
If you know of a TH-camr who is doing that sort of thing, I'm happy to check them out, just let me know.
Keep up all the cool videos. Cheers!
We have level 1 tech,level 2 jeff what’s next level 3 steve 😂
I know you won't tribute aVe, but, that would be hilarious!😂
before the WHAT it's compelling to know the WHY.
waited for the payoff but I guess I don't get enough why I'd want to use all these TOPs
I'm excited about using new tech but using it is the key. The excitement about the build comes after the excitement about the functional power. So I guess I don't get this video too much.
So like, objectively I know that the Hailo isn't really meant for inference of production grade LLMs or anything.
...But like, I still want to see if a person couldn't do something silly like a bespoke MoE architecture with 128M active parameters and still get okay quality and speed.
There a Qualcomm AI accelerator that use Dual M.2 slots, that board could be perfect for it (I've never been able to find the spec for dual m.2, so the spacing might be off), just one of those could get 200TOPS/25W
Good luck finding one though...
Can you show us actual runs? Also would love to see the 10H version when it comes to your hands.
Can these boards be plugged into a full-fledged PC?
It's cool to see that working _technically_, but in terms of inference, that's kinda the equivalent of having a 1000 CPU cores running on a single, shared stick of DDR4 memory. Do you hypothetically have enough cores to run a ton of applications at the same time? Yeah. But how much you'll be ably to do with the system will depend on the throughput of your memory. And it's the same with these little compute sticks. They rely on your Pi's memory so while you - on paper - have a lot of TOPS, running any task that operates on a reasonable amount of data or live streaming data will quickly saturate your onboard memory and pcie bus and prevent you from doing anything useful with multiple NPUs
I wonder how hard it would be for a Pi to orchestrate all these TPUs together. Could it offload some orchestration onto another TPU? I kind of want to write some code for this thing to see how it performs with multiple processors.
thanks for the video!
Can those neural chips be used for something custom other than video recognition?
Hi, do you have an idea on how to utilize 2 TPUs of that double coral chips?
I want to make similar board for deskyop PC, how to do that?
We love you for your tinkering!
Hell yeah, finally you dont just tease😂 i was waiting soo badd, am i rpi addicted🤨
Very soon, we will have an external NPU device connect via usb 4 just like eGPU 🚀💻
You choose to switch "geerlings" and it seems it paid off for the better :)
8:20! Google doesn't support the software for the Coral?! Is that at all? Or are you meaning on PiOS? Or?
I wonder how feasible it is to put an abstraction layer in front of the 25 coral TPUs so that the software only sees one big TPU?
That's more-or-less what RAID drivers for ZFS and BTRFS do so clusters of 25+ drives appear as one block device. And what CPUs do to make scalar code think it's still being executed in-order as the only process on a single-threaded CPU in a virtual memory space, despite out-of-order execution, superscalar execution, hyperthreading, multithreading, branch prediction, and more, all happening in the background.
I really like your videos they always bring novel information
Do you know when it will support running NVMe SSD and Hailo AI Kit simultaneously? (with the NVMe SSD used as the system boot disk)
You need that 12xm.2 card with a 16x connector and put it in your ampere workstation.
People need to try ridiculous things. The ridiculous of today become the feasible of tomorrow and the routine of the day after.
can we add this PCIe HAILO AI chips (i.e. 50+ TOPS) in our Mini PC?
I meant, If we want add it to NUC or Dell Opt. Mini PCs instead of Raspberry PI
@level2Jeff Where did you buy the Hailo-8 card? On their website they only provide an option for product enquiry, not for purchase.
I wonder if Bend would work with this config ❤
What's the best lowcost solution today for running local image-detection CNNs on a Pi4 or Pi5? Those USB-TPUs?
first thing i did with chatgpt was to have it write the code to house a llm on a local raspberry system (with 4 units and 1 master)
so Google TPU might be the LPU arranged in certain way? doesn't seems so, because LPU have huge memory chips... well the tops seems matched I wonder where this experiment will go into this is really interesting. Really interesting! indeed!
Wait, why does he say that the Hailo-8 is only 13 TOPs when on the RPi kit it is said that it is "26-TOPS"??
I had trouble getting my coral to work with codeproject ai long-term. It would work for a few hours but then stop responding. I don't know where the issue was on that. I mostly just gave up on it.
I had a similar issue in my testing, though didn't take too much time to debug it. One time it seemed to lock up the frontend of the Pi, had to force-poweroff!
Nice!
So, other SBC with PCIE Gen3x4 can leverage the full power of all AI chips installed?
Might be interesting for scientific computing. Can you send instructions via MPI or something?
Marco Reps at the end?
You don't need to make software for them to all work together just create something in python and back off it. you know what have you using the python multithreading library to trick it into multithreading ?
r u gonna do it w/ the halio?
think you might be able to look over the nanopi boards since most have built in NPU? I know the Nanopi R5S has one. Not that much of one but still there
Arent they all sharing memory bandwidth? Are there any AI m.2s with onboard memory?
can this be used for llm's and stable diffusion or is it only useful for video tracking.
one day graphics cards will be replaced by AI accelerator cards, and all you'll need is a low power gpu.
software support for parallel processing huh, something similar to Bend programming language?
Find someone who makes custom boards. Try to collaborate to make one that can handle all the TOPS you want.
Great video, does this also works with Pi4 ? Please also mention in future if the projects also works with Pi4 possible since not everybody has the latest and greatest :-)
It works with CM4, but not Pi 4 directly since the Pi 4 doesn't expose PCIe without a pretty serious hardware hack.
time to find some splitters for converting m.2 x4 into 4 m.2 x1 plugs for those dual-edge tpus
Shame that Hailo uses the PCIe slot, because I have a NVME bottom using that slot
This channel name sounds suspiciously like Level 1 Tech. Pi’s instead of massive and exotic server and workstation hardware?
Interesting, but cool!
but the real question is can it blend? what can one use this for?
Hey, thanks a lot for your videos. I'm wondering something that i'm sure has been adressed but i can't find a definitive answer on this. Does TPUs accelerate local LLM answer generations (Ollama for example) ? Thanks
thanks for tinkering
This *might* be the most unusual contraption you've made out of a Raspberry Pi to date. Beware! Future Jeff might break our reality with a Pi.
you are saying hold back while other people are saying the trillion dollar cluster is on the way - what would be nice is to combine a cluster of pi each with 25 top into one like nvidia does with its cards when they are in a cluster
Btw what's the SSD and WiFi card for?
Crap, I need one of those PCIe expansion boards for my NanoPC-T6 to be able to give it dual WiFi plus 5G Cellular and possibly a couple other nifty adapters. Dang it