AI Hardware Will Be Everywhere (for a fee)
ฝัง
- เผยแพร่เมื่อ 23 มิ.ย. 2024
- This is a recording of a talk I've done now at multiple events - in this case, it is the @eetimes747 AI Everywhere Forum, hosted Dec 13th 2022. The title of the talk is State of Play: AI Hardware Startups, and covers the different types of architectures being built by a lot of the companies in the AI semiconductor hardware space building chips for either machine learning training or inference. As of recording, around $10B has been invested into these startups (at least, $10B that we know of), and estimates are in the $100B if we include software and the big players investments too (NVIDIA, Intel).
It is free to sign up to the EE Times AI Everywhere Forum at
aieverywhere.eetimes.com/
Timestamps
Intro
- [0:00] The role of AI Hardware
- [0:40] Talk Overview
- [1:50] Intro to me #selfpromotion
- [2:45] AI Hardware: A New Era
Types of AI Hardware
- [4:37] CPUs
- [5:43] GPUs
- [6:56] FPGAs
- [7:37] ASICs
Types of Architectures
- [8:00] Systolic Arrays and Google TPU
- [9:21] CGRAs
- [10:45] Parallel Pipeline / Dataflow
The AI Hardware Startup Market
- [12:04] Current investment levels
- [13:36] @SambaNovaSystems Cardinal SN10/SN30
- [14:13] @CerebrasSystems Wafer Scale Engine
- [15:36] @Graphcore IPU and Bow
- [17:11] Honorable Mentions: @GroqInc , Lightmatter , Rain, Tachyum
Conclusions
- [18:00] How to Scale
- [19:12] @tenstorrentinc CTO Jim Keller Quote
-----------------------
more-moore.com : Sign up to the More Than Moore Newsletter
/ techtechpotato : Patreon gets you access to the TTP Discord server!
Follow Ian on Twitter at / iancutress
Follow TechTechPotato on Twitter at / techtechpotato
If you're in the market for something from Amazon, please use the following links. TTP may receive a commission if you purchase anything through these links.
Amazon USA : geni.us/AmazonUS-TTP
Amazon UK : geni.us/AmazonUK-TTP
Amazon CAN : geni.us/AmazonCAN-TTP
Amazon GER : geni.us/AmazonDE-TTP
Amazon Other : geni.us/TTPAmazonOther
Ending music: • An Jone - Night Run Away
-----------------------
Welcome to the TechTechPotato (c) Dr. Ian Cutress
Ramblings about things related to Technology from an analyst for More Than Moore
#techtechpotato #machinelearning #venturecapital
------------
More Than Moore, as with other research and analyst firms, provides or has provided paid research, analysis, advising, or consulting to many high-tech companies in the industry, which may include advertising on TTP. The companies that fall under this banner include AMD, Armari, Facebook, IBM, Intel, Linode, MediaTek, NordPass, Qualcomm. - วิทยาศาสตร์และเทคโนโลยี
Are there any AI hardware companies, particularly startups, that I'm missing here? Preferably where I can track their funding on Crunchbase :) I already need to add in Achronix for example...
Maybe you could talk about the future of photonics sometime in the future... you mentioned Lightmatters 'optical computing' and I wondered if that was the same as photonics.... no matter, this is blowing my mind lol
Hey … just watch you on LTT your issue is a windows update! Literally had the same issue random blue screens every soo often and then all of a sudden black screen after login page with the same error opening explorer or any windows settings.
You can fix it by removing the latest update via CMD. Hope this helps!
You may want to take a look at FuriosaAI - they have consistently been performing well MLPerf benchmarks
Amazing summary ! this channel is such a gem !
When a 20 minute video creates hours and days of follow-up learning for hardware noobs like me
I've got a fun series of videos coming in the new year on similar lines :)
Thank you for the brief mention of Groq (th-cam.com/users/groqinc). We hope we can chat more with you this coming year and start talking about our next-gen product solution.
Id like to know about AMDs AI engine in phoenix point made by Xilinx . also interested to compare it with intel's VPU m2 card.
This was the best AI Industry overview presentation.
Very well put together. Congrats! And loving little Ian in the corner 🙂
Good roundup, look forward to the next one.
I just saw LTTs video where you were having a black screen after login, and I was coming over here to say that I had the EXACT same thing happen after a windows update, and I figured out my issue was due to a program called OldNewExplorer I believe? Basically anything that directly modifies the Windows shell causes this after an update. I had to open run (WIN + R) and type "control" then uninstall it and restart. Fixed my issue, you should check and see if there's anything installed that seems like it could be a problem. Not even sure if you're still having this issue since Linus' videos are uploaded weeks later.
Yeah I fixed it about a week later. Installed CCleaner and opened Task Manager to run it, uninstalled about 50 non-vital apps. It now boots every time.
wow!
amazing video! thanks!
very surprised to see something like this on youtube
Thank you! Very interesting!
Nitpicking - CGRA stands for Coarse-Grained Reconfigurable Architecture
Hey Ian, what is the name of your video on "roadmap for semiconductors to 2036"?
I found the one on IBM and quantum computing, and you listed that one as a related video (shown in below the "chapters" of this video).
Great, great content. Making it easier to find videos on good topics like the roadmap to 2036 will only help your channel grow. I’ll be interested to see how much it grows in the coming years.
Ah. Perhaps never mind? I googled instead of TH-cam searching and I see that the video is probably "How we get down to 0.2nm". I already had added that video to my watch later queue.
yup that's the one! It should have come up as a card in the video
Despite all these AI hardware startups, AI hardware is still domiated by nvida at least for application like large languate model (GPT) or image generation most if not all of those are trained and run on nvidia stuff (A100s)
Deliver a talk on AI hardware ... tick
Pester Linus on LTT ... tick
It's been a good day
Quality
I predicted years ago that AI will be available via interface nodes for a fee. Such hardware could hardly be for free (unless they find a way to use adverts the way they do on the internet to make it "freely" available to the public; but that is a long term prospect).
I feel like I have heard this talk before, at least parts of it. The recent upload where you held it in London?
If there were a better solution than just buying the nvidia RTX 6000 Ada (short of a H100 PCIe) to run large model inference in a local workstation... AMD has some workstation cards with a lot of vram now. Intel has Gaudi2 as well as PVC - but don't sell either to workstation, they aren't in that market at all.
Also why spend 6000€ on a large GPU just for model inference that sleeps 98% of the time, especially if your Gaming GPU costs maybe 250€.
I hope some of these products make it to consumer workstation soon - because all products currently sold as "ai accelerators" are from 5 years ago and for tiny computer vision models. not 30B language models.
Mythic was a big loss, I reckon it was the most interesting Interference start up out there. The promise of high output and low power could have been a winner.
I would invest in anything Jim Keller (aka Miles Dyson!) is involved in.
Do you guys also look at photonics? I follow a company in very late development stage, polymer photonics (LWLG) 3 times the speed with 1/10 of the power. Please check it out
Ayar Labs, Lightmatter. Though photonics compute vs photonics networking are two different fields.
Damn i wish i could put some money into those companies.
Actually you can. There is secondary market operated by few platforms like forgeglobal, equityzen, equitybee etc.. where employees of some of these companies sell their stocks. So technically you can be part of their wonderful growth.
@@cyberguy3694 even if you are a non american person?
@@lorabex791 Not entirely sure about that. But I have seen the option for international investors as well. You can enquire with equityzen as they are one of the more popular platform for such transactions
That was a lot of info, but in my opnion power of AI still in cloud , not in a particular hardware vendor
Every smartphone has dedicated AI processing in it these days.
@@TechTechPotato right a processing , but the core. Distributed computing rules.
You don’t really believe Tachyum is onto something, do you?
Their 2022 architecture is a big departure from the 2018 one announced ages ago. It could work. I've got a video on it on the channel with the Chips and Cheese crew.
@@TechTechPotatoI’ll check it out.
A little heads-up, Graphcore’s WoW tech increased frequency by 40%
This year will mark mass consolidation or end of most of these AI startups, since they will struggle to raise money and ballooned their valuations at covid peak. Now wants someone to hold the bag. Dark times ahead.
You forgot to mention the Akida family of chips and IP from Brainchip which are extremely energy efficient! Maybe you could interview the Founder Peter Van Der Made or other company people at CES 2023?
$10 Billion... 😳
🤔 have you heard of my AI Startup. 😏
New products will always be there. Best is to wait and watch. Money loses value depending on production power. In fact the world has excess than shortages. Sales is where they make goofing. Loss in some products because of timing is way out of time of release. Even if it is worth billions it is out of reach for some. People always look for cheaper items except companies which need them.
Startups don't matter, they only exist to try to scare one of the big players into buying them.
No. You’ll see true innovation from these startups. But yeah, bigger companies will likely but some. But not because the startup scared them into buying them. Because they came up with something useful. Startups matter!
Who giving loans
Everything is worth nothing until someone puts a price on it. Wow! Maybe I should be an economist. 🤣
Maybe YOU should be an advocate for AI laws that limit it's uses and ethics... Or at least get the ball rolling.. we have ALL seen the scary stories and now it actually seems like it's ABSOLUTELY possible with all these amazing advances in just the past 5 years....
You could bring the world together to begin the SERIOUS discussions about the implications of unbridled advancement. Think about it Doc :)
lol laws. I'd like to see them enforce them.
first