[24] Jim Keller Interview 2: AI Hardware on PCIe Cards
ฝัง
- เผยแพร่เมื่อ 19 มิ.ย. 2024
- Written version: morethanmoore.substack.com/p/...
Updated Audio! • [24b] Jim Keller Inter...
Well, it's official. I finally got to meet the guy in person. Jim now leads an startup called Tenstorrent, designing ML hardware and software, with a lot of clever people in the team. In this interview, we discuss RISC-V, Chiplets, IP, Expertise, and if they'll sell PCIe cards.
[00:00] Spicy talk
[00:33] Q1: Moving from CTO to CEO
[01:48] Q2: Tenstorrent and Software
[05:30] Q3: Incoming Hiring Mentality
[07:29] Q4: Why have cores, why RISC-V?
[09:28] Q5: Chiplets?
[11:37] Q6: Is Tenstorrent an AI company or an IP company?
[14:56] Q7: Differentiation and Expertise
[16:48] Q8: Use Tenstorrent To Make Tenstorrent
[19:50] Q9: AI on EDA: Unintelligible Outputs
[21:25] Q10: Auditable Hardware
[22:47] Q11: New Clean Sheet Designs
[24:25] Q12: Comfortable!
[24:58] Q13: One Core = One Chip = One System
[27:00] LOLs
[29:00] Q14: Customer System Sizes
[30:30] Q15: Selling PCIe Cards? Yes!
[33:17] Q16: Customer Announcements?
Big Ball of Mud Paper: www.researchgate.net/publicat...
Even though Tenstorrent is a client of my analyst business, this interview was not sponsored content.
-----------------------
Need POTATO merch? There's a chip for that!
merch.techtechpotato.com
more-moore.com : Sign up to the More Than Moore Newsletter
/ techtechpotato : Patreon gets you access to the TTP Discord server!
Follow Ian on Twitter at / iancutress
Follow TechTechPotato on Twitter at / techtechpotato
If you're in the market for something from Amazon, please use the following links. TTP may receive a commission if you purchase anything through these links.
Amazon USA : geni.us/AmazonUS-TTP
Amazon UK : geni.us/AmazonUK-TTP
Amazon CAN : geni.us/AmazonCAN-TTP
Amazon GER : geni.us/AmazonDE-TTP
Amazon Other : geni.us/TTPAmazonOther
Ending music: • An Jone - Night Run Away
-----------------------
Welcome to the TechTechPotato (c) Dr. Ian Cutress
Ramblings about things related to Technology from an analyst for More Than Moore
#techtechpotato #jimkeller #tenstorrent
------------
More Than Moore, as with other research and analyst firms, provides or has provided paid research, analysis, advising, or consulting to many high-tech companies in the industry, which may include advertising on TTP. The companies that fall under this banner include AMD, Armari, Facebook, IBM, Infineon, Intel, Lattice Semi, Linode, MediaTek, NordPass, ProteanTecs, Qualcomm, SiFive, Tenstorrent. - วิทยาศาสตร์และเทคโนโลยี
![[27] Insights with Tenstorrent's Latest Board Director: COO Keith Witek](http://i.ytimg.com/vi/LIHJIXvBDQA/mqdefault.jpg)
![[34] Ian Interviews Aart de Geus, Synopsys](/img/n.gif)
UPDATED AUDIO VERSION: th-cam.com/video/YOiXomG9FhE/w-d-xo.html
Used some recommended tools to clean up audio
Everytime it's a pleasure to listen to your interviews with Jim Keller. As other people have pointed out though he deserves to be recorded with a better microphone next time. :D
Written version at more-moore.com
Need some AI to clean up the audio.
mic was on the table. i guess he didn't want to clip it on.
Jim Keller somehow manages to always give the best interviews with the worst audio.
universal watermark of brilliance
Lol, yep...
keck xD
Great to see another interview with Jim! The audio could be a lot better however.
Yeah room was filled with whiteboards, and it's just me hauling stuff. Tried to do a lot of AI post processing. I need a proper audio kit for these interviews that I can set up in 5 mins
Oh and now a written version for you: morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent
@@TechTechPotato You just need a little tripod to get the Rode Wireless Mic as close to the mouth of your talent/subject (I suspect putting a Lav Mic or the wireless Mic on Jim was not possible). No mic in the world can sound good if it is not near the source, unless you want to invest many thousands in shotgun mics and haul huge c-stands and tripods and boom poles around.
As an alternative, the "DJI Mic" has better mounting options, you can mount it via magnet so mounting it on clothing is much more easy. And Setup and user experience is an upgrade from the Rode Wireless Go II you are using.
IMO, just get some 50$ Rode Lav mics, and wire your subjects up, or just clip the Rode on the chest of your subject. Or as said before, at least put the Mic on a tripod in front of your talent if you can not do that.
Proximity to sound source is king in mics.
@@TechTechPotato One or two mics could have been dedicated to delayed remote room audio that would need to be attenuated significantly so that only nearby uncorrelated original voice signal audio remains. The room acts as a filter with frequency group delays and such that produce the noise artifacts that obscure clear reception of the audio. One method would be to focus audio reception from just one of the people and then have a second pass to optimize for the other subject speaker.
--> convolution dsp audio delayed echo paths stackexchange
The multi-signal path problem is more solvable when you have a fixed geometry and record using multiple microphones. Background echoed audio can then be used to filter sound by means related to one-dimensional convolution with the other signal sources.
Around 54 years ago I saw a science fair where a fellow classmate, the son of a polymath genius-level rocket scientist, produced an amazing prefiltered directional mic that employed systemic strategically varying length tubular channels that focused audio upon a parabolic focal situated transducer so that various frequency range noises would be pre-canceled using this elaborate scheme prior to the sound being recorded.
Doing an image search using various combinations of keywords seems to fail to produce any meaningful examples that replicated what I had witnessed those many years ago.
In prior decades I have seen many science and engineering videos produced that were done recklessly improperly for the ideal image or sound quality. I suppose that each subject and cameraman should be fitted with two lapel mics lest any subject speaker turns their heads from too far to their left or right-hand side. Often the cameraman would be asked to read a digital display reading from a meter and if not microphone-equipped, his response would be much less intelligible. Ambient lighting can also be a cause of low-quality production. The solution for this is to use expensive aspherical lenses that have close to ideal low-light exposure capabilities.
--> Stanley Kubrick Zeiss f/0.7 lens candle-lit scenes "barry lyndon"
Quotation from website/ The Solution -- Kubrick obsessively researched the problem. He eventually discovered that Nasa had commissioned Carl Zeiss to build ten Planar 50mm f/0.7 stills lenses in the sixties, which were used to take photos of the dark side of the moon. /End website quotation
The lower the f/stop value of the given lens, the faster motion could be recorded or lower light situations could be recorded. These various lenses are able to view things in ways many magnitudes better than the human eye.
The price of those lenses was many tens of thousands of dollars back in the 1960s, which translates into millions of dollars now.
th-cam.com/video/EUjUxKvXgpw/w-d-xo.html
Quotation/ Price is the barrier because good lenses cost as much as rare luxury sports cars /End quotation
Some sports cars are multi-million dollars so that price really is a barrier until manufacturers can robotically crank out millions of quartz crystal aspherical lenses with very low defect rates, then everyone could buy such stuff.
The first time audio was muted. The second time it was loud & clear. So clear, in fact all the tiny nuances of audio became apparent.
Jim seems different from the past interviews. At least at first, it was great to see him gradually open up, and his expensive thoughts flow out.
It was great to hear so much on the software side of things! The commentary on why software stability and parallel programming is hard is super on point. A lot of us in the software space would love to do this well, but that takes a ton of time and money that usually isn't available.
The ability for you to talk to people like Jim Keller and fully understand the conversation/provide micro feedback is something other techtubers (THAT I LOVE) could not match. I have seen people that could not hold this conversation shit on you, but things like this show you are OBVIOUSLY a step above the rest, real talk. I know for a fact people like Paul, Tom, Chris, Steve's are listening to this and realize their place.
I love those other channels too, but who else could sit in this room and have meaningful contributions? Maybe Wendell? But really everyone else are consumer facing computer techs, reviewers, overclockers, leakers and tech news repeaters. There's not a huge number of people with either the hardware design/software background to really discuss industry facing topics with the titans working in the field today.
Tom doesn't even belong in the group of those four. Have you seen his reviews? They're a step above RedGamingTech... rofl. He just tells Mfgs he's a reviewer so he can get hardware direct and not wait in line. "puts up token review like he said he would", What a joke. If he did that kind of work as his last employer, no wonder he's out of a job.
Maybe less realize their place and more be inspired by. But yeah I think that's why a lot of people are here, real talk. The last Jim interview was one of the best I've heard and this one was great as well, I end up pausing every couple minutes just to try to properly absorb what's being said and the implications of it. Also I have a feeling a lot of people are severely underestimating how much work Ian is putting into these video's/interviews beforehand.
@@SamanthaVimes177 Wendell might get too excited 😂. Buutt also begs the question, who is this 'guy over there'. I know Wendell has been the camera guy before.
@@Cinnabuns2009 I do take your point, but just remembered Tom also interviewed Jim in his podcast
Always insightful listening to Jim, and you ask great questions to provide the complete picture! Thanks for taking the time to go out and get this interview done.
Every interview you done with MR. Keller is a pure gem! Thanks
2 absolute legends chatting it up. gives me motivation to get deeper into optimization for specific hardware!
Always get excited when I see that Keller has done another interview.
Ian is looking at Jim like he met his god.. :) And deservedly so i would say. My favorite engineer in this space. Simply a brilliant man.
What a treat, thank you for this interview
Jim os obviously the man itself when it comes to chip architectures…so glad to have been alive at the same time in history
Wow, now this was truly awesome Dr. Cutress. I could listen to you two discussing this for several hours, and with my 5 minute attention span, that is quite a feat. So thank you for this. Now we wait for AI to rule the world.....
Was waiting so long for this, thank you !
Listening to Jim Keller is always interesting, great interview!
Reverberating interview, lads.
wub wub wub
Written version if you need it: morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent
Thank you Jim a rare treat !
thanks for the interview. you definitely need a better mic that can handle meeting rooms that are echo chambers :)
What tenstorrent is doing sounds very interesting. I'm sure other people won't agree, but the approach they're taking is the future of hardware acceleration for neural networks.
Jim is great. Thanks for this chat
jim’s a silicon assassin. great call TTP! def watch his discussions (two parts) with Lex Fridman 🚀
Keller is a very interesting man. Keep up the great interviews, Ian.
Interesting interview, great topic.
Aside from the extra nerdy content, I guess what I like about TTP is they don't even pretend to be stylish like other tech channels...😎
Thanks! Glad to see you guys together! P.S. What happened to sound?
Updated audio version: th-cam.com/video/YOiXomG9FhE/w-d-xo.html
Or written version: more-moore.com
Excellent interview, we might be looking at both investing and significant new specific “neural logic” venture. This approach should increase capability, speed, usefulness in multiple applications.
Wow, Jim sounds like a great systems designer!
Like, in general.
You teased quite something in the Livestream, so I am excited.
Ever since switching to an Intel Arc A750, I have been forced to actually care about hardware with Inference. I had to go through a bunch of onnx, Optimus, openVINO etc just to get something to work. Sadly the deadline is on Tuesday so I won't write a system paper and fancy up my code for my CLIP on Arc integration.
So since your last interview, the exposure of successful models to the public has been massive. We all expected for "AI" to become much larger. But does the reality match with the projections in the industry? Has there been any need for adjustment in scale since products like ChatGPT became commercially viable.
Can Copilot do System Verilog? Competently?
Jim Keller on air, hell yes!
Great interview!!
thanks for this one :) jim is great.
I was pretending to understand half of this and only had a vague understanding of the other half, although its always interesting listening to Jim Keller so I watched the whole thing and even learned a few things.
Please keep doing what you're doing, 'coz you're doing great :)
Cool interview thank you Ian, Jim Keller is a legend, but next time pls put the mic on the shirt of the guest so we can clearly hear everything without any distracting noise. Cheers!
That was a good discussion! Do you think we will see you talking to the other small businesses on the AI hardware map? I think Toronto has a few growing there.
I would like to know how you wangled that one and so glad you did, but you looked on the level with him and he better have got lunch in for you. :)
Cracking interview keep up the great work. :)
Always a pleasure to listen to lord of CPU!
I love this guy, none more insightful
I can't miss at interview with the "CPU King"!. For some who did not know Jim worked on the Alpha EV6!. Not the freaking car, the defunct CPU (from DEC/Compaq/HP)...
Do you know what HDL do they use? Also do they use Quartus or Vivado?
Great interview; on what date was the interview actually conducted? Thanks!
Couple weeks ago
This is super exciting. I can't wait to pop an AI accelerator next to my 6750 and not have to deal with CUDA drivers on my gaming gpu
I'd like to see a comparison with Erlang which has programmers using lots of processes or GA144 which is efficient hardware but lacking programmers.
Hey can we please get Jim Keller another interview as stated below with better audio. It would be great to hear the next 3 year plan from Jim with a little more detail if possible.
Damn that echo is annoying, is it possible to fix the audio post processing ?
Love to watch all Jim Keller interviews.
I did a lot of audio post-processing, even AI post processing. The room wasn't great. There's a written version if you want to read: morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent
Tascam makes a tiny recorder pack with a lav mic. Regardless of what your normal setup is, get one of those and > IN ADDITION < put it onto whoever you're talking to. It is much more important to properly understand the guest than the interviewer.
Written version: morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent?
Big Ball of Mud paper: www.researchgate.net/publication/2938621_Big_Ball_of_Mud
[00:00] Spicy talk
[00:33] Q1: Moving from CTO to CEO
[01:48] Q2: Tenstorrent and Software
[05:30] Q3: Incoming Hiring Mentality
[07:29] Q4: Why have cores, why RISC-V?
[09:28] Q5: Chiplets?
[11:37] Q6: Is Tenstorrent an AI company or an IP company?
[14:56] Q7: Differentiation and Expertise
[16:48] Q8: Use Tenstorrent To Make Tenstorrent
[19:50] Q9: AI on EDA: Unintelligible Outputs
[21:25] Q10: Auditable Hardware
[22:47] Q11: New Clean Sheet Designs
[24:25] Q12: Comfortable!
[24:58] Q13: One Core = One Chip = One System
[27:00] LOLs
[29:00] Q14: Customer System Sizes
[30:30] Q15: Selling PCIe Cards? Yes!
[33:17] Q16: Customer Announcements?
When will they verify these things on a Raspberry Pi ;)
Sorry if I missed this, but was this recorded before the announcement with Jim and his new company? Is he staying with Tenstorrent?
Ian, you should pin your comment to make it appear on top.
Thank you was looking for the paper the audio wasn’t clear after big ball couldn’t get any relevant search results
@claudyla Atomic Semi is not his new company, its just one he's investing in
that was fascinating. a bit tense for the first half though. accoustics sucked, but that wasn't Ian's , but more the rooms fault they met in.
Dancing Jim is the best.
what is the bigball thing ?
i couldn't find it
www.researchgate.net/publication/2938621_Big_Ball_of_Mud
Any plan for Tensotorrent going public ?
Ugh, I was late to see the fixed audio version :(
The Jim Keller Teller
excited to see the future of AI and a little worry about people losing jobs to AI too...
Intel have made a lot of mistakes in the past but I think one of the biggest in recent times was not putting their entire support behind Jim to move the company in the right direction much faster. It looks like they did listen in the end but 5 years too late!
Thanks for the interview. Wish the audio was clearer and less echoey though.
Yeah unfortunately the room wasn't great, and my audio skills are zero. I did apply some proper post processing on it. If you want a written version, then: morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent
@@TechTechPotato awesome, kudos!
@@TechTechPotato Just need a close mic for each person and that would cut out a lot of the echo
Than you Ian! Obviously too short. But thanks nonetheless.
@18:32 Woh. Models building Computer Logic. 🤯
find yourself someone who looks at you like ian looks at jim
Need a much better mic for Jim!
Great interview. God awful audio. When interviewing someone of this caliber you really should have thought about this. Anyway keep up the good work.
Jim an amazing guest as usual, but I have to humbly pick on his comment about "auditability". Great quote btw, do you think 5M LOC written by 500 people during 5 years, most of whom are not in the company anymore, is that auditabl?e 😄but he's also selling his ideas/business.
First, the goalpost is incorrect: we don't need "auditable" code at least not outside safety-critical niches. We need code that people can understand in a reasonable amount of time, in order to maintain it, debug it, evolve it. I work with that kind of code (>5MLOC, way older than 5yr, many primary authors gone) and it's pretty sane: not easy, but manageable. Of course there is a very large distance in the quality of code by humans that are new or or bad at the job and code written by humans who excel at it. But if ML can't be as good as the *best* humans, including the readability / maintainability aspect, I'm not interested.
I also lol'd at Jim's quip about Windows BSOD'ing all the time. Please upgrade your system, Win98 has been EOL'd :) I am a PC/gaming enthusiast so I put some abuse on my system - overclocking CPU/RAM/GPU, running beta BIOS and Windows Insider prereleases etc. - and I don't get any crashes that are not explained by some of these things (e.g. WHEA caused by a bad overclock). If I wanted, using the system more conservatively as a normal user, I'd go years without a single BSOD, especially if I also picked hardware for reliability (e.g. ECC RAM). Humans CAN make software that's fantastically complex like that, that's highly reliable.
On that last point - I used to be a world #2 overclocker. My modern laptop BSODs on a weekly basis, which I think is down to the Gen 4 SSD I put in it, which runs at Gen 4 speeds, but clearly the chassis doesn't like it as much. So I think your 'upgrade from Win98' is a bit flippant, especially given plenty of people, deal day in, day out, with systems that like to incorrectly wake from sleep etc.
@@TechTechPotato Yep it was intentionally flippant; I assumed Jim's "all the time" was a bit rhetorical as well. I worked previously as a Windows admin, Win2K era, even then we didn't get random crashes "all the time". Incidents counted per year not week. For our (small company) two Win2K Servers on Dell PowerEdge boxes, zero BSODs ever. Today I manage 3 Win11 PCs in my household, I check event logs and crash dumps, no random crashes. But one problem with crashes is their attribution, the OS gets blamed for everything. For critical and unrecoverable HW errors, the OS is just doing its job when it dies.
When it comes to third-party drivers, well Windows is not a clean microkernel system so it can die because of a bad driver for a graphics driver or an SSD. Still rare for me. I share some of your experience with power management issues, but mostly in my MacBookPro, which I run docked with external monitors and KVM-switched and sometimes macOS doesn't like sudden disconnect/reconnect of devices and dies. I don't get that issue in my Windows PC, but I don't have a Windows laptop and when I had one I didn't have this complex docked environment. The Mac is employer-issued, I'd never buy one with my money.
For a different perspective, I also use Linux daily: a local Threadripper-based workstation and an EPYC-based cloud VM. Both professionally managed, with WS/Server-class hardware, and without minefields like gaming-oriented GPU drivers. If I discount the crashes that are clearly not Windows's fault, I see no difference in reliability to the Linux systems, they're all extremely solid even though none is perfect. Of course the PC market being what it is, YMMV especially with laptops.
The fact is that with consumer PCs, we just expect too much from both HW and SW. We want to pay around $1K for a full desktop machine, choose every component for performance and price, abuse it with BIOS tunings and OC and bad software, reward GPU vendors for biasing drivers for performance at any cost - then we complain that "Windows crashed". :)
nvidia broadcast removes the echo pretty good
Doesn't offer a post-processing option. No matter how many times I ask about it. I did use several AI enhancement tools. There's a written version if you prefer: morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent
@@TechTechPotato Thank you very much.
Is writing software by AI really working?
Intel, you done goofed letting this guy get away.
@TechTechPotato Jim looks very stressed. Maybe the realisation that nVidia is waaay ahead and probably uncatchable at this point is the cause? CUDA is deeply entrenched.
Tenstorrent selling RISC-V chiplets seems like a bad idea to me.
It sounds like the typical temptation a pre-profitable startup faces when potential customers wave cash around for having their problem solved that you're capable of solving. The question is: How will this help Tenstorrent's mission? At what cost and opportunity cost to the engineering team?
Perhaps it makes sense if it aligns highly with what Tenstorrent were planning on doing anyway? I have no idea really, just seemed weird to me.
The audio quality is very very bad. :( (Super strong reverb and one can not hear clearly what’s being said.) Couldn’t you place a mic between you?
There's an updated audio version now: th-cam.com/video/YOiXomG9FhE/w-d-xo.html
Or a written version: more-moore.com
Audio quality needs work… a lot of work.
Well, everything he said was very understandable.. In this case, PCM Linera audio was not needed.. but it can be improved with a microphone near the table, like a podcast.. 😄
It sounds like Ian has his mic clipped on lapel, but the other mic is sitting on the table relatively close but not super close (so picks up a lot of room noise).
Not a lot you can do to clean up the audio in a large four wall space like that. Could maybe improve with a small shotgun pointed directly at talent but Ian might not have had one suitable for the gear he has for mobile recording-sometimes the people you interview aren't comfortable putting on a mic, and you might be at their mercy for the recording.
I turned on closed captions. Helped a lot.
@@JeffGeerling Jeff has it right - with Jim's high collar, it would have looked odd to have a mic jabbing at his neck, so I put it on the table. The end result was a mix of both mics, lots of AI processing. The thing is, if I carry around 2 mics for podcast/table recording, I just know my next interview will be standing up. etc
It the way this guy talks, dude needs to learn how to use his diaphragm
The audio is terrible, he's not wearing his microphone!
With his jumper it looked like it'd jab into his neck which looks awkward. I did some AI post processing on it to get it this good, but if you prefer a written version, try morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent
He get a lot older. But comparison AI to human was great :D
should have given your mic to the interviewee
so short?
You take what you can get!
Just some constructive feedback: For a rather large channel like yours, and with guests of this caliber, your recording equipment/setup should be better. Hard to hear what Jim says. Echo in the room, muffled, aircondition or something in the background frequencies. 👍
Yeah, I want prepared for the environment. There are links to a written version in the description, and I recently uploaded a better audio version, link to that is in the description too
OMG the echo/reverb is horrible, worst possible meeting room ever.
"We already use massive computers to do simple things"
yet 4bit and 8bit microcontrollers are still very popular in embedded electronics.
Most of the time, better hardware capabilities are used to ease software development.
..we are getting withdrawal symptoms!..where are you?..
A bit too much echo for my liking.
If you want a written version, try morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent
bad sound
written version: morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent
A lot of big claims about Tenstorrent made by Mr Keller. None have yet materialised
To be fair these things tend to have a REALLy long maturation period remember he was gone from AMD years before Ryzen actually hit
In 5 years Jim's Tenstorrent will become more valuable than AMD and will acquire AMD. That's the way I see it because AMD's CDNA is just modified GPU and not ML/AI hardware.
Apple and Qualcomm have their AI/ML NPU HW so they will be fine too. Tesla will be strong because they do their own designed AI/ML chip.
Intel and Nvidia will struggle much as their HW is moreless heavily modified GPU. If they not adopt they will die just like AMD will.
As smart as Jim Keller is he is addled by old though brain. Why wait to release the card when they can sell cards under a beta marketing and get prosumer feedback. If this AI pcie card can outperform Nvidia cards at stable diffusion then I'm sure they can sell at least 10k cards