Tesla Dojo AI Supercomputer Deep Dive and Analysis
ฝัง
- เผยแพร่เมื่อ 4 มิ.ย. 2024
- more-moore.com : Sign up to the More Than Moore Newsletter
A deeper look into how Tesla's Dojo microarchitecture and system level design is constructed.
[00:00] Intro
[00:56] More Than Moore Newsletter
[01:21] Part 1: Training vs Inference
[03:48] Part 2: Dojo D1 Chip and Core
[11:21] Part 3: Packaging and Tiles
[15:50] Part 4: Custom Network Silicon
[19:13] Part 5: The Tesla Dojo ExaPod
[23:40] Part 6: Comparison to Cerebras
[25:32] Part 7: Tesla AI Day 2022
[26:59] Thoughts on Tesla FSD
[29:26] Cat Tax
-----------------------
more-moore.com : Sign up to the More Than Moore Newsletter
/ techtechpotato : Patreon gets you access to the TTP Discord server!
Follow Ian on Twitter at / iancutress
Follow TechTechPotato on Twitter at / techtechpotato
If you're in the market for something from Amazon, please use the following links. TTP may receive a commission if you purchase anything through these links.
Amazon USA : geni.us/AmazonUS-TTP
Amazon UK : geni.us/AmazonUK-TTP
Amazon CAN : geni.us/AmazonCAN-TTP
Amazon GER : geni.us/AmazonDE-TTP
Amazon Other : geni.us/TTPAmazonOther
-----------------------
Welcome to the TechTechPotato (c) Dr. Ian Cutress
Ramblings about things related to Technology from an analyst for More Than Moore
#techtechpotato #Tesla #dojo
------------
More Than Moore, as with other research and analyst firms, provides or has provided paid research, analysis, advising, or consulting to many high-tech companies in the industry, which may include advertising on TTP. The companies that fall under this banner include AMD, Armari, Facebook, IBM, Intel, Linode, MediaTek, NordPass, Qualcomm. - วิทยาศาสตร์และเทคโนโลยี
more-moore.com : Sign up to the More Than Moore Newsletter
[00:00] Intro
[00:56] More Than Moore Newsletter
[01:21] Part 1: Training vs Inference
[03:48] Part 2: Dojo D1 Chip and Core
[11:21] Part 3: Packaging and Tiles
[15:50] Part 4: Custom Network Silicon
[19:13] Part 5: The Tesla Dojo ExaPod
[23:40] Part 6: Comparison to Cerebras
[25:32] Part 7: Tesla AI Day 2022
[26:59] Thoughts on Tesla FSD
[29:26] Cat Tax
Thanks, I absolutly like your deep dives into architectures, like IBM z16 and Cerebras. Would love to see more content like this.
This is the type of content I'm subscribed for. Just started on the VLSI track at uni so it is super interesting to see what is out there.
Dr Ian Cutress. Here your title punches pretty hard, i had to pause from time to time to take a breath and digest. Excellent material 🙂
Great review and perfect timing for a refresher! Hope you do a follow-on in October if Tesla releases enough new details.
29:28 the eagle-eyed among you will notice the cat tax has been doubly paid
Really nice overview. Hopefully someone from Tesla send you an invite!!!
Thank you for this thorough and non biased analysis!!!
One of the subtopics you could explore a bit more in depth are number formats. Take a look at Gustafson's work on unums and posits, they should be the "one format to rule them all" going forward.
Vividsparks has reached out. But yeah, a look into number formats is on my to-do list.
Should Tesla have used posits for dojo instead?
I remember reading a paper which described the fact that posit arithmetic was less efficient to implement in hardware compared to IEEE754 fp formats (on FPGAs at least). Although the article did not discuss smaller more efficient formats such as BF16 or FP16, its conclusion could explain why this format was not used by tesla.
Thanks for SUPER DETAILED presentation
Great insight into this architecture. I’ve got an exam tomorrow focusing on the DOJO Tesla, and this video gave a great summary of the main points.
Thank you so much for this deep dive and especially for the putting the effort into comparing Dojo with Cerebras
Excellent review of the the entire process Tesla has gone through in creation of Dojo. We’ve been working on optical processing for 20+ years in a hybrid format and it interesting to see Tesla’s focused approach.
Our work is hybrid technology, but the optical cores reduce power and provide us with different approach to similar AI problems.
Brilliant walkthrough. Thank you for the detailed presentation.
Great video. Very insightful 👏
Excited to see your breakdown after Ai Day 2
Really enjoyed the video!
thankyou alexia, for reading the excellent writeup
Excellent vid! Thank you!
It is great that you make this video, and I think you at Tesla AI Day 2 would be align with they goals, because not all people would know about this event.
And it would be great, if they gives more numbers, to make deeper comparison between Tesla solution, and not only Cerebras but other systems (as Tesla is using Nvidia GPU cluster today) too.
Awesome video thanks doc😊
Awesome Video doctor.
Great video, thank you for making this.
No way you watched it yet
@@kalebbruwer I did, i commented about 2/3 into it.
@@RoundSparrow My bad
Thank you
Good work DIC!
Good job
DEEP DIVE contents Always be in my favorites in every youtube science channels, please do more of this awesome topics this is very interesting to watch and learn
I'm subbing ♥️
impressive sir!
One word: insane
Interesting designs. Tesla uses NPUs(kinda stripped version of GPU in some manner) for FSD chips but uses CPU cores for training. Thank you for a nice analysis. I'm really curious how their training codes are implemented.
Will there be an update with what you were able to learn at the event?
Impressive!
thanks for the detailed breakdown. It seems similar to Cerebras where cores have small SRAM and designed for stream processing. Makes sense given Tesla is moving to training on video streams, instead of individual frames from separate cameras. It would be interesting to see a comparison of how TPU v3 handles streaming video training to DOJO. Google has been training on video for several years now.
autonomous taxi has already gone live in ShenZhen china and their FSD seems more advanced that the current Tesla FSD beta.
⚠️ Hey, you found an excellent way to hide your microphone wearing a black shirt. Simple and functional. I already tried many solutions to hide my mic, but the best one was to use a mic like yours, pointed to my mouth, and eliminate all reverb from the room. Great 👍 🙏❤️
Yes but is it better to use Brainchip's ultra power efficient Akida chips?
you are the type of journalist that Elon would want at the event that can read the data with understanding compared to places like CNN Fox etc
@27:10 - FSD in passive mode. Pothole mapping would be interesting, especially of shock movement data is available.
I think they focusing not to not fail at weird intersections ;) As they did talk in past, they pulling data they want to optimize in given moment.
What's the cat tax song?
(Sorry if you've mentioned the name in a past video - I don't see any mention in this video or the description)
th-cam.com/video/2N0tmgau5E4/w-d-xo.html
Cat pictures infest the internet and actual cats plage computer users.
They should consider making idle time on theirs public, or make a small one for public use. Mostly for getting devs experience and or excited with the architecture and then maybe hire them :D
Do you think Dojo is ready for production as they recently stated?
It would be interesting to see how the Dojo chip can do computational fluid dynamics. That would help with weather forecasts, for example. Then again, an AI-based approach to weather forecasting might be more appropriate for the Dojo chip.
AVP
yeah / now I'm trying t pressure freeze diamonds / so the chips don't get hot making them //
Is this the first major RISC-V chip? I know there are many micro-controllers, this would probably fall under accelerators.
It's not RISC-V. Don't get RISC confused with RISC-V.
@@TechTechPotato Ah, thanks. It's some kind of TESLA version of RISC
Im new to the channel. Have you made a video comparing a Dojo chip to the latest Nvidia chip or Dojo wafer to Nvidia GPU?
Love to drive a Tesla through white outs.
That should give them some interesting events.
Using hydro poles as markers and feeling road tracks, because it is impossible to see the road.
I wonder if Tesla is open sourcing Dojo?
The idea of each chip being a grid of cores, and the whole supercomputer being a grid of those chips that link their individual grids together is very similar to what Adapteva originally was planning for their chips.
First time listener here. You have a plesant voice.
I would think you'd put out a video like this on the "day of" or "week of" Tesla's full self driving release which was promised 5ish years ago? Is it out yet?
Great info. Q: Who would you want to build a next gen Personal Computer, Cerbras or Tesla?
wait did you say telsa leading the way in automous self driving??? Hello?? What has Google been doing with with Waymo???
google and what mass-deployable neural net?
Hi Ian, long follower here !!! Just as feedback not criticism, but the audio quality is not great haha
The architecture looks a lot like the CPU in a Cyberdyne Systems T-800 model 101 Terminator
I'm certainly interested to see some kind of real comparison between this and the Cerebras. If they were purely running convnets, punishing FMA ops, branch prediction and scalar units are wasted silicon. Even nvidia's "sparse" tensor engine is wasted space for most applications, a mux choosing the non-zero element of a pair of contiguous elements is just a neat trick to increase theoretical numbers that doesn't really help the majority of workflows. You have two non zero elements next to each other? Sorry, you can't use sparse op/its output is malformed.
The sparse tensor stuff is good for slightly sparse inferencing (if you have trained for sparsity), but useless for training.
You can even train the network to make sparsity in the pattern that the sparse tensor engine likes.
This musta been one Hellva script to write. AI day 2 next week!!
The Dojo system performance and efficiency is heavily dependent on the software Tesla can develop. Chip specs are really meaningless unless they build the right software.
I have no doubt they can do it, do you ? 🙏👍😄
kinda looks like tenstorrents architecture :P
I feel the missing keywords are "Organic electron flow"
But isn‘t it so that DOJO might come online with so much delay that off the shelf alternatives like nVidia Hopper would already have caught up in performance?
Dojo is meant to be intimitely optimized for Tesla workloads. One thing you can't easily get over is chip to chip connectivity. Nvidia hasn't scaled that with Hopper.
@@TechTechPotato NVLink for Chip to Chip interconnects and Mellanox Infiniband to connect the nodes isn‘t too bad. Depends on the exact workloads whether this is a limiting factor or not.
1.25MB of SRAM is not a cashe, it is more like a scratchpad.
My question is: Wouldn't they also benefit from analogue chips for machine learning that Veritasium has talked about? With the recent rise of analogue computers and what not.
That Veritasium video was kinda laughed at in the industry. He failed to make the distinction between chips used for training and chips used for inference. Mythic are building chips that do inference, whereas Dojo is training.
Tesla going this route, how long until we have "Captcha" from Google showing us driving incident footage and we have to input what we'd do, in order to "prove we're human" when actually we're just teaching Google how to drive.
Problem with google solution is "good data", even if you pay and train people they make mistakes. This is why one of big moment of Tesla training was auto label, which cut time and errors.
Simple
Sir, will you teach us about microprocessor, microcontroller, gpu and other things such as logic unit, other specialized hardware ASIC and FPGAs
love the overview. would Tesla's approach of silicon on wafer, as compared to cerebus direct wafer scale chip, limit power, as there is an extra layer to transfer power/heat through?
You gotta wonder how cost effective it is to roll your own vs just buying something from a company that specialises in those things.
It depends on how large you are scaling up. If you are at a warehouse level meaning your computing needs can fill up an entire warehouse then it starts to make sense to roll out your own custom microchips. Tesla is already at that level. Currently they are running three warehouse size clusters of Nvidia GPU and they recently added a new one. Their projection is that they need to 10X their current computing power so they really need custom supercomputer to reduce their computing cost.
Big fan of the channel, but to be honest though this was really in-depth, it felt kinda rushed and hard to follow. Had to pause and repeat quite a few times.
Check out the written version!
This is like the third video this week that the names Tesla as “ leading the way in autonomous driving”. What do they have that no one else has again other than broken promises?
Making videos about tesla guarantees views for TH-camrs. The people love tesla content. That the company is full of shit doesn't matter for the ad revenue
@@gptinsights Dr. Cutress is also angling for that A.I. Day invite, so the logic is pretty simple: Make a Tesla Dojo video, include a note at the end that he's trying to get a A.I. Day invite, get 2 million views from extremely online people who do nothing but watch and talk about Tesla all day, and hope enough people spam Elon's Twitter feed with links to the video and he extends the invite. I'm not being critical of Dr. Cutress here, it's a valuable video in its own right, but Elon only seems to interact with the world through his Twitter feed, so this video is what the meta requires.
@@jasonosmond6896 yeah I don't blame him
It'll make his channel more popular. But he might lose some credibility for it
Seems like you didn't want to do it. it was interesting nonetheless! See you on the next one.
Seems like I didn't want to do it? What?
Dojo deep dive. Maybe it just me, just seem to have little less enthusiasm. As before still enjoyed watching.
Would it be possible for Tesla cars to all do a small amount of training as they go and transmit the data back to the data centre to make the whole fleet of Teslas worldwide help continuously help refine the algorithm?
That's what they do - transmit the interesting clips back and use it for training in the datacenter. Training doesn't happen on the car. Same reason why you don't render graphics on a CPU : 100000x difference in perf and perf/watt
@@TechTechPotato Ok I figured they'd be doing that, but wondered if perhaps there would be a certain critical mass of Teslas all processing that it would become feasible. Thanks for the response!
Why is it everyone in the media keeps saying tesla is leading the way when they only have a fancy cruise control and Mercedes has level 4 self driving and Weemo has taxis literally already driving themselves?
Lol
I want more silicon details!!!!! Muuhhahahah
Dumb question can they use all the waste heat to drive a steam turbine and generate electricity?
Not to be an a$$ but Tesla has a horrible record with their self driving cars, and other car manufacturers already have more certifications for FSD tech in their vehicles across the world whereas Tesla is loosing some of their FSD certifications... Tesla was the first longish range mass market EV that doesn't make them the best and that's becoming more and more apparent as the years/months go by.....
While there is a lot I like about Tesla's approach, I can't help but feel like they (and other self driving tech companies) are still trying to brute force the problem. I think the decision matrix is too large to really be able to use this approach only. I feel like they are going to need a way to vastly speed up "training." Conceptually I'm thinking they are in the stage of where path tracing was a decade or so ago, and they need tech akin to DLSS and other tech that is making "real time" path tracing possible. Yes, I know those use "AI" and I'm not saying to use AI to solve an AI problem, I'm trying to say there is a related tech that actually needs to be developed before we can solve the FSD problem. Call it Artificial Imagination, basically a way to pare down the problem in a way that makes some training possible on the in vehicle hardware, or at least fast enough that it could happen over a network in close enough to real time.
Everyone and their mom is designing their own processors these days...
“What’s your minimum specification?” Is that some kind of inside joke for chip experts?
my minimum specification is potato ;)
Does no one have an answer?
@@ophello Yes, it's an inside joke.
Basically just "in closing", or "I would like to see"
@@popcorny007 I still don’t get the joke. That’s how the video starts, so I’m not even sure what you’re taking about.
Hundreds of billions spent on "training" across the field, and it's not even the right path forward.
Could you please not add the progress bar to your videos or make them very subtle somewhere in the corner. The green bar was so distracting that I kept staring at it and mindlessly skipping all the interesting content you were presenting. Thanks! 🙏
If TechTechPotato knew anything, he would have a JOB, not a YT channel.
I have a JOB outside YT. I'm very well paid. Google me sometime. ✌️
Wish Elon creates desktop risc-v out of this someday.
since when are you making science fiction videos ;) tesla dojo is prob a fugazi
You are not keeping up. Dojo is a complete flop.
Musk fired the head module designer for non-performance.
He designed the Dojo module with little to no memory.
The idea was that it would make the system faster
Turns out AI is a memory pig to work efficiently!!!
So at the very least the entire Dojo project will need to be redesigned or scrapped altogether
or
Maybe repurpose the chips for a huge data center after adding memory
The good news is that FSD AND Optimus are NOT dependent on Dojo
Our original supercomputer is still the one we lean on for all FSD and Optimus iterations
The 10,000 H100 Invidia chips went to our old supercomputer -- NOT Dojo
You're commenting on a 16month old video.
"superscalar multi-threaded organization" for their subprocessor nodes on a transport oriented architecture. That right there tells me they don't know what the f*ck they're doing. I think chip design automation tools have become sophisticated enough we are seeing the rise of the equivalent of "script kiddies" and "code monkies" in chip design. They just click on features they think sound cool based on the buzzword and make the design tools throw them in the final design, LOL
It really triggers me when people talk about GPU SIMD lanes as if they along constitute cores. Like people that will tell you "this GPU has X thousand cores" and you try to tell them that technically it doesn't, but they keep saying it 🙄
Good thing these are actual cores, and not simple SIMD. My whole PhD was programming GPUs.
Modern NVIDIA GPUs are very good at pretending that those SIMD lanes are separate cores, so good that it has caused us very nasty bugs. (Most of our code was written with awareness of the SIMD lanes, and optimised for them)
My PhD was mostly programming GPUs too. (and designing an architecture which looks quite similar to Dojo, to solve the multi GPU scaling issues)
I'd call it a model but I am an ass
Reading slides.
Chip On Wafer, You have a C.O.W., Well time to Milk this C.O.W. And Pet the Tabby boy...
Per your last thoughts..... It's been my contention from the beginning that Tesla risks being accused of being "racist", if FSD begins to mimic driving behaviors that are only seen in America in the very risky, high crime neighborhoods. For example, It is normal procedure to never, ever stop at stop signs or traffic lights after 10 pm for example, for fear of having your car hijacked or shot at.
If Tesla ever implements algorithms to mimic local driving behavior (which is 100% necessary to Drive in Mumbai, Bangkok, Germany, and Harlem), then that will be the level at which jeers of racism will occur.
No one I know is interested in self drving,nd no one I know plans on using self driving. So quit wasting time and instead build an EV 450 mile range, quick charging, lots of charging stations and priced in the mid twents.
AI has no ability to determine context on it's own....a very serious issue and short-coming.
You should learn about AI before you talk about them. Machine learning is all about learning context from a vast amount of data.
A car company that is giving the likes of Nvidia and AMD a run for their money... Even Intel with their vast sums of R&D couldn't make Itanium work (but they didn't have TSMC).
The power desity is immence (more than most racks in just 2U?).
LOL... Oh the ignorant.
넌 뭐하는 얘니?
I love that EVs are moving into the mainstream but I hate Musk and his pseudo intellectual, fake liberal, kind. My hate for him and those like him drive me away from EVs, etc. I see his name attached and cant bring myself to be interested. I know this video isnt about Musk BUT this is how much I hate him. I dont hate many things, hate is a strong word.
You should touch grass. I hate Putin but I would not late that hate consume my life.
Blah blah blah … you are out of your depth when talking about ai… stick to hardware dude…
How so?
What's My Minimum Irritation?
Fake accents... 😕
#Grrrrr
After just watching the NASA DART hit an asteroid, and then stumbling across this, I think almost ANYTHING is possible with Humankind! Plus I'm getting ready to build my Intel 12th Gen Computer. Thank you for your video! ❤🤍💙