The original thumbnail had a low click-through rate, so I'm testing out the test&compare feature If you saw a dumpster-grade laser-eyes Jensen meme, I apologize, but we serve the algorithm!
The idea of a world that doesn’t lose up to 90% of the original energy content of fossil fuels to heat loss at the multiple steps along the extraction, refinement, and use pathway is pretty incredible.
Awesome video! The +2% per year seems reasonable. However what you have missed is the possibility of on the edge inference with 1 billion robots, which do have enough power to burn. To your question of GPU prices: 1. The H100 has HBM memory instead of GDDR memory, which is ~6-8x more expensive for the same size. 2. The yield of big silicon chips is much lower than smaller ones, and the H100 is at the reticle limit. 3. Corporate greed. They know that you have money and they take it. 4. Yeah, and they have to cover the development cost with a smaller number of chips. But lately this seems not to be the case.
I love your channel and your videos. But don't think the problem with the 20% is not really the amount of energy, but rather the centralization of energy. The thing about big AI models and data centers is that they become more and more useful the bigger they get, so the data centers will somehow be centralized, so we will get problems with local peaks in the grid and consumption forecasting for buffering demand. Please take this as an open question.
Yeah, the possibility of multi-GW installed power in a single facility is kind of hard to imagine, although it's basically like a power plant in reverse. The biggest NPP can generate 8GW of electricity, which means it has the interconnections to export 8GW, and has the cooling infrastructure to dissipate 16GW of waste heat. An 8GW datacenter might look very similar, just the direction of electricity changes But it's also important to distinguish what exactly centralization is good for. It's good for training a base neural model, which is likely impossible to do in a distributed fashion, but the inference can be distributed, and the fine tuning too, so I think gigawatt scale facilities will be rare
Your break down of the issues and variables is great for normal people to see. Thanks. Could you analyze water use by data centers? Also, where the heat goes after it leaves? Thanks for this.
So far the most powerful AI data centre is the Colossus from xAI at 150MW. I think we're still a few years away from GW class AI. Great modelling, really liked how you checked different scenarios at the end.
I like that you define the question you are answering: will AI crash my electricity grid. For some of us, the question, will AI contribute to global warming, is also important.
It looks like we will use up savings in our use of electricity and use of fossil fuels for electricity through the energy demands of data centers and crypto. Is that what you are showing there?
My guess - no, not unless we have another breakthrough as large as the Transformer. LLMs are limited by it's context length and we haven't found a good way around it. We can do tricks, but tricks have limitations. Either a hardware or software revolution has to happen. Also we (including me) sees the power consumption of AI as a serious problem and is doing research to get that power DOWN. Current generation hardware is still in the research and low volume phase. There's a lot of hard nut to crack. I hope we'll get there soon. But also yeah.. AI uses a lot of power. I am considering solar so I can keep my research systems at home running 24/7. Can't compete with Nvidia yet with chips made from older generation processes.
I heard that some companies are getting around the context limitations by adding their proprietary data to the training set and retraining I could imagine a future where your personal AI assistant has enough of a context window for about a day, and everything from the day gets added to your personal training set and gets retrained in the cloud overnight
Can you make a Video about a Robotaxi Fleet of 10 Million Teslas starting operation in 2027. Basically over night after regulatory approval in the EU. How would that affect the electricity demand in Europe and the kilometres driven in electric cars?
I have a video about energy demand in the net zero future, and that covers the conversion of each combustion sector to electric, including oil for road transport 👉 th-cam.com/video/8Y7_GyWgh-o/w-d-xo.html It turns out that electricity demand isn't actually going to increase that much, since most of the energy used by the combustion economy is usually wasted as rejected heat. Most of the energy disappears, and only a minority is found as new electricity demand
The big question mark in this equation is CAPEX. Nvidia might charge $40k, but the BOM is more like $3k-5k and that is in a market where Nvidia has probably bought components and fab capacity at a premium to meet demand, rather than optimize their cost. This isn't just true for Nvidia, but the entire data center. Just for fun assume a total CAPEX of $5k/kW. At 50% utilization that gives me $2k per year or including energy $2.5k per year. At your revenue assumptions this translates to 2k TWh/year or 83% or current consumption. The problem is that the error margins here are absolutely huge. I really have no idea if AI actually has the ability to generate $150/capita/month in the next 10 years or if price per kW really could come down to $5k.
It feels illegal to have access to this type of information for free. Please allow monetization through memberships in your channel. And please don't stop making videos!
The energy consumption of Bitcoin is a constant over time, regardless of how many transactions occur You can certainly calculate it backwards to see what the average energy consumption per transaction turned out to be, but the equation is different when you look forward
@@grafity1749 The energy consumption decreases when halvings happen, but increases in correlation with price appreciation per coin / energy prices. Right now transaction costs $1/tx so clearly this too low to cover the electricity expenditure. The number of transactions per Watt is meaningless because transaction count per unit time is not directly related to value of bitcoin! 1. There are potentially thousands of transactions on exchanges and other blockhains before that one final transaction is executed on the bitcoin network. 2. A lot of bitcoin energy usage originates from anticipation of the coin price appreciation - which can be explained by arbitrage (in time) on inflating Fiat currencies - which people unsurprisingly capitalize on. - How this works in practise? - E.g. if you give people 1 Trillion dollars (e.g. from 2020 in the US, but globally it is obviously more), some of it will be utilized on food (i.e. plants grow from sun) as usual, but a lot of it gets to stocks (companies have extra money, pay employees, lawyers, they food - 100Wh per worker - food grow from the sun) and then some goes to bitcoin (which uses energy sources in one form or another.). The point is that the distributions and channels of how energy flows from source (sun, fossil, nuclear) to the people changes, when certain pressures are being applied (e.g. big money "printing" powerful governments). Bitcoin is one such channel but with very distinct energy flow path! (Unlike companies where energy usage is very fuzzy, albeit also high.) If the currencies were not inflated almost certainly bitcoin energy usage would be lower and TVs would be cheaper. 😀
The original thumbnail had a low click-through rate, so I'm testing out the test&compare feature
If you saw a dumpster-grade laser-eyes Jensen meme, I apologize, but we serve the algorithm!
The idea of a world that doesn’t lose up to 90% of the original energy content of fossil fuels to heat loss at the multiple steps along the extraction, refinement, and use pathway is pretty incredible.
I was glued to my screen!!!
Thanks for understanding & respecting units 🥰
🥰🙌
Awesome video! The +2% per year seems reasonable.
However what you have missed is the possibility of on the edge inference with 1 billion robots, which do have enough power to burn.
To your question of GPU prices:
1. The H100 has HBM memory instead of GDDR memory, which is ~6-8x more expensive for the same size.
2. The yield of big silicon chips is much lower than smaller ones, and the H100 is at the reticle limit.
3. Corporate greed. They know that you have money and they take it.
4. Yeah, and they have to cover the development cost with a smaller number of chips. But lately this seems not to be the case.
Jevons is gonna get your model ;). Energy = wealth now more than ever with AI. This is why it’s going to grow faster than we expect.
I love your channel and your videos.
But don't think the problem with the 20% is not really the amount of energy, but rather the centralization of energy. The thing about big AI models and data centers is that they become more and more useful the bigger they get, so the data centers will somehow be centralized, so we will get problems with local peaks in the grid and consumption forecasting for buffering demand.
Please take this as an open question.
Yeah, the possibility of multi-GW installed power in a single facility is kind of hard to imagine, although it's basically like a power plant in reverse. The biggest NPP can generate 8GW of electricity, which means it has the interconnections to export 8GW, and has the cooling infrastructure to dissipate 16GW of waste heat. An 8GW datacenter might look very similar, just the direction of electricity changes
But it's also important to distinguish what exactly centralization is good for. It's good for training a base neural model, which is likely impossible to do in a distributed fashion, but the inference can be distributed, and the fine tuning too, so I think gigawatt scale facilities will be rare
Your break down of the issues and variables is great for normal people to see. Thanks. Could you analyze water use by data centers? Also, where the heat goes after it leaves? Thanks for this.
So far the most powerful AI data centre is the Colossus from xAI at 150MW. I think we're still a few years away from GW class AI.
Great modelling, really liked how you checked different scenarios at the end.
These AI cooling plants 00:16 look like starcraft 1 supply depots.
I like that you define the question you are answering: will AI crash my electricity grid. For some of us, the question, will AI contribute to global warming, is also important.
It looks like we will use up savings in our use of electricity and use of fossil fuels for electricity through the energy demands of data centers and crypto. Is that what you are showing there?
My guess - no, not unless we have another breakthrough as large as the Transformer. LLMs are limited by it's context length and we haven't found a good way around it. We can do tricks, but tricks have limitations. Either a hardware or software revolution has to happen.
Also we (including me) sees the power consumption of AI as a serious problem and is doing research to get that power DOWN. Current generation hardware is still in the research and low volume phase. There's a lot of hard nut to crack. I hope we'll get there soon.
But also yeah.. AI uses a lot of power. I am considering solar so I can keep my research systems at home running 24/7. Can't compete with Nvidia yet with chips made from older generation processes.
I heard that some companies are getting around the context limitations by adding their proprietary data to the training set and retraining
I could imagine a future where your personal AI assistant has enough of a context window for about a day, and everything from the day gets added to your personal training set and gets retrained in the cloud overnight
Can you make a Video about a Robotaxi Fleet of 10 Million Teslas starting operation in 2027. Basically over night after regulatory approval in the EU. How would that affect the electricity demand in Europe and the kilometres driven in electric cars?
I have a video about energy demand in the net zero future, and that covers the conversion of each combustion sector to electric, including oil for road transport 👉 th-cam.com/video/8Y7_GyWgh-o/w-d-xo.html
It turns out that electricity demand isn't actually going to increase that much, since most of the energy used by the combustion economy is usually wasted as rejected heat. Most of the energy disappears, and only a minority is found as new electricity demand
The big question mark in this equation is CAPEX. Nvidia might charge $40k, but the BOM is more like $3k-5k and that is in a market where Nvidia has probably bought components and fab capacity at a premium to meet demand, rather than optimize their cost. This isn't just true for Nvidia, but the entire data center. Just for fun assume a total CAPEX of $5k/kW.
At 50% utilization that gives me $2k per year or including energy $2.5k per year. At your revenue assumptions this translates to 2k TWh/year or 83% or current consumption.
The problem is that the error margins here are absolutely huge. I really have no idea if AI actually has the ability to generate $150/capita/month in the next 10 years or if price per kW really could come down to $5k.
Some datacenters are buying natural gas generators so they need gas pipeline instead of electrical grid
I have the view that AI will have to get much more efficient in both power and cost to grow to ubiquity.
yooo you kinda sound like the guy from Subject Zero...
It feels illegal to have access to this type of information for free. Please allow monetization through memberships in your channel. And please don't stop making videos!
I think mining for cryptos, especially Bitcoin will be a much bigger problem. Alone one transaction needs 800 kwh of electricity.
The energy consumption of Bitcoin is a constant over time, regardless of how many transactions occur
You can certainly calculate it backwards to see what the average energy consumption per transaction turned out to be, but the equation is different when you look forward
@@SizeMichael but i think the overall consumtion also increased. How can that happen?
@@grafity1749 The energy consumption decreases when halvings happen, but increases in correlation with price appreciation per coin / energy prices. Right now transaction costs $1/tx so clearly this too low to cover the electricity expenditure. The number of transactions per Watt is meaningless because transaction count per unit time is not directly related to value of bitcoin!
1. There are potentially thousands of transactions on exchanges and other blockhains before that one final transaction is executed on the bitcoin network.
2. A lot of bitcoin energy usage originates from anticipation of the coin price appreciation - which can be explained by arbitrage (in time) on inflating Fiat currencies - which people unsurprisingly capitalize on.
- How this works in practise? - E.g. if you give people 1 Trillion dollars (e.g. from 2020 in the US, but globally it is obviously more), some of it will be utilized on food (i.e. plants grow from sun) as usual, but a lot of it gets to stocks (companies have extra money, pay employees, lawyers, they food - 100Wh per worker - food grow from the sun) and then some goes to bitcoin (which uses energy sources in one form or another.).
The point is that the distributions and channels of how energy flows from source (sun, fossil, nuclear) to the people changes, when certain pressures are being applied (e.g. big money "printing" powerful governments). Bitcoin is one such channel but with very distinct energy flow path! (Unlike companies where energy usage is very fuzzy, albeit also high.)
If the currencies were not inflated almost certainly bitcoin energy usage would be lower and TVs would be cheaper. 😀