I remember the HP Autonomy fraud scandal - it was big deal, impacted badly HP and I guess precipitated the HP split. BTW, this GPU short is caused chiefly by the first Gen (that is, inferior) AI models (mainly LLM/Transformers) which require immense resources for training. Think of the 60s of the previous century when the US auto market was flooded with gas guzzling cars which worsened the impact of the oil shortages. I think the situation with GPU demand will be quite different when the next gen AI models for NLP tasks appear. On the other topic: we do not need TikTok, Temu, Shein and the likes; unfortunately, these entities are being cajoled in the US market by the Big Tech which rely on ads - Alphabet and Meta. On the VMWare topic - I hope Broadcom is not Thoma Bravo..
GPU crash?? Huh? Most LLM ‘s are taking 12-18 months to train (on GPU’s), when released they are already outdated by 18 months. Then everyone has to use GPU’s to “refine” the LLM’s (mostly redundantly on the same public data) to catch them up to date with specific data interests. Then there is the massive backlog of about 2 petabytes per enterprise that need to use GPU’s to convert their data into vector databases so AI LLM’s can be used by enterprises to access their own data, privately/securely. The limiting factor for the US, assuming the GPU demand can continue to be met, the US does not have enough electrical power generation capacity to meet the never ending amount of AI LLM training, re-training, and enterprise vectoring and refining backlog. We need to start building cross border power distribution from Canada now. We can manufacture and meet the needed GPU demand but do we have the political will to meet the required corresponding power generation demand?
Just like CPU’S, AI chip generations price will fall quickly as new generations/versions are released. Intel CPU prices will drop 10-20% in the first 6-12 months, and then fall another 20% 12-24 months, and fall further another 50% shortly after that. AI chips like the A100, H100, GH200 will be subject to this same discounting/price drops as new versions/generations of these AI chips are released. So, I guess you could say the price of Intel i7 CPU’s price “crashed” and created an excess of un-utilized inventory of Intel i7’s when the Intel i9’s became more widely available. Demand for i9’s remained high, even while i7’s were still sold as at lower price to still get work done. AI chip price is irrelevant, the cost and availability of power to train and re-train AI models with historical, current and real-time ENTERPRISE data is what will matter the most.
The key thing that literally no one talks about, is education gap/ skills gap, and most people in their 30s and 40s around the world are addicted to their social media, so most office workers just don’t function properly. The AI level, esp Gen AI can increase productivity in the work place (see JP Morgan 2024 investor day transcript), but most CEOs don’t understand tech, and most workers are too stupid and lazy to reskill and upskill. It’s a human problem not AI problem.
I wish someone would talk about specific workloads that are not being done now that can be done better.
Why can we find better models in D Wave tech. Would any new technologies be applied other than what has been described on this Cube?
I would have liked to hear a more detailed construct of the GPU crash scenario. Backup, data, etc and not just hearsay.
the best in tech!
I remember the HP Autonomy fraud scandal - it was big deal, impacted badly HP and I guess precipitated the HP split. BTW, this GPU short is caused chiefly by the first Gen (that is, inferior) AI models (mainly LLM/Transformers) which require immense resources for training. Think of the 60s of the previous century when the US auto market was flooded with gas guzzling cars which worsened the impact of the oil shortages. I think the situation with GPU demand will be quite different when the next gen AI models for NLP tasks appear. On the other topic: we do not need TikTok, Temu, Shein and the likes; unfortunately, these entities are being cajoled in the US market by the Big Tech which rely on ads - Alphabet and Meta. On the VMWare topic - I hope Broadcom is not Thoma Bravo..
We are beyond gpu and micros.
The cube will sign a $150 million podcast exclusivity deal with Broadcom audio
GPU crash?? Huh? Most LLM ‘s are taking 12-18 months to train (on GPU’s), when released they are already outdated by 18 months. Then everyone has to use GPU’s to “refine” the LLM’s (mostly redundantly on the same public data) to catch them up to date with specific data interests. Then there is the massive backlog of about 2 petabytes per enterprise that need to use GPU’s to convert their data into vector databases so AI LLM’s can be used by enterprises to access their own data, privately/securely. The limiting factor for the US, assuming the GPU demand can continue to be met, the US does not have enough electrical power generation capacity to meet the never ending amount of AI LLM training, re-training, and enterprise vectoring and refining backlog. We need to start building cross border power distribution from Canada now. We can manufacture and meet the needed GPU demand but do we have the political will to meet the required corresponding power generation demand?
Are there any articles from reputable sources tying the NVDA GPU sales to actual or reported purchases and deployments?
Just like CPU’S, AI chip generations price will fall quickly as new generations/versions are released. Intel CPU prices will drop 10-20% in the first 6-12 months, and then fall another 20% 12-24 months, and fall further another 50% shortly after that. AI chips like the A100, H100, GH200 will be subject to this same discounting/price drops as new versions/generations of these AI chips are released. So, I guess you could say the price of Intel i7 CPU’s price “crashed” and created an excess of un-utilized inventory of Intel i7’s when the Intel i9’s became more widely available. Demand for i9’s remained high, even while i7’s were still sold as at lower price to still get work done. AI chip price is irrelevant, the cost and availability of power to train and re-train AI models with historical, current and real-time ENTERPRISE data is what will matter the most.
Da cube 🧊
The key thing that literally no one talks about, is education gap/ skills gap, and most people in their 30s and 40s around the world are addicted to their social media, so most office workers just don’t function properly. The AI level, esp Gen AI can increase productivity in the work place (see JP Morgan 2024 investor day transcript), but most CEOs don’t understand tech, and most workers are too stupid and lazy to reskill and upskill. It’s a human problem not AI problem.