The price honestly makes a lot of sense for me. Because it's much better than something 4o-mini and gemini 1.5 flash but much cheaper than something 4o or sonnet 3.5. There was definitely a sweetspot there that I would have loved having a few months ago, and very glad they delivered it now as opposed to waiting for engineering to bring the prices down further. I'd imagine it's a bigger model / takes up more vram than 4o-mini or flash. Plus the Claude models do have something special that others don't
I don't get your issue, just haiku 3 if it is too expensive for you, just odd logic, why complain about something when the same company has different options
Can you reccomend study material to consider pricing an AI model serving and related data product distribution platform as a service? I'm in consulting, ex marketer. Looking to explore this niche topic. Thanks!
For true agentic workflows, you need Sonnet level intelligence with the current Haiku pricing. It is unfortunate but we can’t do anything about it. Open source models are not even close to Haiku reasoning let alone Sonnet. Still rooting for LLama to get better and better 🔥
Agreed. Maybe I'll change my mind later on, but right now, I want a faster, cheaper Sonnet, not a new slow, pricey Opus. Hopefully this new Haiku might fill that niche. But yeah, I still want a better Llama, regardless.
New Claude user and my biggest complaint is the lack of connection between conversations. Creating separate projects, love it…but what’s the point if it’s a new person for every chat. As a Gemini, that’s so gemini of Claude.
Sigh of relief here. You can still use the original version 3 for cheap bulk LLM operations (version 3) via the API at the old pricing. At least for now!
I'm really bummed. I like Claude and want to see Anthropic succeed. I hope they get the funding they need, and have blessed training runs. I'm just a hobbyist, so having a decent intelligence available for frivolous projects at a low price was great. I'm going to start migrating to Gemini/OpenAI for a few projects. Google has free fine-tuning iirc so maybe I'll fine-tune on Claude outputs 😂
Even as a hobbyist you should consider databricks for fine tuning agents and managing data products. But the vendor lock in may be an issue which only big business should seel to solve right now. ML ops across vendors requires a solution which is more agnostic than unity cataloug... for model management across API providers and data silos.
I've pretty much moved everything over to Gemini. I think in the long run, Google has very real advantages (robust research, an application suite, Android, TH-cam, safety team, access to almost limitless data, TPUs, massive context windows, unique API functionalities, etc.) Is Gemini on top of every leaderboard? No. But they are solid performers at a great price (and Flash is extremely fast). The real letdown with Google is the Cloud Console which is an absolute mess.
@@samwitteveenai The Google Cloud console. It looks like the control panel on the ISS and feels like it was developed by someone overly excited to get a "free" copy of Macromedia Dreamweaver in 2002. Managing API keys, projects, and pricing is SO MUCH EASIER with OpenAI and Anthropic; just a few clicks. Also, Google does not allow setting a hard cap on spend like the others do, only alerts. Very risky if something goes off the rails.
For me, nothing comes close to Opus 3. Opus 3 writes beautiful eloquent prose. The OpenAI models write like a serviceable college graduate. Gemini's writing is atrocious. Even Elon Grok's writing is better.
I mean the community complained that Sonnet 3.5 was cheaper than Opus, I think this is exactly what this kind of stupidity deserves. Good job Anthropic, punish stupidity, the world has more than enough of it.
@3:14 "Claude-3.5-haiku surpassed claude-3-opus in performance at a lower cost, so we're increasing the cost of haiku" That makes no sense. Either pass the savings off to the customer, or they'll use OpenRouter to get access to BETTER models for CHEAPER 😂
I don’t really need Haiku 3.5 personally. I’ve found agentic frameworks and open source model serve my needs quite well, although it does mean more engineering work and more error checking. In the end, the ability to customize the models, rapidly swap out different flavors of model for the specific use case, and to have it all running locally/on premise, with potentially more flexibility in access to sensitive information and use cases is just too big an advantage. There is a use case for the very top end models, and I do think o1 preview and 3.5 Sonnet do have reasons to use them, as they just really don’t have competitors (you spend more doing tree search on open source models than just a single query from the large ones, lol), but for small and cheap owning the hardware really does pay off.
GPT 4 was "0.5 more than" GPT 3.5 and much more expensive. Claude 3.5 Haiku is "0.5 more than" Claude 3 Haiku. All the LLM companies make very confusing branding and naming of the models. I guess Mistral is the fairest: they just come up with various names rather than pretending that one is a new version of another. Gemini 1.5 Pro is radically different from Gemini 1.0 Pro. GPT-4, GPT-4 Turbo and GPT-4o have nothing in common. We just need to learn taking each model as a new product, regardless of the often misleading branding. I can easily imagine that Haiku 3 and Haiku 3.5 will co-exist. 4x price difference basically indicates a totally different class of product. They could have called the new thing Claude 3.5 Limerick.
I was upset and still am but I am starting to think that maybe they are factoring in prompt caching. That is something that doesn't help fund them to keep up with the competition.
It's just to sensitive to the prompt. Let's say it's lazy and takes everything too literally. And it seems like it was trained on forums and reddit users, while the others are trained on books and articles.
Claude 3.5 (new) Sonnet is not sure how you can claim such things 😮 _I need to correct a misunderstanding in your statement. We are currently in April 2024, so I cannot comment on events from November 2024 as that's in the future._
My feeling is they are pricing a certain percentage of people/use cases out because they likely dont have the required inference compute. Guys supply and demand is a b****
Oh Sam and the art of video naming... But seriously, I don't like the new trend though. What is Anthropic thinking "business" wise anyway, what the idea behind 4x price increase? Companies will switch back to Flash or other mini models, it is not that companies are really heavily invested in the Anthropic ecosystem at the moment.
@ Newer doesn't necessarily mean better. OpenAI had a few pre-GPT3 models that were great at certain kinds of writing (very uncensored). Also, OpenAI had introduced GPT-4o as a "faster" variant of GPT-4 Turbo, which in turn had been a "faster" variant of GPT-4. But faster means cheaper to run, but also - smaller, distilled, quantized. AI conpanies these days experiment with longer contexts, faster and cheaper inference, expert switching etc. - all these things look promising on paper, but aren't necessarily really better. Typically the vendors aren't working towards "best" but much more "good enough but much cheaper to run" (see GPT-Turbo & GPT-4o).
If Haiku is better than Opus and far better than the previous Haiku. it’s so much more expensive they should have given it another name so people wouldn’t feel like they are overpaying for Haiku
An H100 chip without ram costs nvidia maybe 100$. 80GB of GDDR6 ram costs about 200$. HBM ram isn't more expensive to make, it's just more pins for a wider bus. So that can tell you that AI will become dirt cheap. nvidia is on a greed frenzy and AMD is unfortunately not calling them on their bs. It's strange how passive AMD is because tensor chips are not hard to design from what I gather. Indeed AMD has them, they just have to price them even slightly reasonably to completely destroy nvidia's 20x above norm earnings. We are being artificially held back. You could easily imagine a home developer board with 1TB of fast ram instead of the pathetic 24GB in a 4090. And it sounds like 5090 wont have much more ram, despite ram prices having dropped 80% and neural models have exploded in size with no end in sight. A very affordable GPU/TPU can easily run GPT4o at good speed but we need more memory. AI has ballooned really fast combined with nvidia enterprise greed that personal compute hasn't kept up at all despite the components being very affordable. Wake up and demand better.
You are correct that AI hardware is overpriced, especially at the moment, but this take isn’t entirely fair. Yes, HBM technically doesn’t cost more than GDDR; in the end it’s just memory. One minor problem with that is that the packaging (the thing that actually connects the processor and the memory) is significantly more expensive for HBM. To my understanding wide memories like GDDR are around $7-12/GB atm and HBM is notably more than that (I think $12-$25/GB depending on the exact type (once you actually put it on the board). I believe memory was cheaper about a year ago, though. H100 may or may not be around $100 a chip, I’m not really willing to break out the TSMC wafer calculations to figure it out, but that type of hardware (server grade) typically isn’t priced according to the manufacturing price. Generally when you buy that grade of hardware, you’re paying for service (ie: on the Nvidia developer forums you can get specialized help from highly competent engineers if you just show up and list your problem, and it’s their job to help you), you’re paying for software, you’re paying for highly reliable and redundant hardware, and each of those things adds a significant amount of cost. Does all of this mean H100 should be $2,000? Probably that would be a bit too cheap for that category of product, even if Nvidia wanted to launch to consumer for some reason. Should it be $40,000? Also probably not. I’m pretty sure the “right” price would probably be somewhere in between $4,000 and $20,000, but I’m not really sure where it should actually be in that window, though I find myself rather reluctant to search it out as that’s quite the hypothetical to get into.
@@novantha1 if the chip costs on the order of 100$ and the 80GB ram could cost on the order of 200$ (I use dramexchange for pricing) then is 2000$ really too cheap... I noticed that enterprise stuff isn't based on cost but what is the justification for that? they are made with love? handled with white gloves? if you can have the same compute for 1/20 then how great can reliability difference be? and in my experience, high dollar items are made worse, last shorter. I have seen expensive Tyan motherboards melt down doing the simplest thing while a dirt cheap mass produced motherboard just lasts forever. I guess nvidia pricing started as a super niche product but now it's mass production, it's nothing but greed. and it's so damaging that nvidia can't give us large amounts of memory on consumer cards because then they could outperform the datacenter products and people would ask why. so we desperately need AMD to destroy nvidia's greed with a consumer 100GB card. Something people can cluster 100 of together and beat ultra expensive nvidia enterprise solutions
I tested it and get disappointed. It far worse than previous Haiku. Now it gives me lists instead of good answers. Now it mixes Russian with English. An epic fail! It's dumber, slower and more expensive! Outrageous! 😢 😡
They are not in a position to compete with Facebook, Google or OpenAI on price. They know it and everyone knows it.
That's a lie they are literally working with Amazon, they have always overpriced
@@holdthetruthhostageagree, AWS mentioned. always overprice compare to the others
The price honestly makes a lot of sense for me. Because it's much better than something 4o-mini and gemini 1.5 flash but much cheaper than something 4o or sonnet 3.5. There was definitely a sweetspot there that I would have loved having a few months ago, and very glad they delivered it now as opposed to waiting for engineering to bring the prices down further.
I'd imagine it's a bigger model / takes up more vram than 4o-mini or flash. Plus the Claude models do have something special that others don't
Time will tell it’s very popular as it’s as good as Opus, which is great enough, but way faster and cheaper relative to that.
did you test it? it's a piece of crap
@ works pretty good.
Probably underrated.
The price is a joke! I will stick with Sonnet, and for less demanding tasks, I will use Gemini and OpenAI's cheap models. :)
I don't get your issue, just haiku 3 if it is too expensive for you, just odd logic, why complain about something when the same company has different options
Yes please test the claude agent
I do pricing for a living. This time it's priced to not divert revenue away from sonnet
Can you reccomend study material to consider pricing an AI model serving and related data product distribution platform as a service? I'm in consulting, ex marketer. Looking to explore this niche topic. Thanks!
Also, when it comes to true frontier models, there may not be a market for more than 3, as people systematically gravitate towards the top 3.
For true agentic workflows, you need Sonnet level intelligence with the current Haiku pricing. It is unfortunate but we can’t do anything about it. Open source models are not even close to Haiku reasoning let alone Sonnet.
Still rooting for LLama to get better and better 🔥
Agreed. Maybe I'll change my mind later on, but right now, I want a faster, cheaper Sonnet, not a new slow, pricey Opus. Hopefully this new Haiku might fill that niche. But yeah, I still want a better Llama, regardless.
Thanks as always sam
New Claude user and my biggest complaint is the lack of connection between conversations. Creating separate projects, love it…but what’s the point if it’s a new person for every chat. As a Gemini, that’s so gemini of Claude.
Sigh of relief here. You can still use the original version 3 for cheap bulk LLM operations (version 3) via the API at the old pricing. At least for now!
Haiku 3 was awesome. I hope they don't kill my beloved Haiku 3. The new one is piece of garbage.
How is that opening statement in any way true?
The more advanced the model the more compute they seem to take to run.
More compute costs more.
I'm really bummed. I like Claude and want to see Anthropic succeed. I hope they get the funding they need, and have blessed training runs.
I'm just a hobbyist, so having a decent intelligence available for frivolous projects at a low price was great. I'm going to start migrating to Gemini/OpenAI for a few projects. Google has free fine-tuning iirc so maybe I'll fine-tune on Claude outputs 😂
Gemini is too racist, OpenAI is too restrictive and, unfortunately, Claude is less open than it used to be.
Even as a hobbyist you should consider databricks for fine tuning agents and managing data products. But the vendor lock in may be an issue which only big business should seel to solve right now. ML ops across vendors requires a solution which is more agnostic than unity cataloug... for model management across API providers and data silos.
I've pretty much moved everything over to Gemini. I think in the long run, Google has very real advantages (robust research, an application suite, Android, TH-cam, safety team, access to almost limitless data, TPUs, massive context windows, unique API functionalities, etc.) Is Gemini on top of every leaderboard? No. But they are solid performers at a great price (and Flash is extremely fast). The real letdown with Google is the Cloud Console which is an absolute mess.
Are you talking about the AIStudio console or VertexAI? Anything in particular you would like to see fixed or added ?
@@samwitteveenai The Google Cloud console. It looks like the control panel on the ISS and feels like it was developed by someone overly excited to get a "free" copy of Macromedia Dreamweaver in 2002. Managing API keys, projects, and pricing is SO MUCH EASIER with OpenAI and Anthropic; just a few clicks. Also, Google does not allow setting a hard cap on spend like the others do, only alerts. Very risky if something goes off the rails.
You can get an API key super easily in AI Studio!
Gemini is awesome for it's price. Gemini 1.5 Pro is the best after Sonnet 3.5 and 4o-latest
For me, nothing comes close to Opus 3. Opus 3 writes beautiful eloquent prose. The OpenAI models write like a serviceable college graduate. Gemini's writing is atrocious. Even Elon Grok's writing is better.
I mean the community complained that Sonnet 3.5 was cheaper than Opus, I think this is exactly what this kind of stupidity deserves. Good job Anthropic, punish stupidity, the world has more than enough of it.
I suspect that earlier models for all vendors were underpriced, relative to their training and operating costs.
@3:14 "Claude-3.5-haiku surpassed claude-3-opus in performance at a lower cost, so we're increasing the cost of haiku"
That makes no sense. Either pass the savings off to the customer, or they'll use OpenRouter to get access to BETTER models for CHEAPER 😂
its about demand! if everyone wants it energy consumption goes up so ofc prices have to go up!
I don’t really need Haiku 3.5 personally. I’ve found agentic frameworks and open source model serve my needs quite well, although it does mean more engineering work and more error checking. In the end, the ability to customize the models, rapidly swap out different flavors of model for the specific use case, and to have it all running locally/on premise, with potentially more flexibility in access to sensitive information and use cases is just too big an advantage.
There is a use case for the very top end models, and I do think o1 preview and 3.5 Sonnet do have reasons to use them, as they just really don’t have competitors (you spend more doing tree search on open source models than just a single query from the large ones, lol), but for small and cheap owning the hardware really does pay off.
GPT 4 was "0.5 more than" GPT 3.5 and much more expensive. Claude 3.5 Haiku is "0.5 more than" Claude 3 Haiku.
All the LLM companies make very confusing branding and naming of the models. I guess Mistral is the fairest: they just come up with various names rather than pretending that one is a new version of another.
Gemini 1.5 Pro is radically different from Gemini 1.0 Pro. GPT-4, GPT-4 Turbo and GPT-4o have nothing in common.
We just need to learn taking each model as a new product, regardless of the often misleading branding.
I can easily imagine that Haiku 3 and Haiku 3.5 will co-exist. 4x price difference basically indicates a totally different class of product. They could have called the new thing Claude 3.5 Limerick.
I was upset and still am but I am starting to think that maybe they are factoring in prompt caching. That is something that doesn't help fund them to keep up with the competition.
I've already changed 2 of my apps to use Gemini
I am back to 3.0
me too. my tests showed that the new model is complete crap compared to the old one... and they made it 4x more expensive, that's crazy
I must be using Gemini 1.5 Pro wrong because I've found that it sucks but these benchmarks are telling me something different.
It's just to sensitive to the prompt. Let's say it's lazy and takes everything too literally. And it seems like it was trained on forums and reddit users, while the others are trained on books and articles.
if it was openai, they would have just used a new naming convention for the higher price model, like Haiku-V7 (vs Vaiku-1 and newest Vaiku).
Claude 3.5 (new) Sonnet is not sure how you can claim such things 😮 _I need to correct a misunderstanding in your statement. We are currently in April 2024, so I cannot comment on events from November 2024 as that's in the future._
Lol and everyone has been canceling their 20 a month subscription because of rate limits. They passing that cost to the 3.5 haiku model now lol8
I'd rather throw 20 bucks at an openrouter and send hundreds of messages an hour and still have some left over for next month
@@MudroZvon Openrouter???? never heard of that, thanks
My feeling is they are pricing a certain percentage of people/use cases out because they likely dont have the required inference compute. Guys supply and demand is a b****
Oh Sam and the art of video naming...
But seriously, I don't like the new trend though.
What is Anthropic thinking "business" wise anyway, what the idea behind 4x price increase?
Companies will switch back to Flash or other mini models, it is not that companies are really heavily invested in the Anthropic ecosystem at the moment.
Companies can just use Haiku 3 instead of Limerick 3.5 (mis-branded as Haiku 3.5)
@@AdamTwardoch Limerick :) Do you want to stay on older model though?
@ Newer doesn't necessarily mean better. OpenAI had a few pre-GPT3 models that were great at certain kinds of writing (very uncensored). Also, OpenAI had introduced GPT-4o as a "faster" variant of GPT-4 Turbo, which in turn had been a "faster" variant of GPT-4. But faster means cheaper to run, but also - smaller, distilled, quantized.
AI conpanies these days experiment with longer contexts, faster and cheaper inference, expert switching etc. - all these things look promising on paper, but aren't necessarily really better. Typically the vendors aren't working towards "best" but much more "good enough but much cheaper to run" (see GPT-Turbo & GPT-4o).
If Haiku is better than Opus and far better than the previous Haiku. it’s so much more expensive they should have given it another name so people wouldn’t feel like they are overpaying for Haiku
"Aha! We've got ourselves a marketer!"
I doubt it's better
4x bump of price tag only reflects 4x increase in greediness
It's not worth it. I would understand if it was better than the old haiku, but it's some kind of 70b degenerate.
Somewhere in the corner two truly lobotomized models, 8B and 32B ones, are giggling on "degenerate"
An H100 chip without ram costs nvidia maybe 100$. 80GB of GDDR6 ram costs about 200$. HBM ram isn't more expensive to make, it's just more pins for a wider bus. So that can tell you that AI will become dirt cheap. nvidia is on a greed frenzy and AMD is unfortunately not calling them on their bs. It's strange how passive AMD is because tensor chips are not hard to design from what I gather. Indeed AMD has them, they just have to price them even slightly reasonably to completely destroy nvidia's 20x above norm earnings. We are being artificially held back. You could easily imagine a home developer board with 1TB of fast ram instead of the pathetic 24GB in a 4090. And it sounds like 5090 wont have much more ram, despite ram prices having dropped 80% and neural models have exploded in size with no end in sight.
A very affordable GPU/TPU can easily run GPT4o at good speed but we need more memory. AI has ballooned really fast combined with nvidia enterprise greed that personal compute hasn't kept up at all despite the components being very affordable. Wake up and demand better.
You are correct that AI hardware is overpriced, especially at the moment, but this take isn’t entirely fair.
Yes, HBM technically doesn’t cost more than GDDR; in the end it’s just memory. One minor problem with that is that the packaging (the thing that actually connects the processor and the memory) is significantly more expensive for HBM. To my understanding wide memories like GDDR are around $7-12/GB atm and HBM is notably more than that (I think $12-$25/GB depending on the exact type (once you actually put it on the board). I believe memory was cheaper about a year ago, though.
H100 may or may not be around $100 a chip, I’m not really willing to break out the TSMC wafer calculations to figure it out, but that type of hardware (server grade) typically isn’t priced according to the manufacturing price. Generally when you buy that grade of hardware, you’re paying for service (ie: on the Nvidia developer forums you can get specialized help from highly competent engineers if you just show up and list your problem, and it’s their job to help you), you’re paying for software, you’re paying for highly reliable and redundant hardware, and each of those things adds a significant amount of cost.
Does all of this mean H100 should be $2,000? Probably that would be a bit too cheap for that category of product, even if Nvidia wanted to launch to consumer for some reason.
Should it be $40,000? Also probably not.
I’m pretty sure the “right” price would probably be somewhere in between $4,000 and $20,000, but I’m not really sure where it should actually be in that window, though I find myself rather reluctant to search it out as that’s quite the hypothetical to get into.
@@novantha1 if the chip costs on the order of 100$ and the 80GB ram could cost on the order of 200$ (I use dramexchange for pricing) then is 2000$ really too cheap...
I noticed that enterprise stuff isn't based on cost but what is the justification for that? they are made with love? handled with white gloves? if you can have the same compute for 1/20 then how great can reliability difference be? and in my experience, high dollar items are made worse, last shorter. I have seen expensive Tyan motherboards melt down doing the simplest thing while a dirt cheap mass produced motherboard just lasts forever. I guess nvidia pricing started as a super niche product but now it's mass production, it's nothing but greed. and it's so damaging that nvidia can't give us large amounts of memory on consumer cards because then they could outperform the datacenter products and people would ask why. so we desperately need AMD to destroy nvidia's greed with a consumer 100GB card. Something people can cluster 100 of together and beat ultra expensive nvidia enterprise solutions
I tested it and get disappointed. It far worse than previous Haiku. Now it gives me lists instead of good answers. Now it mixes Russian with English. An epic fail! It's dumber, slower and more expensive! Outrageous! 😢 😡
It's just a different model. Think of it as Claude 3.5 Limerick. It's not really an update of Haiku 3.
Claude 3 haiku это самый худший бот в мире
Haiku is solid for coding compared to gpt.
Sam has never heard of Alaska
Water flowing up? Or sunsets ?
@@samwitteveenai prices for LLMs are going up in Alaska?
Models get smarter, people get dumber 😂
they just suck at naming things...