Anthropic’s new 100K context window model is insane!
ฝัง
- เผยแพร่เมื่อ 8 ก.พ. 2025
- Anthropic released a new LLM with a 100K token context window. In this video I'll explain what this means and we look at a demo.
Learn more about Anthropic's new model: www.anthropic....
Get your Free Token for AssemblyAI👇
www.assemblyai...
▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬
🖥️ Website: www.assemblyai...
🐦 Twitter: / assemblyai
🦾 Discord: / discord
▶️ Subscribe: www.youtube.co...
🔥 We're hiring! Check our open roles: www.assemblyai...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#MachineLearning #DeepLearning
You literally picked THE PERFECT podcast to summarize. That's literally the one I've been trying to find a time window to digest (having a 2 year old makes this extremely difficult!)
Ausgezeichnet!!
haha glad to hear that!
Wow, spent weeks learning about embeddings and this throws it all out the window. 😂
Embeddings will always be useful tbf
@@absta1995 I totally agree with you, Actually, embeddings can be used on their own for Q&A systems. Not as chatty as LLMs but much, much cheaper, as ingesting 100k tokens in a single prompt will surely be costly.
haha
Still useful for over 100k tokens.. for now 😁
No, no. Throw it *in* the window
In the context window 😊
Great video, this model looks so powerful! Thanks for sharing!
thank you!
Amazing, what a time to be alive!
Great walkthrough/explanation and video format as usual!
thank you!
Pretty incredible the speed of progress here
Great demo - thx!! Best from Hamburg
how expensive is the claude-v1-100k model?
So the advantage of Claude over e.g. OpenAI with LangChain are: 1) I don't have to summarize parts of larger documents to later retrieve the ones deemed relevant for answering a given question and thus 2) I don't have to worry about any kind of Vectorstore. Did I get this right?
When using Claude, as the Input prompt contains the large text corpus, I'm feeding a lot more tokens during inference thus paying more?
sounds correct
It's correct conceptually and if you want to use personally to summarize papers and books for example. But in practical terms calling the API recurrently with 100k tokens it's not feasible cost wise for production arquitetures.
@@rafaelfigueroa2479 hopefully in the future tokens cost becomes dirtcheap that this would be no issue.
Hello, the video is very good. I would like to ask if claude's api has similar SYSTEM prompts as in gpt3.5？Or there's a strong system of cue weights to keep him from forgetting to give him an identity or a situation？
Do i need to always send the whole data[text] , always when doing prompt engineer
How long before you create an agent that listens to your youtube videos and then responds to the commenter questions that were already answered in your video ?
That section about putting 100K tokens into context was eye opening, that means you could put the original 3 Star Wars scripts and ask it to write a whole sequel??
yes!
@@AssemblyAI No, because the output is still constrained to like 2k tokens.
@@computerex so run it several times, output is produced sequentially anyway as I understand. It might get a bit expensive but likely cheaper than George Lucas at least.
Man, this is crazy
Does it work just in the chatting interface without calling API now?
What was the response time of the prompts you demoed?
Has Claude improved over the past month? Last time I tested it, GPT4 was far superior. Little point switching unless the underlying Claude system has also improved. I'd rather use embeddings with a superior GPT4 model, than 100k context in mediocre Claude model
HI @AssemblyAI Excellent video as always! I just had a question is Anthropic API free to use or do we need to buy a subscription for this model?
no it's not free
Yes, but how much does it cost if for each prompt we have to send the 100,000 tokens of the document to the API ? It seems quite expensive to use ...
Starting from 1.60 to 33 usd/ million tokens
@@baggebilla hopefully, price for LLMs use will go down as the technology advances and we will even have the ability to do it locally for free with performant models on one's laptop...
How much money does it cost to feed it 100k tokens?
Hi,
1- Do I have to pay for the API key?
2- Can this read with pdf files? Thanks
If it can't read PDF's directly, there are plenty of services out there that will, for example, convert PDF's to Markdown etc.
@@pmarreck thanks for the reply. Do I have to pay for the API?
@@yusufkemaldemir9393 of course lol, inference isn't cheap. they did say their pricing is similar to their standard pricing rn
I requested access, got accepted, and then nothing. No email, no web page. Nada. Fail. Not ready for prime time.
Does this get better if you give the same context multiple times?
I don't think so
cool!
there is a research paper showing 2M context size is possible.
This model is probably just the beginning
Finally we'll get the ending for the GoT novels!
the 2M context size is not the same as what's happening here. The RMT paper just showcases the ability to retrieve facts and do very simple reasoning tasks (uses 1 or 2 facts at most and even with just that the performance drops quite a bit), while claude is claiming to be able to properly synthesize the data throughout the whole 100k context windox
The link is dead?
now it's live!
@@AssemblyAI - Thank you. It works.
sounds amazing, but sorry, neither url exists nor code works as is.
Please stop over-using the word "insane" when describing an AI.
We may need it someday.
ok maybe you're right ;)
still no one can actually use it thou
Is the code available for free @AssemblyAI?
I would love the code too. I use Claude a lot more than ChatGPT and have been happy with the results.