Meta Announces Llama 3 at Weights & Biases’ conference
ฝัง
- เผยแพร่เมื่อ 8 พ.ค. 2024
- In an engaging presentation at Weights & Biases’ Fully Connected conference, Joe Spisak, Product Director of GenAI at Meta, unveiled the latest family of Llama models, Llama 3.
Highlighting a significant milestone in AI development, the Llama 3 models, including the impressive 8 billion and 70 billion parameter models released during the conference, along with a glimpse into the future with a 400 billion parameter model still in the works.
Joe shared insights into the training processes and alignment of Llama 3, which now ranks as the top-performing model in the open weights category on the MMLU, GSM-K, HumanEval benchmarks.
Weights & Biases is proud to support our customers such as Meta as they push the boundaries of AI, to learn how to fine-tune your LLMs using torchtune and Weights & Biases, start here: wandb.me/torchtune
Timestamps:
00:00 Introduction
03:05 Overview of Llama at Meta
05:59 Introducing Meta Llama 3
7:04 Advancements in Llama 3: Training and Data Scale
10:02 Benchmarking Llama 3 Performance
14:01 Enhancing Model Safety and Red Teaming
16:23 Expanding the Ecosystem and Future Directions
23:00 Closing remarks: Future plans for Llama models, and an invitation to use Meta's Lama 3.
#MetaLlama #ArtificialIntelligence #AITrends #TechInnovation - วิทยาศาสตร์และเทคโนโลยี
My favorite fact from this is that the smarter the model, the more it violates rules. Just like us :)
Very true ! People who are way smarter on tax laws are the one who violate most , innocent people pay more than what they are supposed to etc . Same goes with many other laws
Or, the rules it uses instead of the rules we assumed are different.
Thanks for this W&B
Our pleasure!
I really enjoyed this. Thanks
Glad you enjoyed it!
Congratulations!
There's a universe where Joseph Spisak is Mark Zuckerberg's brother. Oh, and nice presentation. Wonderful work they are doing at Meta AI.
so all those supervisor/safeguard models are only utilized during training? i mean, once the weights of llama3 are out, there is no safeguard network between user and inference engine right?
I'm sure they have a safety model that tries to review every request and catch some negative responses.
How much did it cost to build, including hardware and engineering costs?
What will be the SQLite of LLMs, with capability for local use? Llama?
I think he could have said "ridiculous" a bit more often
vin diesel!
Summary: Safety and size. The end.
a few hours go by...llama 3 no longer SOTA
That's why they open source it. They let the community figure things out and iterate. For Meta LLM is just a tool and not a product on itself
Wait what is sota now?
@@SkepticButOptimist "state of the art"
Which model is sota?
I think it's supposed ro be either phi or sensenova, neither of which are released @@JeiShian
How silly is to redteam a model which you control the training data to check for bioweapons capabilities. How stupid should you have to be? Isn’t easier to run a search on the data 😂😅
I’m glad they saw how useless they made codellama 😂, it was waaaay overly aligned