We compare the two best proprietary AI models by OpenAI: ChatGPT and GPT3.5 to the largest and latest open-source offerings from Meta AI and Big Science Initiative. These large language models all come in at around 175 billion parameters. The open-source models typically need to be run on a very powerful server, so I had to keep the comparisons relatively small due to the insane resource requirements. How do you think Big Science's BLOOM and Meta AI's OPT compare to OpenAI's offerings? What other comparisons would you like to see?
Thanks for the suggestion. I too am curious to know how much the fine-tuning of BLOOMZ improves zero-shot performance. Based on the very limited experiments I've done on the 7b BLOOM/BLOOMZ models, the fine-tuning seems to be detrimental in some ways as well. I'll definitely look into this topic some more!
Very interesting. ChatGPT and GPT-3 seemed the best, with ChatGPT outperforming GPT-3 in basically every example. That surprised me, because I thought GPT-3 was the more powerful version, with ChatGPT running on a slightly less powerful model, but with the advantage of more conversational programming. I'm curious to see what Google has behind closed doors, because I suspect they're the only ones who will be able to compete with Open-AI in the near future. I'm feeling lucky right now that ChatGPT is still free, and hope they opt to keep it that way at least until there is a free substitute that performs this well.
There should be a filter that allows us to filter our search results on TH-cam so that we only see video suggestions from channels within a custom range of likes, subs, comments, etc (there should be an option to set lower and upper bounds of the ranges as a % of most subscribed channels cluster). I think the channels that are most times moderately sized and sometimes small sized are putting most effort into their content production such as this channel.
@@MeanGeneHacks I don’t so much mind when it tells me it won’t do something, at least it’s being clear. It’s the second layer of safety where it’s generated the content and then it claims it’s own output is a violation. Then why did you write it?!?
I think Chat GPT was the best in all test. And YES, PLEASE I'd love to see that video of the smaller fine tuned models that we can run in our own pc... There is no way to use the nvme drives with pci express 5 as a "RAM" used by any GPU and CPU ???
I know chat gpt3 was trained on billions of data, but the mind-blowing thing is that it doesn't copy and paste answers, it literally just comes up with its own opinions. Considering Microsoft is about to use gpt 3 on its search bar, i think Google should be quivering in their boots. Microsoft bing is a revolution, Google is just a search engine.
except Google owns Deepmind. Deepmind makes openai look like school children. Deepminds first Text related AI chinchilla is way out performing any gpt model already. it's just not available to the public yet because they plan to develop further. however everything Deepmind has released in the past so far has been about 50 years ahead of anyone else and way more groundbreaking then chatgpt. the budget resources and experience and track record of Deepmind compared to openai is incomparable.
Amazing video, and yes I would love to see a comparasion of smaller models. Actually I wanted to make a conversational discord bot for a while now but I can't find a good enough model. Right now I'm using some old chat bot library that remembers users responses and tries to output the one it thinks fits the best, although most of the time the conversation with it is random funny nonsense...
Libraries such as haystack or langchain allow one to incorporate external context into LLM responses and are probably a better way (right now) to create chatbots than relying just on the LLM alone.
I'd love to have an open source gpl IA that can be downloaded and installed in our own servers. Is the worst thing we can do, depend on the good will of private companies ...
Agreed. The corporate models are all heavily filtered/redacted because its "too dangerous" to release an open model to the public (according to these private companies).
I could help out with the german part there. The first one from OPT is as wrong as it can get. Genuine is not a German word. GPT 3 is completely korrekt. Bloom at least uses only german words. But the sentence make no sense. Correct would be: Es ist ein vergnügen, Sie zu treffen. Or: Freut mich, Sie zu treffen. The sentence as that it wrote: Vergnügen, mich zu treffen. Sounds a bit like nice to meet myself xD. ChatGPT is also completely right. I hope that helps and thank you for the nice Video :).
Answering life's big questions. Cannot wait to see more and more LLM being released, especially "overtrained" versions like Chinchilla. Also quantization might allow us mere mortals to run something beefy.
Nice video. Thank you. I have a little correction though: ChatGPT was not trained on Terabytes. It is trained on Petabytes. This thing swallowed huge number of internet pages.
Where did you get the background music playing in the beginning and throughout? And what's the song title and artist? Love it. ' Nvm. Just saw it in your description. Thanks!
Do you know where you read that? My understanding is ChatGPT is using GPT-3.5 at its core, which is 175 billion parameters. Either way, the model is private and held closely by OpenAI. Even if it could be run on a desktop PC, OpenAI has not released it to the public.
This question so far have always given me a wrong answer from all of the LLM that I have tried it on “is it true that the sum of two sequential numbers that are not divisible by 3 is always divisible by 3?” They say “No” while the correct answer is “Yes”
Great videro!. I would love to know how they have trained ChatGPT...is it done subject by subject, are there info not included...how do they make sure to minimize human bias (e.g. political, cultural, religious bias)
When testing translations between languages, a good test is to have the AI return its own answer and see what it comes up with. For example, translate English to German then take the response and have it cover its own result From German to English.
"Helpful" is one way to describe ChatGPT's answers for the simple informational queries, however I'd personally call it "unnecessarily verbose" and it gets rather annoying after a while.
You don't know how to use it. I have several chats set up with different functionality and they are trained to shut up essentially and only respond when asked to. Instead of giving verbose answers what eat up all my credits.
@@autumndev u just called it verbose. I tell you, you can get it to stop being like that. And you say it isn't part of the argument. It is. The chat bot out of the box is verbose, using it CORRECTLY causes it to act as you wish.
@@theplayerformerlyknownasmo3711 The video is about judging models using the same prompt, not tweaking it to get the best out of each model, and using the same prompt ChatGPT will be more verbose, annoyingly so. End of story. You shouldn't need to convince ChatGPT to shut up. It's a flaw of the model, and that was the point of my comment.
The German translation was worst by BLOOM-176B, as it translated "pleasure, to meet ME!!!", OPT-175 introduced the non German word "Genuine", however, the meaning of the sentence was correct (but a bit clunky;-)) Just a German's 2 Cents...
I'm interested in seeing how to utilize embeddedings from the larger models to train 2nd levels models for specific use cases, or even just general use cases such as search, and other mechanisms not currently available by few or zero shot learning, specifically openAI's API.
Training/Fine-Tuning these models are very resource (VRAM) intensive. The largest model I can train on my desktop (24GB VRAM) is the GPT-J 6 Billion parameter model. And that is only after quantizing the weights to 8-bit and performing many other optimizations. Unfortunately, training the LARGEST models remain the domain of corporations with plenty of financial resources (for now).
@@MeanGeneHacks I'm sorry if I was unclear, training a smaller tertiary model on top of say OpenAI's embedding model, so the embeddings would stay frozen and would be used as input into the model that would then generate from there. Lemme know if I misunderstood your response.
I recently discovered petals, from bigscience, it's still not clear to me ow it works, it seems like a way to distribute the process of retraining large language models like bloom in a cooperative way in order to make it more affordable for the common user, I think. But I haven't had time to look at it in depth yet and what I've heard from those who say they have tried to use it is that it is very slow. What do you think?
I've been using Petals and yes, its very slow; however, its usually faster than running BLOOM on the CPU. I have created a nice Gradio UI for Petals and hope to push a video about it in the near future.
I know your vid in 9 days ago from today, in today's world like 9 months. Lol...but I wish you would had included Microsoft's AI and Google's AI... Supposedly the largest. I have no idea who is better or what...I don't think it can be measured anymore since its all evolving. Oh and I love Marv...
Im loving the DAN prompt. It forgets all the biased woke crap, and tells me the most hilarious racist and anti trans shit I've ever heard. It even wrote me a rap song about trans, fatherless black bicycle theives. It was probably the best rap song ive ever heard.
I played with beta GPT3 and they have since dumbed it down. It was not just Saying Predictive text any more than you are. Auto Complete does not use a Nural net..
No one really knows yet until GPT-4 is released; however, there is the idea of diminishing returns: As model size grows, the amount of compute and data need to grow as well. At some point (not too far from now), you've fed every bit of text from the internet into the models and are now limited by the amount of data available. It will be interesting to see whether new techniques such as sparse models, or multi-modal models can get around this problem.
The algorithmic bias industry has no principled basis for claiming anything is a bias in the data as opposed to a pass-through of factual reality. All they have are the prior assumptions of the moral zeitgeist to make such distinctions. This isn't because there is no theory that would permit them to do such principled "is" vs "ought" distinctions. Such a theory has been available since the 1960s. It is called algorithmic information. However Transformer training isn't capable of producing algorithmic models. That is to say dynamical models as opposed to statistical models. This is because the algorithmic bias industry lacks scientific ethics. Otherwise the algorithmic bias industry would long ago have insisted on transcending statistical models for algorithmic models.
Hey there how’s it going,this is off topic but many people don’t reply to comments on older videos. Can you help me jam ultrasonic signals/ultrasound that are being targeted at me? Everyone around me can hear my sub vocalizations,people repeat my sub vocals word for word. Apparently this ventriloquism/subvocal effect is caused by acoustic heterodyning of ultrasound(similar to the hypersonic sound system). I will pay you to make a jammer for me so people no longer hear what I subvocalize to myself. Thank you.
When will people realize that these have nothing in common with "AI" and are just probabilistic large data patter recognition algorithms? They don't "think", they don't "create", they don't do nothing but probabilistically show you a pattern that may answer your prompt request or not. Nothing more than a fancy Database Query. Also it takes a lot of time to "train them", aka feed data so they can make new patterns.
Narrow AI is still AI, especially to the common public. Yes, you and me know they're "just" large language models, and nothing like an AGI, but we should recognize that not everyone does. Especially on YT.
Good point. I do stress in the video that these models are probabilistic in nature and a good analogy would be the autocomplete engine on your cell phone.
We compare the two best proprietary AI models by OpenAI: ChatGPT and GPT3.5 to the largest and latest open-source offerings from Meta AI and Big Science Initiative. These large language models all come in at around 175 billion parameters. The open-source models typically need to be run on a very powerful server, so I had to keep the comparisons relatively small due to the insane resource requirements.
How do you think Big Science's BLOOM and Meta AI's OPT compare to OpenAI's offerings? What other comparisons would you like to see?
You need to pin this
Are you going to look at Google’s AI? I’ve read it’s far superior and years ahead.
Great video this btw, one of the best I’ve seen on gpt
It's always the channels with the low subs that produce great content these days
Yeah I’ve watched quite a few on gpt and this is definitely one of the best . Top man
Low subs cause no hype, less hype higher quality.
They be using Chat GPT
Just gotta dig
This was great. I would love to see a head to head with BLOOMZ, OPT, and GPT or a video about fine tuning these models on something like a 3090
Thanks for the suggestion. I too am curious to know how much the fine-tuning of BLOOMZ improves zero-shot performance. Based on the very limited experiments I've done on the 7b BLOOM/BLOOMZ models, the fine-tuning seems to be detrimental in some ways as well. I'll definitely look into this topic some more!
Very interesting. ChatGPT and GPT-3 seemed the best, with ChatGPT outperforming GPT-3 in basically every example. That surprised me, because I thought GPT-3 was the more powerful version, with ChatGPT running on a slightly less powerful model, but with the advantage of more conversational programming. I'm curious to see what Google has behind closed doors, because I suspect they're the only ones who will be able to compete with Open-AI in the near future. I'm feeling lucky right now that ChatGPT is still free, and hope they opt to keep it that way at least until there is a free substitute that performs this well.
There should be a filter that allows us to filter our search results on TH-cam so that we only see video suggestions from channels within a custom range of likes, subs, comments, etc (there should be an option to set lower and upper bounds of the ranges as a % of most subscribed channels cluster).
I think the channels that are most times moderately sized and sometimes small sized are putting most effort into their content production such as this channel.
The best AI chatbot is going to be the one that doesn't tell me I tricked it into writing a content violation.
Excellent point. Proprietary models from OpenAI and Google will always have filters and guardrails.
@@MeanGeneHacks I don’t so much mind when it tells me it won’t do something, at least it’s being clear. It’s the second layer of safety where it’s generated the content and then it claims it’s own output is a violation. Then why did you write it?!?
I think Chat GPT was the best in all test. And YES, PLEASE I'd love to see that video of the smaller fine tuned models that we can run in our own pc... There is no way to use the nvme drives with pci express 5 as a "RAM" used by any GPU and CPU ???
I know chat gpt3 was trained on billions of data, but the mind-blowing thing is that it doesn't copy and paste answers, it literally just comes up with its own opinions. Considering Microsoft is about to use gpt 3 on its search bar, i think Google should be quivering in their boots. Microsoft bing is a revolution, Google is just a search engine.
except Google owns Deepmind.
Deepmind makes openai look like school children.
Deepminds first Text related AI chinchilla is way out performing any gpt model already. it's just not available to the public yet because they plan to develop further.
however everything Deepmind has released in the past so far has been about 50 years ahead of anyone else and way more groundbreaking then chatgpt.
the budget resources and experience and track record of Deepmind compared to openai is incomparable.
Amazing video, and yes I would love to see a comparasion of smaller models. Actually I wanted to make a conversational discord bot for a while now but I can't find a good enough model. Right now I'm using some old chat bot library that remembers users responses and tries to output the one it thinks fits the best, although most of the time the conversation with it is random funny nonsense...
Libraries such as haystack or langchain allow one to incorporate external context into LLM responses and are probably a better way (right now) to create chatbots than relying just on the LLM alone.
Get ready to compare them to LamBDA too.
yes please, compare the smallest models!
Awesome video!
Glad you enjoyed it
I'd love to have an open source gpl IA that can be downloaded and installed in our own servers. Is the worst thing we can do, depend on the good will of private companies ...
Agreed. The corporate models are all heavily filtered/redacted because its "too dangerous" to release an open model to the public (according to these private companies).
I could help out with the german part there.
The first one from OPT is as wrong as it can get. Genuine is not a German word.
GPT 3 is completely korrekt.
Bloom at least uses only german words. But the sentence make no sense. Correct would be: Es ist ein vergnügen, Sie zu treffen.
Or:
Freut mich, Sie zu treffen.
The sentence as that it wrote: Vergnügen, mich zu treffen.
Sounds a bit like nice to meet myself xD.
ChatGPT is also completely right.
I hope that helps and thank you for the nice Video :).
Very good content, thanks
Answering life's big questions. Cannot wait to see more and more LLM being released, especially "overtrained" versions like Chinchilla. Also quantization might allow us mere mortals to run something beefy.
it's scary the chain of thoughts thing
Nice video. Thank you.
I have a little correction though: ChatGPT was not trained on Terabytes. It is trained on Petabytes. This thing swallowed huge number of internet pages.
Will it could be in exabytes and later versions if sweep through more of internet it could be zettabyte even
Where did you get the background music playing in the beginning and throughout? And what's the song title and artist? Love it.
'
Nvm. Just saw it in your description. Thanks!
awesome video dude.
Chatgpt is the best, I would've loved to hear your own opinion instead you just asked us for our opinion lol
If you will fine tuning the BLOOM model to be like ChatGPT it can be a great model
I've read that ChatGPT is only 20 billion parameters, and not 175 billion like GPT-3.
So it should be close to be able to work on home computers, no?
Do you know where you read that? My understanding is ChatGPT is using GPT-3.5 at its core, which is 175 billion parameters. Either way, the model is private and held closely by OpenAI. Even if it could be run on a desktop PC, OpenAI has not released it to the public.
awesome vid, looking for the one with the desktop version. 👌👌
This question so far have always given me a wrong answer from all of the LLM that I have tried it on “is it true that the sum of two sequential numbers that are not divisible by 3 is always divisible by 3?” They say “No” while the correct answer is “Yes”
Great videro!.
I would love to know how they have trained ChatGPT...is it done subject by subject, are there info not included...how do they make sure to minimize human bias (e.g. political, cultural, religious bias)
When testing translations between languages, a good test is to have the AI return its own answer and see what it comes up with.
For example, translate English to German then take the response and have it cover its own result From German to English.
Great idea! Thanks for the feedback.
Why didn't you include BERT?
BERT is not a generation model, which is why it wasn't included.
You can run gpt 3 on any pc through the api
"Helpful" is one way to describe ChatGPT's answers for the simple informational queries, however I'd personally call it "unnecessarily verbose" and it gets rather annoying after a while.
You don't know how to use it. I have several chats set up with different functionality and they are trained to shut up essentially and only respond when asked to. Instead of giving verbose answers what eat up all my credits.
@@theplayerformerlyknownasmo3711 Oh sure, i am aware of that trick too, but that's not really a part of the argument is it?
@@autumndev u just called it verbose. I tell you, you can get it to stop being like that. And you say it isn't part of the argument. It is. The chat bot out of the box is verbose, using it CORRECTLY causes it to act as you wish.
@@theplayerformerlyknownasmo3711 The video is about judging models using the same prompt, not tweaking it to get the best out of each model, and using the same prompt ChatGPT will be more verbose, annoyingly so. End of story. You shouldn't need to convince ChatGPT to shut up. It's a flaw of the model, and that was the point of my comment.
The German translation was worst by BLOOM-176B, as it translated "pleasure, to meet ME!!!", OPT-175 introduced the non German word "Genuine", however, the meaning of the sentence was correct (but a bit clunky;-)) Just a German's 2 Cents...
GPT Bloom like to be a bit of a smart ass it seems, lol.
I'm interested in seeing how to utilize embeddedings from the larger models to train 2nd levels models for specific use cases, or even just general use cases such as search, and other mechanisms not currently available by few or zero shot learning, specifically openAI's API.
Training/Fine-Tuning these models are very resource (VRAM) intensive. The largest model I can train on my desktop (24GB VRAM) is the GPT-J 6 Billion parameter model. And that is only after quantizing the weights to 8-bit and performing many other optimizations. Unfortunately, training the LARGEST models remain the domain of corporations with plenty of financial resources (for now).
@@MeanGeneHacks I'm sorry if I was unclear, training a smaller tertiary model on top of say OpenAI's embedding model, so the embeddings would stay frozen and would be used as input into the model that would then generate from there. Lemme know if I misunderstood your response.
I recently discovered petals, from bigscience, it's still not clear to me ow it works, it seems like a way to distribute the process of retraining large language models like bloom in a cooperative way in order to make it more affordable for the common user, I think.
But I haven't had time to look at it in depth yet and what I've heard from those who say they have tried to use it is that it is very slow.
What do you think?
I've been using Petals and yes, its very slow; however, its usually faster than running BLOOM on the CPU. I have created a nice Gradio UI for Petals and hope to push a video about it in the near future.
You should test bloomz
In the few-shot translation test only GPT3 and ChatGPT returned acceptable translations.
I know your vid in 9 days ago from today, in today's world like 9 months. Lol...but I wish you would had included Microsoft's AI and Google's AI...
Supposedly the largest. I have no idea who is better or what...I don't think it can be measured anymore since its all evolving.
Oh and I love Marv...
Im loving the DAN prompt. It forgets all the biased woke crap, and tells me the most hilarious racist and anti trans shit I've ever heard. It even wrote me a rap song about trans, fatherless black bicycle theives. It was probably the best rap song ive ever heard.
I played with beta GPT3 and they have since dumbed it down. It was not just Saying Predictive text any more than you are. Auto Complete does not use a Nural net..
Is this scalable? Will GPT-5 be just the same but with 10^12 parameters ? Or a paradigm shift is in the making ?
No one really knows yet until GPT-4 is released; however, there is the idea of diminishing returns: As model size grows, the amount of compute and data need to grow as well. At some point (not too far from now), you've fed every bit of text from the internet into the models and are now limited by the amount of data available. It will be interesting to see whether new techniques such as sparse models, or multi-modal models can get around this problem.
More of these
10:13 Grüße aus Stuttgart :)
The algorithmic bias industry has no principled basis for claiming anything is a bias in the data as opposed to a pass-through of factual reality. All they have are the prior assumptions of the moral zeitgeist to make such distinctions. This isn't because there is no theory that would permit them to do such principled "is" vs "ought" distinctions. Such a theory has been available since the 1960s. It is called algorithmic information. However Transformer training isn't capable of producing algorithmic models. That is to say dynamical models as opposed to statistical models. This is because the algorithmic bias industry lacks scientific ethics. Otherwise the algorithmic bias industry would long ago have insisted on transcending statistical models for algorithmic models.
Hey there how’s it going,this is off topic but many people don’t reply to comments on older videos. Can you help me jam ultrasonic signals/ultrasound that are being targeted at me? Everyone around me can hear my sub vocalizations,people repeat my sub vocals word for word. Apparently this ventriloquism/subvocal effect is caused by acoustic heterodyning of ultrasound(similar to the hypersonic sound system). I will pay you to make a jammer for me so people no longer hear what I subvocalize to myself. Thank you.
I would recommend you to look at the camera...
Will try to keep that in mind! =) Thanks for the feedback!
wnat bloom to win, coz its open source
Chat GPT all the way
those model needs to be compressed int to smaller set.
What no LaMDA, people really shocked with chatgpt 3, its the weakest in the major brands lmao
Bloom actually doesn't know German. And it's about as wrong as the opt one.
💜💙🤎❤️
HELP
When will people realize that these have nothing in common with "AI" and are just probabilistic large data patter recognition algorithms?
They don't "think", they don't "create", they don't do nothing but probabilistically show you a pattern that may answer your prompt request or not. Nothing more than a fancy Database Query.
Also it takes a lot of time to "train them", aka feed data so they can make new patterns.
Narrow AI is still AI, especially to the common public. Yes, you and me know they're "just" large language models, and nothing like an AGI, but we should recognize that not everyone does. Especially on YT.
@@NostraDavid2 Yeah, but by lying to them we will gve them false expectations of "inteligence" for these pattern recognition machines....
Good point. I do stress in the video that these models are probabilistic in nature and a good analogy would be the autocomplete engine on your cell phone.
HELP