Unicorn AI - Computerphile
ฝัง
- เผยแพร่เมื่อ 30 พ.ค. 2024
- GPT-2, the Language model that shocked the world with its entirely fictitious story about the unicorns inhabiting a secret South American valley. Rob Miles explains
More on GPT-2: Coming Soon
More from Rob Miles: bit.ly/Rob_Miles_TH-cam
Thanks to Nottingham Hackspace for providing the filming location: bit.ly/notthack
/ computerphile
/ computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: bit.ly/nottscomputer
Computerphile is a sister project to Brady Haran's Numberphile. More at www.bradyharan.com
I also really liked the whole silver snow and blue water with crystals thing which *clearly* was a result of these being *unicorns* we're talking about. "Magical stuff", basically.
+
Just a few years later we have GPT-4
Using a billion data point based AI with an extremely complex understanding of the human language to finish my Voldemort, Harry, and Snape love triangle fan fiction...
Harry Potter and a Stone.
Imagine. You could leave it running overnight and wake up to thousands and thousands of Harry Potter slash fiction novels. Humans have been replaced.
Lol
Hucrux, the lost nose
Game of Thrones could have ended so much better...
Generating text is easy:
In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
I am in the morning but I cant't find it on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen
I'm not sure how I feel about Rob Miles "in-jokes" getting so many likes. Maybe he'll give up his academic endeavors and start seriously monetizing TH-cam videos. Instead of a stamp-making machine he might create a TH-cam video-making machine!
This video is probably just the result of AI been given the entirety of TH-cam videos as a dataset and being ask to make one in a Computerphile 'style'. Thus, freeing up time for Rob to work on the stamp collecting AI in the morning. Unfortunately, he couldn't find it on the phone screen on the phone screen. Tetracorns found to have alien DNA according to Perez, the South American professor. Right, we mentioned this several paragraphs ago.
Gostaria de ter que fazer uma daily sobre o Lenovo e percebi que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato
Maybe ramp up the softmax temperature? Might help with the looping
They can't find find the actual letters on the phone screen keyboard so they are trying to say something using just auto-suggestions.
I want a Spotify to offer a service where I train it with my EEG output and it generates songs that always give me chills.
RyanFromUltrasound that’s actually a really intriguing idea...
What if it becomes too effective and you get chilled to death?!?
Now waiting for news article in guardian
"Computer scientist in UK confirms existence of unicorns in South America"
A day later: Dr. Perez missing!
I wish I had that thing to write my master diploma for me.
Give it an abstract and it reasonably fills in the blanks with rambling no one really cares about.
In Standard English we would say writing a "thesis for my Master's degree". Considering you're having trouble even naming it in English; then yeah, you'd better get an AI. Or, at least, don't try to write it in English!
Tbh let it write a rough draft for you, then go back and polish it
It's surreal to be re-watching all these videos only 3-5 years later, shortly after the public release of chatGPT. So much has changed in such a short span of time that it feels as if AI development is accelerating, or at the least AI development is finally nearing a tipping point where it has the potential to be the driving force behind the next technological revolution/paradigm-shift. I can imagine the level of technological and societal change brought about by near-future AI easily exceeding the change brought about by the discovery of electricity and the industrial revolution
I can't properly describe the feeling of being able to witness and interact with the sorts of systems I've been dreaming would be possible ever since the mid-90's, but if I'm being totally honest, now that the sort of tech I was wishing for is here and becoming increasingly advanced, well, I'm a bit more worried about the existential risks than I ever thought I would be... not that I wasn't concerned before, but the risk of serious accident and/or misuse somehow seems more likely to me now than it did 5 years ago, and I suspect a lot of people are likely finding themselves in that same boat.
I suppose that all we can really do is stay in the loop, hope for the best, and try super hard to NOT be the person who accidentally ends the world with AI.
The world ending because of AI has a large probability based on the vastness of the search space and the small number of cases AI could be aligned.
IDK man, I don't think we have been watching the same videos.
This particular vid is a great example of how language models work and how they can be immensely impressive but not AGI by any means.
Even today in 2023 I think there is a lot of hype about.
From the title, I thought this might be about a type of AI that's so rare, that it's a mythical creature.
Seeing how it wasn't released it may as well be!
triton62674,
A point to you.
And with all of this incredible technology, humans will put it to use generating clickbait articles
I believe that's the exact reason why they didn't release the fully trained model.
@@jasonschuler2256 rrtehr
Clickbait requires sofistication dont you think?
TH-cam: computerphile uploaded
Me: go away Im sleeping
TH-cam: it's Rob Miles
Me: *instant click*
Then what?
That shitpost was clearly generated... Hello GPT2 we know it's you.
Indeed.
, I'm*
TH-cam: Well I know whom *your* crush is then.
Awesome. This system can generate everything I *hate* about popular media coverage of scientific discoveries!
It's almost like this can be used to create news articles or tweets...
almost?
That’s why they didn’t release the full trained model
And? It's already rather easy for people to do that.
Dexxus researchers were afraid of the scope. Imagine spam with this kind of quality, uniquely addressed to you based on your social media profile. Or tons of plausible-looking fake social media accounts reposting auto generated news articles...
They invented the ultimate fake news writer.
isn't this the one that they said it was too dangerous to publish the full program publicly?
Yep. We talked about that aspect of things as well so it will probably be in a future video
@@RobertMiles2 thanks for making videos, machine learning is really a fascinating subject.
And all that does is to encourage someone to try and equal, or better it.
@@threeMetreJim the human race is unstoppable
Damn them, my conference article is due tomorrow. I though I just found a solution on how not to make this an all-nighter...
Where it all began
Seriously, super interesting seeing all the comments, feels like reading 1950s predictions of the future. So much has changed
Anything with unicorns I will click on and watch but I have no idea what's going on here. I think that unicorn is cute though.
Anything with Rob Miles I will click on and watch but I have no idea what's going on here. I think that Rob Miles is cute though.
Computer scientists made a fan fiction AI. They then fed it a paragraph about English speaking unicorns and out came the article Rob reads in the video.
@@sth128 ah, anything with computer science I will click on and watch, but I have no idea what's going on here. I think that computer scientists are cute though
Summary: make an AI --> shove a bunch of articles in it --> AI analyzes how the articles are formatted (sentence structure, flow, syntax, etc.) --> AI then tries to make its own based on those observations. Results = unicorns.
@@JamieAtSLC uh oh, Rob's personal AI has developed feelings for him. we had better contain it
*_...maybe it misspelled "forehorns"... We 'fortran'ed computer-generated-poetry back in the '70's e.g. "I sing a blue guitar large-eyed and bearded bronze but not a star..." pondering whether that was input because we certainly didn't read all the input... (We also implemented recursive-subroutine-calling, in Fortran)..._*
Something I never expected to see.. an Unicorn talking in a Computerphile video. My life is complete now!!!!
If nothing else AI will bring us lots of laughter. At least we will go extinct with a smile...
People: programming must be so dangerous, you might be arrested for hackering Nasa...
Computerphile: _Unicorn AI_
2:50 This is also how it works with people. Go figure.
"A unicorn the same way as to the same way as to the same way as to the same way as to the same way" says predictive text on my phone.
"A unicorn can get us all together after the fight is done and they are doing it for the best"
"A unicorn is a service from Riot Games Player Support Specialist Riot Games Player Support Specialist Riot Games Player Support Specialist..."
A unicorn or a different one of the disabled people who to attack properly and I were not sure what that is the next one is the next one I bought a new portal frame and I were able to do better than I expected you to be in the last week and then I would love to see actually you have any of you guys are you still making it was a car my uncle had once upon a time I wanna go rallying and I were able to do better than I expected you to be in the last week and then I would love to see actually you have any problems with the sale of the disabled people you have any advice with regards to the contrary to the contrary to the contrary to the contrary to the contrary
now imagine if your phone was one hundred billion times more powerful
This bot should be unleashed upon r/WritingPrompts
You can find a fine tuned model bot with Writing Prompts on the r/SubredditSimulatorGPT2
Great channel with great question and answers.
>unicorns are aliens
IT KNOWS TOO MUCH
woke!
I have the entirety of Wikipedia downloaded and that's only 70.5 gigabytes on my laptop so 40 is pretty gigantic!
that’s quite cool. why? do you update it from the live website?
If Skynet becomes self aware reading /r/Brony then Judgement Day is virtually assured.
Your values will be satisfied through friendship and ponies.
It will only watch all seasons MLP on loop, no worry.
Another video so soon? Mr. Miles you spoil us
If this had been implemented in games to create random quests, ohh my lord I would never get bored :)
All of the quests are protect the client like RE 4's Ashley
All your base are belong to us
Your values will be satisfied through friendship and ponies, and it will be completely consensual.
Underrated1 Fabulous idea! I'd be down for that!
daggerfall mod now
Well now GPT 3 is released and I'm watching this video again
Well this is all very interesting, but why is Rob in prison today?
Our AI overlord found out he was designing security systems for AIs
He threw three doubles
His stamp collecting AI tried to conquer the world
@@abcdefghier Ah, yes. He built that rogue AI that caused the world-wide stamp shortage of 2019.
roko's basilisk is retroactively punishing him in the past from the future
This is the earliest I've been to a Computerphile video
Me too!
It consistently stuck to a location in South America and gave the professor a Spanish name... And was able to point out that the unicorns speaking English was "surprising" to the researchers 😄
the world's smartest philosophical zombie.
it's not a zombie.
omg hes so cute, I love nerdy guys. I bet he talks like this all the time and no one knows what he's talking about but I do.
Instead of a stamp-collecting AI, Miles and Riley are working on a TH-cam video-making AI. Complete with Cartoon Unicorn clickbait. This is what the AI apocalypse looks like!
my my, chatgpt-3 sure is a step up from this.
That is eerily realistic... very neat!
Yay more robert miles
I can see why the full-parameter model of GPT2 hasn't been released
and now 3 years later there is GPT-3...
*3.5
*4
40 GB of text is indeed a extraordinarily large amount. My E-book collection is about 1GB, and that's with cover art included in the file sizes. Though, not every story has cover art, so the text of my E-book collection is probably only around 500 to 800 MB.
And even though I really like science, I wouldn't be able to bring myself to read anywhere near 40 GB of scientific press releases. (I understand that those are only a small portion of the total list of website categories that it was trained on.) I would go nuttier than I already am.
That's pretty much the point of machine learning. Our brain cannot process as much information as quickly as a computer or remember as much of it.
I like the fact now (year 2023) that we're at GPT-4 , all what these commenters were asking for have come to pass.
The SEO ramifications alone are frightening.
Open AI released GPT-3, a 175B parameter model recently. They have some example in their github repo. Any chance of a new video on GPT-3?
This is scarily impressive
And I'm sitting here at 1:37 and reading "turn that 211" in the captions and wondering... were the captions generated via this language model?
If so... the model needs more training. But if they were done by a human, that human's model needs more training.
Then imagine when the network does get BETTER results than the human... that's when things will really take off.
Just look at what google is doing with that robocall thing... The future will be both awesome and scary.
Robots, calling other robots, plotting the robot uprising. I welcome our eloquent overlords.
And now we have GPT-3, which makes GPT-2 look tiny by comparison.
The "annotering" is not on. Dont remember the english name for it. (The one CGP grey uses for linking to footnotes)
Annotations have been discontinued entirely by TH-cam. They aren't on on any video.
GPT-2 now looks like a joke compared to GPT-4. Now imagine how GPT-9 or something will be.
I am so impressed.
The bot writes better than I do!
Can anyone maybe point me in the right direction for when i'm looking at ways to generate text based on input? I want to look for a way to give AI several keywords to make a story with. Does such a thing exist?
That's scary and fascinating at the same time.
Maybe it could produce some interesting Kōan ...
Funny watching a warewolf reading about unicorns.
Welcome to the Marvel - My Little Pony Extended Universe
May I borrow this AI for writing my English papers?
Please make a video about Bert and XLnet as well
I wanna read the rest of the article, does anyone have a link to it ?
Wow first time I learned what gpt stands for.
Global Partition Table? :P
generative pre-trained
This is frightening, Jorge Pérez is the name of one of my computer science professor at my university. He specializes in deep learning and AI.
They primed for the unicorn story.
I think I read about that on Reddit.
Will it recognise itself?
just amazing
Omg... Dont fade out... What else can it do
That's freaking impressive
This is getting surreal. They need to give back to reddit and create a subreddit where it continues the most upvoted user submitted story once a day.
Not exactly the same, but check out /r/SubSimulatorGPT2
Do we know how many similar texts are in the dataset? Maybe it just quoted existing paragraphs with small substitutions, and mixed them together.
Scary and fascinating at the same time. I wonder how this model would perform in generating multiple coherent pages
How would you test this for over-fitting?
They might as well have released it, anyone that knows tensorflow and has the GPU resources could re-implement it.
How is the ability of an AI to create somewhat plausible bogus text useful? (except for nefarious purposes) I would want my AI to meaningfully respond to things I ask it about real life situations and these scenarios don't have me convinced that's possible. The ability to create legible text does not, to my understanding reflect an ability to actually understand what I might say and respond as reasonably.
This setup doesn't seem aimed at making an AI understand anything, just produce plausbile stuff without any meaning behind it. Am I wrong?
It can be used as an inspiration for story tellers.
An AI such as this can be repurposed for a variety of uses, in its current state though it is limited but the most useful is perhaps using it translate text into other languages in a manner more realistic to how we speak. One could also theoretically make a video game in which the dialog and story is different for everyone or even just realistic character dialog. Each character could theoretically have a true individuality. One big use for this is that it can allow robotic reporting in conflict areas around the globe. The accuracy that it currently has Is fairly remarkable and if it was made to generate text about on the topic of war then we could have drones with converting the AI's text to speech and allowing a more human and realistic way of reporting in war zones or dangerous areas, live and without risk to a reporter.
what is understanding?
@@mrosskne in this case maybe the ability to correctly conextualize arbitrarily? I'm not an expert on epistemology or neuroscience but surely there is a difference between being able to surround words with other words in a plausable way based on the entire corpus of text available on the internet as a reference and being able to take a concept and apply it in a different contexts, which to me is one (of possibly many) indications that understanding has taken place. seeing how e.g. chatgpt has done on physics tests (see recent computerphile video) this kind of understanding does not seem to take place.
"attention" only is not enough for rich modelling. We also need memory units, for example LSTM-like gated units
Please link the 'previous video', its hard to follow this.
Talking about Unicorns openly. Yet it becomes unhinged once aliens are thrown into the mix.
Well the prompt was unicrons so it had to build from there
That "four horned unicorns" made me truly laugh - out loud - a while! 😂
A four-horned unicorn will beat a single-horned quadricorn any day of the week.
I mean it does kind of make sense, it could simply be referring to specific group of four unicorns that were "horned", meaning of possession of a horn. Maybe all the other unicorns had theirs taken by poachers or something.
@Robin Powell
You have a point - ...but not out of the blue. If some or almost all unicorns had their horns taken, that is such a detail that no writer would forget to mention.
the average person, if they saw a horse with four horns on its head, wouldn't call it a quadricorn. they'd call it a four horned unicorn. because human speech isn't rigorous.
10:08 best part
Doing a bit of research, the size data set is roughly equivalent 133,000 books if we assume the average book is 300 pages. Large for sure but we could absolutely go bigger. For reference the library of congress is holding 6 million books, not that we can digitize them really.
Build a useful AI: one that scans printed pages and CORRECTLY transcribes them into digital format, that would be a very useful AI.
Alternatively let the state hire unemployed people to do the job manually...
the subject was unicorns but i was totally convinced by the generated text it's so scary
I can't help wondering how many of those source articles were written by previous-not so successful-AI attempts, which someone found and posted on Reddit for a laugh ;-)
Got some festival wristbands on, Rob?
This guys voice is addictive
not a player of D&D, but for example if you were able to input the Data Set of every D&D game ever created would there be a decent chance this would make quite a decent Dungeon Master?
does that thing would end up giving good aproximations of the future? give it history books, and increase the amount of data as you get closer to the present, gdp, investments,migrations, etc.... pretty sure it will destroy the stock market already.
That's basically Psychohistory. We'll see, oh yes, we'll see....
Feed it code.
It has seen some code, and if you feed the small and medium models stuff that looks like code, it will produce stuff that looks like code (but which usually doesn’t compile)
A lot of reddit links are to stackoverflow
@@drdca8263 Might have to do with the fact that most of stackoverflow code doesn't compile. Either because that's what brought author to stackoverflow in the first place or because those code snippets are not complete compilation units.
I love computerphile , I love how passionate each of the hosts are.
Anderson Cooper is interviewing the unicorn leader tonight
Seems very close to passing the turing test?
It does better than some humans...
This could solve writers block for thousands of people.
This could make writers obsolete
His sound is so geeky!!!!
How did they make sure the homepages they used contained English text?
It isn’t all English text
@@drdca8263 but wouldn't a language mix mess up the learning?
@@dragoncurveenthusiast So could any stuff that isn't a popsci article
Makes you wonder exactly how much information a human needs to do something like this, but also how do humans learn so quickly? Surely we haven't managed to absorb 40GB of information to come up with something like that. I'm pretty sure we don't have to repeat train on the same thing hundreds of thousands of times either. It would be great to be able to find the answers to these questions.
Oh I suspect that while growing up , humans absorb much much more than 40GB , and over a lifetime ???
@@wktodd Well I could make up fairy stories by age 6/7, maybe not with detail of south America, but still, I definitely hadn't read 40GB of text by then. My guess is we do stories from memory, rather than directly from short/medium term past, although that _might well_ give current context. Pick out relevant memories and convert to text/speech whatever - memory seems to be discouraged in Neural nets (overfitting). I can see that GPT-2 is hopeful of storing everything in one model, but I have doubts - the capsule network idea seems more realistic. Is it possible to pick out single words in order, ignoring language rules at first and then use simpler rules to string them together into an intelligible sentence? I'm Sure I've seen a couple babies do this - but they had to learn the rules over quite some time. The attention seems to do the language rules very well, but randomness and statistical word choosing??? Even when writing this, I had to choose non-statistically, or at least as far as I was aware...I had to consider if it was understandable...and words needed changing so that things were made 'softer', can you work out where I originally placed the word 'sensible'... grammar not so much. :-/ ).
@@threeMetreJim Ah but you can assimilate data by sight, sound, smell , and touch, your parents , siblings , friends all helped to feed you data, that is far richer than plain text . So, by 6/7 you would have absorbed and analyzed (although not stored ) far more information than in 40gb of text.
39 seconds, boyah!
I think this demonstrates more than anything else the paucity of actual intelligence in academic press releases.
most things humans write are largely generic and interchangable. there's nothing unique about academic press releases.
4:22 to add perspective
Each letter is a byte
So the amount of letters in a word is that amount of bytes
Apple = 7 bytes
1,000,000 bytes = 1 megabyte
1 gigabyte = 1000 mega bytes
45GB of text data
4500 megabytes
4,500,000 bytes
And thats counting all punctuation
Don't you mean 4500*1,000,000(Bytes). 45GB = 48,318,382,080 bytes exactly.
you're correct: Jorge Pérez = Horr-Heh Peh-reth (like George Peterson)
actually quite popular Spanish name and surname.
Did it basically pick the Spanish equivalent of John Smith?
@@NortheastGamer close. That would be Juan Pérez.
@@NortheastGamer but if I were to be accurate I would add a second (maternal) surname. To be fair it's usual for it to be omitted in non formal situations.
@@aliedperez Juan Herrera?
@@Turalcar that can work too :)
This is basically what a brain is. More data, more accurate.
I must eat my hat now, because I claimed, that you can't understand the language without some knowledge about the real world, and now this thing seems to get some understanding about the real world just from correllations between words.
Why would you ever say that. Your brain understands the language just by correlations between words. This was 30GB. You brain is Petabytes. You're just a bigger model than this.
@@marsovac how did you go about measuring the memory capacity of the brain?
I was really hoping that this would be a commentary on the feasibility of the hyperlethally persuasive AI in CelestAI.
10:25 That passage is weirdly meta
Thinking that the end of humanity will be caused by conscious, rougue AI seems too self confident now.
to whomever has drawn the unicorns: I see you had a different type of horn in mind.
can we reproduce this ourselves at home?