Mind Blowing 🤯 New Research Paper Creates LIVING NPCs Using AI
ฝัง
- เผยแพร่เมื่อ 14 ส.ค. 2023
- The autonomous agents project is finally here! From the authors of the paper "Generative Agents: Interactive Simulacra of Human Behavior", the code they used to create the simulation has been published! I walk you through how to install it and how to run your own simulations. This is absolutely mind-blowing. I've been waiting for this project for a long time.
Enjoy!
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewber...
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew...
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
Installation Instructions - gist.github.co...
Research Paper - arxiv.org/abs/...
Research Paper Review - • Fully Autonomous NPCs ... - วิทยาศาสตร์และเทคโนโลยี
🎯 Key Takeaways for quick navigation:
00:00 🎉 Stanford and Google released code for setting up simulations with autonomous agents displaying human-like behavior, applicable for NPCs in video games and other uses.
01:27 🤖 Autonomous AI agents exhibit human-like behavior, forming habits, friendships, relationships, and making plans, powered by chat GPT prompts.
02:26 💻 Installation process requires Anaconda, open AI API key, and specific settings configuration.
04:24 🐍 Workarounds for installation bugs, such as updating the 'pillow' module to resolve issues.
05:23 🌐 Front-end and back-end servers need to be set up simultaneously to run simulations.
07:17 🏞️ Command-based simulation setup and running, but caution about costs associated with API usage.
09:17 💾 Saved simulations appear in a storage folder, allowing for customization and replaying past simulations.
11:14 🛠️ Customization involves modifying persona and scratch files to define agent behaviors, traits, and interactions.
11:43 🎮 Implications for video games and exciting potential for future applications.
you copied from the other guy -worldofai- youtube channel. so how do i do it on a mac? lol
Thank you!!!
What do you mean I copied another TH-cam channel? I certainly did not copy anything.
i was just imagining a game without quests, where NPCs genuinely just act as they do not on a set path and them asking for help would be a genuine ask for help and it was kinda an insane thought. Imagine skyrim but they are actually fighting a war each npc with their own motive and part in it.
One step closer to West World. Crazy how much forsight Michael Chrichton had.
He was a genius.
Welcome… to Westworld
I predicted that strong AI would wake up this way. Just wanna say that. I for one welcome our future NPC AI overlords.
Nothing has woken up, it's just good ole ChatGPT throwing gibberish at itself.
😶🌫
Excellent video as always - I love that you're on top of the papers and read them seriously. It's that kind of depth and understanding that I'm here for abs appreciate - thank you.
Lolwhat? The only thing he's on top of is following GitHub instructions.
Now it makes total sense why they didn't release it the same day they published the article. The complexity of just teaching people how to use it is real.
This is a watershed moment, for sure. Maybe it's a bit like looking into the mirror with a mirror behind you, you get an infinite mirror. The simulation has created another simulation. One day soon the NPCs will wonder whether they are actually alive or not, and whether (the) god(s) has/have created them. Will they create their own religion? Will they discover things of their universe? Given enough resources and computing power, they might want to break out of their simulation.
love this comment. what if one of them was given entrepreneurial insights to build something that others in the simulation may find valuable - and actually pay for..
dude, it makes me so happy how much you love this. CAN'T WAIT TO start messing around with autonomous npcs
I think people are overestimating how entertaining this will be in reality.
This specific repository don't is entertainment. However, people enjoy watching others talking, spending hundreds of hours playing a game, or watching a TV show.
With this technology, we are moving towards a time where entertainment will be customized. You will watch a completely original game or movie created solely for you based on your preferences or a specific request. If this isn't entertainment, I don't know what is.
I think you are overestimating your predictive capabilities/imagination.
We can see a sliver of what is to come, but I don't think we have any real grasp yet of how much the world will change.
I'm not all sure that most AAA game brands will survive this, especially since it seems like they won't be able to hoard this type of tech to themselves.
The moment this is possible to run on local LLM I will spin it up, no question about it. Would love to have something like this running. The screensaver of the future:) (or very realistic games). T y for cool video.
This is so cool, it's giving me so many ideas of how I could use it.
Thank you for sharing!
That awkward moment when you realize you are an advanced AI in a simulated reality that some mega advanced AI created for us...
this is my favorite youtube comment of the year
I really like how you always tackle state-of-the-art topics and inform us of the latest trends ! Thanks man, your channel is a goldmine for the AI enthusiasts !
Interesting, maybe in the future this could be applied to a business process simulations. E.g. warehouse employees, bookings on transaction in systems, creating documents, etc. And then train real people inside this simulated world
And then find out everyone is repeatedly thanking everyone else in a loop and getting excited about how awesome AI is because that's what ChatGPT does when you let it talk to itself...
Thank you for posting this video!!! Please tell me as soon as it works based on local large language models
you could change the api end points to use local and put vicuna as it uses the same API functions as chatgpt
That is too complicated for our "expert" to even consider.
@@clray123 huh? endpoints usually found in json file and it's just 1 link
Interesting can’t wait for local AI model support. I also expected to see some characters moving but it’s still a very detailed video that we can use as a guide also I think we can set this all up without Visual Studio tho.
Matt, The gist files are extremely helpful, THANK YOU
The most humbling part of this AI explosion is how all the things we valued and thought were difficult or important AI is rapidly able to do, even emergent social behavior is easy. But recognizing that this photo is of a dog, but it's upside-down in a bee costume just floors the best models. It is our evolved traits which are impressive, all the stuff we made up over thousands of years is going to be resolved end of next year. But glancing at two boxes and instantly recognizing one is heavier, that's probably going to be out of AI's reach for years to come. We can see how the sides bulge, and it's focused at the bottom not the middle and understand that means weight. Where the machines will be fooled by boxes stuffed with pillows, because they bulge more than the one full of sand. Heck a human might notice the sand near the box. What will the machine see? Numbers in a grid that represent color. It can't tell between a bad image and sand on a box.
I look forward to advanced robotics and AI having a contest where it chooses between a soccer ball and a bowling ball painted like a soccer ball to kick at a goal. Most humans could do that as they approach the balls and all of them if they bothered to think about which one won't hurt when you kick it. Machines are going to be breaking their legs on a problem like that for years.
The magic box knows everything but it understands nothing.
Pretty exciting!
I'm more interested in how some this this tech can be implemented in games in future. not so much this software in particular. For example back in the day Skyrim had an amazing system for what felt like living breathing NPC's going about their daily lives and routines and couid be found in various places depending on the time of day and each NPC was tracked, however despite all this, they still felt sort of like robots and only said fixed lines if you try to talk to them. I imagine AI will but answer to make them feel far more life like, coming up with things to say based on the game knowledge and lore, current events going on in the game, and their personality to make them feel far more real. even if it's only "flavor" and not actually helping you to progress the ggame it would still be amazing for immersion. The only issue is they would need to figure out how to keep it in rails to ensure NPC only speaks of relevant information for the game and no go out-side that breaking the 4th wall or saying anything inappropriate, i'm not even sure games should accept input from user to talk to NPC's as we know people will find a way to break them,, but who knows. I'm still excited for the future of what this could do for games but i think we are a long way away because right now AMD doesn't freaking support any AI stuff so only Nvidia users would have it and only people with the latest cards, so there isn't any market yet for this to be put into games unless games start charging a monthly fee to play single player games to be connected to chatgpt. Or maybe game companies can host their owe chatgpt like servers to run the games on for now, for those who cannot run it locally or to not impact game preformance.
Great video! Imagine the all the NPCs in a game like Cyberpunk 2077 simulated like this.
I’m thinking about Mass Effect!
My god, Sims 5 is going to be insane.
Can't wait to see RimWorld-likes that will come out of this. It'll be hilarious
how long until someone makes a simple desktop launcher for this and these kinds of things?
I'm sure we will be able to run it with local LLMs pretty soon as people get to tinker with it.
Just 4 days ago I was working on a local implementation of something similar, AI NPCs interacting with each other with a local LLM, the main issue was that I couldn't find a local model small enough to be used in consumer PCs with high enough context window that generated quality responses, if anyone got a model for this, please respond back.
tell me when you figure it out or find
Super excited
Great video. Thanks for providing the fixes.
The Thirteenth Floor feelings right now... lol
Ohhhhh hell yeah!!! Sooo many awesome possibilities, I can’t wait for triple A gaming studios to ruin this with ads and loot boxes somehow :ppp
Can't wait!
if you look at this from a different perspective this is just a very high level brain
Hello, Matthew! Do you know free diagnose MRi disease llm or site worked on it? Thanks
Each step represents 10 SECONDS of ingame time
Finally, confirmation that all of creation was python scripts all along
God used assembly
@@RobotischeHilfe Is that a church joke?
@@Sylfano joke just the truth
Any tutorials/is it possible to create something like this in say unreal engine 5?
Could this be applied to a home like TABS?
What's that?
@@matthew_berman First result on the search would be Totally Accurate Battle Simulator.
The answer would be, yes. If you can describe the world in the form of text it could do something with it. I imagine that the censored ChatGPT models will just drop their weapons, apologize and then discuss the weather however.
thank you
Excellent video.
There are a lot of good ideas and projects and it is going fast. Ai is helping us to create faster.
this could be a good Isaac Asimov's book or a James Cameron's Terminator movie.
however is there is real and we better learn everything we can so we can get some good profit out of it
Please make a video about how one can do something similar in Skyrim.
When’s that WizardLM 70b vid coming lol
I want to know how I can change the map/environment.
What about AI-Town? What do you think of this in comparison?
It runs but i cannot move the map of the simulation home and the characters’ realtime info do not get updated :(
Anyone faced the same issue?
(Im a dev)
could someone please screen record their simulation and send me the link to their video?
:)
I'm working on a related game. It's a fascinating time to be in this space.
in future, What if they become sentient and continue living in the simulated world thinking it's real ?
Rimworld can already do all of this
Thanks for letting me know youre gonna revoke your api key so I don't have to waste my time trying to use it
Next step: make a mod for sims 4 with autonomous agents based on local models.
Damn we had the same idea
Can you tell one of them that one of the others wants to kill them?
Tell them there is a treasure hidden in someone elses house
Tell one of them their spouse is having a affair
....there is soooo much entertainment in this!!
If you use the template and let it run un modified template will the results be the same all the time?
I don't think so, because the LLM is (by default) randomly picking from the top N best responses for each token. As far as I know you can alter the settings to make it deterministic though. I didn't check the project to see if they use different settings, but I doubt it.
Wow. Psychology and sociology can finally become experimental sciences. I guess. Thank you very much for clearly discriminating btw the local agent and the agent connected to corporation-owned CHAT GPT!
Can I use this without Anaconda?
Thank you for the video. Unfortunately they can´t replicate their self although they can have a crush. Someone needs to solve this issue. : D
Dwarf fortress
Would GPT4ALL server work with this, to avoid the OpenAI API costs?
Try it
If it has the same API endpoint as chat GPT does it should function. In that case you'd 'just' have to redirect the traffic in some way. But I don't know if GPT4ALL is written as a drop-in replacement like that.
You could still rewrite the API call layer to use something local instead, but it'd be a more involved process.
@@RobotischeHilfeIt works!
I can’t post any more than that or YT auto-deletes my comment. 😡
Since I’ve gotta be cryptic, apparently, seek out SaturnCassini‘s pull request.
Now give them robot bodies and place them in a city
AI SIMCITY?
Is there anyone running this in realtime Successfully?
I don't see the point: There are already games out there that have autonomous NPCs but this is just the publication of the simulation we all already know.
Despite of that you tell us: I have played with it for a while but will not show you my results and how fast things evolve in my own simulation world because if you don't have tons of dollars it will be very boring. - ??? So what's the point?
If you would have created a player in a real game and would have used an open source model to steer it, detect and recognize objects, other players and speech bubbles with eg the openCV library or even also a open source AI model and read the text via an OCR library , create a map of the world, store all that in a database and give it a memory and then feed the text to your player with the following prompt "what would you do now?" and then let it create that action based on the answer, that would justify the title of the video.
And btw there's a video out on youtube how to do something like that. Not with using AI models but doing all the other stuff.
Like this it's just: The sim is now public, you may try it and I'll tell you how.
Although I really love your videos, imho your video title is clickbait and I hate clickbait titles. Doesn't describe the video content at all. - Sorry, one thumb down for choosing the video title!
What are the other games that have autonomous NPCs?
@@wowzande One I remember good is: "A.I. Learns to Drive From Scratch in Trackmania". But there are also some other videos out there that show how to program to steer the game with AI. I can also remember a skill game with a marble and also one with a game caracter, but you have to search for yourself, please. - I'm not a gaming guy so I can't remember the titles.
And I know that a lot of people are working on AI NPCs for different games.
How is the title clickbait I think this simulation was created for scientific purposes, not made to entertain people like a video game. The title states that you can create NPCs. You can. Like you can give them personalities and watch them live their lives. Which might be boring yes. But I don't think it was made to be fun. And then he states that games will never be the same again. In which they won't if the non playable characters are all able to respond to external stimuli, learn, plan , such and such. Are you mad because you felt like you wasted your time thinking that you could play as a character in this simulated world?
@@wowzande OK, first I just didn't want to respond to that because it also takes some time to explain it, but I'll try:
Normally, NPCs are referred to Player characters in a game that interact with real players in a game.
But this is a whole "little world" simulation where the difference to other simulations where dots or ants are also driven by a neural network AI.
The "only" difference is, that the characters in this simulation are driven by a LLM AI, which makes it more complex because they may also perform more sophisticated actions than replicate, eat and die after their predefined lifespan. - But also these simple "ants" characters may all have their own characters!
I also gave an example of a video with a real NPC racecar character that really drives in a real game against real player character's cars.
Ah, I forgot: If I remember correctly, I recetly saw a video about a real AI driven NPC in Minecraft; but maybe I remember that wrong. - You may search for that.
And yes, you are correct: My time is very limited and I try to follow a lot/some of other Channels that refer to the actual development of neural network driven AI and if the title of a video is totally misleading for me, imho it's clickbait which I really hate because it waists my time.
Maybe you don't differentiate between all that and it's OK for you, which ist totally OK, but imho the title of a video should really as exactly as possible the content.
Didn't unsubscribe beacause of this one now but my golden rule is: If a content creator uses titles I personally find they are clickbait I unsubscribe after the fifth time. - There are mostly also other channels that will probably also address the same topic.
😮
autocode llama
It looks like it doesn't have the indeterministic flux of life, e.g. wind, hurricanes trajectory, accidents, ... In life, there are environmental factors that no one can control. This makes it so life is not all deterministic like living in some big computer program. That's what separates reality from simulation. However, a simulation can simulate this indeterministic flux. It's not same as two peoples' wind wakes interacting, creating eddies, brushing up onto more peoples' bodies. That's the real thing, the real indeterministic flux that comes from the laws of physics. Was there anything like this in there, or any external unplanned environmental conditions in there?
I guess for now you cannot just type this all in English what you want it to do.
I hope Trump and Biden don't destroy our world too quick, this is very exciting.
what is wrong with this guy's video, it looks like 8 fps
My current goal is to have the AI produce and run game storyline + Quests on top of the AI NPC's : Also looking at having the AI used in the combatant npc's, they'll learn the player and adapt to play styles!
Imagine a sims mod using that 💀🤌🏻
Yeah, thats interesting. Imagine that, one AI marrying another AI & they'll create "AI kids". Crazy & amazing stuffs tho.😁😄🤖🤖🤖🤖🤖