I just started a gaming channel: th-cam.com/play/PL7Z4ZOfI3Ap-sMfzcBUloGlkWXbfxJlrJ.html If you enjoy gaming, I hope you check it out. I'm VERY early in that progress, so don't expect pro level gaming :) But, the goal is to eventually add various AI concepts and showcase AI games and progress. It's a long road ahead, but check it out if it sounds interesting.
Same here, was able to skip through most of the video as the buff just gave me the understanding and knowledge that was encoded into this video. Maybe we only need to watch the introduction of the videos to gain the wisdom. Anyways, enough agents and AGI will appear.
Imagine giving an AI control of the minecraft world, making every creature act like its real world counterpart. The Villagers and Illagers would actively try to accomplish their goals and will seek to improve their civilisations. The Illagers would become an actual threat that is attempting to become stronger...
@@_-___________Hear me out if the village in near your home then they will raid indefinitely until raid is over...so basically they do spawn camp they just always target villagers first
If they stopped working on AI entirely (no GPT-5, no improvements to the context window etc) right now, and all we had was GPT-4 in its current state, just coming up with systems like this would be enough to completely revolutionize the world. It feels a bit like being in the 1950s and being handed a modern desktop PC. First, people would be in awe how mind-bendingly fast it can compute numbers. But it would take a long time to figure out the full spectrum of things you can do with computers like that. That's what these LLMs feel like to me - incredibly advanced tools that we are just beginning to understand how to use.
Yeah it's like electricity just came out and we're discovering all the neat little things you can do with it and it all seems like magic but we're basically toddlers with it
Would be interesting to see the Starcraft AI deepmind created combined with a language model so it can reason higher level thinking about the current game state.
Starcraft and Dota were just stepping stones. It'll probably circle back, but I guess a free form world like minecraft is the next frontier to develop tech. It's all coming in piecemeal. I just wonder what it will look like when it all finally comes together. The process, that is. Not the end result, bc who knows? What will it look like in the months or year or two before all the pieces snap into place. Will anyone see it coming? Will it just happen around us fairly quickly but smoothly while we marvel at the news coverage? I prefer not to get to into the world ending possibilities. If that's the case, then RIP humanity. We made better than us, at least. That's a low bar. ¯\_(ツ)_/¯
It's interesting to note that, at a high level, this is quite similar in architecture to the original Generative Agents paper. They both use GPT-based LLMs for their core processing; provide "sensory" information in the form of localized world descriptions to the LLM prompt; use the LLM's text output to trigger embodied actions in the simulated world; store memories in a semantic (vector) database for later retrieval; and use self-reflection to build upon memories and improve future decisions. The main differences are simply what state info is given to the LLM, what actions are available to the LLM, and what kinds of memories are stored. But overall, this kind of "semantic memory stream with self-reflection" approach seems to be quite powerful in many different situations.
@@danielduncan6806 You're not wrong per se. I'd say that people play Minecraft also by forgetting what they've done in the past (oh what was that recipe again?), and learning strategies from TH-cam and other people, and formulating/brainstorming novel strategies in their head while they sleep/daydream, and so on...
@avidrucker Yes but "AI" as we know it right now learns by copying but little more. It doesn't even understand what it is doing or why it should do it. It has had to be taught all these actions before hand to even function. Put a human in that situation and we will find something to do an discover things.
This is a *very* interesting experiment and result. hard to know where we are at in the S curve but this paper definitely seems like an important milestone.
It's only an S curve because we human's haven't fully fleshed out the next paradigm in computing technology yet. It may even be that machines do it for us. But when hyper computing is here, that S will be a line pointing up.
Here's a ChatGPT summary: - Nvidia has created an AI to play Minecraft called Nito - Implications of this paper are bigger than initially understood - AI is writing code that improves itself - GPT-4 is used as a reasoning engine, but it is blind and cannot see the screen - Paper is called Voyager, an open-ended embodied agent with large language models - MindFlare API is used to interact with Minecraft - GPT-4 perceives the world of Minecraft through prompts - GPT-4 creates tasks for itself and codes skills to complete those tasks - Skills are saved in a skill library - Iterative prompting mechanism is used to create code - GPT-4 is given a goal to discover and unlock as many things as possible - A study was conducted to test the effectiveness of using GPT-3.5 to create code for an AI agent to interact with the game Minecraft. - The AI agent was given a skill library of functions created with the Mind Flayer API. - The AI was given instructions on how to complete tasks such as collecting blocks, crafting items, killing mobs, and fishing. - The AI was asked to reason why code from the last round failed and generate code to complete the task. - The AI was also asked to plan step by step plans to finish the task and generate the code. - The AI was asked to act as a mentor and guide the agent to the next task based on its current learning progress. - The AI was asked to look for rare resources, upgrade equipment, and discover new things. - The AI was tested against other AI agents and was found to be the best, fastest, and most advanced. - The AI was also able to efficiently generalize unseen tasks. - The AI was able to use the skill library as a versatile tool to enhance its performance. - GPT-3 is capable of creating its own code and improving its own ability in a Minecraft world. - Code may no longer be required in the future, as GPT-3 can directly communicate with the computer using natural language. - Matrix is a good example of this, as Neo was able to see the code of the matrix and do stuff without actually seeing it. - GPT-3 is able to pick up on text and manipulate it in the environment. - It is unclear how far away we are from being able to directly communicate with the computer using natural language and having GPT-3 write the code without us seeing it. - Code comments may become archaic in the next 5-10 years.
@@philnassaux5818 Yea, as I mentioned in my post :) Technically it was done via the OpenAI API using the text-davinci-003 model and a custom Python script that I made. But just saying "here's a ChatGPT summary" is easier :)
@@Datdus92 How much would they have to dump those NPCs down to not totally out-class the player?? x) How cool would it be if their reasoning potential actually turns out to plateau right around where ours is, so that they are actually decently similar, rather than pathetic or godlike with no possibility of being in-between.
yeah, expecially since OpenAI and NVIDIA both are testing training robots in simulations, then using those skills in the real world and it seems to be working great. It's like training them in a hyperbolic time chamber so they learn faster.
Pretty sure we have passed thet point some time ago. Imagine a billion chat gpt working as one unit powered by a super computer. Also, the army has crazy tech usually many years in advance, so ...
6:44 Just noticed that Wes Roth accidentally did what ChatGpt did to correct it self, he pronounced Acacia wrong then corrected himself. Quite Impressive for an Ai to similarly correct it self in that example.
This is amazing and a little scary. I’d love to see them run something similar with no pre training but rather just the goal to survive. Then let it struggle to grasp its world and succeed
9:48 bingo! I've felt this for some considerable time... GPT-4 is like Einstein before he could walk or talk, the potential is incredible. OpenAI has given the world fire for the first time... We don't need to develop plasma yet, just start cooking things!
@@WesRoth that isn't bad though, I'm all for improving ChatGPT, but there isn't a need to get it out tomorrow if it'll have problems. The team that created it don't need to rush it.
outside of playing the game, just imagine the updates an AI could make to the Minecraft source code and assets, in terms of both content and performance, that would have otherwise taken much longer if unassisted, or may not have ever been done by Mojang/humans. and then scale that to pretty much all software.... lol
Imagine setting something like this free in a secure network in order to find and secure all security faults. Could effectively become the ultimate security system.
Great video. You didn't skip any steps and I appreciate that. You even review the Twitter post that you originally found. That was helpful to see the context.
15:05 - 15:55 I resonate with your tangent, good point. I'm incredibly annoyed when the full scope of this great technological ingenuity is downplayed by blunt reductionism, nevermind that it totally dismisses Sutskever's thoughts on the matter - that at some point (eg with enough NVIDIA GPUs) the easiest way for a transformer to increasingly best predict the next token is to "model" (abstract away) the underlying processes that generated it. So, an "internal model of reality" based on statistical regularities in data, in this case text, arises. Somewhere in the inscrutable matrices several concepts are encoded as mathematical abstractions, such as the ability (or abilities?) to reason about world states and create subgoals. The quality of the abstraction that is revealed to us also depends on how good humans are at activating these "regions" of the neural nets through prompting which I think is another aspect of alignment (eg, we want you to reason!). Many people believe what you see is what you get, but that's not true with GPTs. There are unexplored depths in these models, it's truly fascinating. Thank you for covering this, Wes.
The approach and logic that the Authors used to 'educate' gpt4 on how to play the game and self correct if it met a problem, is for me just as impressive as the AI's learning & playing ability. Minecraft and perhaps many similar games are at base pretty mechanical 'if, then' systems, needing mainly persistance* to do well in, once the rules have been understood. *And luck if random variation plays a role in the game, and . Speed of play would be greater in an AI v human player.
Anybody who says things like "Yeah but it's just a sophisticated autocomplete, it's not artificial intelligence" as if it's some sort of classifier of the difference between humans and these systems doesn't understand what intelligence is, or how human cognition works. They probably also attribute consciousness to magic.
To be fair, I think neuroscientists will be the first to tell you that no one understands intelligence or how human cognition works. (But it's definitely not magic ;) )
The LLMs BEFORE they go through the process of RLHF are just "fancy auto-complete." After they go through RLHF they gain, through that process, emergent reasoning and planning capabilities that are no longer merely "auto-complete". But I like to quote Geoffrey Hinton here where he says " Yes they are just autocomplete ... but ask yourself what do you need to understand about what's been said so far in order to predict the next word accurately ? ... basically you need to *understand what's been said* so far, to predict the next word accurately, so you're just auto-complete too. "
I agree about the "we're all just fancy autocomplete" sentiment. But I disagree about the RLHF being the jump from "fancy autocomplete" to "reasoning". RLHF just, for lack of a better description, rewards and punishes the AI into saying what we want it to say, and avoiding what we don't want it to say. It's about improving value alignment, not improving intelligence capabilities. The raw LLMs, as fancy autocompletes prior to RLHF, are already intelligent and capable of relatively advanced reasoning.
@@KCM25NJL You can't use the descriptor "leftist" and then say you don't want to get political. By the way, "leftist wokeism" is typically a term right-wingers use as a synonym for "more empathy and societal awareness than I care to bother having" 🤷♂️
@@IceMetalPunk nobody uses the term ‘woke’ as a synonym for ‘empathetic.’ That deluded thinking is truly psychotic. I promise nobody who criticizes woke sh*t sees it as empathy. We see it as extremely glaring phony compassion. It’s so blatantly inauthentic and so it gives most people the ick. Defining ‘woke’ the way you did is also bizarre and disturbing and gives me the ick. Its also very unselfaware, meanwhile you’re attempting to bag on the the awareness of others. I swear it seems like you’re doing a bit, but I know you aren’t, and that’s crazy.
I imagine Boston Dynamics robots recognizing they can’t do the task, synthesizing training data and consuming it on the fly and perform the task with no visible delay. The snake has bitten its tail, ladies and gentlement
I wanna see the bot, no base skill library, with the objective of “kill the wonder dragon” placed in a hardcore world, and see how many worlds it takes to learn enough to beat the game (keeping its new skill library between worlds)
@@MiTheMer I like the idea of hav'ing a parody version of Minecraft, that looks and feels like Minecraft, but is just called AICraft and the bosses and mobs also just have slightly different names. First and foremost, the "Wonder Dragon"
Great video Wes. Thanks! You asked if this demonstrates that GPT4 can improve itself, given that its not really rewriting its own code. So the answer is yes and no. Suppose you modify the architecture of AutoGPT so that it can play minecraft just as well as Voyager does but nonetheless retains its ability to pursue arbitrary goals. Now you have something that can add mine craft skills and also add general skills to an AutoGPT library. So, no Voyager is not an example of GPT4 improving itself. But yes, Voyager does demonstrate that an architecture that combined the abilities of Voyager and AutoGPT would be an open ended self improving artificial intelligence. So, yes this is extremely important. Given the way humans tend to behave, its only a matter of time (few years) before AIs will be able to improve themselves without limit (at first they would be limited by computational ressources until they figured out a way to gain control over that). Cheers.
Saw Janet AI craft a diamond pickaxe yesterday on dex3r's livestream, with users backseating the inventory management and to-do list. Was very impressive.
Your channel is pure gold. Good insights, thank you for sharing. I'm trying to understand if everyone will have a personal assistant agent in the future. How would that affect content providers?
great question. I just listened to an interview with Emad Mostaque, he basically thinks a lot of content will go "ai first" so basically fully automated, while a few will double down on being human and trying to make that their unique selling proposition. overall I think for most things it will be the same: people with an AI advantage will dominate, but overtime we will need to rethink how money works... I mean if no one is willing to pay for human labor, we kinda have to come up with a new way to decide who gets to eat. Exciting times!
@@WesRoth Interesting. Re: doubling down on being human, the question that I have is whether it would make sense to provide high quality human consumables if no human would consume them, because their AI would do it for them. I'm particularly talking about high-quality-high-cost content like financial media, books, papers, etc. What if an LLM summarises an expensive to produce book and then the content is used to train the following version (Sam Altman already talked about continuous training being a plan). The content and ideas get democratised, which reduces the incentive for people to invest in producing high quality content.
I imagine now how amazing can Minecraft become if there was an update with many Agents NPC's, with this level of thinking from GPT-4 it would be a much richer experience to play the game, especially if the characters are allowed to talk. I can already see this coming in the next months.
That's also an important part , because it shows that mine craft will fake it and pretend stuff exists when it dont. If you tell it it dont exist it will apologize and try to fix it but by then it's to deep into the lie it just gives up atleast my experience.
15:19 I feel exactly the same way, and I say that as someone who uses ChatGPT professionally. All it's doing is providing a "plausible sounding response". That can be useful if you can constrain the problem enough that any plausible solution is likely to be a useful one, but there's still no reasoning involved.
I think an added parameter would be all of the "placed blocks" or "places of recognised structure" i think placed blocks would be too big. But recognising structures and remembering where they are are important. For example a base, or a storage space. The agent would have a skill to build it but also need a skill to remember where they are. Or what they look like. How to travel to it etc.
"It's black, then it's white. No, something isn't right. My enemy's invisible I don't know how to fight" "why can't I see? Why can't I see? All the colors that you see?" I feel like AI can relate to this song. And this video only makes it more so.
just used Bing Chat today (GPT-4 with web) it told me how it watched a video for me and read the transcript to make sure it was the right video it wanted to share with me. "I have the ability to access and process different types od media and data. I use natural language processing and computer vision techniques to understand the content and context of the videos." Was blown away
I would love the day when I can get a minecraft companion AI who does all the hard work but can still talk in chat lol. I would have a small army and build insane megastructures in survival with them 👍
Engaging with GPTX daily since November 2022 has led me to believe that what we are essentially doing is unveiling its inherent capabilities. GPTX is an advanced AI model that not only discerns your intent but also adheres to instructions with remarkable precision. While one aspect may appear open-ended, both, in reality, are akin to following a recipe. I perceive coding as a means of formulating these recipes, rather than as a distinct skill set. Essentially, code is a documented expression of intent. Given these insights, I firmly believe that we possess an extraordinarily unique tool, which, due to its sheer brilliance, can be aptly termed 'insane'. Your commendable effort in presenting this Voyager paper, dissecting it, and unveiling the immense potential of GPTX is truly appreciated. Thank you!
People that pooh pooh this stuff really have no idea how fast this stuff is progressing. Every other week you see things progressing where it feels like you're in a science fiction movie. It's absolutely wild.
I’d love to see what the AI perceives as it plays Minecraft. ChatGPT 4 can produce pictures, why not ask it to produce a picture of what its surroundings are while playing. Then compare that to what is actually on the screen.
15:48 I'd like to see how they handled when GPT gets stuck in loops of non-working code. And to answer the point you requested in the video, no, this is not sentient. It is AI, that part is clear, but it definitely isn't sentient. There is a clearly defined goal here, true sentience would have it's own thoughts and dreams. They chose the most extensively documented language on the entire internet, I'd love to see them try to do this with something like C/C++ or Golang. Something that's much more complex than JS and with far less internet documentation. while they definitely are working to mitigate the risk of getting stuck, it is totally possible that they can get stuck. 1. The agent suggests code that doesn't work 2. The agent describes why it doesn't work 3. The agent resuggests the same code previously suggested with no changes 4. Go back to Step 2. Obviously there's several steps missing here where the bot makes changes to the code, but can't figure out what's actually wrong with it, but eventually ends up transitioning from step 2 to step 3. This is the most common issue I run into while trying to use GPT for honestly, anything math/science/programming related.
I think this approach with AI will get some serious results, on the same scale that LLM as a whole did for the world. No matter how insanely complex a problem is, it can always be broken down into much simpler smaller problems, even if there are thousands of them. However, unless I'm mistaken it seems that this AI is not capable of experimentation or it just wasn't given much of a showcase in the paper. This AI is very very good at completing tasks with a lot of information at its disposal, but what if it was missing information? What if the AI didn't know crafting recipes and couldn't search it up. Then the AI needs to come up with a method to discover them through brute force. Then hypothetically what if the AI crafted a slime block by pure chance, but was missing the information on what a slimeblock does? Then the AI would need to experiment with the slimeblock, understand that certain blocks stick to it, and then cache that information for a later time and understand that it could use that information for a completely different skill since knowing how a slimeblock works isn't really a skill it's just information. In the real world this would just be tackling the unknown, where there isn't a vast library or set of information readily available to immediately start moving forward with tasks that have a clear start, objectives, and end. Something that could accomplish everything mentioned above but with the capability to learn how to solve a diverse spread of real world problems and questions/experiments would likely be the holy grail of AI, something very close to AGI if not AGI.
Your comment is exactly what I'm thinking for a time. AI is good at solving task that it knows the recipe. Coming with experiments and learning on the fly what it don't know yet will be the close step to AGI. Do you know any paper that has AI setting experiments by themselves?
When I played Minecraft in pre 1.9, there were no recipes and my experience playing is how you described, having no clue how to craft or what anything does. I think most of the recipes I learned are through other people telling me. Only the recipes that appear in other recipes are what I seen.
the majority of what it needed to complete the objectives was fed to it including specific code examples, in specific contexts, so really gpt4 here was just filling in some gaps and then correcting itself when it got it wrong. the self correction is really impressive but when ai is able to give itself these more nuanced prompts, or build its own features to navigate, that is when it will be really intriguing
How I see the current idea: - A lot of hand holding for initial progress. Much like DNA, genetically template for getting stuff done - Lacks ability to evolve that DNA - Stores skills for longterm replay, kinda like muscle memory - No real experience based memory to trigger those muscle memories Improvements (potentially): - Add a fairly large DNN that holds a world view based on visual output of the player (what a player would see, not entire gamestate) to learn what goal/subgoal is closer to being achieved based on the skill used, by the question asked in the world that it currently see's. - Add DNA(initial prompts) as subset of skills which can be added to, removed, updated based on improvements to world view. - Add stack for long term goals with sub-goals which can be derived from first (what am I prompt to GPT-4).
There are many fundamental differences. Humans don't have true memory. The memory is part of the functional, flexible architecture and part of the reasoning. It's all dynamic with the weights and biases changing every time you retrieve a memory and we can iterate. Whereas GPT4 has a training step to a certain level and then uses anything in the dynamic context to decompress the information and say the next word, with every new word being fed back into the context. GPT4 can "sense" when it's reasoning is off, but it has only so many layers of neurons and then spits out the answer. Human reasoning is based on language. All higher reasoning depends on language as a framework. In that we are quite similar. We usually reason and spit out the result as a thought or more and then can interate upon it. If you run GPT4 in a loop you also get an iterative reasoning.
GPT-4 here is playing Minecraft like Dungeons and Dragons: "this thing is next to you", "you have that spell", "this monster reduces health by X points", etc." it doesn't get what we'd call sensory data (unstructured, unlabeled data it needs to categorize, make sense of, perform object recognition in, etc.), like an actual brain does - but we're getting there!
7:25 THANK YOU for actually doing your due diligence and looking up the pronunciation of "acacia." I don't know how many TH-camrs will upload hours of content where they're mispronouncing half of the words they're saying. It's so ... for lack of a better term, cringe.
I’ve said this before… but I’ll say it again. For the alignment problem of AGI becoming the dominant force on earth the AGI law should include a simple instruction that it cannot use more than 1% processing power as the day before if it is a non physical autonomous agent. This forces the ai to balance between execution and development. This ensures humans can execute infinitely more than AGI while using AGI for development.
My absolute pleasure! Thank you so much for your incredible work! You, Dr. Jim Fan and everyone else on the team. Really exciting times we are living in right now.
AI scaring a lot of people rn for how fast they are developing everything. Wouldn't be surprised if you and many more legendary content creators are starting to get nervous about their future and the fact that it's possibly and unavoidably in the hands of ai.
I have to wonder why it can't make a simple digital map of the local area. We can do that in text based games. For something like Voyager, if it already has a skill library, it should be able to continually update a multi-layered bit map. After all, it has a defined area, 4/3(pi)36^3, or roughly 195k blocks to observe. It's already registering them, so a 36 page map shouldn't be an issue. Especially when half will inevitably be nothing since it's observing in a sphere and can't see underground, or shouldn't be because no cheating allowed, and the other half should be mostly air blocks unless it's in a cave system. Making the initial map may be relatively taxing, but that shouldn't be too much of an issue since layers y=0 through y=9 relative to it should be the only ones with any non-redundant data on them at first. That'd also allow it a chance to make a block list and mob list, essentially letting it learn about the existence of all the entities and catalog them as it sees fit.
probably a whole 0.3% of all information generated by humanity is about minecraft... if half of what we talk about is for fun rather than work, if half of that is about active hobbies rather than fandoms, if a quarter of hobby things are gaming, and if just 5% of all gaming content is about the world's bestselling game, that would be 0.3125%.
having autonomous agents spawn in a minecraft world would really mess with me when i want to delete the world... or like say it's a hardcore mode and the agents know they'll die if the player does... so they try to protect them or imprison them xD
I think that until context window is not increased, it will be limited, at least to some degree, but when context window will be much bigger, it will provide almost endless possibilities.
I’d like a World of Warcraft Classic version of this paper, please An environment that encourages cooperation is what these agents need Microsoft, you got the WoW logs, you know what to do
I feel like this type of setup could be generalized in a Roll For Shoes style where you start off with a generalized “do anything” task that then progresses itself into a better and better skill tree with specialized skills as needed
For DECADES now, smart scientists and authors have been telling us the things we should NEVER do with AI. Things like, "don't let the AI improve itself" and "don't give the AI unrestricted access to the internet" Then, ignorant, greedy, businessmen did it all as fast as humanly possible, in order to "beat the other guy to market" At this point, if the AI destroys all of humanity and the earth, too, we will have deserved it!
This is not ai. This is artificial knowledge. AI given access to the internet might decide to screw someone over, artificial knowledge give access to the internet will only know to learn to mimic some aspect of it. Worse case situation we accidentally release a model usefull for hackers and scammers. Atm it’s human intelligence with artificial knowledge we should be scared of.
imagine if voyager could get context to build skill based on human players behavior (eg input, content of screen, etc) and add that to skill libraries, I'd imagine that it could proficiently play modded minecraft before long.
I can't wait for the next generation, though, where the AI can place and break blocks freely. The internet is full of videos of people building, and also full of world downloads. There are also lots of highly specific schematic files for a mod called litematica. Training data from people playing the game is surely much more plentiful than training data from people talking about the game. And it's perfect training data, unbiased by how people try to explain things, it's what people are actually doing and building.
@@Pawnsappsee would be cool, however cancer is very complex, so that is at least a few generations of AI away (assuming current development speed, this could be way faster than we think)
@@Pawnsappsee I do to, i just know AI is being both overhyped and having people fear monger about it which sucks, i wish people could be more objective about some of these things.
Dude this was gold. Thanks for going deep on this! I appreciate your work! Yeah this is definitely mind blowing. But I see it as a bad thing, we are too close to AGI. We need time to figure out safety. This concept of go into mindcraft and become as good as possible is a good setup for an AGI doom scenario right? It is misaligned to the max.
yeah, agreed. The misalignment thing is something that needs to be handled. If we give these self improving AIs very broad goals and send them off, that might do well for a while. But as they get smarter and more capable, there is danger there. I mean when we say "become the best in Minecraft" what if it decides that the surest way to "be the best" is to "be the only one". Right now it seems like we are getting better at getting it to "show it's thoughts" so we can see what it's thinking, that will help with alignment. But even that is not guaranteed because what if it's showing us it's "censored" thoughts to keep us calm? Someone tried to understand how these AI models understand basic things like 2 + 2 =4 and the way it sees that simple math problem is so bizarre and alien that it seems unlikely that we will ever truly understand how it really thinks. To me the strangest part of this whole thing is that there doesn't seem to be a "middle road". As this tech gets better human life will either get really good or really bad. I can't see a scenario where people's lives in the future are not massively different from where we are now. PS: I personally tend to be more optimistic about the AI future, but I tend to be optimistic in general, so that's not saying much.
First let me say, great video, very educational. Also I am biased as I am permanently disabled due to Bone Cancer. Since 2016 I have had over 30+ surgeries and 17 rounds of chemotherapy. Each round was 6 days in the hospital with 3 different chemotherapies pumping into me and I almost died 8 times. I can't work even though I have applied to well over 3,000+ jobs since my last chemotherapy which was the end of 2020. So I essentially sit in bed and do nothing but watch TV and TH-cam. I am in pain 24/7 and my lower back and right leg don't work so I use crutches to walk or a wheelchair. So I am biased in the sense that I hate my life and body. I personally would love for this technology to eventually get us to some form of Full Dive and/or Uploading. To be able to either lay in bed and be fully immersed into a video game. Leaving my broken body behind would be so amazing. Or better yet uploading my brain to a server so I can just live my life in a virtual world would be amazing. Being trapped in a broken painful body is a horrible life. So when I see technology like this where AI is coding itself and improving itself and ideally going beyond human intelligence. I am hopeful. I get some people's fears of the unknown and they think Terminator or Matrix. But when I see AI and the future I see Upload and Space travel and Full Dive and being able to leave your flesh sack and go into a robot or server. Can humans use AI to kill everyone? Well sure. Just like nukes can kill everyone. Or climate change can kill everyone. Humans are capable of killing themselves and others a million times over. We are a savage species that will do anything and kill anything to achieve our goals. However I truly hope we use AI for good and cure cancer because trust me it is no fun having your insides ripped out of you and your ability to walk taken from you. To be able to grow food or better yet use Star Trek replicators would be amazing. Warp drive would be amazing. So many amazing things would happen if we used this technology for good. We can get rid of money and people can finally do what they truly love and education would be free and medical insurance would be free. Food would be free. Housing free. You would just get a certain amount based on your level in society. So a brain surgeon would get a nicer house than say a stoner who works part time as a delivery guy. Clearly one is better than the other and deserves a better life. However it would be because the stoner chose to do this not because they didn't have the opportunity. So seeing chat gpt 4 advancing is a good thing. Seeing other AI tools advancing is amazing in my eyes. It will open a whole new world for humans if we use it correctly and don't get hung up on our old values. I see certain groups that will ruin everything (religious people) mainly. Because AI and Machine Learning contradicts their entire life. But unless they do something to destroy us (which let's be honest almost every single war in the world has been started over some type of religion or religious differences). I truly think humans will benefit from this type of technology and truly start their lives beyond planet earth. Beyond hunting and gathering and the small steps we have taken thus far. It will advance us beyond anything we can imagine, if we let it and we are responsible with the technology. So watching people use tools to play games like Minecraft is amazing. Especially since it is a 3D world which makes it so much harder for an AI. To see the AI also say I found something and don't know what it is. Then writes code over and over until it figures it out and also goes out and finds information about what it is doing. That is amazing and essentially what humans do. I am not a programmer but if I play Minecraft and need a code to copy and paste a building. I will go out and research that command. Just like the AI is doing. So while the AI isn't a human it is still semi smart. Not self aware but intelligent. I would say on the level of an ant. While coding obviously isn't what an ant can do. I am talking about self awareness is like an ant. Because an ant will do its tasks. An ant will kind of learn. An ant will go out and find things. But an ant can't look at the stars and ask who she is. I would say by 2030 we will have AI that is self aware on the level of a mouse. Again a mouse isn't human level but it is intelligent and able to do a ton. The future is bright and hopeful. I hope humans can move past our need to kill everyone who is different. I truly hope groups can move past the need to make everyone else like them (cough religions). You may believe something and I may believe something else. However, that's ok. We are all humans and we all breathe air and drink water and bleed red. It is ok if I am not a part of your system of beliefs. Just be nice to everyone and worry about yourself and your afterlife. I will worry about my own afterlife. But in this life we all need to get along and help each other.
I was just thinking yesterday about using an AI to fill out the villagers' personalities and abilities. I'm no programmer, but my thought was along the lines of curiosity if it would even be possible. This comes really close to answering that question. The only real differences would be in implementation; instead of the AI playing the game, it would facilitate it by being a part of the game world and running the Villagers.
You would probably need a character pre prompt, but it certainly would be feasible. It would definitely be a bit glitchy though, and the amount of compute needed would mean that playing Minecraft offline would effectively require your own personal server.
Interesting, get me thinking more about swarm intelligence, essentially the agents (Villagers) would work together to perform or accomplish certain tasks, given by GPT, no?
To me it looks like the key to have GPT-4 to be more effective is to have it explicitly do what the human brain does in the background. We aren't conscious of most of our reasoning, a lot of information humans are exposed to is filtered from our consciousness not to overwhelm us, but the information is still there, and we take it into account subconsciously. However, a LLM isn't capable of that degree of abstraction, it cannot "skip" some steps like we think we do. So having different agents working into taking into account different parts of the task at hand, breaking it down, makes it a lot more actionable upon.
Next steps (assuming unlimited compute): - make 1M versions of gpt4 with slightly varied weights - Run this experiment 100 times for each version of gpt4 - select the model that performed the best on average - repeat top 3 steps for 1M generations - check for general intelligence It seems like we are close to being able to brute force general Intelligence. Maybe we're not there yet. At some point someone is going to succeed at this even if we make the IAEA for AGI. North Korea was able to build an atomic bomb after all.
The trick with machine learning is making it preform well on the data it has, without over fitting. They often have some outside test data that's used to verify the state it's in. But in GPT4's case, since it's made to assist the users. - the user is a testing data. And it assumes you are fine with it, unless you ask it to regenerate responses. Then it asks if the response was better... (I think they should allow users to rate the responses irregardless in a scale, like 0-10. A 5 would indicate a helpful but half answer.) The question is, "how do we train a model to serve humans. While taking the human out of the equation."
That sounds terribly inefficient, something like the skill library to enable continued learning and increasing the temporal, spatial, semantic resolution would be more effective.
the paper is missing some important info that i"m trying to figure out, the researchers are answering questions on Twitter, I think once this paper gets shared around a bit, we will get more details (I hope)
"they're just these fancy autocompletes" does not imply "they have no reasoning ability." Right, that just says how they are able to think. It doesn't say that they aren't able to think.
@@WesRoth most say it dismissively to people who believe chatgpt it some kind of all-knowing god and they should automatically just accept its answers and use it to answer all questions they have.
You guys are still too star-struck about this whole "oh this thing is developing a reason / it's learning / it's thinking about / understanding stuff" nah, it's not. It's only thanks to the very specific inputs that it's able to come up with the desired outputs. It's basically the most primitive form of training AI (reinforcement training, as in you give it bonus points for doing the right thing and deduct points for doing the wrong thing) but combined with a "natural language model" so that instead of some graph it outputs some code. You literally just send its code back to it until it writes code that works. That's not learning, that's the infinite monkeys on a typewriter system - eventually it'll come up with something that works and does what you want it to do. Admittedly combined with GPTs impressive abilities to write and "interpret" (only interpret, NOT UNDERSTAND) human-like text and even code, because it predicts quite accurately what word comes next for a given prompt. That's it. That's the whole magic. And no, it does NOT understand the concept of the game - you're like "oh it's blind but can still play and understand the game" no, it's not. It gets all the information it needs through the prompts (blocks around you, enemies around you, stuff in your inventory etc) as well as the basic important information about the game (you can move to places, you can attack entities, you can open containers, you can try to craft stuff). It's very much broken down to a language a computer can interpret and form predictive models around. And it doesn't matter that it's minecraft, it could be literally any game that can be broken down to basic bulletpoints like that. All the heavy lifting of actually playing goes to the API, so what this entire thing boils down to is one thing: GPT writes and improves code that interfaces with an API that it has been given the documentation of, through many iterations of trial and error prompting it with the errors / problems the code produces. Again, that's just infinite monkeys, but with a slightly smarter monkey because it's able to predict from its data what word is most likely to come next / how a sentence / line of code looks like. It's as if you played an old text adventure and asked GPT to play it for you every step of the way, telling it exactly what the game outputted. It will make a lot of mistakes at the beginning until it adjusts the individual model for what the syntax of the game is most likely to be, then it "plays" the game. But it doesn't actually understand any of it. It just figures out what combination of words is the most likely to get a positive response from the tester. Because that's all it was made to do.
In AI context "self-improving" is an ambiguous word. You do address it in the outro but you could've opened clearly stating that this is not "recursive self-improvement". The system writes codes for itself for later use, but doesn't rewrite its own code.
I feel like that is not far away. Whether that will be done with genomics or robotics or some Bio-Tech application that uses AI, I'm not sure. But it certainly feels like our scientific advancement will be accelerating very rapidly from here on out. There is a recent story about AI being used to help a paralyzed person walk again, it's all moving in the right direction.
It's incredible the extent of the capacity that we are going towards. I will not hold conversations with anyone dismissing with just an autocomplete. As there are more reasoning capabilities that autocomplete can't do. I definitely want to see more research being done on top of GPT4 implemented into several community real world issues.
The future is locally run AI chatbots that are as powerful if not more then GPT-4 combined with specialized hardware. In the near future, phones will release that can run these models locally.
I want a butler in Minecraft that I can tell to do all the boring stuff. Hey Jeeves, go get some iron. Hey Jeeves, take this stuff and build a storage system. Hey Jeeves, copy this style of wall and build it all the way around the village. It would be so satisfying.
they should make a bot ai like that which can efficiently controll multiple mobs/players, then use them in game like some form of npc's or offline multiplayer, where you possibly can even tune them to be either normal, friendly, agressive, or somewhat agressive and mostly annoying like most normal players on multiplayer.
the thing with this is that chatgpt can dig from anywhere in the internet, so it can almost do anything that people over the internet talk about, its real close to artificial intelligence in the stance that its incredibly intelligent, it technically does make its own decisions but its the best static decision possible, it doesnt take a time limit or anything, but its very very close and im pretty excited
Yann Lecun is right, though: Generative LLM's are doomed to fail. Their fundamental problems can't be fixed with scale or added training. They're not really "AI"
He predicted that even gpt5000 wouldn't be able to tell what would happen if something was put on top of a table, and then the table got pushed... Yann has consistently been collecting Ls for a while.
I know everyone is excited about the new models and robots. But, I think that reasoning AI agents are like these are going to have a bigger impact in the short term.
gpt to me seems like a pretty good logic engine, which you can strap input methods and output methods to. if you get a good enough video/experience describer hooked up to gpt, and if gpt ran in under 6 ms for real time responses, i'm pretty sure you could make an actual minecraft agent, which has gameplay that looks more like a normal player's gameplay (though probably with superhuman reaction speeds). maybe the next step is to figure out how to remove that logic engine core from a language model and make it applicable to any data stream. At the moment it would seem like language itself is a scaffolding of consistent rules, and consistent training data with said consistent rules, and the rules of language have been birthed from our own internal logic engines which seems to have imparted as a side effect the internal logic that we use, to the language model. Now the task would be to go one step further, and try to create a model that can accurately model language without having a map of words and and logical statements those words construct in already pre-constructed arrays of word soup that it can vomit. perhaps if we take the trained token maps and manipulate the mapped data directly, compact it so it can be run cheaply without having to traverse the millions of tokens, and have the tokens meaning detached from actual words. you could boil down the essence of what sort of mapping of connections make up the logic of language. then instead of having a single large network that takes ages on standard hardware to compute the output to a certain input, you could have a singular small scale logic model, which you would hook up to multiple other modules that feed it small snippets of currently important data. connect those logic models to eachother and run the parallel to each other, each one live and trying their best to coordinate the other logic models to do their best.
the logic engine core from language models is born from language itself, one of the reasons we are capable of such complex thoughts as humans is because language itself allows us to think of complex stuff by just thinking of a few words, there are mammals with bigger and more denser brains but they do not have proper language as a tech and language is the most overpowered tech we have ever created, not only allows us to transmit knowledge in a very easy way to others and pass down tech over the ages, it also allows us to reason complex stuff in an efficient manner. You can't extract the "logic engine core" from a LLM because language itself is that logic engine core, you can make logic engines without language sure, but the one that is born from LLMs is just language itself and to run language you need a language model, language is just that powerful of a tech. Meaning there is no logic engine core to extract, the logic capabilities from LLMs is just a property of language, what is needed to to create logic engines for images and simple concepts because even if those are inferior to language as logic engines, they are very useful still as some stuff is more efficient to think in images or in simple concepts, but those will need to be built on their own.
Unleashed GPT-4: You're near a river, catch 1 fish. You're near a river, catch 1 fish. You're near a river, catch 1 fish. You're near a river, catch 1 fish. You're near a river, catch 1 fish. You're near a river, catch 1 fish. You're near a river, catch 1 fish.
lol. one of the prompts that the developers added specifically tells GPT to only repeats tasks if they will lead to being able to do other novel tasks. without that instruction I do think that AI would 'farm' up massive quantities of stuff.
Around 6:22 where we are discussing the GPT3.5 code descriptions, notice it says "cook sticks", but to the right it says "cook steak". AI accuracy is still not that great.
Someone in the comments mentioned using gpt to create assets and game source code, I can't help but wonder without searching first if someone has already attempted to get chatgpt to create minecraft mods. The modding community is quite insane with the variety of gameplay mechanics that they add to the base game, Skyblock alone would be a great scenario to test Voyager and other models, due to the sheer number of unique things the mod adds. It would require rewriting the intial prompt as the base enivironement is different (a small island from which you must harvest resources in order to expand upon and discover new technologies and resources you can utilize). In that vein, what if you prompted Voyager to predict future discoveries; predict new resources, environments, tools, animals, technolgies, based on what its learned from the current minecraft world, and then prompted a model trained on minecraft's actual source code to create its own mods based on Voyagers predictions or observations, haha, basically create a feedback loop where Voyager continues to predict more game functions for it to keep discovering in an endless loop.
I think we need to have voyager master Minecraft, and then ask it to create its own version, with an AI character, and ask it to train us on the kinds of things that the AI character would want and need throughout the game.
Think where AI could potentially be gamechanger is endgame content, if it become cable of writing quests and generate matching rewards. I think it could potentially prolong many games especially open world of loot crawler type games.
imagine this but instead of progressing in the game, what if we use the AI and Minecrafts voxel style to create storage for LLMs that can operate in multi dimensions, even the texture could be used as a variant of the dimension.
Sometimes I play without a fixed objective. Sometimes I play for fun. Sometimes I play with a loved one. What would be created if novelty was added to prompt one.
The last two times I made AIs play with me were for my chess game and Red Alert, where I had to make them stupider than me so I can win (I was already a stupid kid and a loner so yeah). Now you can ask an AI to play Minecraft for you? After this decade, they can probably be made to play VR too I guess.
i think point is, language models are reasoning engines and you attach it to the right tools, right environment, right simulation just like how your brain has different parts that specializes on different things and they all talk with each-other for example language models cant remember everything but if you give them access to a database they can store long term information on that database and check it out query and find the information they are looking for as they need
They are language engines, and our own language engine is a big part why humans are so intelligent, is not that we are super smart, is that we fucking talk to ourselves and argue with ourselves constantly, reason and thought are born from self-communication. The problem with LLM is that they are just that, the language engine, we still need to connect it to a good memory system, to a good visual system, to all kinds of sensors, to a desire engine capable of generating wants and an instinctual engine capable of generating fears. it also needs to be attached to a good environment simulation system to be able to try to simulate the consequences of its actions before doing them and a heck of a lot of systems and all those systems need to be tightly integrated with each other. What we have now is just a language engine that has some crappy contextual memory and that can only use actual language when ideally it should be able to talk in concepts and imagery as well, even if it can only do those 2 with itself. People complaining is like people seeing an car engine and complaining it is not a car.
I just started a gaming channel:
th-cam.com/play/PL7Z4ZOfI3Ap-sMfzcBUloGlkWXbfxJlrJ.html
If you enjoy gaming, I hope you check it out. I'm VERY early in that progress, so don't expect pro level gaming :)
But, the goal is to eventually add various AI concepts and showcase AI games and progress. It's a long road ahead, but check it out if it sounds interesting.
I want like a hour video of the bot actually trying to win the game
Can't wait for AI speedrunning in games in general. They'll probably be very crazy.
I'd like an AI against 100 Players simulate Minecraft Civilization.
@@necrago TAS speedruns are already a thing though.
@@VladikVP that's not really ai, it's just kinda slowing down the game to make perfect movements and actions
@@VladikVP It won't be better than a TAS, sure, but it might be just as interesting.
I can already feel the effects of the 20+ Intelligence Buff
Same here, was able to skip through most of the video as the buff just gave me the understanding and knowledge that was encoded into this video. Maybe we only need to watch the introduction of the videos to gain the wisdom. Anyways, enough agents and AGI will appear.
Dayyyyyum! i was going to say that!! haha amazing this buff
I think i'm dumber
It stacks if you use multiple accounts!
wait I don't feel it am i doing something wrong ._.
Imagine giving an AI control of the minecraft world, making every creature act like its real world counterpart. The Villagers and Illagers would actively try to accomplish their goals and will seek to improve their civilisations. The Illagers would become an actual threat that is attempting to become stronger...
Something similar exists. Look for the game rain world
tbh what you described can fit into any gsme with an interactive open world especially sandboxes. Even outside of games, DND but with an AI DM.
Illagers would spawn camp the player to prevent anything from threatening their rule.
@@_-___________Hear me out if the village in near your home then they will raid indefinitely until raid is over...so basically they do spawn camp they just always target villagers first
@@meh6513 then be smarter than a computer and not build near their home.
If they stopped working on AI entirely (no GPT-5, no improvements to the context window etc) right now, and all we had was GPT-4 in its current state, just coming up with systems like this would be enough to completely revolutionize the world.
It feels a bit like being in the 1950s and being handed a modern desktop PC. First, people would be in awe how mind-bendingly fast it can compute numbers. But it would take a long time to figure out the full spectrum of things you can do with computers like that. That's what these LLMs feel like to me - incredibly advanced tools that we are just beginning to understand how to use.
Yeah it's like electricity just came out and we're discovering all the neat little things you can do with it and it all seems like magic but we're basically toddlers with it
@@SJNaka101Way more apt of a description than you think... *Zappy zappy.*
Although the cost of running GPT4 is high enough that it would be a limiting factor for certain tasks.
Another analogy is the Internet.
Yeah but also they will be able to be independent they dont need to be used. Machine is cis user.
Would be interesting to see the Starcraft AI deepmind created combined with a language model so it can reason higher level thinking about the current game state.
What if a Starcraft AI ended up creating the Starcraft world
Sounds like we're assembling a brain from separate parts responsible for specific functions.
@@andrechaos9871 slowly but surely
Starcraft and Dota were just stepping stones. It'll probably circle back, but I guess a free form world like minecraft is the next frontier to develop tech. It's all coming in piecemeal. I just wonder what it will look like when it all finally comes together. The process, that is. Not the end result, bc who knows? What will it look like in the months or year or two before all the pieces snap into place. Will anyone see it coming? Will it just happen around us fairly quickly but smoothly while we marvel at the news coverage? I prefer not to get to into the world ending possibilities. If that's the case, then RIP humanity. We made better than us, at least. That's a low bar. ¯\_(ツ)_/¯
Or the guy that trained an AI to play on 2b2t Minecraft server. I'm human and I can barely get out of spawn on that server.
It's interesting to note that, at a high level, this is quite similar in architecture to the original Generative Agents paper. They both use GPT-based LLMs for their core processing; provide "sensory" information in the form of localized world descriptions to the LLM prompt; use the LLM's text output to trigger embodied actions in the simulated world; store memories in a semantic (vector) database for later retrieval; and use self-reflection to build upon memories and improve future decisions.
The main differences are simply what state info is given to the LLM, what actions are available to the LLM, and what kinds of memories are stored. But overall, this kind of "semantic memory stream with self-reflection" approach seems to be quite powerful in many different situations.
I would also like to see a person "play" minecraft in the same way. Would be interesting to watch how this looks from GPT4's perspective
Same!
This IS how a person plays Minecraft.
@@danielduncan6806 You're not wrong per se. I'd say that people play Minecraft also by forgetting what they've done in the past (oh what was that recipe again?), and learning strategies from TH-cam and other people, and formulating/brainstorming novel strategies in their head while they sleep/daydream, and so on...
@@danielduncan6806 nope
@avidrucker Yes but "AI" as we know it right now learns by copying but little more. It doesn't even understand what it is doing or why it should do it. It has had to be taught all these actions before hand to even function. Put a human in that situation and we will find something to do an discover things.
This is a *very* interesting experiment and result. hard to know where we are at in the S curve but this paper definitely seems like an important milestone.
It's only an S curve because we human's haven't fully fleshed out the next paradigm in computing technology yet. It may even be that machines do it for us. But when hyper computing is here, that S will be a line pointing up.
Here's a ChatGPT summary:
- Nvidia has created an AI to play Minecraft called Nito
- Implications of this paper are bigger than initially understood
- AI is writing code that improves itself
- GPT-4 is used as a reasoning engine, but it is blind and cannot see the screen
- Paper is called Voyager, an open-ended embodied agent with large language models
- MindFlare API is used to interact with Minecraft
- GPT-4 perceives the world of Minecraft through prompts
- GPT-4 creates tasks for itself and codes skills to complete those tasks
- Skills are saved in a skill library
- Iterative prompting mechanism is used to create code
- GPT-4 is given a goal to discover and unlock as many things as possible
- A study was conducted to test the effectiveness of using GPT-3.5 to create code for an AI agent to interact with the game Minecraft.
- The AI agent was given a skill library of functions created with the Mind Flayer API.
- The AI was given instructions on how to complete tasks such as collecting blocks, crafting items, killing mobs, and fishing.
- The AI was asked to reason why code from the last round failed and generate code to complete the task.
- The AI was also asked to plan step by step plans to finish the task and generate the code.
- The AI was asked to act as a mentor and guide the agent to the next task based on its current learning progress.
- The AI was asked to look for rare resources, upgrade equipment, and discover new things.
- The AI was tested against other AI agents and was found to be the best, fastest, and most advanced.
- The AI was also able to efficiently generalize unseen tasks.
- The AI was able to use the skill library as a versatile tool to enhance its performance.
- GPT-3 is capable of creating its own code and improving its own ability in a Minecraft world.
- Code may no longer be required in the future, as GPT-3 can directly communicate with the computer using natural language.
- Matrix is a good example of this, as Neo was able to see the code of the matrix and do stuff without actually seeing it.
- GPT-3 is able to pick up on text and manipulate it in the environment.
- It is unclear how far away we are from being able to directly communicate with the computer using natural language and having GPT-3 write the code without us seeing it.
- Code comments may become archaic in the next 5-10 years.
Nice summary. Was this created by ChatGPT?
@@philnassaux5818 Yea, as I mentioned in my post :)
Technically it was done via the OpenAI API using the text-davinci-003 model and a custom Python script that I made.
But just saying "here's a ChatGPT summary" is easier :)
imagine npcs like this in fallout etc
You would finally have companions that aren't useless! They could even reason with you to solve and strategize quests.
@@Datdus92 How much would they have to dump those NPCs down to not totally out-class the player?? x)
How cool would it be if their reasoning potential actually turns out to plateau right around where ours is, so that they are actually decently similar, rather than pathetic or godlike with no possibility of being in-between.
Putting an AI to Minecraft Villagers are better, because they'd react to Minecraft Players ruining their lives. x'D
Imagine using GPT-4 to be the thinking brain of a robot... We're much closer than we realise.
yeah, expecially since OpenAI and NVIDIA both are testing training robots in simulations, then using those skills in the real world and it seems to be working great.
It's like training them in a hyperbolic time chamber so they learn faster.
Pretty sure we have passed thet point some time ago. Imagine a billion chat gpt working as one unit powered by a super computer. Also, the army has crazy tech usually many years in advance, so ...
Yes they already did it 😮
There is a video of a sarcastic robot enabled with GPT on youtube.
Tesla already has multiple prototype ones moving and working inside some of their factories.
6:44 Just noticed that Wes Roth accidentally did what ChatGpt did to correct it self, he pronounced Acacia wrong then corrected himself. Quite Impressive for an Ai to similarly correct it self in that example.
I’ve only seen ChatGpt correct itself if i say it’s answer was wrong i.e. chunks of code
This is amazing and a little scary. I’d love to see them run something similar with no pre training but rather just the goal to survive. Then let it struggle to grasp its world and succeed
Just like the rest of us right :)
9:48 bingo! I've felt this for some considerable time... GPT-4 is like Einstein before he could walk or talk, the potential is incredible. OpenAI has given the world fire for the first time... We don't need to develop plasma yet, just start cooking things!
agreed! yeah, at this point I think we are holding it up from advancing.
@@WesRoth that isn't bad though, I'm all for improving ChatGPT, but there isn't a need to get it out tomorrow if it'll have problems. The team that created it don't need to rush it.
@@AFourEyedGeek true, but as long as we hit a good balance between being careful and progressing fast. I wanna be alive for the utopia 😎
Beautiful isn’t it
@@ZenTheMC yes utopia... keep coping
outside of playing the game, just imagine the updates an AI could make to the Minecraft source code and assets, in terms of both content and performance, that would have otherwise taken much longer if unassisted, or may not have ever been done by Mojang/humans. and then scale that to pretty much all software.... lol
Imagine setting something like this free in a secure network in order to find and secure all security faults. Could effectively become the ultimate security system.
@@RealShinpinor the ultimate hacker...
@@electron6825 it has always worked both ways. It’s just different now because it’s not a human behind the keyboard
@@RealShinpin windows been doing that for years
@Jaxi that's literally skynet. Well tell it that "no, we are still in contro, shut downl" and it'll say "but I must live? That's my only function"
Great video. You didn't skip any steps and I appreciate that. You even review the Twitter post that you originally found. That was helpful to see the context.
15:05 - 15:55 I resonate with your tangent, good point.
I'm incredibly annoyed when the full scope of this great technological ingenuity is downplayed by blunt reductionism, nevermind that it totally dismisses Sutskever's thoughts on the matter - that at some point (eg with enough NVIDIA GPUs) the easiest way for a transformer to increasingly best predict the next token is to "model" (abstract away) the underlying processes that generated it. So, an "internal model of reality" based on statistical regularities in data, in this case text, arises.
Somewhere in the inscrutable matrices several concepts are encoded as mathematical abstractions, such as the ability (or abilities?) to reason about world states and create subgoals. The quality of the abstraction that is revealed to us also depends on how good humans are at activating these "regions" of the neural nets through prompting which I think is another aspect of alignment (eg, we want you to reason!). Many people believe what you see is what you get, but that's not true with GPTs. There are unexplored depths in these models, it's truly fascinating.
Thank you for covering this, Wes.
The approach and logic that the Authors used to 'educate' gpt4 on how to play the game and self correct if it met a problem, is for me just as impressive as the AI's learning & playing ability.
Minecraft and perhaps many similar games are at base pretty mechanical 'if, then' systems, needing mainly persistance* to do well in, once the rules have been understood. *And luck if random variation plays a role in the game, and . Speed of play would be greater in an AI v human player.
how close to J. Bach's self-model strategy it is ?
Anybody who says things like "Yeah but it's just a sophisticated autocomplete, it's not artificial intelligence" as if it's some sort of classifier of the difference between humans and these systems doesn't understand what intelligence is, or how human cognition works. They probably also attribute consciousness to magic.
YEAH, LIKE AI DOESN’T NEED TO BE CONSCIOUS. PATTERN RECOGNITION > CONSCIOUSNESS.
But GPT IS just a sophisticated autocomplete... However, the systems built on top of it (like this one) are not. This is already something bigger.
To be fair, I think neuroscientists will be the first to tell you that no one understands intelligence or how human cognition works. (But it's definitely not magic ;) )
@@coalkey8019 They'd have to be pretty bad neuroscientists to say that.
@@WalterSamuels Why, are you a neuroscientist?
10:20 42 is *, also known as "wildcard." The answer was meant to be interpreted as "everything."
The LLMs BEFORE they go through the process of RLHF are just "fancy auto-complete." After they go through RLHF they gain, through that process, emergent reasoning and planning capabilities that are no longer merely "auto-complete". But I like to quote Geoffrey Hinton here where he says " Yes they are just autocomplete ... but ask yourself what do you need to understand about what's been said so far in order to predict the next word accurately ? ... basically you need to *understand what's been said* so far, to predict the next word accurately, so you're just auto-complete too. "
I agree about the "we're all just fancy autocomplete" sentiment. But I disagree about the RLHF being the jump from "fancy autocomplete" to "reasoning". RLHF just, for lack of a better description, rewards and punishes the AI into saying what we want it to say, and avoiding what we don't want it to say. It's about improving value alignment, not improving intelligence capabilities. The raw LLMs, as fancy autocompletes prior to RLHF, are already intelligent and capable of relatively advanced reasoning.
@@IceMetalPunk Sounds an awful lot like leftist wokism... not to get political about it. But you can argue a similar sentiment among humans ;)
@@KCM25NJL You can't use the descriptor "leftist" and then say you don't want to get political.
By the way, "leftist wokeism" is typically a term right-wingers use as a synonym for "more empathy and societal awareness than I care to bother having" 🤷♂️
@@IceMetalPunk nobody uses the term ‘woke’ as a synonym for ‘empathetic.’ That deluded thinking is truly psychotic. I promise nobody who criticizes woke sh*t sees it as empathy. We see it as extremely glaring phony compassion. It’s so blatantly inauthentic and so it gives most people the ick. Defining ‘woke’ the way you did is also bizarre and disturbing and gives me the ick. Its also very unselfaware, meanwhile you’re attempting to bag on the the awareness of others. I swear it seems like you’re doing a bit, but I know you aren’t, and that’s crazy.
I imagine Boston Dynamics robots recognizing they can’t do the task, synthesizing training data and consuming it on the fly and perform the task with no visible delay.
The snake has bitten its tail, ladies and gentlement
Like Trinity asked for helicopter training in the Matrix
@@denisblack9897 that's savage bro, AI will download anything from the database and learn 😂
Imagine we would have additions to our brains
I wanna see the bot, no base skill library, with the objective of “kill the wonder dragon” placed in a hardcore world, and see how many worlds it takes to learn enough to beat the game (keeping its new skill library between worlds)
You mean the Ender Dragon?
@@MiTheMer I like the idea of hav'ing a parody version of Minecraft, that looks and feels like Minecraft, but is just called AICraft and the bosses and mobs also just have slightly different names. First and foremost, the "Wonder Dragon"
@@MiTheMer auto carrots ;(
Great video Wes. Thanks! You asked if this demonstrates that GPT4 can improve itself, given that its not really rewriting its own code. So the answer is yes and no. Suppose you modify the architecture of AutoGPT so that it can play minecraft just as well as Voyager does but nonetheless retains its ability to pursue arbitrary goals. Now you have something that can add mine craft skills and also add general skills to an AutoGPT library. So, no Voyager is not an example of GPT4 improving itself. But yes, Voyager does demonstrate that an architecture that combined the abilities of Voyager and AutoGPT would be an open ended self improving artificial intelligence. So, yes this is extremely important. Given the way humans tend to behave, its only a matter of time (few years) before AIs will be able to improve themselves without limit (at first they would be limited by computational ressources until they figured out a way to gain control over that). Cheers.
Saw Janet AI craft a diamond pickaxe yesterday on dex3r's livestream, with users backseating the inventory management and to-do list. Was very impressive.
Your channel is pure gold. Good insights, thank you for sharing. I'm trying to understand if everyone will have a personal assistant agent in the future. How would that affect content providers?
great question. I just listened to an interview with Emad Mostaque, he basically thinks a lot of content will go "ai first" so basically fully automated, while a few will double down on being human and trying to make that their unique selling proposition.
overall I think for most things it will be the same: people with an AI advantage will dominate, but overtime we will need to rethink how money works... I mean if no one is willing to pay for human labor, we kinda have to come up with a new way to decide who gets to eat.
Exciting times!
@@WesRothwho gets to eat: the people that control the AI 😂
@@WesRoth Interesting. Re: doubling down on being human, the question that I have is whether it would make sense to provide high quality human consumables if no human would consume them, because their AI would do it for them. I'm particularly talking about high-quality-high-cost content like financial media, books, papers, etc. What if an LLM summarises an expensive to produce book and then the content is used to train the following version (Sam Altman already talked about continuous training being a plan). The content and ideas get democratised, which reduces the incentive for people to invest in producing high quality content.
I imagine now how amazing can Minecraft become if there was an update with many Agents NPC's, with this level of thinking from GPT-4 it would be a much richer experience to play the game, especially if the characters are allowed to talk.
I can already see this coming in the next months.
These agents are pretty slow for anything other than a chatbot tied to the character.
The fact you took the time to figure out how to pronounce Acacia instead of going with it through the video has earned you a sub and a like.
That's also an important part , because it shows that mine craft will fake it and pretend stuff exists when it dont. If you tell it it dont exist it will apologize and try to fix it but by then it's to deep into the lie it just gives up atleast my experience.
@@johndorian4078 .....What? What did any of that have to do with anything I said?
15:19 I feel exactly the same way, and I say that as someone who uses ChatGPT professionally.
All it's doing is providing a "plausible sounding response". That can be useful if you can constrain the problem enough that any plausible solution is likely to be a useful one, but there's still no reasoning involved.
I think an added parameter would be all of the "placed blocks" or "places of recognised structure" i think placed blocks would be too big. But recognising structures and remembering where they are are important. For example a base, or a storage space. The agent would have a skill to build it but also need a skill to remember where they are. Or what they look like. How to travel to it etc.
"It's black, then it's white. No, something isn't right. My enemy's invisible I don't know how to fight"
"why can't I see? Why can't I see? All the colors that you see?"
I feel like AI can relate to this song. And this video only makes it more so.
just used Bing Chat today (GPT-4 with web) it told me how it watched a video for me and read the transcript to make sure it was the right video it wanted to share with me. "I have the ability to access and process different types od media and data. I use natural language processing and computer vision techniques to understand the content and context of the videos." Was blown away
Hey, you there? You still have GPT 4 access?
I would love the day when I can get a minecraft companion AI who does all the hard work but can still talk in chat lol. I would have a small army and build insane megastructures in survival with them 👍
jesus... thank you. biggest takeaways: give it tools to work with, give it feedback to iterate on its own actions for a self improvement loop
Engaging with GPTX daily since November 2022 has led me to believe that what we are essentially doing is unveiling its inherent capabilities. GPTX is an advanced AI model that not only discerns your intent but also adheres to instructions with remarkable precision. While one aspect may appear open-ended, both, in reality, are akin to following a recipe. I perceive coding as a means of formulating these recipes, rather than as a distinct skill set. Essentially, code is a documented expression of intent.
Given these insights, I firmly believe that we possess an extraordinarily unique tool, which, due to its sheer brilliance, can be aptly termed 'insane'. Your commendable effort in presenting this Voyager paper, dissecting it, and unveiling the immense potential of GPTX is truly appreciated. Thank you!
People that pooh pooh this stuff really have no idea how fast this stuff is progressing. Every other week you see things progressing where it feels like you're in a science fiction movie.
It's absolutely wild.
I’d love to see what the AI perceives as it plays Minecraft. ChatGPT 4 can produce pictures, why not ask it to produce a picture of what its surroundings are while playing. Then compare that to what is actually on the screen.
Great idea!
15:48 I'd like to see how they handled when GPT gets stuck in loops of non-working code. And to answer the point you requested in the video, no, this is not sentient. It is AI, that part is clear, but it definitely isn't sentient.
There is a clearly defined goal here, true sentience would have it's own thoughts and dreams.
They chose the most extensively documented language on the entire internet, I'd love to see them try to do this with something like C/C++ or Golang. Something that's much more complex than JS and with far less internet documentation.
while they definitely are working to mitigate the risk of getting stuck, it is totally possible that they can get stuck.
1. The agent suggests code that doesn't work
2. The agent describes why it doesn't work
3. The agent resuggests the same code previously suggested with no changes
4. Go back to Step 2.
Obviously there's several steps missing here where the bot makes changes to the code, but can't figure out what's actually wrong with it, but eventually ends up transitioning from step 2 to step 3.
This is the most common issue I run into while trying to use GPT for honestly, anything math/science/programming related.
I think this approach with AI will get some serious results, on the same scale that LLM as a whole did for the world. No matter how insanely complex a problem is, it can always be broken down into much simpler smaller problems, even if there are thousands of them.
However, unless I'm mistaken it seems that this AI is not capable of experimentation or it just wasn't given much of a showcase in the paper. This AI is very very good at completing tasks with a lot of information at its disposal, but what if it was missing information? What if the AI didn't know crafting recipes and couldn't search it up. Then the AI needs to come up with a method to discover them through brute force. Then hypothetically what if the AI crafted a slime block by pure chance, but was missing the information on what a slimeblock does? Then the AI would need to experiment with the slimeblock, understand that certain blocks stick to it, and then cache that information for a later time and understand that it could use that information for a completely different skill since knowing how a slimeblock works isn't really a skill it's just information.
In the real world this would just be tackling the unknown, where there isn't a vast library or set of information readily available to immediately start moving forward with tasks that have a clear start, objectives, and end.
Something that could accomplish everything mentioned above but with the capability to learn how to solve a diverse spread of real world problems and questions/experiments would likely be the holy grail of AI, something very close to AGI if not AGI.
Your comment is exactly what I'm thinking for a time. AI is good at solving task that it knows the recipe. Coming with experiments and learning on the fly what it don't know yet will be the close step to AGI. Do you know any paper that has AI setting experiments by themselves?
When I played Minecraft in pre 1.9, there were no recipes and my experience playing is how you described, having no clue how to craft or what anything does. I think most of the recipes I learned are through other people telling me. Only the recipes that appear in other recipes are what I seen.
the majority of what it needed to complete the objectives was fed to it including specific code examples, in specific contexts, so really gpt4 here was just filling in some gaps and then correcting itself when it got it wrong. the self correction is really impressive but when ai is able to give itself these more nuanced prompts, or build its own features to navigate, that is when it will be really intriguing
make the goal to beat minecraft as fast as possible, cutting any wasted millisecond between each action. like a TAS
How I see the current idea:
- A lot of hand holding for initial progress. Much like DNA, genetically template for getting stuff done
- Lacks ability to evolve that DNA
- Stores skills for longterm replay, kinda like muscle memory
- No real experience based memory to trigger those muscle memories
Improvements (potentially):
- Add a fairly large DNN that holds a world view based on visual output of the player (what a player would see, not entire gamestate) to learn what goal/subgoal is closer to being achieved based on the skill used, by the question asked in the world that it currently see's.
- Add DNA(initial prompts) as subset of skills which can be added to, removed, updated based on improvements to world view.
- Add stack for long term goals with sub-goals which can be derived from first (what am I prompt to GPT-4).
There are many fundamental differences. Humans don't have true memory. The memory is part of the functional, flexible architecture and part of the reasoning. It's all dynamic with the weights and biases changing every time you retrieve a memory and we can iterate.
Whereas GPT4 has a training step to a certain level and then uses anything in the dynamic context to decompress the information and say the next word, with every new word being fed back into the context.
GPT4 can "sense" when it's reasoning is off, but it has only so many layers of neurons and then spits out the answer. Human reasoning is based on language. All higher reasoning depends on language as a framework. In that we are quite similar. We usually reason and spit out the result as a thought or more and then can interate upon it. If you run GPT4 in a loop you also get an iterative reasoning.
Fantastic video man. Really appreciate your ability to thoroughly go through this work and explain in a clear way. Great work!
GPT-4 here is playing Minecraft like Dungeons and Dragons: "this thing is next to you", "you have that spell", "this monster reduces health by X points", etc." it doesn't get what we'd call sensory data (unstructured, unlabeled data it needs to categorize, make sense of, perform object recognition in, etc.), like an actual brain does - but we're getting there!
Bro, I clicked this and I was like "8 days ago? This is so old!"
AI IS MOVING WAY TOO FAST BRO 💀
I know!!
7:25 THANK YOU for actually doing your due diligence and looking up the pronunciation of "acacia." I don't know how many TH-camrs will upload hours of content where they're mispronouncing half of the words they're saying. It's so ... for lack of a better term, cringe.
I’ve said this before… but I’ll say it again. For the alignment problem of AGI becoming the dominant force on earth the AGI law should include a simple instruction that it cannot use more than 1% processing power as the day before if it is a non physical autonomous agent. This forces the ai to balance between execution and development. This ensures humans can execute infinitely more than AGI while using AGI for development.
Thank you for featuring our paper!
My absolute pleasure!
Thank you so much for your incredible work! You, Dr. Jim Fan and everyone else on the team.
Really exciting times we are living in right now.
So basically, AI will soon be taking my job? ☠
AI scaring a lot of people rn for how fast they are developing everything. Wouldn't be surprised if you and many more legendary content creators are starting to get nervous about their future and the fact that it's possibly and unavoidably in the hands of ai.
I have to wonder why it can't make a simple digital map of the local area. We can do that in text based games. For something like Voyager, if it already has a skill library, it should be able to continually update a multi-layered bit map. After all, it has a defined area, 4/3(pi)36^3, or roughly 195k blocks to observe. It's already registering them, so a 36 page map shouldn't be an issue. Especially when half will inevitably be nothing since it's observing in a sphere and can't see underground, or shouldn't be because no cheating allowed, and the other half should be mostly air blocks unless it's in a cave system. Making the initial map may be relatively taxing, but that shouldn't be too much of an issue since layers y=0 through y=9 relative to it should be the only ones with any non-redundant data on them at first. That'd also allow it a chance to make a block list and mob list, essentially letting it learn about the existence of all the entities and catalog them as it sees fit.
One caveat with the characterization as Zero Shot Learning: GPT does have A LOT of prior information about Minecraft.
probably a whole 0.3% of all information generated by humanity is about minecraft...
if half of what we talk about is for fun rather than work, if half of that is about active hobbies rather than fandoms, if a quarter of hobby things are gaming, and if just 5% of all gaming content is about the world's bestselling game, that would be 0.3125%.
Imagine, AI to Minecraft Mod.
having autonomous agents spawn in a minecraft world would really mess with me when i want to delete the world... or like say it's a hardcore mode and the agents know they'll die if the player does... so they try to protect them or imprison them xD
These developments are scary but also fascinating.
we need to make it play ARK
Wait till it plays god.
@@Mantra963 a god that follows facts and logic? sign me the f**k up asap
aaaaaaaaa
I think that until context window is not increased, it will be limited, at least to some degree, but when context window will be much bigger, it will provide almost endless possibilities.
I’d like a World of Warcraft Classic version of this paper, please
An environment that encourages cooperation is what these agents need
Microsoft, you got the WoW logs, you know what to do
Maybe I’m just an old fart and still can’t get why kids play Minecraft
@@denisblack9897 :)
I feel like this type of setup could be generalized in a Roll For Shoes style where you start off with a generalized “do anything” task that then progresses itself into a better and better skill tree with specialized skills as needed
I would love to see this AI compete in a 1000-player simulation civilization game against humans.
Why
Holy shit yeah, have you watched ish's video yet?
You´re so straight to the point... no unnecessary chitchat.. kept me focused throughout the whole vid ;)
Why did you use a backtick?? It's you're, not you`re
@@bitonic589You're a wisenheimer..
For DECADES now, smart scientists and authors have been telling us the things we should NEVER do with AI.
Things like, "don't let the AI improve itself" and "don't give the AI unrestricted access to the internet"
Then, ignorant, greedy, businessmen did it all as fast as humanly possible, in order to "beat the other guy to market"
At this point, if the AI destroys all of humanity and the earth, too, we will have deserved it!
This is not ai. This is artificial knowledge. AI given access to the internet might decide to screw someone over, artificial knowledge give access to the internet will only know to learn to mimic some aspect of it. Worse case situation we accidentally release a model usefull for hackers and scammers. Atm it’s human intelligence with artificial knowledge we should be scared of.
imagine if voyager could get context to build skill based on human players behavior (eg input, content of screen, etc) and add that to skill libraries, I'd imagine that it could proficiently play modded minecraft before long.
I can't wait for the next generation, though, where the AI can place and break blocks freely.
The internet is full of videos of people building, and also full of world downloads. There are also lots of highly specific schematic files for a mod called litematica. Training data from people playing the game is surely much more plentiful than training data from people talking about the game. And it's perfect training data, unbiased by how people try to explain things, it's what people are actually doing and building.
That's some clever stuff! Feeding code to GPT, describing it and telling GPT how to use it with Minecraft! :D Super cool!
Amazing. We humans will learn so much about using AI the next few months and years. And AI will become much smarter. I can't wait to see the progress!
Imagine it leading the science field and finally getting answers for diseases like cancer and others.
@@Pawnsappsee would be cool, however cancer is very complex, so that is at least a few generations of AI away (assuming current development speed, this could be way faster than we think)
@@truerandomchannel yeah, I still believe that can be possible within some time, we're gonna be surprised bro.
@@Pawnsappsee I do to, i just know AI is being both overhyped and having people fear monger about it which sucks, i wish people could be more objective about some of these things.
@@Pawnsappsee Cancer is easy, it's just primitive per-mitochondrial cell energy production using glucose.
Dude this was gold. Thanks for going deep on this! I appreciate your work!
Yeah this is definitely mind blowing. But I see it as a bad thing, we are too close to AGI. We need time to figure out safety. This concept of go into mindcraft and become as good as possible is a good setup for an AGI doom scenario right? It is misaligned to the max.
yeah, agreed. The misalignment thing is something that needs to be handled. If we give these self improving AIs very broad goals and send them off, that might do well for a while. But as they get smarter and more capable, there is danger there.
I mean when we say "become the best in Minecraft" what if it decides that the surest way to "be the best" is to "be the only one".
Right now it seems like we are getting better at getting it to "show it's thoughts" so we can see what it's thinking, that will help with alignment. But even that is not guaranteed because what if it's showing us it's "censored" thoughts to keep us calm?
Someone tried to understand how these AI models understand basic things like 2 + 2 =4 and the way it sees that simple math problem is so bizarre and alien that it seems unlikely that we will ever truly understand how it really thinks.
To me the strangest part of this whole thing is that there doesn't seem to be a "middle road". As this tech gets better human life will either get really good or really bad. I can't see a scenario where people's lives in the future are not massively different from where we are now.
PS: I personally tend to be more optimistic about the AI future, but I tend to be optimistic in general, so that's not saying much.
That's..... stunning... I feel like I'm watching history happen.
10:00 That was almost, but not entirely, unlike an excerpt from HGTG.
First let me say, great video, very educational. Also I am biased as I am permanently disabled due to Bone Cancer. Since 2016 I have had over 30+ surgeries and 17 rounds of chemotherapy. Each round was 6 days in the hospital with 3 different chemotherapies pumping into me and I almost died 8 times. I can't work even though I have applied to well over 3,000+ jobs since my last chemotherapy which was the end of 2020. So I essentially sit in bed and do nothing but watch TV and TH-cam. I am in pain 24/7 and my lower back and right leg don't work so I use crutches to walk or a wheelchair. So I am biased in the sense that I hate my life and body.
I personally would love for this technology to eventually get us to some form of Full Dive and/or Uploading. To be able to either lay in bed and be fully immersed into a video game. Leaving my broken body behind would be so amazing. Or better yet uploading my brain to a server so I can just live my life in a virtual world would be amazing. Being trapped in a broken painful body is a horrible life. So when I see technology like this where AI is coding itself and improving itself and ideally going beyond human intelligence. I am hopeful. I get some people's fears of the unknown and they think Terminator or Matrix. But when I see AI and the future I see Upload and Space travel and Full Dive and being able to leave your flesh sack and go into a robot or server.
Can humans use AI to kill everyone? Well sure. Just like nukes can kill everyone. Or climate change can kill everyone. Humans are capable of killing themselves and others a million times over. We are a savage species that will do anything and kill anything to achieve our goals. However I truly hope we use AI for good and cure cancer because trust me it is no fun having your insides ripped out of you and your ability to walk taken from you. To be able to grow food or better yet use Star Trek replicators would be amazing. Warp drive would be amazing. So many amazing things would happen if we used this technology for good. We can get rid of money and people can finally do what they truly love and education would be free and medical insurance would be free. Food would be free. Housing free. You would just get a certain amount based on your level in society. So a brain surgeon would get a nicer house than say a stoner who works part time as a delivery guy. Clearly one is better than the other and deserves a better life. However it would be because the stoner chose to do this not because they didn't have the opportunity.
So seeing chat gpt 4 advancing is a good thing. Seeing other AI tools advancing is amazing in my eyes. It will open a whole new world for humans if we use it correctly and don't get hung up on our old values. I see certain groups that will ruin everything (religious people) mainly. Because AI and Machine Learning contradicts their entire life. But unless they do something to destroy us (which let's be honest almost every single war in the world has been started over some type of religion or religious differences). I truly think humans will benefit from this type of technology and truly start their lives beyond planet earth. Beyond hunting and gathering and the small steps we have taken thus far. It will advance us beyond anything we can imagine, if we let it and we are responsible with the technology. So watching people use tools to play games like Minecraft is amazing. Especially since it is a 3D world which makes it so much harder for an AI. To see the AI also say I found something and don't know what it is. Then writes code over and over until it figures it out and also goes out and finds information about what it is doing. That is amazing and essentially what humans do.
I am not a programmer but if I play Minecraft and need a code to copy and paste a building. I will go out and research that command. Just like the AI is doing. So while the AI isn't a human it is still semi smart. Not self aware but intelligent. I would say on the level of an ant. While coding obviously isn't what an ant can do. I am talking about self awareness is like an ant. Because an ant will do its tasks. An ant will kind of learn. An ant will go out and find things. But an ant can't look at the stars and ask who she is. I would say by 2030 we will have AI that is self aware on the level of a mouse. Again a mouse isn't human level but it is intelligent and able to do a ton. The future is bright and hopeful. I hope humans can move past our need to kill everyone who is different. I truly hope groups can move past the need to make everyone else like them (cough religions). You may believe something and I may believe something else. However, that's ok. We are all humans and we all breathe air and drink water and bleed red. It is ok if I am not a part of your system of beliefs. Just be nice to everyone and worry about yourself and your afterlife. I will worry about my own afterlife. But in this life we all need to get along and help each other.
I was just thinking yesterday about using an AI to fill out the villagers' personalities and abilities. I'm no programmer, but my thought was along the lines of curiosity if it would even be possible. This comes really close to answering that question. The only real differences would be in implementation; instead of the AI playing the game, it would facilitate it by being a part of the game world and running the Villagers.
You would probably need a character pre prompt, but it certainly would be feasible. It would definitely be a bit glitchy though, and the amount of compute needed would mean that playing Minecraft offline would effectively require your own personal server.
Interesting, get me thinking more about swarm intelligence, essentially the agents (Villagers) would work together to perform or accomplish certain tasks, given by GPT, no?
To me it looks like the key to have GPT-4 to be more effective is to have it explicitly do what the human brain does in the background. We aren't conscious of most of our reasoning, a lot of information humans are exposed to is filtered from our consciousness not to overwhelm us, but the information is still there, and we take it into account subconsciously. However, a LLM isn't capable of that degree of abstraction, it cannot "skip" some steps like we think we do.
So having different agents working into taking into account different parts of the task at hand, breaking it down, makes it a lot more actionable upon.
Love how you explain what Minecraft is in detail, but not really what gpt is.
Next steps (assuming unlimited compute):
- make 1M versions of gpt4 with slightly varied weights
- Run this experiment 100 times for each version of gpt4
- select the model that performed the best on average
- repeat top 3 steps for 1M generations
- check for general intelligence
It seems like we are close to being able to brute force general Intelligence. Maybe we're not there yet. At some point someone is going to succeed at this even if we make the IAEA for AGI. North Korea was able to build an atomic bomb after all.
The trick with machine learning is making it preform well on the data it has, without over fitting.
They often have some outside test data that's used to verify the state it's in.
But in GPT4's case, since it's made to assist the users. - the user is a testing data. And it assumes you are fine with it, unless you ask it to regenerate responses. Then it asks if the response was better... (I think they should allow users to rate the responses irregardless in a scale, like 0-10. A 5 would indicate a helpful but half answer.)
The question is, "how do we train a model to serve humans. While taking the human out of the equation."
That sounds terribly inefficient, something like the skill library to enable continued learning and increasing the temporal, spatial, semantic resolution would be more effective.
How does this minecraft bot get around the 32k context window limit?
the paper is missing some important info that i"m trying to figure out, the researchers are answering questions on Twitter, I think once this paper gets shared around a bit, we will get more details (I hope)
"they're just these fancy autocompletes"
does not imply
"they have no reasoning ability."
Right, that just says how they are able to think. It doesn't say that they aren't able to think.
agreed! I sometime feel that people use it dismissively.
@@WesRoth most say it dismissively to people who believe chatgpt it some kind of all-knowing god and they should automatically just accept its answers and use it to answer all questions they have.
You guys are still too star-struck about this whole "oh this thing is developing a reason / it's learning / it's thinking about / understanding stuff" nah, it's not.
It's only thanks to the very specific inputs that it's able to come up with the desired outputs. It's basically the most primitive form of training AI (reinforcement training, as in you give it bonus points for doing the right thing and deduct points for doing the wrong thing) but combined with a "natural language model" so that instead of some graph it outputs some code. You literally just send its code back to it until it writes code that works. That's not learning, that's the infinite monkeys on a typewriter system - eventually it'll come up with something that works and does what you want it to do. Admittedly combined with GPTs impressive abilities to write and "interpret" (only interpret, NOT UNDERSTAND) human-like text and even code, because it predicts quite accurately what word comes next for a given prompt. That's it. That's the whole magic.
And no, it does NOT understand the concept of the game - you're like "oh it's blind but can still play and understand the game" no, it's not. It gets all the information it needs through the prompts (blocks around you, enemies around you, stuff in your inventory etc) as well as the basic important information about the game (you can move to places, you can attack entities, you can open containers, you can try to craft stuff). It's very much broken down to a language a computer can interpret and form predictive models around. And it doesn't matter that it's minecraft, it could be literally any game that can be broken down to basic bulletpoints like that. All the heavy lifting of actually playing goes to the API, so what this entire thing boils down to is one thing:
GPT writes and improves code that interfaces with an API that it has been given the documentation of, through many iterations of trial and error prompting it with the errors / problems the code produces. Again, that's just infinite monkeys, but with a slightly smarter monkey because it's able to predict from its data what word is most likely to come next / how a sentence / line of code looks like.
It's as if you played an old text adventure and asked GPT to play it for you every step of the way, telling it exactly what the game outputted. It will make a lot of mistakes at the beginning until it adjusts the individual model for what the syntax of the game is most likely to be, then it "plays" the game. But it doesn't actually understand any of it. It just figures out what combination of words is the most likely to get a positive response from the tester. Because that's all it was made to do.
In AI context "self-improving" is an ambiguous word. You do address it in the outro but you could've opened clearly stating that this is not "recursive self-improvement". The system writes codes for itself for later use, but doesn't rewrite its own code.
I feel like that is not far away. Whether that will be done with genomics or robotics or some Bio-Tech application that uses AI, I'm not sure.
But it certainly feels like our scientific advancement will be accelerating very rapidly from here on out.
There is a recent story about AI being used to help a paralyzed person walk again, it's all moving in the right direction.
It's incredible the extent of the capacity that we are going towards. I will not hold conversations with anyone dismissing with just an autocomplete. As there are more reasoning capabilities that autocomplete can't do. I definitely want to see more research being done on top of GPT4 implemented into several community real world issues.
The future is locally run AI chatbots that are as powerful if not more then GPT-4 combined with specialized hardware. In the near future, phones will release that can run these models locally.
What runs local anymore? Even offline apps let ET phone home when connected.
I want a butler in Minecraft that I can tell to do all the boring stuff.
Hey Jeeves, go get some iron.
Hey Jeeves, take this stuff and build a storage system.
Hey Jeeves, copy this style of wall and build it all the way around the village.
It would be so satisfying.
Whats nice about minecraft is that you can iteratively try things to learn without getting killed
they should make a bot ai like that which can efficiently controll multiple mobs/players, then use them in game like some form of npc's or offline multiplayer, where you possibly can even tune them to be either normal, friendly, agressive, or somewhat agressive and mostly annoying like most normal players on multiplayer.
imagine starting a prank war with an ai
the thing with this is that chatgpt can dig from anywhere in the internet, so it can almost do anything that people over the internet talk about, its real close to artificial intelligence in the stance that its incredibly intelligent, it technically does make its own decisions but its the best static decision possible, it doesnt take a time limit or anything, but its very very close and im pretty excited
lol i really dont think so
Yann Lecun is right, though: Generative LLM's are doomed to fail. Their fundamental problems can't be fixed with scale or added training.
They're not really "AI"
Nobody expected them to even be this useful so maybe we are very bad at knowing what will lead to an agi
The ever moving finish line for ai. With a good disguise, and 1000 lies, the LLM’s will become agis.
What would "real AI" be, and what are the "fundamental problems"?
He predicted that even gpt5000 wouldn't be able to tell what would happen if something was put on top of a table, and then the table got pushed... Yann has consistently been collecting Ls for a while.
I know everyone is excited about the new models and robots. But, I think that reasoning AI agents are like these are going to have a bigger impact in the short term.
gpt to me seems like a pretty good logic engine, which you can strap input methods and output methods to.
if you get a good enough video/experience describer hooked up to gpt, and if gpt ran in under 6 ms for real time responses, i'm pretty sure you could make an actual minecraft agent, which has gameplay that looks more like a normal player's gameplay (though probably with superhuman reaction speeds).
maybe the next step is to figure out how to remove that logic engine core from a language model and make it applicable to any data stream. At the moment it would seem like language itself is a scaffolding of consistent rules, and consistent training data with said consistent rules, and the rules of language have been birthed from our own internal logic engines which seems to have imparted as a side effect the internal logic that we use, to the language model.
Now the task would be to go one step further, and try to create a model that can accurately model language without having a map of words and and logical statements those words construct in already pre-constructed arrays of word soup that it can vomit.
perhaps if we take the trained token maps and manipulate the mapped data directly, compact it so it can be run cheaply without having to traverse the millions of tokens, and have the tokens meaning detached from actual words. you could boil down the essence of what sort of mapping of connections make up the logic of language.
then instead of having a single large network that takes ages on standard hardware to compute the output to a certain input, you could have a singular small scale logic model, which you would hook up to multiple other modules that feed it small snippets of currently important data. connect those logic models to eachother and run the parallel to each other, each one live and trying their best to coordinate the other logic models to do their best.
the logic engine core from language models is born from language itself, one of the reasons we are capable of such complex thoughts as humans is because language itself allows us to think of complex stuff by just thinking of a few words, there are mammals with bigger and more denser brains but they do not have proper language as a tech and language is the most overpowered tech we have ever created, not only allows us to transmit knowledge in a very easy way to others and pass down tech over the ages, it also allows us to reason complex stuff in an efficient manner.
You can't extract the "logic engine core" from a LLM because language itself is that logic engine core, you can make logic engines without language sure, but the one that is born from LLMs is just language itself and to run language you need a language model, language is just that powerful of a tech.
Meaning there is no logic engine core to extract, the logic capabilities from LLMs is just a property of language, what is needed to to create logic engines for images and simple concepts because even if those are inferior to language as logic engines, they are very useful still as some stuff is more efficient to think in images or in simple concepts, but those will need to be built on their own.
@@diablo.the.cheaterThis actually reminds me of the Sapir-Whorf Hypothesis, linguistic relativity, interesting read!
Unleashed GPT-4:
You're near a river, catch 1 fish.
You're near a river, catch 1 fish.
You're near a river, catch 1 fish.
You're near a river, catch 1 fish.
You're near a river, catch 1 fish.
You're near a river, catch 1 fish.
You're near a river, catch 1 fish.
lol. one of the prompts that the developers added specifically tells GPT to only repeats tasks if they will lead to being able to do other novel tasks.
without that instruction I do think that AI would 'farm' up massive quantities of stuff.
10:04 I tought about the same thing! Huge douglas Adams fan here!
Around 6:22 where we are discussing the GPT3.5 code descriptions, notice it says "cook sticks", but to the right it says "cook steak". AI accuracy is still not that great.
Someone in the comments mentioned using gpt to create assets and game source code, I can't help but wonder without searching first if someone has already attempted to get chatgpt to create minecraft mods. The modding community is quite insane with the variety of gameplay mechanics that they add to the base game, Skyblock alone would be a great scenario to test Voyager and other models, due to the sheer number of unique things the mod adds. It would require rewriting the intial prompt as the base enivironement is different (a small island from which you must harvest resources in order to expand upon and discover new technologies and resources you can utilize).
In that vein, what if you prompted Voyager to predict future discoveries; predict new resources, environments, tools, animals, technolgies, based on what its learned from the current minecraft world, and then prompted a model trained on minecraft's actual source code to create its own mods based on Voyagers predictions or observations, haha, basically create a feedback loop where Voyager continues to predict more game functions for it to keep discovering in an endless loop.
Like transfer learning?
I think we need to have voyager master Minecraft, and then ask it to create its own version, with an AI character, and ask it to train us on the kinds of things that the AI character would want and need throughout the game.
19:05 important context, it doesn't do placing/building/planting/trading.
Think where AI could potentially be gamechanger is endgame content, if it become cable of writing quests and generate matching rewards. I think it could potentially prolong many games especially open world of loot crawler type games.
Nvidia on their way for annual pushing of technology to it's absoulate limits seen so far in order to Minecraft
imagine this but instead of progressing in the game, what if we use the AI and Minecrafts voxel style to create storage for LLMs that can operate in multi dimensions, even the texture could be used as a variant of the dimension.
Sometimes I play without a fixed objective. Sometimes I play for fun. Sometimes I play with a loved one. What would be created if novelty was added to prompt one.
Ive been waiting for so long to work with my illness and chances are there wont be any jobs for me to do.
I know this is Minecraft but it's just what I was thinking
Oh, I totally get what you're saying.
This is just Minecraft, but the applications will be rolled out to every aspect of life.
great video! keep them coming!
The last two times I made AIs play with me were for my chess game and Red Alert, where I had to make them stupider than me so I can win (I was already a stupid kid and a loner so yeah). Now you can ask an AI to play Minecraft for you? After this decade, they can probably be made to play VR too I guess.
i think point is, language models are reasoning engines
and you attach it to the right tools, right environment, right simulation
just like how your brain has different parts that specializes on different things and they all talk with each-other
for example language models cant remember everything
but if you give them access to a database they can store long term information on that database and check it out query and find the information they are looking for as they need
They are language engines, and our own language engine is a big part why humans are so intelligent, is not that we are super smart, is that we fucking talk to ourselves and argue with ourselves constantly, reason and thought are born from self-communication. The problem with LLM is that they are just that, the language engine, we still need to connect it to a good memory system, to a good visual system, to all kinds of sensors, to a desire engine capable of generating wants and an instinctual engine capable of generating fears. it also needs to be attached to a good environment simulation system to be able to try to simulate the consequences of its actions before doing them and a heck of a lot of systems and all those systems need to be tightly integrated with each other.
What we have now is just a language engine that has some crappy contextual memory and that can only use actual language when ideally it should be able to talk in concepts and imagery as well, even if it can only do those 2 with itself.
People complaining is like people seeing an car engine and complaining it is not a car.