As an older developer, we used 'virtual memory' because in 1989 computers only had 640k and in DOS, there was no OS memory management. We would swap CAD/CAM geometry objects in and out of memory as they were needed. Please keep us informed as this project moves forward especially when it can use open source LLM's.
The more things change, the more they stay the same. I've been writing code professionally for 30 years, and every generation of 20-somethings "discovers" something some greybeard taught me when I was 20-something. Virtual context management. Imagine that. New since about 1970.
nitpick; PC from 1989 likely had more RAM than 640k, DOS by default just couldn't address more than 1MB directly (with 384k reserved for system leaving 640k for user) because of legacy architectural limitation of the original IBM PC from 1981 and the holy tenets of backwards compatibility. since around dos4.0 in the backwards compatible "real mode" himem.sys and emm386 could give access to higher memory areas but the proper way was to switch to "protected mode" that could address rest of the system memory directly (16M for 24bit 286 and 4GB for 32bit 386) usually with an extender library like DOS/4G which were around in -89 but maybe not so widely spread yet.
AGI doesn't need to be a memory machine. An agi good enough at comprehension and decision making is enough. Now if you talk about AGI with conscience, we are talking about elon musk level extinction
Your channel has distinctly carved its niche in the AI TH-cam arena. Among the myriad of AI TH-camrs I'm subscribed to, your channel, particularly over the last six months, has excelled in quality, presentation, and professionalism. Your videos have become my go-to source, superseding others that now seem laden with filler content. Your knack for diving straight into the core topic, elucidating not only the 'what' but the 'why,' is refreshing. The structured walk-throughs, practical guidance, and anticipatory glimpses into the future keep me engaged throughout. Your closing phrase, "And...I'll see you in the next one," has amusingly become a segment I look forward to; it encapsulates the essence of your engaging delivery. Being a part of your channel feels like being immersed in a thriving community. The clear, concise factual delivery, balanced with simplicity, makes the content accessible for newcomers while remaining enriching. Despite the crowded space of AI discussions on TH-cam, your channel effortlessly ranks within my top 10. Thank you for the enriching content and the community you've fostered.
Well deserved and well said! Amazing how you explain these matters for everyone. Any exec will be so pleased to have you guide them. And btw it doesn't matter whether the words were perfected by AI, it's the thought - the gratitude that counts.
Separating the conversation from an internal dialogue the system can have will prove very helpful: you can ask where the system has learned something to prevent hallucinations, have a space to run logical reasoning until confirmation, and now spout, “The ball has to be 10c and the bat $1.10… Wait, no.”
@@avi7278i used TheBloke/MistralLite-7B-GGUF and it was good TheBloke/Mistral-7B-Phibrarian-32K-GGUF is another option i've tried it wasn't as good for what I was doing but it might be better on academic datasets
My favourite open source model is currently _Falcon 180B_ with the web search feature. I was impressed by M$'s _Bing Chat_ in Edge, but I mainly use Falcon instead now, since it seems just as good for grabbing information from the web, at least from my perspective. Although I don't fancy paying to run Falcon on a server, just to test it with MemGPT, despite my eagerness to try it out. It could be interesting if there was a _Falcon 180B_ API, similar to OpenAI's API, only much cheaper.
The issue with uneven attention in the context window (that phenomena when only beginning and end is memorized well, but everything else in the middle is foggy blurry) was partially solved by Mosaic with their MPT models
D@MN you're good. You are on the forefront. Thanks for finding the material, breaking it down, and explaining how to implement..... It is much appreciated....
One of my first function calling experiments was having GPT-4 manage a couple of it's own context windows, and it really does a good job! Told it to use regex. Didn't go to this scale tho... Sounds really expensive!!!
Once we have a robust way of handling memory, like MemGPT, we can simply fine tune the LLMs to utilize the system. Then we no longer need to use context window space for the system prompt to operate the memory. The LLM will just "naturally" do it.
About lossy compression: it’s fascinating to me that lossy *text* compression can act as a normalizer, including replacing misspelled words or typos. I wonder if the output of recursive reflection is text or an embedding? As embeddings they can have more nuance than can be expressed in words (eg, “like a unicorn but even more mythical”) but that nuance could accumulate noise as well.
They should use both the uncompressed and compressed memories, the compressed memories offer a "preview" to the AI which it can then call more details if it needs to.
This was the first thing I thought of when I learned about token limits. I even asked GPT to create a micro shorthand language to condense info. It didnt work in April but seems like were getting close!
Thanks a ton, Matthew! That's such great news. One step closer to a real LLM-OS. Can't wait till they implement opensource model support. I also would like to see how things like LangChain, HuggingFace and others can get integratied into solutions. Would highly appreciate another video about this topics from you. Thanks for your great work! 💪
It looks like UC Berkley intends to release their own tuned version of Mistral-7B, Sounds like that project combined with their memory might have some amazing results for local independent research. Interesting they are on the Mistral 7B and not the Llama2 7b or Llamav3, This is institutional recognition of the value in this new open commercially viable solution and its plasticity to being fine tuned... I cant wait to see what comes of it definitely make a vid when its working with a local LLM, I suspect if Berkley is tuning Mistral for this use case it *could* be all local!
I've almost exclusively heard of Berkley in relation to jazz music education, so I can't help but wonder if they're *tuning* it to the jazz standards of "The Real Book"?! :D
things like textgen have qlora training built in that runs on fairly low spec hardware... have an option to train a lora from the long term memory on a schedule . start with a default lora trained on synthetic MemGPT input/output text pairs with the FULL Memgpt system header, then use short hand system messages during inference to give it 'reminders' on whatever aspect of the complete system protocol is the most important for that step.
Thank you very much for sharing this information! I'm very interested in using MemGPT with open-source LLM models installed locally. If you come across any new developments in that space, I would highly appreciate hearing about it!
🎯 Key Takeaways for quick navigation: 00:00 🧠 AI currently lacks memory beyond training data and is limited by its context window. 00:29 📈 Progress has been made to increase context window size, but still limited (e.g., Chat GPT-4 has 32,000 tokens). 00:58 📚 Introducing MemGPT: A solution to expand AI's memory. The video reviews this research and the open-sourced code. 01:11 ✍️ Paper titled "M GPT: Towards LLMs as Operating Systems" has several authors from UC Berkeley. 01:51 🗣️ Limited context window issues arise especially in long-term chat and large document analysis. 02:20 💽 MGPT mimics computer OS memory management, with an "appearance" of large memory resources. 03:27 📊 Increasing context window in Transformers is not optimal due to computational and memory costs. 04:08 🔄 MGPT autonomously manages its memory through function calls, enhancing its ability. 04:52 🖥️ Diagram explanation: Inputs go through parsers, get processed in virtual contexts (main and external), and get outputted after further processing. 06:14 🖱️ MGPT allows AI to self-manage context, treating longer context as virtual memory and own context as physical memory. 06:40 📟 Main context (like RAM) has a size limit while external context (similar to a hard drive) is virtually unlimited. 07:08 📏 Various models have different token limits, impacting how many messages can be processed. 07:48 ⚠️ Actual usable context is often less than advertised due to system messages and other requirements. 09:00 🔄 Recursive summarization is another way to manage limited context, previously discussed in another video. 09:15 🧠 MemGPT stores its "memories" in a vector database, but it eventually compresses them through a process called "reflecting on memories" to manage space. 09:56 🔄 Recursive summarization can address overflowing context but is lossy, leading to gaps in the system's memory, much like video compression degradation. 10:38 📝 MemGPT splits context into: system instructions, conversational context (recent events), and working context (agent's working memory). 12:02 🎂 MemGPT can store key information from conversations in its working context, as shown by a birthday conversation example. 12:43 💽 External context acts as out-of-context storage (like a hard drive), separate from the main context but can be accessed through function calls. 13:25 🔍 There are two types of external contexts: recall storage (history of events) and archival storage (general data store for overflow). 14:09 🧩 MemGPT manages its memory using self-directed memory edits and retrievals, executed via function calls and based on detailed memory hierarchy instructions. 15:32 🔄 MemGPT can correct its memory when false information is detected, updating its stored context. 16:14 🤖 The effectiveness of MemGPT as a conversational agent is evaluated based on its consistency (alignment with prior statements) and engagement (personalizing responses). 17:10 🎵 Through a function call, MemGPT can delve into its past memory to recall previous conversations, like discussing a music artist. 17:52 🕰️ Deep Memory Retrieval (DMR) enables the agent to answer questions that refer back to very specific details from past conversations. 18:05 📊 The accuracy of MGPT's responses is better than GPT 3.5 or GPT 4 alone. 18:19 🍪 Personalized conversation openers (like referencing a user's cookie preference) increase user engagement. 19:01 ☕ Examples illustrate how MGPT uses context and recall differently to engage with users. 20:12 📜 Many documents exceed the token limits of current models, creating challenges in document analysis. 21:06 🧠 Large language models exhibit a bias in recalling information towards the beginning or end of their context, mirroring human memory patterns. 22:44 📈 Charts indicate that MGPT maintains consistent accuracy regardless of the number of documents or nested information, unlike GPT 3.5 and 4. 23:12 ⚖️ A trade-off with MGPT is that some token budget is used for system instructions. 23:41 🤖 Discussion about LLMS as agents and their emergent behaviors in multi-agent environments. 24:21 💻 Tutorial on how to activate and use MGPT, starting with code setup. 27:35 📁 MGPT's document retrieval feature allows users to chat with their documents; using wildcards can fetch multiple text files. 28:15 💵 Embedding files come with a computational cost; example given shows 3 documents for 12 cents. 28:44 🔄 MGPT's persona is customizable, allowing users to tailor how the model interacts with information, like referencing archival memory. 29:38 🔍 MGPT can retrieve specific data from documents, such as annual revenues of companies. 30:06 🌐 Introduction to MGPT emphasized its rapid evolution and potential for open-source models in the future. 30:33 🎙️ Interview with MGPT authors Charles and Vivian discussing inspiration and plans for the project. 30:46 🧠 MGPT addresses the memory limitations of current language models by actively saving crucial data into a permanent memory store.
Thanks for the great content, Ive learned a lot from your Ai curation. Due to the pervasive sound effects I use subtitles on your channel. Keep up the great work
wow, very exciting video. It'll be awesome to get an updated video once they release a way of running open source models with this, even if they're not entirely capable yet. I'd wager that if mistral can successfully scale their models up to ~34B they'll probably be able to be fine tuned into a very competent function calling model!
Epic vid thanks for the insight. Seems like a much better solve than the attention sink paper. Excited to see this in the wild. The very basic install instructions seemed out of place.
Thanks 🙏 for all of the highlights. I have trouble focusing on my Sales job because I am trying so hard to learn bot 🤖 skills and reconsidering juggling the Newsletter information
A lot of things discussed here are very similar to how human memory works. We can hold a limited amount of data in our short attention window. Our brain can store a lot of long-term info but buries it deeper and out-of-reach if it thinks it's not currently relevant. It also seems to compress memories by letting us remember easily the most important details of an event but burying the rest deeper. And we have all kinds of techniques or "functions" to jog our memory to bring back old data we know we have somewhere, store more short-term stuff efficiently when cramming for an exam, and so forth.
The more i think about multi model systems the more i see similarities with how our brains divy up task priorities. Its both freaking me out and exciting me to be honest
Great video, Exciting stuff. Memory access is a huge limiting factor especially within coding projects so I can see this really moving things forward. It seems like this would give you the benefit of a huge LLM like ChatGPT that then can be "trained" or augmented to your specific use and data set while still retaining all of its full training data.
Amazing bit of work by this team. A thought... While I'm not going to be silly and say open source models are currently as good all around as openAI's offerings they're close in so many regards and are catching up fast in most areas BUT where openAI really has a lead is in things like Function Calling. I'm really, really hoping we see some innovation in this area in the open model space soon. Thankfully I do think that innovation is coming and openAI's closed ecosystem is going to be under more and more pressure. imo open models will eventually win. Thanks for the coverage.
I like all your stuff but this video blew me away. Even though you include a transcript, I had to keep stopping it to make notes. Well done. The only place I stumbled was the many different but slightly similar constructs. I'm still working to wrap my mind around that. For instance, you had a reference to system instruction, conversational context and working context. Later you refer to recall storage and archival storage which I assume are the same as main context and external context. Later you have working context and recall. I'm sure it's just me, but I'm trying to sort that out in my own mental model. But again, well done!
Matthew, this is sn amazing video. Remember the format, it's really good. Of course there must be a paper which is as good as this. But your way of explaining it is really well done
Thinking about what you’re thinking… subconscious analysis of thoughts based on beliefs… Seems like the CPU/Ram/HD analogy could be better replaced by how our minds actually process information. Love this stuff! Thanks for breaking it down
Yes please do another video if they incorporate open source. This looks awesome, but I don't want to use an Open AI key. Another great video BTW :) You are my go to AI TH-camr.
Matt, MemGPT is a further abstraction of a context window from an application level (Chat) to an OS level. OS level Context window could be in-memory (similar to cache) and on-disk. Cache size can be controlled. Anything that needs to be in context is cached, and the rest is flushed to disk. This process is operated as a function call. A long context window is almost essential to maintain a holistic context. The context is saved on your personal or work environment/device according to your usage. Context can leveraged for any required automation or building work efficiencies. 🙂How different is this from a traditional Windows-Registry with Name-Value pairs!! 🙂
this looks super neat. i have been having fun though with creativity of lllms by specifically pushing them past the context and token limits. that's when you seem to get some good creativity. though that's kinda the opposite of what you are going for here haha. great video buddy
Thanks for digging into this and explaining it so well. I have looked at this project a couple of times and didn't quite "get" it. Keep up the good work!
Imagine using MemGPT + Agents for large or multi-document analysis, where each Agent takes on a document (or section) and discusses with the other Agents their documents to answer user questions?
I wonder, if that could be used for example for applications to companies (for work): essentially, create a CV based on the profile of the company. for this, it would need to know a lot about the user and know which information is relevant for a job application and which one is not. maybe the system can also need something like memory-files? essentially store important facts about a person together so they can be queried together. I see massive potential in these context-based usecases.
Long term and short term memory. Imagine how cool its gonna be when they realize they can on schedule or on event splice in NEW info to the prompts... Cant wait to start my AI project at work...
Gotta pont out, that's pretty much an analog of how humans process memory as well. also when it says :working-context-append Key Personality high-speed, adrenaline-rush activities and intense gaming sessions in CSGO) a slightly more charitable reading of it would take the CSGO part as just a descriptor of things that are fast paced and adrenal pumpin activates like formula one racing.
While I acknowledge there are other similar concepts floating around, I think the MemGPT ease of use, documentation and open sourcing is a great resource. Maybe I need to read the entire paper, but I am curious whether the "working_context_apend" feature is self guided, or a schema specified by the programmers, eg. "Key Personality Trait" - did the LLM decide this was something to remember, or that was pre-defined ?
So who’s building something with AutoGen + MemGPT?
Please do! Autogen had blown my mind before and now combined with this it could be unstoppable!
I'm currently working on something very similar to that.
Ha! I was just going to suggest that. How can we get it working with a local LLM and autogen?
@@TheRealDOSmile How to contact you?
I probably am ;)
As an older developer, we used 'virtual memory' because in 1989 computers only had 640k and in DOS, there was no OS memory management. We would swap CAD/CAM geometry objects in and out of memory as they were needed.
Please keep us informed as this project moves forward especially when it can use open source LLM's.
LLMs*
Hold up a sec, I gotta find disk 2
The more things change, the more they stay the same. I've been writing code professionally for 30 years, and every generation of 20-somethings "discovers" something some greybeard taught me when I was 20-something.
Virtual context management. Imagine that. New since about 1970.
nitpick; PC from 1989 likely had more RAM than 640k, DOS by default just couldn't address more than 1MB directly (with 384k reserved for system leaving 640k for user) because of legacy architectural limitation of the original IBM PC from 1981 and the holy tenets of backwards compatibility.
since around dos4.0 in the backwards compatible "real mode" himem.sys and emm386 could give access to higher memory areas but the proper way was to switch to "protected mode" that could address rest of the system memory directly (16M for 24bit 286 and 4GB for 32bit 386) usually with an extender library like DOS/4G which were around in -89 but maybe not so widely spread yet.
@@snooks5607
You need to understand at $1,500 per mb of memory, there were not many computers with more than a mb of memory.
AGI would be impossible without a memory system, so I agree this is another step towards it. It's really cool.
🎉🎉
I have the same fellings
AGI doesn't need to be a memory machine. An agi good enough at comprehension and decision making is enough. Now if you talk about AGI with conscience, we are talking about elon musk level extinction
What possibilities regarding MemGPT could be accomplished with AI like ChatGPT?
@@kloszino one cares. 😄
Your channel has distinctly carved its niche in the AI TH-cam arena. Among the myriad of AI TH-camrs I'm subscribed to, your channel, particularly over the last six months, has excelled in quality, presentation, and professionalism. Your videos have become my go-to source, superseding others that now seem laden with filler content.
Your knack for diving straight into the core topic, elucidating not only the 'what' but the 'why,' is refreshing. The structured walk-throughs, practical guidance, and anticipatory glimpses into the future keep me engaged throughout. Your closing phrase, "And...I'll see you in the next one," has amusingly become a segment I look forward to; it encapsulates the essence of your engaging delivery.
Being a part of your channel feels like being immersed in a thriving community. The clear, concise factual delivery, balanced with simplicity, makes the content accessible for newcomers while remaining enriching. Despite the crowded space of AI discussions on TH-cam, your channel effortlessly ranks within my top 10.
Thank you for the enriching content and the community you've fostered.
This is such a kind comment, thank you so much!! Glad you’re learning from my videos :)
I agree, but this looks like it was written by GPT
Well deserved and well said! Amazing how you explain these matters for everyone. Any exec will be so pleased to have you guide them.
And btw it doesn't matter whether the words were perfected by AI, it's the thought - the gratitude that counts.
@@theChotkiyOne that is exactly what I was just saying. LMAO
Separating the conversation from an internal dialogue the system can have will prove very helpful: you can ask where the system has learned something to prevent hallucinations, have a space to run logical reasoning until confirmation, and now spout, “The ball has to be 10c and the bat $1.10… Wait, no.”
Yes, Self Awareness would great improve LLMs.. some humans need it to learn that skill too!
Yes please do another tutorial with MemGPT! This is huge!
Ok!
@@matthew_berman Thank you!!!👏🏻👍🏻
Please also do a tutorial on Salesforce ai.
Some of the new Mistral based local LLM's have 32k context and hence beat GPT-4 at certain tasks, it's amazing
Good to know!
which ones exactly?
@@avi7278i used TheBloke/MistralLite-7B-GGUF and it was good TheBloke/Mistral-7B-Phibrarian-32K-GGUF is another option i've tried it wasn't as good for what I was doing but it might be better on academic datasets
TheBloke/Mistral-7B-Phibrarian-32K-GGUF
TheBloke/Llama-2-7B-32K-Instruct-GGUF
I think the information is valuable and is explained up to the point where you can't understand more without a deep dive into AI. Good job!
Please let us know and do this again when they have open source models!
Love this video and thanks so much for the in-depth post! Accurately explains the theoretical science along with the practical implementation 🙏🏾
Man! I for one, am fully ready to welcome our AGI overlords!
I'm right there next to you my friend
@@Seriph001I'm right behind you bro.
Bow to the chosen One.
Same
this meme is so overdone i cringe everytime i see it
Matthew thank you so much for keeping us up to date. Your rock. Can't wait to play with this.
My favourite open source model is currently _Falcon 180B_ with the web search feature. I was impressed by M$'s _Bing Chat_ in Edge, but I mainly use Falcon instead now, since it seems just as good for grabbing information from the web, at least from my perspective. Although I don't fancy paying to run Falcon on a server, just to test it with MemGPT, despite my eagerness to try it out. It could be interesting if there was a _Falcon 180B_ API, similar to OpenAI's API, only much cheaper.
Thanks a million Matthew! Your videos are so clear and easy to follow 🙂Looking forward to your follow-up videos on MemGPT and open source models.
The issue with uneven attention in the context window (that phenomena when only beginning and end is memorized well, but everything else in the middle is foggy blurry) was partially solved by Mosaic with their MPT models
What innovation did they have?
This was such a good episode. The fact that the LLMs have memory like humans remembering the first and last, wow. I want this. Great episode!
D@MN you're good. You are on the forefront. Thanks for finding the material, breaking it down, and explaining how to implement..... It is much appreciated....
One of my first function calling experiments was having GPT-4 manage a couple of it's own context windows, and it really does a good job! Told it to use regex. Didn't go to this scale tho... Sounds really expensive!!!
I recently found this channel and really enjoy the videos. Great job Matt.
That video was awesome, very informative. I love how you ACTUALLY read the paper along the video
Once we have a robust way of handling memory, like MemGPT, we can simply fine tune the LLMs to utilize the system. Then we no longer need to use context window space for the system prompt to operate the memory. The LLM will just "naturally" do it.
you can simply fine-tune right now for that
@@gidmanone Yes, when we can fine tune GPT-4. But it will be better if OpenAI implement this directly themselves.
About lossy compression: it’s fascinating to me that lossy *text* compression can act as a normalizer, including replacing misspelled words or typos. I wonder if the output of recursive reflection is text or an embedding? As embeddings they can have more nuance than can be expressed in words (eg, “like a unicorn but even more mythical”) but that nuance could accumulate noise as well.
They should use both the uncompressed and compressed memories, the compressed memories offer a "preview" to the AI which it can then call more details if it needs to.
Wow. I just set up my own MemGPT bot on Discord and had a long conversation, Impressive, though still a bit artifically cheerful. Thanks for this!
This was the first thing I thought of when I learned about token limits. I even asked GPT to create a micro shorthand language to condense info. It didnt work in April but seems like were getting close!
Thanks a ton, Matthew! That's such great news. One step closer to a real LLM-OS. Can't wait till they implement opensource model support. I also would like to see how things like LangChain, HuggingFace and others can get integratied into solutions. Would highly appreciate another video about this topics from you. Thanks for your great work! 💪
always excited to see what you're posting next, really excited for more on AutoGen series
Thanks!
Well done! That was brilliant and the synergy between NN OSs and AutoGen seems like the way forward for sure.
I love that this came from young researchers and not from more senior ppl at a big company.
Love the videos. Definitely want to see more on MemGPT as I believe this is going to be the next huge move in development.
More to come!
Great and helpful walkthrough. Love your channel.
It looks like UC Berkley intends to release their own tuned version of Mistral-7B, Sounds like that project combined with their memory might have some amazing results for local independent research. Interesting they are on the Mistral 7B and not the Llama2 7b or Llamav3, This is institutional recognition of the value in this new open commercially viable solution and its plasticity to being fine tuned... I cant wait to see what comes of it definitely make a vid when its working with a local LLM, I suspect if Berkley is tuning Mistral for this use case it *could* be all local!
I've almost exclusively heard of Berkley in relation to jazz music education, so I can't help but wonder if they're *tuning* it to the jazz standards of "The Real Book"?! :D
wow thats amazing! thanks for sharing (you and the researchers as well) !! 🙏🙏🙏
Great, thanks a lot for a valuable update, Matthew!
things like textgen have qlora training built in that runs on fairly low spec hardware... have an option to train a lora from the long term memory on a schedule . start with a default lora trained on synthetic MemGPT input/output text pairs with the FULL Memgpt system header, then use short hand system messages during inference to give it 'reminders' on whatever aspect of the complete system protocol is the most important for that step.
gotta add this one to the memgpt playlist brotha thanks for the great vids love you long time
Thank you very much for sharing this information! I'm very interested in using MemGPT with open-source LLM models installed locally. If you come across any new developments in that space, I would highly appreciate hearing about it!
🎯 Key Takeaways for quick navigation:
00:00 🧠 AI currently lacks memory beyond training data and is limited by its context window.
00:29 📈 Progress has been made to increase context window size, but still limited (e.g., Chat GPT-4 has 32,000 tokens).
00:58 📚 Introducing MemGPT: A solution to expand AI's memory. The video reviews this research and the open-sourced code.
01:11 ✍️ Paper titled "M GPT: Towards LLMs as Operating Systems" has several authors from UC Berkeley.
01:51 🗣️ Limited context window issues arise especially in long-term chat and large document analysis.
02:20 💽 MGPT mimics computer OS memory management, with an "appearance" of large memory resources.
03:27 📊 Increasing context window in Transformers is not optimal due to computational and memory costs.
04:08 🔄 MGPT autonomously manages its memory through function calls, enhancing its ability.
04:52 🖥️ Diagram explanation: Inputs go through parsers, get processed in virtual contexts (main and external), and get outputted after further processing.
06:14 🖱️ MGPT allows AI to self-manage context, treating longer context as virtual memory and own context as physical memory.
06:40 📟 Main context (like RAM) has a size limit while external context (similar to a hard drive) is virtually unlimited.
07:08 📏 Various models have different token limits, impacting how many messages can be processed.
07:48 ⚠️ Actual usable context is often less than advertised due to system messages and other requirements.
09:00 🔄 Recursive summarization is another way to manage limited context, previously discussed in another video.
09:15 🧠 MemGPT stores its "memories" in a vector database, but it eventually compresses them through a process called "reflecting on memories" to manage space.
09:56 🔄 Recursive summarization can address overflowing context but is lossy, leading to gaps in the system's memory, much like video compression degradation.
10:38 📝 MemGPT splits context into: system instructions, conversational context (recent events), and working context (agent's working memory).
12:02 🎂 MemGPT can store key information from conversations in its working context, as shown by a birthday conversation example.
12:43 💽 External context acts as out-of-context storage (like a hard drive), separate from the main context but can be accessed through function calls.
13:25 🔍 There are two types of external contexts: recall storage (history of events) and archival storage (general data store for overflow).
14:09 🧩 MemGPT manages its memory using self-directed memory edits and retrievals, executed via function calls and based on detailed memory hierarchy instructions.
15:32 🔄 MemGPT can correct its memory when false information is detected, updating its stored context.
16:14 🤖 The effectiveness of MemGPT as a conversational agent is evaluated based on its consistency (alignment with prior statements) and engagement (personalizing responses).
17:10 🎵 Through a function call, MemGPT can delve into its past memory to recall previous conversations, like discussing a music artist.
17:52 🕰️ Deep Memory Retrieval (DMR) enables the agent to answer questions that refer back to very specific details from past conversations.
18:05 📊 The accuracy of MGPT's responses is better than GPT 3.5 or GPT 4 alone.
18:19 🍪 Personalized conversation openers (like referencing a user's cookie preference) increase user engagement.
19:01 ☕ Examples illustrate how MGPT uses context and recall differently to engage with users.
20:12 📜 Many documents exceed the token limits of current models, creating challenges in document analysis.
21:06 🧠 Large language models exhibit a bias in recalling information towards the beginning or end of their context, mirroring human memory patterns.
22:44 📈 Charts indicate that MGPT maintains consistent accuracy regardless of the number of documents or nested information, unlike GPT 3.5 and 4.
23:12 ⚖️ A trade-off with MGPT is that some token budget is used for system instructions.
23:41 🤖 Discussion about LLMS as agents and their emergent behaviors in multi-agent environments.
24:21 💻 Tutorial on how to activate and use MGPT, starting with code setup.
27:35 📁 MGPT's document retrieval feature allows users to chat with their documents; using wildcards can fetch multiple text files.
28:15 💵 Embedding files come with a computational cost; example given shows 3 documents for 12 cents.
28:44 🔄 MGPT's persona is customizable, allowing users to tailor how the model interacts with information, like referencing archival memory.
29:38 🔍 MGPT can retrieve specific data from documents, such as annual revenues of companies.
30:06 🌐 Introduction to MGPT emphasized its rapid evolution and potential for open-source models in the future.
30:33 🎙️ Interview with MGPT authors Charles and Vivian discussing inspiration and plans for the project.
30:46 🧠 MGPT addresses the memory limitations of current language models by actively saving crucial data into a permanent memory store.
Wow, nice. Thank you!
What did you use for this
What was used for this?
Sounds pretty complicated, regardless, things like ChatGPT could use this to improve the performance of the ai they use, right?
@@eraldcala9125 I think its HARPA ai. I'm seeing tons of videos spammed with this.... already over it lol
Thanks for the great content, Ive learned a lot from your Ai curation.
Due to the pervasive sound effects I use subtitles on your channel. Keep up the great work
You don’t like the sound effects you’re saying? I’ll reduce them in future videos if people don’t like them.
Please do more! Great content lad! Learning and using something much!
Please keep following this with opensource! Great for analysis
+1
You got it!
wow, very exciting video. It'll be awesome to get an updated video once they release a way of running open source models with this, even if they're not entirely capable yet. I'd wager that if mistral can successfully scale their models up to ~34B they'll probably be able to be fine tuned into a very competent function calling model!
glad to see the progress, been working on a solution using roughly the same approach for months now.
I loved your simple explanation of MemGPT; thank you!
As always, highest quality content, many kind thanks!
Much appreciated!
Definitely want a tutorial with open source models and thank you so much for doing all of this I've learned a ton from you
I got to watch your video for 2nd time. your video is a huge boost for my startup idea. thanks a lot
9:59 as an amateur author use recursive summarization to communicate my ideas to LLMs all the time so i can't wait to see if this will be better
Thanks for doing a video about MemGPT I can't wait to see what can be done with all this.
I started writing something like this a few days ago. Realized it's a monumental task. So glad someone beat me to it. 😁
Very cool and valuable information! Thank you! I am looking forward to them adding open source LLms!
Epic vid thanks for the insight.
Seems like a much better solve than the attention sink paper.
Excited to see this in the wild.
The very basic install instructions seemed out of place.
Definitely would love follow-up on the future open source model usage!
You help me so much my dude. Thank you 🙏
Thanks 🙏 for all of the highlights. I have trouble focusing on my Sales job because I am trying so hard to learn bot 🤖 skills and reconsidering juggling the Newsletter information
A lot of things discussed here are very similar to how human memory works. We can hold a limited amount of data in our short attention window. Our brain can store a lot of long-term info but buries it deeper and out-of-reach if it thinks it's not currently relevant. It also seems to compress memories by letting us remember easily the most important details of an event but burying the rest deeper. And we have all kinds of techniques or "functions" to jog our memory to bring back old data we know we have somewhere, store more short-term stuff efficiently when cramming for an exam, and so forth.
The more i think about multi model systems the more i see similarities with how our brains divy up task priorities. Its both freaking me out and exciting me to be honest
Super exciting project! I totally agree that document chat is a key app. Please do another video when they support an open source LLM.
This was such a great breakdown, even I understood. Wow.
Great video, Exciting stuff. Memory access is a huge limiting factor especially within coding projects so I can see this really moving things forward. It seems like this would give you the benefit of a huge LLM like ChatGPT that then can be "trained" or augmented to your specific use and data set while still retaining all of its full training data.
Absolutely!
Exactly.
I literally was just about to ask about a memGPT your a freak…. You earned that sub
I joined the discord the other day, it's pretty awesome in there too
you're*
@@adelinrapcore why does that always happen. Not lying I always mess it up and someone corrects me.
Haha thank you. I’m reading your mind :)
@@93cuttywelcome!
Amazing bit of work by this team.
A thought... While I'm not going to be silly and say open source models are currently as good all around as openAI's offerings they're close in so many regards and are catching up fast in most areas BUT where openAI really has a lead is in things like Function Calling. I'm really, really hoping we see some innovation in this area in the open model space soon. Thankfully I do think that innovation is coming and openAI's closed ecosystem is going to be under more and more pressure. imo open models will eventually win. Thanks for the coverage.
Cant wait for the open source model. Thanks so much for presenting this information.
Wow, what a brilliant concept!
I like all your stuff but this video blew me away. Even though you include a transcript, I had to keep stopping it to make notes. Well done. The only place I stumbled was the many different but slightly similar constructs. I'm still working to wrap my mind around that. For instance, you had a reference to system instruction, conversational context and working context. Later you refer to recall storage and archival storage which I assume are the same as main context and external context. Later you have working context and recall. I'm sure it's just me, but I'm trying to sort that out in my own mental model. But again, well done!
Thank you!
Wow, very inspiring video. I’m working in my PhD and would be a must for it
Thank you, for this video it was awesome. Yes, I would love to see you implement this with open source models.
Awesome information. Keep it up. Cheers Terence
Matthew, this is sn amazing video. Remember the format, it's really good. Of course there must be a paper which is as good as this. But your way of explaining it is really well done
Much appreciated. I don’t think some people liked the glitch transition or the sound effects. But can’t please everyone!
@@matthew_berman i have not noticed any glitch transition. But maybe my brain is running too low Herz to notice them 😉
HUGE yes from me, please cover it again when it can use llama or the webUI api :-) suuuper cool project!
Thinking about what you’re thinking… subconscious analysis of thoughts based on beliefs… Seems like the CPU/Ram/HD analogy could be better replaced by how our minds actually process information. Love this stuff! Thanks for breaking it down
An awesome piece. Thank you.
You are my goto Mac Ai guy! Keep it up!
Can I also be your PC AI guy too? :)
Quality content as usual 😊
Glad you think so!
Yes please do another video if they incorporate open source. This looks awesome, but I don't want to use an Open AI key. Another great video BTW :) You are my go to AI TH-camr.
Thanks :)
Matt, MemGPT is a further abstraction of a context window from an application level (Chat) to an OS level. OS level Context window could be in-memory (similar to cache) and on-disk. Cache size can be controlled. Anything that needs to be in context is cached, and the rest is flushed to disk. This process is operated as a function call. A long context window is almost essential to maintain a holistic context.
The context is saved on your personal or work environment/device according to your usage. Context can leveraged for any required automation or building work efficiencies.
🙂How different is this from a traditional Windows-Registry with Name-Value pairs!! 🙂
this looks super neat. i have been having fun though with creativity of lllms by specifically pushing them past the context and token limits. that's when you seem to get some good creativity. though that's kinda the opposite of what you are going for here haha. great video buddy
Thanks for digging into this and explaining it so well. I have looked at this project a couple of times and didn't quite "get" it. Keep up the good work!
Glad it was helpful!
Imagine using MemGPT + Agents for large or multi-document analysis, where each Agent takes on a document (or section) and discusses with the other Agents their documents to answer user questions?
Would love to see more on this technique and new models for memgpt.
I wonder, if that could be used for example for applications to companies (for work): essentially, create a CV based on the profile of the company. for this, it would need to know a lot about the user and know which information is relevant for a job application and which one is not. maybe the system can also need something like memory-files? essentially store important facts about a person together so they can be queried together.
I see massive potential in these context-based usecases.
Amazing. Thanks very much Matt
Surprised this hasn't happened already! Huge step forward.
Very promising stuff. Liked, subscribed, following. Keep it coming :-)
Great news, great project! Thank you
Long term and short term memory. Imagine how cool its gonna be when they realize they can on schedule or on event splice in NEW info to the prompts... Cant wait to start my AI project at work...
Thanks for sharing the helpful content
Amazing news! And yes, ofcourse we want to see it working with open source models..
Gotta pont out, that's pretty much an analog of how humans process memory as well. also when it says :working-context-append Key Personality
high-speed, adrenaline-rush activities and intense gaming sessions in CSGO) a slightly more charitable reading of it would take the CSGO part as just a descriptor of things that are fast paced and adrenal pumpin activates like formula one racing.
While I acknowledge there are other similar concepts floating around, I think the MemGPT ease of use, documentation and open sourcing is a great resource. Maybe I need to read the entire paper, but I am curious whether the "working_context_apend" feature is self guided, or a schema specified by the programmers, eg. "Key Personality Trait" - did the LLM decide this was something to remember, or that was pre-defined ?
At 14:00, archival actually means that GPT is formatting the data in a more summarized way, for storage in a simpler way
Definitely interested when this goes open source. Thx for this!
holy zeropoint/singularity, it's as if a solution appears as soon as i find a crippling limitation in ai-human interface. this one is a gem
This one is my top 1 thing about AI I am really excited about. Cant wait for local LLM integration.
Very nice thanks!!
Very ingenious workaround. Although I thought vector stores were the answer? Would definitely like to see video on open source
R astores
Awesome content, thank you
Yes, please!
Thank You for this video.
And please , as soon as they release it with Open source LLMs, love your videos.
Hey that is what I was working on for my Dungeon and Dragons DM AI :D
This is one of your best videos
So exciting!