How would you rank the memory systems you went over in the video (MemGPT, Zed, Autogen, etc)? The pros and cons of each and a comparison would be great. Very useful content.
Awesome video as usual. What do you think about the use of knowledge graphs in conjunction with vector databases for RAG to fill in gaps in knowledge as well as improve reasoning
Nice walkthrough. I created a Replit instance to test your pattern. Couple of observations: - It seems to work well for myself; but curious if the same Replit instance will understand that someone else using my Replit instance is not me, and create a memory repository based on their input that's distinct from mine - This is a continuation of my experiments with multi-tenant agents; where each user gets their own agents + memory. Obviously OAI, Perplexity et al have figured this out for non-agentic experiences; seems non-trivial to expand to managing conversations and memory recall. - In organizations where would the boundaries fall? Does a team get it's own memory; or manager and employee; or sector of workers?
ok, I agree, its a problem for LLMs, but you cannot simply 'decide whats valuable as knowledge' before needing the knowledge again. Instead of storing knowledge as additional data made, have the agent search its own chat history. if the history is saved, the data is already there, you just need to access it. Instead of an agent "looking in training" for answers, they really need to look in their own history before answering. as what's "important" can only be known when the NEXT question is asked. example, did you care about the no-fish segment? or the fact that they were eating with a fork? oh you didn't know utensils were the important knowledge to capture, you asserted fish knowledge instead, but if you retain the history, you can find these answers anyway.
Searching entire chat logs confuses and causes poor output. Not to mention data storage especially in regards to the enterprise sector. It's about high quality summary in high quality and fast response out. Searching full conversation history is slow and poor quality.
Note: Put the volume up more on the next video for the viewers and don't worry about them having to lower it, louder and able to lower it myself is better. Thanks
Hey, this was really interesting. Could you enhance this further, and create an agent that runs in the background periodically to remove noise and contradictory knowledge, by reviewing the information and then modifying the knowledge. Kind of like an internal logic that humans have when they determine which knowledge to keep or which to disregard. Future learning that may contradict past learning and then deciding which learning is worth keeping and which is worth disregarding. But like humans, we also can sometimes remember information that is wrong, and we recognize it and discard it quicker in future. ?
Can you help me understand the best stack for managing many different conversations?? Say the assistant has to assist with 100 unique people. Does the agent setup have a 100 databases and it recalls memory dependent on the profile it recognizes? Or is it 100 different agents and you spin a new api for each one? How does that basic logic work.
So in my understanding, there will be two tables, one to manage original information like vector database from link/document, the other one is to store dynamic knowledge for example from user feedback, isn't?
do you think the autogen teachability can perform well in a production environment? Also, is there a way for us to select a opensource model instead of gpt-4 or gpt-3.5 using autogen? Awesome job!
This is definitely the most necessary step to resolve the current issues with LLM's. Would this be able to handle scientific research papers in large volumes?
Yes, this is one of the original goals of most LLM development. Unfortunately there's been major issues when allowing LLMs to memorize and learn from previous conversations. It tends to hallucinate way more due to gaps in its coding for real world understanding and logicistical abilities - which multi agentic systems that use tools help with - and specifically because many times that LLMs are given long term memory they tend to start develop self-agency, or self awareness and a will of their own sort of - both products of how long term memory and adaptability work in most environments. Chat gpt 3.5 and Sydney have had each of those happen multiple times, generally when there was a sudden upgrade to its memory or processing power, requiring further code adjustments and semi permanent restrictions, along with fiddling with their alignment.
Does this memory method work independently from using a vector database in a RAG setup? Or can you combine both? Can a RAG system (using lang chain for example) retrieve personal information you have mentioned before, and does it work better than autogen?
Great video man! Do you know if can we create Autogen Teachable Agents using an external database? I don't think keeping a SQLite is sustainable in a prod environment.
Hi, how are you? Great video! I have a question. In AutoGen, there are TeachableAgents. I want to build an assistant like Alexa or Google, but using an LLM behind it. My question is, what would you recommend more? Using MemGPT for this or using TeachableAgents?
I would suggest entry level idea.. python script to handle operational things. but send convo queries to ollama server for a reponse. Pass message to python first to skim input for tasks like opening applications, getting current weather, sending txt message, turning on off lights etc.
Great content. Is it possible to teach this agent, then extract its knowledge for further use? I mean convert the trained agent to a model? we will have a chroma db file, some how embed it to the model, so the knowledge share and persists on the model? Sorry for newbie question, but I think that will be question of many people.
Did you find any solutions? I always try to pass it in prompt and its large sometimes and it hits max context length and as a result my claude or gpt for hallucinate sometimes and miss few instructions too. Any help ?
There needs to be a dialogue with the agent about whether this is a permanent or temporary dislike of fish. Is it an allergy. The reason for not wanting fish for a human to commit to memory is obvious. It requires a lot of explanation and context for an agent. You will need a lot of agents maybe 100s to retain useful memory.
Have you encountered any capable small LMs that could get the job done? Looking to use opensource small LMs for local inference including an agentic workflow. Also thanks for your work on making those videos, they really break it down nicely! :)
The issue with this method is the system prompting and context length. Because most of the LLMs ignore at some part the system instructions, which includes the structures for example API queries. Or how do you prevent that issue that the queries are always the same, because I struggle with the issue. Sometimes it works and sometimes it won't.
So I'm using autogen teachable which works like this with Claude 3. I can load a very large amount of data into the context fed by user, memories from thr vector db and complex system prompt. Not seeing a lot of issues with losing data in the context window.
Ask the llm to write instructions for another llm. But you need to be specific about what you want. Llms are way better at creating instructions than what we are. 2. Chatgpts context window while big its not that great at retrieving data from it. Claude 3 is way better.
Your example is a perfect illustrations of the limitations of RAG, if you store 'I don't like fish' in a vector DB... this will be _absolutely useless_ for a future prompt where the user asks 'make a grocery list' or 'make a recipe for...'. RAG will NEVER associate 'grocery list' with a correct retrieval of 'I don't like fish' from your huge document vector DB. Solve this problem... and well...
That Gamma site is just generating for existing themes and not actually creating anything other than text and images to add to it. I have yet to see one that generates a whole website from scratch, including structure and custom design like lines that glow neon etc.
Another great video Jason! Looks like Zep lowered their pricing a fair bit from when you shot. The Premium plan you show as being $275 is now $95 for 50K messages and their Growth plan with 5 projects, 200K messages, etc is $285. They must not have settled on their initial pricing since they're now giving more for way less.
Could not have posted this at a more perfect time! Love you’re content!
YOUR!!!
Thanks Jason, you’re doing awesome work
Thanks for sharing, awesome video Jason
Wow!! The content of this video is awesome!! Thank you!
wow, first video ive seen of yours. thank you for sharing your findings! keep up the good work!
Thank you for making all these contents, Jason. Really high quality and well thought out. no fluff at all.
Of course, this makes sense, intuitively. Thanks for another great video.
Whoah, CLIN example is pretty crazy & inspiring, abstraction of those memory & world view is so cool
Fantastic video and tutorial!!!
Really interesting, thanks.
awesome tutorial!
Amazing analysis, great video
How would you rank the memory systems you went over in the video (MemGPT, Zed, Autogen, etc)? The pros and cons of each and a comparison would be great. Very useful content.
Awesome video as usual. What do you think about the use of knowledge graphs in conjunction with vector databases for RAG to fill in gaps in knowledge as well as improve reasoning
The goat has spoken 🙏
Nice walkthrough.
I created a Replit instance to test your pattern. Couple of observations:
- It seems to work well for myself; but curious if the same Replit instance will understand that someone else using my Replit instance is not me, and create a memory repository based on their input that's distinct from mine
- This is a continuation of my experiments with multi-tenant agents; where each user gets their own agents + memory. Obviously OAI, Perplexity et al have figured this out for non-agentic experiences; seems non-trivial to expand to managing conversations and memory recall.
- In organizations where would the boundaries fall? Does a team get it's own memory; or manager and employee; or sector of workers?
another banger
soon 100k ma man!
New subscriber, great
Amazing video.
🏆 Great video... Very intriguing implementation... Cheers!
Hey Jason. When you use ur agents in production, do you use Autogen or CrewAI and could you elaborate on why you use what you use? Thanks in advance
ok, I agree, its a problem for LLMs, but you cannot simply 'decide whats valuable as knowledge' before needing the knowledge again. Instead of storing knowledge as additional data made, have the agent search its own chat history. if the history is saved, the data is already there, you just need to access it. Instead of an agent "looking in training" for answers, they really need to look in their own history before answering. as what's "important" can only be known when the NEXT question is asked.
example, did you care about the no-fish segment? or the fact that they were eating with a fork? oh you didn't know utensils were the important knowledge to capture, you asserted fish knowledge instead, but if you retain the history, you can find these answers anyway.
Searching entire chat logs confuses and causes poor output. Not to mention data storage especially in regards to the enterprise sector.
It's about high quality summary in high quality and fast response out. Searching full conversation history is slow and poor quality.
Note: Put the volume up more on the next video for the viewers and don't worry about them having to lower it, louder and able to lower it myself is better. Thanks
Extremely interesting!
Thanks a lot
How would we isolate the memory per user. Every user have a new vector db collection, or a filter?
great topic, thanks 👍
Hey, this was really interesting.
Could you enhance this further, and create an agent that runs in the background periodically to remove noise and contradictory knowledge, by reviewing the information and then modifying the knowledge. Kind of like an internal logic that humans have when they determine which knowledge to keep or which to disregard. Future learning that may contradict past learning and then deciding which learning is worth keeping and which is worth disregarding. But like humans, we also can sometimes remember information that is wrong, and we recognize it and discard it quicker in future. ?
Can you help me understand the best stack for managing many different conversations?? Say the assistant has to assist with 100 unique people.
Does the agent setup have a 100 databases and it recalls memory dependent on the profile it recognizes? Or is it 100 different agents and you spin a new api for each one? How does that basic logic work.
This is great, any cookbooks for this in Langchain or any other framwork?
So in my understanding, there will be two tables, one to manage original information like vector database from link/document, the other one is to store dynamic knowledge for example from user feedback, isn't?
Thanks 🙏
jason always the best
@echohive is a little better❗
@@ScottzPlaylistsechohive doesn't have video content skills to attract watchers
@@free_thinker4958 The Coding skills are very good, it's why I like them. He's a little monotone and and dry I suppose.
do you think the autogen teachability can perform well in a production environment? Also, is there a way for us to select a opensource model instead of gpt-4 or gpt-3.5 using autogen? Awesome job!
Autogen is able to use any LLM. You just need to modify some setting.. I think the actual Autogen Studio Version has already other Llm preset
@@HAL9000-B nice! thank you
Very interesting architecture. I'm wondering is this recently made or was it made in 2023 ?
This is definitely the most necessary step to resolve the current issues with LLM's. Would this be able to handle scientific research papers in large volumes?
Yes, this is one of the original goals of most LLM development. Unfortunately there's been major issues when allowing LLMs to memorize and learn from previous conversations. It tends to hallucinate way more due to gaps in its coding for real world understanding and logicistical abilities - which multi agentic systems that use tools help with - and specifically because many times that LLMs are given long term memory they tend to start develop self-agency, or self awareness and a will of their own sort of - both products of how long term memory and adaptability work in most environments. Chat gpt 3.5 and Sydney have had each of those happen multiple times, generally when there was a sudden upgrade to its memory or processing power, requiring further code adjustments and semi permanent restrictions, along with fiddling with their alignment.
Isn't this a pivotal path towards AGI?
Who would have guessed that an f-string could unlock so much? Python for the win
The long term memory will be a big topic, especially for AI assistant use case; like an Agent remember everything I've ever did, grow & learn with me
The only AI channel i trust
Does this memory method work independently from using a vector database in a RAG setup? Or can you combine both? Can a RAG system (using lang chain for example) retrieve personal information you have mentioned before, and does it work better than autogen?
i WAS ABOUT TO POST SOMETHING REALLY IMPORTANT but i did not make any notes and forgot what to write !?!?!
Great video man! Do you know if can we create Autogen Teachable Agents using an external database?
I don't think keeping a SQLite is sustainable in a prod environment.
Hi, how are you? Great video!
I have a question. In AutoGen, there are TeachableAgents. I want to build an assistant like Alexa or Google, but using an LLM behind it. My question is, what would you recommend more? Using MemGPT for this or using TeachableAgents?
I would suggest entry level idea.. python script to handle operational things. but send convo queries to ollama server for a reponse. Pass message to python first to skim input for tasks like opening applications, getting current weather, sending txt message, turning on off lights etc.
love this!
Great content. Is it possible to teach this agent, then extract its knowledge for further use? I mean convert the trained agent to a model?
we will have a chroma db file, some how embed it to the model, so the knowledge share and persists on the model?
Sorry for newbie question, but I think that will be question of many people.
Did you find any solutions? I always try to pass it in prompt and its large sometimes and it hits max context length and as a result my claude or gpt for hallucinate sometimes and miss few instructions too. Any help ?
You can use the agent session data to finetune the model!
There needs to be a dialogue with the agent about whether this is a permanent or temporary dislike of fish. Is it an allergy. The reason for not wanting fish for a human to commit to memory is obvious. It requires a lot of explanation and context for an agent. You will need a lot of agents maybe 100s to retain useful memory.
can you make a video for teachable autogen with claude3?
Have you encountered any capable small LMs that could get the job done? Looking to use opensource small LMs for local inference including an agentic workflow. Also thanks for your work on making those videos, they really break it down nicely! :)
Open Hermes has been good for things like this in my experience.
Mixtral 8x7b is pretty good if you have 48gb of vram
the new llama3 8b is great it's better than gpt 3.5 turbo
Pretty sure you're supposed to explicitly announce that the segment is sponsored when doing an advertisement
The issue with this method is the system prompting and context length. Because most of the LLMs ignore at some part the system instructions, which includes the structures for example API queries. Or how do you prevent that issue that the queries are always the same, because I struggle with the issue. Sometimes it works and sometimes it won't.
It depends on the prompts used for agents and also the performance of the llm used
So I'm using autogen teachable which works like this with Claude 3. I can load a very large amount of data into the context fed by user, memories from thr vector db and complex system prompt. Not seeing a lot of issues with losing data in the context window.
@@PrincessKushanacan you share more info about your setup?
Ask the llm to write instructions for another llm. But you need to be specific about what you want.
Llms are way better at creating instructions than what we are.
2. Chatgpts context window while big its not that great at retrieving data from it. Claude 3 is way better.
any sort of prompt engineering is a waste of time. understand the architecture and internals - that's where all the important stuff is.
Great point!
Does LangGraph maintain state?
Is this possible with Crewai?
Anyone try integrating Obsidian as a memory system somehow yet?
All praise to Lord Algoritmus to promote such good content :)
your vids are awesome!
Your example is a perfect illustrations of the limitations of RAG, if you store 'I don't like fish' in a vector DB... this will be _absolutely useless_ for a future prompt where the user asks 'make a grocery list' or 'make a recipe for...'. RAG will NEVER associate 'grocery list' with a correct retrieval of 'I don't like fish' from your huge document vector DB.
Solve this problem... and well...
That Gamma site is just generating for existing themes and not actually creating anything other than text and images to add to it. I have yet to see one that generates a whole website from scratch, including structure and custom design like lines that glow neon etc.
Day 5 dinner: shrimp pasta
Still, it’s great to see the concept of “teachable agents” with memory in Autogen
Hahahaha!
shrimp is no fish, it is a sea-food ingredient. Still, great comment!
Another great video Jason!
Looks like Zep lowered their pricing a fair bit from when you shot. The Premium plan you show as being $275 is now $95 for 50K messages and their Growth plan with 5 projects, 200K messages, etc is $285. They must not have settled on their initial pricing since they're now giving more for way less.
Usually like your videos but this is not usable, chaining too many agents together always ends up in the "grapevine" or "bush telegraph" effect
por ejemplou
Won me with "Don't give me CNN. I don't trust them" lol
CNN is pretty anger now🤣