AI Chat Memory Boost with Hybrid Storage and Query in SillyTavern
ฝัง
- เผยแพร่เมื่อ 28 ธ.ค. 2024
- In this video, I will share how I use hybrid mode for chat memory improvement (using both summary and raw logs)
The summarization plus hybrid storage approach: Condensing chats provides a concise overview for quick context retrieval, while keeping original transcripts preserves granular details that could prove invaluable in certain scenarios.
AI Long Term Memory Workflow Locally (no cloud!)
Explanation of why Summarization is important for context
Share that Summarization has flaws: raw chat logs can remediate
Using both Attachment and Databank in SillyTavern allows for control of chunks ratio between details and context
Add padding in detailed chunks to improve details raw logs context
Bonus: Stable Diffusion image at the end
Tools used:
--
Oogabooga: download and installation instructions
github.com/oob...
SillyTavern: download and installation instructions
github.com/Sil...
Finally someone explains about SillyTavern bank data via video, thanks a lot.
I'm happy to hear that it helped you. If you ever tried this out, let me know how it goes and if you find something interesting, don't hesitate to share :)
Holy shit dude, thank you so much for the sharing. My biggest struggle with the summarization has been the lost of details. I have spent so much time tweaking the summs and it not so fun. But I think you found the solution for it. Please keep these videos coming!!
Hi @SerahMiller-Q7f, I'm super happy to hear that my video share helps you :) I'm always finding ways to better the output of the memory retention and extration, so even if I know this is not perfect, I do think it improves it. I've been using this technique since I've found it and been liking the results alot. Hope to be able to share more of my experience with you and the community.
@@AIArtistryAtelier have you thought about making a discord community? a place where we can share ideas and what not
@@SarahMiller-q7f I think it's something I would be interested in when there is more interest towards what I'm sharing and doing. Let's see how this channel grows and based on that, I could look forward to planning for something like discord. Thanks for the suggestion! In the meantime, don't hesitate to send me DM if needed :)
I'm not having much luck with this method and the AI's detail recall ability. I only have 8k context and I just might have too much data to go though. I have 12 long chat logs that take about 10 minutes to vectorize. I had issues with even pulling data from the Summarized Memory text file in chat. I will keep messing with it and see.
I got a question. At 7:35 what was the "summe" button and how does it function?
Hi @nuklear22, happy to see your interest. Another user got similar question
that yours. Here was my answer:
For the summarize, you can just simply paste directly in the chat the following:
Pause your roleplay. {{user}} is interested in developing a long-term memory system that will help him keep track of events and states. The goal is to write a detailed report of about five hundred words of the events that have happened today so far in the chat history, focusing on major events and each character's associated emotions. The report should be concise and factual, relying solely on information from the chat history and avoiding any speculation or guesswork. Do not use words that is not present in the chat history. The report, restricted only from chat information, should include a list of main characters, events and each character's associated emotions, all of which must be directly related to {{user}} or have potential for development in further stories. Do not give extra details or description if not specified in the chat. Describe seperately for each character, how they are feeling in great details during event list descriptions. Unique words in this chat must be used throughout the whole summary and not just a list. Do no add events or objects in the summary if not infered from the chat history. Limit the events and each character's associated emotions list to ten only. At the end, summarize in details with at least one hundred words what was the most meaningful topic that was discussed today. This summary should have a bullet point list format.
If you want that same button like I my setup (which basically does exactly that, which is paste that big command prompt in the chat, but just more easy to access), you can add the silly tavern extension "Quick Reply", then paste that command in there.
Details about the Quick Replies extention of ST here: docs.sillytavern.app/usage/st-script/
Search for "Quick Replies"
I hope this works for you. Good luck!
@@AIArtistryAtelier It does thank you very much.
Thanks for this but im gonna need your summarize settings or template i guess? because my default summarize does not look like that. please of course.
Hi @kul-tigin, happy to see your interest.
For the summarize, you can just simply paste directly in the chat the following:
Pause your roleplay. {{user}} is interested in developing a long-term memory system that will help him keep track of events and states. The goal is to write a detailed report of about five hundred words of the events that have happened today so far in the chat history, focusing on major events and each character's associated emotions. The report should be concise and factual, relying solely on information from the chat history and avoiding any speculation or guesswork. Do not use words that is not present in the chat history. The report, restricted only from chat information, should include a list of main characters, events and each character's associated emotions, all of which must be directly related to {{user}} or have potential for development in further stories. Do not give extra details or description if not specified in the chat. Describe seperately for each character, how they are feeling in great details during event list descriptions. Unique words in this chat must be used throughout the whole summary and not just a list. Do no add events or objects in the summary if not infered from the chat history. Limit the events and each character's associated emotions list to ten only. At the end, summarize in details with at least one hundred words what was the most meaningful topic that was discussed today. This summary should have a bullet point list format.
If you want that same button like I my setup (which basically does exactly that, which is paste that big command prompt in the chat, but just more easy to access), you can add the silly tavern extension "Quick Reply", then paste that command in there.
Details about the Quick Replies extention of ST here: docs.sillytavern.app/usage/st-script/
Search for "Quick Replies"
@@AIArtistryAtelier ah yes this is nice thanks
I get a summary but it's very concise and doesn't have bullet points etc. you must have another setting that controls this. is it a template or something like that?
Hi @user-hi3ke6qh7q, you are correct, having something come out similar, there's quite a few parameters and elements beyond just the summary prompt. Here are a few pointers that hopefully will help:
1. Not all AI model are good for summaries. Maybe try another AI LLM model? I usually run my quick "AI LLM Model Scoring" to give me an idea how good the AI model is before doing more with it knowing the expectations. th-cam.com/video/1w0uVeUnLxs/w-d-xo.html. This is the one I used in this video based on my use case and rig : LoneStriker_Phi-3-medium-128k-instruct-8.0bpw-h8-exl2
2. Play around with your text gen AI Response Configuration Settings? (top left of the silly tavern menu). Instead of Default, maybe try Deterministic?
3. As usual, it's ai gen, so regen it if you don't get what you like. For the summary, it usually takes me 2-3 tries before getting something I like.
I hope that helps!
@@AIArtistryAtelier Appreciate you taking the time to reply. This is helpful. I'll give it a shot. Thanks so much.