Chatbots and Long-Term Memory Explained

Chatbots and long-term memory explained with ‪@jamesbriggs‬. This video is a simple introduction to how chatbots like OpenAI's ChatGPT, Anthropic's Claude, and Google's LaMDA are able to use long-term memory to avoid hallucinations, keep their knowledge up to date, and even integrate with internal or domain-specific data.
We use the Pinecone vector database as the long-term memory for out chatbot systems.

[@sarkedev]

But you didn't actually explain it, you just talked about the problem.
How does the LLM interface with the vector database?

[@nucularsr]

can't wait to start working with this!

[@mintakan003]

The other thing that external storage allows, is provenance, citation, and ease of "fact checking".
I've been testing (and help training) Bard. Hopefully, I'm also giving feedback to the Google developers, on what features I would like. Bard has a "Google it" button. But it seems to give related links (probably through another semantic similarity mechanism). What I would like, are not only the links, but highlighting on the relevant sections in the page, that justifies the LLM's response. This makes it easier for the user to fact check, than having to use the browser "Find" option , or just manually scanning down the page, trying to see where the response came from. It would be a "killer app", if they can pull this off.
Also, I realize not everything is in the external source. Some maybe pre-trained in the LLM. Knowing which is which, is important. (Probably specific facts, proper names, and places, should be in the document.)
I can imagine case law, where the verbatim text might be important. Besides the LLM's summary, or paraphrase, one needs the literal text in the vector database, so can go back to the original document. One needs both, as well as a way to trace back to the source.

[@arrezki1]

Hello,
Thank you for the Interesting Video. I am a final year student and thinking of making a chatbot that can read around 100 books on a small library. Can you please advise me what resources i need for that, and how long it will take raphly to finish this project if i work hard on it. My time limit is 6 weeks of training.

[@jidun9478]

I mean everyone is aware of the issues of remote hosting of data and for private or even many small to midsize businesses need their own private and secure AI data.

[@jidun9478]

Personally, I think memory is a huge problem. I also think we need to really divide ourselves into two camps... One mega large as looking toward gigantic databases dealing with thousands of people at a time and then there should be an equal amount of development toward personal and or private AI's whose memory will not be near as demanding i.e. numpy, scikit, Django ORM. What do you think as that could set a vector db for dialog, job memory and so on .... What do you think?

[@troylowry4239]

You can run local vector databases. The pgvector extension for postgres has served me well so far this week for the embeddings produced by text-divinci-002

[@user-zk5st1kk8s]

yes but the number of token they need to call a data sometime goes to 500K tokens in 10 minutes