- 48
- 931 769
Tim Carambat
United States
เข้าร่วมเมื่อ 10 มี.ค. 2014
Software Engineer, Founder and CEO of Mintplex Labs. Creator of AnythingLLM and Senate/House Stock Watcher.
Create fine-tuned models with NO-CODE for Ollama & LMStudio!
👋 Hey everyone,
Back with a new video highlighting a super cool feature that we just added into AnythingLLM where you can create a full fine-tuned model from your chats and documents and run that **locally** with Ollama and LMStudio - no privacy issues or vendor lock-in. At AnythingLLM we believe models trained from your data are your models - we don't gatekeep this technology or restrict you to a single provider.
**This is not a local service** If you want to fine-tune a model locally you can export your data directly from AnythingLLM and do this yourself. A locally running no-code trainer video will be coming soon - subscribe to get notified!
Downloading AnythingLLM: anythingllm.com/download
Star on Github: github.com/Mintplex-Labs/anything-llm
Send me an email: team@mintplexlabs.com
Chapters
0:00 Introduction to no-code fine-tuning with AnythingLLM
0:25 Disclaimers and caveats
0:47 How to download AnythingLLM
1:27 no-code Fine-tuning overview
2:00 Who this video is for
2:37 Gathering good training data with RAG + AnythingLLM
3:44 Reviewing the dataset we will train on
5:28 Fine-tuning vs RAG for the everyday person
7:00 Ordering a fine-tune
8:04 Privacy & Data Handling
8:44 Selecting a base model
9:18 Uploading our dataset
10:32 What happens once you start
10:54 What you get once tuning is done
11:49 Loading our custom fine-tune into Ollama
15:38 Testing our new fine-tune out!
16:43 How to uninstall a custom fine-tune from Ollama
17:30 Loading our custom fine-tune into LMStudio
20:57 Conclusion and Thanks!
Back with a new video highlighting a super cool feature that we just added into AnythingLLM where you can create a full fine-tuned model from your chats and documents and run that **locally** with Ollama and LMStudio - no privacy issues or vendor lock-in. At AnythingLLM we believe models trained from your data are your models - we don't gatekeep this technology or restrict you to a single provider.
**This is not a local service** If you want to fine-tune a model locally you can export your data directly from AnythingLLM and do this yourself. A locally running no-code trainer video will be coming soon - subscribe to get notified!
Downloading AnythingLLM: anythingllm.com/download
Star on Github: github.com/Mintplex-Labs/anything-llm
Send me an email: team@mintplexlabs.com
Chapters
0:00 Introduction to no-code fine-tuning with AnythingLLM
0:25 Disclaimers and caveats
0:47 How to download AnythingLLM
1:27 no-code Fine-tuning overview
2:00 Who this video is for
2:37 Gathering good training data with RAG + AnythingLLM
3:44 Reviewing the dataset we will train on
5:28 Fine-tuning vs RAG for the everyday person
7:00 Ordering a fine-tune
8:04 Privacy & Data Handling
8:44 Selecting a base model
9:18 Uploading our dataset
10:32 What happens once you start
10:54 What you get once tuning is done
11:49 Loading our custom fine-tune into Ollama
15:38 Testing our new fine-tune out!
16:43 How to uninstall a custom fine-tune from Ollama
17:30 Loading our custom fine-tune into LMStudio
20:57 Conclusion and Thanks!
มุมมอง: 43 248
วีดีโอ
Unlimited AI Agents running locally with Ollama & AnythingLLM
มุมมอง 157K6 หลายเดือนก่อน
Hey everyone, Recently in AnythingLLM Desktop, we merged in AI Agents. AI Agents are basically LLMs that do something instead of just replying. We support both tool-call-enabled models like OpenAI but have even now have a no-code way to bring AI agents to every open-source LLMs like with Ollama or LMStudio. Now, with no code required, you can take any LLM and get automatic web scraping, web-bro...
Stop paying for ChatGPT with these two tools | LMStudio x AnythingLLM
มุมมอง 279K9 หลายเดือนก่อน
In this video, we are installing two user-friendly tools that make downloading, running, and managing a powerful local LLM to replace ChatGPT. Seriously. Today, with only a desktop computer with a retail GPU, and two free applications you will get a fully private Local LLM RAG chatbot running in less than 5 minutes! This is no joke - the teams at LM Studio and AnythingLLM are now fully integrat...
Unleash the power of Local LLM's with Ollama x AnythingLLM
มุมมอง 127K9 หลายเดือนก่อน
Running local LLMS for inferencing, character building, private chats, or just custom documents has been all the rage, but it isn't easy for the layperson. Today, with only a single laptop, no GPU, and two free applications you can get a fully private Local LLM RAG chatbot running in less than 5 minutes! This is no joke - the teams at Ollama and AnythingLLM are now fully compatible, meaning tha...
Custom AI Chatbot for Websites using any LLM | No-Code | Open-Source
มุมมอง 15K9 หลายเดือนก่อน
In a world of pay-per-token LLMs you probably have looked everywhere to get your Local or Cloud-hosted LLM model to act like an AI assistant on your website - well now with AnythingLLM it is free and open source! Today, we are going to create an AI chatbot for our website trained on the data from our own website! All of this is built into the AnythingLLM cloud or our self-hosted version you can...
🔐 Deploy a PRIVATE Chroma vector database on Render.com | tutorial
มุมมอง 1.7Kปีที่แล้ว
Hey everyone, In Discord & TH-cam, I have seen a ton of people looking for an even easier option for deploying Chroma on more "user-friendly" server instance providers like Render.com Render is like AWS, but if they cared about their customers. I am a customer of Render and that simplicity and ease of use comes at a cost! This deployment of Chroma (v0.4.18) is super easy but it will cost you ab...
🚀 Deploy a PRIVATE Chroma Vector DB to AWS | Step by step 🚀
มุมมอง 3.7Kปีที่แล้ว
Cloudformation template: s3.us-west-1.amazonaws.com/public.mintplexlabs.com/chromadb-with-api-key.json Script to reboot Docker: gist.github.com/timothycarambat/754969f00b9815459153ff4f66a5910f Hey there! This is the second video in a three-part series on how to launch Chroma (trychroma.com) and have it be fully private with no unauthorized access so you can store your vectors without worrying a...
How to run a private Chroma Vector Database locally in 5 mins!
มุมมอง 26Kปีที่แล้ว
Hey everyone, I wanted to take some time to show how simple it is to get Chroma (trychroma.com), an open-source vector database, to run locally on your machine so you can use it with AnythingLLM (github.com/Mintplex-Labs/anything-llm) or other popular services as well as even secure the data between updates and use API key authentication! Watch the AWS or Render videos for how to take these sam...
AnythingLLM | How to get a Private Uncensored ChatGPT using Mistral-7B, LLama2, & more
มุมมอง 12Kปีที่แล้ว
Another update is hot off the press for AnythingLLM, a fully open-source and private ChatGPT equilivant software you can use. Now, we support using custom LLM models you can run on your machine or servers you own. Now, you can use models like Mistral-7B, CodeLlama, Falcon-40B, and others and get a ChatGPT experience more! Github: github.com/Mintplex-Labs/anything-llm Hosted: useanything.com Con...
OpenAI just launched custom ChatGPTs for all pro users!!
มุมมอง 256ปีที่แล้ว
OpenAI just launched custom ChatGPTs for all pro users!!
ChatGPT displays random dudes selfie in thread!
มุมมอง 462ปีที่แล้ว
Link below! a user reported that during a programming question thread chat GPT 3.5 responded with a picture of a random man selfie during response. chat.openai.com/share/ba0faec8-bd98-46ee-a010-fe871709bc5d
[FREE] AnythingLLM v2 | The last document chatbot you will ever need
มุมมอง 56Kปีที่แล้ว
AnythingLLM is an open-source full-stack "Chat with your documents" application that is constantly evolving and will enable you to chat with your documents in a private and enterprise environment. Lean more: useanything.com GitHub: github.com/Mintplex-Labs/anything-llm Chapters 0:00 What is AnythingLLM & what’s new?0:46 Where to find on GitHub 1:29 Onboarding - Introduction 1:51 Onboarding - LL...
Launch a Private AI Document Chatbot in 10 minutes! | Open Source
มุมมอง 12Kปีที่แล้ว
AnythingLLM is a private, customizable, and extendable single or multi-user document chatbot software. useanything.com In this video, we will show how to deploy a working instance of AnythingLLM in 10 minutes! While this method is more expensive to run it can be a quick solution for a proof-of-concept! AnythingLLM supports multiple vector databases like Chroma, Pinecone, QDrant, and Weaviate as...
VectorAdmin | The universal GUI for vector databases
มุมมอง 11Kปีที่แล้ว
VectorAdmin | The universal GUI for vector databases
AnythingLLM | The easiest way to chat with your documents using AI | Open Source!
มุมมอง 33Kปีที่แล้ว
AnythingLLM | The easiest way to chat with your documents using AI | Open Source!
why I even built @congressstockwatcher .its a free tool for public good!
มุมมอง 328ปีที่แล้ว
why I even built @congressstockwatcher .its a free tool for public good!
How to add partytown.js to your website for 10x gains
มุมมอง 7Kปีที่แล้ว
How to add partytown.js to your website for 10x gains
OpenAI is building a Github CoPilot Killer with ChatGPT? & How to find it
มุมมอง 2.2Kปีที่แล้ว
OpenAI is building a Github CoPilot Killer with ChatGPT? & How to find it
[No-Code] Mint an NFT with ANY ERC-20 token using Rampp.xyz
มุมมอง 5592 ปีที่แล้ว
[No-Code] Mint an NFT with ANY ERC-20 token using Rampp.xyz
Create an ERC-1155 Collection without code | Rampp.xyz
มุมมอง 1.5K2 ปีที่แล้ว
Create an ERC-1155 Collection without code | Rampp.xyz
Token-Gated Discounts for your Shopify store without code!
มุมมอง 2.1K2 ปีที่แล้ว
Token-Gated Discounts for your Shopify store without code!
Launch a No-code minting website to earn ETH | Rampp.xyz
มุมมอง 2.4K2 ปีที่แล้ว
Launch a No-code minting website to earn ETH | Rampp.xyz
Listing your Smart Contract on OpenSea & Airdrop without code | Rampp.xyz
มุมมอง 2.2K2 ปีที่แล้ว
Listing your Smart Contract on OpenSea & Airdrop without code | Rampp.xyz
Deploying your NFT smart contract without code! | Rampp.xyz
มุมมอง 1.3K2 ปีที่แล้ว
Deploying your NFT smart contract without code! | Rampp.xyz
Create an NFT Smart Contract WITHOUT CODE | ERC-721A | Rampp.xyz
มุมมอง 3.3K2 ปีที่แล้ว
Create an NFT Smart Contract WITHOUT CODE | ERC-721A | Rampp.xyz
Upload NFTs to IPFS without any code! | Rampp.xyz
มุมมอง 2.4K2 ปีที่แล้ว
Upload NFTs to IPFS without any code! | Rampp.xyz
Using Hashlips to create a custom NFT collection | Rampp.xyz
มุมมอง 1.5K2 ปีที่แล้ว
Using Hashlips to create a custom NFT collection | Rampp.xyz
[Web3] Using OpenSea's API to display NFTs with Javascript!
มุมมอง 17K2 ปีที่แล้ว
[Web3] Using OpenSea's API to display NFTs with Javascript!
Decentralizing your NFT project with Rampp + IPFS
มุมมอง 1.9K2 ปีที่แล้ว
Decentralizing your NFT project with Rampp IPFS
[Web3] Using Web3.js to login with Ethereum + read and modify smart contracts!
มุมมอง 21K3 ปีที่แล้ว
[Web3] Using Web3.js to login with Ethereum read and modify smart contracts!
Thanks.
Possible without a gpu?
@@M4XD4B0ZZ absolutely
hello tim. i am pretty interested in the possibilities anythingLLM and Ollama presents, but the question for you (as a person with lots of experience in my book): if I want to make an agent, who would analyse some twitter threads / post in a theme (let's say, nature, or sports), and then write / reply based on what was read. what would be the tool or a service to make such a beast? would it be done greatly in a no-code pipeline (many of them, possibly), or there would be an easier way around?
Great work , congrats . Im Very impressed with your gift of comunicate and the impressive amount of work that it took to develop this tool!!!
Omg I've found u !!!! Been searching over the net. None of it are legit. Yours are true value
This does not work as expected. I still can do the request without any token to the server.
Amazing🎉
Hey Tim, great video! I have a question: I write short 10-minute stories, all based on a five-step storytelling structure. Is it possible to fine-tune a model so that, whenever I ask it to write a story on a given topic, the model consistently follows this five-step process?
why you need anythingLLM when LMStudio already got chat function?
@@ariyako they have simple chat to test the model out. There is no rag, agents, custom agents, data connectors or any other tools besides the chat with the model directly and their api
@@TimCarambat now it make sense why need anyllma. appreciate it! thanks!
@@ariyako you can connect anythingllm to lmstuido and get best of both worlds. Lmstudio is a great llm engine and has many runtime, more model selection, and gpu layering. So both can be used and frankly it's a better experience when used together
Excuse me. I use the Agent Skill SQL Connector and seems it doesn't work. How can I verify that my LLM is connect to SQL database?
@TimCarambat is multimodal coming? website form data entry? THANKS for the wonderful work. Starring github now.
what are the recommended minimally needed windows and mac hardware specs for this... I'm asking for a friend on a budget :;p ? eGPU possible?
That was a really good video. Thank you so much.
Hi, I'm new to llama. How to start with a Windows computer?
Hey @Tim, I didn't see it on the github page but do you have support for SAML? I want to deploy this at work.
Hi, is there a way I can have this running on my machine but then access it from my phone or another laptop through the browser when im on 4g or a different network?
@@willdarby9259 yes, but you should run the docker version since that is accessible via a browser. Then you can just expose the required IP and port and use AnythingLLM on the go
Great content! Thanks for "birthing" this cool tech. I'm a documentarian with a library of interview transcripts. I'm trying to find the best local method to create a RAG to search and interact with them. Currently, they're PDFs and CSV files. I installed AnythingLLM and have been getting some lackluster results. Probably my fault! What's the best set-up for my use case re. LLM, embedder, vector db, etc? Currently, I'm using all the native options and I find the RAG is hallucinating a lot and/or not showing all results of a search term. Any tips would be appreciated! Thank you!
Sure! So PDFs are pretty cut and dry and the defaults work. CSVs and other tabular data though are HARD. The nature of CSV's often requires full-document comprehension which basically means you need a massive model (Google Gemini, Anthropic) to digest them. The alternative is to load them into a proper database so at least relationships exist. CSV's cannot be effectively used in RAG since there is no "semantic" relationships among datapoints for the most part. This makes "chunking" effectively worthless. Checkout Document pinning here for solving CSV issues, you may have to use a cloud model for those since they are probably thousands of lines long docs.anythingllm.com/llm-not-using-my-docs
@@TimCarambat Thanks so much for your reply, Tim! I first started with PDFs, but because the timecode info wasn't consistent across all PDFs, I converted them into 3 column CSVs. In my system prompt, I explain how the CSVs are structured and where to look for the data. But, you're saying, even with my instructions, CSVs are still difficult to work with? Side note...I'm about to start creating content for my filmmaker's channel. I know transcript wrangling is a popular topic/pain point. I'm sure you're super busy, but if you'd be interested in doing a quick interview, perhaps we can shine the light on a local llm solution for filmmakers who typically wouldn't consider it. Let me know!
Great software, great video, a lot to learn from it so, way to go man! thanks for such a brilliant AI piece.
50usd pm min ?!
@@louisduplessis5167 you can run it free for desktop. You can self host at your cost, and if you can't do any of that, yeah you can pay us to host it for you.
Thanks Tim for this excellent tutorial. Best wishes for your venture .
So good!! Thank You Man.
Can you make a tutorial how we can make either or the other to TTS for the AI-Response in a chat? I don't mean speech-recognition. just AI voice output.
Could not respond to message. Ollama call failed with status code 404: model "llama2" not found, try pulling it first
If you see this error, add your local ollama again and this time save it. The save button was positioned on the far right and I never noticed it. This is an irritating af trend caused by the pressure to make all apps "mobile friendly."
This is actually not caused by that because the app doesnt even have a mobile form. Its place there for a totally unrelated reason
Is possible to use AnythingLLM, RAG and then download model and upload to other local device ? I dont want to use AnythingLLM on linux device and use its API
You can use the API from any other device over LAN, so yes
Can I finetune models from the cloud version? I have absolutely no coding skills.
Yes, but i would recommend still using the desktop app so you can easily load that model in locally once its ready
Bro, this is exactly what I was looking for. Would love to see a video of the cloud option at $50/month
@@monbeauparfum1452 have you tried the desktop app yet (free)
It works on my desktop with an nVidia RTX4080 Super . However, what I need is to be able to upload a 120 pages .pdf technical document (eg 9MB pdf) and get a brief extract of main issues raised in the document. I use Ollama models - so far all say that they can only see snippets , so useless. ChatGPT can do it perfectly, but there is a confidentiality problem. Can anybody help , what is the best way to run such an analysis on my private desktop? Can I uise Anything LLM and how ?
This is because when you use ChatGPT it injects all 120 pages into the context window. All of this is also able to replicated with local models and in AnythingLLM. By default, we chunk the document since that is fundamentally how RAG works. docs.anythingllm.com/llm-not-using-my-docs
Amazing, the way you have explained a complex concept. Thank you
Ok, the Claude API works very well. I have a question: how can I fetch an entire website, not just a single page link? If I want to load all pages of a website, how can I do that?
Open the file manager/upload modal -> Data connectors -> Bulk link scraper
Either I don't know how to set it up correctly, or it works extremely slowly on the local computer. It's practically unusable.
@@webagewebdesign2114 I'm gonna go with you trying to run a massive model on CPU only and probably a mid range device. The software is as fast as your machine will run an LLM. If it's slow, use a smaller model or get a machine that has better specs, and even better, has a GPU!
@@TimCarambat I use an HP desktop with AMD Ryzen 7 3700X, 16GB RAM, and ASUS AMD Radeon RX 6600 V2 Dual GPU, 8GB GDDR6, 128-bit. It's not a new or top-tier configuration, but it's well above most laptops. And if this configuration has problems running local LLMs, then installing a local LLM on laptops makes no sense. For a simple question like what's 2+2, the processor usage reaches 60%, the fans go to maximum speed as if the desktop is about to take off. I tested with Claude.ai's API and it works well. Only now do I realize the insane computing power of OpenAI, Anthropic, etc. servers if my desktop takes 1 minute to give the result for 2+2.
@@TimCarambat I noticed that the GPU is not used at all. But, it's ok, I'm using Claude's API variant and it works fine. Thanks.
Screw giving a Github star (I did anyway). Tell me when you're going public so I can buy shares!! lol Seriously, you have a winner here.
@@JRo250 ill be sure to preallocate shares for star givers. Note to SEC: this is a joke. Maybe
THANKS! how to port on WEBSIDE local ??
have one doubt from langchain.from langchain.llms import Ollama llm=Ollama(model='llama3') The response going through rest api using post method , can we acess server side code as it is local host , I want to see the internal working how my input is going to lma model and all the internal detail of model in .py itself
Is it possible to stay in OpenWEBUI and use AnythingLLM by using API on port 3001 like RAG ? I don't find solution in Internet :(
Yes, We have an openAI compatible endpoint so you can just use that. You can find the exact endpoint by going to the Developer API page in AnythingLLM and opening the documentation. OpenAI Compatible endpoints are at the bottom
This guy looks like my former dealer lmao 😂
@@bigsmoke4568 former because they found a new line of work or the other outcome???
Do you have agent zero support?
I have never heard of agent zero, is that a framework for Agents like CrewAI?
Could we set up a trigger to then use the ai agent? For example a new slack message comes in then it uses the agent we built in anything llm?
Like a webhook I am presuming? By default all agent skills are invoked via chat, but you could build a webhook middleware and use the AnythingLLM api to make exactly what you are requesting occur since the API can also leverage agents just like the normal UI
Really awesome stuff. Thank you for bringing such quality aspects and making it open-source. Could you please help to understand on how efficiently the RAG pipeline in AnythingLLM works ? For example: If I upload a pdf with MultiModal content or If I want my document to be embedded in a semantic way or use Multi-vector search, Can we customize such advanced RAG features ?
yeah we need this, rather then deleting entire namespace
You're an excellent teacher. Thanks
Very cool. Thanks You! This open-source, ethically crafted and versatile application makes ollama RAG and more easy. Clear and transparent Linux install and update instructions too. Well done.
Followed the instructions. At my first question in chat with mistral-7b, I get "Only user and assistant roles are supported!"
Using LMStudio?
Greetings! And thanks for your dedicated work! I'm no great programmer but I'm getting the hang of AnythingLLM. How can Vector-Admin be set up to run with AnythingLLM vector databases? Thanks.
After integrating with Chroma and making a Workspace in AnythingLLM every chat in that Workspace generates the following in the AnythingLLM logs. Any ideas? [backend] error: ChromaDB::namespaceExists InvalidCollection
Instead of dragging files, can you connect it to a local folder? Also, why does the first query work but the second always fail? (it says "Could not respond to message. an error occured while streaming response")
This is great! So we woukd always have to run lm studio before running anything llm?
If you wanted to use LMStudio, yes. There is not specific order but both need to be running of course
good tutorial. helped me make my script
Wao what a great tool. Congratulations and thank you. Can you make a video explaining licence and commercial use to sell this to clients? Thank you.
License is MIT, not much more to explain :)
love u !! that's crazy stuff dude i test it for special case .. that's insane can i specialize my llm with my local architecture ?