how to run deepseek-code-7B in ML Studio? its perfect for coding but i dont get a good answer. I dont know which Preset (on the right) to use for this model.
I just tried the local server of LM Studio...using it, and its examples I had a c# console app setup and talking to it in about 15 minutes. Easiest API to use, maybe ever. So good I came here to mention it! :)
Do you have a running list of the best models for each category? I can't always remember which one you tested last for either coding or uncensored ect. Thanks for any comments.
Matthew, can these models be DLed to an external drive and used from there? Can you set up Agents? No capability to upload files? Can you report how the M processor does against a GPU? How well does the locally ran dev version scale? Obviously based on the size of the computer this is running on, but will it handle multiple requests from developers?
you can def. have the models on a separate drive, which is super useful. not sure about the rest, but to the last question: via the API mode (emulates style of open AI api) you can have several requests, that then get queued up one after the other.
In your demo you seem to only use 1 GPU layer. For my "old" Nvidia 2060 with 6GB I can easly do 40 layers on the GPU and it is very fast with for example the mistral dolphin 2.2.1 Q5 models. The API feature is brilliant, I use it for developing my own agent using a system message to give it some interesting features in its output (calling functions).
LM studio is awesome . running the server and operating open source models from an IDE I was able to get it to perform pretty much on par with gpt 3 j. just a bit slower . running the server is the way to give your llm the most tokens possible for inference while you formulate your questions around json's and SPR sparse primary representation prompts in the IDE. At one point I had dolphin 2.2 telling a story for over an hour strait with out stopping and not even repeating itself until I shut it off . Massive unexplored potential there.
I know this is very easy to use but there are plenty open source solutions to do the same. It would be good to inform about the data collection these companies are doi g on the users who use their software.
looks nice, but I wouldn't rely on it for testing in your videos until you can specify prompt formats (there's a good chance the model might be handicapped by the wrong format, currently it only lets you edit context, not the full prompt format). Also it only uses llama.cpp, which means anyone with an nvidia GPU could double their speed by switching to ExLlamaV2 and EXL2 quants.
please do a tutorial with this for memgpt. ive been using lm studio for a couple weeks now. ive seen people get memgpt to work with the server but some people have issue, me included
Is it possible doing document retrieval within LMStudio? For example, a chatbot that can chat about .pdfs / .csvs / .txts, given to it?. If not possible, would privategpt be a better alternative? It seems very intuitive there. Couldn't find anything on google.
This looks much easier than TextGen WebUI. I haven't looked into it but I hope LM Studio will not record my usage for anything. Another interesting thing would be if we can use AutoGen or MemGPT to extend its capabilities. And if we can "chat with our own doc" using LM Studio.
Is there a way to add your own text files, datafiles etc.? So when using the chat, it also knows the specific info about a subject from the files I provided?
I am also very interested in this. i want to add tons of local video and audio content to the chosen LLM. would love to batch it in. anyone else doing that sort of thing?
I'm waiting for an open model to implement OpenAI's function stuff reliably. That would make up for a lot regarding the differences in intelligence between GPT-4 and open models.
I'd say at the rate open llms are advancing, we'll probably have this ability within a years time. Although it's nice that we have the framework for when that does happen.
I believe that if you are enabling Apple Metal requires specific models that was trained with Apple Metal. Also if you are on Mac you can run - open -n -a "LM Studio" - to spawn multiple instances to run different models. I am going to try to do the Linux beta and see if you can get more configuration via CLI for a real server.
I have been yelling about this and Faraday (my favorite) for quite some time now, but for some reason you never see any of the ai channels telling you about. These are the easy to use programs that the majority of ai users want.
Those 'should work' etc is not based on your system, it's about compatibility with the LM Studio app. (GGUF models) I have 128GB system and 40GB VRAM, and it also shows the 30GB+ required warning.
Would you be able to cover more in-depth about the developer side? I would like to host on my local PC, but be able to access it from my android phone.
can you make a tutorial on how to run and test local llm models on a linux server for business us. also using vector embeddings to allow way more data to chat with. ?
im a complete coding/compter numty and got it running fine. Quest i dont know is how to take a basic base model and add learning to it. It told me it can extract information from webpages, but it dosnt seem very effective
Its funny how everything that is "dead simple" is just simple enough for most people to start using. But yes, LM studio is maybe the best entrance to using open source LLM's.
Great video. Would you happen to have "toyed" with the RAG option of this tool? If you happen to have done a video, I would gladly view it. Thank you, very quick and instructive.
Looks great, Thank you. But LM Studio didn't work with own text, PDF or Docx files, right? Also no dialogue mode possible. Is there a video that shows how to create own LLM? Thank you.
I would love to install this, but I dont think it has a local web option like Gradio. Which would allow me to access it from any device in my network or from outside my home through a local IP / port
@@Pyriold I did see that but from the video it seems its an api backend type of connection only, not a connection that has a gui and complete usage that you can use by simply browsing to the ip and port and see the entire front end like you can with Oobabooga, and Automatic 1111
Thanks @matthew_berman One challenge I haven't solved yet is moving an environment. At the office I have the OK to explore LLM potential BUT within the existing software and hardware constraints. My PC is good enough, but our network is so locked down none of the scripts can pull down requirement files and libraries. I'd need to setup an environment on an 'internet facing' computer and then be able to move it. And run it. Is that possible??
Could try using a VM. I noticed a company called Massed Compute has VMs specifically for Matthew. All of the tools he has used in his videos are pre-loaded
@@etunimenisukunimeni1302 ig, someone will make open source version of this in the near future.. may be not with all features and this much polished, but mostly..
Not sure if the title changed in the last 3 hours. But he only said about the model being open source. The closed source part is only the frontend which should be fine.
TBH this is just a UI for open source models. I really can live with a closed source product here. There are actually OSS alternatives. So, no need to worry.
This was well made and useful thanks. Could I ask: can LM Studio use the models to process some data e.g. take a csv file, process the data, and output an excel sheet? I do this with Chat GPT usually in a Chrome window and it seems to save the output file somewhere on my PC and gives me a link to click. Can LM Studio do something similar?
Thank you for you tutorial! Some questions for you: Can I upload text or other format document and question the model about the content? And I need to be sure that the privacy of the text/document uploaded is granted. If the computer is online, the contents uploaded would always and anyway be private?
First I want to thank for sharing the useful AI content. The LM Studio software was a key step to bring AI assistants a step closer to the customers and consumer. I made use of the software as well and was recently experimenting with dolphin mistral llm 2.2.1 and wondered after a while what the token count 4984/2048 at the bottom right below the chat input means. As far as I understood, it's some sort of counter how many tokens the llm already has written and answered, but why does it matter? Is the chat history fed into the language model each time we enter something new, and this happens somehow behind the scenes? When these language models are working like this, I would understand that the natural limit of the input the language model supports also is the maximum size of the chat history. I am not very familiar with LLM s and just started experimenting with them. Could someone please explain why the token count: yxcd/yxcd number is there and how it affects the Assistants' performance or affect the chat in which way? Thanks in advance
Can you please provide a specification for PC build of $2400, which will run ai models locally in fastest way possible at this price. What things we should consider when building PC solely for running ai models locally and rarely gaming? What really helps to run this model fastest locally? please provide related information also. I want to build a PC with budget of $2400. Thank you.
Could you explain more about how that "Apple Metal" configuration works? Is it only for models trained on apple metal? What changes if it is enabled or not?
This looks awesome. I have a query, let's say I finetune the llama2 model on my custom data. In this case, can I use that new model and integrate with LM studio?
There are so many models out there Ive been looking and my question is really can i find a model that would be able to answer any computer error messages i get and how to fix them, what model would i pick if I wanted that? ANd i guess as a side note im also a musician / music producer and would like to find a model that is best for music production or creative writing songs both chord structure/progressions as well as song ideas and lyrics. How do i find that? Ive already searched any and all the keywords I could think up to find that stuff but nearly any of the keywords I try does not bring up anything so im guessing I just dont understand how to look for things. Any help like a link of something to read or anything would be amazing.
Thank you for your tutorial and the channel. It would be nice if you can teach how to process files with LM Studio. For example, I have an email (HTML) that I want to parse and structure. I would be interesting to learn all the details how to tune temperature, tokens, context window, etc. Thanks
I Have a suggestion.... If is it possible make a video about how to get and see image ressponses from a image generation model like FLUX.1 or Stable Diffusion 2. I ran it in LM Studio by i got a markdown with an unuseful link to imgur or other similiar.. None of thos links works.. Thanks
Very cool tool. Thanks for this nice tutorial. I wish some day you give lollms a try. It has a models zoo and can run multiple types of models including GGUF , gptq and now awq. It has a persona system and can be installed with a single file install script. It supports basically all remote and local LLMs. Asits name suggests, it is built to support everything that crawls out there. It can be used to generate text, image and audio. It has an extension system (WIP) and It took me hell lot of effort to make. it is 100% free under apache 2.0 licence and there are documentations on my modest youtube channel. I think you can present it way better than I do :) .
lollms is absolutely worth giving a try, I installed it and have been using it for a week now. It has so many features and functions it's almost unbelievable that it's just a couple of devs behind it all. It's a bit rough around the edges in some aspects, but still very much functional and new additions and bugfixes are published constantly. It's been my favorite so far.
Thanks for this video! I was confused why you hadnt shown or seen this before. LM Studio has been my go to, it runs fast and has GPU / CPU support. I dont know another tool that works as well.
Is there any playground studio like that but for commercial llms where you put your API keys and can play with anthropic, openAI, Cohere models in one interface?
Awesome! are you going to create a video on OpenAI's upcoming 'create your own gpt'? would love a video covering that, and exploring any alternatives for Mistral or Llama (ie open source).
I'm trying this on an older computer. I got the message "This processor does not support AVX2 instructions." is there a way to get this to work? Obviously, it would work on a newer processor, but I don't want to test it on computers I actually use. Or, is there a model that will work on less than AVX2 instructions?
The best discount for Black Friday: bit.ly/46bDM38
Hi Matthew, i love you videos, watching all of them lately, but i have to ask, did you forget the Autogen Expert Tutorial??
how to run deepseek-code-7B in ML Studio? its perfect for coding but i dont get a good answer. I dont know which Preset (on the right) to use for this model.
Fake?
It would be great to see how it works with autogen and memgpt. And thank you Mat for this great job your are doing! Keep doing!
@@MrAndi1281 haha no, but that’ll take a bit longer to put together
I m blown by the possibilities it brings to the users who are not comfortable with the earlier present methods.
I just tried the local server of LM Studio...using it, and its examples I had a c# console app setup and talking to it in about 15 minutes. Easiest API to use, maybe ever. So good I came here to mention it! :)
Do you have a running list of the best models for each category? I can't always remember which one you tested last for either coding or uncensored ect. Thanks for any comments.
Matthew, can these models be DLed to an external drive and used from there? Can you set up Agents? No capability to upload files? Can you report how the M processor does against a GPU? How well does the locally ran dev version scale? Obviously based on the size of the computer this is running on, but will it handle multiple requests from developers?
you can def. have the models on a separate drive, which is super useful. not sure about the rest, but to the last question: via the API mode (emulates style of open AI api) you can have several requests, that then get queued up one after the other.
@@just..someone thanks for the reply
In your demo you seem to only use 1 GPU layer. For my "old" Nvidia 2060 with 6GB I can easly do 40 layers on the GPU and it is very fast with for example the mistral dolphin 2.2.1 Q5 models. The API feature is brilliant, I use it for developing my own agent using a system message to give it some interesting features in its output (calling functions).
Any link?
@@irakmendez9985 You can search from inside the LM Studio software
Just in time! Thank you. How is the MemGPT setup process? I’m gonna try this out after work. Thanks.
Do they have a document upload feature, so that we can chat about our document like the custom GPTs?
Seconded.
I’m wondering about this too
LM studio is awesome . running the server and operating open source models from an IDE I was able to get it to perform pretty much on par with gpt 3 j. just a bit slower . running the server is the way to give your llm the most tokens possible for inference while you formulate your questions around json's and SPR sparse primary representation prompts in the IDE. At one point I had dolphin 2.2 telling a story for over an hour strait with out stopping and not even repeating itself until I shut it off . Massive unexplored potential there.
OMG! What was the quality of the story like ? You think it could readable and enjoyable novels on demand ?
@@retex73 I too am interested in this magic?
got difusionbee for local picture generation
so its about time we had an easy way to use LLM on our local Maschine
Would be a little more useful if the system would allow you to upload documents so you could perform actions like summarization
You can do that with GPT4ALL with a plugin.
learn how to code lol
I know this is very easy to use but there are plenty open source solutions to do the same. It would be good to inform about the data collection these companies are doi g on the users who use their software.
Do you know if LM Studio actually collects anything? Has anyone run a packet sniffer to check if it actually sends packets somewhere?
Like which one? Can you tell us some open source examples?
Like which one? Can you tell us some open source examples?
Any recommendation for doing stuff in CLI?!
I like how with nearly no context, "Mario" just pumped out a romance novel scene.....
OMG I stopped the video to read it and honestly I never imagined Mario in such a context. What did he do to Peach!? Is it even Peach!?
that was wild as hell, mario and his taboo activities
looks nice, but I wouldn't rely on it for testing in your videos until you can specify prompt formats (there's a good chance the model might be handicapped by the wrong format, currently it only lets you edit context, not the full prompt format). Also it only uses llama.cpp, which means anyone with an nvidia GPU could double their speed by switching to ExLlamaV2 and EXL2 quants.
please do a tutorial with this for memgpt. ive been using lm studio for a couple weeks now. ive seen people get memgpt to work with the server but some people have issue, me included
or with anything that calls an openai api for that matter i just really wanna try memgpt and chat dev with this thing
Is it possible doing document retrieval within LMStudio? For example, a chatbot that can chat about .pdfs / .csvs / .txts, given to it?. If not possible, would privategpt be a better alternative? It seems very intuitive there.
Couldn't find anything on google.
thanks for video, please could you advice how to work with local documents with the model?
That was amazing. You are helping us so much introducing to all this tools. Thank you very much.
I've been trying this out and it honestly delivers. So easy to run your own AI's!
This looks much easier than TextGen WebUI. I haven't looked into it but I hope LM Studio will not record my usage for anything. Another interesting thing would be if we can use AutoGen or MemGPT to extend its capabilities. And if we can "chat with our own doc" using LM Studio.
Autogen and memgpt sound good with this.
Is there a way to add your own text files, datafiles etc.? So when using the chat, it also knows the specific info about a subject from the files I provided?
I am also very interested in this. i want to add tons of local video and audio content to the chosen LLM. would love to batch it in. anyone else doing that sort of thing?
I'm waiting for an open model to implement OpenAI's function stuff reliably. That would make up for a lot regarding the differences in intelligence between GPT-4 and open models.
Have you tried airoboros ? It's trained on function calling and works for me
I'd say at the rate open llms are advancing, we'll probably have this ability within a years time. Although it's nice that we have the framework for when that does happen.
Check out Gorilla open functions
I accidentally subscribed to you a long time ago, best decision I ever made.
Thanks!
I believe that if you are enabling Apple Metal requires specific models that was trained with Apple Metal. Also if you are on Mac you can run - open -n -a "LM Studio" - to spawn multiple instances to run different models.
I am going to try to do the Linux beta and see if you can get more configuration via CLI for a real server.
What telemetry does it capture/send?
In their website they say that they capture no data from user, we can only pray that this is true lol
I have been yelling about this and Faraday (my favorite) for quite some time now, but for some reason you never see any of the ai channels telling you about. These are the easy to use programs that the majority of ai users want.
Those 'should work' etc is not based on your system, it's about compatibility with the LM Studio app. (GGUF models)
I have 128GB system and 40GB VRAM, and it also shows the 30GB+ required warning.
that is a setting that you can turn off or make it less sensitive. key word is "safeguard" I believe
Would you be able to cover more in-depth about the developer side? I would like to host on my local PC, but be able to access it from my android phone.
Off-topic question- how can i save a fine-tuned model that i fine-tuned using gradientAI to run it locally.
Please reply, love your videos!❤❤
Love this!!! Thanks for always giving a walkthrough.
We are about 1 year into gpt pandemonium and the momentum is only getting faster and everything is easier
can you make a tutorial on how to run and test local llm models on a linux server for business us. also using vector embeddings to allow way more data to chat with. ?
im a complete coding/compter numty and got it running fine. Quest i dont know is how to take a basic base model and add learning to it. It told me it can extract information from webpages, but it dosnt seem very effective
Is it possible to upload documents and query them?if yes, can you show how? Is it also available through the API server? Thanks in advance!
Wish they would add the chat with your docs feature
Its funny how everything that is "dead simple" is just simple enough for most people to start using. But yes, LM studio is maybe the best entrance to using open source LLM's.
But can you fine tune the models or have it access your private documents locally on your computer?
Great video. Would you happen to have "toyed" with the RAG option of this tool? If you happen to have done a video, I would gladly view it. Thank you, very quick and instructive.
It's sadly not allowed for Commercial use. But indeed very easy to install and run.
Looks great, Thank you. But LM Studio didn't work with own text, PDF or Docx files, right? Also no dialogue mode possible.
Is there a video that shows how to create own LLM? Thank you.
I would love to install this, but I dont think it has a local web option like Gradio. Which would allow me to access it from any device in my network or from outside my home through a local IP / port
It has a local server mode, that was shown in the video.
@@Pyriold I did see that but from the video it seems its an api backend type of connection only, not a connection that has a gui and complete usage that you can use by simply browsing to the ip and port and see the entire front end like you can with Oobabooga, and Automatic 1111
Running the open-source models, but the software itself is not open source, lol
awesome review! I wonder how to make the LLM constrained to read and analyze local files?
LM studio is great. My only issue with it is that it's got very small font that I haven't found a way to change.
Same issue here.
Thanks @matthew_berman One challenge I haven't solved yet is moving an environment. At the office I have the OK to explore LLM potential BUT within the existing software and hardware constraints. My PC is good enough, but our network is so locked down none of the scripts can pull down requirement files and libraries. I'd need to setup an environment on an 'internet facing' computer and then be able to move it. And run it. Is that possible??
Could try using a VM. I noticed a company called Massed Compute has VMs specifically for Matthew. All of the tools he has used in his videos are pre-loaded
10:04
😂 Indeed. I think we know what its finetuned on hahaha.
Mario is definitely a hor.y little llm😂
TheBloke also gives recommendations for which models to use or not use - not necessarily which one is the biggest that you can run.
The only problem is...its CLOSED source, not open source program.
I agree, but what I'm seeing here, that seems to indeed be the _only_ problem. Which is cool, unless closed source is a showstopper for you.
@@etunimenisukunimeni1302 It looks great but unfortunately it is a showstopper, at least for now.
@@etunimenisukunimeni1302 ig, someone will make open source version of this in the near future.. may be not with all features and this much polished, but mostly..
Not sure if the title changed in the last 3 hours. But he only said about the model being open source. The closed source part is only the frontend which should be fine.
TBH this is just a UI for open source models. I really can live with a closed source product here. There are actually OSS alternatives. So, no need to worry.
Good afternoon Mr Berman... You have a talant doing these videos, you come over as clear as glass. well done.
Great video! Id love a lm studio with memgpt and autogen video if possible
LMStudio looks pretty awesome, great job on giving us a tour, going to try it, thanks !
Great video, thank you for explaining this.
This was well made and useful thanks. Could I ask: can LM Studio use the models to process some data e.g. take a csv file, process the data, and output an excel sheet? I do this with Chat GPT usually in a Chrome window and it seems to save the output file somewhere on my PC and gives me a link to click. Can LM Studio do something similar?
maybe with AnythingLLM you could do that, but I'm not sure, can work with LM studio too
would love some tips on which specs can run it or some recs on laptops
keep me updated
Thank you for you tutorial! Some questions for you:
Can I upload text or other format document and question the model about the content?
And I need to be sure that the privacy of the text/document uploaded is granted. If the computer is online, the contents uploaded would always and anyway be private?
Great video. How would you host this on provider like Linode or AWS?
crisp and concise content...
First I want to thank for sharing the useful AI content.
The LM Studio software was a key step to bring AI assistants a step closer to the customers and consumer.
I made use of the software as well and was recently experimenting with dolphin mistral llm 2.2.1 and wondered after a while what the token count 4984/2048 at the bottom right below the chat input means. As far as I understood, it's some sort of counter how many tokens the llm already has written and answered, but why does it matter? Is the chat history fed into the language model each time we enter something new, and this happens somehow behind the scenes? When these language models are working like this, I would understand that the natural limit of the input the language model supports also is the maximum size of the chat history.
I am not very familiar with LLM s and just started experimenting with them. Could someone please explain why the token count: yxcd/yxcd number is there and how it affects the Assistants' performance or affect the chat in which way?
Thanks in advance
How many millions of tons of carbon are being wasted listening to these models apologise?
Can you please provide a specification for PC build of $2400, which will run ai models locally in fastest way possible at this price. What things we should consider when building PC solely for running ai models locally and rarely gaming? What really helps to run this model fastest locally? please provide related information also. I want to build a PC with budget of $2400. Thank you.
does it have long term memory? or is it just chat like chatgpt? can we upload pdfs?
It has “chat” and no pdf support yet
Could you explain more about how that "Apple Metal" configuration works? Is it only for models trained on apple metal? What changes if it is enabled or not?
I think training and inference are totally decoupled, so it does not matter how it was trained, you can use whatever hardware for inference.
LMStudio lacks local documents, what a bummer I will continue to use GPT4All
This looks awesome. I have a query, let's say I finetune the llama2 model on my custom data. In this case, can I use that new model and integrate with LM studio?
waiting for this answer or updates if you get it working
Awesome content as always!
One thing I don't really understand is where I should place the php files, if I want to use php and curl, to access the local instance of the model.
Brilliant. Can you possibly use llava with lm studio to talk to images?
This is so helpful tutorial. Thank you so much!
brilliant! thank you Matthew and thank you LM Studio
There are so many models out there Ive been looking and my question is really can i find a model that would be able to answer any computer error messages i get and how to fix them, what model would i pick if I wanted that? ANd i guess as a side note im also a musician / music producer and would like to find a model that is best for music production or creative writing songs both chord structure/progressions as well as song ideas and lyrics. How do i find that? Ive already searched any and all the keywords I could think up to find that stuff but nearly any of the keywords I try does not bring up anything so im guessing I just dont understand how to look for things. Any help like a link of something to read or anything would be amazing.
quick question, when you download how do you specify what hard drive to download to?
Will watch. But question… Ollama or Llm studio?
Wow in your previous video commented on LMstudio and now I see a video on it. Wow you are the best
Can you train or fine-tune the models?
Thank you for your tutorial and the channel. It would be nice if you can teach how to process files with LM Studio. For example, I have an email (HTML) that I want to parse and structure. I would be interesting to learn all the details how to tune temperature, tokens, context window, etc.
Thanks
I Have a suggestion.... If is it possible make a video about how to get and see image ressponses from a image generation model like FLUX.1 or Stable Diffusion 2. I ran it in LM Studio by i got a markdown with an unuseful link to imgur or other similiar.. None of thos links works.. Thanks
Very cool tool. Thanks for this nice tutorial.
I wish some day you give lollms a try. It has a models zoo and can run multiple types of models including GGUF , gptq and now awq. It has a persona system and can be installed with a single file install script. It supports basically all remote and local LLMs. Asits name suggests, it is built to support everything that crawls out there. It can be used to generate text, image and audio. It has an extension system (WIP) and It took me hell lot of effort to make. it is 100% free under apache 2.0 licence and there are documentations on my modest youtube channel. I think you can present it way better than I do :) .
Look who's here, guys!
lollms is absolutely worth giving a try, I installed it and have been using it for a week now. It has so many features and functions it's almost unbelievable that it's just a couple of devs behind it all. It's a bit rough around the edges in some aspects, but still very much functional and new additions and bugfixes are published constantly. It's been my favorite so far.
Thanks for this video! I was confused why you hadnt shown or seen this before. LM Studio has been my go to, it runs fast and has GPU / CPU support. I dont know another tool that works as well.
Mario started writing 50 shades of grey for you 💀
GPT killer other than Sora. This will be what LLM will look like in the near future! So excited to get my hands dirty! 😮
How can I do fine tunning with my context? Is there any video that you explain it? It should be amazing! I tried LM studio! So easy! Thanks a lot!
keep me updated
Is there any playground studio like that but for commercial llms where you put your API keys and can play with anthropic, openAI, Cohere models in one interface?
Do you prefer this to ollama?
One Love!
Always forward, never ever backward!!
☀️☀️☀️
💚💛❤️
🙏🏿🙏🙏🏼
can you load pdf files for summarization etc. in those models?
Super One ! Thanks for this Video !
Does the mac version work on macs with dual-core intel i5 or just the m1/m2?
Love this! Thanks for sharing.
Can we load safetensor models as well ?
Awesome! are you going to create a video on OpenAI's upcoming 'create your own gpt'? would love a video covering that, and exploring any alternatives for Mistral or Llama (ie open source).
can you make a video about how to make that server in localhost, python and curl versions
Could you make a video about using this server Mode to power aider? I couldn't figure how to do it
Is there a way to chat with docs or doc repos from LM Studio?
So i could run this (10:34) local server as a local lan server on a PC that family could use? Not following the replace the base section.
hi nice video how did you make the intro transition ? thanks
is this an alternative to Midjourney , so text to image for macs?
I'm trying this on an older computer. I got the message "This processor does not support AVX2 instructions." is there a way to get this to work? Obviously, it would work on a newer processor, but I don't want to test it on computers I actually use. Or, is there a model that will work on less than AVX2 instructions?
Are you saying if we import OpenAi api to this the syntax is the same as openai but we're actually using the open source model?
Will it serve multiple models like ollama?
this was actually a really good video