วีดีโอ

You're right that your local ressources will limit the models you can use. For Mac users, an entry-level M1 MacBook Air should run quantized version of Llama3 at a reasonable speed.

Thanks Amin, glad you find them useful!

You can try, but it will probably drain your battery at record speed. The installation instructions are different than those in my long-form video, have a look at this blog post for Android: davidefornelli.com/posts/posts/LLM%20on%20Android.html For OpenWebUI as well you will have to follow the regular instructions, I wouldn't use docker.

That's a good question. I have only looked at local solutions for RAG, both in built-in solutions and DIY in python using langchain and vector dbs. So far the Msty implementation is the best that I have come across, but it's local only. I did make a video on AnythingLLM which also offers a cloud version, but to be honest I was unimpressed at the time. If I was to go at it myself I would probably build a custom solution in Python with a vector database to have more control. I don't know any no-code solutions, but I'm curious so please let me know if you find one that works well!

Most of the data is saved in "Library/Application Support/Msty/" directory. It is usually hidden on Mac, but you can access it in Finder using the menu item Go->Go to Folder... Some of the data will also end up in Library/Caches. The downloaded models are stored in the default Ollama directory, "~/.ollama/models".

Yes, kind of. It will let you download all the filing information. But keep in mind that 10Ks contain more than just the filing text. To get financials, you have to extract the XRBL information from the filing. Most filings also have attachments. Edgartools will let you download all of these, the hardest part for RAG is not the scraping, it will be to structure all that data in a useful way for querying.

I have tried uninstalling and reinstalling Msty, but the issue still persists!

Models downloaded directly with Ollama should be in the "C:/Users/username/.ollama" directory on Windows. If you copy the content, it should be ok. I presume Msty also saves them there, but I don't have a windows machine to confirm.

​@@VincentCodesFinance Thank you for your prompt response. I couldn't locate the models there, so I looked in the appdata folder and found them here: C:\Users\username\AppData\Roaming\Msty\models

Text models are optimized for text completion instead of chat-style querying. In most uses cases à la ChatPGT that we usually think of, the chat variant is the one you want. 2.You can copy any of the ones that are there (you should use llama3 no, not llama2, or one of the newest uncensored models.) The only thing to be aware of if that the command that gets copied is "ollama run modelname" instead of "ollama pull modelname". The run command will trigger a pull if necessary and the model will be downloaded, but run will also load the model in memory and make it available in the command line.

They added this feature (still in beta) in the new version. I've had mixed results with it using local models, it was not as good as copilot. For it to be useful, you have to use a very small model otherwise latency becomes an issue. Their documentation is a useful guide on picking the right model: docs.continue.dev/walkthroughs/tab-autocomplete

Thanks for the kind words! I have a few more scraping-related videos planned in the near future.

I'm using a macbook pro with m3 max cpu and 64gb of ram. If using a pc, you'll want a fast gpu with a decent amount of ram on the gpu otherwise ollama will run on the cpu (much slower).

Depends on the json file. You can try to embed it in a knowledge stack, but this is meant for querying, i.e. letting your LLM search the stack for the answer you're looking for. To process the whole file, I would first check if it can fit within the context of the model (llamma 3 support 8k tokens I think). If so, you can include the full file in your query. Otherwise, you could chunk your file and process it using a map-reduce approach like I do in this video: th-cam.com/video/Tnu_ykn1HmI/w-d-xo.html

@@VincentCodesFinancethank you so much for the detailed reply. I'm going to have a go at both of these methods 🙏😸

I should have been more precise. This gives you the financial data from the latest annual report (form 10-K). In the case of Apple, the latest financials available would be from the quarterly report issued earlier this month. You can get the latest quarterly financials using: `apple.get_filings(form="10-Q").latest().obj().financials`. Note that companies only file reports quarterly, some there is always some lage in the data.

It should work if you call .fit(cov_type="kernel") . By default it will use the bartlett kernel (same a Newey-West). You can specify the bandwidth using fit(cov_type="kernel", cov_config={"bandwidth": 3}) bashtage.github.io/linearmodels/devel/iv/iv/linearmodels.iv.model.IV2SLS.fit.html#linearmodels.iv.model.IV2SLS.fit

I don't know these two tools much. From a quick glance at their docs, it seems that they do not support Ollama, which is what Msty uses. Msty has a built-in Ollama instance, but it will share models with a stand-alone Ollama install if you have one. However, as far as I know it cannot share models with other local LLM servers. In another video I show how to setup Open WebUI, which is similar to TextGen WebUI and supports Ollama.

​@@VincentCodesFinance I can't wait for your next tutorial!

Glad you like it! thanks for the kind words.

Already working on it! It will likely be more than one video given all the new features.

I haven't tried to build a setup as involved as this. I think you can save the loaded documents, but I only tried it as a single user, so I'm not sure if sharing is possible, and what types of files are supported besides pdf. Be aware however that this is a recent open source project under active development, I would be careful before loading any sensitive medical data in a server software that has not undergone a security audit

Yes! You can specify the url with the apiBase parameter. See docs.continue.dev/reference/Model%20Providers/ollama for an example.

@@VincentCodesFinance Thank you 🙏

I haven't done this specifically, but you have to look at the filings that would have that information. For most companies, cash would be available on a quarterly basis in the 10-Q form, as part of the XBRL (I'm not sure of the exact xbrl field). I would approach this by searching for all filings of type "10-Q" that I'm interested in (based on year and company) and then retrieving the xbrl data for each filing, then extracting the cash flow column.

Thanks! I guess the main benefit of having it directly in VS Code is convenience: it lets you stay within VS Code, is integrated with the UI, and the buttons and slash commands will automatically wraps prompts around your code.. In terms of output quality, it will mostly depend on the model you use.

Awesome, thank you!

So far the easiest to install and use is Msty. However, it does not super documents (yet), it is just a pure chat for now.

That I don't know. You should ask Ashok (Msty developer) on their discord channel, he is very responsive to user questions: discord.gg/2QBw6XxkCC

​@@VincentCodesFinance Thanks for the response

I agree that the UI is inferior to pretty much all the other ones I tried. As for inference speed, I configured it to use my local Ollama installation, so the speed is the same. The one use case where I found AnythingLLM better than WebUI is for document search (Msty doesn't have it yet). I haven't tried LMstudio yet but it's high on my list.

@@VincentCodesFinance Try Msty inferece spped is really fast, i run a low en laptop, with 4g vram 16gb ram. Hope it helps

Thanks for the kind words, it's always nice to hear!

It is! In my latest video I show a new even simpler UI called Msty, which is a simple desktop app that comes bundled with Ollama (or can use your existing Ollama). If you don't mind using a closed-source UI, it's the nicest one I've seen so far: th-cam.com/video/REEYqYEtqAc/w-d-xo.html

@@VincentCodesFinance That sounds cool! It's awesome that you're checking out simpler UI options. Thanks for sharing!

Glad it was helpful!

Thanks for the tips. I never used them but will look into them. TaskWeaver seems like something that would be useful for my research work.

Hum, I don't have much experience with WSL. Could it be related to this issue? github.com/docker/for-win/issues/13845

Thanks! I have a MacBook Pro M3 Max with 64GB of ram. The Apple chips are great for running these model because the ram is shared between cpu and gpu, so the gpu can use all of it when needed. In my most recent video, I test the new models released this month to see which ones will run smoothly on my laptop: th-cam.com/video/0ujRg04fDW4/w-d-xo.html

Happy to help! Make sure to try the new Llama 3 that was recently released, it's a big step up from llama2 in the video.