วีดีโอ

Thanks. You saved me an headache.

Just use the ollama plugin for Rivet and rename the chat node from ollama to "output"

@@AIMadeApproachable It seems like there is no support for ollama chat. I get the following error message: 'Unknown node type: ollamaChat2'

@@AIMadeApproachable It seems like there is no support for ollama chat. It states on the debug console that the "ollamaChat2 " node is unknown

No it is not. Rivet projects are not using python. Basically a rivet project is just a yaml file describing the nodes etc. and then there is a node.js library that knows how to execute it

That is a topic I purposely left out. Not because it is not solvable, but because I did not want to make the prototype more complex. Currently the option "Enable Parallel Function Calling" is set to "off" in the chat node, but I also saw it happening once and assume that this on/off-switch in Rivet is not working as intended. If you want to enable it, the rivet project needs to be extended roughly like this: - Check function calling result (parallel or single call) - Bring the function calls into a common format (best is probably to format single function calls in some format as parallel function call) - Activate splitting on the match node and the following nodes, so that each will be executed for each entry in the array (= each function) - Then either send the function call results seperately into the 2nd chat node or merge them all before (although out of my head I also do not know how they need to be formatted for it to work)

@@AIMadeApproachable The `tool` features of the Chat Node are quite challenging to grasp. I attempted the method you suggested, and here's what I found: - When the 'parallel function calling' option is activated, the Chat Node does indeed initiate multiple functions at the same time as per the request. These can be seen under the 'Function Calls' output port of the Chat Node. However, in the 'All Messages' output port, only the message from the first function called by the assistant appears. - This indicates that the first function needs to be processed separately by providing a response to the function call. - The rest function listed in the 'Function Calls' port array must be processed individually. Afterward, their outputs should be appended to 'All Messages' (this includes the function request, function response, summarization, etc.). I apologize if my explanation of the algorithm comes across as unclear. My understanding of it is still evolving as I continue experimenting with it myself.

Well you can send multiple function call results back in one message (tested that today)... but Rivet does not handle it correctly at the moment. I created an issue and hope they will fix it: github.com/Ironclad/rivet/issues/381

Not really, sorry. Maybe this video helps you: th-cam.com/video/P1PhHWK6n9I/w-d-xo.html

Sure. Ollama can run gguf models. If you have a model that is not in the ollama model list, you just need to manually create a modelfile and use the "ollama create" command. There are some good instructions here: otmaneboughaba.com/posts/local-llm-ollama-huggingface/

Thank you! Take my answer with a grain of salt, as I am not a finetuning expert: To my knowledge there is no general answer how to handle training/validation data. Generally if you want some metrics about the effect of your training, it is advisable to generate validation data. Now depending on what you actually want to train, the first decision is already if your validation data should be disjoint from your training data or not. As in this case we wanted to not exactely answer with the phrases we trained it on, but generate a general understanding, I chose the data to be disjoint, to test if that works in untrained scenarios as well. About the data split itself: I read that it is typical to use 10 - 30% as validation data. I just picked one of those values ;) Most likely 10 or 20% will also suffice.

First of all: I am not the developer of the software, those are mainly Andy + Cai from Ironclad. I am just a user (and recently started creating some plugins for it). Bot about your request: Video about finetuning any local model will be out in a few days!

Yeah, it is open source.

@@AIMadeApproachable Thank you!

Not sure I am going to create a video about it. But I got this project, which creates an OpenAI SDK compatible API on any Rivet graph: github.com/ai-made-approachable/rivet-chat-api

I would generally advice you to use Retrieval Augmented Generation (RAG) for that use-case. But about finetuning: Well the only method you have for finetuning with OpenAI is SFT (Supervised Finetuning), which means you need to give it positive examples of conversations to learn from. You will need to think about your use case of that data from the 1000 pdf's and create training data based on that. am not sure what this would be, if you are not going to ask questions about the pdfs, which would bring you back to question/answer pairs. Generally you will need a unique workflow for your purpose, that e.g. chunks the PDFs into smaller parts and automatically generates question/answer/other desired conversation examples (using some LLM).

Just us an update: Maybe in the next 1-2 days you will not need to use my subgraphs anymore to make the ollama chat node work with any model. Just requested a pull request where I added the "promptFormat": "auto" option and ollama will handle it. Just needs to get approved :)

Problably tackling that next: The prompt needs to be split up into lots of steps that can be handled by weaker models. Only gpt-4 can handle a prompt with that many conditions properly. And then there is also the challenge of function calling: open source models cannot do/are not good at function calls. Or there are models that can only do function calls. Will need to see how to solve that, there are workarounds.

@@AIMadeApproachable Yes! And using chatbot UI

Thank you!

Hm the first case should theoretically be possible, but Chatbot-UI does not offer you the option to upload an image without selecting "GPT-4-Vision" as a model. I think both cases need to go as candidates on the feature list. There is much more I want to add (e.g. also ChromaDB).

I guess you could search and replace the logo file in the chatbot-ui github repository that is created once you deploy the solution.