วีดีโอ

@@KeesFluitman no xml file, it’s a csv but yes it’s direct access to the database. In a real environment this would run against a data warehouse, lake, or some backup/export of the database

@@KeesFluitman there has to be a way in which people can find information stored in a database today. And generally they write SQL to find that information, or have to ask an analyst to write the SQL for them and send them the results. This pipeline just simplifies that process a little bit.

@@kosarajushreya6578 yes, but you’ll need to modify the pipeline code to do this. You’ll need to know how to connect to your DB in python.

so maybe the whole setup if relevant... like is ollama on a linux server? is openwebui on windows or on the same linux in a container etc..?

@@Earthvssuna you can run vLLM as a docker container or k8s deployment. For docker use this doc: docs.vllm.ai/en/latest/serving/deploying_with_docker.html Once the model (typically one from huggingface, like mistral or llama) is running, it’s an OpenAI-compliant endpoint. You can use the OpenAI python client for custom apps, or just add it as an endpoint in open-webui in the admin settings page. If you’re interested I was considering making some more videos describing how to install docker, kubernetes etc on a GPU server?

@@jim02377 I haven’t played with filters, but that concept does exit as a type of pipeline: github.com/open-webui/pipelines/blob/main/examples/filters/detoxify_filter_pipeline.py

Thanks, good to know. When I opened it up in Word, the bottom left says “Kazakh”. Too late now to correct the video

@@azmat8250 I’ve only seen a string work when returning from a pipeline. Even a list throws an error for me. However it’s all open source… if you look at the component during a web search, or with an audio input, it seems like you could create something custom.

@@azmat8250 looked into this and v0.3.30 of open-webui has experimental support for image generation via OpenAI’s api and a few others. It’s not via a pipeline, but still may be worth upgrading and checking out if you haven’t seen it yet.

thanks, @jordannanos . I'm on .30 but it seems like it's not working...at least for me. I'm still toying around with it. If I find something, I'll share here.

@@geovannywing1648 if you’re using docker you’ll also need to run a separate “pipelines” container from the open-webui project, make sure networking is setup correctly between the containers, and then a connection is established between the two containers.

@@geovannywing1648 docs here: docs.openwebui.com/pipelines/

@@firstland_fr yes I don’t see why not. Ollama will work with any model in GGUF format (llama.cpp). And vLLM works with just about any transformers model from huggingface: docs.vllm.ai/en/latest/models/adding_model.html Both ollama and vLLM are tested with this pipeline

@@renatopaschoalim1209 yes, it’s tested with Postgres and MySQL. If you know how connect to SQL server with python, you’ll be able to use the pipeline

@@Mohsin.Siddique llama-index is a python package that is installed via pip, you don’t need an API key. No API keys required for this pipeline

@@RickySupriyadi hi, code is here: github.com/JordanNanos/example-pipelines video reviewing the code: th-cam.com/video/iLVyEgxGbg4/w-d-xo.html

@@jordannanos wow cool, thank you.

@@gilkovary2753 you’ll need to docker exec or kubectl exec into the container called “pipelines” Then run: pip install llama-cloud==0.0.13 llama-index==0.10.65 llama-index-agent-openai==0.2.9 \ llama-index-cli==0.1.13 llama-index-core==0.10.66 llama-index-embeddings-openai==0.1.11 \ llama-index-indices-managed-llama-cloud==0.2.7 llama-index-legacy==0.9.48.post2 \ llama-index-llms-ollama==0.2.2 llama-index-llms-openai==0.1.29 \ llama-index-llms-openai-like==0.1.3 llama-index-multi-modal-llms-openai==0.1.9 \ llama-index-program-openai==0.1.7 llama-index-question-gen-openai==0.1.3 \ llama-index-readers-file==0.1.33 llama-index-readers-llama-parse==0.1.6 \ llama-parse==0.4.9 nltk==3.8.1

Did you configure well pipeline ?

@@dj_hexa_official what do u mean with that ?

@@netixc9733 what error are you seeing? docker logs -f or kubectl logs -f your pipelines container and it may report an error

hi, code is here: github.com/JordanNanos/example-pipelines video reviewing the code: th-cam.com/video/iLVyEgxGbg4/w-d-xo.html

hi, code is here: github.com/JordanNanos/example-pipelines video reviewing the code: th-cam.com/video/iLVyEgxGbg4/w-d-xo.html

hi, I think if you've got a db you should be able to query it. especially if you already know how using python. I posted another video. code is here: github.com/JordanNanos/example-pipelines video reviewing the code: th-cam.com/video/iLVyEgxGbg4/w-d-xo.html

@@jordannanos Thanks a lot Jordan . Super cool

code is here: github.com/JordanNanos/example-pipelines video reviewing the code: th-cam.com/video/iLVyEgxGbg4/w-d-xo.html

Jordan thanks, I have a single gpu runpod setup would you recommend just adding a docker postgresql to existing pod? and is the python code using langchain stored in the pod pipeline settings? this sort of reminds me of AWS serverless Lambda but simpler

@@RedCloudServices if you’d like to save money I would run Postgres in docker on the same VM you’ve already got. That will also simplify networking. Over time you might want to start/stop those services independently in the event of an upgrade to docker or your VM. Or you might want to scale independently. In that case you might want a separate VM for your DB and a separate one for your UI. Or you might consider running kubernetes. Yes the python code is all contained within the pipelines container and uses llama-index not langchain (though you could use langchain too). Just a choice I made.

@@RedCloudServices in other words, you’ll need to pip install the packages that the pipeline depends on, inside the pipelines container. Watch the other video I linked for more detail on how to do this.

@@jordannanos yep! just watched it. I just learned openwebui does not allow Vision only models or multi modal LLMs like Gemini. Was hoping to setup a pipeline using a vision model 🤷‍♂️ also it’s not clear how to edit or setup whatever vector db it’s using

@@martinsmuts2557 it’s a single .py file that is uploaded to the pipelines container. I’ll cover that in more detail in a future video

@@jordannanos Do create this video soon!

@@KunaalNaik @martinsmuts2557 just posted a video reviewing the code: th-cam.com/video/iLVyEgxGbg4/w-d-xo.html repo is here: github.com/JordanNanos/example-pipelines

working on getting it to run on both vLLM + ollama endpoints with llama3.1 + mistral. prompt uses llamaindex for text-to-sql.

similar to this guide: docs.llamaindex.ai/en/stable/examples/index_structs/struct_indices/SQLIndexDemo/

Great job can't wait to see more

@@jvannoyx4 hi, code is here: github.com/JordanNanos/example-pipelines video reviewing the code: th-cam.com/video/iLVyEgxGbg4/w-d-xo.html

AI, apparently. (LLM = Large Language Model)

Why would you need more than that? Be glad for what you already have or you won't find happiness :)

Bitch i have a p40 and im over the moon. Being poor in ml is hard.

Why not podman tho

Yes, working to improve the LLM response and SQL query time