GPT-4V Alternative (Self-Hosted): Deploy CogVLM on AWS

Deploy CogVLM, a powerful GPT-4V alternative, on AWS with this step-by-step technical guide. Learn how to set up and run a self-hosted AI model, gaining independence from standard APIs and enhancing your computer vision capabilities.
Chapters:
- 00:00 Intro
- 00:40 Introduction to CogVLM
- 01:43 Setting Up the AWS Infrastructure
- 03:56 Configuring the Inference Server
- 05:41 Running Inference and Testing the Model
- 09:08 Outro
Resources:
- Roboflow: roboflow.com
- Roboflow Universe: universe.roboflow.com
- How to Deploy CogVLM on AWS blog post: blog.roboflow.com/how-to-depl...
- GPT-4 Vision Alternatives blog post: blog.roboflow.com/gpt-4-visio...
- Inference Server code: github.com/roboflow/inference
- CogVLM Client code: github.com/roboflow/cog-vlm-c...
- CogVLM: Visual Expert for Pretrained Language Models arXiv paper: arxiv.org/abs/2311.03079
- CogVLM code: github.com/THUDM/CogVLM
- Multimodal Maestro GitHub: github.com/roboflow/multimoda...
- Multimodal Maestro: Advanced LMM Prompting blog post: blog.roboflow.com/multimodal-...
Remember to like, comment, and subscribe for more content on AI, computer vision, and the latest technological breakthroughs! 🚀
Stay updated with the projects I'm working on at github.com/roboflow and github.com/SkalskiP! ⭐

[@abdellatifBELMADY]

Great job, thank you 😉

[@Roboflow]

Thanks a lot!

[@cyberhard]

Excellent as usual! BTW, nice hat.

[@Roboflow]

Thanks! It’s been a while since my last video. I’m a bit rusty.

[@cyberhard]

@@Roboflow seems like you edited the rust out. 😉

[@Roboflow]

@@cyberhard hah, what do you mean?

[@body1024]

amazing as always. can you make another video on how to use this on google colab ?

[@tomaszbazelczuk4987]

Awesome!!!

[@SkalskiP]

Thank you!

[@romroc627]

As always your video are very helpful and clear. I use a VM too for object detection inference in cloud. I still didn't find a good serverless solution to run inference. Maybe one of the next videos could be to run inference with serverless architecture, with or without Gpu. Thanks

[@SkalskiP]

Could you be a bt more specific? What you are looking for? Server that you could youse for deployment? Contenerization? Terraform?

[@romroc627]

@@SkalskiP i need to deploy my object detection trained model for inference. Currently I have a VM in cloud to do that. Using a VM has some disadvantages: I pay even when it is idle, I need to mantain it, and so on. I tried to use a serverless solution to run inference, like Aws Lambda or Azure Functions, but they have limitations, and they don't run on gpu.

[@william-faria]

Hello from São Paulo, Brazil! Thank you for your help and time. I have a question: Is it possible to train this model with another language, like Brazilian Portuguese? If yes, how can I do that?

[@filipemartins1721]

Is there any way to use FastAPI with this solution? Instead of using the UX provided I would like to send a API call. Any ideas?

[@eliaweiss1]

The 'inference server start' command always start a new container, while the old one stays on disk, this clog the disk and take long start up time
How can I make the inference use the previous container?

[@user-yw6wf3uu1o]

It would be nice to also show running cogvlm in live cam mode.

[@Roboflow]

We will probably make video like this with next big multimodal LLM.

[@akhileshsharma5067]

@Robloflow I made a project in roboflow and annotated 300 images but I only want to use 100 images for dataset generation. how to do that? There is no option to select number of images for dataset generation.

[@gexahedrop8923]

is it possible to run it on T4 with transformers library?

[@mohamednayeem2602]

Can you make a video to fine tune cog vlm

[@jimshtepa5423]

why did. you use roboflow? what function does it have? what would you do otherwise if roboflow was not available?

[@Roboflow]

CogVLM is one of the models available in Inference Server. I used it because it is free and required only 2 commands to deploy. All you need is RF API KEY, and you can generate it with free tier account.

[@Roboflow]

As to what other options do you have, you would probably need to wrap the model in TorchServe.

[@jimshtepa5423]

thank you. don't get me wrong, I was not criticizing. I just didn't understand the role of roboflow. Just to clarify, what is the purpose of api key when an ml model is deployed? compute resources are provided by aws, source code of the model is available on hf, what was the purpose for roboflow? what does it do?@@Roboflow

[@user-yw6wf3uu1o]

Do I have to utilize AWS or can I do it on my own server?

[@Roboflow]

You can run it on your own server!

[@user-hc5os4fs5k]

make a video on finetuning cog-vlm and llava also

[@Roboflow]

Cool idea. I’m scared to even think how much compute you need to find tune this model.

[@user-hc5os4fs5k]

@@Roboflow yes i understand that the required compute will be too high, but still we can get some insight about fine-tuning vision-llm's , as currently there is very less to No articles or videos are available for this. Hoping that you will come up with some video or article on this 👍👍👍

[@mohamednayeem2602]

Is any update on fine tuning it.. I did fine tune lava but not sure how do I do it for cogvlm. Can you help me if you have any resources??

[@slider0507]

How much did this cost on aws? 🤔

[@Roboflow]

It is around $0.50 per hour

[@Designer598]

I am professional thumbnail Designer

[@Roboflow]

Please reach out to me on Twitter: twitter.com/skalskip92

[@Designer598]

@@Roboflow send email address

[@eliaweiss1]

Amazon Machine Image (AMI)
Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.0.1 (Amazon Linux 2) 20231219

[@Roboflow]

Thanks!