วีดีโอ

Thanks for the feedback. That's a great suggestion for another video.

GPT 3 was trained but not trained explicitly on movie trivia.

@@simonwidjaja5025 thanks for the feedback. The name did indeed change. It is now called Agent Builder.

Hi @whitetiger1810 - no, I didn't need to do so. To test the deployed model in the UI I did need to get the test data point formatted exactly right (as shown in the video).

Thanks for the comment. For automating fetching of data from an API on a given schedule, I suggest that you take a look at cloud.google.com/run/docs/execute/jobs-on-schedule. Cloud Run gives you significant flexibility and this topic shows how to schedule Cloud Run jobs.

Hi - if you haven't looked at it already, you might find the accompanying article helpful: markryan-69718.medium.com/rag-hello-world-for-vertex-ai-7d789ba99b31?sk=e62c212738a036d7ed3adafbc82ae1e0

Hi - I ran the pipeline script in Cloud Shell. That is, I just treated Cloud Shell as a Linux instance to run things from. You can upload files one at a time to Cloud Shell from a local system, or clone a repo to get files there. For the training script, it needs to be in the container.

@@markryan2475 Thanks for quick response. In case if we want to schedule a pipeline script, how we can automate that? Please suggest your opinion.

@@AdhvaithG - I didn't go through the process of scheduling the pipeline script. However, if I were to do it, I would start with the procedure described here: cloud.google.com/vertex-ai/docs/pipelines/schedule-pipeline-run

Thanks for your comment. Pinecone isn't mandatory - you could use Vertex AI Vector Search. For an all-Google RAG implementation, this cloud.google.com/vertex-ai/generative-ai/docs/learn/rag-access/rag is currently in private preview.

Hi @eralcala9125 - this may be what you are looking for: cloud.google.com/vertex-ai/generative-ai/docs/multimodal/send-chat-prompts-gemini#gemini-chat-samples-console

Thanks very much for the recommendation. I have created a public playlist for my Vertex AI videos: th-cam.com/play/PLDr6KjrVE4CU7gVu5znuNesTAG6wYU9d_.html&si=LtY8Vx04v0m75FRh

Hi @rafaelmagossi137 - thanks very much for your feedback. I really appreciate it! Good question about tuning vs. grounding. The documentation for tuning cloud.google.com/vertex-ai/generative-ai/docs/models/tune-models states it well - it's designed for specific tasks, and you specify the tuning with JSONL files that give a pattern for input and response that you want the tuned model to follow. By contrast, grounding cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview is for more open-ended use cases where you want queries that may not follow a particular input format to get responses that are grounded in the datastore that you specify. In addition, the data for grounding can be a set of URLs, BQ tables, and content in Cloud Storage. See cloud.google.com/generative-ai-app-builder/docs/create-data-store-es for details.

Hi Julian. Thanks for your comment. Vertex AI Search is aimed at non AI/ML developers so you get a simpler experience with fewer choices and the details (such as how the docs are chunked) are handled behind the scenes. For open ended RAG with Vertex AI, Google just shared a reference architecture cloud.google.com/architecture/rag-capable-gen-ai-app-using-vertex-ai

for example my docker container contains script on how to use this model, so without the script how is it possible to give a set of inputs in json and expect an desired output ? Because for me model files are everything related to model only. The question is because as we see in the video, an IP JSON returns right output.

Hi Amit - thanks for your feedback. First, on the move from Azure to Vertex AI, good move! a) the use case I show in the video is for a model that is trained outside of Vertex AI. In that case, you do need to get the model files into a bucket. If you have been working on the model in Vertex AI already you don't nec. need to do that. For example, you can deploy the new Gemma open model to a Vertex AI endpoint without needing to upload model files to a bucket - see th-cam.com/video/BM3wzKSPOoI/w-d-xo.html b) from the Vertex AI model garden cloud.google.com/vertex-ai/docs/start/explore-models you can one-click deploy a variety of LLMs, including Gemma, Llama 2. Just click on the model card in the model garden and click on the Deploy button. The model will be deployed to an endpoint. For your container question, check out this article: medium.com/the-modern-scientist/training-a-custom-model-in-a-pre-built-vertex-ai-container-8a34e244db53?sk=5486fc72b783977e03f8215339d58596. It describes how to take the same model that I show how to I trained the model described in this video by adapting a Vertex AI training container. That article also describes how you can specify the location of the training dataset and parameters for training so that the training process that is running in the container can get at them. I hope that this helps and I hope that you enjoy working with Vertex AI.

Thanks a ton@@markryan2475 ! I will go through the articles you have linked. I have a question though, because I am working with huggingface models post fine-tuning the model files are stored in .safetensors format. How do we go about this ? Because acceptable model weights in VertexAI is .pb, .pt etc... and that is not the case with my model files. In my case, I have trained all my models outside GCP. I think I can still use my docker code the way it is for the moment except for the fact that the endpoint URL is a GCP container URL. But to do that, I need to have the model registered in GCP which is a blocker at the moment for the aforementioned reason.

Hi @@AmitKumar-hm4gx - I haven't imported models in .safetensors format yet. However, this example seems to show something that is related to what you want to do, so it might be useful for you: github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_stable_diffusion_xl_lcm.ipynb

hey@@markryan2475 thanks for replying, so I went back to the following steps: - Build Docker image (download model inside the docker) - Push the image to GAR - Upload the GAR to Model Registry so now we have model created. - Deploy the endpoiint (my inputs are formatted as it is expected here) - the problem is in my output I don't get predictions, I get other deployment details like model id etc.. but prediction keys are missing and I don't see anything in log that will help me understand this. Have you faced or are aware of such issues ?

Hi - thanks for your comment. You can find documentation on chat models in Vertex AI here cloud.google.com/vertex-ai/docs/generative-ai/chat/test-chat-prompts and, for Gemini, here: cloud.google.com/vertex-ai/docs/generative-ai/multimodal/send-chat-prompts-gemini

Thanks very much!

For a datastore that uses website search, you need turn on Enterprise: cloud.google.com/generative-ai-app-builder/docs/enterprise-edition

Thanks for your comment. It cost me about $30 CDN a day while it was running.

@@markryan2475 Thank you for your reply.

Hi - thanks for your comment. Right now, text-bison and chat-bison are supported: cloud.google.com/vertex-ai/docs/generative-ai/grounding/ground-language-models#before_you_begin

Hi - thanks for your comment. You need to replace "project-id" with the ID of the project that you are using and replace "datastor_id" with the datastore ID shown in Vertex AI Search for your datastore. You can see more details in this Vertex AI documentation topic: cloud.google.com/vertex-ai/docs/generative-ai/grounding/ground-language-models

Hi and thanks for your comment. I don't know for sure whether the request and response are saved. I believe that is up to the application.

Hi - thanks for your comment. I haven't seen an error like the one you reported before. I suggest saving a simple model and verifying that you can use that as a starting point.

thanks for your comment. I just uploaded the whole directory structure of the saved model to Cloud storage. See the Import the model to Vertex AI section here for details: medium.com/@markryan-69718/deploy-keras-models-with-vertex-ai-32e5b6d59f4f?sk=f36754de23fccdb3fa8a06b43c1271a9. I haven't worked through your particular use case (deploying a model from hugging face on Vertex AI), but this blog looks like it would be helpful for that use case: blog.devops.dev/tuning-and-deploying-huggingface-transformers-with-vertex-ai-part-3-start-distributed-tuning-and-e0943bfc9d4b

Thanks for your comment. There are a couple of choices. The simplest one is to incorporate the data processing into the model (examples shown here github.com/ryanmark1867/deep_learning_best_practices/blob/master/notebooks/model_training_keras_preprocessing.ipynb - see Keras preprocessing layers) so that the preprocessing is wrapped into the model and you don't need to explicitly worry about it at inference time. Alternately, you can create custom classes using the sklearn pipeline features and then call the pipeline in the Flask module to do the preprocessing there (example here: github.com/ryanmark1867/deep_learning_for_structured_data/blob/master/deploy_web/flask_server.py). It took a lot of fiddling around to get the second method to work and using the Keras preprocessing layers is easier and more efficient. See this article for more details: medium.com/the-modern-scientist/a-better-way-to-deploy-keras-models-a9d5764de964?sk=7ee732fad6eb2310fb0094cc44bff4bd

The blog just covers running it in Vertex AI on Google Cloud.

Yes, you can make the endpoint public. Some more details here: cloud.google.com/vertex-ai/docs/general/deployment

Thank you

Hi - I did some experiments going from Python to COBOL (described here towardsdatascience.com/back-to-the-future-with-codex-and-cobol-766782f5ae8f?sk=a31c8847df5dd5937547cb03fc6b192f). The results weren't great. I assume that there is more interest in going from COBOL to a modern language, rather than the other way around, so I didn't look into it more deeply. It may be worth trying JavaScript to COBOL since COBOL to JavaScript worked better than COBOL to Python.

Thanks for the comment. I'm glad you found the video helpful.

you need to install dependencies in case of yolo8, it will require custom docker image

Hi - thanks for the question. I assume you're referring to github.com/sokrypton/ColabFold. One of the goals of the blog I mention in the video is to show an integration with Vertex AI. I'll check with the authors of the blog whether there are any plans to update it.