Hi ... I wanted to take a moment to express my appreciation for your videos. They have been incredibly helpful as I learn about GCP, and your clear explanations make complex topics much more accessible. One question here, here every thing we are doing it manually using GUI and alternatively can we do the same process using Python SDK right from building package and pushing the package to Cloud Bucket and then training setup, Model Registry, Deployment and end point creation? If yes, when you get a time can you please post a video on this?
Thank you for the kind words! You are right, majority of the things can be done also by using Python SDK or gcloud CLI tool. I always try to focus first on the GUI approach as once you learn it it’s much easier to automate entire process with code. Unfortunately due to lot of project work on my end recently, I am not planning any new videos on that topic in the nearest future. I might come back to it, but I don’t know when it might happen. Hope that this video will be sufficient starting point for you!
Great tutorial, I have a question, is it optimal or good practice to initiate training job from cloud run? If so, can you add short description on how you'd do it? I'm trying to automate custom container training on vertex ai through cloud workflow service where I can invoke cloud run job since there is no direct support of workflow support to vertex ai.
I am not sure if that is the best practice to initiate it from this service, it depends on a lot of things. If that is the service that you would like to use maybe you can use this together with Cloud Scheduler and on a defined schedule Cloud Scheduler would trigger the Cloud Run container with a training job. This sounds like a simple yet correct architecture to me :)
i followed your steps, but when i try to run, i get an error: Unable to start training due to the following error: Required field is not set. Where possibly could i be wrong? I tried various factors but nothing worked (I am training an image classification model)
Hey, are you using my code for training from the task.py file or your own? Also, what is the command that is raising this error? There might be a possibility that Google has changed the mandatory fields for some API calls, please give me some more details and possibly I will be able to figure out what is going on :)
@@cloud4datascience772 Thank you for your reply. I am using my own file for this and i am training a image processing model . My script runs well on local machine, but while training it on GCP, it fails giving the above error. I have input all the parameters as mentioned in your video, I followed the steps but still I get an error. It does not provide any specific field because of which the error is generating, it just says, field not set
I tried to use the created model in batch / online prediction. I get "Prediction failed: Exception during sklearn prediction: 'DecisionTreeClassifier' object has no attribute 'n_features_'" or "Prediction failed: Exception during sklearn prediction: float() argument must be a string or a number, not 'dict'" Pls guide how to test this model with online /batch prediction . what should be the format of test data
Hi, I think it could be a good idea to deploy your model to the local endpoint and test it there. There is a very helpful section on that: cloud.google.com/vertex-ai/docs/predictions/get-predictions#test_your_model_locally - On this page, you can also find more details regarding the input data requirements.
excellent tutorial! Do you have a video on pipeline with vertex AI?
this is very helpful, thank you
Hi ... I wanted to take a moment to express my appreciation for your videos. They have been incredibly helpful as I learn about GCP, and your clear explanations make complex topics much more accessible.
One question here, here every thing we are doing it manually using GUI and alternatively can we do the same process using Python SDK right from building package and pushing the package to Cloud Bucket and then training setup, Model Registry, Deployment and end point creation? If yes, when you get a time can you please post a video on this?
Thank you for the kind words! You are right, majority of the things can be done also by using Python SDK or gcloud CLI tool. I always try to focus first on the GUI approach as once you learn it it’s much easier to automate entire process with code. Unfortunately due to lot of project work on my end recently, I am not planning any new videos on that topic in the nearest future. I might come back to it, but I don’t know when it might happen. Hope that this video will be sufficient starting point for you!
Excellent tutorial , very easy to follow up 😀
Thank you! I am super happy that you found this tutorial easy to follow ;)
Great tutorial, I have a question, is it optimal or good practice to initiate training job from cloud run? If so, can you add short description on how you'd do it? I'm trying to automate custom container training on vertex ai through cloud workflow service where I can invoke cloud run job since there is no direct support of workflow support to vertex ai.
I am not sure if that is the best practice to initiate it from this service, it depends on a lot of things. If that is the service that you would like to use maybe you can use this together with Cloud Scheduler and on a defined schedule Cloud Scheduler would trigger the Cloud Run container with a training job. This sounds like a simple yet correct architecture to me :)
Thank you, that was great help.@@cloud4datascience772
great video
Thank you!
i followed your steps, but when i try to run, i get an error: Unable to start training due to the following error: Required field is not set. Where possibly could i be wrong? I tried various factors but nothing worked
(I am training an image classification model)
Hey, are you using my code for training from the task.py file or your own? Also, what is the command that is raising this error? There might be a possibility that Google has changed the mandatory fields for some API calls, please give me some more details and possibly I will be able to figure out what is going on :)
@@cloud4datascience772 Thank you for your reply. I am using my own file for this and i am training a image processing model . My script runs well on local machine, but while training it on GCP, it fails giving the above error. I have input all the parameters as mentioned in your video, I followed the steps but still I get an error. It does not provide any specific field because of which the error is generating, it just says, field not set
I tried to use the created model in batch / online prediction.
I get "Prediction failed: Exception during sklearn prediction: 'DecisionTreeClassifier' object has no attribute 'n_features_'" or
"Prediction failed: Exception during sklearn prediction: float() argument must be a string or a number, not 'dict'"
Pls guide how to test this model with online /batch prediction . what should be the format of test data
Hi, I think it could be a good idea to deploy your model to the local endpoint and test it there. There is a very helpful section on that: cloud.google.com/vertex-ai/docs/predictions/get-predictions#test_your_model_locally - On this page, you can also find more details regarding the input data requirements.
I have the same problem, have you managed to solve it?