Cloud Composer Best Practices

Best practices for Cloud Composer environments.
Resources referred to in the video
- Cloud Composer Terraform provider: registry.terraform.io/provide...
- Failed DAG runs notifications - setup tutorial: • Cloud Composer - Notif...
- Example (UNSUPPORTED!) cross-environment monitoring dashboard: github.com/filipknapik/compos...
- CI/CD pipeline guide for Composer Airflow DAGs: cloud.google.com/composer/doc...
- Cloud Composer 2 performance and cost tuning guide: cloud.google.com/composer/doc...
- Private Service Connect guide for Composer 2: cloud.google.com/composer/doc...
- Secrets Management integration for Cloud Composer: cloud.google.com/composer/doc...

[@sudhirkenguva4095]

Great, it was really helpful :)

[@Ar001-hb6qn]

I am unable to create a GCP Composer environment. After around 45 minutes it shows the error "Some of the GKE pods failed to become healthy. Please check the GKE logs for details, and retry the operation." I have configured the setting and given the necessary access. I am using composer-2.7.0-airflow-2.7.3. But it failed to create the environment. Can you please help with this? Thanks.

[@hrishikeshgodbole5236]

I am also facing the same error since yesterday...

[@FC8KHD]

Very helpful video indeed! Few follow-up questions... Is the in-place environment upgrade only available via the Cloud Console UI? How do you do in-place environment upgrade if the environment is deployed via Terraform?

[@ChristopherSlattery1980]

This would be great to know

[@estebanr1297]

Hi, very hepfull!! Do we have news about Composer local support?

[@doppalapudiashi5083]

Is it possible to have high availability across 2 regions in one composer environment. I am using terraform to acheive that.