Hi! Thanks for the insightful talk. One question is do we already have something like pgbouncer for connecting cloudrun with cloudsql? Or we need to spin up managed gce for cloud bouncer?
We recommend using a client library that supports connection pools that automatically reconnect broken client connections. This could be SQLAlchemy for Python, Knex for Node.js, HikariCP for Java, etc. For more detailed examples on how to use connection pools, do a web search for "cloudsql postgres manage database connection". Best of luck with your project!
@@TheMomander Thanks for your quick response. I see. So you mean that even if we have multi cloud run instances spinning up each instance will have its own pg-pool seems have some overhead . You don’t think it is necessary to use pg bouncer because even if pg bouncer can manage all cloud run instances connection pool cleverly but it does not worth the effort to provision a stateful machine only for ph bouncer.
@@TheMomander some context for my question is I am worrying about that the cloudsql max connection will throttle my cloud run instances. So I am thinking put a pg bouncer to manage all cloud run instances connection so the total number of connections will not exceed the cloud sql limit. To be specific say I have 10 cloud run instances each have max 200 connections so I can have maximum 2000 connections in full concurrent case. But my cloud sql instance let’s say can only accept 1000 connections max my application will definitely be throttled right? In that case if I put a pgbouncer in front of my cloudsql I will not be throttled by a large chance.
@@yansongguo8354 Thank you for providing the additional background. I would approach this problem by writing the simplest possible code first, which is to use a standard library that gives you one connection pool per Cloud Run instance. Run a realistic load test and see if it causes any trouble. Add complexity, like a separate machine for managing connection pools, only if needed. And remember to set up redundancy so you don't introduce a single point of failure. Otherwise the extra machine may reduce your system's overall availability, not improve it. Also, keep in mind that each Cloud Run instance can handle concurrent requests. You mentioned 10 instances. I am running a system where a single Cloud Run instance handles 20 requests per second, using 20% of its memory and 20% of its CPU. Ten Cloud Run instances would let the system handle at least 200 rps. But every workload is different, so you should load test yours, as mentioned above.
Google PSO recommends NOT using Compute engine default service account to create compute engine instances. The recommend implementation is to use a service account in the project and bind that service account cloud function invoker role at the function level. I can't find any official documentation of how choosing allow internal traffic only would work if the cloud function is in shared VPC service project prj-svc-paas while the compute engine instances for the HR application are in another shared VPC service project prj-svc-iaas while the only VPC is in the shared VPC host project prj-hst-hr. Do I need to create a VPC service control service perimeter and add the 3 projects, cloud function in VPC accessible services? Can Dina request Vinod Ramachandran to comment?
Good question. I think it would be a great idea to see more documentation around shared VPCs in general. I often struggle to find good information around host projects and service projects when terraforming.
Get $300 and start running workloads for free → goo.gle/3lbdtEj
This is a great style of teaching, keep going 👍
Thank you! 😃
I like it how you ask at the beginning of the video then you answer it yourself 4:10
Very nice short tutorials 👍
Thank you!
I needed this two weeks ago 😹
I figured it out with app engine the hard way
Hi! Thanks for the insightful talk.
One question is do we already have something like pgbouncer for connecting cloudrun with cloudsql? Or we need to spin up managed gce for cloud bouncer?
We recommend using a client library that supports connection pools that automatically reconnect broken client connections. This could be SQLAlchemy for Python, Knex for Node.js, HikariCP for Java, etc. For more detailed examples on how to use connection pools, do a web search for "cloudsql postgres manage database connection". Best of luck with your project!
@@TheMomander Thanks for your quick response. I see. So you mean that even if we have multi cloud run instances spinning up each instance will have its own pg-pool seems have some overhead . You don’t think it is necessary to use pg bouncer because even if pg bouncer can manage all cloud run instances connection pool cleverly but it does not worth the effort to provision a stateful machine only for ph bouncer.
@@TheMomander some context for my question is I am worrying about that the cloudsql max connection will throttle my cloud run instances. So I am thinking put a pg bouncer to manage all cloud run instances connection so the total number of connections will not exceed the cloud sql limit.
To be specific say I have 10 cloud run instances each have max 200 connections so I can have maximum
2000 connections in full concurrent case. But my cloud sql instance let’s say can only accept 1000 connections max my application will definitely be throttled right? In that case if I put a pgbouncer in front of my cloudsql I will not be throttled by a large chance.
@@yansongguo8354 Thank you for providing the additional background. I would approach this problem by writing the simplest possible code first, which is to use a standard library that gives you one connection pool per Cloud Run instance. Run a realistic load test and see if it causes any trouble. Add complexity, like a separate machine for managing connection pools, only if needed. And remember to set up redundancy so you don't introduce a single point of failure. Otherwise the extra machine may reduce your system's overall availability, not improve it.
Also, keep in mind that each Cloud Run instance can handle concurrent requests. You mentioned 10 instances. I am running a system where a single Cloud Run instance handles 20 requests per second, using 20% of its memory and 20% of its CPU. Ten Cloud Run instances would let the system handle at least 200 rps. But every workload is different, so you should load test yours, as mentioned above.
@@TheMomander Thanks so much for the insightful response. Issue closed❤️
Any good example on accessing the cloud function from Java using the Google cloud sdk?
How can I authenticate if I call Cloud Functions from my on-prem server? (I already created a Service Account that can access my Cloud Functions)
This setup is not working 😕
Google PSO recommends NOT using Compute engine default service account to create compute engine instances. The recommend implementation is to use a service account in the project and bind that service account cloud function invoker role at the function level. I can't find any official documentation of how choosing allow internal traffic only would work if the cloud function is in shared VPC service project prj-svc-paas while the compute engine instances for the HR application are in another shared VPC service project prj-svc-iaas while the only VPC is in the shared VPC host project prj-hst-hr. Do I need to create a VPC service control service perimeter and add the 3 projects, cloud function in VPC accessible services? Can Dina request Vinod Ramachandran to comment?
Good question. I think it would be a great idea to see more documentation around shared VPCs in general. I often struggle to find good information around host projects and service projects when terraforming.
I have a Nginx with Ubuntu, can I add a subdomain to external IP (ex. 35.355.355.355:2222, or ex. admin.35.355.355.355:2222). Thanks.
Code example are not working from tutorials