GCP Cloud Logging Introduction

GCP Cloud Logging Introduction

[@user-vf1xg9gz2f]

I've used logger module in my pyspark file to capture the logs in a file. I ran the job with this python script, but I can't see any log files in my dataproc environment or even in GCP local disk even after successful completion of the job.

[@duskbbd]

There is one use case, how to get dataflow job status by pubsub with stack driver, in production run