ETL Batch pipeline with Cloud Storage, Scala Dataproc Serverless, BigQuery orchestrated by Airflow

This video shows a complete use case with an ETL Batch Pipeline deployed on Google Cloud.
This pipeline stages are :
- Extract : Cloud Storage
- Transform : Spark Scala and Dataproc Serverless
- Load : BigQuery
Everything is orchestrated by Cloud Composer and Apache Airflow.
The CI CD part is done with Cloud Build.
#googlecloud #Airflow #CloudComposer #ETL #CloudStorage #BigQuery#Spark #Scala #Dataproc #Serverless
▸ Github : github.com/tos...
▸ X : x.com/MazlumTo...
Feel free to subscribe to the channel and click on the bell 🔔 to receive notifications for the next videos.
📲 Follow me on social networks :
▸ Articles : / mazlum.tosun
▸ X : / mazlumtosun3
▸ LinkedIn : / mazlum-tosun-900b1812
▸ WhatsApp : whatsapp.com/c...

[@GCPLearning-ce9bg]

🔔 Subscribe to the channel to learn Google Cloud and click on the bell to receive notifications → bit.ly/gcp-learning-mazlum-gb