Really nice explanation please make more videos on GCP services and some practical project which we can use and learn. Looking for data pipe line processing and cloud data engineer video's
1) But where does the raw data gets stored if you are saying big query is data processing component where one can write queries, do ETL etc......and afterwards where does the processed data gets stored ? 1) Could you mention the names of the storages ? 2) what is the upstream of big query and what is the downstream ?
1.Raw data can be stored in platforms like HDFS, RDBMS, or GCS, serving as the foundation for building data pipelines. 2.For instance, in Google Cloud, GCS is commonly used for raw data storage. 3.Typically, the upstream sources could include traditional data technologies like Teradata or Informatica, while the downstream consumers are often reporting teams.
Really nice explanation please make more videos on GCP services and some practical project which we can use and learn. Looking for data pipe line processing and cloud data engineer video's
nice explanation ... waiting for all videos on GCP 🙂
Is Data engineer and Big Dataengineering same?
Thanks for this
brother your videos are super cool keep doing
1) But where does the raw data gets stored if you are saying big query is data processing component where one can write queries, do ETL etc......and afterwards where does the processed data gets stored ?
1) Could you mention the names of the storages ?
2) what is the upstream of big query and what is the downstream ?
1.Raw data can be stored in platforms like HDFS, RDBMS, or GCS, serving as the foundation for building data pipelines.
2.For instance, in Google Cloud, GCS is commonly used for raw data storage.
3.Typically, the upstream sources could include traditional data technologies like Teradata or Informatica, while the downstream consumers are often reporting teams.
can you post on airflow?