Did you find this video helpful? We’d love to hear from you in the comments! Be sure to subscribe to GCT for more guides like this 😎 → goo.gle/GoogleCloudTech
It's a well produced piece of content. Still, a youtuber doing a real project step-by-step is much more educational than this. You should create more content with real example, step by step. Thank you.
Great video summarising the tech stacks used for a data pipeline. When it comes to using dataflow with Python SDK, we face so many challenges especially connecting to RDBMS sources. The expansion is implemented in Java. It would be helpful to see a Dataflow video with Python SDK connecting to RDBMS.
Thanks for your efforts. Really great video. But I found the music distracting. If there must be music please kindly consider putting it at a volume that doesn't compete with what is being said.
The 1st important step is to bring data into Google cloud ! example if I have BQ Views as sources then how do I import these directly to my BQ DWH and then use Data flow to process and export back to BQ Tables is important. Also just wondering if we have Cloud Data Flow to pickup the file from Cloud Storage bucket and load the data into BQ table ,why do we need Cloud Composer ? Is this for scheduling when to pickup the file from Cloud Storage ?? cant this be done by Cloud Data Flow .. lets find out
Thanks for your questions! Regarding BQ -> Dataflow -> BQ pipelines: many times you can do processing directly in BigQuery using SQL and UDFs, though if you have a use case that isn't covered, you can write your own Dataflow job to do this as well. Regarding the need for Cloud Composer: in this example, you could likely use the scheduling function of Dataflow, however it is simply a scheduler and if you have other dependencies that Dataflow can't handle, you would need to use Composer.
Did you find this video helpful? We’d love to hear from you in the comments! Be sure to subscribe to GCT for more guides like this 😎 → goo.gle/GoogleCloudTech
It's a well produced piece of content. Still, a youtuber doing a real project step-by-step is much more educational than this. You should create more content with real example, step by step. Thank you.
Great video summarising the tech stacks used for a data pipeline. When it comes to using dataflow with Python SDK, we face so many challenges especially connecting to RDBMS sources. The expansion is implemented in Java. It would be helpful to see a Dataflow video with Python SDK connecting to RDBMS.
I Agree it not user-friendly and difficult to manage. java is better in dataflow. or to use Dataporc
Thanks for your efforts. Really great video. But I found the music distracting. If there must be music please kindly consider putting it at a volume that doesn't compete with what is being said.
I want to migrate DataStage jobs to DataFlow and I was looking for some more details exactly on DataFlow part, which was omitted in this video.
A very useful sharing ! Really appreciate your effectiveness and effort.
The 1st important step is to bring data into Google cloud ! example if I have BQ Views as sources then how do I import these directly to my BQ DWH and then use Data flow to process and export back to BQ Tables is important. Also just wondering if we have Cloud Data Flow to pickup the file from Cloud Storage bucket and load the data into BQ table ,why do we need Cloud Composer ? Is this for scheduling when to pickup the file from Cloud Storage ?? cant this be done by Cloud Data Flow .. lets find out
Thanks for your questions! Regarding BQ -> Dataflow -> BQ pipelines: many times you can do processing directly in BigQuery using SQL and UDFs, though if you have a use case that isn't covered, you can write your own Dataflow job to do this as well. Regarding the need for Cloud Composer: in this example, you could likely use the scheduling function of Dataflow, however it is simply a scheduler and if you have other dependencies that Dataflow can't handle, you would need to use Composer.
This channel is creating great content regarding the services of GCP thanks for that.
thanks for sharing this video, if you add any Qwiklab links related to this concepts then i will helpful for learners to check and try it.
Great content! Many thanks for sharing
Good video but the music is distracting.
Thanks! How abt another video for ELT?
Συγνώμη κυριά αλλά δεν πρόλαβα να γράψω πολλά comments στην εργασία ς
HELP
Looks good
Panda. Bike