Hands on Google Cloud Data Fusion

• Cloud Data Fusion is a fully managed, cloud-native, enterprise data integration service for quickly building and managing data pipelines.
• The Cloud Data Fusion web UI allows you to build scalable data integration solutions to clean, prepare, blend, transfer, and transform data, without having to manage the infrastructure.
• Cloud Data Fusion is powered by the open-source project CDAP.
• Code free, Drag-n-drop tool
• 150+ preconfigured connectors & transformations
• Editions are available
o Developer
o Basic
o Enterprise
Subscribe to Channel: lnkd.in/ehFZbVH5
Follow us in LinkedIn: lnkd.in/gDT3ESdm
Follow us in Instagram: lnkd.in/gZ278ShA
Follow us in Facebook: lnkd.in/gQGF_3Eb
Follow us in Twitter: lnkd.in/gh7dZACW
Join in Telegram channel: lnkd.in/guFt2sAg
Join in WhatsApp group: lnkd.in/gAqkuDPA
Connect with me here:
Instagram: / thearrow0494
LinkedIn: / vignesh-sekar-sujatha-...
🙏🙏🙏🙏🙏🙏🙏🙏
YOU NEED TO DO BELOW THINGS to support my channel
1. LIKE
2. SHARE
&
3. SUBSCRIBE
TO MY TH-cam CHANNEL
#gcpcloud #gcpdataengineer #datafusion

[@joyhodling]

This is very helpful. Could you point me a video or create one on how to upgrade data fusion ? There are pipelines connected. How to do it safely. This video is very helpful.

[@madhavab1533]

Very helpful.Thanks

[@sidmakhaji706]

very nice and detailed video

[@jahnavibandaru3900]

Nice content

[@Abilash1able]

Do you think you can share a tutorial on using Cloud Composer? Thanks for sharing the video. Learning a lot!

[@cloudaianalytics6242]

Sure

[@Abilash1able]

@@cloudaianalytics6242 thank you

[@UddhavParab]

How to send Pipeline Alerts like if the pipeline fails how to send emails?
It's not sending when I am trying to do can you please help.

[@emmanuelihetu9848]

Thank you soo much

[@leenaroja2731]

I have started learning Data fusion and tried the pipeline you mentioned. My question is , can we add a new field(lets say date column with current_date as value to all the existing rows) along with the file fields before loading into target?. Please someone help me to find the way .

[@cloudaianalytics6242]

yes, pipelines built are dynamic. It can have n number of columns and transformations can't be customized.

[@ashwinjoshi3331]

Thanks for the video . I want to import the data from mysql /oracle using cloud data fusion. What value to mention for connection arguments ? I tried to search but not able to get any detail on the same .

[@cloudaianalytics6242]

same as other JDBC/ODBC connection parameters

[@ashishvats1515]

How to perform CDC or incremental data pulling using cloud data fusion pipeline(JDBC to Bigquery)

[@cloudaianalytics6242]

Sure will upload video on the same

[@ashishvats1515]

@@cloudaianalytics6242 please do,i’m waiting for it

[@malyadrich330]

why node from wranger to big query is not connected

[@cloudaianalytics6242]

Error is?

[@DataSpace-yw9pl]

Hope you got it solved later.
I wonder it could be the Bigquery node from source instead sink that you tried to connect

[@shashikiran5015]

Can I get this word document 📃 please share us the link

[@cloudaianalytics6242]

😊

[@shashikiran5015]

Please share the link bro

[@RakeshKhade]

can we do this real time data not in batch

[@cloudaianalytics6242]

Yeah.... But depending on your usecase

[@anilrawat25x]

Can you please provide the documents you showed in videos.

[@cloudaianalytics6242]

Sure...working on it

[@kavana3107]

How Can we write the output into XML file ?

[@cloudaianalytics6242]

I’ll do R&D and let you know on this