How to auto ingest semi-structured data from Google Cloud Storage using Snowpipe | Snowflake Demo

แชร์
ฝัง
  • เผยแพร่เมื่อ 14 ธ.ค. 2024

ความคิดเห็น • 13

  • @dt0229
    @dt0229 3 ปีที่แล้ว +1

    I have done simiilar configuation setup for AWS, very helpful content to learn from Google side as well. Thank you for sharing!

  • @utkarshdixit3502
    @utkarshdixit3502 3 ปีที่แล้ว +1

    Great job!

  • @manojkashyap6566
    @manojkashyap6566 2 ปีที่แล้ว +2

    I did the same with GCP (With CSV format), pipe is hitting GCP but NOT reading the data. Any ideas?

    • @mastering_snowflake
      @mastering_snowflake  ปีที่แล้ว

      Check the file format is correct

    • @mastering_snowflake
      @mastering_snowflake  ปีที่แล้ว

      And then you can attempt to run the copy into statement manually to aid with troubleshooting before you wrap it in a create pipe statement

    • @uarpand
      @uarpand ปีที่แล้ว

      @@mastering_snowflake I face the same issue while trying out with csv. My question is while creating the pub/sub topic, should I specify -f json or -f none , since my file is a .csv?

  • @jmpresto
    @jmpresto 10 หลายเดือนก่อน

    Can you please create a video how to move data the other direction, from Snowflake table to GCP using service account? For this direction, does the service account have to be created by GCP or created by Snowflake?

    • @mastering_snowflake
      @mastering_snowflake  9 หลายเดือนก่อน

      It’s actually quite different and straightforward to unload data from Google to a GCS bucket using the COPY INTO command

  • @robertsilva9545
    @robertsilva9545 2 ปีที่แล้ว

    That did not work, the command line......

  • @RaviKiran-ox6iw
    @RaviKiran-ox6iw 2 ปีที่แล้ว +1

    For anyone facing bind issue - give monitoring viewer role to gcp_pubsub_service_account

    • @HimanshuSharma-fg2nj
      @HimanshuSharma-fg2nj ปีที่แล้ว +1

      Try going to GCP dashboard -> Add people to this project -> Add SA and select monitoring viewer

    • @aravindakrishna8640
      @aravindakrishna8640 3 หลายเดือนก่อน

      @@HimanshuSharma-fg2nj does this resolve bind issue

    • @aravindakrishna8640
      @aravindakrishna8640 3 หลายเดือนก่อน

      how can we add this role i am unable to by pass this bind issue