Data Pipelines: Using CDC to Ingest Data into Kafka

แชร์
ฝัง
  • เผยแพร่เมื่อ 5 ก.ย. 2024
  • cnfl.io/data-p... | Using change data capture (CDC), you can stream data from a relational database into Apache Kafka®. This video explains the two types of CDC: log-based and query-based. Follow along as Tim Berglund (Senior Director of Developer Experience, Confluent) covers all of this in detail.
    Use the promo code PIPELINES101 to get $25 of free Confluent Cloud usage: cnfl.io/try-cl...
    Promo code details: cnfl.io/promo-...
    LEARN MORE
    ► No More Silos: Integrating Databases and Apache Kafka: rmoff.dev/no-m...
    ► Kafka Connect Deep Dive - JDBC Source Connector: www.confluent....
    ► Add Key to Data Ingested Through Kafka Connect: kafka-tutorial...
    ► JDBC Source and Sink: www.confluent....
    ► Oracle CDC Source: www.confluent....
    ► Debezium MongoDB CDC Source: www.confluent....
    ► Debezium MySQL CDC Source: www.confluent....
    ► Debezium SQL Server CDC Source: www.confluent....
    ► Debezium PostgreSQL CDC Source: www.confluent....
    ABOUT CONFLUENT
    Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion - designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.
    #kafka #datapipeline #ksqldb #confluent #apachekafka

ความคิดเห็น • 16

  • @raghu7174
    @raghu7174 ปีที่แล้ว +1

    Lovely. This is exactly what I was looking for. Thanks for explaining ithe concepts in such a crisp and brief manner!

  • @andresherrera458
    @andresherrera458 ปีที่แล้ว +4

    loved the "CDC" reference haha

  • @krneki6954
    @krneki6954 11 หลายเดือนก่อน +1

    so query based CDC is really CT (change tracking)?

  • @tauseefjanvekar5201
    @tauseefjanvekar5201 2 หลายเดือนก่อน

    Thanks for this. I was looking for an example on flink cdc. I am trying to write to elasticsearch from kafka in a flink environment

  • @mac.ignacio
    @mac.ignacio 6 หลายเดือนก่อน

    Basically you don't need this if you are using AWS SQS or Event Bridge.

    • @nikhil199029
      @nikhil199029 4 หลายเดือนก่อน

      U mean event bridge that comes with azure?

    • @mac.ignacio
      @mac.ignacio 4 หลายเดือนก่อน

      @@nikhil199029 AWS

  • @ddrsdiego
    @ddrsdiego 2 ปีที่แล้ว +2

    The didactics and diction of the presenter are very difficult but keep going

    • @jester667
      @jester667 2 ปีที่แล้ว +1

      Actually Bare Metallson is the best educator on that "topic" I have watched for a long time. I really enjoy his lectures.

    • @vorandrew
      @vorandrew 2 ปีที่แล้ว +16

      Jesus, Tim's voice is very clear, what did you want? some guy from India doing this talk?

    • @hamidadesokan6528
      @hamidadesokan6528 2 ปีที่แล้ว

      The very best person to teach anything distributed systems. Go watch his intro to distributed system on Oreilly. You will thank me later. Dude is the BEST :) And this one he is killing it. I am not sure whatelse you want here :(

    • @hydtechietalks3607
      @hydtechietalks3607 2 ปีที่แล้ว

      @@vorandrew 😆

    • @andresherrera458
      @andresherrera458 ปีที่แล้ว +1

      I can see what you mean though, his volume and speed fluctuate very quickly, it can throw you off at times