Data Pipelines: Using CDC to Ingest Data into Kafka
ฝัง
- เผยแพร่เมื่อ 5 ก.ย. 2024
- cnfl.io/data-p... | Using change data capture (CDC), you can stream data from a relational database into Apache Kafka®. This video explains the two types of CDC: log-based and query-based. Follow along as Tim Berglund (Senior Director of Developer Experience, Confluent) covers all of this in detail.
Use the promo code PIPELINES101 to get $25 of free Confluent Cloud usage: cnfl.io/try-cl...
Promo code details: cnfl.io/promo-...
LEARN MORE
► No More Silos: Integrating Databases and Apache Kafka: rmoff.dev/no-m...
► Kafka Connect Deep Dive - JDBC Source Connector: www.confluent....
► Add Key to Data Ingested Through Kafka Connect: kafka-tutorial...
► JDBC Source and Sink: www.confluent....
► Oracle CDC Source: www.confluent....
► Debezium MongoDB CDC Source: www.confluent....
► Debezium MySQL CDC Source: www.confluent....
► Debezium SQL Server CDC Source: www.confluent....
► Debezium PostgreSQL CDC Source: www.confluent....
ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion - designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.
#kafka #datapipeline #ksqldb #confluent #apachekafka
Lovely. This is exactly what I was looking for. Thanks for explaining ithe concepts in such a crisp and brief manner!
loved the "CDC" reference haha
so query based CDC is really CT (change tracking)?
Thanks for this. I was looking for an example on flink cdc. I am trying to write to elasticsearch from kafka in a flink environment
Basically you don't need this if you are using AWS SQS or Event Bridge.
U mean event bridge that comes with azure?
@@nikhil199029 AWS
The didactics and diction of the presenter are very difficult but keep going
Actually Bare Metallson is the best educator on that "topic" I have watched for a long time. I really enjoy his lectures.
Jesus, Tim's voice is very clear, what did you want? some guy from India doing this talk?
The very best person to teach anything distributed systems. Go watch his intro to distributed system on Oreilly. You will thank me later. Dude is the BEST :) And this one he is killing it. I am not sure whatelse you want here :(
@@vorandrew 😆
I can see what you mean though, his volume and speed fluctuate very quickly, it can throw you off at times