Great video! I was able to do the entire process, thanks for the materials which made it super easy. Additionally, I tried to handle Tombstone events in this pipeline as the Spark job fails due to this. It seems the connector appends a NULL message to the kafka pipeline which makes the spark job give this error (java.lang.NullPointerException) I tried by setting these parameters in the Kafka Connect for postgres, which was mentioned in the documentation: delete.enabled=true transforms.unwrap.drop.tombstones=false schema.evolution=basic But after this too, it's not able to handle tombstone events. Can you suggest what can be done to handle these events? Thanks!
When I use Oracle for source database, which source-class should I use? org.apache.hudi.utilities.sources.debezium.DebeziumSource ? There isn't a Debezium source class for Oracle.
Step by step written guide
www.linkedin.com/pulse/real-time-data-processing-postgres-debezium-kafka-schema-soumil-shah-li19e?
Great video! I was able to do the entire process, thanks for the materials which made it super easy.
Additionally, I tried to handle Tombstone events in this pipeline as the Spark job fails due to this. It seems the connector appends a NULL message to the kafka pipeline which makes the spark job give this error (java.lang.NullPointerException)
I tried by setting these parameters in the Kafka Connect for postgres, which was mentioned in the documentation:
delete.enabled=true
transforms.unwrap.drop.tombstones=false
schema.evolution=basic
But after this too, it's not able to handle tombstone events.
Can you suggest what can be done to handle these events? Thanks!
Thanks sir.. Really good
Thanks mate I appreciate your positive comments
When I use Oracle for source database, which source-class should I use? org.apache.hudi.utilities.sources.debezium.DebeziumSource ?
There isn't a Debezium source class for Oracle.
Hmm you may have to write custom Java code and extend debezium class