Debezium Does Not Replicate the Source As-Is: Three Ways Data Can Drift Quietly
In the company tech blog, I previously wrote about redesigning our CDC pipeline with Debezium and Flink, and later about moving our replication model toward CDC-based incremental processing. The structure was simple: Debezium captured changes from MySQL, Kafka carried the events, and Flink replicated them into S3 and Iceberg tables. I will skip the architecture and the reasons behind that design here. This post is about what happened after that. The issues we ran into while operating Debezium fell roughly into two groups. One was data quietly drifting from what we expected. The other was connector behavior that did not match our operational assumptions. In this post, I will focus on the first group: cases where the data itself ended up different. ...