Learn Debezium - Change Event Handling Patterns

#Introduction

Consuming CDC events is one thing; modeling them correctly in downstream systems is another. Events with the same op can mean different things depending on the receiving system's state. Episode 18 covers event handling patterns: insert, update, delete, jumbled ordering, duplication, and how to build an up-to-date view from a stream of changes.

The heart of the problem is this: Kafka guarantees ordering per partition, but not at-least-once delivery. That means consumers must be ready for duplicate events and for the same event to be read more than once. Every pattern in this episode is rooted in that reality.

#Modeling Insert, Update, and Delete Downstream

Each op has a different meaning and action:

• c — insert: add a new record.
• u — update: overwrite an existing record.
• d — delete: remove the record, and watch for the tombstone.
• r — read: snapshot result, treat like an upsert.

For deletes, Debezium sends two events: the first with a value containing op: d with before, then a tombstone with a null value on the same key. The tombstone is needed to support topic compaction. Consumers must be ready to receive a null value and not crash when it happens.

docker exec -it kafka /opt/kafka/bin/kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 \
  --topic dbserver1.inventory.customers \
  --property print.value=false

A correct consumer treats op: d as a delete, then ignores the following null tombstone.

#Handling Out-of-Order Events and Idempotent Consumers

Because consumers can read an event more than once, write operations must be idempotent: running the same operation twice gives the same result. Two basic strategies:

• Upsert with primary key: writing with INSERT ... ON CONFLICT ... DO UPDATE so repeated updates are safe.
• Versioning: store source.ts_ms or the log position in the target record and reject older events.

For events arriving out of order across partitions, the source.ts_ms comparison key determines logical order. A JDBC sink with insert.mode: upsert and pk.fields automatically satisfies this idempotency principle.

#Compaction, Deduplication, and Upsert

Log compaction is a Kafka feature that keeps the latest value for each key. Enable it on topics that represent state, not a flow of events:

docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --alter --topic dbserver1.inventory.customers \
  --config cleanup.policy=compact

With cleanup.policy=compact, Kafka keeps the latest value per key and discards older ones as segments are compacted. The tombstones Debezium sends also remove keys during compaction, so the stored state stays accurate.

Combine compaction with the upsert pattern to build a state store that stays in sync with the source database.

#Building Materialized Views and CQRS

A materialized view is a state representation continuously updated from events. Two approaches:

• Streaming aggregation: ksqlDB or Kafka Streams aggregates events into an up-to-date view.
• State store: a consumer applies events to a queryable local state.

Materialized view with ksqlDB

CREATE TABLE customer_state AS
  SELECT id, LATEST_BY_OFFSET(first_name) AS first_name,
               LATEST_BY_OFFSET(email) AS email
  FROM customers_stream
  GROUP BY id
  EMIT CHANGES;

The customer_state table shows the latest state of every customer. This pattern is also the basis for CQRS: the command side writes to the source database, CDC events flow out, and the query side reads from the materialized view — a clean separation of read and write load.

#Handling Snapshot Re-runs and Repeated Events

When an incremental snapshot is re-run (episode 4), consumers receive op: r events again for data that already exists. This is why consumers must treat snapshot events as upserts, not plain inserts. Applying op: r with INSERT ... ON CONFLICT DO UPDATE keeps state consistent without duplication.

Upsert safe for snapshot events

INSERT INTO customers (id, first_name, email)
VALUES (?, ?, ?)
ON CONFLICT (id) DO UPDATE
SET first_name = EXCLUDED.first_name,
    email = EXCLUDED.email;

The ON CONFLICT (id) DO UPDATE query above works for c, u, and r events at once. With one write form, consumers don't need to sort through operation types for write actions.

#Testing Consumer Resilience

Event handling patterns are useless if untested. Recommended exercises:

• Send duplicate events and make sure the final result stays the same.
• Send out-of-order events and make sure versioning rejects older ones.
• Delete a record and make sure the tombstone is processed without errors.
• Trigger a snapshot re-run and make sure there's no duplication in the state store.

Make these scenarios part of the test pipeline so code changes don't break behavior that already works.

#Conclusion

Episode 18 covered proper event handling patterns: mapping op to downstream actions, facing tombstones and duplicates with idempotent consumers, using compaction for state, and building materialized views and CQRS from the change stream.

The key takeaways:

• Insert, update, delete, and read map to different downstream actions.
• A tombstone with a null value accompanies deletes and is needed for compaction.
• Consumers must be idempotent because Kafka delivery is at-least-once.
• Log compaction keeps the latest value per key on state topics.
• Materialized views and CQRS naturally separate write and read paths.

In the next episode, episode 19, we'll discuss operational readiness and runbooks — writing runbooks for connector failures, snapshot restarts, and data replays, incident response for lag and schema issues, and backing up configuration and offsets.