Learn Debezium - Auditability & Data Observability

#Introduction

If you're asked "who changed this row, when, and what did the value become", can your pipeline answer? Episode 13 covers auditability — the ability to trace and reconstruct a change history — and data observability — the ability to see the health of the data flowing through.

The good news is that Debezium already provides the raw material: every change is stored as an append-only event with before and after values. Your job is to build processes that use that material for auditing, reconstruction, and quality control.

#Tracing Database Changes and Reconstructing the Event Stream

Because events are append-only, the full history can be reconstructed at any time. To trace a specific row, use the row key and read its entire history:

docker exec -it kafka /opt/kafka/bin/kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 \
  --topic dbserver1.inventory.customers \
  --from-beginning \
  --property print.key=true \
  --property print.partition=true

Because the primary key is the message key, all changes to the same row live in one partition and in order. From that sequence you can reconstruct the row's value at any point in time — a form of time travel a plain snapshot doesn't have.

#Audit Trails for CRUD and Schema Change Operations

A good audit trail records not just the data, but the operation and its context. Debezium events already carry both: op gives the operation type, source gives the origin, and before/after give the old and new values.

Add transaction time schema and user metadata if your database supports it:

{
  "provide.transaction.metadata": "true",
  "include.schema.changes": "true"
}

With provide.transaction.metadata: "true", Debezium publishes transaction events linking several operations within one database transaction. Schema change events from include.schema.changes complete the trail with a table structure history — who added a column and when.

#Observability with Metrics, Logs, and Tracing

Observability combines three signals: metrics, logs, and traces.

• Metrics: expose Debezium's JMX to Prometheus to monitor throughput, lag, and remaining snapshot rows.
• Logs: structure worker logs in JSON format so they're easy to search in log aggregation.
• Tracing: propagate a trace context from the database to consumers so the journey of one event can be traced end to end.

An example of structured worker log properties:

LOG_LEVEL: INFO

For tracing, connect the worker to the OpenTelemetry agent and export spans to an observability backend. That way, when an event is slow to process, you can see where the time went — in the snapshot, in network transfer, or in the consumer.

#Data Quality Checks and Validation Pipelines

Incoming data must be validated before it's considered safe. Build a quality-check pipeline that consumes CDC events and verifies:

• Completeness: required fields exist and aren't null.
• Type consistency: values match the schema type, for example an email must be valid.
• Monotonicity: ts_ms and source position always increase, detecting duplicate or out-of-order events.
• Coverage: no table silently loses events.

An example of a simple quality rule with ksqlDB:

Detecting malformed events

CREATE STREAM bad_events AS
  SELECT * FROM customers_stream
  WHERE email IS NULL OR email LIKE '%example.invalid%'
  EMIT CHANGES;

The bad_events stream above holds events that violate the rules, then they can be directed to a DLQ or alert. Regular data quality checks detect problems before they spread to downstream consumers.

#Separating Audit Topics from Operational Topics

To keep the audit trail separate from high-volume operational traffic, many teams route audited events to dedicated topics. This can be done with the routing transform from episode 6, so events going to the audit topic stay complete while the original topic is used for normal synchronization.

{
  "transforms": "auditRoute",
  "transforms.auditRoute.type": "org.apache.kafka.connect.transforms.RegexRouter",
  "transforms.auditRoute.regex": "(.*)",
  "transforms.auditRoute.replacement": "$1-audit"
}

With the $1-audit pattern, every original topic gets an audit topic partner. These partners can be kept with long retention or synced to object storage as a compliance archive.

#Metrics You Must Monitor

Some of the most important Debezium metrics for observability:

• Lag: the difference between the streaming position and the latest position in the database log.
• Snapshot progress: the number of rows read and still to be read.
• Record count: events per second per connector.
• Error rate: the number of failed events entering the DLQ.

Combine these metrics with logs and traces to get the full picture of one event's journey from database to consumer.

#Conclusion

Episode 13 turns the pipeline into an accountable system: change history can be reconstructed from append-only events, the audit trail covers CRUD and schema changes, observability monitors health from three signals, and quality checks filter data before it spreads.

The key takeaways:

• Append-only events with a message key enable per-row history reconstruction.
• op and before/after are the raw material of a complete audit trail.
• provide.transaction.metadata links operations within one transaction.
• Combine metrics, logs, and traces for end-to-end observability.
• Validate completeness, types, monotonicity, and coverage in the quality pipeline.

In the next episode, episode 14, we'll discuss cross-region and hybrid topologies — replicating data across regions with Debezium, hybrid cloud and hybrid database architectures, latency and network topology considerations, and data sovereignty.