Learn Apache Kafka - Encryption & TLS/SSL

#Introduction

SASL authenticates identity, but data passing by can still be read by anyone sniffing the network. For that, Kafka uses TLS/SSL: an encryption protocol that keeps data confidential and intact in transit between clients and brokers, and between brokers.

Episode 18 walks you through setting up the certificate infrastructure — CA, broker and client certificates, keystores and truststores — configuring TLS on both sides, understanding in-transit and at-rest encryption, and certificate management practices like rotation and expiry monitoring.

#TLS/SSL Setup

#Creating the CA and Certificates

The standard flow: create a Certificate Authority (CA), then issue a certificate for every broker and client signed by that CA. Using keytool, which ships with the JDK:

keytool -genkeypair -alias ca -keyalg RSA -validity 3650 \
  -keystore ca.jks -ext "SAN=dns:kafka-1.internal"
 
keytool -genkeypair -alias kafka-broker -keyalg RSA -validity 825 \
  -keystore broker.jks -ext "SAN=dns:kafka-1.internal"
 
keytool -certreq -alias kafka-broker -keystore broker.jks -file broker.csr
keytool -gencert -alias ca -keystore ca.jks -infile broker.csr -outfile broker.cer
keytool -importcert -alias ca -keystore broker.jks -file ca.cer
keytool -importcert -alias kafka-broker -keystore broker.jks -file broker.cer

keytool -genkeypair creates a key pair, and keytool -gencert signs the broker CSR with the CA. The CA and broker aliases are stored in separate keystores.

#Keystore and Truststore

Two key storage concepts:

• Keystore: stores your own private keys and certificates. Brokers use broker.jks.
• Truststore: stores certificates of trusted parties — here the CA. Brokers and clients use a truststore to verify the counterparty's certificate.

Create a truststore for all nodes by importing the CA:

keytool -importcert -alias ca -keystore truststore.jks -file ca.cer

keytool -importcert -alias ca adds the CA to the truststore. Brokers need a keystore to present their certificate and a truststore to verify clients; clients need a truststore (and optionally a keystore for mutual TLS).

#TLS Configuration

#Broker Side

Enable an SSL listener in server.properties:

listeners=SASL_SSL://:9093
ssl.keystore.location=/etc/kafka/secrets/broker.jks
ssl.keystore.password=${file:/etc/kafka/secrets/pass}
ssl.key.password=${file:/etc/kafka/secrets/pass}
ssl.truststore.location=/etc/kafka/secrets/truststore.jks
ssl.truststore.password=${file:/etc/kafka/secrets/pass}
ssl.client.auth=required

ssl.keystore.location and ssl.truststore.location point to the key storage; passwords are read from files so they aren't written in the configuration. ssl.client.auth=required enables mutual TLS — clients must also present a valid certificate.

#Client Side

Clients verify the broker through a truststore:

security.protocol=SASL_SSL
ssl.truststore.location=/etc/ssl/certs/kafka/truststore.jks
ssl.truststore.password=changeit

security.protocol=SASL_SSL enables encryption (plus SASL if combined). With mutual TLS, clients also add their own ssl.keystore.location.

#Data Encryption

#In Transit

TLS encrypts all communication: client-to-broker and inter-broker. Every byte crossing the network — records, metadata, offsets, control — is secured. This is the basis of all secure production communication. Without TLS, sensitive data can be intercepted on the network path or at a load balancer.

#At Rest

TLS only protects data in motion. For data stored on broker disks, use at-rest encryption:

• Filesystem encryption: LUKS or dm-crypt on the data volume.
• Cloud-native: AWS EBS encryption, Google Persistent Disk encryption, Azure Disk Encryption — enabled once and automatic.
• Application-side encryption for the most sensitive data, with centralized key management (KMS).

At-rest encryption is not a replacement for TLS; they handle different threats.

#Certificate Management

#Certificate Rotation

Certificates have a limited validity period. Periodic rotation prevents the use of leaked or expired certificates. The broker rotation process: issue a new certificate, update the keystore, then perform a rolling restart of brokers one by one. For clients, update the truststore before the new CA becomes valid.

#Expiry Monitoring

The most common TLS failure is a certificate expiring unexpectedly. Automation is key:

• Monitor the expiry dates of all certificates (CA, brokers, clients) and send alerts well before the deadline.
• Use short-lived certificates (for example 30-90 days) renewed automatically by infrastructure like cert-manager or Vault PKI.
• Test the rotation process periodically in staging so there are no surprises in production.

#Closing

In this episode 18 you've understood creating CAs and certificates with keytool, the difference between keystores and truststores, configuring TLS on brokers and clients, in-transit and at-rest data encryption, and certificate management with rotation and monitoring.

The key takeaways:

• TLS protects data in transit; the keystore holds your identity, the truststore holds trusted parties.
• ssl.client.auth=required enables mutual TLS.
• Don't write keystore passwords directly in configuration — use secret files.
• At-rest encryption (EBS, LUKS) handles data on disk.
• Expired certificates are the most common cause of TLS outages.
• Automate certificate rotation and expiry monitoring.

In the next episode 19 we'll discuss quotas and rate limiting — limiting writes and reads per user or client-id. You'll learn quota types, configuration at the user and client-id levels, throttling mechanics, and their use for multi-tenancy and SLAs.