Learn Vitess - Control Plane & Topology Management

#Introduction

The Topology Service is Vitess's nervous system — every routing decision depends on the metadata it stores. Episode 18 digs deep into this control layer: choosing the right backend, keeping its data healthy, repairing it when corrupted, and automating operations through vtctld.

Episode 18 roadmap: choosing a Topology Service, day-to-day topology operations, topology repair and backup, then the vtctld UI and automation. By the end, you'll manage the "brain" of a Vitess cluster confidently.

#Topology Service: etcd, ZooKeeper, Consul

Vitess supports three backends for the Topology Service:

• etcd — the default and most common choice. Known from the Kubernetes ecosystem, stable for small to large clusters. The Vitess Helm chart default.
• ZooKeeper — Vitess's historical backend, used since the YouTube days. Stable and battle-tested, but heavier to set up.
• Consul — another option with built-in service discovery features.

The choice is rarely a purely performance decision; what matters more is team skill and the existing ecosystem. For labs and most modern production, etcd is the sensible choice.

Checking the topology endpoint in use:

vtctlclient GetTopologyInfo

vtctlclient GetTopologyInfo displays the global topology and registered cells — a view of the cluster map from the control plane side.

#Day-to-day Topology Operations

The topology operations most often done via vtctlclient:

vtctlclient ListAllKeyspaces
vtctlclient ListAllTablets
vtctlclient ListShardHealth
vtctlclient GetKeyspace commerce
vtctlclient GetShard commerce/0

vtctlclient GetShard shows one shard's details: primary tablet, replicas, and its configuration. These operations are read-only and safe to run anytime.

Commands that change topology state — CreateShard, DeleteShard, InitShardMaster — must be used carefully and ideally through planned workflows, not during production.

#Topology Repair and Backup

Topology can get corrupted from storage failures, bugs, or human error. Symptoms: "missing" tablets, inconsistent keyspaces, or VTGate unable to find shards. The main tools:

• vtctlclient with the right flags to fix incorrect entries.
• DeleteTablet to remove tablets that no longer exist.
• Reparenting to restore role consistency after a failure.

vtctlclient DeleteTablet -allow_master <tablet-alias>

vtctlclient DeleteTablet removes a tablet from the topology. Use -allow_master only if you're sure the tablet must indeed be removed — for example, its physical data is already gone.

Topology backup is crucial: if topology data is lost, the cluster loses its map — worse than losing business data because nothing can route. etcd has its own snapshot:

ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-snapshot.db \
  --endpoints=http://etcd:2379

etcdctl snapshot save saves an etcd snapshot. In Helm-managed Vitess, etcd is usually deployed as a StatefulSet — make sure snapshots are taken regularly and their restores tested.

#vtctld UI and Automation

vtctld isn't only an API; it also has a web UI that visually displays cluster state: keyspaces, shards, tablets, and health. Access the UI via port-forward:

kubectl port-forward -n vitess svc/vtctld 15000:15000 &
open http://localhost:15000

kubectl port-forward ... svc/vtctld opens the vtctld UI on port 15000. From here you can view the cluster map and navigate between shards without typing commands.

For automation, everything the UI can do is also available through the API and vtctlclient. Common automations:

• Scheduled health checks via scripts calling ListShardHealth.
• Alerting that triggers automated repair commands.
• CI/CD pipelines calling vtctlclient to apply schema and topology (episode 19).

vtctlclient ListShardHealth -format json | \
  grep -E '"(Master|Replica)"|"State"' | head -20

The vtctlclient ListShardHealth -format json command produces JSON output that automation can parse — a common way to extract status for monitoring.

#Closing

In this episode 18 you understood the Vitess control plane: choosing a Topology Service between etcd, ZooKeeper, and Consul, running day-to-day topology operations, repairing and backing up the topology, and using the vtctld UI and automation to manage cluster state.

Key takeaways:

• etcd is the sensible default; ZooKeeper and Consul are alternatives.
• Topology is small in size but critical in availability.
• Topology read operations are safe; write operations must be planned.
• Topology backup is mandatory and its restore must be tested, as seriously as data backups.
• The vtctld UI gives a useful visual view of the cluster for navigation.
• Automation via vtctlclient JSON output enables scheduled health checks.

In the next episode, episode 19, we streamline the delivery flow: CI/CD and release management — GitOps for Vitess configuration, CI/CD pipelines, versioning schema changes, rolling upgrades, and testing in staging. See you there!