Learn Vitess - Core Concepts & Main Architecture

#Introduction

Episode 1 already answered why Vitess was born. Now we answer how it works: what its components are, how they coordinate, and how a simple query manages to find data that might be spread across dozens of servers. Episode 2 is the architecture foundation — every later episode will refer back to the concepts here.

Episode 2 roadmap: we dissect the four main components (VTGate, VTTablet, Topology Service, and MySQL instances), then the keyspace and shard concepts, then vindex as the routing brain, and finally the complete flow of a query. By the end of the episode, the big picture of the Vitess architecture will be clear in your mind.

#Vitess Main Components

#VTGate: Query Serving Layer

VTGate is the entry point for all queries. Applications connect to VTGate just like they'd connect to regular MySQL, but behind the scenes VTGate does three jobs: parsing SQL, planning the query, and routing it to the right shard. VTGate also manages connection pooling, transactions, and concurrency limits.

One thing to remember: VTGate doesn't store any data at all. It's purely a logic layer that knows where data lives based on metadata from the Topology Service. This makes VTGate easy to scale horizontally — add VTGate instances, add query capacity.

#VTTablet: Storage Layer

VTTablet is the agent that lives side by side with each MySQL instance. One VTTablet manages one MySQL instance, and they're the ones that communicate with VTGate: receiving queries, executing them in MySQL, managing replication, and reporting health and metrics.

VTTablet also runs various background jobs: replication monitoring, backups, and VReplication (which we'll cover in episode 10). Think of VTTablet as a "remote control" that lets Vitess drive MySQL without logging into it directly.

vtctlclient ListAllTablets

The vtctlclient ListAllTablets command lists all tablets along with their keyspace, shard, and role type — your first window into the running architecture.

#Topology Service

Topology Service is Vitess's source of truth: it stores metadata about keyspaces, shards, tablets, and policies. Technically it's a key-value backend like etcd, ZooKeeper, or Consul. Whenever VTGate needs to know which shard serves a given keyspace, it asks the Topology Service.

That's why the Topology Service must be extremely reliable — it's the nervous system of the Vitess cluster. If it goes down, operations that need metadata (like planned failover) are disrupted, even though in-flight queries can usually still be served by VTGate instances that already have topology caches.

#MySQL Instances

At the very bottom is MySQL itself. Vitess doesn't replace MySQL — it wraps it. You're free to use MySQL community, Percona Server, or MariaDB, and Vitess manages them through VTTablet. The MySQL version you use affects the features available — for example, newer versions support more modern failover mechanisms.

#Keyspace, Shard, and Vindexes

#Keyspace

A keyspace is the highest logical unit: a collection of tables that the application treats as one database. An application can have multiple keyspaces, like separate databases in regular MySQL. A keyspace can consist of one shard (no sharding) or many shards.

USE commerce

The USE commerce command selects the target keyspace, similar to USE database in MySQL. This is where we distinguish the concepts: a keyspace is like a database, a shard is like a piece of a database.

#Shard

A shard is a data partition. One shard is one MySQL instance (complete with its replicas) that stores part of a keyspace's data. A keyspace sharded across three shards means its data is split three ways. Two sharding approaches: sharding by range (e.g. ID ranges) and sharding by hash (e.g. a hash of user_id).

#Vindex: The Routing Brain

A vindex is a function that maps a column value to a shard. When a query like WHERE user_id = 42 comes in, VTGate uses the user_id vindex to compute which shard holds that user_id's data. A vindex is required for the sharding key column, and choosing the vindex is the most important design decision in sharding.

{
  "vindexes": {
    "hash": { "type": "hash" }
  },
  "tables": {
    "users": {
      "column_vindexes": [
        { "column": "user_id", "name": "hash" }
      ]
    }
  }
}

The VSchema (Vitess Schema) above defines a vindex named hash applied to the user_id column of the users table. All queries based on user_id will be routed directly to the right shard — no scatter.

#The Routing Flow of a Query

Let's follow the journey of one query from the application down to disk:

1. The application sends SELECT * FROM users WHERE user_id = 42 to VTGate.
2. VTGate parses the query, then looks at the VSchema to find the user_id vindex.
3. VTGate computes the target shard from the value 42, and asks the Topology Service to find the tablet serving that shard.
4. VTGate forwards the query to the correct VTTablet.
5. VTTablet executes it in MySQL, then returns the result to VTGate, which forwards it to the application.

If the query doesn't mention the sharding key column, VTGate has to do a scatter-gather: send the query to all shards and combine the results. This works, but it's slower — one reason good schema design matters so much.

#VReplication and Streaming

VReplication is the MySQL log-based replication engine Vitess uses for many tasks: migrating data between shards, resharding, streaming to data warehouses, and online schema changes. It works by reading the binlog from the source and applying it to the target. We'll use VReplication repeatedly in episodes 10 and 16.

#Closing

In this episode 2 you've understood the core Vitess architecture: VTGate as the stateless query serving layer, VTTablet as the agent managing each MySQL instance, Topology Service as the source of truth for metadata, plus the keyspace, shard, and vindex concepts that govern data distribution and routing.

Key takeaways:

• VTGate routes queries without storing data; it scales horizontally.
• VTTablet manages one MySQL instance and runs background jobs like backups.
• Topology Service is the source of truth: etcd, ZooKeeper, or Consul.
• A keyspace is like a database, a shard is a piece of data, a vindex maps values to shards.
• Queries with the sharding key column = single-shard fast path; without it = scatter to all shards.

In the next episode we get hands-on: basic installation and deployment of Vitess on Kubernetes using Helm, configuring the first keyspace and shard, then verifying the components and MySQL connectivity. See you there!