Learn Vitess - Resharding & Scaling Workflows

#Introduction

Every database eventually hits the same wall: capacity. Episode 16 covers how Vitess gets past that wall without downtime — through resharding. This is the feature Vitess operators fear most and rely on most: frightening because the wrong sequence means disaster, reliable because companies like YouTube and Slack have used it for years.

Episode 16 roadmap: when to reshard, online resharding with VReplication, the difference between splitting and moving shards, scaling replicas, then vertical vs horizontal scaling.

#When to Reshard

Resharding is a big operation — don't do it without a clear reason. Common signs:

• A shard is approaching capacity (storage or CPU).
• Write latency is rising because one primary is overwhelmed.
• Backups take too long because one shard's data is too large.
• Storage is the bottleneck — data no longer fits in one shard.

Resharding doesn't have to change the number of shards. Sometimes what's needed is moving shards: relocating shards to more suitable hardware or regions. This episode covers both.

#Online Resharding with VReplication

Resharding in Vitess runs online: data is copied to new shards while the application keeps serving traffic. The engine behind it is VReplication (episode 10). The main flow:

1. Create new shards (e.g., -80 and 80-) for the keyspace.
2. Create a resharding workflow that copies data from shard 0 to the new shards.
3. VReplication copies a snapshot, then catches up from the binlog.
4. Once lag approaches zero, switch read and write traffic to the new shards.
5. Complete the workflow and deactivate the old shards.

vtctlclient CreateShard users/-80
vtctlclient CreateShard users/80-

The vtctlclient CreateShard command creates new, still-empty shards. The next step is the resharding workflow — in modern Vitess versions, use the workflow with target shards:

vtctlclient Reshard --source-shards=0 \
  --target-shards=-80,80- users create split_users
vtctlclient Workflow --keyspace=users show split_users

The vtctlclient Workflow ... show command monitors progress: how much data is copied, how much lag, and when it's safe to switch.

#Splitting vs Moving Shards

#Splitting (Shard Split)

Splitting breaks one shard into several: 0 becomes -80 and 80-, or -80 becomes -40 and 40-80. Total capacity rises because the data is evenly divided. This is the most common operation for growth.

#Moving Shards

Moving shards relocates shards from one cell/region to another without changing their number. Useful during infrastructure relocation, or when a region is no longer ideal. Data is copied to the new location, synchronized, then traffic is moved over.

vtctlclient MoveTables --source=commerce --tables=orders users create order_move
vtctlclient MoveTables --source=commerce --tables=orders users order_move switchtraffic

vtctlclient MoveTables moves the orders table from the commerce keyspace to users — an example of moving shards at the keyspace level that's also used for relocation.

#Scaling Replicas and Vertical vs Horizontal

#Adding Replicas

Resharding adds primaries (write capacity). For read capacity, add replicas. This is a lightweight operation: bootstrap a new tablet from the latest backup, then let it catch up on replication.

vtctlclient ListAllTablets -keyspace_shard=users/0

vtctlclient ListAllTablets lists tablets per shard — you can see how many replicas exist and add more if read load rises.

#Vertical vs Horizontal Scaling

• Vertical scaling (scale up): enlarging a single server's resources. Simple, but there are physical limits and costs that rise exponentially. Good for buying time.
• Horizontal scaling (scale out): adding servers. In Vitess that means adding shards or replicas. Scales almost without limit, but adds operational complexity.

Practical recommendation: start with vertical scaling to postpone complexity, then switch to horizontal (resharding) when vertical stops making sense financially or technically.

#Cleanup After Resharding

After the traffic switch, the work isn't over. Old shards still hold data that's no longer routed — leaving it is wasted storage and operational cost. The final stage of resharding is cleanup:

1. Make sure all queries are directed to the new shards.
2. Verify no VReplication still depends on the old shards.
3. Deactivate the old shards from the keyspace.
4. Delete the shards and their physical data after a safe waiting period.

vtctlclient Reshard --keyspace=users --source-shards=0 \
  --target-shards=-80,80- complete split_users

vtctlclient Reshard ... complete marks the workflow complete. Don't rush to delete old data before you're sure — allow an observation window after the switch before removing the source.

#Closing

In this episode 16 you understood how to expand Vitess capacity: when resharding is needed, online resharding with VReplication that runs without downtime, the difference between splitting and moving shards, scaling replicas for read load, and the vertical vs horizontal scaling decision.

Key takeaways:

• Resharding is a big operation — do it when there's a clear capacity reason.
• Online resharding copies data while serving traffic, then switches.
• Splitting increases the shard count; moving shards relocates them without changing the count.
• vtctlclient Workflow show is the tool for monitoring resharding progress.
• Don't change two things at once when resharding — change one at a time.
• Start with vertical scaling, move to horizontal when the limit is reached.

In the next episode, episode 17, we combine two worlds: hybrid workloads and OLAP integration — separating OLTP and OLAP, read replicas for reporting, snapshot export, and ETL integration with a data warehouse. See you there!