Learn Ceph - Block Storage with RBD

#Introduction

Welcome to episode 5 of the Learn Ceph series! In episode 4 you understood pools. Now we'll use pools for one of Ceph's most popular interfaces: block storage with RBD (RADOS Block Device).

RBD is the interface used for virtual machine disks, database disks, and workloads that need raw block access. RBD's signature feature is being accessible as a block device just like a physical disk, but with modern storage features such as snapshots, clones, and thin provisioning.

In this episode we'll cover RBD concepts and use cases, creating an RBD image and mapping it as a block device in Linux, integration with OpenStack Cinder and Kubernetes CSI, and snapshots, clones, and thin provisioning.

#RBD Concepts and Use Cases

#What Is RBD

RBD maps RADOS objects into an image that looks like a continuous block device. Clients use the rbd driver — either the Linux kernel, librbd in QEMU, or the CSI driver in Kubernetes — to access the image.

RBD images are stored in a pool with the rbd application. The pool where images are stored is usually named something like rbd-pool or volumes, following the OpenStack convention.

#When to Use RBD

RBD is suitable for workloads that need raw block access:

• VM disks: qcow2/raw images used by hypervisors.
• Database backends: databases need consistent block access with snapshot support.
• Kubernetes: persistent volumes via the CSI driver.

If a workload only needs shared files or HTTP-based access, consider CephFS or RGW as alternatives.

#Creating an RBD Image and Mounting It as a Block Device

#Creating an Image

Images are created in a pool with the rbd create command:

rbd create vol-data --size 10G --pool rbd-pool
rbd ls rbd-pool
rbd info rbd-pool/vol-data

rbd create vol-data --size 10G --pool rbd-pool creates an image with 10 GB capacity. rbd info shows details like size, block size, and the active features. Images support thin provisioning by default.

#Mapping and Formatting

To use an image as a disk in Linux, map it with the rbd kernel module:

rbd feature disable rbd-pool/vol-data exclusive-lock,object-map,fast-diff,deep-flatten
rbd map rbd-pool/vol-data
sudo mkfs.ext4 /dev/rbd0

The rbd map rbd-pool/vol-data command creates the /dev/rbd0 device on the client. Certain features must be disabled for older kernels; mkfs.ext4 then formats the device as a filesystem.

sudo mkdir /mnt/vol-data
sudo mount /dev/rbd0 /mnt/vol-data

After mounting, /mnt/vol-data can be used like a regular disk. To map the device from another node, that node needs a client keyring with access to the pool — we'll cover the details in episode 8.

#Resizing an Image

Increasing an image's capacity is easy:

rbd resize rbd-pool/vol-data --size 20G
sudo resize2fs /dev/rbd0

rbd resize grows the image, and resize2fs expands the filesystem inside it. RBD supports shrinking with --allow-shrink, but that's a risky operation and rarely recommended.

#Snapshots, Clones, and Thin Provisioning

#Creating a Snapshot

A snapshot is a consistent point-in-time copy of an image and is read-only:

rbd snap create rbd-pool/vol-data@before-migration
rbd snap ls rbd-pool/vol-data

rbd snap create creates a snapshot named before-migration. Snapshots are the basis for point-in-time backups and can be restored at any time with rbd snap rollback.

#Cloning from a Snapshot

A clone produces a new, independent image that shares data with the source snapshot until the data changes. Cloning is a fast way to copy an image without copying all the data:

rbd snap protect rbd-pool/vol-data@before-migration
rbd clone rbd-pool/vol-data@before-migration rbd-pool/vol-clone

rbd snap protect marks the snapshot so it can't be deleted while a clone still depends on it. rbd clone creates the vol-clone image linked to that snapshot.

#Thin Provisioning

RBD images are created thin: only the blocks actually written consume pool capacity. The total image capacity can exceed the pool capacity, as long as actual usage doesn't.

ceph df
rbd du rbd-pool/vol-data

rbd du shows the space an image really uses compared to its logical capacity. Monitor this value so over-provisioning doesn't turn into a full disk.

#Integration with OpenStack Cinder and Kubernetes

#OpenStack Cinder

Ceph RBD is Cinder's primary backend in OpenStack deployments. Cinder calls librbd to create volumes, snapshots, and clones, so RBD features are available directly in the Horizon dashboard. Configuration is done in the cinder.conf file with volume_driver = cinder.volume.drivers.rbd.RBDDriver.

#Kubernetes RBD CSI

For Kubernetes, the RBD CSI driver provides persistent volumes with snapshot, resize, and clone features via a StorageClass:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-rbd
provisioner: rbd.csi.ceph.com
parameters:
  clusterID: <fsid>
  pool: rbd-pool
  csi.storage.k8s.io/fstype: ext4

The StorageClass above lets PVCs use RBD images automatically. Full Kubernetes deployment will be covered in episode 16.

#Conclusion

In this episode you've understood and practiced RBD block storage: concepts and use cases, creating images and mapping them as block devices, snapshots and clones, thin provisioning, and integration with OpenStack Cinder and Kubernetes CSI.

The key takeaways:

• RBD maps RADOS objects into a continuous block device.
• rbd create, rbd map, then mkfs and mount is the basic usage flow.
• Snapshots are read-only and form the basis for point-in-time backups.
• Clones share data with the source snapshot, so they're fast to create.
• RBD images are thin: only the used blocks consume capacity.
• Cinder and the RBD CSI use librbd for automatic provisioning.

In the next episode, episode 6, we'll cover file storage with CephFS — the metadata server architecture and data path, creating a filesystem and mounting clients, active-active MDS for scaling metadata, and quotas, snapshots, and client consistency. Get ready for your first file interface!