Learn Ceph - Pre-Requisite Skills & Environment Setup

#Introduction

Welcome to the Learn Ceph series! This series will take you through mastering Ceph — a software-defined, open-source distributed storage system that provides block, file, and object storage in a single cluster — from the foundations of the concepts to production readiness. There are 23 episodes in total, arranged across six phases.

But before touching ceph and cephadm, there are some basic skills and software you must have. Why are these prerequisites important? Because Ceph is not just an application you install and you're done. Ceph is a distributed system where data is sharded, replicated, and placed across many nodes based on the CRUSH algorithm. If you don't yet understand the block, file, and object storage concepts, or aren't familiar with Linux administration, every operation will feel like opening a black box.

Episode 0 is your roadmap: we will make sure the basic skills are in place, prepare the software and hardware, then bootstrap your first Ceph cluster with cephadm and verify its health. Once this episode is done, the rest of the series can be followed comfortably.

#Basic Skills You Must Master

#Storage Concepts: Block, File, and Object

Ceph provides three types of storage at once, so you must understand the differences from the start:

• Block storage: data is stored in fixed-size blocks, used as raw disks for VMs or databases, accessed through RBD in Ceph.
• File storage: data is stored in a hierarchy of directories and files, accessed through NFS/CIFS mounts or CephFS.
• Object storage: data is stored as objects with metadata and a unique ID, accessed through the S3/Swift API, implemented by the RADOS Gateway in Ceph.

#Linux, Networking, and Storage Administration Basics

All Ceph operations run on Linux. You must be comfortable with the command line, disk management, and networking:

• Linux: systemd, SSH, package managers, and permissions.
• Networking: TCP/IP, DNS, and firewalld/iptables to open cluster ports.
• Storage administration: an understanding of disks, partitions, journals, and WAL.

uname -a
cat /etc/os-release
free -h
df -h

Make sure your system is Red Hat-based (RHEL, Rocky, AlmaLinux) or Debian-based (Ubuntu). Both are fully supported by Ceph.

#Distributed Systems and Fault Tolerance

Ceph is built on distributed systems principles. Here are some concepts you must understand:

• CAP theorem: consistency, availability, partition tolerance, and the trade-offs between them.
• Replication: copies of data across several nodes to tolerate hardware failures.
• Quorum: the majority of nodes that must be alive for the service to keep running, for example at least 2 out of 3 MONs.

#Virtualization and Containerization Basics

For the lab, you can run a multi-node Ceph cluster on VMs or containers. Make sure you understand:

• Virtualization: VMs with VirtualBox, libvirt/KVM, or a cloud provider.
• Containerization: Docker or Podman, which cephadm uses to run Ceph daemons.
• Orchestration: basic Ansible is optional for provisioning repeatable nodes.

#Software to Prepare

#Ceph Release and cephadm

Ceph 18 (Reef) and Ceph 19 (Squid) are the recommended releases as of this writing. The main tool for modern deployment is cephadm, which runs every Ceph daemon as a container:

curl -fsSL https://download.ceph.com/rpm-19.2.0/el9/x86_64/cephadm -o /usr/sbin/cephadm
chmod +x /usr/sbin/cephadm
cephadm version

If you are on Ubuntu, replace rpm-19.2.0/el9/x86_64 with deb-squid/. Verify that cephadm version shows the version you downloaded.

#Other Supporting Tools

Prepare the following tools from the start because they will be used in many episodes:

• SSH access to all nodes, with a passwordless key pair for cephadm.
• Git for storing cluster configuration and documentation.
• ceph CLI, which is installed automatically alongside the bootstrap.
• Monitoring: Prometheus and Grafana, or just use the built-in Ceph dashboard.

#Hardware Minimum Requirements

#Number of Nodes and Specifications

For production, Ceph recommends a minimum 3-node cluster. This isn't an arbitrary suggestion: MON quorum and CRUSH replica placement require at least three independent locations.

RAM   : 8 GB or more
CPU   : 4 cores or more
Disk  : 1 disk for OS + 1 disk or more for OSD
Network: 1 Gbps minimum, 10 Gbps recommended

For the lab, creating three VMs that meet the specification above is enough. Note that the disk for the OSD should be separate from the OS disk, because OSDs store DB/WAL which is sensitive to I/O.

#Network for Cluster Traffic

Separate the concepts of public network (used by clients) and cluster network (used between daemons). If you only have one NIC, Ceph can still run, but backfill and recovery performance will share bandwidth with client traffic.

#Environment Setup with cephadm

#Bootstrap the First Node

Bootstrap runs on the first node and automatically creates the first MON, the first MGR, and copies the keyring to /etc/ceph:

cephadm bootstrap --mon-ip 192.168.100.10

Replace 192.168.100.10 with the IP of the first node. At the end of the process, the terminal displays the dashboard URL, username, and the default admin/random password printed to the console.

#Adding Other Nodes

Make sure SSH from the first node to all nodes works without a password, then add the hosts and deploy the OSDs:

ceph orch host add node2 --ssh-user <user>
ceph orch host add node3 --ssh-user <user>
ceph orch apply osd --all-available-devices

The ceph orch apply osd --all-available-devices command will format all available and unused disks on each node as OSDs. Make sure you don't run it on a machine whose disks still hold important data.

#Overall Verification

After all nodes and OSDs are in place, verify the cluster:

ceph status
ceph health detail
ceph orch ps

ceph status shows a summary: version, the active mon/mgr/osd, and the PG status. Ideally ceph health shows HEALTH_OK. If HEALTH_WARN appears in the early episodes, that's normal — for example because the OSDs aren't all up yet or there are incomplete PGs while the cluster is still being set up.

Also run a simple read/write verification through the dashboard to make sure the cluster is really usable. If all indicators are green, your environment is ready for the next phase.

#Conclusion

In episode 0 you've prepared the footing for the whole series: understanding the differences between block, file, and object storage, the basics of distributed systems and fault tolerance, preparing cephadm, bootstrapping the first cluster, adding hosts and OSDs, and verifying cluster health.

The key takeaways:

• Ceph unifies block, file, and object storage in one RADOS-based cluster.
• Understand storage concepts, Linux, networking, and quorum first before touching Ceph.
• The minimum production specification is 3 nodes with 8 GB RAM and 1-10 Gbps networking.
• cephadm runs every Ceph daemon as a container and handles the bootstrap.
• Primary verification is always through ceph status and ceph health detail.

In the next episode, episode 1, we will cover the history, background, and why to choose Ceph — from its birth in 2006 by Sage Weil, the RADOS design that aims to eliminate single points of failure, to its position as a storage foundation in cloud-native and OpenStack. Make sure your cluster is ready, because the Learn Ceph journey is just beginning!