Learn Proxmox VE - Shared Storage (NFS, CIFS/SMB, iSCSI) & Ceph Distributed Storage

#Introduction

Up until episode 7, all of your storage was local — only accessible by a single node. Once your infrastructure grows into a cluster with many nodes, a fundamental need arises: shared storage, where all nodes read and write the same data. This is a prerequisite for migration and high availability.

Episode 8 opens up two worlds at once: classic shared network storage (NFS, CIFS/SMB, iSCSI) that connects Proxmox to a NAS, and Ceph, distributed storage integrated directly with Proxmox to build hyper-converged infrastructure.

#Shared Network Storage

#NFS (Network File System)

NFS is the most commonly used filesystem-sharing protocol in the Linux ecosystem. Many NAS devices like Synology and TrueNAS provide NFS natively. In Proxmox, NFS is ideal for ISO, backups, and templates — data read by many nodes at once.

Check the shares available from your NAS:

showmount -e 192.168.1.100

The showmount -e command shows the list of directories exported by the NFS server. Once you know the share path, add it at Datacenter -> Storage -> Add -> NFS, fill in the server and export path, then choose the desired content types.

#CIFS/SMB and iSCSI

CIFS/SMB is the Windows-style file-sharing protocol that Proxmox can also access — useful if you use a NAS focused on the Windows ecosystem. Meanwhile, iSCSI is completely different: not a filesystem, but a block-level transport that sends raw disk operations over the network. iSCSI offers higher performance for VM disks because there's no filesystem layer on the data path, and it usually needs LVM or ZFS on top so Proxmox can allocate space.

NFS   : filesystem, Linux-native, simple
CIFS  : filesystem, Windows ecosystem, needs credentials
iSCSI : block-level, high performance, needs LVM/ZFS on top

pvesm status

#Configuring Network Storage in Proxmox

#Adding NFS Storage

Adding NFS in Proxmox can be done from the web UI or the CLI. First make sure the NFS share is available and accessible from all cluster nodes:

showmount -e 192.168.1.100
mount -t nfs 192.168.1.100:/volume1/backup /mnt/test

The showmount -e command verifies the exported shares, and mount -t nfs tests read-write access. After the test succeeds, register it as storage:

pvesm add nfs nas --server 192.168.1.100 --export /volume1/backup \
  --content backup,iso

The storage named nas is now available for backup and ISO content on all cluster nodes.

#Adding iSCSI Storage

For iSCSI, Proxmox connects to an iSCSI target on the storage server, then builds LVM or ZFS on top so space can be allocated per VM:

pvesm add iscsi iscsi-pool --portal 192.168.1.100 --target iqn.2026-08.lab:target1

The pvesm add iscsi command registers the iSCSI target. Once connected, create an LVM-thin on top of it to support snapshots.

#Introduction to Ceph Distributed Storage

#Hyper-Converged Infrastructure

Ceph is a distributed storage system that turns the storage of every node into one giant pool accessible by all nodes. Because it runs on the same Proxmox nodes as the VMs, this architecture is called Hyper-Converged Infrastructure (HCI) — compute and storage combined in one hardware.

Monitor  : maintains the cluster map and health
Manager  : provides metrics and a dashboard
OSD      : where data is stored (one per disk)
Pool     : logical group where data is placed
PG       : placement group, small unit of data replication

Each component has a specific role. OSD (Object Storage Daemon) stores the actual data — the number of OSDs determines capacity and parallelism. Monitor ensures all nodes agree on the cluster state, and placement groups divide data into small units spread evenly across the cluster.

#Why Ceph for Proxmox?

Ceph provides value no other storage has: data is spread across multiple nodes with automatic replication, so if one node dies, the data remains available. All features that need shared storage — migration, HA, live snapshots — run on top of it without additional configuration.

#Building a 3-Node Ceph Cluster

#Basic Configuration from the UI

Ceph is best run on at least 3 nodes. In Proxmox, Ceph installation is very easy because it's already integrated:

1. Open Datacenter -> Ceph and click Install Ceph on the first node.
2. Add OSDs one by one from Ceph -> OSD -> Create OSD.
3. Repeat the Ceph installation on the second and third nodes.
4. Create a pool in Ceph -> Pools with size (number of replicas) 3.

ceph -s

The ceph -s command shows the cluster health, OSD count, and placement group status. Aim for a HEALTH_OK status with all PGs in active+clean state.

The replica count and placement group size determine resilience and data distribution. For 3 nodes, size 3 means each data block is copied to three OSDs — surviving two simultaneous failures. Adjust pg_num to your total OSD count; too small a value causes uneven data distribution across the OSDs.

#Closing

Episode 8 expanded your storage from local to network: NFS for file sharing, CIFS/SMB for the Windows ecosystem, iSCSI for high-performance block-level access, and Ceph as hyper-converged distributed storage fully integrated into Proxmox.

The key takeaways:

• NFS is ideal for ISO, backups, and templates shared between nodes.
• iSCSI provides block-level storage with high performance.
• Ceph combines all nodes' storage into one distributed pool.
• Ceph components: monitor, manager, OSD, pool, and placement groups.
• Ceph needs at least 3 nodes and a dedicated high-speed network.
• Aim for a HEALTH_OK status with all PGs active+clean.

In the next episode, episode 9, we will cover networking fundamentals — the vmbr0 Linux bridge, network bonding for redundancy and bandwidth, and VLAN tagging for network isolation at layer 2. Your storage is now distributed; it's time to understand the data paths that connect it all!