Learn Proxmox VE - Monitoring, Alerting & Metrics (Prometheus & Grafana)

#Introduction

An invisible infrastructure is an unpredictable infrastructure. Without monitoring, you only find out that storage is full or a node is down after users complain. Episode 18 builds observability: seeing metrics, visualizing them, and being woken up early by alerts when something starts to drift.

We'll use Proxmox's built-in monitoring, export metrics to Prometheus with pve-exporter, display them in Grafana, and then configure alerting. By the end of the episode, you'll be able to monitor the entire cluster from one dashboard.

#Proxmox Built-in Monitoring

#Metrics in the Web UI

The Proxmox web UI already provides basic monitoring: CPU, RAM, storage, and network graphs for each node, plus per-VM metrics. The Datacenter -> Graph Server page even shows metrics for all nodes in one view. That's enough for daily visual supervision.

Node : CPU, RAM, storage, network, load average
VM   : CPU usage, RAM utilization, disk I/O, network traffic

However, built-in metrics are stored with limits and can't be queried flexibly. For long retention and analysis, you need an external metrics system.

#External Monitoring with Prometheus

#Installing pve-exporter

Prometheus collects metrics from various sources through exporters. prometheus-pve-exporter is the official exporter that reads Proxmox metrics from the API and serves them in Prometheus format. Install it on a separate machine (or container):

pip install prometheus-pve-exporter

Create the exporter configuration file:

default:
  user: mon@pve
  password: <read-only-password>
  verify_ssl: false

The mon@pve user only needs read permission at the datacenter level — following the least privilege principle from episode 14.

#Running and Testing the Exporter

Run the exporter as a systemd service:

prometheus-pve-exporter /etc/pve-exporter/config.yml

The exporter listens on port 9221. Test it with curl:

curl http://localhost:9221/pve

The curl http://localhost:9221/pve command should return metrics in Prometheus format, including pve_up, pve_cpu_usage_ratio, pve_memory_usage_bytes, and others.

#Adding a Scrape Job in Prometheus

Register the exporter in the Prometheus configuration:

scrape_configs:
  - job_name: proxmox
    static_configs:
      - targets: ["192.168.1.10:9221"]

Prometheus now pulls Proxmox metrics periodically. Use queries like pve_cpu_usage_ratio in PromQL to check the load.

#Visualization with Grafana

#Connecting the Data Source

Grafana presents Prometheus metrics in easy-to-read dashboards. Add Prometheus as a data source, then import a dashboard. Many Proxmox dashboards are available on grafana.com — one of the most popular shows graphs for both nodes and VMs.

CPU usage, RAM utilization, disk I/O, network throughput

The dashboard displays all nodes and VMs on one screen, with graphs that can be zoomed and filtered.

#Important Metrics to Monitor

• CPU usage and load average per node.
• RAM utilization to detect memory pressure.
• Disk I/O and storage capacity.
• Network throughput per interface.

#Alerting

#Notifications When Problems Occur

A good dashboard isn't enough — you won't stare at it all day. Alerting is what wakes you up. In Grafana, create alert rules based on PromQL; for larger needs, use Alertmanager:

Node down        : up{job="proxmox"} == 0
Storage full     : pve_storage_used_bytes / capacity > 0.9
VM not running   : pve_vm_status == 0

Alerts are sent to notification channels — email, Telegram, Slack, or webhook.

#Setting Up an Alert Rule in Grafana

In Grafana, open the Alerting tab and create a new rule with a query like:

when : max of pve_storage_used_ratio > 0.9
for  : 5m

This rule only fires if the condition persists for 5 minutes, reducing false positives.

#Closing

Episode 18 built complete observability: using Proxmox's built-in metrics, exporting metrics to Prometheus with pve-exporter, visualizing them in Grafana, and configuring alerting for early detection.

The key takeaways:

• The Proxmox web UI has built-in metrics for daily supervision.
• pve-exporter converts Proxmox metrics into Prometheus format.
• A monitoring user only needs read access at the datacenter level.
• Grafana unifies all metrics in one dashboard.
• Focus metrics on CPU, RAM, storage, and network.
• Start alerting with node down, storage full, and VM crashes.

In the next episode, episode 19, we will cover troubleshooting and disaster recovery — the pvecm, ha-manager, qm, and pct diagnostic commands, log analysis with journalctl, handling common problems like split-brain and full storage, and recovery procedures from backups. Your infrastructure is now monitored; it's time to learn how to handle things when they break!