Learn MicroCloud - Monitoring & Observability

#Introduction

Episode 19 linked MicroCloud to the automation ecosystem. But automation without visibility is a blind flight — episode 20 builds monitoring & observability. You'll see what's happening in the cluster from one screen: healthy nodes, nearly full OSDs, or problematic networking. Episode 12 touched on Cluster Manager; this episode covers the standard observability layer you can use in production.

An analogy: pilots can't fly without instruments — altimeter, fuel gauge, and warning lights are the nervous system that keeps cockpit decisions informed. MicroCloud monitoring is your instrument panel; this episode installs it.

#MicroCloud's Observability Layer

The common monitoring architecture:

MicroCloud node ──► exporter (metrik) ──► Prometheus (scrape)
                                              │
                                              ▼
                                    Alertmanager ──► alert channel
                                    Grafana ──► dashboard

#The LXD Grafana Dashboard

#The Official Dashboard

The LXD Grafana dashboard is the LXD project's official dashboard — visualizing cluster metrics:

• Instance count per node, container/VM status.
• Storage pool and volume usage.
• Network activity and throughput.

This dashboard works with data from the LXD Prometheus exporter (exposed automatically via the LXD API) — available through MicroCloud Cluster Manager (episode 12) or a standalone deployment.

#Standalone Setup

For standalone production observability, deploy Prometheus and Grafana on an admin node or a separate VM, then register scrape targets for each service:

snap install prometheus grafana

#Prometheus Exporters

LXD, Ceph, and OVN each expose metrics on their endpoints — scrape them all from a single config file:

scrape_configs:
  - job_name: lxd
    metrics_path: /1.0/metrics
    static_configs:
      - targets: ["10.0.0.11:8443", "10.0.0.12:8443", "10.0.0.13:8443"]
  - job_name: ceph
    static_configs:
      - targets: ["10.0.0.11:9283", "10.0.0.12:9283", "10.0.0.13:9283"]
  - job_name: microovn
    static_configs:
      - targets: ["10.0.0.11:9473", "10.0.0.12:9473", "10.0.0.13:9473"]

Key metrics per layer:

• LXD (port 8443): instance count, CPU/memory, storage pool usage.
• Ceph (port 9283): ceph_osd_up, ceph_pool_percent_used, cluster status.
• OVN (port 9473): logical switch/router state, tunnel health.

#Zabbix

Zabbix is a classic monitoring alternative widely used by established infra teams — it supports MicroCloud through the LXD template and host agents:

• LXD template: instance, disk, and CPU status from the LXD API.
• Agent on nodes: system metrics (CPU, RAM, disk, network) from each host.
• Native alerts: Zabbix items and triggers for down conditions.

If your team is already Zabbix-based, use it as the existing layer; Prometheus/Grafana remain great for modern ecosystems. Choose what best fits your team's skill set — not what's most popular.

#Critical Alerting

Three alerts that are mandatory in MicroCloud:

#1. Node Down

• Detection: the Prometheus up metric for LXD/Ceph/OVN targets; or LXD member heartbeats.
• Impact: instances on that node get rescheduled (episode 9); data stays safe thanks to replication.
• Action: the on-call team steps in, the node is checked, evacuate/restore per the runbook (episode 16).

#2. OSD Down

• Detection: ceph_osd_up == 0; Ceph will enter HEALTH_WARN/DEGRADED.
• Impact: data is still available if above min_size, but the redundancy factor drops.
• Action: check the disk (lsblk, logs), add a replacement OSD if permanently broken (episodes 6, 11).

#3. Cluster Degraded

• Detection: Ceph status isn't HEALTH_OK; PGs are degraded/stuck; OVN members offline.
• Impact: capacity drops, data loss risk rises.
• Action: read ceph -s, identify the problematic PGs, let recovery finish (episode 16).

Example alert rules for all three:

groups:
  - name: microcloud
    rules:
      - alert: NodeDown
        expr: up{job="lxd"} == 0
      - alert: OsdDown
        expr: ceph_osd_up == 0
      - alert: ClusterDegraded
        expr: ceph_cluster_health != 1

#Common Pitfalls

• Monitoring only nodes, not storage: an OSD down goes unnoticed until data breaks — always scrape Ceph.
• Alerts without exporters: metrics don't appear before the endpoint is scraped — verify the targets in Prometheus.
• Alert thunderstorm: too many alerts numb the team — limit them to truly critical conditions.
• Forgetting the exporter on new nodes: when adding nodes (episode 11), add their scrape targets too.
• Alerts without runbooks: an alarm without a procedure only adds noise — write the runbook before the alarm fires.

#Closing

Key takeaways:

• MicroCloud observability = exporter → Prometheus → Grafana/Alertmanager.
• LXD, Ceph, and OVN each have their own exporter and key metrics.
• The LXD Grafana dashboard is available via Cluster Manager or a standalone deployment.
• Zabbix is a mature alternative for teams already using it.
• Three mandatory alerts: node down, OSD down, and cluster degraded.
• Every alert needs a runbook — an alert without a runbook is noise.

In the next episode, we'll cover roadmap & community — where MicroCloud is heading (a maturing Cluster Manager, MicroOVN 26.03 LTS, and the awaited 3.x LTS) and the community gathering places: Canonical documentation, the GitHub canonical/microcloud repo, discourse.ubuntu.com, and ubuntu.com/blog. You'll know where the latest news comes from!