Learn NAS - Monitoring & Health Checks

#Introduction

A healthy NAS can't be left running without supervision. Disks can show early signs of failure, ZFS can detect corruption, and power can go out at any time. Episode 16 covers monitoring and health checks: how to monitor disk, pool, system, and power conditions, and get warnings before a problem becomes a disaster.

The goal of monitoring isn't just displaying numbers — it's giving you time to act. Early disk failure warnings give you time to replace a disk. UPS alerts give you time for a clean shutdown. Both save data.

By the end of this episode you'll be able to monitor SMART, schedule scrubs, configure alerts, use TrueNAS Reporting and TrueCommand, monitor with Prometheus and node_exporter, and set up a UPS with NUT.

#SMART Monitoring

#Reading Disk Health

SMART (Self-Monitoring, Analysis and Reporting Technology) is a disk feature that reports internal health. By monitoring SMART attributes, you can detect a failing disk long before total failure.

smartctl -H /dev/sdb
smartctl -A /dev/sdb

The smartctl -H command shows health test results, while smartctl -A shows detailed attributes. Watch for Reallocated_Sector_Ct and Pending_Sector rising as early signs of damage.

#Scheduling SMART Tests

TrueNAS SCALE provides S.M.A.R.T. Tests that can be scheduled. Run routine short tests and periodic long tests. Monitor the results consistently to detect trends.

smartctl -t short /dev/sdb

The smartctl -t short command runs a short test that finishes within minutes. Compare results across periods to spot declining health.

#Scrub and ZFS Status

#Scheduled Scrub

Scrub verifies checksums across the entire pool and repairs corruption if found. As discussed in episode 5, this is a vital part of ZFS integrity. Schedule a weekly scrub for data-heavy pools.

zpool status tank
zpool scrub tank

The zpool status tank command shows an in-progress scrub's progress or the last result. zpool scrub tank starts a manual scrub if one isn't scheduled.

#Understanding the Status Output

While a scrub runs, the scan and repair columns show progress. Make sure no errors accumulate on any vdev. Persistent errors need immediate investigation.

#Alerts and Notifications

#Configuring Alerts

Alerts move monitoring from reactive to proactive. TrueNAS SCALE supports email and webhook notifications; OpenMediaVault also provides email notifications. Enable at least the critical events: disk failure, scrub errors, and low UPS battery.

midclt call alert.test

The midclt call alert.test command sends a test alert to verify the email configuration works. Make sure the email server reaches an inbox that's actually monitored.

#Alerts Worth Setting Up

• Disk failure or vdev offline.
• Scrub errors.
• Disk temperature passing the threshold.
• UPS entering battery mode.
• Replication or backup failures.

#TrueNAS Reporting and TrueCommand

#Built-in Reporting

TrueNAS SCALE has Reporting, which shows CPU, RAM, network, and I/O graphs. These graphs help you understand normal load and detect anomalies, like I/O spikes during backups.

gstat

The gstat command shows real-time I/O statistics. It's a quick way to see which disk is busy.

#TrueCommand for Multiple NAS Units

TrueCommand is iXsystems' multi-node management tool. With TrueCommand, multiple TrueNAS units can be monitored from one dashboard, complete with alerts and reporting. It fits homelabs that are starting to have several servers.

#Monitoring with Prometheus

#node_exporter and Prometheus

Prometheus is an open-source monitoring system, and node_exporter exposes system metrics in a format Prometheus understands. It's the de-facto standard for observability, including for NAS.

curl -s http://192.168.1.100:9100/metrics | head

The curl command fetches metrics from the node_exporter endpoint. From there, Prometheus can collect the data and Grafana can display it.

#Metrics to Monitor

• CPU, RAM, and load average.
• Disk throughput and I/O.
• Network: bandwidth and errors.
• Service availability for NFS, SMB, SSH.

This curl pattern for verifying exporters is also useful when adding other exporters, like one for ZFS.

#UPS and NUT

#Why a UPS Is Needed

A NAS holds data that's actively being written. A sudden power cut can damage unfinished writes. A UPS (Uninterruptible Power Supply) gives time for a clean shutdown. NUT (Network UPS Tools) connects the UPS to the system so the NAS knows when to shut down.

upsc ups@192.168.1.100

The upsc command shows UPS status like battery level and load. When the mains power goes out, NUT signals the NAS to start an automatic shutdown before the battery runs out.

#Configuring Automatic Shutdown

On TrueNAS SCALE, UPS configuration is done through System > Services > UPS. Set the battery threshold for shutdown. On OpenMediaVault, the NUT plugin provides similar settings.

#Closing

In this episode 16 you built a comprehensive monitoring system: SMART monitoring, scrub and ZFS status, email alerts, TrueNAS Reporting and TrueCommand, OpenMediaVault monitoring, Prometheus with node_exporter, and automatic shutdown with a UPS and NUT.

Key takeaways:

• SMART gives early disk failure warnings; schedule periodic tests.
• ZFS scrub detects and repairs data corruption.
• Alerts must reach a place that's actually monitored.
• Prometheus and node_exporter bring standard observability.
• A UPS with NUT prevents damage from power outages.

In the next episode, episode 17, we'll cover advanced ZFS — from special vdevs for metadata, L2ARC and SLOG, recordsize tuning, dataset tiering, ARC size, zstd compression, dedup, to performance monitoring with zpool iostat. Monitoring is installed; now it's time to squeeze out maximum performance.