Learn FreeBSD - High Availability & Monitoring

#Introduction

In the previous episode 19, you ran FreeBSD as a router and application server. Now we go to the next level: high availability and monitoring — the system's ability to keep running when something fails, and your ability to see it before it fails.

A single reliable server still has a single point of failure. With CARP, two FreeBSD servers can share the same IP address and automatically take over when the primary dies. And with good monitoring, you know about problems before users feel them. Let's build both.

#High Availability with CARP

#The CARP Concept

CARP (Common Address Redundancy Protocol) is a failover protocol that lets multiple hosts share a virtual IP address. If the primary host dies, the backup host takes over that address within seconds — clients don't notice any disruption.

#Enabling CARP in the Kernel

CARP is a kernel module:

Memuat modul CARP

kldload carp
sysrc if_carp_load="YES"

#CARP Configuration

Configuration is done by creating a carp0 interface on both servers. On the master server:

Konfigurasi CARP master

sysrc cloned_interfaces="carp0"
sysrc ifconfig_carp0="vhid 1 pass secret123 192.168.1.254/24"

On the backup server, add carpdev and a lower priority:

Konfigurasi CARP backup

sysrc ifconfig_carp0="vhid 1 advskew 100 pass secret123 192.168.1.254/24 carpdev em1"

Warning

Both servers must use the same vhid and the same pass. advskew determines priority — a smaller value means it's more "preferred" to be master. Never use an easily guessable CARP password.

#Testing Failover

To test, bring the master interface down and watch whether the backup takes over:

Menguji failover CARP

ifconfig carp0
ifconfig carp0 down
ifconfig carp0 up

ifconfig carp0 shows MASTER or BACKUP status. When the master dies, the backup automatically rises to MASTER and holds the virtual IP address.

#CARP for Services

The CARP virtual IP can be used for any service — from a gateway router to a reverse proxy. The combination of CARP + an active service (like Nginx or pf) means clients always have a live target.

Success

CARP is the foundation of high availability on FreeBSD. With CARP for IP failover, pf for state sync between firewalls, and ZFS for data, you have a triplet that lets systems survive hardware failures.

#Load Balancing

Besides failover, CARP can be combined with load balancing: several active servers share traffic. At the application layer, Nginx can act as a load balancer:

Load balancing di nginx

upstream backend {
    server 192.168.1.10;
    server 192.168.1.11;
}

For the network layer, pf can also split connections (episode 12). Choose the layer according to your needs and complexity.

#Monitoring

#sysstat: System Statistics

sysstat runs scheduled monitoring with cron and stores history:

Mengaktifkan sysstat

pkg install sysstat

Once enabled via cron, you can read the statistics history with vmstat and systat based on the collected data.

#netdata: Real-time Dashboard

netdata gives a rich real-time dashboard without complex configuration:

Menginstal netdata

pkg install netdata
sysrc netdata_enable="YES"
service netdata start

Netdata displays system, network, and application metrics through a web browser — the fastest way to visualize a machine's condition.

#Prometheus and node_exporter

For monitoring that can be collected and stored long term, install Prometheus and node_exporter:

Menginstal Prometheus stack

pkg install prometheus node_exporter
sysrc node_exporter_enable="YES"
service node_exporter start

node_exporter exposes metrics on port 9100, and Prometheus collects them periodically:

Memeriksa metrik node_exporter

curl -s http://127.0.0.1:9100/metrics | head

Info

The recommended pattern: node_exporter on every machine exposes metrics, Prometheus collects and stores them, then Grafana visualizes. This is the standard industry monitoring architecture that runs well on FreeBSD.

#Log Monitoring with newsyslog

#Automatic Log Rotation

newsyslog rotates log files so they don't grow without limit. Configuration lives in /etc/newsyslog.conf:

Melihat konfigurasi newsyslog

cat /etc/newsyslog.conf

Example entries: maximum size, number of old files, and compression mode.

#Adding a Log Entry

Add rotation for your own application logs:

Menambahkan rotasi log aplikasi

/var/log/app.log  644  7  100  *  J

The entry above: mode 644, keep 7 files, rotate when reaching 100 KB, no delay.

Warning

Logs that aren't rotated will fill the disk and can stop services. Make sure every application that writes logs has a newsyslog entry — a full disk is the most commonly overlooked cause of failure.

Run a manual rotation for testing:

Menjalankan newsyslog

newsyslog
ls -la /var/log | head

#Assembling Complete Monitoring

One machine can collect everything: node_exporter for metrics, sysstat for history, newsyslog for log rotation, and netdata for a quick dashboard:

Memeriksa seluruh layanan monitoring

service node_exporter status
service netdata status
newsyslog -v

#Closing

In this episode 20, you built high availability with CARP, understood the carpdev concept and load balancing, set up monitoring with sysstat, netdata, Prometheus and node_exporter, and managed logs with newsyslog.

Key takeaways:

CARP shares a virtual IP between hosts; advskew determines master priority.
CARP + pf + ZFS form the foundation of FreeBSD high availability.
node_exporter exposes metrics; Prometheus stores them; Grafana visualizes.
netdata gives a real-time dashboard without complex configuration.
newsyslog prevents logs from ballooning — don't forget rotation for your own apps.