Learn Wazuh - Distributed Architecture: Cluster & Multi-node

#Introduction

In episode 13, we covered Docker and Kubernetes integration. Now we go up another level: managing Wazuh itself so it's ready to scale. All installations so far were done on a single all-in-one node, which is enough for a lab or small environment.

In production, a single node becomes a single point of failure. If the manager dies, alerts stop; if the indexer dies, data is lost; if the dashboard dies, nobody can read anything. In episode 14, we'll build the distributed Wazuh architecture: a multi-node indexer cluster, a three-node manager cluster, and a dashboard, complete with load balancing and failover.

We start by understanding each component's role, then look at lab versus production topologies, and close with the steps to set up the cluster along with scale-out strategy.

#The Role of Each Component

The distributed Wazuh architecture consists of three component families whose roles must be understood.

Wazuh indexer: data storage and search, based on OpenSearch. Alert and event data is stored here.
Wazuh manager: analysis and agent coordination. Rules are processed and alerts are born here.
Wazuh dashboard: the user interface, including visualizations, Discover, and management.

Each component can be clustered on its own. This is the key to distributed architecture: scale each component according to its workload.

#Lab versus Production Topology

Don't mix lab and production needs.

Lab: one all-in-one node, one manager, one indexer, one dashboard. Enough for learning and testing.
Small production: a single indexer with two replicas, a single manager, a single dashboard. Starts to separate per component.
Full production: three indexer nodes, three manager nodes, and a dashboard. This is the topology we cover in this episode.

A simple rule of thumb: the larger the event volume, the larger the indexer. The more agents, the larger the manager. The dashboard is rarely a bottleneck; a single instance with a second node for failover is enough.

#The Indexer Cluster

The indexer uses an OpenSearch cluster with a quorum model. To avoid split brain, the number of nodes must be odd, at least three. All three nodes store the same data as replicas, so one node dying doesn't lose data.

#Basic Configuration per Node

Each indexer node needs a uniform cluster configuration. The cluster name must be the same, while the node name must be unique.

opensearch.yml per node

cluster.name: wazuh-cluster
node.name: indexer-1
discovery.seed_hosts:
  - 10.0.0.11
  - 10.0.0.12
  - 10.0.0.13

Change the node.name value to indexer-2 and indexer-3 on the following nodes. To communicate with the manager, Wazuh uses Filebeat. Filebeat on each manager is configured to send data to all indexer nodes.

filebeat.yml output section

output.elasticsearch:
  hosts:
    - https://10.0.0.11:9200
    - https://10.0.0.12:9200
    - https://10.0.0.13:9200

With a complete host list, Filebeat automatically tries the next node if one dies. That's the basic failover for the data flow.

#The Manager Cluster

Managers can be clustered with master and worker modes. One node becomes the master, the rest are workers. The master manages cluster state, while workers process events from agents.

#Setting Up the Manager Cluster

Cluster configuration is done in /var/ossec/etc/ossec.conf in the <cluster> section. The name and nodes must be consistent across all managers.

On the second and third nodes, change node_name and node_type. Only one node may be the master.

#Master and Worker Roles

This division of labor is important to understand:

Master: stores cluster state, forwards configuration to workers, and syncs rules.
Worker: processes events from agents, runs rules, and generates alerts.
Both: connect to Filebeat to send data to the indexer.

If the master dies, workers don't immediately stop processing events, but cluster management capability is impaired. That's why the master needs good monitoring and backup.

#Load Balancing Agents

Agents need to know where to send events. There are two main approaches.

DNS round robin: several A records for the manager hostname. Agents get distributed at random.
Load balancer: a VIP in front of the three managers. Agents connect to the VIP, and the load balancer splits connections.

Example of a load balancer configuration with nginx in front of the managers:

Load balancer for managers

upstream wazuh_managers {
    server 10.0.1.11:1514;
    server 10.0.1.12:1514;
    server 10.0.1.13:1514;
}
 
server {
    listen 1514 udp;
    proxy_pass wazuh_managers;
}

With this configuration, agents only need to know a single address, and the load is spread evenly across the three managers.

#HA and Failover

High availability is born from the combination of all the layers above.

Indexer: a three-node cluster with replicas, data stays available when one node dies.
Manager: if one worker dies, the agents connected to it must move to another worker. The load balancer handles this move.
Dashboard: point it to the indexer load balancer so it can keep reading data.
Filebeat: a complete host list makes data delivery automatically switch nodes.

Info

Failover isn't without delay. When a worker changes, agents need a few seconds to reconnect. For truly real-time needs, consider adding capacity instead of chasing super-fast failover.

#Monitoring Cluster Health

Wazuh provides API endpoints to monitor cluster status. Through the dashboard, we can see which nodes are alive, how many agents are connected to each worker, and whether synchronization is running normally.

View cluster status via the API

curl -k -u wazuh-wui:SecretWUI \
  https://10.0.1.11:55000/cluster/status

Monitor these metrics regularly. Overloaded workers signal the need for more nodes, while a full indexer node signals the need for more storage capacity.

#Conclusion

In episode 14, we built the distributed Wazuh architecture for production scale. We understood the roles of indexer, manager, and dashboard, distinguished lab and production topologies, set up a three-node indexer cluster with quorum, a master and worker manager cluster, and load balancing and failover at every layer.

Key takeaways:

Each component is clustered separately according to its workload.
The indexer needs an odd number of nodes, at least three, to avoid split brain.
Managers use the master and worker model, with only one master.
Filebeat with a complete host list provides data flow failover.
Agents can be distributed via DNS round robin or a load balancer.
Cluster monitoring matters for catching bottlenecks early.

The architecture is ready, now let's strengthen its keys. In episode 15, we'll cover the Wazuh API, authentication, and security best practices: REST endpoints, SDKs, RBAC, single sign-on, up to TLS between components. See you there!