Learn Zabbix - Performance Tuning & Capacity Planning

#Introduction

A healthy Zabbix today isn't necessarily healthy when the host count triples. Episode 17 covers the two disciplines that keep Zabbix strong as it grows: performance tuning to maximize existing capacity, and capacity planning to estimate needs before they arrive.

The two work together. Tuning optimizes parameters without adding hardware; capacity planning tells you when tuning is no longer enough and hardware must be added. In this episode you'll also learn to monitor Zabbix with Zabbix — a practice that ensures the monitoring server doesn't become the last blind spot in your infrastructure.

#Key Performance Parameters

#Caches in zabbix_server.conf

The Zabbix server stores data in various in-memory caches. The cache size determines how much data can be handled before it's written to the database:

• CacheSize: the configuration cache — hosts, items, and triggers.
• HistoryCacheSize: the buffer of history waiting to be written to the database.
• TrendCacheSize: the trend data buffer.
• ValueCacheSize: the item value cache to speed up queries.

CacheSize=128M
HistoryCacheSize=64M
ValueCacheSize=64M

The right size depends on data volume. Monitor cache usage via internal checks — if a cache is consistently near full, raise its value.

#Worker Counts

The number of worker processes determines collection parallelism. The parameters most often adjusted:

• StartPollers: workers for passive checks and SNMP.
• StartTrappers: workers that accept data from active agents and proxies.
• StartPollersUnreachable: dedicated workers for unreachable hosts.

zabbix_server --config /etc/zabbix/zabbix_server.conf | grep -i start

The command zabbix_server --config ... | grep -i start shows the current worker parameter values. A simple rule: add workers until there's no data backlog — indicated by a queue metric near zero — then stop.

#Database Tuning

The database is the most common bottleneck on large Zabbix deployments. A few adjustments that often have a big impact:

• Make sure the history and trends tables sit on fast enough disk (SSD).
• Adjust the MySQL buffer pool (innodb_buffer_pool_size) or PostgreSQL shared_buffers so a large share of the database lives in memory.
• Consider TimescaleDB from episode 12 for automatic partitioning.

MySQL: innodb_buffer_pool_size ~ 70% of allocated RAM
PostgreSQL: shared_buffers ~ 25% of allocated RAM

The numbers above are starting points, not final formulas. Measure real performance and adjust based on observation, not assumptions.

#Capacity Planning

#The NVP Formula

NVP (new values per second) is the most important metric for Zabbix sizing — the number of new values arriving per second. The formula is simple:

NVP = (number of collected items) / (average interval in seconds)

Example: 6000 items with an average interval of 60 seconds produce 100 NVP. If half of them use active checks, the effective NVP drops — active agents are more efficient because the server doesn't wait for connections.

#Sizing for 1K, 10K, and 50K Hosts

The official Zabbix hardware guide provides estimates per NVP tier:

10 thousand NVP  → 2 vCPU, 4 GB RAM
30 thousand NVP  → 4 vCPU, 8 GB RAM
70 thousand NVP  → 8 vCPU, 16 GB RAM
160 thousand NVP → 16 vCPU, 32 GB RAM

For 1000 hosts with an average of 500 items per host at 60-second intervals, the NVP is around 8000 — still comfortable in the first tier. 10 thousand hosts jump to tens of thousands of NVP and need several servers or distributed proxies.

#Monitoring Zabbix Itself

#Internal Checks and Queue

Zabbix provides internal items to monitor itself: zabbix[queue] shows unprocessed data, zabbix[history] and zabbix[trends] measure database write speed, and zabbix[process,<type>] monitors workers.

zabbix[queue]                 → the queue of data waiting to be processed
zabbix[process,history]       → history write speed per second
zabbix[stats,127.0.0.1,10051] → overall server statistics

The metric zabbix[stats,127.0.0.1,10051] gives a picture of server health in a single item. Combine internal items with the official "Zabbix Server" template — this template already contains items and triggers for monitoring the server itself.

#Steps Before Adding Hardware

The suggested order when performance degrades:

• Monitor queue and cache utilization to find the bottleneck.
• Adjust caches and worker counts first.
• Tune the database and consider partitioning.
• Add hardware or nodes only after tuning isn't enough.

#Closing

Episode 17 made Zabbix ready to grow: caches and workers adjusted via zabbix_server.conf, the database tuned to avoid bottlenecks, NVP as the basis for capacity planning, and internal checks keeping Zabbix able to monitor itself.

Key takeaways:

• Caches buffer data in memory; their size follows data volume.
• StartPollers and StartTrappers determine collection parallelism.
• NVP = new values per second, the basis of sizing calculations.
• The database is the main bottleneck; allocate buffer pool according to RAM.
• Attach the Zabbix Server template so Zabbix monitors itself.

In the next episode 18 we'll discuss advanced templates and integrations — official webhook integrations like PagerDuty, Slack, and Jira, pushing data to Prometheus, custom agent2 plugins, and YAML template import-export.