Learn Zabbix - Data Management: History, Trends & Housekeeper

#Introduction

Up to episode 11, you focused on collecting data. Episode 12 addresses the question that starts to appear as data piles up: where are all these numbers stored, for how long, and how do we keep the database from blowing up? This is a lesson in data management — the side of Zabbix most often underrated until the infrastructure grows.

Two concepts are at the center of this episode: history as raw data and trends as aggregation. Once you understand both, you'll learn to set retention, use the housekeeper, and leverage TimescaleDB for deployments with large data volumes.

#History: Raw Data

#What Is Stored in History

History stores every value collected by an item. If an item is collected every minute, that's 1440 history rows per day. This data is the source for full-resolution graphs and trigger evaluations using functions like last() and avg().

Because of its granular nature, history takes up the most space in the database. The default Zabbix history retention is 90 days for text data, while numeric data is usually limited to a shorter period.

#Setting History Retention

Retention is configured in the item form or template, in the History parameter. The question that helps determine the value: how long do you really need raw data? A common answer is 7-31 days; beyond that, trend aggregation is sufficient for almost all analysis needs. For text and log item types, consider shorter retention — log data is usually only useful for short-term debugging, and its size grows far faster than numbers.

#Trends: Hourly Aggregation

#Why Trends Exist

Trends store hourly aggregations of item values: minimum, maximum, average, and the number of values within that hour. An item only produces 24 trend rows per day, compared to 1440 history rows. This is Zabbix's answer to storing long-term trends without eating space.

Graphs in Zabbix combine both transparently: short time ranges are displayed from history, long ranges from trends. Operators don't need to think about which is used — but understanding this matters for retention decisions. When planning storage, remember that trends retention is usually set far longer than history, so long-term graphs remain available without storing expensive raw data.

mysql -uzabbix -p zabbix -e "SHOW TABLE STATUS LIKE 'history';"
mysql -uzabbix -p zabbix -e "SHOW TABLE STATUS LIKE 'trends';"

The command SHOW TABLE STATUS LIKE 'history' shows the history table size in MySQL. Comparing the sizes of both tables is a quick way to convince yourself why trends are a must.

#Retention and the Housekeeper

#Setting Retention in Templates

The best policy places retention in the template, not on each host. That way, one change applies to all hosts linked to the template. A widely used rule of thumb: numeric history 7-31 days, trends 365 days, and text shorter than numbers.

#The Role of the Housekeeper

The housekeeper is the server process that periodically deletes data past its retention. Without the housekeeper, the database grows without end and the disk fills up quickly. The housekeeper runs inside the server; its status can be seen via the internal item zabbix[process,housekeeper].

zabbix_get -s 127.0.0.1 -k zabbix[process,housekeeper]

The command zabbix_get -s 127.0.0.1 -k zabbix[process,housekeeper] shows how many records the housekeeper has deleted. If the value stays at zero while data has passed its retention, check whether the housekeeper process is running and the retention settings are correct.

#TimescaleDB for Large Scale

#The Problem with Giant Tables

At large scale, housekeeper delete operations on giant tables become expensive. The common solution is partitioning — splitting tables per day or per month. Zabbix with PostgreSQL offers TimescaleDB integration, which manages partitions as hypertables automatically. Partitioning also helps keep index fragmentation low, because each chunk is small and isolated from the others.

#Hypertables and Automatic Management

TimescaleDB divides a hypertable into chunks per time interval. Deleting old data becomes a very fast chunk-drop operation, and queries with time filters only use the relevant chunks. A second benefit is parallel writes: new data goes into the active chunk without disturbing historical chunks.

history → chunks per 1 day → drop chunk when past retention
trends  → chunks per 1 month → drop chunk when past retention

Zabbix provides the timescaledb.sql script to convert a standard PostgreSQL database schema into hypertables. The conversion is done once and the result is permanent — existing values remain usable.

apt install -y postgresql-16-timescaledb
timescaledb-tune --conf-path /etc/postgresql/16/main/postgresql.conf

The package timescaledb-tune adjusts PostgreSQL parameters for the TimescaleDB workload automatically. After the extension is enabled in the Zabbix database, run the schema conversion script provided by Zabbix in the order given in the official documentation.

#When to Move to TimescaleDB

Moving to TimescaleDB makes sense when the database starts showing symptoms: the housekeeper takes too long, history queries are slow, or the data growth plan is clearly large. For environments with hundreds of hosts and short intervals, TimescaleDB partitioning is often the difference between a healthy database and one that drags down the whole system.

#Planning a Storage Policy

A summary of the decision-making flow:

• Determine your analysis needs: how long raw data is really used.
• Set retention in the template according to need, not on individual hosts.
• Monitor history and trends table sizes periodically.
• Consider TimescaleDB when data growth starts slowing the system down.

Good retention decisions balance operational needs against storage costs. Data kept too long wastes space; data deleted too quickly hurts during audits or trend analysis.

#Closing

Episode 12 explained Zabbix's storage architecture: history stores granular raw data, trends summarize it into hourly aggregations, the housekeeper keeps the database lean, and TimescaleDB handles large scale with hypertables.

Key takeaways:

• History is raw data; trends are compact hourly aggregations.
• Retention is set in templates, typically 7-31 days for history and 365 days for trends.
• The housekeeper periodically deletes data past its retention.
• TimescaleDB turns history and trends into chunked hypertables.
• A full backup is mandatory before converting to TimescaleDB.

In the next episode 13 we'll discuss infrastructure monitoring: network, cloud, and applications — monitoring network devices with SNMP v1/v2c/v3, ICMP ping, AWS/Azure/GCP cloud templates, HTTP and JMX for applications, and Docker and Kubernetes container integrations.