Learn Elasticsearch - Aggregations - Analytics & Data Summarization

#Introduction

Search answers "which documents match". But businesses don't ask that directly — they ask "how much total sales this month?", "which category sells best?", "what's the average price per category?". The answer isn't a set of documents, but aggregations: statistical computations that run directly inside Elasticsearch, without moving data to the application. Episode 9 covers aggregations — one of the features that makes Elasticsearch unique: metrics (sum, avg, min, max, stats, cardinality, percentiles), buckets (terms, histogram, date_histogram, range, nested), and pipeline (moving average, derivative, cumulative sum, bucket sorting). With these you can build analytics dashboards without extra tools.

#Aggregation Fundamentals

Aggregations are sent together with the query in the aggs block via POST /produk/_search. All hits that match the query are grouped, and each aggregation computes its result. The simplest example:

{ "size": 0, "aggs": { "rata_harga": { "avg": { "field": "price" } } } }

Notice "size": 0 — we don't need the documents, only the aggregation results. This is the standard pattern: the query determines what data, the aggregation determines what gets computed.

#Metrics Aggregations

Metrics compute values from all matching documents:

{
  "size": 0,
  "aggs": {
    "harga_stats": { "stats": { "field": "price" } },
    "merek_unik": { "cardinality": { "field": "brand.keyword" } },
    "harga_percentiles": { "percentiles": { "field": "price" } }
  }
}

cardinality is useful for "unique users per day" metrics without storing all IDs; percentiles answers "what price is above 90% of other products" — the basis of price filter features in e-commerce.

#Bucket Aggregations

Buckets group documents into categories — then another aggregation can run inside each bucket. This is Elasticsearch's GROUP BY concept.

#Terms Aggregation

terms groups by unique values of a keyword field:

{
  "size": 0,
  "aggs": { "per_kategori": { "terms": { "field": "category.keyword", "size": 10 } } }
}

The result is a list of categories ordered by document count. Only the top 10 are returned by default — set a larger size if you need more.

#Histogram and Date Histogram

histogram groups numerics into fixed intervals; date_histogram does the same for time ranges — the most important aggregation for time-series data:

{
  "size": 0,
  "aggs": {
    "per_bulan": {
      "date_histogram": { "field": "@timestamp", "calendar_interval": "month" },
      "aggs": { "total_penjualan": { "sum": { "field": "amount" } } }
    }
  }
}

This is the standard pattern for "sales per month" charts in dashboards — query and aggregation run in a single request.

#Range Aggregation

range groups by intervals you define yourself, for example price segments:

{
  "size": 0,
  "aggs": {
    "segmen_harga": {
      "range": { "field": "price", "ranges": [ { "to": 50000 }, { "from": 50000, "to": 100000 }, { "from": 100000 } ] }
    }
  }
}

#Nested Aggregations

Aggregations can be stacked — buckets inside buckets. For example: product count per category, and within each category, the average price:

{
  "size": 0,
  "aggs": { "per_kategori": { "terms": { "field": "category.keyword" }, "aggs": { "rata_harga": { "avg": { "field": "price" } } } } }
}

If the field is of type nested, use a dedicated nested block so the aggregation respects the boundaries between objects — in episode 5 we discussed why this matters.

#Pipeline Aggregations

Pipeline aggregations work on top of the results of other buckets, not raw documents. This is what enables time-series analysis.

{
  "size": 0,
  "aggs": {
    "per_bulan": {
      "date_histogram": { "field": "@timestamp", "calendar_interval": "month" },
      "aggs": { "total_penjualan": { "sum": { "field": "amount" } }, "rata_gerak": { "moving_avg": { "buckets_path": "total_penjualan" } }, "perubahan": { "derivative": { "buckets_path": "total_penjualan" } } }
    },
    "kumulatif": { "cumulative_sum": { "buckets_path": "per_bulan>total_penjualan" } }
  }
}

• moving_avg — smooths out fluctuations, great for spotting trends behind noisy data.
• derivative — computes the rate of change between buckets (sales up/down).
• cumulative_sum — gradual accumulation, e.g. the year-to-date cumulative total.
• Bucket sorting — order bucket results by their child metric, not just doc count, via the bucket_sort pipeline.

#Significant Terms

significant_terms is an interesting bucket aggregation: it finds field values that are highly unusual compared to the background of the entire dataset. For example: among the products sold this week, which brand's share jumped compared to normal trends? This is the basis of "trending" features and simple anomaly detection.

{
  "size": 0,
  "query": { "range": { "@timestamp": { "gte": "now-7d" } } },
  "aggs": { "merek_trending": { "significant_terms": { "field": "brand.keyword" } } }
}

The result is a list of brands that are statistically the most "prominent" in this data subset — without needing to know the definition of trending in advance.

#Common Mistakes

1. Forgetting size: 0. Otherwise Elasticsearch still returns hit documents you don't need — wasting bandwidth and memory.

2. Aggregating a text field. Aggregations need doc_values — use the keyword sub-field, not text.

3. terms with the default size. The default of 10 buckets can be misleading. Set size explicitly to match your needs.

4. Date histogram interval doesn't match data granularity. calendar_interval and fixed_interval serve different purposes — choose accordingly, don't just slap "1d" on everything.

5. Pipeline aggregation without a correct buckets_path. The path must point at the child aggregation hierarchically, e.g. per_bulan>total_penjualan.

#Conclusion

In episode 9 you mastered aggregations: metrics (sum, avg, min, max, stats, extended_stats, cardinality, percentiles, percentile_ranks), buckets (terms, histogram, date_histogram, range, nested, significant_terms), and pipeline (moving_avg, derivative, cumulative_sum, bucket_sort) — complete with the nested-bucket pattern for analytics dashboards.

Key takeaways:

• Use size: 0 when you only need aggregation results, not documents.
• Metrics for statistical values, buckets for grouping.
• date_histogram is the foundation of time-series analysis.
• Buckets can nest; pipelines work on top of bucket results.
• significant_terms finds statistically unusual values.

Now data can be stored, searched, and analyzed. Time to think about the data lifecycle. In episode 10 we'll cover Index Lifecycle Management (ILM): the hot-warm-cold-frozen architecture, rollover, ILM phases, index priority, allocation filtering, creating policies, and searchable snapshots for the frozen tier. See you there!