Learn Observability with the LGTM Stack - SLI, SLO, SLA & Error Budgets

#Introduction

How reliable is your service? This question can't be answered with feelings. The SRE (Site Reliability Engineering) world answers it with numbers: SLI measures, SLO targets, SLA binds, and the error budget gives room to move.

This episode covers that SRE foundation, how to define the right SLIs, SLO implementation with dashboards and burn rate alerts, error budget management, and using the Grafana SLO plugin.

#SRE Foundation

#Four Basic Concepts

• SLI (Service Level Indicator): a real measurement of service reliability — for example the percentage of successful requests.
• SLO (Service Level Objective): the target set on an SLI — for example 99.9 percent.
• SLA (Service Level Agreement): an agreement with customers, usually looser than the SLO.
• Error budget: failure tolerance = 100 percent minus the SLO.

SLI (measure) -> SLO (target) -> SLA (promise)
error budget = 100% - SLO

The SLI (measure) -> SLO (target) -> SLA (promise) pattern shows the logical order — measure first, then target, only then promise.

#Defining SLIs

#Request-Based SLIs

Request-based SLIs compute the ratio of good events to the total:

• Availability: the proportion of successful requests to total requests.
• Latency: the proportion of requests that complete faster than a threshold, for example 300ms.

sum(rate(http_requests_total{status!~"5.."}[5m])) /
sum(rate(http_requests_total[5m]))

The query above computes the ratio of non-5xx requests — exactly the availability SLI form from episode 19.

#Windows and Data Sources

• Windows-based SLIs: computing the percentage of time the condition was good against total time.
• Measurement strategy: make sure the metric used truly represents the user experience.
• Data source selection: PromQL from Mimir is the primary choice for request-based SLIs.

name: checkout.availability
type: ratio
numerator: sum(rate(http_requests_total{status!~"5.."}[5m]))
denominator: sum(rate(http_requests_total[5m]))

The type: ratio definition above is the most common pattern for an availability SLI.

#SLO Implementation

#Setting Targets

Choose realistic targets, not just 99.99 percent:

• 99 percent: roughly 7 hours of downtime per month — suitable for internal services.
• 99.9 percent: about 43 minutes per month — the common standard for public services.
• 99.99 percent: about 4 minutes per month — expensive and hard.

#Time Windows and Dashboards

SLOs are evaluated within time windows — a rolling 30 days or a calendar month. SLO dashboards show the actual SLI, the target, and the remaining error budget.

#Burn Rate Alerts

Burn rate is how fast the error budget is being consumed. Burn rate 1 means the budget runs out exactly at the end of the period; burn rate 14 means it runs out in 2 days.

sum(rate(http_requests_total{status=~"5.."}[1h])) /
sum(rate(http_requests_total[1h])) / (1 - 0.999)

The value (1 - 0.999) is the error budget for a 99.9 percent SLO. If the query result above exceeds 14, the error budget is burning too fast.

#Error Budget Management

#Calculation and Decisions

• Calculating error budgets: 100 percent minus the SLO, in time or request form.
• Budget consumption tracking: monitor budget consumption on dashboards.
• Budget-based decision making: while budget remains, teams are free to ship features; when it runs out, focus shifts to reliability.

The error budget turns the "can we release or not" debate into an agreed, numbers-based decision.

#Balancing Speed and Reliability

An SLO that's too high stops innovation; one that's too low destroys trust. The error budget gives measurable negotiation room between product and technical teams.

#Grafana SLO Plugin

Grafana provides the official SLO plugin that simplifies management:

• Configuration: defining SLOs based on PromQL queries.
• Visualization: showing SLI, error budget, and burn rate in a single panel.
• Alerting integration: burn rate alerts plug directly into Grafana Alerting.
• Reporting: periodic SLO status reports for management.

#Closing

In episode 20 you understood the foundation of SLI, SLO, SLA, and error budgets, how to define request-based and windows-based SLIs, SLO implementation with realistic targets, dashboards, and burn rate alerts, error budget management, and using the Grafana SLO plugin.

The key takeaways:

• SLI measures, SLO targets, SLA promises.
• The error budget is 100 percent minus the SLO.
• Choose realistic targets according to business needs.
• Burn rate alerts warn about budgets burning too fast.
• Error budgets become the basis for measurable release decisions.

In the next episode 21 we'll discuss incident response and postmortems — incident detection, triage and escalation flows, debugging with observability, a blameless postmortem culture, and incident metrics like MTTD and MTTR. All the tools you've built will be tested at this very moment.