Learn Cloud Computing - Cloud Observability, Monitoring & Logging

#Introduction

After building layered defense in episode 14 — firewalls, WAF, and encryption — one question remains: how do you know your system is actually healthy? Security protects a system from external attacks; observability protects a system from within: from code errors, dwindling resources, and unintended changes.

A reliable engineer isn't one who claims "our system never goes down", but one who can show the numbers: what percentage of availability, what average latency, and where errors flow when there's a problem. This episode equips you with the three pillars of observability, centralized log collection, alerting, and a comparison of tools on AWS, GCP, and Azure.

#Main Discussion

#The Three Pillars of Observability

Observability stands on three pillars that complement each other. Mastering all three at once — not just one — is what distinguishes modern operations from merely "monitoring":

1. Metrics — quantitative numbers in time series: CPU, memory, request rate, error rate, and latency. Concise, cheap to store, and ideal for alerting. The analogy is a speedometer: it tells you the speed is rising, without explaining why.
2. Logs — structured event records with timestamps: who called what, with what parameters, and what the result was. Rich in detail and ideal for troubleshooting. The analogy is an airplane's black box: complete, but only really opened when there's a problem.
3. Traces — the journey of a single request across many services, from the API gateway to the database. The analogy is a staff member following one customer through an entire mall to find out which stores they linger in.

The relationship between the three is simple: metrics answer is there a problem?, logs answer what happened?, and traces answer why did it happen?. Monitoring tells you something is wrong; observability tells you what and why, without having to guess.

#Metrics: Measure What Matters, Not What's Easy

For a request-serving service, the RED method pattern is a good starting point: Rate (number of requests per second), Error (number of failed requests per second), and Duration (latency distribution). For supporting infrastructure, watch CPU, memory, disk, and network.

Notice the alarm column: momentary peak values are usually not problem indicators — the danger is sustained trends. A 30-day baseline is your compass for judging whether this rise is normal or a problem signal.

#Centralized Log Collection and Alerting

Why must logs be collected centrally? Cloud instances are ephemeral — they can be replaced at any time — and containers move between nodes. Logs stored on an instance disk disappear when the instance is stopped. Manually grepping 50 servers is also impossible during an incident. The solution is central aggregation: an agent on every instance sends logs to one place, and that place can be searched, queried, and turned into alarms.

The standard flow: the agent on the instance sends logs to a centralized log service (CloudWatch Logs, Cloud Logging, Log Analytics), then metric- or log-based alarms send notifications to email, Slack, or PagerDuty when a threshold is crossed. Here's an example of retrieving instance CPU statistics via the CLI:

aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=i-0abc123def456 \
  --statistics Average \
  --period 300 \
  --start-time 2026-08-03T00:00:00Z \
  --end-time 2026-08-03T01:00:00Z

The aws cloudwatch get-metric-statistics command returns a series of data points in the requested time range. The result is JSON like this:

{
  "Datapoints": [
    {
      "Timestamp": "2026-08-03T00:30:00Z",
      "Average": 42.7,
      "Unit": "Percent"
    },
    {
      "Timestamp": "2026-08-03T00:45:00Z",
      "Average": 61.3,
      "Unit": "Percent"
    }
  ],
  "Label": "CPUUtilization"
}

These numbers are the raw material for decisions: an average of 42 to 61 percent indicates the instance is reasonably utilized. If the result consistently shows 95 percent, that's a signal to increase capacity — or evaluate the application configuration.

#Log Example: Centralized Search

When metrics say there's a problem, logs give the details. Once logs are centralized, you no longer open a terminal per server — one place is enough for searching. Here's an example CloudWatch Logs Insights query to find the latest errors:

fields @timestamp, @message, @requestId
| filter @message like /ERROR|Exception/
| sort @timestamp desc
| limit 20

This query pulls the timestamp, message contents, and request ID from all instances at once, filters for those containing ERROR or Exception, then sorts from the newest. The same pattern applies in Cloud Logging (with SQL-like queries) and Log Analytics (with KQL). What matters isn't the syntax — it's the concept: logs from the entire fleet are narrowed down to a few relevant lines within seconds.

#Traces: Following a Single Request

When one request passes through many services — API gateway, authentication, service A, service B, down to the database — which one is the slowest? Per-service logs don't answer this, because each service only sees a small piece. This is where traces come in: one trace ID marks the entire journey of a single request, and each hop is recorded as a span.

From the table above it's clear: the 120 ms in the User service isn't an application failure — it's the biggest latency contributor. Without traces, you'd be guessing which service to optimize. With traces, the decision is data-driven. Behind the scenes, tools like X-Ray, Cloud Trace, and Application Insights automatically build tables like this from requests passing through your system.

#Healthy Alerting

Alarms are useless if every alarm is treated as a false positive. Healthy practices:

• Metric-based, not raw-log-based — metrics can be computed and thresholded stably, while logs are better for searching during diagnosis.
• Add a delay — alarms should fire after a condition persists for a few minutes, not immediately on momentary fluctuation, so they don't get hysterical.
• Assign an owner — every alarm must clearly state who's responsible for follow-up, and how far the escalation path goes.

#Comparing the Big 3 Observability Services

All three major clouds built identical observability foundations, with different names:

They all think in the same pattern: metrics stored as time series, logs stored and queryable (CloudWatch Logs Insights, Cloud Logging, Log Analytics), and traces following one request across services. If you already understand this episode's concepts, switching clouds is just learning names and query syntax — the conceptual foundation is already the same.

#Conclusion

In this episode 15 you built the ability to prove system health: the three pillars of observability — metrics for alerts, logs for diagnosis, and traces for understanding cross-service flows; centralized log collection — gathering logs from ephemeral instances into one searchable place; healthy alerting — alarms with clear thresholds, delays, and owners; and a tool comparison across AWS, GCP, and Azure.

But there's a paradox you must understand: observability is only useful if the observed system can be reproduced reliably. Manually building a second staging environment through console clicks is almost guaranteed to produce a different system — and monitoring on top of that different foundation becomes misleading. The next episode, Infrastructure as Code & Cloud Automation, gives you the key to build, duplicate, and change infrastructure with the same precision as writing code.