Learn Docker - Production-Grade Architecture Case Study & Checklist

#Introduction

After building a smooth development workflow in episode 21 — bind mounts, hot reload, dev containers — it's time to assemble everything you've learned since episode 0 into one whole: a real production-grade architecture. So far we've discussed concept by concept: multi-stage Dockerfiles (episode 7), volumes (episode 8), networking (episode 9), Compose (episodes 10-11), security (episode 13), observability (episode 15), and reverse proxies (episode 20). This episode is where they all meet and work together.

Many people can run docker compose up -d for a simple application, but the working world demands more: services that know when they're healthy, resources that are limited so one service doesn't consume everything, networks that are separated so the API doesn't leak to the public, and monitoring that alerts before damage spreads. This episode gives two things at once: a complete case study you can imitate, and a production-readiness checklist you can use to audit your own deployment. At the end, we also dissect the troubleshooting problems that most often wake engineers in the middle of the night.

#Main Discussion

#The Case Study Architecture

We'll build a simple online store platform with a microservices architecture commonly found in the real world. Six main components, each teaching one distinct lesson:

                    Internet (80/443)
                         |
                     [ Traefik ]
                    /      |      \
            app.example  api.example  monitor.example
                   |        |            |
              [frontend] [api]       [grafana]
              Next.js    Node/Go       metrik
              port 3000  port 8000   port 3000
                   |        |
                   |    [redis] (cache)
                   |        |
                   |   [postgres] (volume)
                   |
        [cadvisor] ---> [prometheus] ---> [grafana]

• Frontend (Next.js) — on the frontend network with Traefik.
• API (Node.js/Go) — on both networks: frontend (accepted by Traefik) and backend (talks to the database/cache).
• PostgreSQL — persistent volume, only on the backend network.
• Redis — cache, only on the backend network.
• cAdvisor + Prometheus + Grafana — the monitoring network, collecting metrics from the Docker host and the API service.

Note the security pattern: the database is never on the same network as Traefik. The only path to PostgreSQL is through the API on the backend network. This is network segmentation in practice — a concept that felt abstract in episode 9, and now its importance is clear: even if the frontend is breached, the attacker has no direct path to the database.

#The Complete Production Compose

Here's the compose.prod.yaml file that summarizes everything we've learned. Note not just what's defined, but why each part exists:

services:
  traefik:
    image: traefik:v3.1
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./data/acme.json:/letsencrypt/acme.json
    command:
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --certificatesresolvers.letsencrypt.acme.tlschallenge=true
      - --certificatesresolvers.letsencrypt.acme.email=ops@example.com
      - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
    networks:
      - frontend
 
  frontend:
    image: ghcr.io/arman/web-prod:1.4.0
    restart: always
    read_only: true
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.frontend.rule=Host(`app.example.com`)"
      - "traefik.http.routers.frontend.tls.certresolver=letsencrypt"
      - "traefik.http.services.frontend.loadbalancer.server.port=3000"
    depends_on:
      api:
        condition: service_healthy
    environment:
      - NEXT_PUBLIC_API_BASE=https://api.example.com
    networks:
      - frontend
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://localhost:3000/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: "0.50"
 
  api:
    image: ghcr.io/arman/api-prod:1.4.0
    restart: always
    read_only: true
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    env_file:
      - .env.prod
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.rule=Host(`api.example.com`)"
      - "traefik.http.routers.api.tls.certresolver=letsencrypt"
      - "traefik.http.services.api.loadbalancer.server.port=8000"
    networks:
      - frontend
      - backend
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://localhost:8000/health"]
      interval: 15s
      timeout: 5s
      retries: 5
      start_period: 30s
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: "0.75"
        reservations:
          memory: 256M
 
  postgres:
    image: postgres:16-alpine
    restart: always
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=${POSTGRES_DB}
    networks:
      - backend
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
      interval: 10s
      timeout: 5s
      retries: 5
    deploy:
      resources:
        limits:
          memory: 1G
          cpus: "1.0"
 
  redis:
    image: redis:7-alpine
    restart: always
    command: redis-server --appendonly yes
    read_only: true
    volumes:
      - redisdata:/data
    networks:
      - backend
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 5
    deploy:
      resources:
        limits:
          memory: 256M
          cpus: "0.25"
 
  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    restart: always
    privileged: true
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    networks:
      - monitoring
 
  prometheus:
    image: prom/prometheus:v2.53.0
    restart: always
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - promdata:/prometheus
    networks:
      - monitoring
      - backend
    deploy:
      resources:
        limits:
          memory: 256M
 
  grafana:
    image: grafana/grafana:11.2.0
    restart: always
    volumes:
      - grafanadata:/var/lib/grafana
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.grafana.rule=Host(`monitor.example.com`)"
      - "traefik.http.routers.grafana.tls.certresolver=letsencrypt"
      - "traefik.http.services.grafana.loadbalancer.server.port=3000"
    networks:
      - monitoring
      - frontend
    deploy:
      resources:
        limits:
          memory: 256M
 
volumes:
  pgdata:
  redisdata:
  promdata:
  grafanadata:
 
networks:
  frontend:
  backend:
  monitoring:

The Prometheus scraping configuration is stored separately. Note that Prometheus can reach api:8000 because it's on the backend network, and reaches cadvisor:8080 via the monitoring network:

global:
  scrape_interval: 15s
  evaluation_interval: 15s
 
scrape_configs:
  - job_name: "cadvisor"
    static_configs:
      - targets: ["cadvisor:8080"]
 
  - job_name: "api"
    metrics_path: /metrics
    static_configs:
      - targets: ["api:8000"]

Note the important patterns in compose.prod.yaml above:

• A healthcheck on every service — not just a formality. depends_on with condition: service_healthy makes the startup order follow readiness, not just "the container is running". The API waits until PostgreSQL and Redis genuinely accept connections before starting.
• Resource limits — every service has memory and CPU limited via deploy.resources.limits. A single memory leak in the API won't cripple the host: the container will be killed by the kernel (OOM), not make the whole server swap to death. reservations on the API guarantee a baseline.
• restart: always — Docker will restart a crashed container, unless stopped manually. Without this, a single exception can kill a service forever.
• Security hardening — read_only: true (read-only root filesystem), cap_drop: ALL (drop all Linux capabilities), and no-new-privileges applied to the frontend, API, and Redis. These are the episode 13 practices. Redis can be read_only because its data is on a volume; only the data directory needs to be writable.
• env_file: .env.prod — credentials are never hardcoded in the image (which anyone can extract) nor in a compose file that goes into git. Database values take ${VAR} from a local .env file whose real contents only exist on the server.
• Network topology — three separate networks; each service only gets the access it truly needs.

Also note two easily-missed details. The frontend doesn't need to be on the backend network at all — it only knows the public API URL via NEXT_PUBLIC_API_BASE, so even if the frontend is successfully breached, the attacker has no network path to PostgreSQL or Redis. Conversely, prometheus deliberately joins the backend network so it can scrape api:8000 metrics — it's the only "outside guest" allowed into that network. This smallest-sufficient-network pattern is the concrete form of the least privilege principle at the network level, and far easier to maintain than inter-application firewalls.

#Production Readiness Checklist

Let's translate the case study above into a checklist you can use to audit any deployment. This isn't a theoretical list — every item was born from real incidents in the field:

1. Security — Run as non-root (USER node in the Dockerfile), cap_drop: ALL, read_only: true (add tmpfs for /tmp), no secrets in the image, and use a minimal base image (Alpine/distroless). Audit with Trivy in CI (episode 14).
2. Multi-stage builds — The production image only contains runtime and artifacts, not compilers/dev dependencies (episode 7).
3. Resource limits — deploy.resources.limits for memory and CPU on every service; add pids-limit to stop fork bombs.
4. Healthchecks — Every service has a healthcheck; dependencies between services use condition: service_healthy.
5. Log rotation — Configure log-driver and log-opts in daemon.json (max-size: "10m", max-file: "3"), or a centralized logging driver (episode 15).
6. Volume backups — Data on named volumes, not the writable layer. Schedule pg_dump backups via a temporary container and store them outside the host.
7. Zero-downtime strategy — The rolling restart pattern (episode 20) or a Swarm update; make sure the application shuts down gracefully.
8. Monitoring — cAdvisor + Prometheus + Grafana (as above), with alerting for disk, CPU, and service health.

#Troubleshooting Production Problems

Even with a complete checklist, incidents still happen. What distinguishes a senior engineer is speed and calm in diagnosis. Here are five of the most common problems and how to dissect them:

1. Container restart-loop (Docker's CrashLoopBackOff). Symptom: docker ps shows a Restarting status with a restart count climbing continuously. First step: docker logs --tail 50 <container> to see the last error — logs are the most honest witness. If the logs are empty or truncated, check the exit code and error message: docker inspect -f '{{.State.ExitCode}} {{.State.Error}}' <container>. Exit code 137 means killed by OOM (raise the memory limit), 1/2 is usually an application error (wrong env, failed migration), and 127 means the command isn't found in the image. Limit the restart cycle with restart: on-failure:5 so the host doesn't degrade.

2. docker inspect + logs as the diagnosis sequence. Rule of thumb: always start with docker logs (what the application prints), continue to docker inspect (metadata: exit code, env, mounts, networks), then docker exec only if you need to see the runtime state. Diagnose from the outside in, don't immediately blame the code.

3. A full disk. Classic symptom: containers stop writing logs, the database errors "No space left on device". Check docker system df — it separates usage into images, containers, local volumes, and build cache. The most common causes: unrestricted logs (remember checklist #5) and unused images. Clean up with docker system prune -af for images/containers, or --volumes if you're sure the data isn't needed. Database volume size needs special attention: df -h /var/lib/docker reveals whether named volumes are filling the disk.

4. Port conflict. Symptom: docker compose up -d fails with "port is already allocated". Find the culprit with ss -tlnp | grep :8080 or docker ps to see another container using that port. Often it's a leftover container from an old experiment — stop it with docker rm -f if truly unused, or move the new service's port mapping.

5. DNS resolution between containers. Symptom: the application errors "getaddrinfo ENOTFOUND postgres" even though everything looks up. The cause is almost always one of two: (a) a mistyped service name, or (b) containers on different networks. Check each container's networks with docker inspect -f '{{json .NetworkSettings.Networks}}' <container>, and make sure both share the same network. Test connectivity directly: docker exec api getent hosts postgres — if it returns an IP, DNS works; if not, the containers aren't network-sharing.

6. 502/504 Bad Gateway. Symptom: the browser shows a 502 while the container looks "running normally". The cause is almost always in the proxy→service communication layer, not the application itself: a mismatched service name, an internal port in the Traefik label or NGINX upstream that doesn't match the port the application opens, or the service isn't ready to accept connections when the first request arrives. Check docker logs on the destination service — if no request is recorded at all, the proxy isn't finding it: check the traefik.http.services.*.loadbalancer.server.port label or the upstream block, and make sure the application binds to 0.0.0.0, not just localhost — many frameworks only listen on loopback by default, so connections from the proxy (arriving via the container IP) are rejected.

#The Compact Checklist

As a technical closing, here's a one-glance summary:

#Conclusion

In this episode 22 we assembled the entire series' material into a real production-grade architecture: a frontend and API behind Traefik with automatic TLS, PostgreSQL and Redis on an isolated backend network, and cAdvisor, Prometheus, and Grafana for observability — all in one compose.prod.yaml full of healthchecks, resource limits, hardening, and network segmentation. We also turned that experience into an eight-point production-readiness checklist and dissected the five most common production problems: restart loops, full disks, port conflicts, DNS, and the correct diagnosis sequence.

Core takeaways:

• Network segmentation — the database is never on the same network as the public proxy.
• Healthcheck + depends_on healthy is the foundation of a correct startup order.
• Resource limits prevent one service from crippling the whole host.
• The 8-point checklist is a reusable audit tool for any deployment.
• Problem diagnosis: docker logs → docker inspect → docker exec, from the outside in.

In the next episode, episode 23, we'll pull back the bottom layer: Image Internals — how images are actually stored as stacks of read-only layers on the Union File System, the role of Copy-on-Write, the overlay2 storage driver, and advanced commands like docker image save/load, export/import, and why docker commit is rarely recommended. See you in episode 23!