Learn Docker - Docker Daemon Configuration & Host Maintenance

#Introduction

After dissecting the internals of images in episode 23 — layers, the union file system, the overlay2 storage driver, and the Copy-on-Write concept — this time we climb one level higher: managing the Docker daemon itself. This is the episode where you shift roles from "Docker user" to "Docker operator" — the person responsible for the health of the host where Docker runs.

Why is this topic so critical? Every docker run command, every volume, and every network you learned about in earlier episodes is ultimately executed by a single process named dockerd and stored in a single directory: /var/lib/docker. When the host disk fills up because logs are never rotated, when /var/lib/docker runs out of space because images keep piling up, or when you have to move all of Docker's data to a new, larger disk — that's where the knowledge in this episode saves you. Not to mention when you manage more than one server and want to control all of them from a single terminal without SSHing into each machine.

This episode is divided into three major domains: daemon configuration through the daemon.json file, host maintenance (garbage collection and data migration), and Docker contexts for managing many hosts from a single CLI. Let's start from the foundations.

#Main Discussion

#daemon.json: The Control Center of the Docker Daemon

The Docker daemon (dockerd) isn't a program configured only through startup arguments. There's one JSON file that acts as the "home" for all daemon settings: /etc/docker/daemon.json. This is the file the daemon reads every time it starts — where you specify the log driver, storage location, communication protocol, and even restart behavior.

{
  "data-root": "/mnt/docker",
  "storage-driver": "overlay2",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "metrics-addr": "0.0.0.0:9323",
  "live-restore": true,
  "iptables": true,
  "default-address-pools": [
    { "base": "172.28.0.0/16", "size": 24 }
  ],
  "insecure-registries": ["registry.internal:5000"]
}

To use an analogy: if Dockerfile is the cooking recipe (how to build an image) and compose.yaml is the restaurant's menu (how to orchestrate many containers), then daemon.json is the kitchen and the restaurant building itself — the infrastructure where all the food is made. Many engineers spend their time perfecting the menu but forget the kitchen, and when the kitchen breaks down, the whole restaurant grinds to a halt.

One golden rule: daemon.json is read only once at daemon startup. There's a SIGHUP mechanism (kill -SIGHUP <pid_dockerd>) that reloads some options without a full restart, but not all options support it. Changes to data-root, storage-driver, and hosts require a full daemon restart — and we'll discuss why that's dangerous in the pitfalls section.

#Log Driver & Log Rotation: Stopping the Full-Disk Mystery

The number one cause of a full disk on Docker servers is container logs growing without bound. By default Docker uses the json-file driver, which writes the container's entire stdout/stderr to JSON files under /var/lib/docker/containers/. Without limits, a single application writing 1 GB of logs per day will fill the disk within weeks.

Log rotation is configured via log-opts:

• max-size: the maximum size of one log file before it's rotated (for example 10m).
• max-file: the maximum number of log files kept before the oldest is discarded (for example 3, meaning a maximum of 30 MB per container).

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

There's also the local driver, optimized for storage — binary format, no JSON auto-rotation, and more disk-efficient than json-file for logs that are rarely read by humans. The real choice in production: json-file + rotation (the default most compatible with log tooling like docker logs) or structured drivers such as journald, syslog, awslogs, or loki, which we touched on in episode 15. The principle you should carry with you: container logs are not archive storage — they're a bridge to your centralized logging system. Configure rotation as a safety net, not a replacement for a log aggregator.

#Relocating data-root: When /var/lib/docker Runs Thin

All Docker data — images, containers, volumes — lives under data-root, which defaults to /var/lib/docker. The classic problem on real servers: the / partition (where /var/lib/docker sits) is too small, while there's a large external disk mounted at /mnt or /data. The solution isn't adding fiddly symlinks; it's relocating data-root.

Identifikasi titik penuh sebelum bertindak

df -h /
df -h /mnt
sudo du -sh /var/lib/docker

The migration process is four steps whose order must not be shuffled: stop daemon → sync data → change configuration → restart.

sudo systemctl stop docker docker.socket

sudo rsync -aP /var/lib/docker/ /mnt/docker/

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<'EOF'
{
  "data-root": "/mnt/docker"
}
EOF

sudo systemctl start docker
sudo docker info --format '{{.DockerRootDir}}'

The /mnt/docker output in step 4 confirms the daemon now uses the new location. Why use rsync instead of cp? Because rsync accurately preserves owners, permissions, and symlinks — and for hundreds of GB of data it can be resumed. Never change data-root while the daemon is still running: the daemon doesn't "know" its data has moved, and running containers will suddenly lose access to their volumes.

#storage-driver: Changing the Storage Engine

storage-driver determines how image layers and a container's writable layer are represented on disk. The modern default driver is overlay2, which we covered in episode 23. When should you touch this option?

• Almost never on modern hosts. overlay2 has been the default since kernel 4.x and is the most recommended for performance and efficiency.
• Rootless Docker uses fuse-overlayfs (we cover it in episode 26) because non-root users can't mount overlays.
• If you change the driver on a host that already has data, all images and containers become "invisible" — because layer structures differ between drivers. This means switching storage drivers practically starts you from scratch, so decide at installation time.

#hosts: Unix Socket vs TCP

By default the daemon listens for CLI commands over a Unix socket at /var/run/docker.sock. Unix sockets can only be accessed by processes on the same machine, which is exactly why it's secure by default. The hosts option lets you add a TCP endpoint for remote access:

{
  "hosts": [
    "unix:///var/run/docker.sock",
    "tcp://0.0.0.0:2375"
  ]
}

Important note: when hosts is written in daemon.json, the -H option on the command line is ignored — this configuration file is authoritative.

#live-restore: Containers Stay Alive During Daemon Restart

This is one of those features that's most "underused" despite having a huge impact on uptime. By default, when the daemon is restarted (for example during a package upgrade or systemctl restart docker), every container is shut down too — because the daemon is the parent of all containers. Imagine running a routine apt upgrade and taking all your production services down with it. With live-restore: true, running containers keep running during a daemon restart, because the containerd runtime takes over temporarily and the daemon merely "takes back control" after it wakes up.

{
  "live-restore": true
}

Requirements and limitations: this feature requires an engine that uses containerd as its runtime, iptables: false isn't recommended alongside it (because restore depends on Docker's iptables chains remaining intact), and --live-restore does not keep containers alive if the daemon is stopped outright (systemctl stop) — only on restart. It's not a replacement for orchestration (that's Swarm/Kubernetes' job), but it's the right safety net for single-node hosts.

#iptables, default-address-pools & insecure-registries

Three companion options that often appear together:

• iptables: Docker uses iptables to translate published ports (-p 80:80) and isolate networks. The iptables: false option only makes sense if you manage the firewall manually yourself (for example with a custom firewalld/ufw policy) — and you must be ready to handle all port forwarding on your own. That's why it's rarely disabled.
• default-address-pools: defines the CIDR ranges for automatically created bridge networks. If your host already uses the 172.17.0.0/16 subnet for something else, conflicts arise; narrowing the pool to 172.28.0.0/16 with size: 24 keeps those conflicts in check from the start.
• insecure-registries: the list of registries allowed to communicate without TLS. For internal registries on an office network (registry.internal:5000) or registries still on HTTP, register them here. Public registries and private registries with proper TLS do not need to be on this list — adding them actually weakens security.

#Host Maintenance: Garbage Collection

Over time, the daemon accumulates garbage: old unused images, build cache, stopped containers, and orphaned networks. They silently consume tens of GB without you noticing. Docker provides the prune family of commands:

docker system prune
docker image prune -a
docker builder prune
docker volume prune
docker container prune
docker network prune

The details you must understand before hitting the button:

• docker system prune — removes stopped containers, unused networks, dangling images, and build cache. Safe for persistent data (volumes are not removed without the --volumes flag).
• docker image prune -a — removes all images not used by running or stopped containers. This is the most "aggressive" command: images just waiting to be used (for example a production version you'd roll back to) are deleted too and must be pulled again.
• docker builder prune — cleans up the BuildKit build cache. Safe, and often a lifesaver when disk is full because build caches can grow very large.
• docker volume prune — removes all volumes not used by any container. Dangerous: volumes hold persistent data (databases!). If a volume containing important data isn't currently mounted, this command destroys it without a second warning.

Before pruning, there's a far more informative way to know how much you can reclaim: docker system df.

TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          42        18        12.3GB    8.02GB (65%)
Containers      15        6         1.2GB     1.1GB (91%)
Local Volumes   9         4         6.7GB     2.4GB (36%)
Build Cache     114       -         4.6GB     4.2GB (91%)

The Reclaimable column and its percentage show how much space would come back if you used the corresponding prune command: 4.2 GB of build cache can be cleaned with docker builder prune, 8.02 GB of images with docker image prune -a, and so on. Make docker system df a weekly ritual on every production host — like checking the fuel gauge before a long trip, not waiting for the warning light to turn on mid-journey.

#Monitoring & Verifying a data-root Migration

Before deciding on a migration, you need to know where the space is going. The structure under data-root:

Peta penggunaan ruang di data-root

sudo du -sh /var/lib/docker/* | sort -rh | head -10
sudo du -sh /var/lib/docker/containers/* | sort -rh | head -5
sudo du -sh /var/lib/docker/volumes/* | sort -rh | head -5

The three directories that most often swell: containers/ (json-file logs), volumes/ (persistent data), and overlay2/ (image layers + writable layer). After migrating data-root, verify not just with docker info — run a container and make sure it can read its volumes:

sudo docker run --rm -v test-vol:/data alpine ls -la /data
sudo docker volume ls
sudo docker ps -a

If anything is missing, you still have the old data at /var/lib/docker — don't delete it right away. Keep it until every container is proven healthy, then clean it up to free the space.

#Docker Contexts: Many Hosts, One CLI

Up to this point, all your commands have run on one machine. In the working world, you'll face dozens of hosts: a local laptop, staging servers, several production servers. docker context solves this problem elegantly: one CLI, many daemons, switching with a single command.

docker context ls

NAME        TYPE    DESCRIPTION                               DOCKER ENDPOINT
default *   moby    Current DOCKER_HOST based configuration   unix:///var/run/docker.sock

The most useful context in production is docker@ssh://..., which leverages SSH you already have — no need to open TCP port 2375 and no changes on the target server, because the connection is made over normal SSH:

docker context create staging \
  --docker "host=ssh://deploy@staging.example.com"

docker context create production \
  --docker "host=ssh://deploy@prod.example.com"

docker context use staging
docker context ls
docker context inspect production

After docker context use staging, every subsequent docker command — docker ps, docker compose up, docker logs — automatically targets the staging host. That's why DevOps engineers can manage dozens of servers without opening dozens of SSH terminals. Important note: docker compose and contexts work hand in hand; make sure your compose files and volume paths refer to the target machine, not the local one.

#Common Pitfalls

1. Editing daemon.json while the daemon is running. After you change the file, the daemon won't read it automatically. If the changed option requires a restart (data-root, storage-driver, hosts), don't wait for the daemon to "notice" — restart it deliberately within a planned downtime window, or enable live-restore so containers stay alive.

2. data-root pointing at the wrong place after migration. If you write a path that doesn't exist yet or mistype it, the daemon will start with empty data — and all containers "disappear" from the new daemon's point of view, even though the data is still in the old location. Always check docker info and docker ps -a after a restart, and don't delete the old data before verification is complete.

3. prune -a deleting images you still need. docker image prune -a removes images not used by running or stopped containers. A 1.2.0 version you wanted to keep for a quick rollback gets deleted too. Use docker image prune -a --filter "until=720h" (for example "more than 30 days") to keep recent versions, or use tags and keep critical images in active use.

4. Opening a TCP port without TLS. Port 2375 without TLS is an invitation to a root shell for anyone who can reach your network. Prefer an SSH context; if you must use TCP, configure TLS.

#Conclusion

In this episode 24 you've leveled up to become a Docker operator: understanding daemon.json as the control center (log rotation, data-root, storage-driver, hosts, live-restore, iptables, address pools, insecure registries), mastering the prune command family with full awareness that docker volume prune is an operation without undo, reading docker system df to measure reclaimable space, running a data-root migration in the stop → rsync → configure → verify order, and managing many hosts through SSH-based Docker contexts without opening dangerous ports.

Core takeaways:

• daemon.json is read at startup; changes to data-root/storage-driver require a planned restart.
• Container logs without rotation are the number one disk killer — set max-size and max-file.
• Run docker system df first, then prune — and be careful with volume prune and image prune -a.
• data-root migration = stop, rsync, configure, verify, then clean up the old one.
• Docker contexts let one CLI control many hosts without manual SSH.

But a healthy machine is only half the battle. In the next episode, episode 25, we'll discuss how to constrain and tune the resources running inside that machine: Docker Resources — CPU, Memory & Performance Tuning. You'll understand why one "greedy" container can bring down an entire host, how the --memory, --cpus, --pids-limit, and --ulimit flags work behind the scenes with cgroups, and how to plan limits for databases, caches, and web servers. See you in episode 25!