Learn Docker - Image Internals: Layers, Union File System & Image Storage

#Introduction

After assembling a complete production-grade architecture in episode 22, this time we pull back the very bottom layer of it all. For 22 episodes you've built images with Dockerfiles, written RUN, COPY, and CMD, then called docker build and docker push as if it were all magic. But have you ever asked: what is an image actually made of? The short answer — a stack of read-only layers on top of a Union File System — turns out to explain almost every Docker behavior that seems strange.

Why does changing the order of Dockerfile instructions dramatically change build speed? Why does writing a file into a container "grow" that container but not change the image? Why is storing a database on a volume better than in a container's writable layer? Why don't images derived from the same base double their size on the host? All these questions share one answer: layer structure. This episode dissects that answer, touches the overlay2 storage driver, and closes with advanced commands for moving images between hosts — along with one classic anti-pattern you must know.

#Main Discussion

#Image Anatomy: A Stack of Read-Only Layers

An image isn't one intact file block. An image is a stack of layers, and each layer is a filesystem snapshot representing the result of one Dockerfile instruction. What makes this work is the Union File System: the Linux kernel's ability to merge several directories (layers) into one apparent directory tree, as if stacking sheets of tracing paper on top of each other.

+--------------------------------------+
| Layer 4: CMD ["node", "server.js"]   |  <- layer teratas, paling sering berubah
+--------------------------------------+
| Layer 3: COPY . /app                 |
+--------------------------------------+
| Layer 2: RUN npm ci --omit=dev       |
+--------------------------------------+
| Layer 1: ENV NODE_ENV=production     |
+--------------------------------------+
| Layer 0: FROM node:22-alpine         |  <- layer dasar (base image)
+--------------------------------------+

What the application inside a container sees is the merged view of all layers — it doesn't know that server.js actually comes from layer 3, or that the node binary comes from layer 0. To the processes inside the container, it's one complete filesystem. But to Docker, each layer remains a separate entity that can be shared and cached.

This produces two decisive facts. First, layers are immutable: once a layer is created, its contents never change; updating a file means creating a new layer on top. Second, layers are shared between images: if two images both use node:22-alpine as a base, both images use the exact same layers on disk — that's why a hundred images from the same base don't make the disk explode.

#How a Dockerfile Becomes Layers

Every instruction that modifies the filesystem produces a new layer: RUN, COPY, ADD, and ENV/WORKDIR (which record metadata). Non-filesystem instructions like CMD, ENTRYPOINT, and EXPOSE only write metadata and add no disk size — note in the history output later, their size is 0B. The instruction order determines the amount of change between layers, which is the key to the caching strategy we learned in episode 6: unchanged layers are reused, changed layers — and all layers above them — are rebuilt.

FROM node:22-alpine          # layer A (base)
WORKDIR /app                 # layer B (metadata: working dir)
COPY package*.json ./        # layer C (file dependency)
RUN npm ci --omit=dev        # layer D (node_modules)
COPY . .                     # layer E (source code)
CMD ["node", "server.js"]    # metadata, tanpa layer baru

This is why the order COPY package*.json before npm ci before COPY . . is the golden pattern: when you only change the source code, layers A-D stay valid from cache, and only layer E is rebuilt. Conversely, if COPY . . is written before npm ci, changing one line of code forces npm ci (the slowest process) to rerun every time.

#Peeking at Layers: docker image history & docker inspect

The most direct way to see the layer structure is docker image history. Each row is one layer, ordered from newest:

docker image history my-app:1.0
docker image history --no-trunc my-app:1.0

IMAGE          CREATED       CREATED BY                                    SIZE
6f2e9b0d4c1a   5 min ago     CMD ["node" "server.js"]                      0B
5c1a8f2e6d3b   5 min ago     COPY . /app                                   14.2MB
9b7e4d2a8c1f   6 min ago     RUN /bin/sh -c npm ci --omit=dev              92MB
3f6a1e9c7b4d   6 min ago     COPY package*.json ./                         2.1MB
d0b4a8c2e6f1   8 min ago     WORKDIR /app                                  0B
<missing>       2 weeks ago   /bin/sh -c #(nop) ENV NODE_ENV=production     0B

Note two things. First, the 0B size on CMD — metadata instructions genuinely take no space. Second, the <missing> in the IMAGE column on the base layer: that layer isn't stored locally as part of this image (the base image is pulled separately and shares layers with other images), but its hash is still recorded. That last row is what proves your image is just a "thin layer" on top of node:22-alpine.

To see layer hashes explicitly, docker inspect reveals the RootFS.Layers array — the list of SHA-256 digests forming the image:

docker inspect --format '{{json .RootFS}}' my-app:1.0

{"Type":"layers","Layers":[
  "sha256:1f4e6f2a...",
  "sha256:2b8a9c3d...",
  "sha256:3c5d7e9f..."
]}

Each digest is unique and content-deterministic: images with the same layer set must have the same RootFS, regardless of where they were pulled from. This is the basis of the immutable digest we covered in episode 12.

#The Storage Driver: overlay2 & Copy-on-Write

Now we get to the part that's really the "engine" behind images. Modern Linux Docker uses overlay2 as the default storage driver. overlay2 works in /var/lib/docker/overlay2/ with three important directories per layer:

• lowerdir — the stack of read-only image layers (all but the topmost).
• upperdir — the container's writable layer; where all changes made by processes inside the container go.
• merged — the result of merging lowerdir and upperdir, which processes see as one filesystem.

ls -la /var/lib/docker/overlay2/<layer-hash>/

The concept that ties it all together is Copy-on-Write (CoW). When a process inside a container writes a file:

1. If the file comes from a lower layer (lowerdir) and hasn't been changed yet, overlay2 copies the file to upperdir, then writes the changes there. The original layer stays intact.
2. All subsequent changes to the same file are written straight to upperdir, without copying again.
3. If the container is deleted, upperdir is discarded; lowerdir (the image) stays intact and is shared with other containers.

This is why two containers from the same image can "write" independently without harming each other, and why writing to a container layer never changes the image. But CoW has a dark side you must understand: write amplification. When a file in lowerdir is changed, its entire blocks are first copied upward. Imagine a database writing thousands of times per second to a gigabyte-sized data file on the writable layer — every small write can trigger a large block copy, and upperdir balloons without limit because it's never squashed. This is the definitive answer to the episode 8 question: databases must use volumes, not the writable layer — volumes bypass overlay2 entirely and write directly to the host disk.

#overlay2 vs fuse-overlayfs vs Volumes

The practical conclusion: use overlay2 (default) for images and runtime, accept the rootless trade-off when security requirements demand it, and always put large, frequently written data on volumes. Writing transactional data to the writable layer isn't just a performance problem, but a durability problem: the writable layer disappears with the container.

#Moving Images Without a Registry: save & load

Sometimes you need to move images between hosts without a registry — for example a production server isolated from the internet, or manual transfer when the network is down. docker save packages an image's entire layers into one tar archive (with all metadata and history), and docker load unpacks it back:

# Kemas image menjadi satu file tar
docker save -o my-app-1.0.tar ghcr.io/arman/my-app:1.0.0
 
# Salin ke host tujuan (tanpa registry)
scp my-app-1.0.tar user@server:/tmp/
 
# Bongkar di host tujuan
docker load -i my-app-1.0.tar
 
# Verifikasi — image sekarang ada, dengan semua layer yang sama
docker images ghcr.io/arman/my-app

Since the format is a tar archive, you can also compress it to speed up transfer: docker save my-app:1.0 | gzip > my-app.tar.gz. What distinguishes save/load from merely copying files: the archive carries the complete layer structure, so the image loaded on the destination host is identical to the source — including history and digests. This is a safe, auditable path for offline image transfer.

#export/import: A Filesystem, Not an Image

Alongside save/load there's the docker export and docker import pair — and they're often mixed up. docker export exports the filesystem of a container (including its writable layer!) into a single tarball, without image metadata. docker import unpacks it back into a new image consisting of only one layer, without history:

# Ekspor filesystem container (termasuk data yang diubah runtime)
docker export my-container > rootfs.tar
 
# Import sebagai image satu-layer (history & metadata image hilang)
docker import rootfs.tar my-app:imported

The clearest example distinguishing them: docker save ubuntu > ubuntu.tar produces an archive that can be loaded back into an ubuntu image with intact history. Meanwhile, docker export from an ubuntu container produces a raw filesystem tarball — importing it yields a new image without a base, without layer history, and as one combined layer. When is export useful? For moving data out of a container (e.g. a filesystem snapshot before discarding it), not for distributing applications.

#docker commit: Why It's an Anti-Pattern

Last, the myth most tempting to beginners: docker commit — turning a running container into a new image. At first glance it looks like an instant solution: "run manual commands inside the container until it works, then freeze it into an image".

docker commit my-container my-app:manual

Why is this practice dangerous and not recommended:

1. Ephemeral state gets frozen in. A running container often has temporary files, caches, logs, or runtime data in its writable layer. All of it goes into the image. The result is a non-reproducible image: no one knows why and what is inside it.
2. History is lost. A committed image has no Dockerfile trail. You can't audit it, update dependencies, or rebuild it in a controlled way.
3. No version control. Unlike a Dockerfile that can be reviewed in git, a committed image is a black box stored on only one machine. Coworkers can't do code review.

This is exactly the situation of "a laptop manually configured for years": usable, but not replicable, not fixable, and certain to betray you when the machine dies. A Dockerfile is the source of truth — every production image must be born from a versioned Dockerfile. docker commit is only acceptable for a quick forensics grab (freezing a problematic container's state for analysis), not for building distribution artifacts.

#Layer & Storage Pitfalls

1. A single giant layer. One RUN copying a huge build artifact (e.g. a gigabyte-sized node_modules) produces one large layer that must be fully transferred and stored every time that layer changes. Separate what rarely changes (dependencies) from what changes often (source) with the correct COPY order — the lesson that comes around again from episodes 6 and 23.
2. Too many layers. Although small layers are efficient thanks to CoW, hundreds of layers add overhead when an image is pulled/loaded and when a container runs (each layer adds a mount point). Balance this by combining related RUN commands (e.g. apt-get update + install + clean) — within reasonable bounds, not cramming everything into one line.
3. Writing large files to the writable layer. Remember write amplification: databases, large caches, and log files on the writable layer bloat upperdir, slow writes, and disappear when the container is recreated. Always direct data to volumes.
4. Ignoring --squash. Some people suggest squashing (merging all layers into one) to shrink images. It's rarely used because it destroys caching and layer sharing — a trade-off usually not worth the thin size benefit. Correct Dockerfile ordering remains the primary solution.

#Conclusion

In this episode 23 we dissected image contents down to the deepest layer: images are stacks of read-only layers on the Union File System, each Dockerfile instruction forms one layer, and docker image history/docker inspect prove it via digests. We understood the overlay2 storage driver with lowerdir/upperdir/merged, why Copy-on-Write lets images share layers efficiently while also being a source of write amplification for databases (hence volumes are mandatory for large data), compared it with rootless fuse-overlayfs, and mastered docker image save/load for offline transfer as well as docker export/import which handle filesystems, not images. Finally, we closed with the strong reasons why docker commit is an anti-pattern that sacrifices reproducibility for a moment's convenience.

Core takeaways:

• An image = stacked read-only layers; layers are immutable and shared between images.
• Dockerfile ordering determines cache efficiency: dependencies first, source last.
• CoW is efficient for image layers, but dangerous for large data — use volumes.
• docker image save/load = move a complete image; export/import = move a filesystem.
• docker commit is an anti-pattern; a Dockerfile is the source of truth.

In the next episode, episode 24, we move from the image contents to the machine that runs them: Docker Daemon Configuration & Host Maintenance — configuring daemon.json (log driver, data-root, live-restore), cleaning the host with docker system prune, monitoring the /var/lib/docker disk, and using Docker Context to manage many hosts from one CLI. See you in episode 24!