Learn Docker - History, Background & Why You Need Docker

#Introduction

After securing our environment in episode 0 — basic Linux CLI skills, Docker Engine installed, and verification that docker version responds without sudo — in this episode we step back for a moment to answer a question that's rarely asked but highly decisive: where does Docker come from, and why is it needed?

This question isn't historical trivia. Understanding Docker's background answers three practical questions that will haunt you throughout your career: (1) why an application that "runs on my laptop" can fail on a server, (2) what exactly distinguishes a virtual machine from a container — and when to choose which, and (3) why Linux kernel technologies like cgroups and namespaces are an irreplaceable foundation. Just like understanding the history of a programming language, understanding the history of containerization means you don't just know how to use Docker — you know why it exists — and that's what separates the engineer who types commands from the engineer who understands the system.

In this episode we'll trace the evolution of application deployment from the bare-metal era to the container era, dissect the root cause of "dependency hell", trace the birth of Docker at PyCon 2013, dissect the kernel technologies that form its foundation, and close with a thorough virtual machine versus container comparison.

#The Evolution of Application Deployment

The way we deploy applications is a mirror of how we solve the same problem over and over again: running code on a machine that isn't ours. Let's explore the three major eras.

#The Bare-metal Era

In the early era — the 1980s to the 2000s — one application ran directly on one physical machine (bare-metal). Servers were "iron boxes" that were purchased, assembled, and racked. The consequences:

• One machine runs one application, because stacking two applications risks them interfering with each other — clashing library versions, or one application consuming all the memory.
• Scale = buy a new machine. Need a second application? Buy a second server. Need redundancy? Buy a third server.
• Poor hardware utilization — a server serving only a few requests still has to stay on 24 hours with its CPU 90% idle.

Imagine every application having its own custom-built house — luxurious, but expensive and wasteful.

#The Virtual Machine Era (Hypervisor)

The core problem with bare-metal was cost and utilization. In the early 2000s, hypervisors (VMware, KVM, Xen) split one physical machine into many virtual machines. Each VM contains:

• A complete guest OS — its own kernel (Linux, Windows, etc.)
• The application and its dependencies — just like its own bare-metal server

This was a major shift: one physical server could now run five, ten, even twenty different "servers", each fully isolated. Hardware was utilized far better, and "It works on my machine" began to be solved — an identical VM could be moved between hosts.

But there was a price to pay: every VM copies an entire operating system, complete with kernel, drivers, and libraries. Imagine shipping an entire house — foundation, roof, furniture — when all you needed was to send the contents of one cabinet to a neighbor. Boot takes minutes, consumes hundreds of MB to GB of RAM, and the disk fills with duplicate operating systems that are never used.

#The Containerization Era

This is where containerization answers the question: how do you isolate an application without shipping an entire operating system? The answer is to leverage the Linux kernel that's already there on the host. A container isn't a virtual machine — it's a set of ordinary Linux processes that the kernel isolates and limits. What gets shipped isn't the whole house, but a shipping container: the cabinet contents, shelves, and belongings that are needed — in a standard format that can be loaded onto any ship.

#"It Works on My Machine!" and Dependency Hell

Before closing out the evolution, let's dissect the most classic problem in the software industry. "It works on my machine!" isn't a joke — it's a symptom of a root cause called dependency hell:

1. An application never runs alone. It needs a runtime (Python 3.8, Node 18, JVM), libraries (OpenSSL 1.1 vs 3.0), system tools (curl, jq), and configuration files.
2. Those dependencies depend on each other under complex version rules — often conflicting: Application A needs OpenSSL 1.1, Application B needs OpenSSL 3.0, and the two cannot coexist on the same system.
3. Different environments = different results. The developer's laptop (macOS, latest libraries), the staging server (Ubuntu, mid versions), the production server (CentOS, old versions) — a single differing dependency version can trigger mysterious bugs.

This problem grows as applications grow. One small script with one dependency is easy to manage; a production application with dozens of interdependent libraries is a time bomb. Docker solves this fundamentally: application dependencies are packaged together with the application inside the image, so library versions are "always the same" on every machine — not "hopefully the same".

#The Birth of Docker

#From DotCloud to Docker

Docker's story begins with DotCloud, a Platform-as-a-Service company built by Solomon Hykes and his co-founders. DotCloud sold a platform for deploying multi-technology applications — and to manage isolation between customer applications, they built an internal tool based on LXC (Linux Containers), an isolation technology that leverages cgroups and namespaces.

That internal tool turned out to be more valuable than their main product. In March 2013, Hykes introduced the project — now called Docker — in a famous demo at the PyCon conference. Within minutes on stage, he ran containers and showed how simple it was to package and run applications. The response was extraordinary: the open-source project exploded within months, changing how the industry viewed deployment.

#Why Did Docker Explode?

Docker didn't invent isolation technology — LXC, cgroups, and namespaces had existed for a long time. What Docker invented was ease of use and portability:

1. A simple interface. A single docker run command replaced dozens of manual LXC configuration steps.
2. Images as portable artifacts. The application + environment are packaged into a single image that can be moved between machines, shared, and versioned — like .git for the entire environment.
3. Layered architecture. Images are built from layers that can be cached and shared — efficient in both storage and transfer.
4. Registry. Docker Hub became the "GitHub for images" — sharing applications became as easy as docker pull.

It was this ecosystem — not the isolation technology — that made Docker transform the industry. Before Docker, process isolation was a kernel-level topic rarely touched by application developers; after Docker, isolation became a feature used by millions of developers every day.

#The Linux Kernel Foundation Behind Containers

This is the part that's most misunderstood. A container isn't a new kernel feature — it's a combination of three old Linux technologies brought together. Let's dissect each one.

#cgroups (Control Groups)

cgroups is a kernel mechanism for limiting and measuring resource usage of a group of processes: maximum CPU, amount of memory, disk I/O, even how many processes (PIDs) may be created. Without cgroups, a single "rogue" process could consume the host's entire memory and take down other applications. With cgroups, you can say: "this container uses at most 512 MB RAM and 1 CPU" — and the kernel enforces it strictly.

Analogy: cgroups are room dividers — they regulate how much stuff may go into one room, what its maximum weight is, and keep a record of who uses how much.

#namespaces

namespaces is a kernel mechanism for isolating the view a group of processes has of the system. Every process in Linux "sees" the system through the perspective its namespaces define. The main namespaces used by containers:

Imagine two people living in different rooms in the same house (the kernel). Both use the same "house" (CPU, memory, filesystem), but each only sees the contents of their own room — different doors, different windows, different mailing addresses. That's namespaces: the illusion of a separate environment built on top of shared resources.

#chroot

chroot is the oldest technology in this list — it changes a process's root directory so the process can only see the filesystem beneath that directory. This concept is the ancestor of filesystem isolation and the basis of the idea that "an application brings its own environment". chroot is still used today, but the more modern (and safer) Mount namespaces are what form the foundation of container filesystem isolation.

#Virtual Machine vs Container Comparison

Now we have all the ingredients for an honest comparison. Virtual machines and containers answer the same problem with different approaches:

Let's look at a concrete comparison of boot time and resources:

time virt-boot-vm --image ubuntu-server.img
real    0m45.812s          # VM: menit untuk boot OS lengkap
time docker run --rm ubuntu echo "boot!"
real    0m0.421s            # Kontainer: sepersekian detik

The difference in these numbers isn't just a trick — it reflects a fundamental difference. A VM boots an entire operating system (BIOS → kernel → init system → services), while a container merely creates a new Linux process that immediately uses the host kernel that's already running. There's nothing to boot.

$ ps aux | grep -c sleep
1
$ docker run -d alpine sleep 300
$ ps aux | grep "sleep 300"
root      12345  0.0  0.0   1528   1104 ?   Ss   12:00   0:00 sleep 300

The second line of the example above is the most important proof of concept in this episode: a running container appears as an ordinary process on the host (with a PID in the host process table). It doesn't run its own kernel or init system — it's just a process the kernel isolates and limits. This is why a thousand containers feel far lighter than a thousand VMs.

#Conclusion

In episode 1 we've understood the journey of application deployment from the bare-metal era (one application per machine, wasteful and expensive), through the virtual machine era (strong isolation but copying entire OSes), to the containerization era that isolates processes while sharing the host kernel. We also dissected the root cause of "It works on my machine!" — the dependency hell that led to Docker's birth as a solution for packaging applications along with their environment. Its history is recorded at PyCon 2013: Solomon Hykes introduced Docker, which exploded not because it invented isolation technology (cgroups, namespaces, LXC already existed) but because it made isolation easy and portable.

The core takeaways:

• Deployment evolved from bare-metal → VM → containers in pursuit of efficiency and portability.
• Docker didn't invent isolation — it packaged it behind an easy interface and portable images.
• Containers are built from cgroups (limit resources) + namespaces (isolate views) + chroot (filesystem root).
• Containers share the host kernel — that's why they're lightweight, fast, and only for the same OS.
• VMs = strong and heavy; containers = efficient and light. Choose based on needs, not trends.

Now you know why Docker exists. In the next episode, episode 2, we'll dissect Docker architecture and core concepts: how the client, daemon (dockerd), and registry work together, the role of containerd and runc and the OCI standard behind the scenes, and the four core components — Image, Container, Volume, and Network — that will become your everyday language in all the episodes ahead. See you in episode 2!