Learn ChromaDB - Core Concepts & Main Architecture

#Introduction

After understanding why ChromaDB exists, episode 2 takes you inside its engine: core concepts and main architecture. This is the most important episode to understand before you start coding, because almost all the code in this series revolves around a single primitive: the Collection.

We will cover the data model, the two run modes (embedded and client-server), the difference between the legacy Python FastAPI server and the modern Rust server, and how a query flows from text to results. If you understand this episode well, episodes 3 through 20 will feel like variations on one big theme.

#The Data Model: Collection

#The Four Pillars of a Collection

All data in ChromaDB lives in a Collection, similar to a table in a relational database. Every collection has a unique name, and its contents follow four main components:

• ids: the unique identity of each item, equivalent to a primary key.
• embeddings: numeric vectors representing meaning, generated by the embedding function.
• documents: the original text that was embedded, serving as context for RAG.
• metadatas: free-form key-value pairs for extra attributes like date, category, or source.

The relationship between the four is simple: each id pairs with one embedding, one document, and one metadata dict. You can fill them all at once, or let the embedding be generated automatically from the document.

Empat pilar collection

collection = client.create_collection("berita")
 
collection.add(
    ids=["berita-001", "berita-002"],
    documents=["Inflasi naik dua persen", "Gunung meletus di Jawa Timur"],
    metadatas=[{"kategori": "ekonomi"}, {"kategori": "alam"}],
)

Notice that collection.add(...) does not take an embeddings argument — ChromaDB generates them from the documents using the built-in embedding function. This is the easiest way to get started.

#Why Documents and Embeddings Can Be Separate

This data model is flexible: you may store embeddings only, without documents (for example, for image embeddings), or the reverse. This separation matters for the case in episode 19: multimodal and multi-vector. For ordinary text RAG, you need documents because they are what becomes the LLM's answer context.

#Embedded Mode vs Client-Server Mode

ChromaDB has two very different ways of running. The difference drives many architectural decisions in this series.

#Embedded Mode: Simple and Serverless

In this mode, ChromaDB runs inside your application's process. There is no port, no network, no separate server. You simply create a client and start:

PersistentClient embedded

import chromadb
 
client = chromadb.PersistentClient(path="./chroma-data")

This mode is perfect for development, prototypes, and single-process applications. Data is stored to a local directory (./chroma-data in the example above) via PersistentClient(path="./chroma-data"). There is also EphemeralClient, whose data disappears when the process ends — suitable for testing.

#Client-Server Mode: Chroma as a Service

In this mode, ChromaDB runs as a separate server accessed by many clients over HTTP. This is where chroma run — the Rust server — comes in:

chroma run --path ./chroma-data --port 8000

Clients then connect over HTTP:

HttpClient

import chromadb
 
client = chromadb.HttpClient(host="localhost", port=8000)

This mode is required for multi-process or multi-service production. chromadb.HttpClient(host="localhost", port=8000) is the gateway to the distributed architecture we will cover from episode 12 through 17.

#Server Architecture: Python FastAPI vs Rust

It is important to distinguish the two eras of the ChromaDB server, because many old tutorials still use the Python version.

#The Python FastAPI Server (Legacy)

The 0.x and early 1.x server was built with Python FastAPI. Its strengths: easy to develop and identical to the embedded library. But it has weaknesses that make it less than ideal for production: less deterministic performance, higher memory usage, and — most importantly — a history of security vulnerabilities, including CVE-2026-45829, which we will examine in episode 14.

#The Rust Server (Production Recommendation)

Since version 1.x, ChromaDB introduced the Rust server. With the same command, chroma run, it now runs a Rust binary that is far more efficient. Its advantages:

• Faster and more deterministic performance.
• Smaller memory footprint.
• Not affected by CVE-2026-45829.

For all production episodes in this series, we will always use the Rust server. The Python and JavaScript clients stay the same — only the server differs.

#The Query Flow: From Question to Result

When you send a query, ChromaDB runs a flow made up of three stages:

1. Embed: the query text is turned into a vector by the embedding function.
2. Index: the query vector is compared against all vectors in the collection using the HNSW index and the configured distance function.
3. Re-rank: results with the smallest distance (or greatest similarity) are sorted and returned along with documents and metadatas.

query_text ──embed──> vektor ──HNSW search──> kandidat ──re-rank──> hasil

This flow runs behind a single collection.query(...) call. Understanding the stages matters because each stage has its own tuning knob: the embedding function in episode 6, pre-index filtering in episode 7, and HNSW parameters in episode 9.

#Closing

Episode 2 gave you the architecture map of ChromaDB: the Collection as the main primitive with four pillars (ids, embeddings, documents, metadatas), two run modes (embedded and client-server), the difference between the legacy Python FastAPI server and the recommended Rust server, and the embed → index → re-rank query flow.

Key takeaways:

• All data lives in a Collection with the components ids, embeddings, documents, and metadatas.
• Embedded mode runs in the application process; client-server mode requires a server and HTTP.
• The Rust server (chroma run) is the production recommendation, not the Python FastAPI server.
• Python and JavaScript clients work with both kinds of servers.
• The query flow is always: embed, search the HNSW index, then re-rank the results.

In the next episode, episode 3, we will start writing real code: setup, client, and your first collection — comparing the different client types (Client, PersistentClient, EphemeralClient, HttpClient) and creating your first collection with the right parameters. Get your environment from episode 0 ready.