Learn ChromaDB - Distance Metrics & HNSW Index

#Introduction

All the searches you have run so far run on top of the HNSW index and one of the distance functions. These two components determine how fast and how accurate ChromaDB finds results. Episode 9 takes you into the layer usually considered "advanced", even though the basics are actually quite simple.

We will cover the three distance functions (L2, cosine, inner product), how to choose one based on your embedding model, then dive into HNSW tuning: ef_construction, M, and ef search. The trade-offs you make here — recall versus latency versus memory — are architectural decisions that will follow your collection for its whole life.

#Distance Functions: Three Ways to Measure Distance

#L2 (Euclidean)

L2 computes the Euclidean distance between vectors. It measures the direct distance in vector space — the closer the points, the more similar. ChromaDB normalizes vectors before computing L2 so the results are consistent with similarity.

Collection dengan metrik L2

collection = client.create_collection(
    name="artikel",
    metadata={"hnsw:space": "l2"},
)

create_collection(name="artikel", metadata={"hnsw:space": "l2"}) configures the index to use Euclidean distance. L2 suits cases where your embedding model produces already-normalized vectors or where vector magnitude carries meaning.

#Cosine Similarity

Cosine measures the angle between two vectors — ignoring length (magnitude). This is the most popular metric for text embeddings, because text meaning is usually determined by the vector's direction, not its length:

Collection dengan metrik cosine

collection = client.create_collection(
    name="artikel",
    metadata={"hnsw:space": "cosine"},
)

create_collection(name="artikel", metadata={"hnsw:space": "cosine"}) produces values between 0 and 2 (1 = identical). Cosine is a sensible default for almost all modern text embedding models.

#Inner Product

Inner product (dot product) computes the product of vectors without normalization. It is sensitive to magnitude, so it suits embeddings designed with meaningful magnitude — common with models like OpenAI's text-embedding-3:

Collection dengan inner product

collection = client.create_collection(
    name="artikel",
    metadata={"hnsw:space": "ip"},
)

create_collection(name="artikel", metadata={"hnsw:space": "ip"}) uses inner product. For this metric, a larger value means more relevant — the opposite of L2 and cosine.

#Getting to Know the HNSW Index

#The Hierarchical Navigable Small World Principle

HNSW is an ANN (approximate nearest neighbor) algorithm built on the concept of a small world graph. Vectors are organized into layers: the upper layers are sparse and allow long jumps, the lower layers are dense and allow detailed search. The result: nearest-neighbor search is far faster than a linear scan, with adjustable accuracy.

Because HNSW is approximate, there is a trade-off: results are not always the true nearest neighbors, but very close — and controllable via parameters. These are the three main parameters we will examine.

#Tunable HNSW Parameters

#ef_construction: Quality While Building the Index

ef_construction controls how carefully the index is built when data is added. A higher value means a more accurate built index, but a slower addition process and more memory:

Mengatur ef_construction

collection = client.create_collection(
    name="artikel",
    metadata={"hnsw:space": "cosine", "hnsw:ef_construction": 200},
)

In the example above, metadata={"hnsw:space": "cosine", "hnsw:ef_construction": 200} raises construction from the default 100 to 200. A rule of thumb: raise it if the data is static and accuracy matters; lower it if data is written continuously.

#M: Connections per Node

M sets the maximum number of connections for each node in the graph. A larger M makes the graph denser — more accurate search but more memory:

Mengatur M

metadata = {
    "hnsw:space": "cosine",
    "hnsw:M": 32,
    "hnsw:ef_construction": 200,
    "hnsw:ef_search": 100,
}
collection = client.create_collection(name="artikel", metadata=metadata)

metadata={"hnsw:M": 32, "hnsw:ef_search": 100, ...} uses values that are common for medium-to-large datasets. ChromaDB's default M is 16.

#ef_search: Quality While Querying

Unlike the two parameters above, which affect construction, ef_search is set per query and controls how deep the search runs when querying. A higher value = more accurate results but slower queries:

Query dengan ef_search tinggi

hasil = collection.query(
    query_texts=["deploy aplikasi"],
    n_results=5,
    search_params={"ef_search": 512},
)

query(..., search_params={"ef_search": 512}) raises the search precision for this query only. It is the most flexible parameter because it can be changed without rebuilding the index.

#Trade-Offs: Recall vs Latency vs Memory

The three parameters form a trade-off triangle:

A common strategy: build a good index (ef_construction and M moderate) for data that rarely changes, then play with ef_search per query. You can measure recall using a sample dataset whose true answers are known.

Contoh tuning pragmatis

params = {
    "hnsw:space": "cosine",
    "hnsw:ef_construction": 256,
    "hnsw:M": 32,
}
def build_collection(ef_search):
    c = client.create_collection(name="coba", metadata=params)
    c.add(ids, documents, metadatas)
    return c.query(query_texts, n_results=5,
                   search_params={"ef_search": ef_search})

The pattern above — build_collection(ef_search) — allows quick experiments: measure latency and accuracy for several ef_search values, then pick the optimal point for your case.

#Closing

Episode 9 gave you control over the search engine: three distance functions with their use cases, how HNSW works as an ANN graph, and the three main tuning parameters (ef_construction, M, ef_search) along with the recall, latency, and memory trade-offs.

Key takeaways:

• Cosine for most text embeddings; L2 and inner product have their own cases.
• The metric must match the embedding model's recommendation.
• HNSW is approximate: accuracy is controlled via parameters.
• ef_construction and M affect index construction.
• ef_search is set per query and is the most flexible for tuning.
• The trade-off is always a triangle: recall, latency, and memory.

In the next episode, episode 10, we will discuss data modeling and chunking strategy — recommended metadata schemas, id conventions, timestamps and sources, tenants and namespaces, then document splitting strategies: chunk size, overlap, and granularity for accurate RAG retrieval. The quality of the data above determines the quality of retrieval below.