Learn ChromaDB - Advanced Query Patterns

#Introduction

You mastered the basic retrieval patterns in episode 18. Episode 19 goes up one level with advanced query patterns: techniques that are rarely discussed but often determine the quality of real-world RAG applications — multi-vector, multimodal, and feedback loops that keep retrieval improving.

These three topics complement each other: multi-vector enriches a single document's representation, multimodal extends the reach to images, and feedback loops make your system learn from usage. Let us begin.

#Multi-Vector: Multiple Embeddings per Document

#The One-to-Many Concept

So far, one document has one embedding. Multi-vector flips that assumption: one document is represented by several embeddings — for example one for the title, one for the content, one for keywords. Each embedding searches a different dimension, and the results are combined.

The implementation in ChromaDB is simple: because ids are unique, you store variants as separate items with different ids but the same document:

Menyimpan multi-embedding per dokumen

varian_teks = {
    "judul": "Pengenalan Docker untuk Pemula",
    "isi": "Docker mengemas aplikasi beserta dependency-nya",
    "kunci": "container, image, deployment",
}
 
collection.upsert(
    ids=[f"doc-1:{aspek}" for aspek in varian_teks],
    documents=[teks for teks in varian_teks.values()],
    metadatas=[{"aspek": aspek, "doc_id": "doc-1"} for aspek in varian_teks],
)

ids=[f"doc-1:{aspek}" creates three items for one document. The doc_id metadata keeps the linkage — when retrieval finds any variant, you know the original document.

#Querying and Deduplicating Multi-Vector

When querying, ChromaDB returns results from various variants. The important step: deduplication based on doc_id:

Query lalu deduplikasi

hasil = collection.query(query_texts=["cara deploy container"], n_results=10)
 
terlihat = set()
final = []
for i, doc_id in enumerate(hasil["metadatas"][0]):
    if doc_id["doc_id"] not in terlihat:
        terlihat.add(doc_id["doc_id"])
        final.append((doc_id["doc_id"], hasil["documents"][0][i]))

The loop above takes the first unique document that appears. hasil["metadatas"][0] provides the doc_id for deduplication — this pattern is mandatory when using multi-vector.

#Multimodal: Image and Text Embeddings

#One Collection, Two Data Types

ChromaDB does not care what data is embedded — vectors remain vectors. With multimodal embedding models such as CLIP, images and text are mapped into the same vector space, making cross-media search possible.

Collection multimodal

from chromadb.utils.embedding_functions import OpenCLIPEmbeddingFunction
 
ef = OpenCLIPEmbeddingFunction()
collection = client.create_collection(
    name="katalog",
    embedding_function=ef,
)
 
collection.add(
    ids=["img-1", "img-2"],
    embeddings=[ef([gambar1])[0], ef([gambar2])[0]],
    metadatas=[{"tipe": "image", "judul": "Sepatu merah"}],
)

OpenCLIPEmbeddingFunction() generates embeddings that are comparable for both images and text. collection.add(ids=["img-1"], embeddings=[...]) stores image embeddings directly — using the pre-computed pattern from episode 6.

#Querying Images with Text (and Vice Versa)

The multimodal magic: a text query finds matching images:

Cari gambar dengan teks

hasil = collection.query(
    query_texts=["sepatu berwarna merah"],
    n_results=3,
    where={"tipe": "image"},
)

collection.query(query_texts=["sepatu berwarna merah"], where={"tipe": "image"}) finds red shoe images even though the query is text. Because CLIP unifies the two modalities in one vector space, the query direction can go either way.

#Feedback Loops and Relevance Scoring

#Relevance Scoring from Interactions

Good retrieval does not stop at deployment — it learns. Start by recording relevance scores from user interactions: did the results actually help answer their questions?

Menyimpan feedback pengguna

collection_feedback.add(
    ids=[f"fb-{next_id}"],
    documents=[query_pengguna],
    metadatas={
        "doc_retrieved": doc_id,
        "helpful": 1.0,
        "timestamp": now,
    },
)

collection_feedback.add(ids=[...], metadatas={"helpful": 1.0, ...}) stores quality signals: the query, the chosen document, and whether it helped. This collection of signals becomes a training dataset for retrieval tuning.

#Human Feedback for Iterative Tuning

With feedback collected, you can perform continuous tuning:

• Weighting: give more weight to the multi-vector aspects that frequently help.
• Reranking: train or adjust the reranker threshold based on helpful data.
• Filter tuning: change the metadata filter combinations that yield the best results.
• A/B testing: compare ef_search and M configurations to see which to raise.

Menilai akurasi dari feedback

import statistics
 
skor = collection_feedback.get(where={"doc_retrieved": "doc-42"})["metadatas"]
rata_helpful = statistics.mean([m["helpful"] for m in skor])
print("skor relevansi doc-42:", rata_helpful)

statistics.mean([m["helpful"] for m in skor]) gives the average satisfaction score per document. Documents with low scores can be re-tested with different chunking (episode 10) or removed from retrieval results.

#Iterative Retrieval Tuning: The Complete Cycle

Summarizing this episode into a continuous cycle:

1. Deploy retrieval with the base configuration (multi-vector if needed).
2. Collect user feedback: query, document, and satisfaction score.
3. Measure aggregate scores per document and per configuration.
4. Tune: change embeddings, filters, rerankers, or aspect weights.
5. A/B test the new configuration against the baseline.
6. Promote the winner, repeat.

This pattern turns retrieval from "set and forget" into an asset that keeps improving over time — exactly the spirit of episode 22 on production readiness.

#Closing

Episode 19 opened advanced territory: multi-vector that enriches document representation, multimodal that unifies images and text in a single vector space, and feedback loops that make retrieval keep learning from real usage. You now have the tools to build retrieval that is not only accurate, but also adaptive.

Key takeaways:

• Multi-vector = several embeddings per document, combined and deduplicated.
• doc_id in metadata keeps variants linked to the original document.
• CLIP unifies images and text in one vector space.
• Cross-media queries: text finds images and vice versa.
• Feedback loops store queries, documents, and satisfaction scores.
• Iterative tuning uses user data, not intuition.

In the next episode, episode 20, we will discuss observability and operations — health checks, latency and throughput metrics, logging, OpenTelemetry instrumentation with traceAI-chromadb, routine backup and restore, capacity planning, and incident response. A good system needs eyes; it is time to give ChromaDB sight.