Learn ChromaDB - Integration with LLM Frameworks

#Introduction

All the ChromaDB capabilities you have built since episode 0 now come together. Episode 18 covers integration with LLM frameworks — the part that turns ChromaDB from a database into a retrieval engine ready for RAG pipelines.

We will cover LangChain integration as a VectorStore and retriever, LlamaIndex and other framework adapters, then assemble them into a complete production RAG pattern: chunking, embedding, storage, and querying in one flow. This is the episode that answers "so, how do I actually use it?".

#LangChain: Chroma as a VectorStore

#Installation and Creating a VectorStore

The official LangChain integration lives in the langchain-chroma package. Install and connect:

pip install langchain langchain-chroma

VectorStore LangChain

from langchain_chroma import Chroma
 
vectorstore = Chroma(
    collection_name="dokumen",
    persist_directory="./chroma-data",
    embedding_function=embedding_model,
)

Chroma(collection_name="dokumen", persist_directory="./chroma-data", embedding_function=embedding_model) creates a VectorStore wrapped around ChromaDB. All the complexity from episodes 3-10 is hidden — LangChain manages the collection and embedding function for you.

#Storing Documents

With a VectorStore, storing documents becomes a single call:

Menambah dokumen lewat LangChain

from langchain_text_splitters import RecursiveCharacterTextSplitter
 
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = splitter.split_text(dokumen_panjang)
 
vectorstore.add_texts(
    texts=chunks,
    metadatas=[{"source": "manual"} for _ in chunks],
)

splitter.split_text(dokumen_panjang) splits the document using the chunking from episode 10, then vectorstore.add_texts(...) stores it. Note that chunk_size and chunk_overlap here are the direct realization of episode 10's strategy.

#Retriever and the Full RAG Chain

#Turning the VectorStore into a Retriever

Turning the VectorStore into a retriever is as easy as calling .as_retriever():

Retriever dari VectorStore

retriever = vectorstore.as_retriever(
    search_type="similarity",
    search_kwargs={"k": 4},
)

vectorstore.as_retriever(search_type="similarity", search_kwargs={"k": 4}) produces a retriever that returns the top 4 documents for every question. The k parameter is equivalent to n_results from episode 5.

#Building the Full RAG Chain

Now assemble everything into an answering pipeline:

RAG chain dengan LangChain

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
 
llm = ChatOpenAI(model="gpt-4o-mini")
 
prompt = ChatPromptTemplate.from_template(
    "Jawab berdasarkan konteks ini:\n{context}\n\nPertanyaan: {question}"
)
 
def jawab(pertanyaan):
    dokumen = retriever.invoke(pertanyaan)
    konteks = "\n\n".join(d.text for d in dokumen)
    pesan = prompt.format_messages(context=konteks, question=pertanyaan)
    return llm.invoke(pesan).content

The jawab(pertanyaan) function runs the most core RAG pattern: retrieve context from ChromaDB, attach it to the prompt, then generate an answer with the LLM. This is the embed → store → retrieve → generate flow from episode 0 in concrete form.

#LlamaIndex and Other Frameworks

#ChromaDB as a LlamaIndex VectorStore

LlamaIndex also has an official adapter. The flow: create a storage context, then use VectorStoreIndex:

ChromaDB di LlamaIndex

import chromadb
from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.vector_stores.chroma import ChromaVectorStore
 
chroma_client = chromadb.PersistentClient(path="./chroma-data")
collection = chroma_client.get_collection("dokumen")
 
vector_store = ChromaVectorStore(chroma_collection=collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)

ChromaVectorStore(chroma_collection=collection) wraps a ChromaDB collection into a LlamaIndex vector store. From here, index.as_query_engine() can be used directly for asking questions.

#Other Frameworks in the Ecosystem

ChromaDB also integrates with other frameworks such as Haystack and LlamaStack. The pattern is consistent everywhere: the framework provides the adapter, ChromaDB provides storage and retrieval, and the embedding function stays from episode 6. The key is not memorizing each framework's API, but understanding the VectorStore concept — once you get it, all adapters feel familiar.

#The Recommended Production RAG Pattern

#The Complete Architecture

Summarizing episodes 0-18 into a production-ready RAG architecture:

dokumen ──chunk──> embedding ──store──> ChromaDB
                                                │
pertanyaan ──embed──> query ──filter+rerank──> konteks
                                                │
konteks + pertanyaan ──prompt──> LLM ──> jawaban

The pipeline above combines: chunking (episode 10), embedding (episode 6), metadata filtering (episode 7), hybrid and rerank (episode 8), and the LLM as generator. You have mastered every stage in its own episode.

#Integration Checklist

Before landing in production, make sure:

• The embedding function is consistent between ingestion and query.
• The retriever uses relevant metadata filters (episode 7).
• Consider hybrid + rerank for accuracy (episode 8).
• Server and client connect via HttpClient (episode 12).
• Auth, TLS, and network isolation are active (episodes 13-15).

Retriever production dengan filter

retriever = vectorstore.as_retriever(
    search_kwargs={
        "k": 6,
        "filter": {"status": "published"},
    }
)

retriever.as_retriever(search_kwargs={"k": 6, "filter": {"status": "published"}}) injects a metadata filter directly into retrieval — the realization of episode 17's pre-filtering at the framework level.

#Closing

Episode 18 unified the entire series into one goal: a working RAG. You can now make ChromaDB a VectorStore and retriever in LangChain, use the LlamaIndex adapter, and assemble a production RAG pipeline from chunking to answers. ChromaDB is no longer just a database — it is the retrieval engine of your AI application.

Key takeaways:

• langchain_chroma wraps ChromaDB as a VectorStore and retriever.
• Framework chunking realizes the strategy from episode 10.
• as_retriever() turns a VectorStore into a RAG component.
• LlamaIndex uses ChromaVectorStore as its adapter.
• RAG pattern: chunk → embed → store → retrieve → generate.
• Filters and reranking from episodes 7-8 can be injected directly.

In the next episode, episode 19, we will discuss advanced query patterns — multi-vector and multimodal with multiple embeddings per document, image and text embeddings, and feedback loops with relevance scoring and human feedback for continuous retrieval tuning. Your retrieval is smart; now it is time to make it smarter.