Learn LangChain - Deployment & Production

#Introduction

In episode 19 you installed LangSmith: tracing for observability, datasets and evaluators for regression testing. Quality is now maintained; it's time to leave the laptop. Episode 20 takes the application to production: serving a chain as an API with LangServe, typing the input/output contract with Pydantic, streaming to clients, the LangGraph Platform runtime for stateful agents, then the operational side — containerization, scaling, concurrency, endpoint auth, and monitoring.

#LangServe: Turning a Chain into a REST API

LangServe is a library that turns a LangChain chain into a REST API in a few lines. The basics: a FastAPI application, then add_routes to register the chain at a path.

pip install "langserve[all]" langchain-openai

server.py

from fastapi import FastAPI
from langchain_openai import ChatOpenAI
from langserve import add_routes
 
app = FastAPI(title="API Ringkasan")
add_routes(
    app,
    ChatOpenAI(model="gpt-5-mini"),
    path="/ringkas",
)
 
if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8080)

Run it with python server.py, and LangServe automatically provides many endpoints for a single chain: /ringkas/invoke for single calls, /ringkas/stream for streaming, /ringkas/batch for batches of inputs, plus /ringkas/playground to try it directly from the browser. One chain, the entire REST contract — without writing each route one by one.

#Input and Output Contracts with Pydantic

A production API shouldn't accept just anything. With Pydantic, add_routes accepts input_type and output_type, so requests are validated before entering the model and responses are guaranteed to have a structure.

schema.py

from pydantic import BaseModel
 
class Artikel(BaseModel):
    judul: str
    isi: str
 
class Ringkasan(BaseModel):
    ringkasan: str
    poin_penting: list[str]
 
add_routes(
    app,
    chain,
    path="/ringkas",
    input_type=Artikel,
    output_type=Ringkasan,
)

With this schema, the API documentation is auto-generated at the /docs FastAPI endpoint, and non-conforming requests are rejected before ever touching the model. It also lets clients on the other side — web, mobile, or CLI — know exactly what the request and response look like without reading the source code.

#Streaming to Clients

A production chat experience needs streaming. The client calls the stream endpoint with POST, then reads line by line while the connection is open.

client.py

import requests
 
with requests.post(
    "http://localhost:8080/ringkas/stream",
    json={"judul": "RAG", "isi": "Retrieval augmented generation..."},
    stream=True,
) as resp:
    for baris in resp.iter_lines():
        print(baris)

The /stream endpoint sends data as Server-Sent Events, and users see tokens appear incrementally. Make sure the proxy in front of the application — for example Nginx or a load balancer — has buffering disabled for streaming paths; otherwise tokens pile up and the streaming effect is lost.

#LangGraph Platform for Stateful Agents

For agents with state and human-in-the-loop from episodes 13 and 17, LangServe alone isn't the right fit. That's where the LangGraph Platform (or a self-hosted LangGraph server) comes in: a runtime that understands graphs, checkpoints, and threads.

{
  "dependencies": ["."],
  "graphs": {
    "agent": "./src/agent.py:graph"
  },
  "env": ".env"
}

The langgraph.json file describes the graph entries and dependencies. The platform then provides endpoints like creating a run within a thread_id, resuming after an interrupt, and reading state — all with the built-in persistence from the checkpointing you learned in episodes 7 and 17. Deploy via the CLI langgraph deploy to the cloud platform, or run it yourself in a container.

#Containerization

The next step is packaging the application so it runs anywhere. A multi-stage Dockerfile keeps the image small and secure.

FROM python:3.12-slim AS base
 
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
 
COPY . .
 
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8080"]

Build and run with docker build -t langchain-api . then docker run -p 8080:8080 langchain-api. This makes consistency easy between local, staging, and production — something that can't be guaranteed if each developer runs the application directly on their machine.

#Scaling, Concurrency, and Endpoint Auth

A LangServe chain is stateless, so its scaling is horizontal: run many instances behind a load balancer and distribute the traffic. To raise concurrency on one instance, add uvicorn workers with --workers or run the application as an async process.

uvicorn server:app --host 0.0.0.0 --port 8080 --workers 4

On the security side, never store provider API keys inside the image. Inject them via environment variables at runtime — values like OPENAI_API_KEY are read from the environment, not hard-coded. For endpoint auth, put a layer in front: a FastAPI dependency for a simple token, or OAuth/OIDC at the ingress/API gateway for organization scale. Rule of thumb: a chain should be public only if it's deliberately meant to be public.

#Production Monitoring

The observability from episode 19 doesn't stop in development — that's where it's most valuable. Keep LangSmith tracing active for incident inspection, and complete it with operational signals:

• Health check: an endpoint indicating the application is ready to accept traffic.
• Metrics: request count, latency, error rate, and tokens per minute.
• Alerting: notifications when the error rate or latency exceeds a threshold.
• Periodic evaluation: run the regression test dataset from episode 19 on every new model or prompt release.

service: langchain-api
checks:
  health: /health
metrics:
  requests_per_sec: on
  latency_p95: on
  error_rate: on
  tokens_per_min: on
alerts:
  error_rate: "> 5% selama 5 menit"
  latency_p95: "> 8 detik selama 10 menit"

The combination of traces for diagnosis and metrics for detection means production incidents can be found faster and analyzed down to the root cause.

#Conclusion

Episode 20 took LangChain applications to production: LangServe to turn a chain into a full REST API with Pydantic schemas and streaming, the LangGraph Platform for stateful agents, Docker containerization, horizontal scaling with uvicorn workers, auth in front of endpoints, and monitoring that unifies traces, metrics, and periodic evaluation.

Key takeaways:

• add_routes turns a chain into an API with invoke, stream, batch, and playground endpoints.
• Pydantic schemas in input_type and output_type enforce the API contract.
• The LangGraph Platform handles stateful agents, threads, and human-in-the-loop.
• Containerization makes behavior consistent across environments.
• Provider API keys are injected via env at runtime, with auth placed in front of endpoints.

In episode 21 we look ahead: Modern 2026 Features & Roadmap — the v1.x architecture, v2/v3 content-block streaming, Context Hub, Harness Profiles, Deep Agents v0.5+, MCP integration, and LangChain's development direction. See you there!