A LangChain agent that only runs in a notebook is a demo. The moment anything else needs to call it, a chat frontend, a webhook, a cron job, a colleague, it has to become a server: a process that stays up, holds state, streams output, and answers at a stable address.
This is the practical companion to best servers for LangChain agents in 2026. That guide compares where to host. This one shows how to actually get there, in the order the work happens.
Quick answer
To deploy a LangChain agent as a server, wrap the agent in an HTTP app (FastAPI is the common choice), add a streaming endpoint and a health check, move state into a persistent checkpointer, load API keys from the environment rather than code, and run it under a process manager on a host that allows long-running processes. Serverless functions are usually the wrong target because agent loops outlive their execution caps.
Key takeaways
- An agent server is a long-lived process, not a request handler that happens to call an LLM.
- Streaming is not optional once a human is watching, and it shapes your endpoint design.
- Checkpoints belong on a persistent disk or a database, never in process memory.
- Health checks and readable logs are what make the difference between a deployed agent and a debuggable one.
- Concurrency limits matter more than CPU, because agent loops spend most of their life waiting on network calls.
Step 1: wrap the agent in an HTTP app
Start by separating agent construction from request handling. Build the agent once at startup, not per request, so you are not paying import and setup costs on every call.
# app.py
import os
from contextlib import asynccontextmanager
from fastapi import FastAPI
from pydantic import BaseModel
agent = None
@asynccontextmanager
async def lifespan(app: FastAPI):
global agent
agent = build_agent() # your LangChain or LangGraph construction
yield
app = FastAPI(lifespan=lifespan)
class InvokeRequest(BaseModel):
input: str
thread_id: str = "default"
@app.post("/invoke")
async def invoke(req: InvokeRequest):
result = await agent.ainvoke(
{"messages": [("user", req.input)]},
config={"configurable": {"thread_id": req.thread_id}},
)
return {"output": result["messages"][-1].content}Two details matter here. Use the async methods (ainvoke, astream) rather than their blocking counterparts, because an agent loop is almost entirely I/O wait and async lets one process handle many concurrent runs. And thread the thread_id through from the caller, because that is the handle your checkpointer uses to find the right conversation.
The FastAPI documentation covers the lifespan pattern in more depth if this shape is new to you.
Step 2: add streaming
A synchronous invoke endpoint is fine for webhooks and cron jobs. As soon as a person is waiting, a thirty second silent pause reads as a hang. Stream instead, using server-sent events, which is the simplest transport that browsers already understand.
from fastapi.responses import StreamingResponse
@app.post("/stream")
async def stream(req: InvokeRequest):
async def events():
config = {"configurable": {"thread_id": req.thread_id}}
async for chunk in agent.astream(
{"messages": [("user", req.input)]},
config=config,
stream_mode="messages",
):
token = chunk[0].content
if token:
yield f"data: {token}\n\n"
yield "data: [DONE]\n\n"
return StreamingResponse(events(), media_type="text/event-stream")Watch out for buffering proxies. If you put a reverse proxy in front of this and see the whole response arrive at once at the end, response buffering is on and needs to be disabled for the streaming route.
Step 3: make state survive a restart
This is where most first deployments quietly break. An in-memory checkpointer works perfectly in development and loses every conversation the first time the process restarts, which will happen on every deploy.
Point the checkpointer at something durable. A SQLite file on a persistent disk is enough for a single-process agent, and Postgres is the answer once you run more than one replica.
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
DB_PATH = os.environ.get("CHECKPOINT_DB", "./data/checkpoints.sqlite")
checkpointer = AsyncSqliteSaver.from_conn_string(DB_PATH)The important constraint is that the directory holding that file has to persist across restarts and redeploys. This is exactly the requirement described in why AI agents need persistent workspaces, and it is the single most common reason a "working" agent server loses its memory in production. The LangGraph documentation has the full checkpointer reference, including the Postgres variants.
Vector indexes, cached documents, and scratch files have the same requirement. If it took work to compute and you want it after a restart, it needs a real disk.
Step 4: handle secrets properly
Your model provider key, tool credentials, and database URLs are environment variables, never literals in the source file. Read them at startup and fail loudly if one is missing, because a missing key that surfaces as a confusing error three tool calls into a run costs far more time than a clear crash at boot.
def require(name: str) -> str:
value = os.environ.get(name)
if not value:
raise RuntimeError(f"Missing required environment variable: {name}")
return value
MODEL_API_KEY = require("MODEL_API_KEY")Step 5: add a health check and readable logs
Give the platform a cheap endpoint to poll, and make it honest. A health check that returns OK while the agent is unusable is worse than none.
@app.get("/health")
async def health():
return {"status": "ok", "agent_ready": agent is not None}For logs, log the thread id, the tool name, and the duration on every tool call. When an agent misbehaves it is almost always a loop on one tool or a prompt regression, and both are obvious in a log that records tool names and timings, and invisible in one that only records HTTP status codes.
Step 6: run it as a real process
In production, run the app under Uvicorn and keep it supervised so it restarts if it dies.
uvicorn app:app --host 0.0.0.0 --port 8000 --workers 1Start with a single worker. Agent processes hold state, and multiple workers each get their own memory space, so anything cached in process is inconsistent between them. Scale by raising the concurrency your async endpoints allow before you scale by adding workers, and when you do add workers, make sure the checkpointer has already moved to a shared database.
Step 7: put it behind a stable URL
The last step is an address that does not change on every deploy. Webhooks are registered once, chat frontends are configured once, and a URL that rotates on each release breaks both. This is the argument in why every AI agent should be able to return a URL.
On Server4Agent this step folds into the previous ones. You describe the app you want, an AI assistant connects to a real server, writes the code into a project workspace, installs the dependencies, sets the secrets, runs the process, reads its own logs when something fails, and hands back a working hosted URL. The project keeps its files between runs, so the checkpoint database from step 3 is simply there, and the URL stays private until you decide to publish it. If you prefer to drive it from code rather than from a chat window, the Python SDK exposes the same operations.
Whichever host you choose, review what is running before you open it up. Agent-built services can reach real credentials and real data, which is the case made in private by default.
Common mistakes
- Building the agent inside the request handler. Move it to startup.
- An in-memory checkpointer in production. State disappears on the first redeploy.
- Deploying to a platform with an execution cap. A sixty second limit kills real agent loops midway.
- No timeout on tool calls. One hanging HTTP tool call holds a run open indefinitely.
- Logging only HTTP status codes. You cannot debug a tool loop from a 200.
FAQ
Can I deploy a LangGraph agent the same way?
Yes. A compiled LangGraph graph exposes the same ainvoke and astream interface, so the endpoints above work unchanged. The main difference is that checkpointing is built into the graph, which makes step 3 more important, not less.
Do I need a GPU?
Almost never. If you call a hosted model API, your server is coordinating network calls and holding state. CPU and memory sized for concurrency matter far more than a GPU, which is only relevant if you run model weights yourself.
How do I handle long agent runs that outlive an HTTP request?
Return a run id immediately, process in the background, and let the caller poll or subscribe to a stream. Any run reliably longer than a minute or two should be a background job rather than a blocking request.
What is the difference between this and a sandbox?
A sandbox is for ephemeral code execution and is torn down after use. An agent server is a persistent home for the agent process itself. The comparison is laid out in the LangChain sandboxes alternative page.
Related reading
- Best servers for LangChain agents in 2026
- Why AI agents need persistent workspaces
- The Server4Agent Python SDK
- Why every AI agent should be able to return a URL