Deploying an AI Agent to Production
Table of contents
- Key takeaways
- Why isn't a script enough in production?
- How do you package the agent in a container?
- Where does the state live and how do you handle concurrency?
- How do you observe the agent and survive failures?
- Security and secrets management
- Frequently asked questions
- Do I need a framework or is a FastAPI container enough?
- How do I scale the agent when more traffic arrives?
- Can I deploy the agent without depending on a paid API?
- Conclusion
- Sources
Deploying an AI agent to production is the jump that separates an experiment that works on your laptop from a service that answers real users, day and night, without you sitting in front of it. The agent stops being a script you launch by hand and becomes a containerised process, with state stored outside the container, observability for every model call, well-managed secrets and a clear strategy for failures. In this guide you package an agent built with the OpenAI Agents SDK inside a minimal Docker image, expose it with FastAPI and get it ready for real traffic. The same explanation is available in Spanish.
Key takeaways
- An agent in production is a containerised HTTP service, not a script: you package it in a minimal Docker image (
python:3.13-slimbase, on Debian 13) and run it as a stateless process. - The OpenAI Agents SDK (
openai-agentspackage, version 0.18.2 from 11 July 2026) supports over 100 models, so the same service works with a proprietary model or an open one. - State (conversation memory, checkpoints) lives outside the container, in Postgres and Redis, so you can scale to several replicas behind a load balancer.
- Observability is not optional: every model call is traced with its cost, its latency and its output, and provider requests carry retries with exponential backoff and rate limits.
- Secrets are never baked into the image (they would stay in the layers): they are injected via environment variables or a secrets manager, and the agent runs as an unprivileged user.
Why isn’t a script enough in production?
During development, an agent is a Python file you run in the terminal: it reads a question, calls the model a few times, prints the answer and exits. That model is fine for a demo, but it breaks the moment other people have to use it. A production service must be always on, handle several requests at once, survive a restart without losing data, record what it did and why, and never expose your API keys to anyone inspecting the system.
The first step is to stop thinking about a script and start thinking about a service. In practice that means wrapping the agent logic in a web server that receives HTTP requests. With FastAPI (version 0.139.0) and the Uvicorn server (version 0.51.0), a few lines expose two routes: a health route, which an orchestrator polls to know whether the process is alive, and another that receives the user’s message and returns the agent’s answer.
# service.py
from fastapi import FastAPI
from pydantic import BaseModel
from agents import Agent, Runner
app = FastAPI(title="Support agent")
agent = Agent(
name="Support",
instructions="Answer in English, concisely and citing your source.",
)
class Request(BaseModel):
message: str
@app.get("/health")
def health():
return {"status": "ok"}
@app.post("/answer")
async def answer(r: Request):
result = await Runner.run(agent, r.message)
return {"answer": result.final_output}
With those two routes you already have an agent that speaks HTTP. The handler is asynchronous (async def) because most of the time the process is waiting for the model’s response, not computing, so a single worker can serve many concurrent requests without blocking.
How do you package the agent in a container?
The container is the unit you deploy: it wraps your code, its exact dependencies and the Python interpreter, and guarantees it runs the same way on your machine, in continuous integration and on the server. Always pin the versions in a requirements.txt; never depend on the unpinned latest release, because a silent update can break the service.
# requirements.txt
openai-agents==0.18.2
fastapi==0.139.0
uvicorn[standard]==0.51.0
The image is built in two stages (Docker’s multi-stage pattern). The first installs the dependencies; the second copies only the result into a clean image, with no compilers or pip cache, so the final image is small and has a smaller attack surface. This Dockerfile works with Docker Engine 29.6.1, the stable release from June 2026:
# Stage 1: install the dependencies into an isolated prefix
FROM python:3.13-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
# Stage 2: minimal final image, as an unprivileged user
FROM python:3.13-slim
RUN useradd --create-home --uid 10001 agent
WORKDIR /app
COPY --from=builder /install /usr/local
COPY . .
USER agent
EXPOSE 8000
HEALTHCHECK CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
CMD ["uvicorn", "service:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "2"]
You build and start the image with two commands; the keys come in through an environment file that is never copied into the image:
docker build -t support-agent:1.0.0 .
docker run -p 8000:8000 --env-file .env support-agent:1.0.0
If you use a state-graph framework like LangGraph, you can skip writing the Dockerfile by hand: the langgraph-cli tool (version 0.4.30) reads your langgraph.json file and generates the Dockerfile for you with langgraph dockerfile Dockerfile, including the server and the state stores.
Where does the state live and how do you handle concurrency?
An agent is inherently stateful: it remembers the conversation thread, saves intermediate results and, if it is a graph, keeps checkpoints to resume long tasks. The golden rule is that this state cannot live inside the container, because a container is ephemeral: it restarts, gets replaced and is replicated without notice. If you store the memory in a Python variable, you lose it on the first restart and two different replicas will hold different histories.
The fix is to externalise state to services built for it. The OpenAI Agents SDK offers the openai-agents[redis] extra to store sessions in Redis; LangGraph saves its checkpoints in Postgres and uses Redis to stream events. With state outside, the container becomes stateless and you can launch several identical replicas behind a load balancer that spreads the requests among them.
For concurrency it helps to separate two cases. Fast answers (a few seconds) are served inside the HTTP request itself, with asynchronous handlers and several Uvicorn workers. Long tasks (an agent researching for minutes) must not block the request: you enqueue the job in a task queue, return a job identifier immediately and the client checks the result later. The real bottleneck is almost never your CPU but the model’s latency and its rate limits, so you scale horizontally by adding replicas, not by buying a bigger machine.
How do you observe the agent and survive failures?
An agent in production fails in ways a script never does: the provider returns a 429 error for too many requests, a call takes too long, the model hallucinates a tool that does not exist. Without observability, those failures are invisible until a user complains. That is why every model call must be traced with its exact prompt, its tokens, its cost and its latency. A platform like Langfuse, which you can self-host with Docker, records that full execution tree and lets you debug with data instead of guesswork.
Resilience is built from three pieces. First, timeouts on every external call, so a hung request does not block a worker indefinitely. Second, retries with exponential backoff and a little randomness (jitter): faced with a 429 or a transient 503, you wait 1, 2 and 4 seconds before giving up, instead of hammering the provider. Third, a circuit breaker that, if the provider has been down for a while, stops trying for a few seconds and returns a controlled error. And since you also have a budget to protect, it is worth imposing per-user rate limits and a daily spend cap, so a runaway loop does not drain your account. The same rigour you would apply to serving the model with vLLM is aimed, here, at the reliability of the agent that consumes it.
Security and secrets management
The most common and most expensive mistake is baking the API key into the image. Any ENV OPENAI_API_KEY=sk-... in the Dockerfile is written into an image layer forever, and anyone who can pull it reads it in the clear. Secrets are injected at run time: with --env-file in Docker, with Docker or Kubernetes secrets, or from a dedicated manager like Vault. There are never keys in the code repository, only a .env.example with placeholders.
The rest of the security is good habits you already saw in the Dockerfile. The container runs as an unprivileged user (useradd --uid 10001), not as root, so a process failure does not compromise the host. It is worth mounting the filesystem as read-only and limiting network egress to the domains the agent actually needs, so that, if the agent runs code or tools, it cannot exfiltrate data to any destination. If the agent does run generated code, do it in an isolated environment (a sandbox), never in the service container itself.
Frequently asked questions
Do I need a framework or is a FastAPI container enough?
It depends on the complexity. For a single-loop agent, a container with FastAPI wrapping the OpenAI Agents SDK is enough and gives you full control. When the agent grows into a graph with several steps, branches and resumption of long tasks, a framework like LangGraph brings the checkpoints and state management already solved, and its CLI generates the Dockerfile and the deployment for you. The practical advice is to start simple and adopt the framework when the state gets complicated.
How do I scale the agent when more traffic arrives?
Horizontally. Because the container holds no state (it lives in Postgres and Redis), you launch more identical replicas of the image behind a load balancer that spreads the requests. The upper limit is usually not your server but the model provider’s rate limits: check how many requests per minute it allows you and ask for a quota increase before traffic reaches it. For irregular spikes, autoscaling on the number of queued requests works well.
Can I deploy the agent without depending on a paid API?
Yes. The OpenAI Agents SDK supports over 100 models, including those you serve on your own server. You can run the model locally with Ollama or with vLLM, expose its OpenAI-compatible API and point the agent at that URL. That way the same service container works the same with a proprietary model in the cloud or an open one running on your infrastructure, without changing the agent’s code.
Conclusion
Taking an agent to production is, above all, a shift in mindset: moving from the script you run by hand to the service that runs on its own. The path has five clear stretches. You wrap the logic in an HTTP server with FastAPI; you package it in a minimal, reproducible Docker image; you push the state out to Postgres and Redis so you can replicate; you instrument every model call and protect it with timeouts, retries and limits; and you manage secrets outside the image, with the container running unprivileged. With those five pieces, your agent stops being a demo and becomes a service you can rely on. The next step is to take the Dockerfile above, pin your dependencies, build the image and deploy the first version.
Sources: [1] OpenAI Agents SDK documentation[1], [2] LangGraph deployment[2], [3] Multi-stage builds in Docker[3], [4] openai-agents package on PyPI[4].
