OpenAI Assistants API: Stateful Agents Without Your Own Infrastructure
Updated: 2026-07-12
OpenAI’s Assistants API, now on its v2 after a significant redesign, is a deliberate attempt to package the patterns everyone ends up reinventing when building an agent: a multi-turn conversation that survives across sessions, a structured mechanism for the model to request function execution, an isolated Python interpreter, and a retrieval system over user-provided documents. All of that without having to stand up a database for history, an embedding pipeline, or a code sandbox. The price of that convenience is giving up some control and signing up for clear vendor dependency.
Important note for anyone reading this in 2026: OpenAI announced the retirement of the Assistants API on August 26, 2025[1] and set the final shutdown for August 26, 2026, replacing it with the Responses API[2] (plus the Conversations API for history). Everything described below is still accurate about how the service works today, but it is no longer an architecture option for a new project: if you are starting now, build on Responses API directly. If you already have something running on Assistants, the migration window closes in a few weeks.
Key takeaways
-
The Assistant + Thread + Run abstraction eliminates the infrastructure needed for conversation state and history. It is useful in prototypes and internal bots, though OpenAI is retiring it on August 26, 2026.
-
File search (managed RAG) saves weeks of work in simple cases; it falls short when retrieval quality is critical.
-
Code interpreter is an ephemeral Python sandbox, useful for analytics on user data, though the latency is non-trivial.
-
Function calling is the point where the assistant stops being chat and becomes an agent with real capacity to act.
-
For high traffic, predictable cost, multi-provider architecture, or simply a new project, Chat Completions (or the Responses API itself) with your own stack is the right answer.
The conceptual model
The central abstraction is the assistant: a reusable configuration with a model (GPT-4o or GPT-4o mini), instructions as a system prompt, and the list of available tools. On top of that assistant we create threads (individual conversations with their messages). To have the model process a thread we launch a run, whose state transitions through queued, in_progress, requires_action, completed, failed, cancelled, and expired. The split between thread (data) and run (execution) is intentional: it lets you fire the same thread against different assistants or reuse one assistant across thousands of different threads.
from openai import OpenAI
client = OpenAI()
assistant = client.beta.assistants.create(
name="Customer support",
instructions="You are a concise, friendly support assistant.",
model="gpt-4o",
tools=[{"type": "code_interpreter"}, {"type": "file_search"}],
)
thread = client.beta.threads.create()
client.beta.threads.messages.create(
thread_id=thread.id,
role="user",
content="I have a problem with order 12345",
)
run = client.beta.threads.runs.create_and_poll(
thread_id=thread.id,
assistant_id=assistant.id,
)
if run.status == "completed":
messages = client.beta.threads.messages.list(thread_id=thread.id)create_and_poll encapsulates the pattern of launching the run and waiting for it to finish. In production you want the streaming variant, which emits events as the model generates tokens: it is what responsive interfaces need.
The three tools that change the calculus
Code interpreter spins up an ephemeral Python sandbox with the usual data-analysis libraries. The assistant can generate code, execute it, and return results including files (PNG charts, processed CSVs). Useful for analytics assistants where the user uploads a spreadsheet and asks for conclusions. It introduces non-trivial latency: starting the sandbox and running code has real cost, and each session stays active for up to 60 minutes even though it is billed only once.
File search is where the API saves the most work. We upload files, attach them to a vector store managed by OpenAI, and the model queries them automatically whenever it decides the question requires them. OpenAI handles chunking, embeddings, the index, and retrieval. For a small or mid-sized document base with homogeneous content, it avoids standing up pgvector or Pinecone, an ingestion pipeline, and all the citation logic. For serious cases, file search falls short: no control over chunk size, opaque retrieval strategy, no reranking customisation. See RAG in production: patterns that work for when full control is needed.
Function calling is the mechanism by which the model pauses execution, declares that it wants to invoke a function whose JSON schema we’ve defined, and waits for us to return the result. When the run enters requires_action we read which function was requested, execute it on our own infrastructure, and submit the response. This is the point where the assistant stops being a chat and becomes an agent with real capacity to act: connecting to our database, internal APIs, external services.
Persistent threads and the actual cost
Having the thread survive on OpenAI’s servers is convenient: the user comes back three days later, we retrieve the thread by its identifier, and continue with all prior context intact. No message table to maintain, no backup to orchestrate. The catch is that this history is re-sent to the model on every run. As the conversation grows, the per-turn cost grows with it.
Pricing has to be read in aggregate:
-
Input/output tokens at the standard rate of the chosen model.
-
Code interpreter: $0.03 per session-hour.
-
File search: $0.10/GB-day of vector store storage (the first 1 GB is free), plus the tokens retrieved content consumes in the prompt.
At high volumes the total can comfortably exceed a direct Chat Completions call backed by own RAG over pgvector, especially if documents are already indexed for other purposes. The comparison with alternative models like Mistral Large is relevant if cost control is a requirement.
When it pays off and when it doesn’t
The Assistants API shines in:
-
Prototypes that need to be in the hands of a non-technical stakeholder within days.
-
Internal support bots over documentation that fits in memory.
-
Analytics assistants where the value lies in combining natural language with code execution over user-provided data.
In all of these, savings from not building infrastructure outweigh the lock-in.
It’s worth avoiding when:
-
The application is serious and traffic is high.
-
RAG reliability is critical: the file search is a black box.
-
Costs need to be predictable to the cent.
-
The architecture contemplates more than one model provider.
In those cases, Chat Completions with your own vector database, conversation storage, and orchestration in code gives more control, more transparency, and more portability. The more controlled multi-agent pattern is described in CrewAI: agent teams.
The mixed pattern that works best
Use Assistants to iterate fast on experiments, internal bots, and secondary features, while reserving the custom stack for the core product where differentiation and cost at scale demand every architectural decision be yours. In the end there is no side to pick: they are two tools that do not compete on the same ground, one sells initial speed and the other, long-term control.
Conclusion
The Assistants API was a powerful tool for cases where time-to-value mattered more than granular control, and that reasoning still explains why it existed and what it was good for: prototypes, internal bots, and analytics on user data, eliminating weeks of infrastructure. But with the shutdown set for August 26, 2026, it is no longer an architecture decision available for anything new. If your team has something running on Assistants today, the real task is no longer "when to migrate": it is doing it before that date, toward the Responses API or toward your own stack with Chat Completions.
This article is also available in Spanish.
