ThinkingBox: checking whether your agent does real work

Your agent nails the demo and fails on a Tuesday afternoon. The pattern is familiar: the conversation reads perfectly, the agent announces it made the change, and the system record is empty. ThinkingBox is the test harness Microsoft published in August 2026 to measure exactly that gap. This article covers what it isolates, what it measures, how it differs from a prompt evaluator or a code sandbox, and how far its verdict actually reaches.

Key takeaways

• ThinkingBox runs the agent against simulated tools that hold state and judges it by what changed, not by the transcript.
• Tools are declared as MCP servers, so the agent sees the same surface it would see in your real system.
• The companion benchmark, ThinkingBox-Bench, holds 507 workflows across five sectors and repeats every task twenty times.
• The best model measured succeeds 65.36% of the time on a first attempt, but only 25.25% of tasks come out right all twenty times.
• The framework is MIT; the data ships separately under CDLA-Permissive-2.0. This is genuinely open source, not an announcement.

What ThinkingBox is and what it measures

ThinkingBox is a Python framework for defining mock tools as MCP servers, running an agent against them and evaluating the outcome. Microsoft published it at github.com/microsoft/thinkingbox^[1] under MIT, alongside a second repository, thinkingbox-data^[2], which holds the scenarios, the tool servers and the datasets.

The repository’s own description places it in three uses: generating conversations for offline training or evaluation, putting the whole system inside a reinforcement-learning loop, and evaluating models. The third is the one that matters if you already have an agent running.

What separates it from almost everything else is where it looks. The accompanying paper, submitted to arXiv on 20 August 2026 with twelve authors, defines it as a sandbox providing "isolated MCP-compatible tool sessions, complete execution traces, and outcome evaluation over terminal backend state".

Why the transcript is not evidence

The example Liang-Chun Tsai opens Microsoft’s announcement with is deliberately mundane. You ask a travel support agent to add a quiet-room preference to your hotel booking, away from elevators and ice machines. The agent collects your booking reference, checks the reservation and says the request has been added.

The conversation looks like a success. The booking’s special_requests field is still empty.

No evaluator that reads the response catches that failure. Neither does one that checks the sequence of tool calls, because the sequence can be right and the effect never happen. Tsai puts it plainly: "Tool order measures conformity to one reference trajectory." A reasonable agent might inspect the support ticket first, skip a redundant profile lookup or retry a read after a transient error, and any of those paths completes the task.

So ThinkingBox checks the records left behind. The question it answers is not whether the agent talked well, but whether the system ended up the way it should have.

How it works inside

The central piece is the MCP Session Proxy, a long-running HTTP server listening on port 7111 that fronts a fleet of tool processes. Each of those is an independent MCP server the proxy talks to over stdio with JSON-RPC, one process per server. If the protocol underneath interests you, we have a complete guide to MCP and another on building your own MCP server.

The proxy exposes five operations, and the list alone tells you the design:

Every tool server implements two reserved hooks: __reserved__init, which seeds the starting state, and __reserved__geteffects, which surfaces what was modified. That is the whole trick. Because the scenario always starts from the same state and the server itself knows how to declare what it touched, the final check can be exact rather than approximate.

A full attempt uses three separate models: one plays the agent, another a simulated user answering follow-up questions, and a third acts as judge where judgement is needed. The verdict is a conjunction: every check has to pass. And the checks are written to accept valid trajectories while rejecting wrong, missing or extra effects. That last case is the one to remember: an agent that books the room and also cancels something else has not completed the task.

Installing and running it

The tested target is Linux, WSL included, with Python 3.12. The recommended install uses uv:

git clone https://github.com/microsoft/thinkingbox.git
git clone https://github.com/microsoft/thinkingbox-data.git

cd thinkingbox
uv venv --python 3.12
uv sync --group dev

Everything goes through a single command, tb, with eight subcommands. infer runs one case or a set, mcp-start brings up the proxy, agg aggregates metrics and sbs compares a candidate against a baseline. pp prints a readable result, plus dump-tests, run-test and tui for an interactive session.

The framework repository ships a single scenario, cloud_drive, whose entire purpose is to confirm the install works. In one terminal you start the proxy, in another you launch the case:

uv run tb mcp-start

uv run tb infer -c config/config_o4mini.yaml --dataset ./dataset \
  --agent think --name cloud_drive.py:test_append_some_more_text \
  --output output.yaml
uv run tb pp output.yaml

For real work you point the proxy at the data repository’s catalogue with uv run tb mcp-start --servers ../thinkingbox-data/servers/servers.yaml. Models are configured against Azure OpenAI, after an az login, or against any OpenAI-compatible deployment, with a vLLM example included.

What ThinkingBox-Bench revealed about 2026 models

The benchmark holds 507 policy-conditioned workflows across five sectors: retail, travel and hospitality, auto insurance, internal IT support at a neobank, and IT and HR support in consulting. Twelve models were measured, six proprietary and six open-weight, with twenty attempts per task.

The result worth keeping fits in one line of the paper’s abstract. The strongest model reaches 65.36% success on a first attempt, but only 25.25% counting the tasks it gets right all twenty times.

Tsai frames it in a sentence worth keeping handy the next time someone shows you a demo: "Retries raise the chance of getting one good result without making the agent dependable."

There is a detail I find more uncomfortable than the percentages. According to the paper, a failed trial tends to terminate cleanly and with valid state-changing actions. The agent does not break: it does something reasonable that turns out to be the wrong thing. No alarm in your system is going to fire for that.

How it differs from DeepEval, promptfoo or E2B

This is where people get confused, because all four are marketed as tools for testing agents and none of them compete with each other.

The line separating ThinkingBox from the first two is persistence. DeepEval and promptfoo evaluate an output against a criterion. ThinkingBox lets the agent act over a sequence of turns on a system that remembers, then compares that system with what should have happened.

The line separating it from E2B is simpler still: E2B gives you a safe place to execute, and passes no judgement at all. They are different layers, and in a serious system you end up wanting all three.

What a task suite worth running looks like

If you are going to write your own cases, the ThinkingBox design suggests four requirements that hold up outside it too.

The first is that the task must change something. A question answered by reading does not separate a competent agent from one that sounds competent.

The second is that the checks must reject extra effects rather than only confirm the expected ones; without that, an agent that does what was asked plus three other things passes. The third is that some information should be missing at the start so the agent has to ask for it, because coordinating across turns is precisely what breaks in production. The fourth is repetition: twenty attempts per task sounds excessive until you see that the difference between 65% and 25% lives there.

It also pays to write the domain policy down, because half the interesting failures are not about capability but about obeying a rule nobody wrote.

The limits of testing inside a sandbox

The authors are fairly honest about this, and it is worth repeating because it is what press coverage skips.

In 477 of the 507 tasks the verdict depends only on the final state and the side effects. They spell out the consequence themselves: a trial that executes the correct state transition while misreporting it to the user might still be scored as a success. Only a subset also checks properties of the final response.

The tasks are synthetic reconstructions from a non-public collection, and the paper says explicitly that they are not claimed to represent the distribution of enterprise work. On top of that, every trajectory is shaped by one fixed simulated user, more tractable than a person: it never states an unsupported fact, never revises its goal, and stays cooperative after repeated failures.

Translated to your case: a good result here is necessary and not sufficient. You still need to see inside the system while it is running, with something like Langfuse for agent observability. And you still need to decide what happens when a step dies halfway, which is the territory of durable execution with Temporal.

Frequently asked questions

Do I need Azure to run ThinkingBox?

No. The documented example uses Azure OpenAI, but the framework accepts any OpenAI-compatible deployment and ships a sample vLLM configuration. You do need Linux or WSL and Python 3.12.

Can I use their data in my own benchmark?

Yes, with one condition. The framework is MIT, but the data repository is licensed by content type. Code under servers/ is MIT, and the data under dataset/, support/ and releases/ is CDLA-Permissive-2.0^[3], which requires you to ship the licence file alongside the data when you redistribute it.

Does ThinkingBox replace my current tests?

It replaces nothing. It covers a gap that response-level evaluations leave open, which is verifying the effect. If you are about to deploy an AI agent to production, this is one more layer, and probably the one you were missing.

Conclusion

ThinkingBox is not a tool you will install on Monday and maintain for the rest of the year. It is a small research framework, with 25 stars and thirteen commits on its main branch as I write this, and it shows.

What deserves adopting immediately is its criterion: judge the agent by what changed, require it to reject extra effects, and repeat every task until reliability and capability stop being confused with each other. A 65% that becomes a 25% when you repeat twenty times is the only figure you need to justify that change of criterion. The Spanish version of this article is at ThinkingBox: comprobar si tu agente hace trabajo de verdad.

Sources

1. github.com/microsoft/thinkingbox
2. thinkingbox-data
3. CDLA-Permissive-2.0
4. Microsoft Command Line, how we built ThinkingBox
5. One Success Isn’t Reliability, arXiv 2608.19741

Route: AI Agents in Production: Evaluation and Reliability