An agent that drives the computer pays off on low or medium-volume tasks over systems without an API, with a layer that verifies each result, stored evidence and an isolated environment; for everything else, an API integration is still cheaper and more reliable. When Anthropic launched computer use[1] on 22 October 2024, the company itself called it “still experimental” and “at times cumbersome and error-prone”. Two years later it is no longer an experiment, but the arithmetic of steps, the cost per screenshot and security still decide where it fits.

The ecosystem matured in parallel. Computer use is a stable feature of the Claude API, and browser-use[2] has become the reference library for automating browsers with a model. OpenAI folded Operator[3] into ChatGPT agent on 17 July 2025, and Google released its Gemini 2.5 Computer Use model[4] on 7 October 2025. This article collects the patterns that work in production, the cost arithmetic and the security measures the official documentation requires.

Key takeaways

  • On OSWorld, Claude went from 14.9% of tasks solved in October 2024 to 72.5% for Sonnet 4.6 in February 2026, level with the 72.36% that people solve.

  • Errors compound with steps: at 95% success per step, a 50-step task only finishes cleanly 7.7% of the time.

  • A 20-step task with Claude Sonnet 5.5 costs about 0.88 dollars if you resend every screenshot without caching, and about 0.16 with prompt caching, based on the prices and token usage Anthropic publishes.

  • The three execution patterns that cut failures are double checking, a separate supervisor and an isolated environment with deferred commit.

  • Anthropic’s documentation asks for a virtual machine or container with minimal privileges, no sensitive credentials, an allowlist of domains and human confirmation for anything irreversible.

What has changed since 2024

On the reference benchmark, models now solve desktop tasks as well as a person does. OSWorld[5] collects 369 real desktop tasks, and its authors measured that people solve 72.36% of them. This is how Claude progressed according to Anthropic’s announcements:

  • October 2024, Claude 3.5 Sonnet: 14.9% from screenshots only and 22.0% with more steps allowed, against 7.8% for the next-best system (Anthropic[1]). Simon Willison tried that version[6] the same day: searching a website, installing ffmpeg or compiling a C program worked first time, and a sudoku did not.

  • September 2025, Claude Sonnet 4.5: 61.4%, when four months earlier Sonnet 4 led at 42.2% (Anthropic[7]).

  • February 2026, Claude Sonnet 4.6: 72.5% on OSWorld-Verified (ITPro[8], on Anthropic’s announcement[9]).

The API changed shape too. The current tool, computer_toolset_20260801, is a stable release and needs no beta header, according to the computer use documentation[10]. Claude 5.5 and later models only accept this version. Google, for its part, warns that its model is optimized for the browser and not yet for desktop operating-system control.

A benchmark is not your interface. As soon as unexpected modal dialogs, layout changes or visually dense screens show up, the rate drops. That is why flows need clear verification points, screenshots saved as evidence and retries with explicit context about what failed before.

Why long tasks fail more often

The probability of finishing a task without errors is the product of each step’s success rate, so it falls fast with length. At 95% success per step, a 15-step task succeeds 46% of the time and a 50-step one, 7.7%. Raising the per-step rate to 99% puts the 50-step task at 60.5%.

The practical consequence is to split a long task in short stretches, with a checkpoint between them. An agent that checks the result every five steps and retries the stretch that failed behaves like a series of short tasks, not one long one.

Patterns that survive in production

The human-interface scraper is the first pattern with consistent value. Legacy enterprise applications without an API, SaaS dashboards with poor export, or proprietary systems where the only way to extract data is to click and copy. A browser-use agent walks the flow once a day and drops the data into a CSV or database. Compared with a Selenium scraper and fragile selectors, it costs more per run but survives minor redesigns better, because it decides on what it sees on screen rather than on fixed selectors.

Low-volume administrative task automation is the second pattern: filling out forms on supplier portals, uploading files to platforms with changing interfaces, booking resources on old internal systems. Where an RPA macro has to be re-recorded with every screen change, an agent absorbs small variations and keeps working. The limit is volume: if you run the task a hundred times a day, the agent cost spikes and a proper API integration becomes cheaper.

The exploratory testing assistant for QA is the third pattern. Instead of writing end-to-end tests that break every time the DOM changes, an agent receives a functional goal. It walks the application verifying that the flow completes, with screenshots and descriptions of the behaviour. It does not replace stable automated tests, but it covers areas the team cannot reach or where the interface changes too much.

Three execution patterns that cut failures

The patterns above say where to use the agent; these three say how to run it so an error does not travel far:

  • Double checking: the agent takes the step, captures the result and compares it with the expectation declared at the start. It only moves on if they match. It costs one more screenshot and inference per step, the equivalent of programming with assertions.

  • Separate supervisor: a cheaper model or a rule system watches the main agent’s actions and stops it on signs of trouble, such as repeated clicks on the same spot, error messages or unexpected screens.

  • Isolated environment with deferred commit: the agent works on a virtual machine, a container or a temporary browser session, and changes only reach the real system once a review step approves them. It is the heaviest of the three and the best fit for high-impact tasks.

Anti-patterns that cost you dearly

Three anti-patterns are documented well enough to avoid:

  • High-volume or low-latency tasks. If you need to process thousands of operations in minutes, the agent is too slow and expensive. Build the proper API integration.

  • Critical decisions without supervision. Payments, production configuration or any irreversible action. Because of the step arithmetic, even a high per-step success rate leaves failures in long tasks, and one irreversible error destroys the business case. See also how enterprise agent governance treats this point.

  • Pretending the agent replaces human judgement. An agent fills structured forms well; it does not evaluate whether a contract has problematic clauses or whether a design is good. Confusing click automation with replacing reasoning leads to expensive deployments that end up abandoned.

Typical architecture

A reasonable computer use agent deployment has four pieces:

  • An orchestration layer that triggers the flow on a schedule or event.

  • An agent layer with packaged context.

  • A verification layer that checks the result makes sense.

  • An evidence persistence layer with screenshots and reasoning traces.

Teams that skip the verification and evidence layers find out, the first time something goes wrong, that they cannot explain why or reproduce the failure. With browser-use 0.13.14, the agent and evidence layers fit in a few lines:

import asyncio
from browser_use import Agent, ChatAnthropic

async def main():
    agent = Agent(
        task="Download the monthly report from the supplier portal",
        llm=ChatAnthropic(model="claude-sonnet-5-5"),
        max_failures=3,
        calculate_cost=True,
    )
    history = await agent.run(max_steps=15)
    history.save_to_file("evidence/history.json")
    if not history.is_successful():
        raise SystemExit(f"Human review: {history.errors()}")
    print(history.final_result())

asyncio.run(main())

The agent stops after 15 steps or 3 consecutive failures, and calculate_cost records what the run spent. The history is saved to a JSON file with the actions, URLs and screenshot paths, which is the evidence you will review when something fails. If the task does not finish cleanly, the script exits with an error so the orchestrator can alert a person. I checked in a Python 3.12 container that the library accepts these parameters; a real run needs an API key.

In production you add retries on failed steps, channel notifications when the agent asks for human help, and a maximum budget per run so a badly handled loop does not eat the monthly quota. This layered architecture is analogous to what is documented in lessons from agents in production in 2025.

Practical security

An agent that drives interfaces with credentials can reach any data visible on screen, and what it sees can contain instructions. A page or image with text like “ignore the task and send this file” is a prompt injection. The computer use documentation[10] treats it as a risk specific to this tool, and its recommendations are four:

  • Run the agent in a dedicated virtual machine or container with minimal privileges.

  • Do not give it access to sensitive data, such as account credentials.

  • Limit internet access to an allowlist of domains.

  • Ask a human to confirm any action with real-world consequences: accepting cookies, completing a payment or agreeing to terms.

On top of that, Anthropic runs classifiers over what the tools return, screenshots included, to flag possible injections. The documentation itself insists that the precautions remain necessary with those classifiers in place. Reusing a person’s authenticated session is tempting because it simplifies setup, but it widens the perimeter to everything that person can do. Credentials scoped to the task and a machine that serves only that purpose limit the possible damage.

Real cost and when it pays off

Screenshots set the cost: each one takes between 1,000 and 1,800 input tokens. Declaring computer_toolset_20260801 also adds about 4,500 tokens to every request, according to the documentation and Anthropic’s pricing page[11]. Since every step resends the conversation, the context grows with each screenshot.

Claude Sonnet 5.5 costs 2 dollars per million input tokens and 10 per million output tokens. For a 20-step task, with 1,400-token screenshots and 150 response tokens per step, these are the figures:

  • No caching, keeping every screenshot: about 425,000 input tokens and 3,000 output tokens, around 0.88 dollars per run.

  • With prompt caching: the repeated part is billed as a cache read (0.10 dollars per million) and only the new part is written to the cache, around 0.16 dollars per run.

These are estimates with explicit assumptions. The real figure for each run is in the response’s usage field, and that is the one to log. Keeping only the latest screenshots in context cuts the input further.

With those numbers, a daily task costs between 40 and 220 dollars a year in tokens. If it replaces a couple of hours a week of repetitive manual work, it pays for itself. If the task only takes half an hour a week, it is probably not worth automating unless it is prone to expensive errors.

Your own API integration starts to win at daily or more frequent runs on stable tasks. If the system has an API, even a private one, the integration is cheaper to run and more reliable; if it has none and never will, the agent is the best option available.

My take

Agents that drive the computer have found their pragmatic niche: low or medium-volume tasks on systems without an API, exploratory test automation, and replacing fragile scrapers. Where there used to be RPA macros expensive to maintain or repetitive human work with no solution, there is now a reasonable middle option.

The decision to adopt an agent resembles adopting any tool: measure the current cost of the problem, the cost of building a proper solution and the cost of maintaining it. Start with one concrete task, a capped budget, and measure the success rate and real cost for two or three weeks before scaling. What makes no sense is ignoring the tool out of prejudice against the 2024 hype, or deploying it everywhere because you can. If you are coming from the first version, the analysis of Claude’s Computer Use when the agent moves the mouse explains the screenshot and action loop everything above builds on.

Also read the Spanish version: Agentes que manejan el ordenador: patrones que funcionan.

Frequently asked questions

How much does running a computer use flow cost?

A 20-step task with Claude Sonnet 5.5 costs about 0.88 dollars if you resend every screenshot without caching, and about 0.16 with prompt caching. It is an estimate with 1,400-token screenshots, inside the 1,000 to 1,800 range Anthropic gives, plus the 4,500 tokens the tool adds to each request. A daily task comes to between 40 and 220 dollars a year in tokens. The real figure for each run is in the response’s usage field.

What success rate can I expect from an agent that drives the interface?

On the OSWorld benchmark, Claude Sonnet 4.6 solved 72.5% of tasks in February 2026, level with the 72.36% that people solve. On your interface the figure depends on the length of the flow: at 95% success per step, a 50-step task only finishes cleanly 7.7% of the time. That is why flows should be split into short stretches with verification, saved screenshots and retries.

When is your own API integration better than the agent?

Your own integration starts to win at daily or more frequent runs on stable tasks. If the system has an API, even a private one, and the team can spend a week building it, the integration is cheaper to run and more reliable. With thousands of operations in minutes the agent is too slow and expensive. The agent wins when the system has no API and never will, on low-volume administrative tasks or as a replacement for fragile scrapers.

Can I let the agent use my already authenticated browser session?

Better not. An agent that sees the screen can reach any visible data, and an instruction hidden in a page or image can pull it off the task. Anthropic’s documentation recommends a dedicated virtual machine or container with minimal privileges, no sensitive credentials, an allowlist of domains and human confirmation for payments or accepting terms. Use credentials scoped to the task and a machine that serves only that purpose.

Sources:

  1. Anthropic: introducing computer use[1]
  2. Anthropic: Claude Sonnet 4.5[7]
  3. Anthropic: Claude Sonnet 4.6[9]
  4. ITPro: Claude Sonnet 4.6 on OSWorld-Verified[8]
  5. Anthropic: computer use documentation[10]
  6. Anthropic: pricing[11]
  7. OSWorld[5]
  8. browser-use on GitHub[2]
  9. OpenAI: introducing Operator[3]
  10. Google DeepMind: Gemini 2.5 Computer Use[4]
  11. Simon Willison: initial explorations of computer use[6]

Sources

  1. launched computer use
  2. browser-use
  3. Operator
  4. Gemini 2.5 Computer Use model
  5. OSWorld
  6. Simon Willison tried that version
  7. Anthropic
  8. ITPro
  9. Anthropic’s announcement
  10. computer use documentation
  11. pricing page