When Anthropic released Computer Use in October 2024[1], the reaction was a mix of awe and skepticism. On one hand, a model that could look at a screen, move the mouse, and type. On the other, too many doubts about reliability, security, and cost to imagine it in production. Anthropic itself described it at the time as "still experimental," adding that it was "at times cumbersome and error-prone."

Nearly nine months later, the question has changed. Some teams have taken it to real tasks. Patterns are emerging about where it works, where it isn’t yet suitable, and how to configure it so the promise doesn’t collide with operational reality.

For broader context on AI agents in enterprise flows, the analysis of AI agents in the enterprise covers adoption criteria. Security patterns for agents operating with system access are covered in LLM agent security. The post on CI with AI agents describes a more bounded and controlled agentic use scenario.

Key takeaways

  • Current error rates are 5–15% on well-specified tasks, down from 30–40% at launch; per-step latency and cost have roughly halved.

  • Works well for data extraction from interfaces without an API, repetitive bureaucratic flow navigation, and visual validation of web application changes.

  • Not suitable for tasks with irreversible consequences without human approval, information-dense interfaces, or tasks exceeding 50 steps without intervention.

  • The double-check pattern (the agent captures the result and compares it to the stated expectation before advancing) reduces error on long tasks at the cost of doubling token spend.

  • Approximate cost: 50–150 cents per 20-step task; around 200 euros annually for a task run once per day.

What has changed since October 2024

The initial version of Computer Use was a functional proof of concept. On the OSWorld benchmark, Anthropic reported 14.9% success in screenshot-only mode and 22% with extended steps, against 7.8% for the closest comparable system: a real improvement, but far from reliable.

The model understood screenshots and could decide where to click and how to type. Latency was high, error rate on long tasks was 30–40%, and each step consumed significant tokens.

Simon Willison tested the initial release[2] and found the same pattern. Structured tasks like browsing a website or installing a package worked on the first try, but it failed at something as simple as solving a sudoku. For demos it impressed; for real work it frustrated.

Later releases have improved on three axes:

  • Per-step latency has dropped roughly in half.

  • Error rate on well-specified tasks has come down to the 5–15% range depending on complexity.

  • Token consumption per step has dropped thanks to better visual compression.

Not magic yet, but it has crossed the threshold where certain tasks begin to be economically viable.

Where it’s working now

The first category where Computer Use shines is data extraction from interfaces without an API. There are old enterprise applications, vendor portals, internal systems with green-screen layouts where navigating with an agent is cheaper than writing a fragile scraper or paying for custom integration. In a project seen up close, a team replaced four weeks of scraper development with two days of agent configuration, with better maintainability because the agent adapts to minor interface changes without sharp breakage.

The second category is navigation of repetitive bureaucratic flows: filling multipage forms with conditional logic, downloading periodic reports from portals with authentication, marking sets of items in filtered lists. These are tasks where classic RPA works, but where RPA configuration cost exceeds that of an agent that understands natural language and doesn’t need every step explicitly recorded.

The third, more experimental, is visual validation of changes in web applications. An agent can open a preview branch, navigate critical screens, describe what it sees, and compare with the previous version. It doesn’t replace automated tests, but catches visual and flow regressions that unit tests don’t cover.

Where it isn’t yet suitable

The list of cases where Computer Use is still problematic is longer than where it works well:

  • Tasks with irreversible consequences without human approval: purchases, shipments, financial approvals. The error cost is asymmetric: the agent can err with low probability but the loss when it does is large.

  • Information-dense interfaces: large spreadsheets, dashboards with dozens of fields, tables with complex horizontal scroll. On those screens the agent loses track.

  • Multilingual or ambiguous content: if the agent has to decide whether a button says "Send" or "Cancel" in a medium-resolution screenshot, the outcome depends too much on factors the model doesn’t control.

  • Tasks exceeding 50 steps without intervention: if each step has 95% success, a 50-step task has 7% probability of completing without error. For 500-step tasks, completing without intervention is essentially impossible.

Emerging patterns

The first pattern consolidating is double-check execution. The agent makes a step, captures the result, compares against the expectation declared at the start, and only advances if there’s a match. This pattern roughly doubles token cost, and it buys a lower error rate on long tasks. It’s the equivalent of programming with assertions at each step.

The second is supervised execution. A cheaper model or a classic rule system watches the main agent’s actions and stops execution if trouble signs appear: click loops in the same place, error messages, unexpected screens. Separating execution and supervision into two processes is more expensive but much more robust.

The third, more recent, is execution in an isolated environment with deferred commit. The agent works on a temporary environment, whether a virtual machine, a container, or a private browser session, and changes only apply to the real environment after a review step approves them. This pattern is heavier but fits well for high-impact tasks.

Real cost

A task the agent completes in 20 steps consumes on the order of 50 to 150 cents, depending on the model and screenshot size. For tasks run once a day, the annual cost per task is around 200 euros. For tasks run more than once a day, the math scales fast.

The useful comparison isn’t against zero, it’s against the cost of an alternative:

  • A custom scraper costs weeks of development and ongoing maintenance.

  • Formal integration via API, when it exists, costs double.

  • A person doing the same task costs much more per hour.

Computer Use starts to make economic sense when the task is repetitive but changing, the API doesn’t exist, and volume is enough to amortize initial configuration.

Practical security

The least-discussed risk is security. An agent operating interfaces with credentials lives close to any data visible on screen. Suppose the machine where the agent operates has access to email, internal systems, or authenticated browsers. Any agent failure, or malicious instruction injected into what it sees, can lead it to act outside the intended perimeter.

Anthropic’s own tool documentation[3] warns about exactly this: instructions hidden in a page or an image can override the user’s. Anthropic recommends limiting use to virtual machines or containers with minimal privileges.

The recommendation for any serious use is to run the agent in a dedicated virtual machine, with credentials scoped to the specific task, without access to anything else. The temptation to reuse the user’s authenticated session is huge because it simplifies setup, but it’s the quick door to an incident. Security incidents with computer use agents almost all come through that door: overly generous credentials, overly wide perimeters, no supervision.

When it pays off

My practical rule is: Computer Use pays off when the task is repetitive, lacks an API, the error cost is low or reversible, and volume justifies the initial setup. Outside those four requirements, the equation complicates fast.

For teams that want to explore, the best path is a specific, measured task with bounded budget. Configure the agent, measure success rate for two or three weeks, compute real cost and decide whether to scale. Starting with big ambition and vague strategy is the fast way to spend money without learning anything.

Within two years Computer Use will be in more flows than we imagine today, though not as a replacement for people in complex tasks. Instead, it will replace integration layers currently done with fragile scrapers and RPA. The likely pattern is agents as bridges between old systems and new ones, operating in short supervised shifts with limited authority.

This article is also available in Spanish: Computer Use en producción: agentes que manejan la interfaz.

Sources:

  1. Anthropic: Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku[1]
  2. Simon Willison: Initial explorations of Anthropic’s new Computer Use capability[2]
  3. Anthropic (platform.claude.com): Computer use tool documentation[3]

Frequently asked questions

What is Computer Use's error rate today compared with launch?

On well-specified tasks the error rate sits in the 5-15% range depending on complexity. At the October 2024 launch it was 30-40% on long tasks, when Anthropic reported 14.9% success on OSWorld in screenshot-only mode. Per-step latency and token consumption have roughly halved. Even so, with 95% success per step, a 50-step task completes without error only 7% of the time.

How much does running a task with Computer Use cost?

A 20-step task consumes on the order of 50 to 150 cents depending on the model and screenshot size. Run once a day, that is around 200 euros annually, and with more than one run a day the math scales fast. The double-check pattern, where the agent compares each result against the stated expectation before advancing, roughly doubles token spend, and it buys a lower error rate on long tasks. The useful comparison is against a custom scraper or a person, not against zero.

Can I let the agent use my already authenticated browser session?

Better not: it is the quick door to an incident. An agent that sees the screen lives close to any visible data, and instructions hidden in a page or an image can override the user's, as Anthropic's own tool documentation warns. The recommendation for any serious use is to run the agent in a dedicated virtual machine, with credentials scoped to the specific task and no access to email, internal systems, or other authenticated browsers. Almost all incidents come from overly generous credentials and no supervision.

Sources

  1. released Computer Use in October 2024
  2. Simon Willison tested the initial release
  3. Anthropic’s own tool documentation