Definition
Computer use is an AI system's operation of browser or desktop interfaces. The system observes interface state, chooses an action, and checks the resulting state as it works toward a task. The application supplies the environment and executes the requested operations.
Implementations can use structured mouse and keyboard actions, generated code through interface-control libraries, or existing UI tools. Screenshots are one source of observations. The capability is broader than any one provider's action format.
Origin and attribution
Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang's World of Bits research studied web agents acting through keyboard and mouse at ICML 2017. Anthropic released its Claude computer-use API beta in October 2024. OpenAI introduced the Computer-Using Agent powering Operator in January 2025.
These projects document different approaches to interface operation. Their history does not establish a sole inventor or first coiner of the general term. CUA can denote OpenAI's named model or abbreviate computer-using agent; context is needed before treating it as a product name.
Scope and evidence limits
A visible interface can change between observation and action. Layout changes, ambiguous controls, missed confirmations, and incomplete operations can all affect the result. A requested click records an attempted action; the next observation needs to establish what happened.
Anthropic's research account identifies prompt injection through screenshots. OpenAI's current guide treats screen content as untrusted and requires bounded execution and outcome checking. Instructions visible in a webpage are content encountered during a task, not permission from the user to expand it.
Operational significance
Limit the reachable sites, credentials, files, and actions to the task's scope. Enforce those limits in the application and environment. The ability to reach a send or purchase button does not establish authorization to press it.
Check completion against the requested outcome and retain enough evidence to reconcile uncertain effects. After a lost connection, verify whether an action took effect before repeating it. Use explicit review at consequential boundaries according to the application's policy.
Distinguish it from nearby terms
- Computer vision interprets images. Computer use also operates an interface and checks the effects.
- Function calling can invoke a domain-specific API without visiting its UI. Computer use works through interface state and interaction.
- Browser automation includes fixed scripts. Model-driven computer use can choose actions as it observes the interface.
- Autonomy describes delegated decision authority. A computer-use capability does not determine how much authority the system receives.
- Programmatic tool calling describes how a program coordinates tools. It can implement computer use or other kinds of work.
Check your understanding
An agent clicks Submit on a government form, then loses its connection. What evidence would distinguish an accepted submission from an incomplete attempt, and which permission should have been checked before the click?