Computer Use: Definition and Examples

Computer Use refers to an advanced feature that allows an artificial intelligence model to take control of a computing environment. Concretely, the AI can see what is displayed on the screen via screenshots, move the mouse cursor, click on elements, type text on the keyboard, and navigate between different applications.

This capability represents a major paradigm shift in human-machine interaction. Rather than being limited to generating text or code, the AI becomes a true agent capable of executing complex tasks in a real graphical environment. For example, it can fill out a web form, manipulate a spreadsheet, use design software, or perform internet searches autonomously.

Computer Use relies on a perception-action loop: the model receives a screenshot, visually analyzes its content, decides on the action to perform (click, keystroke, scroll), then observes the result to plan the next step. This iterative approach allows managing multi-step workflows without constant human intervention.

Introduced by Anthropic with Claude in October 2024, Computer Use paves the way for automating repetitive tasks that previously required human intervention. It differs from APIs and traditional scripts because it interacts with existing graphical interfaces without requiring prior technical integration with the software used.