Click through macOS app builds to catch visual bugs
Claude Code computer use lets Claude open apps, click through UIs, and test native builds on macOS. Task: Claude Code computer use lets Claude open apps, click through UIs, and test native builds on macOS. Here is how to enable it and what it handles best. The slow part after a coding session is often verification. You finish building a macOS app feature with Claude Code, then stop to manually open Xcode, compile, launch, and click through every screen to check for crashes. Claude Code computer use closes that loop. It lets Claude open your apps, click through UIs, screenshot results, and report back without you ever leaving the terminal. Computer use arrived in research preview with Claude Code v2.1.85, and it fits native build validation and visual bug reproduction especially well. This article covers what it does, how to set it up, and where it earns its place in a real workflow. What Claude Code Computer Use Does Computer use is a built-in capability that gives Claude access to your macOS desktop from within a CLI session. Claude can open applications, click buttons and menus, type into fields, scroll, resize windows, and take screenshots of the results. It sees your screen and acts on it the same way you would with a mouse and keyboard. The feature is most useful for tasks that require a GUI and have no CLI or API alternative. Building a menu bar app, testing an iOS Simulator flow, or reproducing a layout bug that only appears at certain window sizes are all cases where computer use fills a gap that Bash and MCP servers cannot. Claude tries other approaches before falling back to computer use. If a task can be done with a Bash command, Claude runs that first. If you have an MCP server for the target service, Claude uses that. Computer use is the last resort for things that genuinely require a screen. Requirements and Plans Computer use in the CLI is available on Pro and Max plans only. It requires Claude Code v2.1.85 or later, a macOS machine, and an interactive session. It does not work with the -p flag or on Linux or Windows through the CLI. If you are on Windows, the Desktop app provides computer use with a similar interface. It also requires authentication through claude.ai. If you access Claude exclusively through Amazon Bedrock, Google Cloud Vertex AI, or Microsoft Foundry, you need a separate claude.ai account to use this feature. Run /status inside a session to confirm your subscription and verify you meet the requirements. How to Enable It Computer use ships as a built-in MCP server called computer-use. It is disabled by default. Open an interactive Claude Code session and run: Find computer-use in the list and select Enable. The setting persists per project, so you only do this once for each codebase where you want the feature active. The first time Claude tries to use your screen, macOS will ask for two permissions: Accessibility and Screen Recording. Both are required. Open the relevant System Settings panes, grant them, and select Try Again in the Claude Code prompt. You may need to restart Claude Code after granting Screen Recording, since macOS sometimes requires a process restart before the permission takes effect. Per-Session App Approval Enabling computer-use does not grant Claude access to every app on your machine. Each session starts with no app approvals. The first time Claude needs a specific app during a session, a prompt appears in your terminal showing which app is requested, what permissions it needs, and how many other visible apps will be hidden while Claude works. You choose Allow for this session or Deny. Apps with broader reach get an extra warning. Terminals and IDEs are flagged as “equivalent to shell access.” Browsers show a “view-only” note, meaning Claude can read what is on screen but not click interactive elements like form fields in browsers. This tiered trust model is how computer use stays useful without becoming a free pass. How Claude Uses the Screen When Claude acquires control of your screen, other visible apps are hidden so Claude interacts only with the approved application. Your terminal window stays visible and is excluded from screenshots. This means Claude never reads its own output from on-screen captures, which prevents a class of prompt injection where on-screen content could influence Claude’s next action.; tools: Claude Code, Claude computer use. Human role: The original post does not say how a person stayed involved.; limitations: Requires Pro/Max plans, macOS, and Claude Code v2.1.85+. It is a research preview, so behavior may change. Security constraints: Terminals/IDEs are flagged as shell-equivalent; browsers are view-only. Human approval is mandatory for each app access per session. Not available via -p flag or Bedrock/Vertex/Foundry without separate claude.ai auth..
Tools
Published on Agent Work Atlas:
Overview
Claude Code computer use lets Claude open apps, click through UIs, and test native builds on macOS.
Workflow
Claude Code computer use lets Claude open apps, click through UIs, and test native builds on macOS. Here is how to enable it and what it handles best. The slow part after a coding session is often verification. You finish building a macOS app feature with Claude Code, then stop to manually open Xcode, compile, launch, and click through every screen to check for crashes. Claude Code computer use closes that loop. It lets Claude open your apps, click through UIs, screenshot results, and report back without you ever leaving the terminal. Computer use arrived in research preview with Claude Code v2.1.85, and it fits native build validation and visual bug reproduction especially well. This article covers what it does, how to set it up, and where it earns its place in a real workflow. What Claude Code Computer Use Does Computer use is a built-in capability that gives Claude access to your macOS desktop from within a CLI session. Claude can open applications, click buttons and menus, type into fields, scroll, resize windows, and take screenshots of the results. It sees your screen and acts on it the same way you would with a mouse and keyboard. The feature is most useful for tasks that require a GUI and have no CLI or API alternative. Building a menu bar app, testing an iOS Simulator flow, or reproducing a layout bug that only appears at certain window sizes are all cases where computer use fills a gap that Bash and MCP servers cannot. Claude tries other approaches before falling back to computer use. If a task can be done with a Bash command, Claude runs that first. If you have an MCP server for the target service, Claude uses that. Computer use is the last resort for things that genuinely require a screen. Requirements and Plans Computer use in the CLI is available on Pro and Max plans only. It requires Claude Code v2.1.85 or later, a macOS machine, and an interactive session. It does not work with the -p flag or on Linux or Windows through the CLI. If you are on Windows, the Desktop app provides computer use with a similar interface. It also requires authentication through claude.ai. If you access Claude exclusively through Amazon Bedrock, Google Cloud Vertex AI, or Microsoft Foundry, you need a separate claude.ai account to use this feature. Run /status inside a session to confirm your subscription and verify you meet the requirements. How to Enable It Computer use ships as a built-in MCP server called computer-use. It is disabled by default. Open an interactive Claude Code session and run: Find computer-use in the list and select Enable. The setting persists per project, so you only do this once for each codebase where you want the feature active. The first time Claude tries to use your screen, macOS will ask for two permissions: Accessibility and Screen Recording. Both are required. Open the relevant System Settings panes, grant them, and select Try Again in the Claude Code prompt. You may need to restart Claude Code after granting Screen Recording, since macOS sometimes requires a process restart before the permission takes effect. Per-Session App Approval Enabling computer-use does not grant Claude access to every app on your machine. Each session starts with no app approvals. The first time Claude needs a specific app during a session, a prompt appears in your terminal showing which app is requested, what permissions it needs, and how many other visible apps will be hidden while Claude works. You choose Allow for this session or Deny. Apps with broader reach get an extra warning. Terminals and IDEs are flagged as “equivalent to shell access.” Browsers show a “view-only” note, meaning Claude can read what is on screen but not click interactive elements like form fields in browsers. This tiered trust model is how computer use stays useful without becoming a free pass. How Claude Uses the Screen When Claude acquires control of your screen, other visible apps are hidden so Claude interacts only with the approved application. Your terminal window stays visible and is excluded from screenshots. This means Claude never reads its own output from on-screen captures, which prevents a class of prompt injection where on-screen content could influence Claude’s next action.
Reported result
The source describes the capability but reports no measured outcomes, success rates, or time savings. It is unclear if this workflow reduces verification time in practice. Proposed acceptance check: Measure the time difference between manual Xcode verification and automated computer use for a standard build cycle.
Human role
The original post does not say how a person stayed involved.
Limitations
Requires Pro/Max plans, macOS, and Claude Code v2.1.85+. It is a research preview, so behavior may change. Security constraints: Terminals/IDEs are flagged as shell-equivalent; browsers are view-only. Human approval is mandatory for each app access per session. Not available via -p flag or Bedrock/Vertex/Foundry without separate claude.ai auth.
Try it
I want to test Claude Code computer use for verifying a macOS app build. Context: [replaceable: app name, specific UI flow to test, e.g., 'MenuBarStats preferences slider']. Tools: Claude Code, computer-use MCP server. Steps: 1. Ensure I am on Pro/Max plan and v2.1.85+. 2. Run /mcp and enable computer-use. 3. Grant macOS Accessibility and Screen Recording permissions when prompted. 4. Ask Claude to build [replaceable: target], launch it, and interact with [replaceable: UI element]. 5. Request a screenshot of the result. 6. Review the screenshot and Claude's report. 7. Confirm before allowing any further actions. Deliverable: A screenshot and a brief report on whether the UI behaved as expected. Quality check: Verify the screenshot clearly shows the intended state. If inputs are missing, ask me for the specific app target and UI flow.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.