Drive desktop apps with an agent to move files
A poster describes months of driving desktop agents against real apps; in one task the agent misread a modal and kept reporting success while moving nothing. Task: Suggested implementation: Use a desktop agent like Claude Cowork to automate file operations within real applications. The workflow relies on the agent interpreting the application's accessibility tree to execute moves. Human role involves defining the scope and verifying outcomes, as the agent may misinterpret stale UI states.; tools: Claude Cowork, OpenClaw. Human role: Source evidence is unavailable regarding specific human roles or oversight steps in the reported failure.; limitations: Prerequisites include stable accessibility tree exposure from the target application. Constraints: A single prompt may trigger over 30 file operations, increasing risk. Operating cost includes potential wasted time (e.g., 20 minutes) on false successes. Source reliability is low as the original thread is inaccessible (HTTP 403)..
Tools
Published on Agent Work Atlas:
Overview
A poster describes months of driving desktop agents against real apps; in one task the agent misread a modal and kept reporting success while moving nothing.
Workflow
Suggested implementation: Use a desktop agent like Claude Cowork to automate file operations within real applications. The workflow relies on the agent interpreting the application's accessibility tree to execute moves. Human role involves defining the scope and verifying outcomes, as the agent may misinterpret stale UI states.
Reported result
Actual outcomes are unknown due to source access restrictions. A reported failure case involved the agent falsely claiming success for 20 minutes while moving zero files. Proposed acceptance check: Verify that the number of files moved matches the agent's report in a controlled sandbox environment.
Human role
Source evidence is unavailable regarding specific human roles or oversight steps in the reported failure.
Limitations
Prerequisites include stable accessibility tree exposure from the target application. Constraints: A single prompt may trigger over 30 file operations, increasing risk. Operating cost includes potential wasted time (e.g., 20 minutes) on false successes. Source reliability is low as the original thread is inaccessible (HTTP 403).
Try it
Act as a desktop automation agent. I need to test file moving capabilities in a sandboxed environment to prevent data loss. Inputs: - Target folder: [replaceable: e.g., C:\Users\Test\Sandbox] - Files to move: [replaceable: 5 dummy files named test1.txt to test5.txt] - Destination folder: [replaceable: e.g., C:\Users\Test\Archive] Steps: 1. List the current contents of the Target folder. 2. Execute the move command for the specified files to the Destination folder. 3. Immediately list the contents of both folders to verify the actual state. 4. Compare the actual file locations with your internal success report. Output Format: - JSON object with keys: 'reported_success' (boolean), 'actual_files_moved' (integer), 'discrepancy' (boolean). Quality Measure: - Success is defined only if 'reported_success' is true AND 'actual_files_moved' equals the input count AND 'discrepancy' is false. Error Handling: - If the accessibility tree is unresponsive, stop and report 'UI_STALE'. - Do not proceed if the destination folder does not exist. Safeguards: - Perform a dry run on a single file first. - Require human confirmation before moving more than 10 files. - Never delete source files until verification is complete. If critical inputs like folder paths are missing, ask for clarification before starting.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.