Play Wordle in a browser with a computer-using agent
Submission history. Task: Researchers used OpenAI's Computer-User Agent (CUA) to navigate the New York Times Wordle game via a standard browser interface. The agent attempted to identify the input grid, submit guesses, and interpret color-coded feedback to deduce the target word. This workflow served as a benchmark to evaluate multimodal agent capabilities and identify behavioral shortcomings in visual recognition tasks.; tools: Operator. Human role: The original post does not say how a person stayed involved.; limitations: The primary constraint is the agent's unreliable color recognition, which directly impacts its ability to interpret game feedback. Integration requires a standard computer interface and access to the specific web-based game. The source indicates that current frontier models struggle with these visual tasks, suggesting high failure rates for similar automation attempts..
Tools
Published on Agent Work Atlas:
Overview
Submission history
Workflow
Researchers used OpenAI's Computer-User Agent (CUA) to navigate the New York Times Wordle game via a standard browser interface. The agent attempted to identify the input grid, submit guesses, and interpret color-coded feedback to deduce the target word. This workflow served as a benchmark to evaluate multimodal agent capabilities and identify behavioral shortcomings in visual recognition tasks.
Reported result
The agent achieved a 5.36% success rate over several hundred runs conducted across one week. The study identified a significant discrepancy in the model's ability to correctly recognize colors depending on the context, highlighting substantial challenges for frontier AI models even in simple tasks.
Human role
The original post does not say how a person stayed involved.
Limitations
The primary constraint is the agent's unreliable color recognition, which directly impacts its ability to interpret game feedback. Integration requires a standard computer interface and access to the specific web-based game. The source indicates that current frontier models struggle with these visual tasks, suggesting high failure rates for similar automation attempts.
Try it
Act as a QA engineer evaluating an AI agent's visual reasoning. Use OpenAI's Computer-User Agent (CUA) to play 10 rounds of New York Times Wordle in a browser. [replaceable inputs: specific game URL, number of rounds]. For each round, capture screenshots of the agent's view before and after submitting a guess. Log the agent's proposed guess, the actual feedback colors (green, yellow, gray), and whether the agent correctly interpreted the feedback. Calculate the success rate and identify specific instances where color recognition failed. Do not publish or share these logs externally. Provide a summary report detailing the failure modes observed in color interpretation. If the agent fails to load the page or submit a guess, log the error and proceed to the next round. Ensure no real user accounts are used; use a guest or test session only.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.