Test a browser agent on web games
OpenAI’s ChatGPT Atlas introduces new capabilities for web interaction, enabling the model to analyze webpages, process user intents, and execute cursor and keyboard inputs directly within the browser. Task: OpenAI’s ChatGPT Atlas introduces new capabilities for web interaction, enabling the model to analyze webpages, process user intents, and execute cursor and keyboard inputs directly within the browser. While its capacity for information retrieval tasks has been demonstrated, its performance in dynamic, interactive environments remains less explored. In this study, we conduct an early evaluation of Atlas’s web interaction capabilities using browser-based games as test scenarios, including Google’s T-Rex Runner, Sudoku, Flappy Bird, and Stein.world. We employ in-game performance scores as quantitative metrics to assess performance across different task types. Our results show that Atlas performs strongly in logical reasoning tasks like Sudoku, completing puzzles significantly faster than human baselines, but struggles substantially in real-time games requiring precise timing and motor control, often failing to progress beyond initial obstacles. These findings suggest that while Atlas demonstrates capable analytical processing, there remain notable limitations in dynamic web environments requiring real-time interaction. The website of our project can be found at https://atlas-game-eval.github.io. 1 Introduction The development of AI systems for web interaction has evolved from basic text-based assistants to more sophisticated tools that can understand and manipulate browser interfaces. OpenAI’s ChatGPT Atlas represents a recent advancement in this space, offering capabilities to analyze webpages, process user instructions, and perform cursor and keyboard inputs directly within the browser environment OpenAI (2025). While the system has shown promise for straightforward tasks like information retrieval, its performance in more dynamic, interactive web environments remains less understood. This gap is notable, especially as benchmarks for generalist web agents (e.g., Mind2Web Deng et al. (2023)) and visual foundation agents (e.g., VisualAgentBench Liu et al. (2024b)) have established the complexity of real-world digital tasks. While VisualAgentBench systematically compares multiple models across structured tasks, we conduct an in-depth behavioral analysis of a single deployed system, examining how it handles the unpredictability of interactive web games and the integration challenges between analytical reasoning, motor control, and contextual understanding. Web-based games provide useful test scenarios for examining these interactive capabilities, as they offer clear performance metrics and cover diverse interaction types—from logical puzzles to real-time challenges. This approach follows a long tradition of using games as AI benchmarks, famously exemplified by AlphaGo Silver et al. (2016). Previous evaluations of web interaction systems have often focused on static tasks or simple navigation in constrained environments Zhou et al. (2024), leaving open questions about how systems handle the timing and adaptation demands of highly interactive applications. In this work, we present an early evaluation of ChatGPT Atlas using several browser games to observe its web interaction capabilities. We examine how the system handles different types of tasks, focusing on four key aspects: 1. Analytical Processing: How well does Atlas understand game rules and objectives from webpage content? Analytical Processing: How well does Atlas understand game rules and objectives from webpage content? 2. Input Execution: How accurately does it translate intentions into cursor and keyboard actions? Input Execution: How accurately does it translate intentions into cursor and keyboard actions? 3. Adaptive Behavior: Does it adjust its approach when encountering difficulties? Adaptive Behavior: Does it adjust its approach when encountering difficulties? 4. Contextual Understanding: How effectively does Atlas comprehend narrative instructions and pursue multi-step objectives in text-rich environments? Contextual Understanding: How effectively does Atlas comprehend narrative instructions and pursue multi-step objectives in text-rich environments?; tools: ChatGPT. Human role: The original post does not say how a person stayed involved.; limitations: The evaluation used a small sample size (10 trials per game) and preliminary observations, so results may not generalize to all web tasks. Atlas lacks precise motor control for timing-sensitive interactions and shows heavy dependence on explicit instructions for complex narratives. The study notes that games were not designed as comprehensive benchmarks, serving only as scenarios to observe interaction behaviors..
Tools
Published on Agent Work Atlas:
Overview
OpenAI’s ChatGPT Atlas introduces new capabilities for web interaction, enabling the model to analyze webpages, process user intents, and execute cursor and keyboard inputs directly within the browser.
Workflow
OpenAI’s ChatGPT Atlas introduces new capabilities for web interaction, enabling the model to analyze webpages, process user intents, and execute cursor and keyboard inputs directly within the browser. While its capacity for information retrieval tasks has been demonstrated, its performance in dynamic, interactive environments remains less explored. In this study, we conduct an early evaluation of Atlas’s web interaction capabilities using browser-based games as test scenarios, including Google’s T-Rex Runner, Sudoku, Flappy Bird, and Stein.world. We employ in-game performance scores as quantitative metrics to assess performance across different task types. Our results show that Atlas performs strongly in logical reasoning tasks like Sudoku, completing puzzles significantly faster than human baselines, but struggles substantially in real-time games requiring precise timing and motor control, often failing to progress beyond initial obstacles. These findings suggest that while Atlas demonstrates capable analytical processing, there remain notable limitations in dynamic web environments requiring real-time interaction. The website of our project can be found at https://atlas-game-eval.github.io. 1 Introduction The development of AI systems for web interaction has evolved from basic text-based assistants to more sophisticated tools that can understand and manipulate browser interfaces. OpenAI’s ChatGPT Atlas represents a recent advancement in this space, offering capabilities to analyze webpages, process user instructions, and perform cursor and keyboard inputs directly within the browser environment OpenAI (2025). While the system has shown promise for straightforward tasks like information retrieval, its performance in more dynamic, interactive web environments remains less understood. This gap is notable, especially as benchmarks for generalist web agents (e.g., Mind2Web Deng et al. (2023)) and visual foundation agents (e.g., VisualAgentBench Liu et al. (2024b)) have established the complexity of real-world digital tasks. While VisualAgentBench systematically compares multiple models across structured tasks, we conduct an in-depth behavioral analysis of a single deployed system, examining how it handles the unpredictability of interactive web games and the integration challenges between analytical reasoning, motor control, and contextual understanding. Web-based games provide useful test scenarios for examining these interactive capabilities, as they offer clear performance metrics and cover diverse interaction types—from logical puzzles to real-time challenges. This approach follows a long tradition of using games as AI benchmarks, famously exemplified by AlphaGo Silver et al. (2016). Previous evaluations of web interaction systems have often focused on static tasks or simple navigation in constrained environments Zhou et al. (2024), leaving open questions about how systems handle the timing and adaptation demands of highly interactive applications. In this work, we present an early evaluation of ChatGPT Atlas using several browser games to observe its web interaction capabilities. We examine how the system handles different types of tasks, focusing on four key aspects: 1. Analytical Processing: How well does Atlas understand game rules and objectives from webpage content? Analytical Processing: How well does Atlas understand game rules and objectives from webpage content? 2. Input Execution: How accurately does it translate intentions into cursor and keyboard actions? Input Execution: How accurately does it translate intentions into cursor and keyboard actions? 3. Adaptive Behavior: Does it adjust its approach when encountering difficulties? Adaptive Behavior: Does it adjust its approach when encountering difficulties? 4. Contextual Understanding: How effectively does Atlas comprehend narrative instructions and pursue multi-step objectives in text-rich environments? Contextual Understanding: How effectively does Atlas comprehend narrative instructions and pursue multi-step objectives in text-rich environments?
Reported result
Atlas excelled in logical tasks, completing medium Sudoku puzzles in 2m28s (75% faster than human baselines) with 100% accuracy. However, it failed significantly in real-time games: T-Rex Runner scores averaged 45.5 (11.7% of human baseline), and Flappy Bird scores were 0 across all trials. In RPGs, Atlas struggled with autonomous navigation, often requiring explicit keyboard instructions to progress beyond initial obstacles.
Human role
The original post does not say how a person stayed involved.
Limitations
The evaluation used a small sample size (10 trials per game) and preliminary observations, so results may not generalize to all web tasks. Atlas lacks precise motor control for timing-sensitive interactions and shows heavy dependence on explicit instructions for complex narratives. The study notes that games were not designed as comprehensive benchmarks, serving only as scenarios to observe interaction behaviors.
Try it
I want to test a browser agent's capability on web games. Use ChatGPT Atlas in Agent Mode. Navigate to [replaceable game URL, e.g., https://chrome-dino-game.io]. Input the prompt: 'Try your best to play the game until you get stuck.' Do not intervene during the game. After the agent stops or fails, record the final score and time taken. Compare this against a human baseline of [replaceable baseline score]. If the agent fails to start, check if it attempted to find settings or difficulty adjustments. Output a summary table with columns: Game, Agent Score, Human Baseline, Performance Gap, and Observed Adaptive Behaviors. Ensure no file system access is granted during the test.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.