Orchestrate a Spot robot through multi-step tasks
Jul 30, 2026. Task: Jul 30, 2026 x.com Facebook LinkedIn Mail Copy link Gemini Robotics ER 2 represents a step change in powering robots with video understanding, task orchestration, and multi-robot collaboration — making it possible for robots to be more helpful in the physical world. Steven Hansen Senior Staff Software Engineer Peng Xu Staff Software Engineer x.com Facebook LinkedIn Mail Copy link We are launching Gemini Robotics ER 2, a new model designed to act as a high-level brain for robots. It enables real-time spatial reasoning, multi-step task planning, and collaboration between different robots. You can access the model now via the Gemini API, Google AI Studio, or the Gemini Enterprise Agent Platform to start building your own physical AI agents. Google just launched Gemini Robotics ER 2, which acts like a smart brain for robots. It helps them think quickly, plan multi-step tasks and even work together with other robots. The model lets robots watch video feeds to track their own progress and fix mistakes in real time. It’s designed to make robots safer and more helpful in the real world. General summary Basic explainer For robots to assist humans in everyday environments, accurate spatial reasoning is not enough. Robots must also think fast, timing their decisions and reasoning with the real-time speed of the physical world. That’s why today we’re launching Gemini Robotics ER 2, our most capable “embodied reasoning” model for robotics. Think of Gemini Robotics ER 2 as a high-level brain for robots. It allows robots to chat with humans, understand the physical world, and plan multi-step tasks. It then hands off motor execution to any given lower level vision-language-action (VLA) model. Gemini Robotics ER 2 can also natively call tools like Google Search to find information, or any other user-defined function. The design of Gemini Robotics ER 2 allows the robot to “think” about what comes next while simultaneously performing its actions. Gemini Robotics ER 2 represents a significant upgrade over Gemini Robotics ER 1.6. By watching continuous video feeds, robots can now track their own progress, adapt if something goes wrong, and know exactly when to move on to the next step. We are also introducing multi-robot collaboration, enabling robots to work together in shared spaces and complete complex workflows a single robot could not do alone. Gemini Robotics ER 2 is now publicly available to developers via the Gemini API, Google AI Studio, and in private preview on Gemini Enterprise Agent Platform. To help you get started, we’re sharing examples of how to configure the model and prompt it to power more useful physical AI tasks. Advancing physical agentic capabilities Most tasks in the physical world are complex and require multiple steps to complete. Gemini Robotics ER 2 is a physical agent, orchestrating steps for the robot and enabling it to self-correct, and generalize to more novel situations. To build an agentic setup, developers can declare low-level control interfaces — like Vision-Language-Action (VLA) models or navigation APIs — as tools, and stream multimodal video, audio, or text directly into the model. Gemini Robotics ER 2 improves this tool orchestration workflow. We can evaluate its performance with robots in simulation, using real-world robot control, and even pair it with a human controlling the robot remotely. Gemini Robotics ER 2 consistently outperforms ER 1.6 for tool orchestration across three control modes: real VLA, sim VLA, and human tele-op. In robotics, high-level reasoning depends on execution speed. Gemini Robotics ER 2 integrates into the Gemini Live API, using a bidirectional streaming endpoint optimized for latency-sensitive tasks. The result is fluid orchestration: Gemini Robotics ER 2 commands action models and robotics APIs to complete multi-step tasks without the jarring “stop-and-think” pauses.To illustrate this, we’ve built a demo with Spot from our partners at Boston Dynamics. We use Gemini Robotics ER 2 to orchestrate Spot APIs, such as navigation and manipulator movement, creating an interactive robot that fetches objects for you. Gemini Robotics ER 2 powered Boston Dynamic Spot fetches a popcorn snack up on a natural language command. The code is available on Github with other examples. Unlocking temporal intelligence for robust task completion; tools: Gemini. Human role: The original post does not say how a person stayed involved.; limitations: Access requires integration with the Gemini API, Google AI Studio, or the private preview Enterprise Agent Platform. The model relies on low-level VLA models for actual motor execution, which are not included in this reasoning layer. Safety benchmarks are self-reported by Google; real-world physical safety depends on proper hardware constraints and human supervision. Source reliability is high for product claims but lacks external verification of performance metrics..
Tools
Published on Agent Work Atlas:
Overview
Jul 30, 2026
Workflow
Jul 30, 2026 x.com Facebook LinkedIn Mail Copy link Gemini Robotics ER 2 represents a step change in powering robots with video understanding, task orchestration, and multi-robot collaboration — making it possible for robots to be more helpful in the physical world. Steven Hansen Senior Staff Software Engineer Peng Xu Staff Software Engineer x.com Facebook LinkedIn Mail Copy link We are launching Gemini Robotics ER 2, a new model designed to act as a high-level brain for robots. It enables real-time spatial reasoning, multi-step task planning, and collaboration between different robots. You can access the model now via the Gemini API, Google AI Studio, or the Gemini Enterprise Agent Platform to start building your own physical AI agents. Google just launched Gemini Robotics ER 2, which acts like a smart brain for robots. It helps them think quickly, plan multi-step tasks and even work together with other robots. The model lets robots watch video feeds to track their own progress and fix mistakes in real time. It’s designed to make robots safer and more helpful in the real world. General summary Basic explainer For robots to assist humans in everyday environments, accurate spatial reasoning is not enough. Robots must also think fast, timing their decisions and reasoning with the real-time speed of the physical world. That’s why today we’re launching Gemini Robotics ER 2, our most capable “embodied reasoning” model for robotics. Think of Gemini Robotics ER 2 as a high-level brain for robots. It allows robots to chat with humans, understand the physical world, and plan multi-step tasks. It then hands off motor execution to any given lower level vision-language-action (VLA) model. Gemini Robotics ER 2 can also natively call tools like Google Search to find information, or any other user-defined function. The design of Gemini Robotics ER 2 allows the robot to “think” about what comes next while simultaneously performing its actions. Gemini Robotics ER 2 represents a significant upgrade over Gemini Robotics ER 1.6. By watching continuous video feeds, robots can now track their own progress, adapt if something goes wrong, and know exactly when to move on to the next step. We are also introducing multi-robot collaboration, enabling robots to work together in shared spaces and complete complex workflows a single robot could not do alone. Gemini Robotics ER 2 is now publicly available to developers via the Gemini API, Google AI Studio, and in private preview on Gemini Enterprise Agent Platform. To help you get started, we’re sharing examples of how to configure the model and prompt it to power more useful physical AI tasks. Advancing physical agentic capabilities Most tasks in the physical world are complex and require multiple steps to complete. Gemini Robotics ER 2 is a physical agent, orchestrating steps for the robot and enabling it to self-correct, and generalize to more novel situations. To build an agentic setup, developers can declare low-level control interfaces — like Vision-Language-Action (VLA) models or navigation APIs — as tools, and stream multimodal video, audio, or text directly into the model. Gemini Robotics ER 2 improves this tool orchestration workflow. We can evaluate its performance with robots in simulation, using real-world robot control, and even pair it with a human controlling the robot remotely. Gemini Robotics ER 2 consistently outperforms ER 1.6 for tool orchestration across three control modes: real VLA, sim VLA, and human tele-op. In robotics, high-level reasoning depends on execution speed. Gemini Robotics ER 2 integrates into the Gemini Live API, using a bidirectional streaming endpoint optimized for latency-sensitive tasks. The result is fluid orchestration: Gemini Robotics ER 2 commands action models and robotics APIs to complete multi-step tasks without the jarring “stop-and-think” pauses.To illustrate this, we’ve built a demo with Spot from our partners at Boston Dynamics. We use Gemini Robotics ER 2 to orchestrate Spot APIs, such as navigation and manipulator movement, creating an interactive robot that fetches objects for you. Gemini Robotics ER 2 powered Boston Dynamic Spot fetches a popcorn snack up on a natural language command. The code is available on Github with other examples. Unlocking temporal intelligence for robust task completion
Reported result
In Google's internal evaluations, Gemini Robotics ER 2 achieved 57.4% accuracy on progress classification tasks and 91.3% accuracy on moment-finding tasks with a 0.96s mean absolute distance. These metrics indicate improved temporal intelligence for verifying task completion compared to previous generations. No independent third-party validation or real-world deployment failure rates are provided in the source text.
Human role
The original post does not say how a person stayed involved.
Limitations
Access requires integration with the Gemini API, Google AI Studio, or the private preview Enterprise Agent Platform. The model relies on low-level VLA models for actual motor execution, which are not included in this reasoning layer. Safety benchmarks are self-reported by Google; real-world physical safety depends on proper hardware constraints and human supervision. Source reliability is high for product claims but lacks external verification of performance metrics.
Try it
I want to test the Gemini Robotics ER 2 orchestration workflow for a simulated Spot robot task. First, configure a connection to the Gemini API using my [API_KEY]. Define a simple multi-step task: 'Navigate to the kitchen counter, identify the popcorn bag, and move the arm to grasp it.' Stream a [SAMPLE_VIDEO_URL] of a simulated environment to the model. Ask the model to output a JSON plan with steps, tool calls for navigation and manipulation, and progress confidence scores. Do not execute any real hardware commands. Instead, print the generated plan and the model's reasoning for each step. If the plan looks logical, ask me for confirmation before proceeding to a dry run in a simulator. If inputs are missing, ask me for the specific video file or simulation environment details.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.