Gemini Robotics ER 2: Whole-Body AI Robots Arrive 2026

Google DeepMind shipped Gemini Robotics ER 2 alongside Gemini Robotics 2, and the headline is not “better grasping.” The model now reasons about a whole body — legs, torso, arms, balance, and the space around all of them — inside a single embodied reasoning model rather than bolting a language model onto a hand-tuned controller. For two years the practical ceiling on robot foundation models was the tabletop: pick up the block, fold the towel, put the mug in the sink. Whole-body control breaks that ceiling, and because ER 2 ships through a developer-accessible API, the question shifts from “when will this be real” to “what do I build with it this quarter.”

What’s actually new in Gemini Robotics ER 2

The release is two models, and confusing them will cost you time. Gemini Robotics 2 is the vision-language-action (VLA) model — it takes pixels and instructions and emits robot actions directly. Gemini Robotics ER 2 is the embodied reasoning model: it does not move a motor. It looks at a scene and returns structured spatial understanding — 2D and 3D object points, bounding boxes, trajectories, grasp poses, affordances, and multi-step plans grounded in what it can see. ER 2 is the brain you query; Gemini Robotics 2 is the spinal cord you delegate to.

The whole-body piece is the genuine step change. Previous generations assumed a fixed base: the camera stayed put, the workspace was a plane, and “reachability” was a solved constant. ER 2 reasons about a robot that relocates itself — that a shelf is unreachable from here but reachable after two steps left, that bending to grab a low object shifts the center of mass, that a doorway is a navigable opening rather than a wall-shaped obstacle. That different spatial model is what makes humanoid robot AI model deployments plausible instead of demo-ware.

Reasoning traces and tool use

Two other details matter for builders. ER 2 does explicit multi-step reasoning before answering, and you can read the trace — it will tell you it identified three candidate mugs, rejected two as out of reach, and chose the third. It also orchestrates: ER 2 calls tools, including Google Search, mid-task, so “find the recycling bin and check whether this county recycles this plastic” becomes one prompt rather than a pipeline you glue together. As a robotics foundation model 2026 offering, that tool-use loop is the part competitors haven’t matched.

Why it matters

• The perception layer is now rentable. Spatial grounding — “where exactly is the handle, in coordinates I can send to a controller” — used to mean a labeled dataset and a custom detector per object class. It is now an API call with open-vocabulary queries.
• You don’t need a humanoid to benefit. ER 2 outputs points and trajectories in image space. A $200 arm, a warehouse AMR, or a fixed inspection camera can consume the same responses. Start where your hardware already is.
• Whole body control robots change the economics of mobile manipulation. When navigation and manipulation share one spatial model, you stop maintaining two systems that disagree about where the table is — the biggest source of integration bugs in mobile manipulation stacks.
• Safety moved into the reasoning loop. ER 2 evaluates whether an instruction is physically sensible and safe before planning — a meaningfully different posture than a policy that executes whatever the prompt says.
• Sim-to-real gets shorter. Because ER 2 reasons in general visual space rather than a trained-on distribution of your lab, teams report far less task-specific fine-tuning to reach a working baseline.
• It reframes the moat. If the Google DeepMind robotics API commoditizes embodied perception, the defensible work moves to hardware reliability, data collection on your specific task, and the last 5% of failure handling — not to computer vision headcount.

How to use the Google DeepMind robotics API today

ER 2 is available through the Gemini API, which means the SDK you already know. Below is a working path from zero to spatial coordinates.

1. Install the SDK and set your key. Get a key from Google AI Studio; the free tier covers evaluation.

   pip install google-genai pillow
   export GEMINI_API_KEY="your_key_here"

2. Ask for points, not prose. The core ER 2 pattern is a query that demands structured spatial output. Be explicit about the JSON shape and the coordinate convention — normalized 0–1000 in [y, x] order is what the model is tuned for.

   from google import genai
   from google.genai import types
   from PIL import Image
   
   client = genai.Client()
   img = Image.open("workspace.jpg")
   
   PROMPT = """Point to each graspable object on the table.
   Return a JSON array. Each entry:
     {"point": [y, x], "label": "<short name>", "reachable": true|false}
   Coordinates normalized 0-1000, [y, x] order.
   Return only JSON."""
   
   resp = client.models.generate_content(
       model="gemini-robotics-er-2",
       contents=[img, PROMPT],
       config=types.GenerateContentConfig(
           temperature=0.2,
           response_mime_type="application/json",
       ),
   )
   print(resp.text)

3. Convert to pixels, then to robot frame. Normalized output is resolution-independent, which is the point — but your controller wants metric coordinates. Two lines of math, then your existing camera extrinsics.

   import json
   
   W, H = img.size
   for obj in json.loads(resp.text):
       y, x = obj["point"]
       px, py = int(x / 1000 * W), int(y / 1000 * H)
       print(f'{obj["label"]:20s} pixel=({px},{py}) reachable={obj["reachable"]}')
       # depth_at(px, py) -> camera frame -> T_base_cam @ p_cam -> robot frame

4. Turn on the reasoning trace for whole-body questions. This is where ER 2 earns its keep. Ask a question that requires body awareness and let it think — then read the trace when it’s wrong, which is how you debug an embodied reasoning model.

   WB_PROMPT = """You control a mobile humanoid, 1.6m tall, reach 0.8m from torso.
   Task: retrieve the blue folder on the top shelf.
   
   Reason step by step, then return JSON:
   {
     "reachable_from_current_pose": bool,
     "base_motion": {"forward_m": float, "lateral_m": float, "yaw_deg": float},
     "posture": "stand" | "reach_up" | "crouch",
     "stability_risk": "low" | "medium" | "high",
     "grasp_point": [y, x],
     "blockers": ["..."]
   }"""
   
   resp = client.models.generate_content(
       model="gemini-robotics-er-2",
       contents=[Image.open("shelf.jpg"), WB_PROMPT],
       config=types.GenerateContentConfig(
           thinking_config=types.ThinkingConfig(include_thoughts=True),
           response_mime_type="application/json",
       ),
   )
   for part in resp.candidates[0].content.parts:
       print("THOUGHT:" if part.thought else "ANSWER:", part.text)

5. Wire it into a real loop with a hard safety gate. Never let model output reach actuators unfiltered. Validate ranges, clamp motion, and require a plausibility check. This is non-negotiable on any machine with mass.

   MAX_STEP_M   = 0.25
   MAX_YAW_DEG  = 30.0
   
   def gate(plan):
       if plan["stability_risk"] == "high":
           return None, "refused: stability risk"
       if plan["blockers"]:
           return None, f'refused: {plan["blockers"]}'
       m = plan["base_motion"]
       m["forward_m"] = max(-MAX_STEP_M, min(MAX_STEP_M, m["forward_m"]))
       m["lateral_m"] = max(-MAX_STEP_M, min(MAX_STEP_M, m["lateral_m"]))
       m["yaw_deg"]   = max(-MAX_YAW_DEG, min(MAX_YAW_DEG, m["yaw_deg"]))
       return plan, "ok"
   
   # perceive -> plan -> gate -> execute one increment -> re-perceive
   while not done:
       plan, status = gate(query_er2(camera.frame()))
       if plan is None:
           log(status); break
       robot.step(plan["base_motion"], posture=plan["posture"])

6. Cache your scene context. High-resolution frames are expensive per call. If you query the same static scene repeatedly, use context caching and send only the delta.

   cache = client.caches.create(
       model="gemini-robotics-er-2",
       config=types.CreateCachedContentConfig(
           contents=[reference_frame],
           system_instruction="You are the spatial reasoner for a mobile humanoid.",
           ttl="600s",
       ),
   )
   # subsequent calls pass cached_content=cache.name

One practical note: run perception at a slower cadence than control. Query ER 2 at 1–3 Hz for planning and let a local controller run the 100 Hz loop. Treating a cloud model as a real-time servo is the most common architectural mistake here.

How it compares

Model / stack	Scope	Whole-body	Developer access	Best fit
Gemini Robotics ER 2	Embodied reasoning: points, trajectories, grasps, plans, tool use	Yes — reasons about base motion, posture, stability	Public Gemini API, standard SDKs	Teams that want spatial intelligence without owning a perception team
Gemini Robotics 2 (VLA)	Direct vision-language-action control	Yes, on supported platforms	Partner / trusted-tester access	Groups with a supported robot and a hardware partnership
NVIDIA GR00T N-series	Open humanoid foundation model plus Isaac sim pipeline	Yes, humanoid-first	Open weights, self-hosted	Teams that need on-prem inference and heavy simulation
Physical Intelligence π-series	Generalist VLA policies across many robot types	Partial — manipulation-centric	Some open weights, research-oriented	Researchers cross-training on diverse embodiments
Figure Helix	Vertically integrated humanoid policy	Yes	Closed — ships with the robot	Buyers of a finished humanoid product
OpenVLA and open baselines	Open-source VLA, mostly tabletop	No	Fully open weights	Cheap experimentation and fine-tuning studies

The honest read: ER 2 wins on reasoning quality and time-to-first-result, GR00T wins on control and cost at scale, and the vertically integrated players win on reliability for one specific machine. These are not mutually exclusive. ER 2 as the planner over a self-hosted low-level policy is a legitimate architecture, and probably the most common one by the end of 2026.

What’s next

Watch the latency curve above everything else. Every embodied reasoning model today lives behind a network hop, and that hop is the difference between a robot that plans and a robot that reacts. The obvious trajectory is distillation — a smaller ER variant running on-device for the fast loop while the full model handles deliberation. When an on-device ER lands, the set of viable applications roughly doubles, because you stop designing around a connectivity assumption.

Track hardware breadth second. Gemini Robotics 2 runs on a short list of platforms, and access is gated. ER 2 is embodiment-agnostic by design — it returns coordinates, not joint commands — so the interesting question is how quickly a middleware layer maps ER 2 output onto arbitrary robots. Expect ROS 2 packages and community bridges to appear faster than official support does; that gap is where independent builders can make themselves useful.

Third, evaluation is about to get contentious. Whole-body benchmarks are immature, and “reasoning” claims are hard to falsify when demos are curated. Build your own held-out task set — twenty scenarios your robot must handle, scored the same way every release — and re-run it on every model update. That discipline separates teams shipping robots in 2027 from teams still posting demo videos, and it’s cheap to start today.

Frequently Asked Questions

What is the difference between Gemini Robotics ER 2 and Gemini Robotics 2?

ER 2 is the embodied reasoning model — it perceives, reasons, and plans, returning structured spatial data like points, boxes, trajectories, and grasp poses. Gemini Robotics 2 is the vision-language-action model that outputs robot actions directly. ER 2 is broadly available through the Gemini API; the VLA has narrower, partner-gated access. Most developers should start with ER 2.

Do I need a humanoid robot to use Gemini Robotics ER 2?

No. ER 2 returns coordinates and plans in image space, so any system with a camera can consume its output — a single arm, a mobile base, or a fixed inspection rig with no actuators at all. The whole-body reasoning is an added capability, not a requirement.

Is there a free tier for the Google DeepMind robotics API?

ER 2 is served through the standard Gemini API, so it inherits Gemini’s access model, including a free tier suitable for evaluation with rate limits. Pricing and quotas change, so check current Gemini API documentation before you size a production deployment — do not budget from a blog post.

Can Gemini Robotics ER 2 run in real time on a robot?

Not as a control loop. Treat it as a planner running at roughly 1–3 Hz and pair it with a local controller handling the high-frequency loop. Architect for cloud latency and intermittent connectivity from the start — a plan-then-execute-increment pattern with local safety gating is what actually works.

How does whole-body control differ from arm-only manipulation?

Arm-only models assume a fixed base and a reachable workspace. Whole body control robots must reason about locomotion, balance, center of mass, and the fact that reachability changes as the robot moves. The model can answer “walk two steps left, crouch, then grasp” instead of failing because the target is out of reach.

What’s the biggest practical risk in deploying a robotics foundation model 2026 stack?

Over-trusting model output. A confident-sounding plan can still be physically wrong, and unlike a chatbot, a wrong answer here moves mass. Validate outputs against hard kinematic and velocity limits, require a plausibility check before actuation, and keep a hardware emergency stop that no software path can override.