What Is Gemini Robotics?

Gemini Robotics is introduced in Gemini Robotics: Bringing AI into the Physical World (arXiv 2503.20020) by the Gemini Robotics Team at Google DeepMind, with an official project page and launch post. The report starts from a gap: large multimodal models show generalist skill in digital domains, yet translating that to physical agents remains a significant challenge. Its answer is to build on Gemini 2.0 in two steps, first strengthening embodied reasoning, then grounding it in robot actions. This page is a definition and a reading guide, not a how-to for running the models.

Gemini Robotics is third-party research. Firsthand is not affiliated with Google DeepMind or the report’s authors, and makes no claim that Gemini Robotics or any other named model was trained, fine-tuned or evaluated on Firsthand data. The site states custom egocentric collection to a written spec, fast and on demand, delivering first-person human video with 3D hand pose at 21 joints per hand and a per-joint visibility flag, calibrated stereo depth and 6-DoF IMUs at 200 Hz. The only language data tied to Firsthand capture is one task instruction per episode. Firsthand captures first-person human video, not robot or teleoperation data.

The report introduces two models. Gemini Robotics-ER is a vision-language model with strong embodied reasoning at its core. Gemini Robotics is a vision-language-action model that connects those reasoning priors to low-level control of real robots. The abstract says Gemini Robotics builds on top of Gemini Robotics-ER, and the paper describes the VLA’s backbone as a distilled version of it.

The report also adds an optional specialization stage, in which the VLA is fine-tuned for extreme dexterity, for harder generalization settings, or for new robot embodiments. The official project page now lists later models, Gemini Robotics 2, Gemini Robotics ER 2 and Gemini Robotics On-Device 2. This page covers the 2025 report only.

The paper defines embodied reasoning as the ability of a vision-language model to ground objects and spatial concepts in the real world and to synthesize those signals for downstream robotics applications. It lists open-world 2D object detection, pointing, trajectory prediction and top-down grasp prediction, plus 3D understanding through multi-view correspondence and 3D bounding boxes. Grasp prediction is a new feature introduced in Gemini Robotics-ER.

The authors also release ERQA, a benchmark of 400 multiple-choice, VQA-style questions on embodied reasoning. The paper says its images are either taken by the authors or sourced from existing datasets, among them HoloAssist, MECCANO and EGTEA Gaze+, which are egocentric. Several of its qualitative examples use egocentric images, such as a trajectory drawn from a human hand to a tool. That tells a reader about the evaluation, not about training data.

Per the paper, the model ingests a multimodal prompt of images of the current scene and a text instruction, and outputs action chunks that the robot executes. It has two components: a VLA backbone hosted in the cloud and a local action decoder running on the robot’s onboard computer.

The paper reports that the backbone’s query-to-response latency was optimized from seconds to under 160 ms, that end-to-end latency from raw observations to low-level action chunks is approximately 250 ms, and that with multiple actions per chunk the effective control frequency is 50 Hz. These are the authors’ figures for their own system.

The paper says the authors collected a large-scale teleoperated robot action dataset on a fleet of ALOHA 2 robots over 12 months, consisting of thousands of hours of real-world expert demonstrations and thousands of diverse tasks. Training also includes non-action data: web documents, code, multimodal content, and embodied reasoning and visual question answering data. The model card summarises the training data as images, text, and robot sensor and action data.

For specialization, the authors curate between 2000 and 5000 episodes of demonstrations per task. For fast adaptation, they report that 7 of 8 tasks reached above 70% success with at most 100 demonstrations, which the paper equates to 15 minutes to 1 hour of demonstrations depending on the task. The robot action data is teleoperated, so the recipe starts from robot demonstrations.

In simulated zero-shot control through code generation, the paper reports an average success of 27% for Gemini 2.0 Flash and 53% for Gemini Robotics-ER, over 50 trials with random initial conditions. For the VLA, it reports an average of 79% across six specialized long-horizon tasks, 100% on the lunch-box task, and an average of 63% for an adapted second bi-arm platform. It compares against two baselines it trains on the same data mixture: a re-implementation of an open-weights VLA and a multi-task diffusion policy.

Treat these as the authors’ own evaluations on their robots, tasks and trial counts. The report also states limitations: Gemini 2.0 may struggle with grounding spatial relationships across long videos, and its numerical predictions such as points and boxes may not be precise enough for fine-grained control. The launch post adds that on a generalization benchmark Gemini Robotics more than doubles performance compared to other state-of-the-art VLAs, which is likewise the authors’ claim.

The official project page invites visitors to join a waitlist if they are interested in testing the models. The March 2025 launch post said Gemini Robotics-ER was available to selected trusted testers and that Gemini Robotics was being developed with a hardware partner. The sources read give no public weights, repository or licence for the models, so those are Not stated in the sources read. Check the project page for current availability.

Gemini Robotics is a language-conditioned policy: its prompt includes a text instruction. The paper evaluates instruction generalization, covering paraphrasing, typos, other languages and varying levels of specificity, and it describes a reasoning-enhanced variant trained on a re-labeled version of its robot action dataset that outputs intermediate trajectory steps. In its in-context experiments, the authors interleave descriptions of the performed actions in language with the demonstrations.

For data buyers the point is that the language and reasoning in this recipe sit on top of robot action data and a very large non-robot mixture. They are not something a single per-episode instruction replaces.

Firsthand captures first-person human video, not robot or teleoperation data, so it does not deliver the robot actions, action chunks or proprioception this recipe consumes. The only language data tied to Firsthand capture is one task instruction per episode, and nothing further: no reasoning chains, narration, captions or sub-task language. Whether human egocentric data would help a policy like Gemini Robotics is Not stated in the sources read, and this page draws no conclusion about it.

What the site does state is custom egocentric collection, fast and on demand, to a written spec. Confirm the spec on a Spec Pilot, a small fast first batch. Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour, and delivery is in RLDS, LeRobot, WebDataset or HDF5.

What is Gemini Robotics?

A family of embodied AI models from Google DeepMind, built on Gemini 2.0. It comprises Gemini Robotics-ER, a vision-language model with embodied reasoning, and Gemini Robotics, a vision-language-action model that directly controls robots.

What is Gemini Robotics-ER?

The embodied reasoning model in the family. The paper lists object detection, pointing, trajectory and grasp prediction, and 3D understanding through multi-view correspondence and 3D bounding boxes, and shows it controlling a robot zero-shot through code or few-shot in context.

Is Gemini Robotics a VLA model, and how fast does it run?

Yes. The paper calls it a generalist vision-language-action model. It pairs a cloud-hosted backbone, with query-to-response latency under 160 ms, with a local action decoder. The paper reports approximately 250 ms end-to-end latency and an effective control frequency of 50 Hz.

Can I download or fine-tune Gemini Robotics?

The sources read do not state public weights, a repository or a licence, so those are Not stated in the sources read. The official project page offers a waitlist for testing the models, so check it for current availability.

Does Gemini Robotics train on human egocentric video?

The paper describes teleoperated robot action data on an ALOHA 2 fleet plus a non-robot mixture, and the model card lists images, text, and robot sensor and action data. Whether human egocentric video was part of training is Not stated in the sources read. The paper draws some ERQA benchmark images from egocentric datasets.

Did Gemini Robotics use Firsthand data, and can I request a custom dataset?

No. Firsthand is not affiliated with Google DeepMind and makes no claim that Gemini Robotics or any model was trained on Firsthand data. You can request a custom egocentric dataset: write the tasks, fields and format into a spec, confirm it on a Spec Pilot, and billing is per validated hour. The only language data is one task instruction per episode, and Firsthand captures first-person human video, not robot or teleoperation data.