Skip to content

AI, decoded

Can a language model actually control a robot?

Yes, and Google DeepMind's Paige Bailey describes it working with hobby hardware today: a Stanford-designed, 3D-printable robot dog with a Raspberry Pi, taking spoken instructions through the Gemini APIs. The interview doesn't say which parts run on the Pi and which call remote APIs, and robot software still turns the model's choices into movement. Bailey told Chain of Thought the robot dog costs about $2,000 if you order the parts and 3D print it yourself, or about $2,500 preassembled.

· Updated · Chain of Thought

Level 5: Production agents · 5.5 The frontier

Multimodal AIModel ArchitectureAI Hardware

  • 1 episode
  • 2 guest clips
  • 5 glossary terms
A small robot dog with a speech bubble reading shake my hand, under the words Hey, puppy, beside the line: the model drives.

In detail

1. It already runs, on cheap hardware

Paige Bailey, AI Developer Relations Lead at Google DeepMind, states the current position plainly: “Gemini already runs on robots.” (36:30)

The hardware in the demo she describes is modest. Stanford’s open-source “Pupper” robot dog can be built from 3D-printed components and runs on a Raspberry Pi. She credits the Gemini APIs: “you can say like, hey, pupper, shake my hand. Hey, Pupper, follow me… Like, all of these things are are possible today just with the Gemini APIs by themselves running and invoking all of the different robotic components on device on a Raspberry Pi.” (37:32)

This is what physical AI means concretely: a general-purpose model taking part in perception and action choices, alongside robot-specific software that executes them.

2. The model decides which components to invoke

Her sentence puts the Gemini APIs in charge of “invoking all of the different robotic components”, but the interview doesn’t specify which parts run locally and which use remote API calls, and her wording doesn’t establish that the model’s weights run on the Pi. The Pupper project’s AI documentation (labeled work in progress) describes one layered setup: model calls handle the conversation, robot software executes tool calls through ROS2, and tricks use prerecorded motions. It is not a record of Bailey’s exact demo.

The general pattern separates responsibilities. Dellaert and Hutchinson’s robotics text describes task plans executed by low-level controllers, and Google DeepMind’s Gemini Robotics introduction describes connecting its reasoning model to roboticists’ existing low-level controllers.

ResponsibilityWhat it contributesWhat still needs checking
Perception and stateInterprets observations of objects, obstacles and the robotSensor quality and whether observations are current
Task reasoningInterprets the request and chooses a goal or actionThe intended object, permitted task and available actions
Motion executionTurns the chosen action into movementReachability, actuator limits and the particular machine
FeedbackObserves whether the intended result happenedA completed physical task, not only a successful API response

Fictional example: a robot is asked to put a blue block in a tray. If the block misses the tray, the task is incomplete even if the model reports success. When testing a prototype, check what happens when observations are stale, the model is unavailable or an action can’t be executed.

That is why the hardware being commodity matters rather than being a footnote. Bailey expects the pattern to spread on that basis: “We’ll start to see many more kinds of robots built with commodity hardware,” (39:04) citing Raspberry Pi and cheaper alternatives that ship faster. She frames this as a forecast, not a fact.

3. Who is testing it

DeepMind runs a robotics trusted-tester program for the Gemini models and for open robotics models it publishes on Hugging Face. Bailey names the participants as “Enchanted Tools, Boston Dynamics, the Figure Team,” (38:38) describing them as “testing out or using Gemini models in order to operate robotics.”

At the time of the episode, that was a testing program rather than shipped products. It has moved since. Google now offers its Gemini Robotics reasoning model, Gemini Robotics-ER 2, to any developer as a preview in the Gemini API, and Boston Dynamics runs the earlier ER 1.6 in a live inspection feature for Spot. The models that directly drive robot motion are still limited to early-access partners.

4. The application she is most interested in is not a robot dog

Bailey’s own enthusiasm points at laboratories rather than consumer hardware. She cites Periodic Labs as “using AI to to control robotics, but also to to have those robots do really interesting material science work. So they can design experiments, run the experiments, test out, you know, the likelihood of different component parts being successful for the creation of semiconductors.” (39:33)

Her argument for it is about tedium and error rates rather than speed: the work is boring and repetitive, and that is where humans make mistakes. Her conclusion is hedged: “I think the more that AI is incorporated into the scientific process, both for designing experiments, but also executing on them, I think the better off we’re all gonna be.” (41:52)

Why it matters

Robotics used to be gated on control software written for one body and one task. When the controller is a multimodal model that takes instructions in language, more of the behavior can come from a general model, which is part of why cheaper hardware becomes workable. A 3D-printed dog taking spoken commands (Bailey puts the self-build cost at roughly $2,000) is still a demo, but a simpler version of the pattern is now for sale. Hugging Face’s Reachy Mini, a $399 to $499 desktop robot, runs a conversation app in which a language model decides when to move the robot’s head and which gestures to play, and that pipeline can run entirely on a local computer.

Disclosure: Episode 47 was sponsored by Galileo, the Season 3 presenting sponsor (read at 49:58).

Physical AI: a general model you can talk to picks the robot's action, and robot software turns it into movement Before, as an illustration: a robot controlled by software written for one body and one task, with a separate program per behaviour. Now: a spoken instruction such as "Hey, Pupper, shake my hand" goes to the Gemini APIs, which choose the action, deciding which robotic components to invoke. Robot-specific software then turns that choice into movement for this one body, and the Stanford-designed, open-source Pupper robot dog, which runs on a Raspberry Pi, acts. Robotic components run on the Pi; the interview does not establish where Gemini inference runs. The dog can be 3D-printed and self-built for roughly $2,000, Paige Bailey's figure. Underneath: more of the behaviour can come from a general model, which is part of why cheaper hardware becomes workable. The model picks the move. Robot code makes it. Google DeepMind's Paige Bailey: "Gemini already runs on robots," including a 3D-printable robot dog. BEFORE Code per body, per task program for task A program for task B program for task C bot A new behaviour means new control software, written for that one robot. NOW, ON HOBBY HARDWARE "Hey, Pupper, shake my hand." plain spoken language 1 · CHOOSES THE ACTION Gemini APIs decide which robotic components to invoke 2 · MAKES THE MOVE Robot software turns the choice into movement for this body Robotic components: on the Pi. Gemini inference location: not established in the interview. Raspberry Pi Stanford's open-source Pupper: 3D-printable,about $2,000 to build yourself (Bailey's figure) More behaviour can come from a general model, so cheaper robots become workable.
The model you talk to picks the action; robot software makes the move. The Pupper demo and the roughly $2,000 self-build figure are Google DeepMind’s Paige Bailey’s, from episode 47; the before panel is this page’s illustration. Download the image

Hear it from the guest

“So you can say like, hey, pupper, shake my hand. Hey, Pupper, follow me. Hey, Pupper, do the spider or, like, go swimming. Or, hey, Pupper, tell me a joke. Like, all of these things are possible today just with the Gemini APIs by themselves running and invoking all of the different robotic components on device on a Raspberry Pi.”
Watch on YouTube
“Periodic Labs, which are using AI to control robotics, but also to have those robots do really interesting material science work. So they can design experiments, run the experiments, test out … the likelihood of different component parts being successful for the creation of semiconductors.”
Watch on YouTube

Quotes lightly edited to remove filler words.

From the conversation

This explainer is drawn from these episodes — each carries its full transcript.

Concepts in this explainer

RoboticsHugging FacePhysical AIReasoning ModelsMultimodal AI