A home robot asked to open a drawer needs more than the label “drawer.” It must estimate the handle’s position, choose a reachable grasp and account for an object blocking the path. Geometry, the robot’s own pose and possible motion all affect the action.
These capabilities overlap with computer vision but add a focus on spatial relations and action. A caption saying “a cup is on a table” does not supply the metric distance or clearance needed to grasp the cup. A robot may combine camera data, depth measurements and a map rather than rely on a text description alone.
The label names a group of capabilities, not one standard model architecture. Evaluate the spatial task the application needs: object-relative reasoning, measured distance or collision-free movement require different evidence. A plausible description of a scene is not proof of a physically executable plan.
Sources
- Dellaert and Hutchinson: Models for robotics — Connects representations of world and robot state, sensor models, action models and task planning.
Go deeper
- Dellaert and Hutchinson: A Mobile Robot With Simple Kinematics course
Study perception, state estimation and planning for a robot that moves through its environment.