A grocery chain points a camera at a shelf and wants to know which products are there and which gaps need restocking. A model that answers “this photo contains cereal” isn’t enough. The chain needs every box of cereal located, counted and named. Object detection does that: for each object it finds, it returns a label, a confidence score and a bounding box, the rectangle around the object. The same task lets a car spot pedestrians and other vehicles, a factory camera find defects, and a photo app put boxes around faces.
Detection sits between two simpler and harder computer vision tasks. Image classification gives one label for the whole picture. Image segmentation goes further than detection and marks the exact pixels belonging to each object. Classic detectors are built on convolutional neural networks: two-stage models such as Faster R-CNN propose candidate regions and then classify them, while single-stage models such as YOLO predict boxes and labels in one pass, fast enough for live video. Results are usually reported on benchmarks such as COCO, using measures that combine whether the box lands in the right place with precision and recall.
In episode 12 at 19:38, Google’s Logan Kilpatrick says general multimodal models like Gemini can now do image segmentation, bounding boxes and object detection, “all of the domain specific computer vision problems” that used to need dedicated models. His point is that a team can try a vision feature with a general model before building its own. Episode 12 aired in season two, which Galileo sponsored (sponsor mention).
Why it matters (editorial synthesis): many practical vision products, from inventory counts to safety monitoring, depend on knowing where things are, not just what’s in the picture. Errors have different costs: a missed pedestrian is far worse than a false alarm. Test a detector on footage from your own cameras, lighting and angles, because accuracy on a public benchmark doesn’t guarantee accuracy in your store or on your road.
Hear it from the guest
“The models can do image segmentation now, and they can do bounding boxes and object detection”
Quotes lightly edited to remove filler words.
Sources
- Ren et al.: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks — Two-stage detector that first proposes regions likely to contain objects, then classifies each region and refines its box; checked October 24, 2026.
- Redmon et al.: You Only Look Once: Unified, Real-Time Object Detection — The YOLO paper frames detection as a single regression from image pixels to bounding boxes and class probabilities, fast enough for real-time video; checked October 24, 2026.
- Lin et al.: Microsoft COCO: Common Objects in Context — Introduces the COCO dataset of everyday scenes with per-instance object labels, which became a standard benchmark for detection and segmentation; checked October 24, 2026.
