Cross-Model Review

A Claude model reviewing a patch produced by Codex is one example from Conor’s essay. A second instance of the same model adds a reviewer without adding another family.

In episode 76, at 35:30, Conor describes using different model families at review steps to challenge an implementing model’s assumptions. He later describes this as an individual or small-team setting.

Conor reports that same-family reviewers can share assumptions and that different families give him different feedback. This is an observation from his workflow, not a measured improvement in defect detection.

Andrea Malagodi, Sonar’s CTO, likes the idea of challenging work with different families and asks how to distinguish useful review from merely getting a different opinion over time. Malagodi also raises the complexity of setting up and managing many models across an organization with 10,000 engineers.

Malagodi recommends functional verification while discussing Sonar’s tools. That product discussion is separate from evidence of a benefit from different-family review.

Episode 76 includes G2i’s episode sponsorship read at 33:36, plus Season 4 presenting-sponsor reads for Svix at 14:55, Walrus Memory at 16:01, and Inngest at 34:17.

Conor sets out the full loop in his October 2026 essay on reviewing AI code. The implementing agent’s change first passes cheap automated checks. Reviewers from different model families then examine it, and a lead model verifies each finding before the merge. He argues that a different family is more likely to challenge the implementer’s blind spots. He expects labs to study biases shared across model generations and says improvements could narrow that advantage over time. His loop retains human judgment for permissions, editorial taste, architecture, and unresolved tradeoffs.

Research gives a reason for caution. Kim, Garg, Peng and Garg studied more than 350 language models overall across multiple settings. On one leaderboard dataset, when two models both answered wrong, they agreed with each other 60% of the time. Larger, more accurate models had highly correlated errors even when their architectures and providers differed. That measures error correlation on benchmark questions, not the effect of review on code. A different family alone does not establish that a reviewer’s errors are independent.

Our recommendation is to verify each reported defect against the files, source material, or runtime behavior. Record confirmed defects, incorrect findings, missed defects, time, and cost when comparing reviewers. Use repeatable tests to check the findings; agreement among models alone does not establish correctness.

Multi-model review is a broader description: several models can come from the same family. Testing an AI system covers repeatable checks that can complement model review.