Worked example: a team spends $100 on model calls during a week and opens 20 pull requests. Eight are merged by the end of that week. Cost per opened PR is $5; cost per merged PR is $12.50 for that accounting window. These fictional figures use the same spending and different denominators. Neither includes human review time.
In episode 76, at 4:33, Sonar CTO Andrea Malagodi describes one engineer with over 500 PRs and low cost per PR, and another with high spending and few PRs. Examining their work revealed different problems: an agent workflow optimized for throughput versus difficult analysis requiring long AI turns. Malagodi cautions against quick judgments from the ratio.
Keep the accounting window visible
The worked example mixes two populations: the $100 pays for all 20 PRs, but only the eight merged ones appear in the merged count. Pick the boundary before comparing ratios. A weekly window divides that week’s spending by that week’s merges, even if some PRs were opened earlier. A cohort follows one set of PRs and their costs for a stated time. Label which one you used and when you stopped counting. Two edge cases: if nothing merges, cost per merged PR is undefined, not zero; and splitting one change into several PRs lowers the ratio without adding any value.
Complements help. DORA’s delivery metrics look at throughput and instability for a whole service, not individuals. Productivity studies are hard to read too: METR’s February 2026 update reports that developers unwilling to work without AI dropped out of its study, and that time measurements were unreliable for people running several agents at once, so it treats its own new speedup estimate as weak evidence.
Our recommendation is to compare similar work and record the accounting boundary. PR count does not measure correctness, useful outcomes, or business value. Cost per task asks the same denominator question about tasks rather than pull requests, and measuring whether AI is paying off considers those broader outcomes.
Hear it from the guest
“Without really understanding the context of the problems that are being worked on.”
Quotes lightly edited to remove filler words.
Sources
- Episode 76: Andrea Malagodi on cost per PR — Sonar's CTO describes the metric and cautions against judging engineers without understanding their work; the arithmetic below is illustrative.
- DORA: Software delivery performance metrics — Measures a team's delivery throughput and instability for an application or service; a complement to PR ratios, not a way to rank individuals. Checked November 6, 2026.
- METR: Uplift study update (February 24, 2026) — Reports selection effects in its developer-productivity study and unreliable time measurement for developers running several AI agents at once. Checked November 6, 2026.