Skip to content

AI Glossary · Metrics

What is Pareto Frontier (Cost-Performance Curve)?

Pareto Frontier (Cost-Performance Curve)

A Pareto frontier contains options for which no other option improves one objective without worsening another, such as cost and task performance.

Also known as: pareto curve, cost-performance curve, capability-to-cost curve

· Updated · Chain of Thought

  • 1 episode
  • 1 source

On a cost-performance chart, an option is dominated if another is at least as cheap and at least as capable, with a strict improvement on one measure. A tie on both does not establish dominance. McDermott’s multi-objective optimization lecture explains this relationship (UCD, Multi-objective Optimisation, slide 29). The frontier depends on which objectives and candidate options you include.

Consider hypothetical models A, B and C. A costs one unit per completed task and succeeds on 80% of the test set. B costs two units and succeeds on 90%. C costs two units and succeeds on 85%. B dominates C, while A and B represent different tradeoffs. These are illustrative values, not current model prices or benchmark results.

Two more cases change the reading. Add a model D that, like A, costs one unit and succeeds on 80%. A and D tie on both measures, so neither dominates the other and both sit on the frontier at the same point; the chart cannot choose between them, and an objective you left out, such as latency, might. Now add a hard requirement of at least 85% success: only B and C qualify, and B is the choice within that set. Require a cost of at most 1.5 units as well, and nothing qualifies. A frontier point never overrides a requirement; change the requirement or the candidates instead of calling an infeasible option the winner.

Introducing a capability-and-cost chart on screen, Paige Bailey says, “one of the things that I’m most excited about is this kind of Pareto curve” (episode 47, 3:59).

For builders, make your own chart from the task you need to complete. Cost per task is a useful cost axis, and model routing can send each request to a model on the frontier. Choose whether the cost measure includes retries, retrieval and failed runs. Add a latency constraint if the product needs one, and state the evaluation set and date so comparisons can be repeated. How to cut AI agent costs follows cost through a workflow; small language model vs frontier model covers selecting a model for a task.

Sources

From the conversation