Suppose you ask a model a word problem five times, with some randomness switched on. It works through the problem differently each time and answers 42, 42, 40, 42 and 38. Self-consistency picks 42, the majority. The idea is that a hard problem usually has several valid ways to reach the right answer, while mistakes tend to scatter across different wrong answers (Wang et al.).
Wang and colleagues introduced it as a replacement for taking the single most likely chain of thought. On arithmetic and commonsense benchmarks it improved accuracy, including 17.9 points on the GSM8K math set in their experiments (Wang et al.). The variety comes from sampling: temperature or top-p sampling must allow different outputs, or every run will say the same thing.
It works best when answers are short and easy to compare, such as a number, a multiple-choice letter or a category label. For an essay or a block of code there is no simple majority, so teams use a judge or tests to pick among candidates instead; pass@k measures a related idea for code. The cost scales with the number of samples, which makes it one of the plainest forms of test-time compute. Agreement is also not proof: if a model is consistently wrong, the vote is confidently wrong, so check self-consistency against labeled examples before relying on it.
Sources
- Wang et al.: Self-Consistency Improves Chain of Thought Reasoning in Language Models — Introduces sampling diverse reasoning paths and selecting the most consistent answer; reports gains such as +17.9 points on GSM8K over a single chain of thought in the paper's setup; checked October 24, 2026.