Self Consistency: Definition and Examples

Self Consistency is an advanced prompting method introduced by Wang et al. in 2022. It relies on a simple but powerful principle: rather than relying on a single response generated by a language model, it asks the model to produce several different reasoning chains for the same problem, then retains the answer that appears most often.

This approach is based on the intuition that if multiple independent reasoning paths converge to the same conclusion, that conclusion is likely correct. It is comparable to asking multiple experts: if the majority arrives at the same answer via different routes, it can be trusted more.

Concretely, Self Consistency works in three steps. First, a Chain-of-Thought type prompt is used to encourage step-by-step reasoning. Second, several responses are generated by increasing the temperature parameter to introduce diversity in reasoning. Third, a majority vote is applied on the final answers to determine the most consistent one.

This technique is particularly effective for mathematical reasoning, logical reasoning, and common sense tasks, where it significantly outperforms standard Chain-of-Thought. It reduces random errors due to a poor reasoning path, without requiring additional model training.