Active Prompting: Definition and Examples

Active Prompting is an advanced prompt engineering technique inspired by active learning in machine learning. Its core principle is not to arbitrarily choose the few-shot examples included in a prompt, but to strategically select those that will bring the most benefit to the language model.

The process proceeds in several steps. First, the model generates multiple responses for a set of candidate questions. Then, the degree of uncertainty of the model on each question is measured (for example by observing the variance between the different responses). The questions where the model shows the most hesitation are then annotated by a human with detailed reasoning (chain-of-thought), and used as examples in the final prompt.

This approach solves a major problem of classic few-shot prompting: the choice of examples is often random or intuitive, which can lead to suboptimal results. Active Prompting rationalizes this choice by precisely targeting the model's weak areas.

Introduced by Diao et al. in 2023, this method has demonstrated significant improvements on arithmetic, logical, and commonsense reasoning tasks, surpassing fixed-example Chain-of-Thought prompting approaches.