Automatic Prompt Engineer: Definition and Examples

Automatic Prompt Engineer (APE) is a technique introduced by researchers at the University of Toronto and Google DeepMind in 2022, which consists of automating the search for the optimal prompt for a given task. Instead of a human manually drafting and adjusting their instructions, the system automatically generates a set of candidates, tests them on examples, and then selects the most effective formulation.

The process works in three main steps. First, an LLM generates several instruction variants from a task description or input-output examples. Next, each candidate is evaluated on a validation dataset by measuring the quality of the responses produced. Finally, the best prompts are selected, possibly refined through successive iterations, until converging to an optimal formulation.

This approach is inspired by optimization metaheuristics: the prompt becomes a variable to optimize rather than a fixed parameter. Results show that automatically generated prompts often match or surpass those manually crafted by experts, including popular techniques like chain-of-thought prompting.

APE paves the way for scalable prompt engineering, where instruction optimization can be systematically applied to hundreds of tasks without mobilizing human expertise for each one. It represents a paradigm shift transforming prompt engineering from a craft into a reproducible engineering process.