Soft Prompting: Definition and Examples

Soft prompting is a method for optimizing language models that fundamentally differs from classic prompting (so-called "hard prompting"). Instead of formulating instructions in natural language, continuous vectors — sequences of numbers — are injected directly into the model's embedding space. These vectors are trained via backpropagation on a task-specific dataset, while the main model weights remain frozen.

This approach was popularized by the paper "The Power of Scale for Parameter-Efficient Prompt Tuning" (Lester et al., 2021) from Google Research. The researchers demonstrated that with sufficiently large models, simply adding a few dozen learnable tokens as a prefix could rival full fine-tuning of the model, while only modifying a tiny fraction of the parameters.

The major advantage of soft prompting lies in its resource efficiency. Rather than duplicating and retraining an entire model for each task, only the prompt vectors are stored and trained — often a few kilobytes versus several gigabytes for a full model. This allows deploying a single base model with multiple interchangeable "modules" of soft prompts depending on the task.

It is important to note that soft prompting is a technique reserved for practitioners who have access to the model's internal layers. Users of APIs like ChatGPT or Claude cannot directly use soft prompting — they practice hard prompting. Soft prompting belongs more to applied machine learning than to prompt engineering in the common sense.