Prompt Compression: Definition and Examples

Prompt compression refers to the set of methods for shortening a prompt sent to a language model without significantly altering the quality of the responses obtained. Faced with the context limitations of LLMs and the increasing cost related to the number of tokens processed, this technique has become a major challenge for prompt engineering practitioners.

There are several compression approaches. Manual compression involves rewording instructions more concisely: removing redundancies, using abbreviations understandable by the model, or restructuring information as lists or tables rather than long paragraphs. Algorithmic compression, on the other hand, uses specialized tools (such as LLMLingua or AutoCompressors) that automatically identify and eliminate the least informative tokens in a prompt.

The benefit of prompt compression goes beyond simple cost reduction. A shorter prompt can paradoxically improve response quality by reducing informational noise and allowing the model to focus on essential elements. This is especially true for RAG (Retrieval-Augmented Generation) tasks where long retrieved documents can dilute the main instructions.

However, compression carries risks: too aggressive compression may remove important nuances, key examples, or subtle constraints that guided the model. The art of prompt compression therefore lies in balancing conciseness and preservation of meaning, a trade-off highly dependent on the model used and the complexity of the task.