ROUGE Score: Definition and Examples

ROUGE Score is a set of automatic evaluation metrics originally designed to measure the quality of text summaries. Developed by Chin-Yew Lin in 2004, ROUGE compares an automatically generated text to one or more human-written reference texts by measuring the overlap between them. The higher the score (on a scale of 0 to 1), the closer the generated text is to the human reference.

The ROUGE family includes several variants. ROUGE-N measures the overlap of n-grams (sequences of N consecutive words) between the generated text and the reference. ROUGE-1 compares individual words (unigrams), ROUGE-2 compares consecutive word pairs (bigrams), and so on. ROUGE-L, on the other hand, uses the longest common subsequence (LCS) to capture structural similarity between texts, even if words are not strictly consecutive.

In prompt engineering, understanding the ROUGE Score is essential when working on text generation tasks, especially automatic summarization, translation, or paraphrasing. This metric allows an objective evaluation of whether a prompt produces results that are faithful to the expected content. For example, by comparing the outputs of different prompts with a reference text, one can identify which wording generates the most complete and relevant summaries.

It is important to note that ROUGE mainly measures lexical recall — that is, the presence of the right words — but does not necessarily capture semantic coherence, fluency, or factual accuracy. That is why ROUGE is often used in conjunction with other metrics such as BLEU, BERTScore, or human evaluation to obtain a more complete view of the quality of a generated text.