F1 Score: Definition and Examples

The F1 Score is a fundamental metric in machine learning and natural language processing. It represents the harmonic mean of precision (proportion of correct positive predictions) and recall (proportion of detected true positives). Its formula is: F1 = 2 × (Precision × Recall) / (Precision + Recall). The score ranges from 0 to 1, where 1 indicates perfect performance.

The main interest of the F1 Score lies in its ability to balance two often contradictory objectives. A model can have excellent precision by being very selective (few false positives), but at the expense of recall (many false negatives). Conversely, a model that predicts 'positive' for everything will have perfect recall but poor precision. The F1 Score penalizes these imbalances through the harmonic mean, which pulls the value toward the lower of the two scores.

In the context of prompt engineering, the F1 Score is commonly used to evaluate the quality of responses generated by an LLM, especially for tasks such as classification, entity extraction, or question answering. For example, when asking a model to extract information from a text, one can measure whether all relevant information was found (recall) and whether the extracted information is actually correct (precision).

There are several variants of the F1 Score for multi-class problems: macro F1 (unweighted average of F1 per class), micro F1 (global calculation over all predictions), and weighted F1 (average weighted by the number of examples per class). The choice of variant depends on the relative importance given to each class in the problem at hand.