Accuracy: Definition and Examples

Accuracy (or exactness) is an evaluation metric that quantifies the ability of an artificial intelligence model to produce correct results. It is calculated by dividing the number of correct predictions by the total number of predictions made. A model with 95% accuracy means it gives the correct answer 95 times out of 100.

In the context of large language models (LLMs) like GPT-4 or Claude, accuracy takes on a more nuanced dimension. Unlike a binary classifier where the answer is either right or wrong, an LLM generates free text whose correctness can be partial, contextual, or subjective. We then speak of factual accuracy (are the stated facts verifiable?), semantic accuracy (does the meaning of the response match the question?), or logical accuracy (is the reasoning coherent?).

In prompt engineering, accuracy is directly influenced by the quality of instructions given to the model. A vague or ambiguous prompt will produce less precise responses, while a structured prompt with clear constraints, examples, and a defined output format will significantly improve the accuracy of results. Techniques like Chain-of-Thought, few-shot prompting, or cross-checking allow measurable increases in accuracy.

It is important to note that accuracy alone is not always sufficient to evaluate a model. On imbalanced datasets, a model can display high accuracy while consistently failing on minority cases. That is why it is often complemented by other metrics like precision, recall, or F1-score to obtain a more complete view of performance.