Discriminative Model: Definition and Examples

A discriminative model (or discriminant model) is an approach in machine learning that focuses on learning the relationship between input data and output labels. Unlike generative models that seek to understand how data is produced, a discriminative model directly learns the conditional probability P(y|x), i.e., the probability that an output y corresponds to a given input x.

Concretely, these models draw decision boundaries in the data space to separate different categories. For example, a discriminative model trained to distinguish spam emails from legitimate ones will learn the characteristics that differentiate these two classes, without trying to model how an email is composed in general. This targeted approach often makes them more performant for pure classification tasks.

Among the most well-known examples of discriminative models are logistic regression, support vector machines (SVMs), classic neural networks used for classification, random forests, and convolutional neural networks (CNNs) for image recognition. These models are ubiquitous in industrial AI applications.

In the context of prompt engineering, understanding the distinction between discriminative and generative models helps to better grasp the capabilities and limitations of large language models (LLMs). LLMs like Claude are fundamentally generative models, but they can perform discriminative tasks (classification, sentiment analysis) when guided properly through well-crafted prompts.