Unsupervised Learning: Definition and Examples

Unsupervised learning is a machine learning method in which an algorithm is trained on a dataset without labels or expected outputs. Unlike supervised learning, where each example is associated with a known output, unsupervised learning lets the model explore the data on its own to extract underlying structures.

The most common techniques include clustering (automatic grouping of similar data, such as K-means or DBSCAN), dimensionality reduction (such as PCA or t-SNE, which simplify complex data while preserving essential features), and anomaly detection. These methods are particularly useful when dealing with large amounts of raw data without human annotations.

In the context of large language models (LLMs), unsupervised learning plays a fundamental role. The pre-training phase of models like GPT or Claude largely relies on unsupervised principles: the model learns to predict the next word in vast text corpora, without being explicitly given the 'correct answers'. It is this ability to learn rich language representations autonomously that makes these models so versatile.

For prompt engineering practitioners, understanding unsupervised learning helps to better grasp how an LLM acquired its knowledge and why it can sometimes generalize surprisingly or, conversely, produce unexpected results. This understanding aids in formulating prompts that best leverage the patterns the model internalized during training.