Continual Learning: Definition and Examples

Continual Learning, also called continuous learning or incremental learning, is a machine learning paradigm in which a model is trained on a stream of data or tasks that evolve over time. Unlike classical learning where all data is available simultaneously, the model must progressively integrate new information while preserving its acquired knowledge.

The central challenge of Continual Learning is the phenomenon of catastrophic forgetting: when a neural network learns a new task, it tends to overwrite the weights associated with previous tasks, thus losing its prior skills. Several families of methods exist to counter this problem: regularization approaches (EWC, SI), memory replay approaches, and architectural approaches that allocate dedicated subnetworks for each task.

In the context of large language models (LLMs), Continual Learning takes on a particular dimension. Fine-tuning a model on new data can degrade its general performance, a phenomenon called alignment tax. Techniques like LoRA, prompt tuning, or retrieval-augmented generation (RAG) allow adding knowledge without deeply modifying the model's weights.

In prompt engineering, understanding Continual Learning helps grasp why a model has a knowledge cutoff date, why it may hallucinate on recent events, and how to structure interactions to compensate for these limitations by providing updated context directly in the prompt.