Transfer Learning: Definition and Examples

Transfer learning is a fundamental approach in artificial intelligence based on a simple principle: knowledge acquired while solving one problem can be transferred to solve a different but related problem. Rather than training a model from scratch, you start with a model already trained on large amounts of data and adapt it to a specific use case.

This technique lies at the heart of the revolution in large language models (LLMs) like GPT or Claude. These models are first pre-trained on massive text corpora to acquire a general understanding of language, then fine-tuned on specific tasks such as translation, summarization, or conversation. It is precisely this mechanism that allows a single base model to become proficient in hundreds of different tasks.

In computer vision, transfer learning has also transformed the field. A model trained on ImageNet (millions of classified images) learns to recognize general shapes, textures, and patterns. These basic visual knowledge are then transferred to detect tumors in X-rays or identify defects on a production line, even with only a few hundred specific examples.

For prompt engineering practitioners, understanding transfer learning is essential because it explains why LLMs can adapt to very diverse contexts simply by formulating instructions in a prompt. Few-shot prompting, for example, directly exploits the model's transfer capacity by providing a few examples to guide its response toward a specific format or style.