Synthetic Data: Definition and Examples

Synthetic data refers to datasets artificially created using algorithms, statistical models, or generative artificial intelligence systems. Unlike real data collected from human observations or interactions, they are generated from scratch while preserving the statistical properties, distributions, and correlations of the original data they mimic.

The main interest in synthetic data lies in its ability to solve several major problems in machine learning and AI. They notably allow overcoming privacy constraints (GDPR, HIPAA), generating massive volumes of training data when real data is scarce or costly to collect, and creating specific scenarios that are difficult to observe in reality (edge cases, rare events).

Generation techniques include generative adversarial networks (GANs), variational autoencoders (VAEs), diffusion models, and rule-based simulations. In the context of prompt engineering, synthetic data is frequently used to create training examples for fine-tuning language models, generate diverse test sets, or produce structured data on demand.

Although very useful, synthetic data has important limitations. If the generator model is biased, the produced data will inherit these biases. Moreover, they may lack the richness and subtleties of real data, which can affect the performance of models trained exclusively on synthetic data. The most effective approach often involves combining real and synthetic data.