Embedding: Definition and Examples

An embedding (or vector embedding) is a fundamental technique in artificial intelligence that consists of transforming complex data — such as words, sentences, images, or documents — into fixed-size numerical vectors. These vectors capture the meaning and semantic relationships between items: two texts close in meaning will have vectors close in the vector space.

Concretely, an embedding model analyzes a text and produces a list of numbers (e.g., 1536 dimensions for OpenAI's text-embedding-ada-002). These numbers are not individually interpretable by humans, but their arrangement encodes the meaning of the text. One can then calculate the distance or cosine similarity between two vectors to determine how semantically close two texts are.

Embeddings are at the heart of many modern applications: semantic search (finding relevant documents even without exact keyword matches), recommendation systems, text classification, and especially RAG (Retrieval-Augmented Generation), which enriches an LLM's responses with external knowledge.

In prompt engineering, understanding embeddings is essential for designing systems that efficiently leverage large knowledge bases. Rather than sending the entire context into a prompt, embeddings are used to identify the most relevant passages and provide only the necessary information to the model.