Memory System: Definition and Examples

A Memory System refers to the set of technical and architectural mechanisms that allow a language model to retain information between interactions. By default, LLMs are 'stateless': each new conversation starts from scratch. A memory system bridges this limitation by storing contextual data—user preferences, decision history, learned facts—in files, databases, or vector embeddings.

Several levels of memory can be distinguished. In-context memory corresponds to the information present in the active token window: system instructions, previous messages, injected documents. Persistent memory goes beyond the current session: it relies on external files, vector databases, or retrieval systems (RAG) that feed the model with relevant information for each new request.

Modern memory systems are often organized by type: user memory (who the interlocutor is, their preferences), project memory (business context, ongoing decisions), feedback memory (past corrections not to be repeated), and reference memory (where to find information). This categorization allows the model to mobilize the right type of information at the right time.

The central challenge of a Memory System is the trade-off between completeness and relevance. Storing too much information saturates the context and degrades performance; storing too little loses continuity. The best implementations use indexes, summaries, and semantic search mechanisms to load only the memories truly useful for the current task.