Conversation Memory: Definition and Examples

Conversation memory is a fundamental mechanism of generative AI systems that allows the model to retain context from previous exchanges within the same session. Without this capability, each message would be processed in isolation, as if the user were addressing an amnesiac interlocutor. Conversation memory is what transforms a simple question-answering interface into a true dialogue.

Concretely, conversation memory works by integrating the message history (user prompts and model responses) into the context sent with each new request. This context is limited by the model's context window, which varies by architecture — from a few thousand to several million tokens. When the conversation exceeds this limit, the oldest messages are truncated or summarized, which can lead to information loss.

There are several types of conversation memory. Short-term memory corresponds to the current session's history. Long-term memory, more recent, allows some systems to retain preferences and information across sessions through persistent storage mechanisms. Some platforms also offer structured memory, where the user can explicitly ask the model to remember or forget certain information.

Mastering how conversation memory works is essential in prompt engineering. It allows structuring exchanges to maximize retention of important instructions, avoid context loss in long conversations, and effectively leverage the personalization capabilities offered by persistent memory.