Whisper: Definition and Examples

Whisper is an automatic speech recognition (ASR) system created by OpenAI and released as open source in September 2022. Trained on over 680,000 hours of multilingual audio collected from the web, it can transcribe speech to text, automatically detect the spoken language, and translate from many languages into English.

Unlike traditional ASR systems that require language- or domain-specific training, Whisper adopts a large-scale multitask supervised approach. Its architecture is based on an encoder-decoder Transformer model: audio is divided into 30-second segments, converted to a log-Mel spectrogram, processed by the encoder, and then the decoder generates the corresponding text. This architecture gives it exceptional robustness to accents, background noise, and technical jargon.

Whisper is available in several model sizes (tiny, base, small, medium, large), allowing a trade-off between accuracy and speed. The large model achieves near-human transcription performance on many benchmarks. Its open-source nature has spawned a rich ecosystem: faster-whisper for optimized inference, whisper.cpp for local CPU execution, and integrations into many productivity tools.

In the field of prompt engineering, Whisper plays a key role by converting voice instructions into text usable by language models. It thus serves as an essential link in multimodal pipelines where voice becomes a natural interface for interacting with AI.