Test Time Compute: Definition and Examples

Test Time Compute (TTC), also called 'inference-time compute', refers to the amount of computation a language model mobilizes when generating a response. Unlike train-time compute, which is fixed once and for all during training, TTC can vary dynamically based on the complexity of the request. It is a lever for performance improvement that has gained significant importance since 2024.

The core idea is simple: rather than answering immediately, the model can 'think longer' about a difficult problem. Concretely, this translates into techniques such as extended chain-of-thought, generating multiple candidate responses followed by selection (best-of-N), or tree search over the space of possible reasoning paths. Models like OpenAI o1, o3, and Claude with 'extended thinking' mode directly exploit this principle.

The appeal of Test Time Compute lies in its flexibility: you can allocate more computation only when needed, providing a better cost-performance ratio than systematically increasing model size. Research has shown that beyond a certain threshold, increasing inference-time compute can be more effective than increasing training-time compute.

For users of AI models, understanding TTC allows optimizing interactions: some problems greatly benefit from a model that 'takes its time', while for simple tasks, the added token cost and latency are not justified. This is a key parameter in choosing between a fast model and a reasoning model.