Temperature changes how a model picks each next token from its list of likely options. Near zero, it almost always chooses the top option, giving consistent, focused answers. Higher values spread the chances more evenly, producing more varied and surprising text, but also more mistakes. It is available in most AI APIs and developer playgrounds.
How it works
At each step, a language model calculates a score for every possible next token in its vocabulary. These scores are turned into probabilities using a function called softmax. Temperature is applied just before that: the scores are divided by the temperature value. A temperature below 1 makes the differences between scores larger, so the most likely token becomes even more likely. A temperature above 1 shrinks the differences, giving less likely tokens a real chance of being picked. At or near 0, generation becomes close to deterministic, always taking the top token, sometimes called greedy decoding. Many APIs accept values from 0 up to 1 or 2, with a default around 1, though ranges vary by provider. Temperature usually works alongside other sampling settings. Top-p, or nucleus sampling, limits choices to the smallest set of tokens whose combined probability reaches a threshold, such as 0.9. Top-k limits choices to the k most likely tokens. Providers often recommend adjusting temperature or top-p, not both. Even at temperature 0, outputs can vary slightly between runs because of how computations are batched on hardware.
Why it matters
Temperature lets you match the model's behaviour to the task. Low settings suit factual answers, data extraction, coding and anything that must be consistent, such as product descriptions in a fixed format. Higher settings suit brainstorming, taglines, story ideas and creative variety. Knowing this helps you understand why the same prompt gives different answers, and how to make an automated workflow more reliable.
Where you will see it
- Setting a low temperature for extracting invoice data into a consistent spreadsheet format.
- Raising temperature to brainstorm dozens of varied video titles or slogans.
- Keeping customer support replies steady and on-policy with a low setting.
Limits and common misconceptions
- Low temperature does not make answers true; a model can confidently repeat the same wrong answer every time.
- High temperature quickly leads to rambling, off-topic or nonsensical text; creativity has a sweet spot, not a straight line.
- Many consumer chat apps do not expose temperature, and reasoning models may fix or restrict it.
How it differs from related terms
- Temperature vs Parameters Parameters are learned during training and stay fixed; temperature is a setting you choose each time you generate output.
- Temperature vs Hallucination Hallucination is a model stating false information; higher temperature can increase it, but low temperature does not prevent it.
Frequently asked questions
What temperature should I use?
For facts, data extraction, code and fixed formats, use a low value, around 0 to 0.3. For general writing, the default is usually fine. For brainstorming and creative variety, try somewhat higher values. Test a few settings on your actual task, as the best value depends on the model and prompt.
Does temperature 0 make AI output identical every time?
Usually very close, but not always. Temperature 0 tells the model to pick the most likely token each time, so outputs are highly consistent. Small differences can still appear due to how calculations are processed on GPUs, and updates to the model can change results.
What is the difference between temperature and top-p?
Both control randomness. Temperature reshapes the probabilities of all possible next tokens, making choices sharper or flatter. Top-p cuts off the unlikely tail, only sampling from the most probable tokens that together reach a set probability. Providers generally suggest adjusting one or the other, not both at once.
Related terms
- Speech to text AI that converts spoken audio into written text, also called automatic speech recognition (ASR) or transcription.
- Synthetic data Data created by algorithms or AI models, rather than collected from real events, used to train, test or fine-tune AI systems.
- Text to image Generative AI that turns a written prompt into a new image by learning how words relate to visual features.
- Text to speech (TTS) Technology that converts written text into spoken audio, with modern neural systems producing voices that sound close to human speech.
- Text to video Generative AI that creates moving video clips from a written prompt, and often from a starting image as well.