Skip to content
Omnikalc

Softmax with temperature calculator

Enter four logits and a temperature to get the softmax probabilities, the entropy in bits and the effective number of choices, as used to sample the next word in a language model.

Updated on
Your inputs

Value out of range

Value out of range

Value out of range

Value out of range

Value out of range

Results

Probability of choice 1

63.81 %

Probability of choice 2
23.47 %
Probability of choice 3
9.54 %
Probability of choice 4
3.18 %
Entropy of the distribution
1.386 bits
Effective number of choices
2.61
Several choices remain plausible

The softmax with temperature calculator converts four logits into probabilities, and gives the entropy and the effective number of choices, to see how the temperature sharpens or flattens a distribution.

How the calculation works

  • Enter the four logits.
  • Enter the temperature.
  • The calculator gives the four probabilities, the entropy in bits and the effective number of choices.

Example

The logits 2, 1, 0.1 and −1 at temperature 1 give exp values 7.39, 2.72, 1.11 and 0.37, which sum to 11.58. The probabilities are 63.81 %, 23.47 %, 9.54 % and 3.18 %. The entropy is 1.386 bits, equal to 2.61 equally likely choices. Lowering the temperature to 0.5 gives the first choice a much larger share.

Why it matters

The calculator helps students in machine learning, developers who tune the sampling of a language model and anyone who wants to understand how scores become probabilities.

Good to know

In language models, the temperature is applied to the logits before sampling. Values around 0.2 give focused, repeatable text, around 1 give varied text, and above 1.5 the output becomes erratic. Sampling methods such as top-k and top-p cut the tail of the distribution before the draw. The softmax is also the output layer of classifiers, where it pairs with the cross-entropy loss. It is not a calibrated probability by itself: a model can be overconfident, and a temperature fitted on validation data is a standard way to fix it. The effective number of choices, the exponential of the entropy, is the perplexity of the distribution. With equal logits it equals the number of choices, here 4.

The Shannon entropy calculator explains the entropy measure, the logistic regression probability calculator covers the two-class case, the LLM memory calculator sizes models and the probability calculator handles basic chances.

Frequently asked questions

What does the softmax do?

It turns any list of scores, the logits, into probabilities that are positive and add up to 1. Each probability is exp(z / T) divided by the sum of these values over all choices, so a higher logit always gets a higher probability.

How does the temperature change the result?

A low temperature sharpens the distribution: the best choice gets almost all the probability, and the output becomes more predictable. A high temperature flattens it toward equal chances. At T = 1 the logits are used as they are, and as T tends to 0 the choice becomes a plain maximum.

Why subtract the maximum?

Subtracting the largest value before taking the exponential does not change the probabilities, since it cancels in the ratio, but it prevents numbers from overflowing when the logits are large. This calculator does it, as most libraries do.

Was this calculator helpful?

Thanks for your feedback!

Results are estimates provided for information purposes. They do not replace professional advice.