Glossary
Softmax — Glossary
A function that converts logits into a probability distribution over classes or tokens.
Full definition
Softmax is used in attention weights and output token distributions. Temperature modifies logits before softmax during sampling.