Logits and log probabilities

When a large language model picks its next token, its final layer produces one number for every token in its vocabulary. Those numbers are logits. They can be any size, positive or negative, and they do not add up to anything in particular. Softmax converts them into probabilities that sum to 1. Taking the logarithm of each probability gives log probabilities, usually negative numbers where values closer to zero mean “more likely”.

Why logs. Multiplying many small probabilities together, one per token in a long answer, quickly produces numbers too tiny for a computer to store accurately. Adding their logarithms is equivalent and numerically stable. That is why training uses cross-entropy loss, the negative log probability of the right token, and why perplexity is computed from average log probability.

Business example. A support team uses an LLM to label incoming emails as “refund”, “complaint” or “other”. Some APIs can return logprobs along with the text, as the OpenAI Cookbook example shows. If the chosen label’s probability is low, the team routes the email to a person instead of trusting the label. Some teams also use logprobs as a rough signal for possible hallucinations or to check calibration.

On the show. In episode 72, at 15:13, AMD’s Anush Elangovan describes levels of user sophistication, ending with the user who says “just give me the raw. logits from the frontier model” and handles everything else. That is his framing of AMD’s developer tools, not a measured finding. The episode names Svix and Walrus Memory as the season’s presenting sponsors in its opening sponsor read.

Why it matters. Logits and logprobs are the model’s own uncertainty, before any sampling choice such as temperature. Editorial synthesis: they are useful signals, but a confident probability is not proof of a correct answer, and many hosted APIs expose only limited logprobs or none at all.