LLM Fundamentals · TRACK 04/06
How Text Generation Works
From logits to the next word: how an LLM actually turns your prompt into text, one token at a time.
// THE TRACK
01 · START HERENext-Token PredictionThe one idea behind every LLM: predict the next token, append it, and repeat.BEGINNERAutoregressive GenerationHow LLMs generate text strictly left-to-right, one token at a time, by feeding each output back in as the next input.INTERMEDIATELogitsLogits are the raw, unnormalized scores an LLM assigns to every token, which softmax turns into a probability distribution.INTERMEDIATEThe Softmax FunctionHow softmax exponentiates and normalizes raw scores into a probability distribution that sums to 1.INTERMEDIATEGreedy Decoding vs SamplingGreedy decoding always takes the most likely token; sampling rolls the dice for diversity. Learn the trade-off and when to use each.INTERMEDIATESpeculative DecodingUnderstand the clever trick where a small 'draft' model guesses several tokens ahead and a big model verifies them in one shot, often doubling generation speed for free.ADVANCEDBeam Search vs SamplingSee how beam search hunts for the single most likely sentence while sampling trades a bit of likelihood for variety, and why chat models prefer sampling.INTERMEDIATEStop SequencesLearn how stop sequences cut off generation at exactly the right spot, why they're useful for structured output, and how a bad stop string can truncate your answer.BEGINNER