Follow One Token
FOLLOW ONE TOKEN
Watch a single token travel through the whole model
Pick a word below, then step through exactly what happens to it — from raw text to its contribution to the final answer.
STEP1 / 9
This journey uses simplified, illustrative numbers to make the trip easy to follow — the sequence of operations (tokenize, embed, attend, transform, predict) mirrors real models.
In more detail
This walk-through follows a single token from input to output:
- 1. The text is split into tokens, and each becomes an ID.
- 2. The ID is looked up in the embedding table to get a vector.
- 3. Position information is added.
- 4. The vector passes through the blocks. In each block, attention gathers information from earlier positions and the MLP processes it, with the results added to the residual stream.
- 5. A final normalization is applied, and the unembedding turns the last vector into logits.
- 6. Softmax gives probabilities, and one token is chosen.
- 7. That token is appended, and the process repeats for the next one.
Simplification: the numbers are illustrative. The order of operations matches real models.