Complete Pipeline

Here is the whole journey in order:

1. 1. Text is split into tokens.
2. 2. Each token becomes an ID and then an embedding vector.
3. 3. Position information is added.
4. 4. The vectors pass through many blocks: attention moves information between positions, and MLPs process each position, all with residual connections and normalization.
5. 5. The last vector is turned into logits, one per vocabulary token.
6. 6. Softmax (with temperature and a sampling method) gives a token.
7. 7. The token is appended, and steps 4 to 6 repeat until the answer ends.

All the numbers involved, the weights, were learned earlier in training, mainly by predicting the next token, and then shaped by fine-tuning and preference training to make the model a useful assistant.