What is attention in Transformer models?
Attention is how, for each token, a model decides how much to look at every other token in context. The result mixes values so related words can receive higher weight.
Transformers often use multi-head attention — several “heads” that can learn different relationship types.
mej7.com provides illustrative attention maps and Q/K/V sections for intuition — not measurements from a production model.