Definition
A mechanism that computes how strongly elements in a representation should influence one another when producing a new representation.
Distinguish it from nearby terms
Attention is a learned computation inside a model, not human-like awareness or guaranteed focus on the correct fact.
Check your understanding
A model can use attention over a token and still fail to follow the instruction it contains.