◆ Definition
Attention
The mechanism at the heart of the transformer: for every token, the model weighs how relevant every other token in the context is, letting meaning flow across a whole document rather than word by word.
Learn more in these free lessons
- The Transformer Architecture · AI Deep Dive
- Ordering, Recency and Attention · Agent Engineering: Building the Harness
- Beyond Vanilla Transformers: MoE and Friends · AI Deep Dive
- Human Review That Scales · Does Your AI Actually Work?
- Ending a Run on Purpose · Agent Engineering: Building the Harness
- Context Engineering · AI Deep Dive