The AI Learning Hub
◆ Definition

Attention

The mechanism at the heart of the transformer: for every token, the model weighs how relevant every other token in the context is, letting meaning flow across a whole document rather than word by word.

Learn more in these free lessons

From the journal

Related terms