Home › Glossary › Attention
Models · intermediate

What is Attention?

A plain-English explanation of Attention (Attention Mechanism) — what it means, why it matters, and how it is used in AI.

Attention
Attention Mechanism
Attention is a mechanism in the Transformer architecture that allows a model to focus on the most relevant parts of the input when generating each part of the output.
"When translating "The cat sat on the mat because it was tired," attention allows the model to correctly link "it" to "cat" rather than "mat"."

Also known as: Self-attention, multi-head attention, scaled dot-product attention

TransformerHeadsLLM

Why does Attention matter?

Attention is the core mechanism behind all Transformer-based models including GPT, BERT, and Claude.

Practice this term

The best way to remember Attention is to practice unscrambling it. AI Terminology Scrambler uses spaced repetition to help you learn and retain AI vocabulary in just a few minutes a day.

Practice Attention now →

Related AI terms

TransformerHeadsLLM