Glossary
Transformer — Glossary
A neural network architecture that uses attention mechanisms to model relationships among tokens in a sequence, foundational to modern LLMs.
Full definition
The transformer architecture processes sequences with self-attention, allowing each token representation to incorporate information from other tokens in the context window. Multi-head attention, feed-forward blocks, residual connections, and layer normalization are common building blocks.
Transformers scale effectively on parallel hardware and underpin most large language models, many speech models, and an increasing share of vision systems. Variants adjust positional encoding, attention sparsity, and encoder/decoder layouts for different tasks.