Transformer

The transformer is a neural network architecture introduced in "Attention Is All You Need" (Vaswani et al., 2017) that replaced recurrence with multi-head self-attention. Transformers process entire sequences in parallel, enabling efficient training on large datasets with GPU/TPU hardware.

The architecture is the foundation of virtually all state-of-the-art language models (GPT, BERT, T5) and has expanded to vision, audio, and multimodal domains.