# Ashish Vaswani > ~1986– · Computer Scientist, Lead Author of Transformer > > **Recorded contribution:** Lead author "Attention Is All You Need" — Transformer architecture (2017) ## How to use this dossier Read for a causal chain, not a hero story: inherited problem → contribution → mechanism → downstream capability → limit. Then close the page and complete the reconstruction exercise from memory. ## 1. Historical orientation Indian computer scientist Ashish Vaswani was the first author of the 2017 paper Attention Is All You Need, written with seven Google colleagues. The paper introduced the Transformer, an encoder-decoder architecture built around multi-head self-attention rather than recurrence or convolution. Vaswani later co-founded Adept AI and Essential AI. Public biographical claims should remain secondary to the well-documented paper record. The Transformer replaced recurrence with repeated attention and position-wise transformation, allowing training to process all sequence positions concurrently. Vaswani's first authorship marks leadership in a team result whose importance lies in the complete encoder-decoder architecture, not merely the existence of an attention formula. ## 2. The problem inherited Recurrent sequence models limited parallel training and made long-range information pass through many sequential states. ## 3. The central contribution Vaswani led the Transformer paper, demonstrating that attention-only sequence transduction could train efficiently and achieve strong translation results. ## 4. Reconstruct the mechanism 1. Project token representations into queries, keys, and values. 2. Compute scaled query-key similarities, mask when required, and normalize them into weights. 3. Mix values in multiple heads, then apply feed-forward, residual, and normalization layers. 4. Add position information and stack encoder and autoregressive decoder blocks for sequence transduction. ## 5. What changed downstream - Transformers became the dominant architecture for language models and spread to vision, audio, biology, and multimodal systems. - Their parallel training characteristics enabled very large models and new accelerator workloads. - The architecture became a general substrate for language, vision, audio, biology, robotics, and multimodal systems because attention exposes a flexible interface for relating sets or sequences of representations. ## 6. Attribution, limits, and uncertainty - The Transformer has eight authors and predecessors in attention, memory networks, convolutions, and sequence modeling; first authorship is not sole invention. - Quadratic attention cost, data and compute concentration, hallucination, bias, and misuse are not resolved by the architecture. - Quadratic full attention can be expensive, positional information must be supplied, and the architecture's later success depended on scaling, datasets, accelerators, software, and many post-2017 innovations. ## 7. Reconstruction lab Compute scaled dot-product attention for four tokens, split the dimensions into two heads, and show a dependency lost when causal masking is applied incorrectly. Remove positional encodings and permute the input, then show exactly which outputs become permutation-equivariant and why content attention alone cannot recover original order. ## 8. Evidence trail - [Attention Is All You Need](https://arxiv.org/abs/1706.03762) — NeurIPS - [Transformer paper proceedings record](https://papers.nips.cc/paper/7181-attention-is-all-you-need) — NeurIPS - [Ashish Vaswani](https://en.wikipedia.org/wiki/Ashish_Vaswani) — Wikipedia contributors --- *Research checked 2026-08-09. Dates, roles, and claims about living people are historical snapshots. Linked sources remain the authority; this dossier is original instructional synthesis.*