# Noam Shazeer > ~1977– · Computer scientist; Transformer co-author; VP at Google DeepMind > > **Recorded contribution:** Transformer co-author; sparse mixture-of-experts research; Character.AI co-founder; Google DeepMind leadership ## How to use this dossier Read for a causal chain, not a hero story: inherited problem → contribution → mechanism → downstream capability → limit. Then close the page and complete the reconstruction exercise from memory. ## 1. Historical orientation Noam Shazeer was a Google Brain researcher and one of eight authors of the 2017 Transformer paper. Earlier and later work explored sparsely gated mixture-of-experts models, in which only part of a very large network is activated for each input. He later co-founded Character.AI, returned to Google in 2024 through a licensing and personnel agreement, and was identified by Google as a Google DeepMind vice president in May 2026. Roles remain volatile; the durable record is the published research. ## 2. The problem inherited Recurrent sequence models limited parallel training, while simply activating every parameter of a larger neural network made compute grow with model size. ## 3. The central contribution Shazeer co-authored the Transformer and pioneered sparsely gated mixture-of-experts approaches that decouple parameter capacity from per-token computation. ## 4. Reconstruct the mechanism 1. Represent tokens as vectors augmented with position information. 2. Use self-attention so each token can combine information from other positions in parallel. 3. In a mixture-of-experts layer, score experts with a learned router for each token. 4. Send the token only to selected experts, combine their outputs, and regularize routing so load is usable. ## 5. What changed downstream - Transformers became the dominant architecture for modern language and multimodal models. - Sparse experts offered a route to larger model capacity without activating every parameter on every token. ## 6. Attribution, limits, and uncertainty - Both contributions are multi-author results with extensive predecessors and later community development; Shazeer is not the sole inventor of either general idea. - Routing can create load imbalance, communication overhead, expert collapse, and opaque specialization; more parameters do not guarantee truth or safety. ## 7. Reconstruction lab Build a toy router with four experts and top-one selection. Plot expert loads for 100 tokens, add a balancing loss, and show the trade-off between specialization and even utilization. Feed the router an out-of-domain token distribution and watch whether a few experts overload. Compare total parameters, active parameters per token, communication cost, and achieved throughput; “larger” means different things under each measure. Disable one expert at inference and trace the failure. These experiments expose mixture-of-experts as conditional computation with a routing and distributed-systems problem attached, not free capacity or evidence that every parameter contributes equally to every answer. ## 8. Evidence trail - [Attention Is All You Need](https://arxiv.org/abs/1706.03762) — NeurIPS - [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://arxiv.org/abs/1701.06538) — ICLR - [Gemini 3.5: frontier intelligence with action](https://blog.google/intl/en-africa/products/explore-get-answers/gemini-3-5/) — Google --- *Research checked 2026-08-09. Dates, roles, and claims about living people are historical snapshots. Linked sources remain the authority; this dossier is original instructional synthesis.*