# Alex Graves

> 1975– · Computer Scientist, AI Researcher
>
> **Recorded contribution:** Connectionist temporal classification (CTC); attention in RNNs; neural Turing machines; handwriting generation

## How to use this dossier

Read for a causal chain, not a hero story: inherited problem → contribution → mechanism → downstream capability → limit. Then close the page and complete the reconstruction exercise from memory.

## 1. Historical orientation

British AI researcher Alex Graves developed methods for learning sequential outputs without pre-segmented labels. His work on Connectionist Temporal Classification (CTC), multidimensional recurrent networks, and neural sequence generation contributed to handwriting and speech recognition. At DeepMind he also co-authored the Neural Turing Machine, which explored differentiable external memory.

## 2. The problem inherited

Sequence datasets often supplied an input stream and an output transcript but not the exact frame-by-frame alignment required by conventional supervised training.

## 3. The central contribution

Graves introduced CTC, an objective that marginalizes over valid alignments so recurrent models can learn unsegmented sequence labeling end to end.

## 4. Reconstruct the mechanism

1. Add a blank symbol to the output alphabet and predict a distribution at every input time step.
2. Define paths whose repeated labels and blanks collapse to the target transcript.
3. Use dynamic programming to sum probabilities over all valid alignment paths.
4. Differentiate the negative log probability and update the sequence model.

## 5. What changed downstream

- CTC became a foundational loss for end-to-end speech and handwriting recognition.
- The work showed how latent alignment could be integrated into neural training rather than supplied as preprocessing.

## 6. Attribution, limits, and uncertainty

- CTC assumes conditional independence between output labels given the input representation and cannot model every sequence dependency well.
- The results build on recurrent-network, dynamic-programming, speech, and handwriting research by many collaborators; later systems often combine or replace CTC with attention or transducer objectives.

## 7. Reconstruction lab

For a three-step alphabet {a, blank}, enumerate every CTC path that collapses to 'a', sum its probability, and verify the result with a small dynamic program. Extend the alphabet to include a and b and explain why repeated output symbols require an intervening blank. Compare greedy decoding with a beam that combines acoustic scores and a tiny language model. Construct a timestep distribution where the greedy path is not the most probable collapsed label. This demonstrates why CTC’s contribution is marginalization over alignments rather than a magical removal of all sequence-modeling and decoding assumptions. Inspect numerical underflow and redo the recurrence in log space, connecting the mathematical objective to the engineering details required for reliable training.

## 8. Evidence trail

- [Connectionist Temporal Classification](https://www.cs.toronto.edu/~graves/icml_2006.pdf) — ICML proceedings
- [Alex Graves publications](https://www.deepmind.com/researchers/alex-graves) — Google DeepMind
- [Neural Turing Machines](https://arxiv.org/abs/1410.5401) — arXiv

---

*Research checked 2026-08-09. Dates, roles, and claims about living people are historical snapshots. Linked sources remain the authority; this dossier is original instructional synthesis.*