# Dzmitry Bahdanau

> ~1990– · Computer Scientist, AI Researcher
>
> **Recorded contribution:** Attention mechanism in neural machine translation; foundational to Transformers

## How to use this dossier

Read for a causal chain, not a hero story: inherited problem → contribution → mechanism → downstream capability → limit. Then close the page and complete the reconstruction exercise from memory.

## 1. Historical orientation

Belarusian-Canadian AI researcher Dzmitry Bahdanau was first author of the 2014 paper Neural Machine Translation by Jointly Learning to Align and Translate with Kyunghyun Cho and Yoshua Bengio. The paper's additive attention mechanism let a decoder select different source annotations for each output token, replacing one fixed-vector bottleneck and supplying a direct predecessor to Transformer attention. Bahdanau attention replaced one global source summary with a query-dependent context at every decoding step. A small learned network scored compatibility between the decoder state and each encoder annotation; normalized scores then produced a weighted context for predicting the next target word.

## 2. The problem inherited

Encoder-decoder translation compressed an entire source sentence into one fixed-length vector, degrading as sentences grew and obscuring word-to-word alignment.

## 3. The central contribution

Bahdanau led the first influential neural machine-translation attention model, jointly learning soft alignment and translation end to end.

## 4. Reconstruct the mechanism

1. Encode the source into a sequence of contextual annotations.
2. Score each annotation against the decoder's current state with a learned additive function.
3. Normalize scores into attention weights and form a weighted context vector.
4. Predict the next target token from context and decoder state, then repeat.

## 5. What changed downstream

- Attention improved neural translation and made learned soft alignment visible.
- The mechanism directly influenced later sequence models and the attention-centered Transformer architecture.
- The mechanism dramatically improved long-sentence neural translation and made alignment-like internal structure learnable end to end, influencing nearly every later sequence-modeling architecture.

## 6. Attribution, limits, and uncertainty

- The paper has three authors and builds on alignment models, encoder-decoders, and differentiable memory.
- Attention weights are not guaranteed faithful explanations, recurrent decoding remains sequential, and the registry's approximate birth year is not verified by the paper record.
- Soft weights can spread over irrelevant tokens or fail to reflect causal reliance, recurrent decoding limits parallelism, and strong translation scores can coexist with hallucinated names or unequal language quality.

## 7. Reconstruction lab

Compute additive attention over three encoder states for one decoder state. Change one score and trace the output probability; compare the weights with a human alignment. Perturb the source token receiving maximum attention and measure the output change, testing whether the visualized alignment is causal or merely correlated.

## 8. Evidence trail

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/abs/1409.0473) — ICLR / arXiv
- [Attention-Based Models for Speech Recognition](https://arxiv.org/abs/1506.07503) — NeurIPS / arXiv

---

*Research checked 2026-08-09. Dates, roles, and claims about living people are historical snapshots. Linked sources remain the authority; this dossier is original instructional synthesis.*