# Sepp Hochreiter > 1967– · Computer Scientist, Co-inventor of LSTM > > **Recorded contribution:** Co-invented LSTM; vanishing gradient problem analysis (1991) ## How to use this dossier Read for a causal chain, not a hero story: inherited problem → contribution → mechanism → downstream capability → limit. Then close the page and complete the reconstruction exercise from memory. ## 1. Historical orientation Sepp Hochreiter analyzed the vanishing-gradient problem in his 1991 diploma thesis and co-invented Long Short-Term Memory with Jürgen Schmidhuber in 1997. His later research spans deep learning, bioinformatics, and architectures that preserve useful information across long sequences. This work belongs to the history of making machine behavior depend on representations, evidence, objectives, and evaluation rather than on a separate hand-written rule for every case. The chronology is used causally: it connects the inherited constraint to an implementable mechanism and then to later reuse, instead of treating fame, job title, or eventual market success as the explanation. ## 2. The problem inherited Backpropagation through an ordinary recurrent network multiplies many derivatives, causing early learning signals to shrink toward zero or grow unstably before they can influence long-range memory. The first-principles difficulty is not simply “make a machine intelligent”: it is to specify what is represented, where evidence comes from, how a procedure changes with evidence, and what observation would count as failure. ## 3. The central contribution The original LSTM introduced a memory cell with a controlled self-connection and gates, creating a path where error can circulate more stably across time. Its importance therefore lies in an inspectable learning or search mechanism, not in an anthropomorphic claim about the system understanding as a person does. ## 4. Reconstruct the mechanism 1. Unroll a recurrent computation and multiply Jacobian factors to observe vanishing or exploding gradients. State the task, representation, and success measure before selecting an algorithm. 2. Introduce a cell state whose self-connection can preserve information across steps. Trace where evidence or feedback changes internal state; do not hide learning behind a product label. 3. Learn gates that control when inputs write state and when stored information affects outputs. Run the resulting procedure on a small case where every intermediate value can be inspected. 4. Measure retention, gradient norm, saturation, computation, and failure as delay and distractors increase. Change the data, objective, or environment and locate the first place behavior ceases to generalize. ## 5. What changed downstream - LSTM enabled major advances in handwriting, speech, translation, and sequence prediction and made gating a standard neural design idea. - Downstream systems inherited both a reusable method and a warning: benchmark performance depends on the data-generating process and evaluation contract. - The transferable first-principles lesson is to separate the artifact named in “Co-invented LSTM; vanishing gradient problem analysis (1991)” from the mechanism, surrounding institution, and evidence that allowed later systems to depend on it. ## 6. Attribution, limits, and uncertainty - Hochreiter shares LSTM credit with Schmidhuber, and later forget gates, implementations, applications, datasets, and hardware involved many researchers. The architecture mitigates but does not abolish optimization difficulty or unlimited-memory constraints. - Later success does not retroactively prove that every historical motivation, cognitive analogy, or priority claim was correct. - The subject is living or the registry has no death year; current titles and institutional affiliations are treated as dated snapshots verified on 2026-08-09, not permanent identity claims. ## 7. Reconstruction lab Implement a delayed-copy task with a basic recurrent network and LSTM. Plot gradient norms at every step, vary delay from 5 to 100, and identify the first failure regime. Report the representation, objective, update/search rule, held-out test, and one deliberately adversarial example. ## 8. Evidence trail - [Long Short-Term Memory](https://www.bioinf.jku.at/publications/older/2604.pdf) — Neural Computation - [Sepp Hochreiter](https://www.jku.at/en/institute-for-machine-learning/about-us/team/sepp-hochreiter/) — Johannes Kepler University Linz - [Sepp Hochreiter](https://en.wikipedia.org/wiki/Sepp_Hochreiter) — Wikipedia contributors · overview and bibliography - [Sepp Hochreiter structured identity record](https://www.wikidata.org/wiki/Q7451965) — Wikidata contributors · CC0 --- *Research checked 2026-08-09. Dates, roles, and claims about living people are historical snapshots. Linked sources remain the authority; this dossier is original instructional synthesis.*