# Tomas Mikolov

> ~1981– · Computer Scientist, Creator of Word2Vec
>
> **Recorded contribution:** Word2Vec — word embeddings revolution; FastText; key bridge from NLP to neural methods

## How to use this dossier

Read for a causal chain, not a hero story: inherited problem → contribution → mechanism → downstream capability → limit. Then close the page and complete the reconstruction exercise from memory.

## 1. Historical orientation

Czech computer scientist Tomáš Mikolov led work at Google on word2vec, a 2013 family of efficient shallow neural objectives for learning word embeddings from large text corpora. He later contributed to fastText at Facebook AI Research. The work did not invent distributed representations, but its speed, released code, and striking vector regularities helped make embeddings standard NLP infrastructure.

## 2. The problem inherited

Earlier neural language models learned useful word vectors but were expensive to train on very large vocabularies and corpora.

## 3. The central contribution

Mikolov and collaborators introduced efficient continuous bag-of-words and skip-gram objectives and practical approximations that made large-scale word embeddings widely accessible.

## 4. Reconstruct the mechanism

1. Sample a target word and nearby context words from a large corpus.
2. Train skip-gram to predict context from the target, or CBOW to predict the target from context.
3. Use negative sampling or hierarchical softmax to avoid a full vocabulary normalization at every step.
4. Reuse the learned input vectors and compare words by geometric similarity.

## 5. What changed downstream

- word2vec accelerated adoption of pretrained embeddings across NLP applications.
- fastText extended the idea with subword features that improve rare-word and morphology handling.

## 6. Attribution, limits, and uncertainty

- Distributed word representations predate word2vec, and the papers and software have multiple authors.
- Static embeddings collapse multiple senses and reproduce corpus stereotypes; vector analogies are sensitive to data, metric, and selection.

## 7. Reconstruction lab

Train skip-gram with negative sampling on a small corpus. Inspect nearest neighbors before and after removing one source document and measure one bias association. Derive the skip-gram training pairs produced by one sentence under two context-window sizes. Compare full softmax with negative sampling and state what objective changed, not merely what became faster. Normalize vectors before cosine comparison and test whether an analogy survives retraining with a new seed. Word2vec’s enduring lesson is distributional: useful geometry can emerge from prediction over context, but frequency, corpus selection, approximation, and social bias all shape that geometry. Compare rare and frequent words and quantify instability for each. Then explain why nearest-neighbor demonstrations are evidence about one trained model, not a universal map of language meaning.

## 8. Evidence trail

- [Efficient Estimation of Word Representations in Vector Space](https://arxiv.org/abs/1301.3781) — arXiv
- [Distributed Representations of Words and Phrases](https://arxiv.org/abs/1310.4546) — NeurIPS
- [Enriching Word Vectors with Subword Information](https://aclanthology.org/Q17-1010/) — Transactions of the ACL

---

*Research checked 2026-08-09. Dates, roles, and claims about living people are historical snapshots. Linked sources remain the authority; this dossier is original instructional synthesis.*