FREE LESSON · Architecture & memory · 2 OF 4
Memory Hierarchy: Cache, RAM and Storage
The memory hierarchy — Locality turns a speed gap into a workable illusion
Fast, large, cheap, and persistent do not arrive together.
Registers are tiny and immediate; caches hold recently useful blocks; DRAM supplies larger working memory; storage provides capacity and persistence. The hierarchy works because programs often reuse recent data and nearby addresses. A miss pays the cost of moving a whole block from a farther layer.
Performance often depends on data movement more than arithmetic.
Average access time exposes the miss penalty
If a cache hit costs 1 ns and one access in 100 misses at a 100 ns extra penalty, misses add about 1 ns on average—roughly doubling the effective time. A small miss-rate change matters because the distant access is so expensive.
Multiply frequency by consequence; rare events dominate when their penalty is large.