Norbert Wiener

Norbert Wiener

Mathematician and a founder of cybernetics · 1894–1964 · United States

Feedback, communication, and estimation are related but different operations. A sensor supplies observations, an estimator infers a state, and a controller chooses an action. Wiener helped develop mathematical tools and a shared vocabulary for investigating these relationships across engineering and biology. Their usefulness depends on assumptions, not on declaring every system the same.

Historical bearings

Norbert Wiener was born on 26 November 1894 and died on 18 March 1964. He graduated from Tufts in 1909, earned a Harvard doctorate in 1913, studied in Europe, and joined MIT in 1919. His mathematical work included probability, harmonic analysis, and integral equations. Wartime prediction research formed part of the background to his cybernetic work. His book Cybernetics appeared in 1948. MacTutor biography

The 1943 paper Behavior, Purpose and Teleology was coauthored by Arturo Rosenblueth, Wiener, and Julian Bigelow. It analyzed behavior in terms of input, output, and purpose, including mechanisms using feedback. Shared functional descriptions can help compare a machine and an organism without proving that their internal structures, capacities, or experiences are identical. Primary paper

Stories about childhood temperament or psychological scars cannot substitute for technical explanation. Likewise, modern control, information theory, and artificial intelligence have multiple contributors. A parallel mathematical pattern does not establish a direct historical line from Wiener to every later device.

Worked mechanism: a noisy measurement should not always be trusted fully

This original scalar example illustrates linear minimum mean-square estimation. Suppose a zero-mean signal S and zero-mean noise N are uncorrelated, with variances 9 and 4. We observe X=S+N and restrict the estimator to S_hat=aX. The objective is to minimize expected squared error, not to recover every individual signal exactly.

Expand the error:

S_hat-S = (a-1)S+aN
E[(S_hat-S)²] = 9(a-1)²+4a²
              = 13a²-18a+9.

The cross term vanishes because S and N are uncorrelated. Differentiation gives 26a-18=0, so a=9/13. The second derivative is 26, confirming a minimum. The minimum mean-square error is 36/13≈2.769, compared with 4 when a=1 and 9 when a=0. For the observed value X=13, the estimate is 9.

That last answer does not prove the actual hidden signal was 9. Different pairs of signal and noise can produce the same observation. The coefficient is optimal on average within the linear class, under the stated moments. Gaussian assumptions are not required for this linear optimization. Additional distributional assumptions are needed to identify it with the unrestricted conditional-mean estimator. It is not automatically a maximum-likelihood estimate either.

For time series, a Wiener filter combines observations across time and uses correlation or spectral information. Whether future observations are allowed matters: a noncausal smoothing estimate and a causal real-time predictor solve different constrained problems. Wiener filtering chapter

An orthogonality check verifies the scalar result. At a=9/13, E[(aX-S)X]=aE[X²]-E[SX]=(9/13)13-9=0. The residual is uncorrelated with the observation used by the optimal linear estimator. This does not imply residual independence without stronger assumptions.

Feedback can become unstable

Here is a separate original control model. Let a plant state x update by x_next=x+u, and choose control u=k(r-x) for a constant target r. Define error e=r-x. Substitution gives e_next=(1-k)e.

Gain k	Error multiplier	Behavior in this model
0.5	0.5	approaches zero without sign changes
1.5	-0.5	approaches zero while alternating signs
2	-1	oscillates with constant magnitude
2.5	-1.5	alternating error grows

Stability requires |1-k|<1, hence 0<k<2. The feedback sign is negative in all four examples, but excessive gain destabilizes the discrete-time system. Delays, actuator limits, noise, and unmodeled dynamics require further analysis. “Negative feedback stabilizes” is therefore a useful intuition only when its conditions are checked.

Estimation and control also interact. Increasing sensitivity to measurement changes can help tracking but amplify measurement noise. Smoothing can reduce noise while adding lag. A sound design must state what is being optimized and what disturbances it expects.

Brownian motion: specify the scale

A standard Wiener process W has W(0)=0, almost surely continuous paths, and independent increments. For t>s, W(t)-W(s) is normal with mean zero and variance t-s. Merely saying that the increments are Gaussian leaves out their scale and drift. The process itself is not stationary: its variance grows with time, although its increments are stationary. MIT probability lecture

For example, the increment from time 2 to 5 has variance 3 and standard deviation √3. The increments over [0,2] and [2,5] are independent. But W(2) and W(5) are not independent: W(5)=W(2)+(W(5)-W(2)), so their covariance is 2. A noisy process's values and its increments must not be conflated.

Information entropy is another distinct concept. A random variable's uncertainty is not synonymous with harmful noise: whether a variation is signal or noise depends on the task and observation model. No amount of cybernetic vocabulary removes the need to define those quantities.

Exercises

1. Repeat the scalar estimation calculation with signal variance 4 and noise variance 9. Find the optimal coefficient and error. Explain the change from the worked example despite the same total observation variance.
2. In the control model, let r=10, x initially 0, and k=1.5. Compute three updates. Then explain what happens for k=2.5. Which omitted physical assumption could make either numerical prediction unrealistic?
3. Let W be standard Brownian motion. Find Var(W(4)-W(1)), Cov(W(1),W(4)), and the variance of 2W(4). Explain why dividing time into four equal pieces does not mean that all four observed positions are independent.

Compare your reasoning with the worked solutions

1. The objective becomes 4(a-1)²+9a²=13a²-8a+4. Its derivative vanishes at a=4/13, giving minimum error 36/13. Although the minimum error happens to match the earlier case, the estimate places much less weight on the observation because a larger fraction of its variance is noise. Total observation variance alone cannot identify the estimator; the covariance between signal and observation matters.
2. With k=1.5 the state values are 15, 7.5, and 11.25. Errors are -5, 2.5, and -1.25, shrinking by magnitude factor 1/2. With k=2.5 the error multiplier is -1.5, so the first states are 25, -12.5, and 43.75; the error grows. A real actuator may saturate, or the plant may have inertia or delay. The model assumes exact instantaneous state access and unrestricted additive action, so its predictions must be interpreted within that scope.
3. The increment spans length 3, so its variance is 3. Write W(4)=W(1)+(W(4)-W(1)); independence of the second term from W(1) gives covariance 1. Scaling a variable by 2 scales its variance by 4, so Var(2W(4))=16. Positions share earlier increments: W(2), for example, includes all of W(1). It is nonoverlapping increments, not accumulated positions, that are independent.

Reading the evidence

The 1943 paper is a historical argument about behavior and purpose. The MIT materials state modern mathematical formulations. The calculations above are original teaching examples and are not presented as Wiener's historical experiments. This separation lets the history and the mathematics support one another without inventing a single-person origin for entire fields.