Vectors, matrices, and linear transformations

Prerequisites: Algebra, functions, and systems of linear equations.

Learning objectives

• Compute a matrix-vector product and interpret its columns.
• Explain why transformation order matters.
• Use rank and null space to diagnose lost information.

A matrix represents a linear map

A vector can represent a displacement, a feature list, or coordinates relative to a chosen basis. A transformation T is linear when T(u+v)=T(u)+T(v) and T(cu)=cT(u). In particular, a linear map sends the zero vector to zero. Translating every point by a fixed nonzero displacement is therefore not linear in ordinary Cartesian coordinates, although homogeneous coordinates provide a useful larger representation.

For a matrix A with two rows and two columns, multiplying A by the column vector (x,y) takes x times the first column plus y times the second column. This column interpretation explains matrix multiplication rather than treating it as a memorized arrangement of arithmetic. If A has rows (2,1) and (0,1), then A(x,y)=(2x+y,y). Applied to (3,2), it produces (8,2).

Composition has an order

Let S scale horizontal coordinates by two: S(x,y)=(2x,y). Let R rotate a vector a quarter turn counterclockwise: R(x,y)=(-y,x). Start with (1,1). Applying S then R gives (-1,2). Applying R then S gives (-2,1). These are different, so matrix multiplication is generally not commutative. In the product RS applied to a column vector, the rightmost map S acts first.

This convention matters in graphics, robotics, and data transformations. A record of matrix dimensions prevents another class of mistakes. An m by n matrix maps n-component input vectors to m-component output vectors. Two transformations compose only if the intermediate dimensions match. A formula that happens to use square matrices can hide this requirement.

Rank tells you what survives

The span of the columns is the set of possible outputs. Its dimension is the rank. The null space contains inputs mapped to zero. For P(x,y)=(x,0), every vertical vector lies in the null space, and outputs occupy only the horizontal axis. P loses information: (3,1) and (3,9) both produce (3,0). No inverse can recover every original input from that output.

Solving Ax=b asks whether b lies in the column span and, if so, whether multiple inputs produce it. Full column rank gives uniqueness when a solution exists; full row rank gives existence for every b. Both hold for an invertible square matrix. Numerical computation adds another question: whether tiny input changes produce large output changes. Exact invertibility alone does not guarantee a numerically reliable calculation, a distinction developed in numerical stability.

Practice problem

A sensor reports two measurements from an unknown vector (x,y). First use A(x,y)=(x+y,x-y). Recover the input when the measurements are (10,4), and show why the input is unique. Then replace the second measurement with twice the first, giving B(x,y)=(x+y,2x+2y). Find two different inputs producing (10,20), identify one nonzero null-space vector, and explain why collecting two numbers does not necessarily provide two independent pieces of information. Finally decide whether (10,21) is a possible exact output of B.

Worked solution

Adding the two equations for A gives 2x=14, so x=7 and y=3. The two independent equations determine one input; equivalently the null space contains only zero. For B, inputs (7,3) and (6,4) both give (10,20). Their difference (1,-1) is a nonzero null-space vector because its coordinates sum to zero. Every output's second coordinate is twice its first, so the second reported number repeats information instead of adding an independent constraint. Output (10,21) is impossible under the exact model. With noisy real measurements it could appear, requiring an estimation problem and a stated error model rather than an exact inverse. Rank concerns independent information, not simply the number of recorded values.

Check your understanding

If a square matrix maps two different vectors to the same output, what follows?

• It has a nonzero null-space vector and cannot be inverted on all inputs.
• Its entries must all be zero.
• It cannot be multiplied by another matrix.

Explanation

Subtract the two inputs. Their nonzero difference maps to zero, so the transformation loses information.

For column vectors, which transformation acts first in RSv?

• R.
• S.
• They must act simultaneously.

Explanation

The expression is R(Sv). The rightmost transformation acts first, and exchanging the order may change the result.

Primary reading

• Linear Algebra: MIT OpenCourseWare

Open the study and save your work →