Orthogonality

When vectors are perpendicular — dot product zero — and orthogonal projections, the foundation of least squares and QR decomposition.

Orthogonal vectors meet at a right angle, so their dot product is zero

u dot v = |u||v|cos(theta)

0.00

theta

At 90 degrees the cosine term vanishes. That is the whole dot-product test for orthogonality.

Definition

Two vectors $\mathbf{u}$ and $\mathbf{v}$ are orthogonal if their dot product is zero: $\mathbf{u} \cdot \mathbf{v} = 0$ .

This generalizes the notion of "perpendicular" from geometry to any dimension.

An orthogonal set is a set of vectors where every pair is orthogonal. If each vector also has unit length, the set is orthonormal: $\mathbf{u}_i \cdot \mathbf{u}_j = \delta_{ij}$ (1 if $i=j$ , 0 otherwise).

Orthogonal matrices: a square matrix $Q$ with orthonormal columns. This implies $Q^T Q = I$ , so $Q^{-1} = Q^T$ .

Key properties

The zero vector is orthogonal to every vector (vacuously, $\mathbf{0}\cdot\mathbf{v}=0$ )
An orthogonal set of nonzero vectors is automatically linearly independent
Orthogonal matrices have determinant $\pm1$ (rotations: $+1$ , reflections: $-1$ )
The inverse of an orthogonal matrix is just its transpose — no computation needed

Common mistakes

Confusing orthogonal with orthonormal: orthogonal only requires zero dot products; orthonormal additionally requires unit length — $Q^TQ=I$ specifically needs orthonormal columns, not merely orthogonal ones
Assuming any matrix with $\det = \pm1$ is orthogonal: determinant $\pm1$ is necessary but not sufficient — shears can also have determinant 1 without being orthogonal

Standard basis

The standard basis vectors $\mathbf{e}_1 = (1,0,0)$ , $\mathbf{e}_2 = (0,1,0)$ , $\mathbf{e}_3 = (0,0,1)$ form an orthonormal set: $\mathbf{e}_i \cdot \mathbf{e}_j = \delta_{ij}$ .

The identity matrix $I$ has these as columns — it's trivially orthogonal.

Try it

Verify that $Q = \frac{1}{\sqrt{2}}\begin{pmatrix}1&1\\1&-1\end{pmatrix}$ is an orthogonal matrix.

Solution

$Q^T Q = \frac{1}{2}\begin{pmatrix}1&1\\1&-1\end{pmatrix}\begin{pmatrix}1&1\\1&-1\end{pmatrix} = \frac{1}{2}\begin{pmatrix}2&0\\0&2\end{pmatrix} = I$ . ✓

Columns $(1/\sqrt{2}, 1/\sqrt{2})^T$ and $(1/\sqrt{2}, -1/\sqrt{2})^T$ have unit length and are perpendicular. This matrix represents a 45° rotation.

Related concepts

Needs first

Dot Product Vector Spaces

Linear Transformations Norms Principal Component Analysis

View in full concept graph →