Orthonormal basis and orthogonal complement

References: edX online course MIT 8.05.1x Week 4.

Sheldon Axler (2015), Linear Algebra Done Right, 3rd edition, Springer. Chapter 6.

Once we have defined an inner product defined on a vector space ${V}$, we can create an orthonormal basis for ${V}$. A list of vectors ${\left(e_{1},e_{2},\ldots,e_{n}\right)}$ is orthonormal if

$\displaystyle \left\langle e_{i},e_{j}\right\rangle =\delta_{ij} \ \ \ \ \ (1)$

That is, any pair of vectors is orthogonal, and all the vectors have norm 1. In 3-d space, the unit vectors along the three axes form an orthonormal list.

Given an orthonormal list, we can construct a vector from the vectors in that list by

 $\displaystyle v$ $\displaystyle =$ $\displaystyle a_{1}e_{1}+a_{2}e_{2}+\ldots+a_{n}e_{n}\ \ \ \ \ (2)$ $\displaystyle$ $\displaystyle =$ $\displaystyle \sum_{i=1}^{n}a_{i}e_{i} \ \ \ \ \ (3)$

for ${a_{i}\in\mathbb{F}}$. The norm of ${v}$ has a simple form:

 $\displaystyle \left\langle v,v\right\rangle ^{2}$ $\displaystyle =$ $\displaystyle \left\langle \sum_{i=1}^{n}a_{i}e_{i},\sum_{i=1}^{n}a_{i}e_{i}\right\rangle \ \ \ \ \ (4)$ $\displaystyle$ $\displaystyle =$ $\displaystyle \sum_{i=1}^{n}\left\langle a_{i}e_{i},a_{i}e_{i}\right\rangle +\mbox{zero terms}\ \ \ \ \ (5)$ $\displaystyle$ $\displaystyle =$ $\displaystyle \sum_{i=1}^{n}\left|a_{i}\right|^{2} \ \ \ \ \ (6)$

The ‘zero terms’ in the second line are terms involving ${\left\langle a_{i}e_{i},a_{j}e_{j}\right\rangle }$ for ${i\ne j}$ which are all zero because of 1.

This result shows that an orthonormal list of vectors is linearly independent, since if we form the linear combination

$\displaystyle v=a_{1}e_{1}+a_{2}e_{2}+\ldots+a_{n}e_{n}=0 \ \ \ \ \ (7)$

then ${\left\langle v,v\right\rangle =0}$ so from 6 we must have all ${a_{i}=0}$, which means the list is linearly independent.

If we have an orthonormal list ${\left(e_{1},e_{2},\ldots,e_{n}\right)}$ that is also a basis for ${V}$, then any vector ${v\in V}$ can be written as

$\displaystyle v=a_{1}e_{1}+a_{2}e_{2}+\ldots+a_{n}e_{n} \ \ \ \ \ (8)$

The coefficients ${a_{i}}$ can be found by taking the inner product ${\left\langle e_{i},v\right\rangle =a_{i}}$ (using 1), so we have

$\displaystyle v=\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i} \ \ \ \ \ (9)$

For example, in 3-d space, the 3 unit vectors along the ${x,y,z}$ axes form an orthonormal basis for the space. However, the unit vectors along the ${x,y}$ axes form an orthonormal list, but this is not a basis for 3-d space since no vector with a ${z}$ component can be written as a linear combination of these two vectors.

If we have any basis (not necessarily orthonormal), we can form an orthonormal basis using the Gram-Schmidt orthogonalization procedure. We’ve already met this in the context of quantum mechanics, and the derivation for a general finite vector space is much the same, so I’ll just quote the result. The procedure is iterative and follows these steps:

The first vector ${e_{1}}$ in the orthonormal basis is defined by

$\displaystyle e_{1}=\frac{v_{1}}{\left|v_{1}\right|} \ \ \ \ \ (10)$

where ${v_{1}}$ is the first vector (well, any vector, really) in the non-orthonormal basis.

Given vector ${e_{j-1}}$ in the orthonormal basis, we can form ${e_{j}}$ from the formula

$\displaystyle e_{j}=\frac{v_{j}-\sum_{i=1}^{j-1}\left\langle e_{i},v_{j}\right\rangle e_{i}}{\left|v_{j}-\sum_{i=1}^{j-1}\left\langle e_{i},v_{j}\right\rangle e_{i}\right|} \ \ \ \ \ (11)$

${e_{j}}$ clearly has norm 1, and we can check that ${\left\langle e_{i},e_{j}\right\rangle =\delta_{ij}}$ by direct calculation. Note that although we’ve indexed the vectors ${v_{i}}$ in the original basis, we can take them in any order when calculating the orthonormal basis via the Gram-Schmidt procedure.

Orthogonal complement

Suppose we have a subset ${U}$ (not necessarily a subspace) of ${V}$. Then we can define the orthogonal complement ${U^{\perp}}$ of ${U}$ as the set of all vectors that are orthogonal to all vectors ${u\in U}$. More formally:

$\displaystyle U^{\perp}\equiv\left\{ v\in V|\left\langle v,u\right\rangle =0\mbox{ for all }u\in U\right\} \ \ \ \ \ (12)$

A useful general theorem is as follows.

Theorem 1 If ${U}$ is a subspace of ${V}$, then ${V=U\oplus U^{\perp}}$. (Recall the direct sum.)

Proof: Given an orthonormal basis of ${U}$: ${\left(e_{1},e_{2},\ldots,e_{n}\right)}$, we can write any ${v\in V}$ as the sum

$\displaystyle v=\underbrace{\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i}}_{\in U}+\underbrace{v-\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i}}_{\in U^{\perp}} \ \ \ \ \ (13)$

On the RHS, we’ve just added and subtracted the same term from ${v}$. Since the first term is a linear combination of the basis vectors of ${U}$, the overall sum is a vector in ${U}$. To see that the second term is in ${U^{\perp}}$, take the inner product with any of the basis vectors ${e_{k}}$:

 $\displaystyle \left\langle e_{k},v-\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i}\right\rangle$ $\displaystyle =$ $\displaystyle \left\langle e_{k},v\right\rangle -\left\langle e_{k},\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i}\right\rangle \ \ \ \ \ (14)$ $\displaystyle$ $\displaystyle =$ $\displaystyle \left\langle e_{k},v\right\rangle -\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle \delta_{ik}\ \ \ \ \ (15)$ $\displaystyle$ $\displaystyle =$ $\displaystyle \left\langle e_{k},v\right\rangle -\left\langle e_{k},v\right\rangle \ \ \ \ \ (16)$ $\displaystyle$ $\displaystyle =$ $\displaystyle 0 \ \ \ \ \ (17)$

Finally, since the two vector spaces in a direct sum can have only the zero vector in their intersection, we need to show that ${U\cap U^{\perp}=\left\{ 0\right\} }$. However if a vector ${v}$ is in both ${U}$ and ${U^{\perp}}$ then it must be orthogonal to itself, so ${\left\langle v,v\right\rangle =0}$ which implies ${v=0}$. $\Box$

Thus any vector space ${V}$ can be decomposed into two orthogonal subspaces (assuming that ${V}$ has any subspaces other than ${\left\{ 0\right\} }$).

6 thoughts on “Orthonormal basis and orthogonal complement”

1. Pingback: Projection operators | Physics pages