Orthonormal basis and orthogonal complement

References: edX online course MIT 8.05.1x Week 4.

Sheldon Axler (2015), Linear Algebra Done Right, 3rd edition, Springer. Chapter 6.

Once we have defined an inner product defined on a vector space {V}, we can create an orthonormal basis for {V}. A list of vectors {\left(e_{1},e_{2},\ldots,e_{n}\right)} is orthonormal if

\displaystyle  \left\langle e_{i},e_{j}\right\rangle =\delta_{ij} \ \ \ \ \ (1)

That is, any pair of vectors is orthogonal, and all the vectors have norm 1. In 3-d space, the unit vectors along the three axes form an orthonormal list.

Given an orthonormal list, we can construct a vector from the vectors in that list by

\displaystyle   v \displaystyle  = \displaystyle  a_{1}e_{1}+a_{2}e_{2}+\ldots+a_{n}e_{n}\ \ \ \ \ (2)
\displaystyle  \displaystyle  = \displaystyle  \sum_{i=1}^{n}a_{i}e_{i} \ \ \ \ \ (3)

for {a_{i}\in\mathbb{F}}. The norm of {v} has a simple form:

\displaystyle   \left\langle v,v\right\rangle ^{2} \displaystyle  = \displaystyle  \left\langle \sum_{i=1}^{n}a_{i}e_{i},\sum_{i=1}^{n}a_{i}e_{i}\right\rangle \ \ \ \ \ (4)
\displaystyle  \displaystyle  = \displaystyle  \sum_{i=1}^{n}\left\langle a_{i}e_{i},a_{i}e_{i}\right\rangle +\mbox{zero terms}\ \ \ \ \ (5)
\displaystyle  \displaystyle  = \displaystyle  \sum_{i=1}^{n}\left|a_{i}\right|^{2} \ \ \ \ \ (6)

The ‘zero terms’ in the second line are terms involving {\left\langle a_{i}e_{i},a_{j}e_{j}\right\rangle } for {i\ne j} which are all zero because of 1.

This result shows that an orthonormal list of vectors is linearly independent, since if we form the linear combination

\displaystyle  v=a_{1}e_{1}+a_{2}e_{2}+\ldots+a_{n}e_{n}=0 \ \ \ \ \ (7)

then {\left\langle v,v\right\rangle =0} so from 6 we must have all {a_{i}=0}, which means the list is linearly independent.

If we have an orthonormal list {\left(e_{1},e_{2},\ldots,e_{n}\right)} that is also a basis for {V}, then any vector {v\in V} can be written as

\displaystyle  v=a_{1}e_{1}+a_{2}e_{2}+\ldots+a_{n}e_{n} \ \ \ \ \ (8)

The coefficients {a_{i}} can be found by taking the inner product {\left\langle e_{i},v\right\rangle =a_{i}} (using 1), so we have

\displaystyle  v=\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i} \ \ \ \ \ (9)

For example, in 3-d space, the 3 unit vectors along the {x,y,z} axes form an orthonormal basis for the space. However, the unit vectors along the {x,y} axes form an orthonormal list, but this is not a basis for 3-d space since no vector with a {z} component can be written as a linear combination of these two vectors.

If we have any basis (not necessarily orthonormal), we can form an orthonormal basis using the Gram-Schmidt orthogonalization procedure. We’ve already met this in the context of quantum mechanics, and the derivation for a general finite vector space is much the same, so I’ll just quote the result. The procedure is iterative and follows these steps:

The first vector {e_{1}} in the orthonormal basis is defined by

\displaystyle  e_{1}=\frac{v_{1}}{\left|v_{1}\right|} \ \ \ \ \ (10)

where {v_{1}} is the first vector (well, any vector, really) in the non-orthonormal basis.

Given vector {e_{j-1}} in the orthonormal basis, we can form {e_{j}} from the formula

\displaystyle  e_{j}=\frac{v_{j}-\sum_{i=1}^{j-1}\left\langle e_{i},v_{j}\right\rangle e_{i}}{\left|v_{j}-\sum_{i=1}^{j-1}\left\langle e_{i},v_{j}\right\rangle e_{i}\right|} \ \ \ \ \ (11)

{e_{j}} clearly has norm 1, and we can check that {\left\langle e_{i},e_{j}\right\rangle =\delta_{ij}} by direct calculation. Note that although we’ve indexed the vectors {v_{i}} in the original basis, we can take them in any order when calculating the orthonormal basis via the Gram-Schmidt procedure.

Orthogonal complement

Suppose we have a subset {U} (not necessarily a subspace) of {V}. Then we can define the orthogonal complement {U^{\perp}} of {U} as the set of all vectors that are orthogonal to all vectors {u\in U}. More formally:

\displaystyle  U^{\perp}\equiv\left\{ v\in V|\left\langle v,u\right\rangle =0\mbox{ for all }u\in U\right\} \ \ \ \ \ (12)

A useful general theorem is as follows.

Theorem 1 If {U} is a subspace of {V}, then {V=U\oplus U^{\perp}}. (Recall the direct sum.)

Proof: Given an orthonormal basis of {U}: {\left(e_{1},e_{2},\ldots,e_{n}\right)}, we can write any {v\in V} as the sum

\displaystyle  v=\underbrace{\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i}}_{\in U}+\underbrace{v-\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i}}_{\in U^{\perp}} \ \ \ \ \ (13)

On the RHS, we’ve just added and subtracted the same term from {v}. Since the first term is a linear combination of the basis vectors of {U}, the overall sum is a vector in {U}. To see that the second term is in {U^{\perp}}, take the inner product with any of the basis vectors {e_{k}}:

\displaystyle   \left\langle e_{k},v-\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i}\right\rangle \displaystyle  = \displaystyle  \left\langle e_{k},v\right\rangle -\left\langle e_{k},\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle e_{i}\right\rangle \ \ \ \ \ (14)
\displaystyle  \displaystyle  = \displaystyle  \left\langle e_{k},v\right\rangle -\sum_{i=1}^{n}\left\langle e_{i},v\right\rangle \delta_{ik}\ \ \ \ \ (15)
\displaystyle  \displaystyle  = \displaystyle  \left\langle e_{k},v\right\rangle -\left\langle e_{k},v\right\rangle \ \ \ \ \ (16)
\displaystyle  \displaystyle  = \displaystyle  0 \ \ \ \ \ (17)

Finally, since the two vector spaces in a direct sum can have only the zero vector in their intersection, we need to show that {U\cap U^{\perp}=\left\{ 0\right\} }. However if a vector {v} is in both {U} and {U^{\perp}} then it must be orthogonal to itself, so {\left\langle v,v\right\rangle =0} which implies {v=0}. \Box

Thus any vector space {V} can be decomposed into two orthogonal subspaces (assuming that {V} has any subspaces other than {\left\{ 0\right\} }).

6 thoughts on “Orthonormal basis and orthogonal complement

  1. Pingback: Projection operators | Physics pages

  2. Pingback: Linear functionals and adjoint operators | Physics pages

  3. Pingback: Gram-Schmidt orthogonalization – a couple of examples | Physics pages

  4. Pingback: Vector spaces – number of dimensions | Physics pages

  5. Pingback: Spectral theorem for normal operators | Physics pages

  6. Pingback: Non-denumerable basis: position and momentum states | Physics pages

Leave a Reply

Your email address will not be published. Required fields are marked *