References: edX online course MIT 8.05.1x Week 4.
Sheldon Axler (2015), Linear Algebra Done Right, 3rd edition, Springer. Chapter 6.
Continuing from our examination of orthonormal bases and the orthogonal complement in a vector space , we can now look at the orthogonal projection, sometimes known in physics as a projection operator.
Suppose we have defined a subspace of and its orthogonal complement , so that . We can define a linear operator called the orthogonal projection operator. It has the property that, given any vector , it ‘projects’ out the component of that lies in . That is, if we write
where and , then
An example of a projection operator is an operator in 3-d space that projects a vector onto the plane. Then the plane is the subspace and the axis is the orthogonal complement .
From the definition of we can list a few properties:
- is not surjective, that is, its range is smaller than the entire space .
- is not injective, since it maps all vectors to , for all . Thus it is a many-to-one mapping.
- is not invertible, since it is not injective.
- Its null space is .
- Once is applied to any vector , all subsequent applications of have no effect. That is, once you’ve projected out the component of that lies in , all further projections into just give the same result. In other words for all integers .
- . This follows from the Pythagorean theorem, since and are orthogonal, so . Geometrically, a projection operator cannot increase the ‘length’ (norm) of a vector. This property relies on the fact that the projection is an orthogonal projection. Other projections can increase the length of a vector (think of the shadow cast by a stick; if the surface onto which the shadow falls is nearly parallel to the direction of the incoming light, the shadow is much longer than the stick).
An explicit form for can be obtained from the decomposition we had earlier
From the definition, it seems reasonable that a vector space can be decomposed into a direct sum of and . We can in fact prove this.
Theorem 1 is an orthogonal projection within the vector space if
Proof: We can take the subspace . From our earlier theorem, we know that , so we need to show that . Since for any , then , but are there vectors in that are not in ? Suppose there is such a vector such that . For such a vector, we can decompose it into where and , with (since if , then would be in , contrary to our assumption).
As , for all . Therefore . Since , (as ). Therefore we must have , implying that also. Thus and , but the only vector that can be in both a subspace and its orthogonal complement is 0, so , which contradicts our assumption above.
From property 5 above, we must have , which implies that the eigenvalues of are 0 and 1. The eigenvectors belong to either the subspace (for eigenvalue 1) or to the orthogonal complement (for eigenvalue 0).
The orthonormal basis of a vector space can be divided into two separate lists of vectors, with one list spanning the subspace and the other list spanning . A matrix representation of can be obtained by considering the action of on each of the basis vectors from the two subspaces. We have
In general, the matrix representation of an operator is defined in terms of its action on the basis vectors by
For a projection operator, we can see that this means that for the basis vectors we must have for all , while for the basis vectors we must have for all . If we list the basis vectors in the order , then is a diagonal matrix with the diagonal elements in the top rows equal to 1, and all other elements equal to zero.
In this basis, we see that (because there is at least one zero element on the diagonal) and , which is the dimension of the subspace . As the trace and determinant are invariant under a change of basis, these properties apply to any basis.