References: edX online course MIT 8.05.1x Week 3.
Sheldon Axler (2015), Linear Algebra Done Right, 3rd edition, Springer. Chapter 3.
On a vector space , a linear operator has an inverse if for all . Here, we’re restricting the arguments given in Axler’s section 3.D by assuming that all linear operators act from back onto . (Axler’s arguments allow to map vectors from into another vector space .) We can show first that is unique:
Theorem 1 If a linear operator has an inverse, the inverse is unique.
Proof: Suppose that there are two distinct inverses and . Then
We’ve made use of the fact that , the identity operator, by the definition of the inverse, and we’ve also used the associativity property of a vector space to go from the third to the fourth term.
Because the inverse is unique, we can refer to it by the notation .
There is an important general result about operator inverses:
Theorem 2 A linear operator is invertible if and only if it is both injective and surjective.
Proof: We first recall the definitions of injective and surjective. An injective operator is one-to-one, so that if , then . A surjective operator has the entire vector space as its range.
As this is an ‘if and only if’ proof, we need to prove the theorem in both directions. First, assume that exists, so that the operator is invertible. Then for suppose that . Applying the inverse, we get
so we must have , making injective.
Next, to prove that the existence of an inverse implies surjectivity, we note that we can write any vector as
That is, there is a vector which, when operated on by , gives any vector . Thus every vector is in the range of , making surjective.
Now we need to prove that if is both injective and surjective, it has an inverse. Let the operator be defined by the property that for any vector , gives a vector such that . Because is injective, we know that must be unique, since only one vector can satisfy . We also know that must exist for every because is surjective, and must produce every vector in in its range. Since , we must have , so is an inverse on the RHS side of . To prove that as well, we have
Comparing first and last terms, we see that . Thus and so has an inverse.
[The full proof also requires that we show that is linear; the details are done in Axler’s theorem 3.56 if you’re interested.]
In the above proof, we did not need to assume anything about the dimension of the vector space , so that the result is valid for both finite and infinite-dimensional vector spaces. For a finite vector space, the result can be made even stricter. The fundamental theorem of linear maps states that for a finite vector space and linear operator
If is injective, then , since 0 is the only vector in the null space. Thus and is surjective. Conversely, if we know that , then the theorem tells us that the null space has 0 dimensions. In other words, for a finite-dimensional vector space, injectivity implies surjectivity and vice versa. Thus, in this case, if we know that is either injective or surjective, we know it has an inverse.