References: edX online course MIT 8.05.1x Week 3.
Sheldon Axler (2015), Linear Algebra Done Right, 3rd edition, Springer. Chapter 5.
While studying quantum mechanics, we have made extensive use of the eigenvalues and eigenvectors (the latter usually called eigenstates in quantum theory) of hermitian operators, since an observable quantity in quantum mechanics is always represented by a hermitian operator and the spectrum of possible values for a given observable is equivalent to the set of eigenvalues of that operator.
It’s useful to re-examine eigenvalues and eigenvectors from a strictly mathematical viewpoint, since this allows us to put precise definitions on many of the terms in common use. As usual, suppose we start with a vector space and an operator . Suppose there is a one-dimensional subspace of which has the property that for any vector , . That is, the operator maps any vector back into another vector in the same subspace . In that case, is said to be an invariant subspace under the operator .
You can think of this in geometric terms if we have some -dimensional vector space , and a one-dimensional subspace consisting of all vectors parallel to some straight line within . The operator acting on any vector parallel to that line produces another vector which is also parallel to the same line. Of course we can’t push the geometric illustration too far, since in general and can be complex vector spaces, so the result of acting on with might give you some complex number multiplied by .
is called an eigenvalue equation, and the number is called the eigenvalue. The vector itself is called the eigenvector corresponding to the eigenvalue . Since we can multiply both sides of this equation by any number , any multiple of is also an eigenvector corresponding to , so any vector ‘parallel’ to is also an eigenvector. (I’ve put ‘parallel’ in quotes, since we’re allowing for multiplication of by complex as well as real numbers.)
It can happen that, for a particular value of , there are two or more linearly independent (that is, non-parallel) eigenvectors. In that case, the subspace spanned by the eigenvectors is two- or higher-dimensional.
Another way of writing 1 is by introducing the identity operator :
If this equation has a solution other than , then the operator has a non-trivial null space, which in turn means that is not injective (not one-to-one) and therefore not invertible. Also, the eigenvectors of with eigenvalue are those vectors in the null space of .
An important result is
Theorem 1 Suppose are distinct eigenvalues of and are the corresponding non-zero eigenvectors. Then the set is linearly independent.
Proof: Suppose to the contrary that is linearly dependent. Then there must be some subset that is linearly independent. Suppose that is the smallest positive integer such that can be written in terms of . That is, the set is a linearly independent subset of . In that case, there are numbers such that
If we apply the operator to both sides and use the eigenvalue equation, we have
We can multiply both sides of 3 by and subtract to get
Since the set of vectors is linearly independent, and for , the only solution of this equation is for . But this would make , contrary to our assumption that is a non-zero eigenvector of . Therefore the set is linearly independent.
It turns out that there are some operators on real vector spaces that don’t have any eigenvalues. A simple example is the 2-dimensional vector space consisting of the plane. The rotation operator which rotates any vector about the origin (by some angle other than ) doesn’t leave any vector parallel to itself and thus has no eigenvalues or eigenvectors.
However, in a complex vector space, things are a bit neater. This leads to the following theorem:
Theorem 2 Every operator on a finite-dimensional, nonzero, complex vector space has at least one eigenvalue.
Proof: Suppose is a complex vector space with dimension . For some vector we can write the vectors
Because we have vectors in an -dimensional vector space, these vectors must be linearly dependent, which means we can find complex numbers , not all zero, such that
We can consider a polynomial in with the as coefficients:
The Fundamental Theorem of Algebra states that any polynomial of degree can be factored into linear factors. In our case, the actual degree of is since could be zero. So we can factor as follows:
Comparing this to 10, we can write that equation as
All the operators in the last line commute with each other since commutes with everything and commutes with itself, so in order for the last line to be zero, there has to be at least one such that . That is, there is at least one such that has a nonzero null space, which means is an eigenvalue.