Transcript — Introduction
In the last lecture, we defined unitary matrices, and so we are now ready to start looking at the complex equivalent of orthogonal diagonalization, which is unitary diagonalization. So, in this lecture, we will begin to try to mimic what we did in the real case with orthogonal diagonalization.
Unitarily Similar Matrices
We start with two familiar definitions. Definition: If A and B are matrices such that A = ((U-conjugate)-transpose)BU, where U is a unitary matrix, then we say that A and B are unitarily similar. Note: If A and B are unitarily similar, then they are similar, and hence have the same determinant, rank, eigenvalues, and trace.
Definition: If A is unitarily similar to a diagonal matrix D, then we say that A is unitarily diagonalizable.
In the real case, we showed that a matrix A is orthogonally diagonalizable if and only if it is symmetric—that is, if A-transpose = A. So, it makes sense to start our look into unitary diagonalization by looking at the complex equivalent of a symmetric matrix.
Definition: A matrix A in M(n-by-n)(C) is called Hermitian if (A-conjugate)-transpose = A.
A couple of notes:
Note number 1. If A is Hermitian, then observe that we have the conjugate of the ijth element of A equals the jith element of A, and so the diagonal entries of A must be real, and for i not equal to j, the ijth entry must be the complex conjugate of the jith entry.
2. If A is an n-by-n real symmetric matrix, then A is also Hermitian.
We now prove that Hermitian matrices satisfy the same important property that symmetric matrices did. Theorem 11.5.1: An n-by-n matrix A is Hermitian if and only if (the inner product of z and Aw) = (the inner product of Az and w) for all vectors z and w in Cn. Proof: On one hand, if A is Hermitian, then we have A-star = A, so by Theorem 11.4.5, we get (the inner product of Az and w) = (the inner product of z and (A-star)w), which equals (the inner product of z and Aw) since A-star = A. On the other hand, if (the inner product of z and Aw) = (the inner product of Az and w), then we have that (z dot product (the conjugate of Aw)) = (Az dot product (the conjugate of w)), which is (z-transpose)(A-conjugate)(w-conjugate) = (z-transpose)(A-transpose)(w-conjugate). Since this is valid for all z and w in Cn, we get, by Theorem 3.1.4 from Linear Algebra I, that A-conjugate = A-transpose. Taking the conjugate of both sides gives A = A-star as required.
We now strongly suspect that every Hermitian matrix is unitarily diagonalizable. We will prove this in the same way that we proved every symmetric matrix is orthogonally diagonalizable: by first proving that every square matrix is unitarily similar to an upper triangular matrix.
Schur’s Theorem: If A is an n-by-n matrix, then A is unitarily similar to an upper triangular matrix whose diagonal entries are the eigenvalues of A. The proof of this theorem is essentially the same as the proof of the Triangularization Theorem, except it is actually a little easier since we don’t have to worry anymore about the eigenvalues being real.
A couple of notes about this:
- We often rearrange this to write A = UT((U-conjugate)-transpose), which is called the Schur decomposition of A.
- The fact that we can unitarily triangularize every matrix in M(n-by-n)(C) is amazing and extremely useful. That is, Schur’s Theorem is an extremely important theorem in linear algebra.
Spectral Theorem for Hermitian Matrices
We can now easily prove that every Hermitian matrix is unitarily diagonalizable. The Spectral Theorem for Hermitian Matrices: If A is Hermitian, then it is unitarily diagonalizable.
Of course, the proof of this theorem should be very similar to our proof of the Principal Axis Theorem. However, I am going to use this opportunity to try to help you take your ability to prove things to the next level. When we study proofs in mathematics, we need to do more than just verify that the steps of the proof are correct. We really need to try to understand the technique and strategies that are being used in the proof. If necessary, refer back to Lecture 19, Orthogonal Diagonalization, and look over the proof of Theorem 10.2.1 and the proof of the Principal Axis Theorem. Really try to think about the strategy being used in both proofs.
In the proof of Theorem 10.2.1, notice that what we did is use the fact that A and D are similar to prove that A must have the same property as D, namely the property that D-transpose = D. In the proof of the Principal Axis Theorem, our strategy was to apply the Triangularization Theorem to get that A is orthogonally similar to an upper triangular matrix T. We then used the fact that A and T are similar to prove that T has the same property as A, namely A-transpose = A. We then proved that every upper triangular symmetric matrix is diagonal.
Therefore, for the Spectral Theorem for Hermitian Matrices, we will use exactly the same strategy. That is, we will use Schur’s Theorem to get that A is unitarily similar to an upper triangular matrix T. We will then use this fact to prove that T must have the same property as A—that is, T must also be Hermitian. Finally, we will prove that every upper triangular Hermitian matrix is, in fact, diagonal.
Here we go. Proof: By Schur’s Theorem, there exists a unitary matrix U such that (U-star)AU = T is upper triangular. Since A-star = A, we get that T-star = ((U-star)AU)-star, which, by properties of conjugate transposes, is (U-star)(A-star)((U-star)-star), which equals (U-star)AU, which is T. Hence, T-star = T, so T is Hermitian. Then, T is also lower triangular since T-star is lower triangular, and this implies that T must be diagonal as required.
Take a minute to read over the strategy and the proof again. Really make sure you understand the strategy being used, as we will use it some more. Additionally, as usual, this proof proves more than just the statement of the theorem is true. Try to figure out what other important result this proves.
The other important result proven is that all eigenvalues of a Hermitian matrix are real. In particular, we have shown that A is similar to the diagonal matrix T, and therefore they have the same eigenvalues. Since T is diagonal, we can write T is diag(t11 through tnn), and we have that the eigenvalues of T are the entries t11 to tnn. Since T is also Hermitian, we have that diag(t11 to tnn) = T, which equals T-star, which is the diagonal matrix whose diagonal entries are the conjugates of t11 through tnn. Thus, every eigenvalue of T—and hence, A—satisfies tii = the conjugate of tii, and hence are real.
Let’s state this as a theorem. Theorem 11.5.4: Every eigenvalue of a Hermitian matrix is real. Note that since every real symmetric matrix is Hermitian, I have also proven that all eigenvalues of a symmetric matrix are real, as I promised that I would back in Lecture 19.
I always say that math is awesome. But, unfortunately, nothing is perfect. We would hope that since Hermitian matrices are the complex equivalent of symmetric matrices, that they would share all the same properties. However, unlike the case of symmetric matrices and orthogonal diagonalization, the converse of the Spectral Theorem for Hermitian Matrices is not true. That is, there are unitarily diagonalizable matrices which are not Hermitian.
Example: The matrix A which equals [0, 1; -1, 0] is unitarily diagonalizable, but not Hermitian.
We notice that this matrix satisfies a condition that is very similar to being symmetric. That is, it satisfies A-transpose = -A. We call such a matrix skew-symmetric. Of course, we extend this to the complex case. Definition: Let A be a matrix in M(n-by-n)(C). If A-star = -A, then A is called skew-Hermitian.
Theorem 11.5.5: Every skew-Hermitian matrix A is unitarily diagonalizable.
Before I give the proof, I will pause the video to give you time to think about the strategy we are going to use, which of course will be the same strategy we used to prove the Spectral Theorem for Hermitian Matrices. You should actually write down what you think the strategy for this proof will be—as detailed as possible—and then compare it with mine. In my experience, most students will have it not quite correct, as they will think too much about the steps we used in the proof of the Spectral Theorem for Hermitian Matrices, rather than the strategy of the proof.
The strategy I will use is first use Schur’s Theorem to get that A is unitarily similar to an upper triangular matrix T, then prove that T has the same property as A—that is, prove that T is also skew-Hermitian. Then, prove that every upper triangular skew-Hermitian matrix is diagonal. If the strategy you wrote down is exactly the same, then excellent job. If it is not the same, then as I mentioned before, that is not unusual, but it does mean that you should take some more time to try to understand the strategy being used.
Here we go with the proof. Proof: If A is skew-Hermitian, then A-star = -A. By Schur’s Theorem, there exists a unitary matrix U and upper triangular matrix T such that (U-star)AU = T. Then, (T-conjugate)-transpose = ((U-star)AU)-starred, which equals (U-star)(A-star)U, which is now (U-star)(-A)U, which is –((U-star)AU), which equals –T. Hence, T is also skew-Hermitian. Thus, T = –(T-star), so T is both upper and lower triangular, and hence diagonal.
What does this say about the eigenvalues of a skew-Hermitian matrix? We have diag(t11 through tnn) = T = –(T-star), which equals the diagonal matrix (–(t11-conjugate) up to –(tnn-conjugate)). Thus, tii = –(tii-conjugate), so all eigenvalues of a skew-Hermitian matrix are purely imaginary or 0.
Again, we will state this as a theorem. Theorem 11.5.6: If lambda is an eigenvalue of a skew-Hermitian matrix A, then lambda = ti for some real number t.
Although this doesn’t match the real case, this doesn’t seem so bad. Instead of just Hermitian matrices being diagonalizable, we have that Hermitian and skew-Hermitian matrices are diagonalizable.
Example: The matrix U = [i/(square root 2), 1/(square root 2); -i/(square root 2), 1/(square root 2)] is unitarily diagonalizable, but not Hermitian nor skew-Hermitian. Notice that U is, in fact, unitary.
Theorem 11.5.7: Every unitary matrix A is unitarily diagonalizable.
Again, I will pause to let you think about the strategy that we will use for this proof. You should write it down explicitly and compare with what I do. The strategy I will use is first use Schur’s Theorem to get that A is unitarily similar to an upper triangular matrix T, and then prove that T has the same property as A—that is, prove that T is also unitary. Then, prove that every upper triangular unitary matrix is diagonal.
Proof: If A is unitary, then A(A-star) = the identity matrix. By Schur’s Theorem, there exists a unitary matrix U and upper triangular matrix T such that (U-star)AU = T. Notice that since U-star, A, and U are all unitary, then T is also unitary by theorem from the last lecture. We now need to show that every upper triangular unitary matrix is diagonal. This will be a little trickier than the two previous proofs. Let T be the upper triangular matrix T = [t11, t12, to t1n; 0, t22, to t2n; all the way down to all zeroes to tnn]. Since T is unitary, the columns of T form an orthonormal basis for Cn. So, the first column must have length 1. That is, we have 1 = (the length of [t11 and all zeroes]), which is the absolute value of t11. Also, the columns are orthogonal, so we have 0 = (the inner product of the first column and the second column), which equals t11(the conjugate of t12). Since the absolute value of t11 equals 1, we know that t11 is not equal to 0, and therefore, we must have t12 = 0. Thus, the second column has the form [0; t22; and all zeroes]. So, repeating what we did above, now using the second and third columns, we get that the absolute value of t22 equals 1, and t13 and t23 both equal 0. Continuing in this way, we get that T is diagonal.
Note that this is not a totally formal proof. To be completely rigorous, instead of just saying “continuing in this way”, we should prove it by induction.
What have we proven about the eigenvalues of a unitary matrix? We have proven the following theorem. Theorem 11.5.8: If lambda is an eigenvalue of a unitary matrix A, then (the absolute value of lambda) = 1. Note, again, that this means the eigenvalue lambda can be any complex number on the unit circle in the complex plane.
So, all Hermitian, skew-Hermitian, and unitary matrices are unitarily diagonalizable. Are there more? Yes. The matrix A = [1, 0, 1; 1, 1, 0; 0, 1, 1] is also unitarily diagonalizable, but not Hermitian, skew-Hermitian, nor unitary. This strategy of guessing and checking which class of matrices is unitarily diagonalizable is obviously not a good approach. What approach would be better? Try to think about this before watching the next exciting lecture, where all will be revealed.
This ends this lecture.