Eigenvalues, Eigenvectors, and Diagonalization

Part 1, Chapter 1: Linear Algebra Toolkit

Learning objectives

Define eigenvalue and eigenvector via $A \mathbf{v} = \lambda \mathbf{v}$
Compute eigenvalues by solving the characteristic equation $\det(A - \lambda I) = 0$
Find eigenvectors by row-reducing $(A - \lambda I) \mathbf{x} = \mathbf{0}$
Connect eigenvalues to trace, determinant, and the geometry of the transformation

Eigenvalues are the intrinsic stretching factors of a linear transformation, the directions in which the matrix acts as a pure scaling. They are the most powerful concept in finite-dimensional linear algebra. They control the long-run behaviour of dynamical systems, the stability of solutions to differential equations, the principal components of high-dimensional data, and the energy levels of quantum systems. If you take only one thing from this chapter, take the geometric picture: an eigenvector is a direction the matrix preserves; its eigenvalue is the scaling factor along that direction.

The definition

A non-zero vector $\mathbf{v}$ is an eigenvector of a square matrix $A$ with eigenvalue $\lambda$ if $A \mathbf{v} = \lambda \mathbf{v}$ . Geometrically, $A$ stretches $\mathbf{v}$ by the factor $\lambda$ without rotating it (though it may flip when $\lambda < 0$ ). The zero vector is excluded by convention, otherwise every scalar would qualify as an eigenvalue.

Finding eigenvalues: the characteristic equation

Rearrange $A \mathbf{v} = \lambda \mathbf{v}$ as $(A - \lambda I) \mathbf{v} = \mathbf{0}$ . For a non-zero $\mathbf{v}$ to exist, the matrix $A - \lambda I$ must be singular, equivalently, $\det(A - \lambda I) = 0$ . This is the characteristic equation of $A$ . For a $2 \times 2$ matrix, it reduces to $\lambda^2 - \operatorname{tr}(A) \lambda + \det(A) = 0$ .

Finding eigenvectors

Once you have an eigenvalue $\lambda_i$ i, solve $(A - \lambda_i I) \mathbf{x} = \mathbf{0}$ iI)mathbfx=mathbf0 by row reduction. The non-trivial solutions form the eigenspace $E_{\lambda_i}$ . Its dimension is the geometric multiplicity of $\lambda_i$ i. The algebraic multiplicity is how many times $\lambda_i$ i appears as a root of the characteristic polynomial. The two coincide for diagonalisable matrices; the gap measures the failure of diagonalisation.

Two beautiful identities

For any $n \times n$ matrix with eigenvalues $\lambda_1, \ldots, \lambda_n$ (counted with algebraic multiplicity):

$\operatorname{tr}(A) = \lambda_1 + \lambda_2 + \cdots + \lambda_n$ , the trace is the sum of eigenvalues.
$\det(A) = \lambda_1 \lambda_2 \cdots \lambda_n$ , the determinant is their product.

The matrix-multiplier above lets you see eigenvectors directly: enter a matrix and look for the directions that get scaled without rotating. For $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ , the eigenvalues are $\lambda = 2, 5$ , with eigenvectors $(1, -2)$ and $(1, 1)$ respectively, you can verify by feeding these vectors through the widget.

Where this shows up

PageRank algorithm: Google's original algorithm computes the dominant eigenvector of a stochastic matrix encoding the web's link structure. Page importance is literally the entries of that eigenvector, the most-cited eigenvalue computation in industry.
Principal Component Analysis (PCA): The principal components of a data set are the eigenvectors of the covariance matrix, ordered by their eigenvalues. Larger eigenvalues = directions with more variance = features that explain the data.
Quantum mechanics, observables and measurement: Every observable (energy, position, spin) is a Hermitian operator. Its eigenvalues are the possible measurement outcomes; its eigenvectors are the corresponding states. Spectroscopy data are literal eigenvalue plots.

Pause and think: If $A \mathbf{v} = 4 \mathbf{v}$ , what is $A^{10} \mathbf{v}$ ? Before computing, predict by powering up the eigenvalue. This is exactly the reason eigenvector decomposition makes matrix powers tractable.

Try it

Predict the eigenvalues of $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ by computing trace and determinant first. Then solve $\lambda^2 - 7 \lambda + 10 = 0$ .
For each eigenvalue you found, predict the eigenvector by inspecting $A - \lambda I$ . Then verify by computing $A \mathbf{v}$ .
A $3 \times 3$ matrix has eigenvalues $1, 2, 3$ . Predict the trace and determinant. Justify with the two beautiful identities.
If $A \mathbf{v} = 4 \mathbf{v}$ , predict $A^3 \mathbf{v}$ before computing. The answer is $4^3 \mathbf{v} = 64 \mathbf{v}$ .
Predict: is $\lambda = 0$ ever an eigenvalue of an invertible matrix? Justify in one sentence using the determinant identity.

A trap to watch for

Eigenvectors are defined up to a non-zero scalar multiple, $(1, -2)$ and $(-5, 10)$ are the same eigenvector in disguise (both lie on the same line through the origin). Students sometimes panic when their textbook lists $(1, -2)$ but they computed $(-1, 2)$ ; both are correct. The eigenspace is a subspace, not a unique vector. Conversely, the zero vector is NEVER an eigenvector (by convention), if you got $\mathbf{v} = \mathbf{0}$ from row reduction, you probably picked a non-eigenvalue $\lambda$ , because $A - \lambda I$ was invertible and forced $\mathbf{v} = \mathbf{0}$ .

What you now know

You can compute eigenvalues from the characteristic equation, find eigenvectors by row reduction, and connect both back to trace and determinant. With eigenvalues in hand, the diagonalisation machinery of §1.7 becomes concrete: count independent eigenvectors, build $P$ , and the diagonal form falls out. This closes Chapter 1: the linear-algebra toolkit that every subsequent chapter of Garrity rests on.

Mark section complete →

References

Garrity, T. (2002). All the Mathematics You Missed. Cambridge University Press, ch. 1, §1.8.
Strang, G. (2016). Introduction to Linear Algebra (5th ed.). Wellesley-Cambridge, ch. 6.
Axler, S. (2015). Linear Algebra Done Right (3rd ed.). Springer, ch. 5 (eigenvalues, eigenvectors, invariant subspaces).
Horn, R. A., Johnson, C. R. (2012). Matrix Analysis (2nd ed.). Cambridge University Press, ch. 1 (spectral theory).
Trefethen, L. N., Bau, D. (1997). Numerical Linear Algebra. SIAM, lec. 24-25 (eigenvalue algorithms).