Matrix Operations and Algebra

Part 18, Chapter 18: Matrices and Determinants

Learning objectives

Define matrices and describe their dimensions
Add matrices and multiply by scalars
Multiply two matrices of compatible dimensions
Identify the identity matrix

You already know how to solve a single equation. What about a system of fifty equations in fifty unknowns? The arithmetic is the same; the bookkeeping is what kills you. Mathematicians invented the matrix, a grid of numbers, precisely so the bookkeeping could be done once and for all by a single set of rules. Once you have those rules, the same algebra that handled $2x = 6$ scales up to systems of any size, to the transformations of geometry, to the data of statistics.

Definition

A matrix is a rectangular array of numbers. An $m \times n$ matrix has $m$ rows and $n$ columns; we denote the entry in row $i$ and column $j$ by $a_{ij}$ ij. For example,

$A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ is a $2 \times 2$ matrix with $a_{11} = 1,\ a_{12} = 2,\ a_{21} = 3,\ a_{22} = 4$ 2,a21=3,a22=4.

The first index always names the row; the second names the column. Read row-major (left-to-right, top-to-bottom) and the labels make sense.

Addition and scalar multiplication

Two matrices of the same shape add entry-by-entry: $(A + B)_{ij} = a_{ij} + b_{ij}$ ij=aij+bij. A scalar $c$ multiplies every entry: $(cA)_{ij} = c \cdot a_{ij}$ . Both operations look exactly like componentwise vector arithmetic, because that is what they are.

Matrix multiplication, row meets column

This is the part beginners trip over. If $A$ is $m \times n$ and $B$ is $n \times p$ , then $AB$ is $m \times p$ , with

$(AB)_{ij} = \sum_{k=1}^{n} a_{ik} \, b_{kj}.$ ,bkj.

In words: the $(i, j)$ entry of $AB$ is the dot product of row $i$ of $A$ with column $j$ of $B$ . The "inner dimension" must match, columns of $A$ count equal to rows of $B$ , or the product is not defined.

Where this shows up

Machine Learning: A neural-network forward pass is a sequence of matrix-vector products $Wx + b$ followed by nonlinearities; transformer models compute thousands of these per token, and matrix multiplication is the bottleneck GPU vendors race to optimise.
Computer Graphics: A camera-view transformation in OpenGL is a 4x4 matrix; chaining multiple transforms (model x view x projection) is matrix multiplication, run once per vertex per frame.
Search Engines: A transition matrix tells you the probability of going from state $i$ to state $j$ ; multiplying the matrix by itself $n$ times gives the $n$ -step distribution, Google's original PageRank algorithm is exactly this iteration.

(Hover any cell of the product matrix on the right. Row $i$ of $A$ lights up green, column $j$ of $B$ lights up orange, and the dot product that produces that cell appears beneath the grid. Edit any entry; the product updates live.)

What the rules buy you

Matrix multiplication is

associative: $A(BC) = (AB)C$ ,
distributive: $A(B + C) = AB + AC$ ,
but not commutative: $AB \neq BA$ in general.

The failure of commutativity is the single most important fact about matrices. It means matrices are not numbers in disguise, they are operators, and the order in which you apply operations changes the result. Try the Swap button in the widget: compute $AB$ , then $BA$ . They almost never agree.

The identity matrix

The $n \times n$ identity matrix $I_n$ n has $1$ on the main diagonal and $0$ everywhere else:

$I_2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \qquad I_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}.$

It does for matrices what $1$ does for numbers: $AI = IA = A$ for any compatible $A$ . This makes $I$ the right base case to test new matrix identities against.

Try it

Predict first: with $A = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$ and $B = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$ , compute $AB$ and $BA$ by hand, which is zero? Enter both matrices in the widget and verify (and note $AB \neq BA$ ).
Predict first: if $B$ is the identity matrix, what should $AB$ equal? Enter $B = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$ in the widget and verify $AB = A$ exactly.
On paper, multiply $\begin{pmatrix} 2 & 3 \\ -1 & 4 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 2 & 5 \end{pmatrix}$ and verify in the widget.
Why is it impossible to multiply a $2 \times 3$ matrix by a $2 \times 2$ matrix on the right? Write the inner-dimension condition.

Pause: matrix addition is commutative ( $A + B = B + A$ ) but matrix multiplication is not. Why? Look at what each operation does: addition combines two grids of numbers entry-by-entry, multiplication combines operators in sequence.

A trap to watch for

The classic trap is to assume $AB = BA$ . Even if both products are defined, they almost never agree, and they may have different shapes. Example: $A$ is $1 \times 3$ and $B$ is $3 \times 1$ . Then $AB$ is $1 \times 1$ (a scalar) and $BA$ is $3 \times 3$ (a matrix). The two products are not the same object, they live in different shape-spaces.

A second trap: writing $(A + B)^2 = A^2 + 2AB + B^2$ . Wrong. Expand the left side: $(A + B)(A + B) = A^2 + AB + BA + B^2$ . The middle terms collapse to $2AB$ only if $AB = BA$ . For general matrices, you must keep both $AB$ and $BA$ separately. The binomial theorem from high school is a special property of commutative arithmetic; it does not survive the move to matrices.

What you now know

You can read a matrix, add two of compatible shape, multiply by a scalar, and compute the product of two matrices via the row-times-column rule. You understand why the order $AB$ vs $BA$ matters, and you have the identity matrix $I$ to anchor your intuition. The next section attaches a single number, the determinant, to every square matrix and shows what it really means geometrically.

Quick check

Mark section complete →

References

Lang, S. (1971). Basic Mathematics. Springer. Chapter 17, §1, the original presentation of matrices and the product rule.
Strang, G. (2016). Introduction to Linear Algebra (5th ed.). Wellesley-Cambridge Press. Chapters 1–2, matrices, columns as vectors, and the row-column dot product worked from scratch.
Axler, S. (2015). Linear Algebra Done Right (3rd ed.). Springer. §3.C, matrices as representations of linear maps and how the product rule falls out.
Hoffman, K.; Kunze, R. (1971). Linear Algebra (2nd ed.). Prentice-Hall. Chapter 1: matrix algebra with proofs for every identity used above.