Matrix Multiplication

Overview

Matrix multiplication is a fundamental operation that combines two matrices to produce a new matrix. Unlike addition, matrix multiplication has specific dimension requirements and is not commutative.

Definition

For an $m \times n$ matrix $A$ and an $n \times p$ matrix $B$, the product $AB$ is an $m \times p$ matrix where:

$(AB)_{ij} = \sum_{k=1}^{n} a_{ik}b_{kj} = a_{i1}b_{1j} + a_{i2}b_{2j} + \cdots + a_{in}b_{nj}$

Entry $(i,j)$ is the dot product of row $i$ of $A$ and column $j$ of $B$.

Dimension Requirements

$A_{m \times n} \times B_{n \times p} = C_{m \times p}$

• Inner dimensions must match: columns of $A$ = rows of $B$
• Result has: rows of $A$ × columns of $B$

Computation Method

$\begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \times \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{bmatrix} = \begin{bmatrix} a_{11}b_{11}+a_{12}b_{21} & a_{11}b_{12}+a_{12}b_{22} \\ a_{21}b_{11}+a_{22}b_{21} & a_{21}b_{12}+a_{22}b_{22} \end{bmatrix}$

Example

$A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}, \quad B = \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix}$

$AB = \begin{bmatrix} 1 \times 5 + 2 \times 7 & 1 \times 6 + 2 \times 8 \\ 3 \times 5 + 4 \times 7 & 3 \times 6 + 4 \times 8 \end{bmatrix} = \begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix}$

Properties of Matrix Multiplication

Property	Formula
Associative	$(AB)C = A(BC)$
Left Distributive	$A(B + C) = AB + AC$
Right Distributive	$(A + B)C = AC + BC$
Scalar	$c(AB) = (cA)B = A(cB)$
Identity	$AI = IA = A$

Non-Commutativity

Important: In general, $AB \neq BA$

Example

$A = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}, \quad B = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}$

$AB = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}, \quad BA = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix}$

Zero Divisors

$AB = O$ does not imply $A = O$ or $B = O$.

Example

$\begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} \times \begin{bmatrix} 0 & 0 \\ 1 & 0 \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix}$

Powers of Matrices

For square matrix $A$:

$A^2 = AA$

$A^3 = AAA = A^2 A$

$A^n = \underbrace{A \times A \times \cdots \times A}_{n \text{ times}}$

$A^0 = I$

Matrix-Vector Multiplication

For matrix $A$ and vector $\mathbf{x}$:

$A\mathbf{x} = x_1(\text{column 1}) + x_2(\text{column 2}) + \cdots + x_n(\text{column } n)$

Example

$\begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} x_1 + 2x_2 + 3x_3 \\ 4x_1 + 5x_2 + 6x_3 \end{bmatrix}$

Row-Column Interpretation

Row Picture

Each row of $AB$ is a linear combination of rows of $B$.

Column Picture

Each column of $AB$ is a linear combination of columns of $A$:

$AB = \begin{bmatrix} A\mathbf{b}_1 & A\mathbf{b}_2 & \cdots & A\mathbf{b}_p \end{bmatrix}$

Block Matrix Multiplication

Matrices can be partitioned into blocks:

$\begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix} \begin{bmatrix} B_{11} & B_{12} \\ B_{21} & B_{22} \end{bmatrix} = \begin{bmatrix} A_{11}B_{11}+A_{12}B_{21} & A_{11}B_{12}+A_{12}B_{22} \\ A_{21}B_{11}+A_{22}B_{21} & A_{21}B_{12}+A_{22}B_{22} \end{bmatrix}$

Worked Example

Compute $AB$ where:

$A = \begin{bmatrix} 1 & 2 & 0 \\ 3 & 1 & 2 \end{bmatrix}, \quad B = \begin{bmatrix} 1 & 4 \\ 2 & 3 \\ 0 & 1 \end{bmatrix}$

$A$ is $2 \times 3$, $B$ is $3 \times 2$, so $AB$ is $2 \times 2$:

$AB = \begin{bmatrix} 1 \times 1 + 2 \times 2 + 0 \times 0 & 1 \times 4 + 2 \times 3 + 0 \times 1 \\ 3 \times 1 + 1 \times 2 + 2 \times 0 & 3 \times 4 + 1 \times 3 + 2 \times 1 \end{bmatrix} = \begin{bmatrix} 5 & 10 \\ 5 & 17 \end{bmatrix}$