Matrix Products, Inverses and LU

Shapes determine valid products

For $A$ of size $m\times n$ and $B$ of size $n\times p$ , the product $AB$ has size $m\times p$ , with entry $(AB)_{ij}=\sum_kA_{ik}B_{kj}$ . Its action is $A(B\mathbf x)$ : the rightmost transformation acts first. Generally $AB\ne BA$ , even when both products exist.

A square matrix has an inverse when some $A^{-1}$ satisfies $A^{-1}A=AA^{-1}=I$ . Then $A\mathbf x=\mathbf b$ has unique solution $A^{-1}\mathbf b$ for every target. For actual numerical solving, elimination or a factorization usually avoids forming the inverse explicitly.

Visual guide

VISUAL GUIDEMatrix columns are images of the basis

For A = [[2, 1], [0, 1]], the unit square becomes a parallelogram. Its edges are the columns (2, 0) and (1, 1). Multiplying a vector forms the same weighted combination of those columns.

Worked example: order matters

Let $A=\begin{pmatrix}2&0\\0&1\end{pmatrix}$ and $B=\begin{pmatrix}1&1\\0&1\end{pmatrix}$ . Then $AB=\begin{pmatrix}2&2\\0&1\end{pmatrix}$ but $BA=\begin{pmatrix}2&1\\0&1\end{pmatrix}$ . Scaling after shearing differs from shearing after scaling.

For a $2\times2$ matrix $\begin{pmatrix}a&b\\c&d\end{pmatrix}$ , the inverse is $(ad-bc)^{-1}\begin{pmatrix}d&-b\\-c&a\end{pmatrix}$ when $ad-bc\ne0$ . Multiplying verifies the identity; zero determinant makes this formula and an inverse impossible.

Worked example: factor once, solve repeatedly

For $A=\begin{pmatrix}2&1\\4&3\end{pmatrix}$ , elimination subtracts twice row one from row two. Hence

A=LU=\begin{pmatrix}1&0\\2&1\end{pmatrix}\begin{pmatrix}2&1\\0&1\end{pmatrix}.

To solve $A\mathbf x=(3,7)^T$ , first solve $L\mathbf y=\mathbf b$ , obtaining $\mathbf y=(3,1)^T$ , then $U\mathbf x=\mathbf y$ , giving $\mathbf x=(1,1)^T$ . The same factors can solve many new right-hand sides. If elimination needs row swaps, the usual form is $PA=LU$ with a permutation matrix $P$ .

Practice

What is the shape of a $3\times2$ matrix times a $2\times4$ matrix?
Invert $\operatorname{diag}(2,5)$ .
Why does $\begin{pmatrix}1&2\\2&4\end{pmatrix}$ have no inverse?

Show worked solutions

$3\times4$ .
$\operatorname{diag}(1/2,1/5)$ .
Its columns are dependent and its determinant is zero. Some targets are unreachable and others have multiple preimages.