Introductory Linear Algebra/Matrix inverses and determinants

Matrix inverses

Template:Colored em are analogous to the Template:Colored em (or reciprocal) in the number system. Template:Colored definition Template:Colored remark In the number system, the multiplicative inverse, if it exists, is unique. Indeed, the matrix inverse, if it exists, is also unique similarly, as shown in the following proposition. Template:Colored proposition

Proof. Suppose to the contrary, that Template:Colored em matrices $B$ and $C$ are both inverses of matrix $A$. Then, $AB=BA=AC=CA=I$ by definition of matrix inverse. If the matrix inverse of $A$ exists, we have $$AB=AC\iff A^{-1}AB=A^{-1}AC\iff IB=IC\iff B=C,$$ which causes a contradiction.

$◻$

Proof.

• (self-invertibility) since $A$ is invertible, $A{A}^{-1}=A^{-1}A=I$, and thus $A^{-1}$ is invertible, and its inverse is $A$
• (scalar multiplicativity) $(cA)(c^{-1}{A}^{-1})=(c{c}^{-1})(A{A}^{-1})=1(I)=I$, as desired
• ('reverse multiplicativity') $(AB)(B^{-1}{A}^{-1})=A(B{B}^{-1})A^{-1}=AI{A}^{-1}=A{A}^{-1}=I$, as desired
• (interchangibility of inverse and transpose) $(A^T)(A^{-1}{)}^T=(A^{-1}A{)}^T=I^T=I$, as desired

$◻$

Template:Colored remark Matrix inverse can be used to solve SLE, as follows: Template:Colored proposition

Proof. $$\begin{array}{cccc}& & A𝐱& =𝐛\\ & \iff & A^{-1}A𝐱& =A^{-1}𝐛\\ & \iff & I𝐱& =A^{-1}𝐛\\ & \iff & 𝐱& =A^{-1}𝐛\\ \end{array}$$

$◻$

Then, we will define the Template:Colored em, which is closely related to EROs, and is important for the proof of results related to EROs. Template:Colored definition Template:Colored remark Template:Colored example Template:Colored exercise Template:Colored proposition

Proof. Outline: $2\times 2$ case: e.g.

• type I ERO:$$\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\stackrel{{𝐫}_1\leftrightarrow {𝐫}_2}{\to }\left(\begin{array}{cc}c& d\\ a& b\end{array}\right)=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)=\left(\begin{array}{cc}0\times a+1\times c& 0\times b+1\times d\\ 1\times a+0\times c& 1\times b+0\times d\end{array}\right)$$
• type II ERO:

$$\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\stackrel{k{𝐫}_1\to {𝐫}_1}{\to }\left(\begin{array}{cc}ka& kb\\ c& d\end{array}\right)=\left(\begin{array}{cc}k& 0\\ 0& 1\end{array}\right)\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)=\left(\begin{array}{cc}k\times a+0\times c& k\times b+0\times d\\ 0\times a+k\times c& 0\times b+k\times d\end{array}\right)$$

• type III ERO:

$$\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\stackrel{k{𝐫}_1+{𝐫}_2\to {𝐫}_2}{\to }\left(\begin{array}{cc}a& b\\ c+ka& d+kb\end{array}\right)=\left(\begin{array}{cc}1& 0\\ k& 1\end{array}\right)\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)=\left(\begin{array}{cc}1\times a+k\times c& 1\times b+0\times d\\ k\times a+1\times c& k\times b+1\times d\end{array}\right)$$

$◻$

Proof. The reverse process of each ERO is an ERO of the same type. Let $E_1$ and $E_2$ be the elementary matrices corresponding to these two EROs (an ERO and its reverse process), which are of the same type. Then, $E_2{E}_1=I\iff E_1^{-1}=E_2$, as desired (since $I$ can be obtained from $I$ by performing an ERO and its reverse process together).

$◻$

Template:Colored remark Template:Colored example Template:Colored exercise Then, we will state a Template:Colored em of invertible matrix theorem, in which some results from the complete version of invertible matrix theorem are removed. Template:Colored theorem

Proof. To prove this, we may establish a cycle of implications, i.e. (i) $\Rightarrow$ (ii) $\Rightarrow$ (iii) $\Rightarrow$ (iv) $\Rightarrow$ (i), then, when we pick two arbitrary statements form the four statements, they are equivalent to each other, which means that the four statements are equivalent.

(i) $\Rightarrow$ (ii): it follows from the proposition about solving SLE, and $𝐱=A^{-1}\mathrm{𝟎}=\mathrm{𝟎}$

(ii) $\Rightarrow$ (iii): since the SLE has a unique solution, the RREF of the augmented matrix of the SLE $(A|\mathrm{𝟎})$ has a leading one in each of the first $n$ columns, but not the $(n+1)$st column, i.e. it is $(I_n|\mathrm{𝟎})$. It follows that the RREF of $A$ is $I_n$, since after arbitrary EROs, the rightmost zero column is still zero column.

(iii) $\Rightarrow$ (iv): since RREF of $A$ is $I_n$, and RREF of $A$ equals $E_k\cdots E_2{E}_{1}A$ for some elementary matrices $E_1,E_2,\dots ,E_k$, it follows that $E_k\cdots E_2{E}_{1}A=I_n$. By definition and general 'reverse multiplicativity' of matrix inverse, we have $$A=(E_k\cdots E_2{E}_1{)}^{-1}=E_1^{-1}{E}_2^{-1}\cdots E_k^{-1},$$

i.e. $A$ is a product of elementary matrices

(iv) $\Rightarrow$ (i): since $A$ is a product of elementary matrices and an elementary matrix is invertible, it follows that $A$ is invertible by general 'reverse multiplicativity' of matrix inverse.

$◻$

Template:Colored remark Template:Colored example Template:Colored exercise The following provides us a convenient and efficient way to find the inverse of a matrix. Template:Colored theorem

Proof. Outline: we can write $E_k\cdots E_1(A|I_n)=(I_n|B)$ for some elementary matrices $E_1,\dots ,E_k$, since $(I_n|B)$ is RREF of $(A|I_n)$ Then, it can be proved that $E_k\cdots E_{1}A=I_n$ and $E_k\cdots E_1{I}_n=B$. It follows that $$BA=(E_k\cdots \underset{E_1}{\underbrace{E_1{I}_n}})A=I_n,$$ and thus $B=A^{-1}$.

$◻$

Determinants

Then, we will discuss the Template:Colored em, which allows characterizing some properties of a Template:Colored em matrix. Template:Colored definition Template:Colored remark Template:Colored example For the formula of determinants of $3\times 3$ matrices, we have a useful mnemonic device for it, namely the Template:Colored em, as follows: Template:Colored proposition

Proof. It follows from the formula in the above example.

$◻$

Then, we will given an example about computing the determinant of a $4\times 4$ matrix, which cannot be computed by the Rule of Sarrus directly. Template:Colored example Template:Colored proposition

Proof.

• $\det O=0\cdot c_{11}+0\cdot c_{12}+\cdots +0\cdot c_{1n}$
• $\det I_n=1\cdot c_{11}+0\cdot c_{12}+\cdots +0\cdot c_{1n}=c_{1}1=\det I_{n-1}$ (since the submatrix obtained after removing the 1st row and 1st column of $I_n$ is $I_{n-1}$)

• so, inductively, $\det I_n=\det I_{n-1}=\cdots =\underset{\text{by definition}}{\underbrace{\det I_1=1}}$

$◻$

Indeed, we can compute a determinant by the Template:Colored em along an arbitrary row, as in the following theorem. Template:Colored theorem Template:Colored remark Its proof (for the general case) is complicated, and thus is skipped. Template:Colored example Template:Colored exercise Then, we will discuss several properties of determinants that ease its computation. Template:Colored proposition

Proof. Outline:

• (type I ERO) e.g.

$$\begin{array}{cc}\left|\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|& =a_{11}\left|\begin{array}{cc}{a}_{22}& a_{23}\\ a_{32}& a_{33}\end{array}\right|-a_{12}\left|\begin{array}{cc}{a}_{21}& a_{23}\\ a_{31}& a_{33}\end{array}\right|+a_{13}\left|\begin{array}{cc}{a}_{21}& a_{22}\\ a_{31}& a_{32}\end{array}\right|\\ \left|\begin{array}{ccc}{a}_{21}& a_{22}& a_{23}\\ a_{11}& a_{12}& a_{13}\\ a_{31}& a_{32}& a_{33}\end{array}\right|& =-a_{11}\left|\begin{array}{cc}{a}_{22}& a_{23}\\ a_{32}& a_{33}\end{array}\right|+a_{12}\left|\begin{array}{cc}{a}_{21}& a_{23}\\ a_{31}& a_{33}\end{array}\right|-a_{13}\left|\begin{array}{cc}{a}_{21}& a_{22}\\ a_{31}& a_{32}\end{array}\right|=-\left|\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|\end{array}$$

• (type II ERO) e.g.

$$\left|\begin{array}{ccc}k{a}_{11}& k{a}_{12}& k{a}_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|=k{a}_{11}\left|\begin{array}{cc}{a}_{22}& a_{23}\\ a_{32}& a_{33}\end{array}\right|-k{a}_{12}\left|\begin{array}{cc}{a}_{21}& a_{23}\\ a_{31}& a_{33}\end{array}\right|+k{a}_{13}\left|\begin{array}{cc}{a}_{21}& a_{22}\\ a_{31}& a_{32}\end{array}\right|=k(a_{11}\left|\begin{array}{cc}{a}_{22}& a_{23}\\ a_{32}& a_{33}\end{array}\right|-a_{12}\left|\begin{array}{cc}{a}_{21}& a_{23}\\ a_{31}& a_{33}\end{array}\right|+a_{13}\left|\begin{array}{cc}{a}_{21}& a_{22}\\ a_{31}& a_{32}\end{array}\right|)=k\left|\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|$$

• (type III ERO) e.g.

$$\begin{array}{cc}\left|\begin{array}{ccc}{a}_{11}+k{a}_{21}& a_{11}+k{a}_{22}& a_{11}+k{a}_{22}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|& =(a_{11}+k{a}_{21})\left|\begin{array}{cc}{a}_{22}& a_{23}\\ a_{32}& a_{33}\end{array}\right|-(a_{12}+k{a}_{22})\left|\begin{array}{cc}{a}_{21}& a_{23}\\ a_{31}& a_{33}\end{array}\right|+(a_{13}+k{a}_{23})\left|\begin{array}{cc}{a}_{21}& a_{22}\\ a_{31}& a_{32}\end{array}\right|\\ & =\underset{\left|\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|}{\underbrace{a_{11}\left|\begin{array}{cc}{a}_{22}& a_{23}\\ a_{32}& a_{33}\end{array}\right|-a_{12}\left|\begin{array}{cc}{a}_{21}& a_{23}\\ a_{31}& a_{33}\end{array}\right|+a_{13}\left|\begin{array}{cc}{a}_{21}& a_{22}\\ a_{31}& a_{32}\end{array}\right|}}+\underset{\left|\begin{array}{ccc}k{a}_{21}& k{a}_{22}& k{a}_{23}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|}{\underbrace{k{a}_{21}\left|\begin{array}{cc}{a}_{22}& a_{23}\\ a_{32}& a_{33}\end{array}\right|-k{a}_{22}\left|\begin{array}{cc}{a}_{21}& a_{23}\\ a_{31}& a_{33}\end{array}\right|+k{a}_{23}\left|\begin{array}{cc}{a}_{21}& a_{22}\\ a_{31}& a_{32}\end{array}\right|}}\\ & =\left|\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|+k\underset{0}{\underbrace{\left|\begin{array}{ccc}{a}_{21}& a_{22}& a_{23}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|}}\\ & =\left|\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right|\end{array}$$

$◻$

Proof. Let $A$ be a square matrix with two identical rows. If we interchange the two identical rows in $A$, the matrix is still the same, but its determinant is multiplied by $-1$, i.e. $$\det A=-\det A\iff 2\det A=0\iff \det A=0.$$ Alternatively, it can be proved by definition and induction.

$◻$

Template:Colored exercise Then, we will introduce a convenient way to determine invertibility of a matrix. Before introducing the theorem, we have a lemma. Template:Colored lemma

Proof.

• (type I: ${𝐫}_i\leftrightarrow {𝐫}_j$) $\det E=-1$ and $\det (EA)=-\det A=(\det E)(\det A)$ (since we are interchanging rows)
• (type II: $k{𝐫}_i\to {𝐫}_i$) $\det E=k$ and $\det (EA)=k\det A=(\det E)(\det A)$ (since we are multiplying a row by nonzero constant)
• (type III: $k{𝐫}_j+{𝐫}_i\to {𝐫}_i$) $\det E=1$ and $\det (EA)=\det A=(\det E)(\det A)$ (since we are adding a multiple of a row to another row)

$◻$

Proof.

• only if part: by simplified invertible matrix theorem, a matrix $A$ is invertible is equivalent to $A$ is product of elementary matrices. So, if we denote the elementary matrices by $E_1,\dots ,E_k$, then

$$\begin{array}{cccc}& & A& =E_1{E}_2\cdots E_k\\ & \Rightarrow & \det A& =(\det E_1)\det (\underset{\text{a matrix}}{\underbrace{E_2{E}_3\cdots E_k}})(\text{not }\iff \text{ since determinant function is many-to-one})\\ & & & =(\det E_1)\det (E_2)\det (E_3\cdots E_k)\\ & & & =\cdots \\ & & & =(\det E_1)\det (E_2)\cdots (\det E_k)\end{array}$$

• if part: Let $A=E_1\cdots E_{k}R$ in which $E_1,\dots ,E_k$ are elementary matrices and $R$ is the RREF of $A$. This Template:Colored em that

$$\det A=(\det E_1)\cdots (\det E_k)(\det R).$$

Since $\det A\ne 0$, so $\det R\ne 0$. Thus, $R$ has no zero row (its determinant is zero otherwise). Since $R$ is in RREF, it follows that $R=I$ (since $R$ is square matrix, if not all columns contain leading ones, then there is at least one zero row lying at its bottom, by definition of RREF). By simplified invertible matrix theorem, $A$ is invertible.

$◻$

After introducing this result, we will give some properties of determinants which can ease the computation of determinants. Template:Colored proposition

Proof.

• (multiplicativity) let $A=E_1\cdots E_{k}R$ in which $E_1,\dots ,E_k$ are elementary matrices and $R$ is the RREF of $A$. Then,

$$\det (AB)=(E_1\cdots E_{k}RB)=(\det E_1)\cdots (\det E_k)\det (RB),$$

and

$$(\det A)(\det B)=(\det E_1)\cdots (\det E_k)(\det R)(\det B).$$

• then, it remains to prove that $\det (RB)=(\det R)(\det B)$
• if $R=I$, then $\det (RB)=\det B=(\det R)(\det B)$
• if $R\ne I$, then the last row of $R$ is a zero row, so $\det R=0=(\underset{0}{\underbrace{\det R}})(\det B)$

• the last row of $RB$ is also a zero row, so $\det (RB)=0=(\det R)(\det B)$

• the result follows

• (invariance of determinant after transpose) we may prove it by induction and cofactor expansion theorem, e.g. $\left|\begin{array}{ccc}1& 2& 3\\ 4& 5& 6\\ 7& 8& 9\end{array}\right|$ vs. $\left|\begin{array}{ccc}1& 4& 7\\ 2& 5& 8\\ 3& 6& 9\end{array}\right|$
• (determinant of matrix inverse is inverse of matrix determinant) using multiplicativity,

$$A{A}^{-1}=I⟹(\det A)(\det (A^{-1}))=\det I=1⟹\det (A^{-1})=(\det A{)}^{-1}$$ ($\det A\ne 0$ since $A$ is invertible)

$◻$

Template:Colored example Template:Colored exercise Then, we will introduce Template:Colored em of matrix, which has a notable result related to computation of matrix inverse. Template:Colored definition Template:Colored remark Template:Colored theorem

Proof. The proof is complicated, and so is skipped.

$◻$

Proof. $$A(\mathrm{adj}A)=(\det A)I_n\iff A(\frac{1}{\det A}\mathrm{adj}A)=I_n\iff A^{-1}=\frac{1}{\det A}\mathrm{adj}A$$

$◻$

Template:Colored example Template:Colored example Template:Colored exercise Then, we will introduce a result that allows us to compute the unique solution of SLE directly, namely Template:Colored em. Template:Colored theorem

Proof. Since $A$ is invertible, the unique solution of the SLE is $𝐱=A^{-1}𝐛$. Using the formula of matrix inverse, we have $$A^{-1}𝐛=\frac{1}{\mathrm{\Delta }}(\mathrm{adj}A)𝐛.$$ Thus, for each $k\in \{1,2,\dots ,n\}$, $$x_k=\frac{1}{\mathrm{\Delta }}(c_{1k}{b}_1+c_{2k}{b}_2+\cdots +c_{nk}{b}_n)=\frac{{\mathrm{\Delta }}_k}{\mathrm{\Delta }}$$ ($c_{1k},c_{2k},\dots ,c_{nk}$ are entries at the $k$th row of $\mathrm{adj}A$ (and at the $k$th column of the cofactor matrix of $A$), so multiplying the entries as above gives the $(k,1)$-th entry of $𝐱$, namely $x_k$)

$◻$