Linear Algebra/Inner product spaces

Recall that in your study of vectors, we looked at an operation known as the dot product, and that if we have two vectors in ${ℝ}^n$, we simply multiply the components together and sum them up. With the dot product, it becomes possible to introduce important new ideas like length and angle. The length of a vector $𝐚$, is just $\Vert 𝐚\Vert =\sqrt{𝐚\cdot 𝐚}$. The angle between two vectors, $𝐚$ and $𝐛$, is related to the dot product by

$\cos \theta =\frac{𝐚\cdot 𝐛}{\Vert 𝐚\Vert \cdot \Vert 𝐛\Vert }$

It turns out that only a few properties of the dot product are necessary to define similar ideas in vector spaces other than ${ℝ}^n$, such as the spaces of $m\times n$ matrices, or polynomials. The more general operation that will take the place of the dot product in these other spaces is called the "inner product".

The inner product

Say we have two vectors:

$𝐚=\left(\begin{array}{c}2\\ 1\\ 4\end{array}\right),𝐛=\left(\begin{array}{c}6\\ 3\\ 0\end{array}\right)$

If we want to take their dot product, we would work as follows

$𝐚\cdot 𝐛=a_1{b}_1+a_2{b}_2+a_3{b}_3=(2)(6)+(1)(3)+(4)(0)=15$

Because in this case multiplication is commutative, we then have $𝐚\cdot 𝐛=𝐛\cdot 𝐚$.

But then, we observe

$𝐯\cdot (\alpha 𝐚+\beta 𝐛)=\alpha 𝐯\cdot 𝐚+\beta 𝐯\cdot 𝐛$

much like the regular algebraic equality $v(aA+bB)=avA+bvB$. For regular dot products this is true since, for ${ℝ}^3$, for example, one can expand both sides out to obtain

$\begin{array}{c}(\alpha v_1{a}_1+\beta v_1{b}_1)+(\alpha v_2{a}_2+\beta v_2{b}_2)+(\alpha v_3{a}_3+\beta v_3{b}_3)=\\ (\alpha v_1{a}_1+\alpha v_2{a}_2+\alpha v_3{a}_3)+(\beta v_1{b}_1+\beta v_2{b}_2+\beta v_3{b}_3)\end{array}$

Finally, we can notice that $𝐯\cdot 𝐯$ is always positive or greater than zero - checking this for ${ℝ}^3$ gives this as

$𝐯\cdot 𝐯=v_1^2+v_2^2+v_3^2$

which can never be less than zero since a real number squared is positive. Note $𝐯\cdot 𝐯=0$ if and only if $𝐯=0$.

In generalizing this sort of behaviour, we want to keep these three behaviours. We can then move on to a definition of a generalization of the dot product, which we call the inner product. An inner product of two vectors in some vector space $V$, written $⟨𝐱,𝐲⟩$ is a function $V\times V\to ℝ$, which obeys the properties

• $⟨𝐱,𝐲⟩=⟨𝐲,𝐱⟩$
• $⟨𝐯,\alpha 𝐚+\beta 𝐛⟩=\alpha ⟨𝐯,𝐚⟩+\beta ⟨𝐯,𝐛⟩$
• $⟨𝐚,𝐚⟩\ge 0$ with equality if and only if $𝐚=0$.

The vector space $V$ and some inner product together are known as an inner product space.

The dot product in ${ℂ}^n$

Given two vectors $𝐚=a_1{\overrightarrow{e}}_1+a_2{\overrightarrow{e}}_2+\dots +a_n{\overrightarrow{e}}_n\in {ℂ}^n$ and $𝐛=b_1{\overrightarrow{e}}_1+b_2{\overrightarrow{e}}_2+\dots +b_n{\overrightarrow{e}}_n\in {ℂ}^n$, the dot product generalized to complex numbers is:

$𝐚\cdot 𝐛=\sum _{i=1}^n{a}_i^{*}{b}_i=a_1^{*}{b}_1+a_2^{*}{b}_2+\dots +a_n^{*}{b}_n$

where $z^{*}$ for an arbitrary complex number $z=c+di$ is the complex conjugate: $z^{*}=c-di$.

The dot product is "conjugate commutative": $𝐚\cdot 𝐛=(𝐛\cdot 𝐚{)}^{*}$. One immediate consequence of the definition of the dot product is that the dot product of a vector with itself is always a non-negative real number: $𝐚\cdot 𝐚\ge 0$.

$𝐚\cdot 𝐚=0$ if and only if $𝐚=\overrightarrow{0}$

The Cauchy-Schwarz Inequality for ${ℂ}^n$

In ${ℝ}^n$, the Cauchy-Schwarz inequality can be proven from the triangle inequality. Here, the Cauchy-Schwarz inequality will be proven algebraically.

To make the proof more intuitive, the algebraic proof for $𝐚,𝐛\in {ℝ}^n$ will be given first. Template:TextBox