\documentclass{amsart}
\newcommand{\F}{\mathbb F}
\newcommand{\vv}{\vec{v}}
\newcommand{\vu}{\vec{u}}
\renewcommand{\L}{\mathcal L}
\newcommand{\R}{\mathbb R}
\newcommand{\C}{\mathbb C}
\newcommand{\bilin}[1]{\langle#1\rangle}

\theoremstyle{definition}
\newtheorem{defn}{Definition}
\theoremstyle{plain}
\newtheorem{thm}{Theorem}
\newtheorem{prop}{Proposition}
\newtheorem{lem}{Lemma}
\newtheorem{cor}{Corollary}

\title{Lecture 21: Inner Products}

\begin{document}
\maketitle

For the remaining lectures, we will work exclusively over $\R$ or $\C$. We'll also look at a very special kind of bilinear form.

\begin{defn}
An \underline{inner product} is a function $\bilin{-,-}\colon V\times V\to\F$ such that
\begin{enumerate}
\item $\bilin{\vv,\vv}\geq 0$ with equality iff $\vv=\vec{0}$.
\item $\bilin{\vv,\vu}=\begin{cases}
\bilin{\vu,\vv} & \F=\R \\
\overline{\bilin{\vu,\vv}} & \F=\C.
\end{cases}$
\item $\bilin{a\vv+b\vu,\vec{w}}=a\bilin{\vv,\vec{w}}+b\bilin{\vu,\vec{w}}$.
\end{enumerate}
\end{defn}

The first condition is that the inner product is ``positive definite'', the second is that it is (conjugate) symmetric, and the third is that it is linear in the first factor.

\begin{prop}
If $\bilin{-,-}$ is an inner product, then
\[
\bilin{\vv,a\vu+b\vec{w}}=\bar{a}\bilin{\vv,\vu}+\bar{b}\bilin{\vv,\vec{w}},
\]
and so if $\F=\R$, then an inner product is bilinear.
\end{prop}
\begin{proof}
Use conjugate symmetry to swap the two coordinates, then use linearity in the first and conjugate symmetry again.
\end{proof}

Over $\C$, this is not bilinearity. The inner product is additive in the second factor, but it has a conjugation that appears in the scalar multiplication. In this case, we call it \underline{sesquilinear} (the prefix ``sesqu-'' means ``one and a half'': it's linear in the first factor and half linear in the second). Of course, we understand complex conjugation completely, so sesquilinearity is as good as bilinearity.

Now some examples.

\begin{enumerate}
\item $\R^n$ with the dot product. This is the prototype of all inner product spaces.
\item $\C^n$ with the standard inner product:
\[
\bilin{(a_1,\dots,a_n),(b_1,\dots,b_n)}=a_1\bar{b}_1+\dots+a_n\bar{b}_n.
\]
\item The space $\ell_2=\{(a_0,\dots) | \sum |a_i|^2<\infty\}$ with the inner product
    \[
\bilin{(a_1,\dots),(b_1,\dots)}=\sum a_i\bar{b}_i.
\]
This is the [countably] infinite dimensional analogue of the previous ones. 
\item $C[a,b]$, the set of continuous functions on $[a,b]$ with inner product
    \[
    \bilin{f,g}=\int_{a}^bf(x)\overline{g(x)}w(x)dx,
    \]
    where $w(x)>0$ for all $x\in [a,b]$. These spaces are essentially all the same (and all the same as $\ell_2$, though this is all too analytical to be fun).
\item If $W$ is a subspace of an inner product space $V$, then the inner product of $V$ restricted to $W$ gives an inner product on $W$.
\end{enumerate}

There are several very important consequences of the existence of an inner product. The first is a notion of distance.

\begin{defn}
The \underline{norm} of $\vv$ is
\[
||\vv||=\sqrt{\bilin{\vv,\vv}}.
\]
\end{defn}

Since the inner product is positive definite, this is always greater than or equal to zero, and moreover, we know that it equals zero iff $\vv=\vec{0}$. We know from sesquilinearity that
\[
||a\vv||=|a|\cdot ||\vv||.
\]
This is one of the nice features of sesquilinearity.

We have two more nice relations: the Cauchy-Schwarz inequality (which will link the inner product to a notion of angles) and the triangle inequality (which will ensure that our norm gives a good notion of distance).

\begin{prop}[Cauchy-Schwarz]
For all $\vv$ and $\vu$ in an inner product space $V$, we have
\[
|\bilin{\vv,\vu}|\leq ||\vv||\cdot ||\vu||.
\]
\end{prop}

\begin{proof}
We first note that if $\vv$ or $\vu$ is zero, then the result is obvious. We therefore assume that neither is zero.

We will consider the square of the norm of $\vv-r\vu$ for various $r$ and then deduce the desired result from this.
\begin{multline*}
||\vv-r\vu||^2=\bilin{\vv-r\vu,\vv-r\vu}=
||\vv||^2-r\bilin{\vu,\vv}-\bar{r}(\bilin{\vv,\vu}-r||\vu||^2)\geq 0.
\end{multline*}
This holds for all $r$, so in particular, it holds for the value of $r$ that simplifies the expression: $r=\bilin{\vv,\vu}/||\vu||^2$. With this, our previous expression becomes
\[
0\leq ||\vv||^2-\frac{\bilin{\vv,\vu}}{||\vu||^2}\bilin{\vu,\vv}=||\vv||^2-\frac{|\bilin{\vv,\vu}|^2}{||\vu||^2},
\]
by sesquilinearity. Rearranging gives the result.
\end{proof}

What does this buy us? Let's say that $\vv$ is a \underline{unit vector} if $||\vv||=1$. If $||\vv||\neq 0$, then we can always divide $\vv$ by its length to get a unit vector. Thus we can restate the previous result as
\begin{prop}
If $\vv$ and $\vu$ are unit vectors, then 
\[
|\bilin{\vv,\vu}|\leq 1.
\]
\end{prop}
Thus we can find a unique $\theta\in [0,\pi]$ such that 
\[
\cos(\theta)=\bilin{\vv,\vu}.
\]

\begin{defn}
The angle between $\vv$ and $\vu$ is that $\theta$ such that
\[
\bilin{\vv,\vu}=||\vv|| ||\vu||\cos(\theta).
\]
\end{defn}

With this, we can get a refined triangle inequality.

\begin{prop}[Triangle Inequality]
For all $\vv$ and $\vu$, we have
\[
||\vv+\vu||\leq ||\vv||+||\vu||.
\]
\end{prop}

\begin{proof}
We show the square of the inequality.
\[
||\vv+\vu||^2=\bilin{\vv+\vu,\vv+\vu}=||\vv||^2+||\vu||^2+\bilin{\vu,\vv}+\bilin{\vv,\vu}.
\]
Since $\bilin{\vv,\vu}+\bilin{\vu,\vv}$ is twice the real part of $\bilin{\vv,\vu}$, we know
\[
\bilin{\vv,\vu}+\bilin{\vu,\vv}\leq 2 |\bilin{\vv,\vu}|\leq 2 ||\vv||\cdot ||\vu||.
\]
This gives
\[
||\vv+\vu||^2\leq ||\vv||^2+2 ||\vv||\cdot ||\vu||+||\vu||^2=(||\vv||+||\vu||)^2.\qedhere
\]
\end{proof}

We can do better, if we are working over $\R$: we can rederive the law of cosines. Recalling that 
\[
\bilin{\vv,\vu}=\bilin{\vu,\vv},
\]
we can simplify the earlier formula and derive the following.

\begin{prop}
For any $\vu$ and $\vv$, we have
\[
||\vu-\vv||^2=||\vu||^2+||\vv||^2-2||\vu||\cdot ||\vv||\cos(\theta),
\]
where $\theta$ is the angle between $\vu$ and $\vv$.
\end{prop}

\end{document} 