\documentclass{amsart}
\newcommand{\F}{\mathbb F}
\newcommand{\vv}{\vec{v}}
\newcommand{\vu}{\vec{u}}
\newcommand{\vw}{\vec{w}}
\renewcommand{\L}{\mathcal L}
\newcommand{\R}{\mathbb R}
\newcommand{\C}{\mathbb C}
\newcommand{\bilin}[1]{\langle#1\rangle}

\theoremstyle{definition}
\newtheorem{defn}{Definition}
\theoremstyle{plain}
\newtheorem{thm}{Theorem}
\newtheorem{prop}{Proposition}
\newtheorem{lem}{Lemma}
\newtheorem{cor}{Corollary}

\title{Lecture 24: Orthogonality and Isometries}

\begin{document}
\maketitle

\section*{Orthogonality}
\begin{defn}
If $\vv$ and $\vu$ are vectors in an inner product space $V$, then $\vu$ and $\vv$ are \underline{orthogonal}, written $\vu\perp\vv$, if 
\[
\bilin{\vu,\vv}=0.
\]
\end{defn}

Since $V$ is an inner product space ``is orthogonal to'' is a symmetric relationship.

If $X$ is a subset of $V$, then we can define the orthogonal complement of $X$ via
\[
X^\perp=\{\vv\in V | \bilin{\vv,x}=0\quad\forall x\in X\}.
\]

The following two results are clear:

\begin{prop}
For any subset $X$,
\[
X^{\perp}=\bilin{X}^{\perp},
\]
and 
\[
\bilin{X}\subset X^{\perp\perp}.
\]
\end{prop}

If $V$ is finite dimensional, then the last inclusion is an equality.

Orthogonality provides a way to easily compute inner products. In particular, if $\{\vu_1,\dots,\vu_n\}$ is a basis such that $\vu_i\perp\vu_j$ for $i\neq j$, then if $\vv=\sum a_i\vu_i$ and $\vw=\sum b_i\vu_i$, then
\[
\bilin{\vv,\vw}=\sum a_i\bar{b}_j\bilin{\vu_i,\vu_j}=\sum a_i\bar{b}_i\bilin{\vu_i,\vu_i}.
\]

\begin{defn}
A set $X$ is orthonormal if for all $\vu_i,\vu_j\in X$,
$\bilin{\vu_i,\vu_j}=\delta_{i,j}$.
\end{defn}

Thus for an orthonormal basis, we can very easily compute inner products. Moreover, we can easily express any vector as a linear combination of basis vectors.

\begin{prop}
If $\{\vu_1,\dots,\vu_n\}$ is an orthonormal basis, then 
\[
\vv=\sum \bilin{\vv,\vu_i}\vu_i
\] for all $\vv\in V$.
\end{prop}

\begin{prop}
If $X$ is an orthonormal set, then $X$ is linearly independent.
\end{prop}
\begin{proof}
Consider a linear dependence relation
\[a_1\vu_1+\dots+a_n\vu_n=\vec{0}.
\]
Taking the inner product with $\vu_i$ yields $a_i=0$, and the result follows immediately.
\end{proof}


A very important result is that given any (linearly independent set) $X$ in an inner product space, we can find an orthogonal (or orthonormal set) $U$ with the same span as $X$. In particular, given a basis, we can find an orthogonal or orthonormal basis. This is the Grahm-Schmidt orthogonalization process. If $\{\vv_1,\dots,\vv_m\}$ is a linearly independent set, then let $\vu_1=\vv_1$, and for $i>1$, let
\[
\vu_i=\vv_i-\sum_{j=1}^{i-1}\frac{\bilin{\vv_i,\vu_j}}{\bilin{\vu_j,\vu_j}}\vu_j.
\]

\begin{prop}
The set $\{\vu_1,\dots,\vu_n\}$ defined above is orthogonal and
\[
\bilin{\vu_1,\dots,\vu_i}=\bilin{\vv_1,\dots,\vv_i}
\] for all $i$.
\end{prop}
\begin{proof}
The second part is immediate. Each of the vectors $\vv_1$ through $\vv_i$ is a linear combination of the vectors $\vu_1$ through $\vu_i$.

For the first part, we compute directly. Since orthogonality is a symmetric relation, it will suffice to show that $\bilin{\vu_i,\vu_k}=0$ for $k<i$:
\[
\bilin{\vu_i,\vu_k}=\bilin{\vv_i,\vu_k}-\sum_{j=1}^{i-1}\frac{\bilin{\vv_i,\vu_j}}{\bilin{\vu_j,\vu_j}}\bilin{\vu_j,\vu_k}.
\]
By induction on $i$, we see that the sum collapses into a single term:
\[
\frac{\bilin{\vv_i,\vu_k}}{\bilin{\vu_k,\vu_k}}\bilin{\vu_k,\vu_k}=\bilin{\vv_i,\vu_k},
\]
and so $\bilin{\vu_i,\vu_k}=0$.
\end{proof}

It is important to realize that this does not depend on the inner product space being $\R^n$ or $\C^n$. This works for any inner product space (since each expression only involves finitely many terms, we can apply this in the infinite dimensional case. We do lose control over the number of terms, however, and the span can change).

At the end of our discussion of isometries, we will see that Grahm-Schmidt actually gives a rather deep connection with matrices. For now, we will set the stage with the following:

\begin{thm}
Choosing an orthonormal basis gives an isomorphism $T\colon\F^n\to V$, where we send the $i$\textsuperscript{th} standard basis to the $i$\textsuperscript{th} basis vector in $V$, and
\[
\bilin{T\vv,T\vu}=\vv\cdot\vu
\]
for all $\vv,\vu\in\F^n$.
\end{thm}

\section*{Isometries}

Let $L\colon V\to W$ be a map of normed spaces. 

\begin{defn}
$L$ is an isometry if 
\[
||L\vv||=||\vv||
\]
for all $\vv$.
\end{defn}

Thus an isometry preserves distance. In particular, an isometry is necessarily an injection, since if $\vv\in\ker(L)$, then
\[
0=||L\vv||=||\vv||,
\]
and so $\vv=0$. On the other hand, if $V$ and $W$ are not finite dimensional, then an isometry need not be an isomorphism.

\begin{prop}
An isometry is always continuous for the metric topologies.
\end{prop}
\begin{proof}
In our $\epsilon$-$\delta$ definition, we need only take $\delta$ to be $\epsilon$.
\end{proof}

Thus we can explain part of the statement about metric spaces and completions: any metric space admits an isometry to a complete metric space (in fact, we showed that the inclusion of $M$ into our $\bar{M}$ was an isometry, so this is immediate).

For an inner product space, an isometry also preserves the inner product:
\[
\bilin{\vv,\vw}=\bilin{L\vv,L\vw}.
\]

This is because of the polarization identities which relate the inner product to the norms of various terms. We show the real case and leave the complex case as an easy, similar exercise.

\begin{prop}
For any $\vv,\vu$, we have
\[
\bilin{\vu,\vv}=\frac{1}{4}\left(||\vu+\vv||^2-||\vu-\vv||^2\right).
\]
\end{prop}
\begin{proof}
Expand the right-hand side and use symmetry.
\end{proof}

This means that not only do isometries preserve distance but also they preserve angles (ie they are ``conformal''). So all geometry is preserved by an isometry.

Some of the most important isometries are ``self-isometries''. 

\begin{defn}
Let $O(V)$ be the set of self-isometries of $V$. This is a group under composition.
\end{defn}

\begin{prop}
If $\{\vu_1,\dots,\vu_n\}$ is an orthonormal set in $V$ and $A\in O(V)$, then
\[
\{A\vu_1,\dots,A\vu_n\}
\] 
is an orthonormal set.
\end{prop}
\begin{proof}
This is immediate from isometries preserving angles.
\end{proof}

Of course, the previous result is also true for isometries in general.

Thus given an ordered orthonormal basis for $V$, applying $A$ gives another ordered orthonormal basis. Our usual arguments then give us an interpretation of $O(V)$: this is the collection of all ``change-of-orthonormal-basis'' matrices.

For $V=\R^n$, we have a preferred orthonormal basis: the standard basis. Let $A\in O(\R^n)$. Then since the columns of $A$ are the values of $A$ on an orthonormal basis, they form an orthonormal set.

\begin{prop}
If $A\in O(\R^n)$, then $A^tA=I$, and therefore the inverse to $A$ is $A^t$.
\end{prop}

\begin{proof}
The element in position $(i,j)$ in $A^tA$ is the dot product of the $i$\textsuperscript{th} column of $A$ with the $j$\textsuperscript{th} column of $A$. Since the columns are orthonormal, we conclude that $A^tA=I$.

Since $\R^n$ is finite dimensional, if $BA=I$, then $AB=I$ and $B$ is the inverse to $A$. We apply this to $B=A^t$.
\end{proof}

Elements of $O(\R^n)$ are called orthogonal matrices. We can restate Grahm-Schmidt in the following way. 
\begin{prop}
If $M\in GL_{n}$ is any matrix, then we can write $M$ as
\[
M=P\cdot R,
\]
where $P\in O(\R^n)$ and where $R$ is upper-triangular.
\end{prop}

The matrix $R$ essentially records the steps required to pass from the columns of $M$ as a set of vectors to an orthogonal set, and $P$ is the normalized form of the orthogonal vectors. This is the so-called ``polar decomposition'' of $M$. Singular matrices also have a polar decomposition, but there is more choice here.

If we work over $\C$, then we have a similar statement. The analogous result for $\C^1$ is that any non-zero complex number $z$ can be written as $z=e^{i\theta}r$, where $r>0$. The $e^{i\theta}$ part is exactly an element of $O(\C)$, and the other part is our ``upper triangular matrix''.


Why would polar form be nice? Two reasons: first the operation of putting a matrix into polar form is continuous and second, the space of upper-triangular matrices is very simple. Thus we see that the topology of the space of all matrices or all invertible matrices is determined by that of the orthogonal matrices. That's a nice simplification.

\end{document}