LECTURE 22: INNER PRODUCTS

For the remaining lectures, we will work exclusively over ℝ or ℂ. We’ll also look at a very special kind of bilinear form.

Definition 1. An \underline{inner product} is a function ⟨−, −⟩: V × V → 𝔽 such that

(1) ⟨v⃗, v⃗⟩ ≥ 0 with equality iff v⃗ = 0.

(2) ⟨v⃗, u⃗⟩ = \begin{cases}⟨u⃗, v⃗⟩ & 𝔽 = ℝ \\ \overline{⟨u⃗, v⃗⟩} & 𝔽 = ℂ. \end{cases}

(3) ⟨av⃗ + bu⃗, w⃗⟩ = a⟨v⃗, w⃗⟩ + b⟨u⃗, w⃗⟩.

The first condition is that the inner product is “positive definite”, the second is that it is (conjugate) symmetric, and the third is that it is linear in the first factor.

Proposition 1. If ⟨−, −⟩ is an inner product, then

⟨v⃗, au⃗ + bw⃗⟩ = ā⟨v⃗, u⃗⟩ + b̄⟨v⃗, w⃗⟩,

and so if 𝔽 = ℝ, then an inner product is bilinear.

Proof. Use conjugate symmetry to swap the two coordinates, then use linearity in the first and conjugate symmetry again. □

Over ℂ, this is not bilinearity. The inner product is additive in the second factor, but it has a conjugation that appears in the scalar multiplication. In this case, we call it \underline{sesquilinear} (the prefix “sesqu-” means “one and a half”: it’s linear in the first factor and half linear in the second). Of course, we understand complex conjugation completely, so sesquilinearity is as good as bilinearity.

Now some examples.

(1) ℝ^{n} with the dot product. This is the prototype of all inner product spaces.

(2) ℂ^{n} with the standard inner product:

⟨(a_{1}, …, a_{n}), (b_{1}, …, b_{n})⟩ = a_{1}b̄_{1} + ⋯ + a_{n}b̄_{n}.

(3) The space ℓ_{2} = {(a_{0}, …)| ∑ |a_{i}|^{2} < ∞} with the inner product

⟨(a_{1}, …), (b_{1}, …)⟩ = ∑ a_{i}b̄_{i}.

This is the [countably] infinite dimensional analogue of the previous ones.

(4) C[a, b], the set of continuous functions on [a, b] with inner product

⟨f, g⟩ = ∫_{a}^{b} f(x)\overline{g(x)}w(x)dx,

where w(x) > 0 for all x ∈ [a, b]. These spaces are essentially all the same (and all the same as ℓ_{2}, though this is all too analytical to be fun).

(5) If W is a subspace of an inner product space V, then the inner product of V restricted to W gives an inner product on W.

There are several very important consequences of the existence of an inner product. The first is a notion of distance.

Definition 2. The \underline{norm} of v⃗ is

||v⃗|| = \sqrt{⟨v⃗, v⃗⟩}.

Since the inner product is positive definite, this is always greater than or equal to zero, and moreover, we know that it equals zero iff v⃗ = 0. We know from sesquilinearity that

||av⃗|| = |a| · ||v⃗||.

This is one of the nice features of sesquilinearity.

We have two more nice relations: the Cauchy-Schwarz inequality (which will link the inner product to a notion of angles) and the triangle inequality (which will ensure that our norm gives a good notion of distance).

Proposition 2 (Cauchy-Schwarz). For all v⃗ and u⃗ in an inner product space V, we have

|⟨v⃗, u⃗⟩| ≤ ||v⃗|| · ||u⃗||.

Proof. We first note that if v⃗ or u⃗ is zero, then the result is obvious. We therefore assume that neither is zero.

We will consider the square of the norm of v⃗ − ru⃗ for various r and then deduce the desired result from this.

||v⃗ − ru⃗||^{2} = ⟨v⃗ − ru⃗, v⃗ − ru⃗⟩ = ||v⃗||^{2} − r⟨u⃗, v⃗⟩ − r̄(⟨v⃗, u⃗⟩ − r||u⃗||^{2}) ≥ 0.

This holds for all r, so in particular, it holds for the value of r that simplifies the expression: r = ⟨v⃗, u⃗⟩/||u⃗||^{2}. With this, our previous expression becomes

0 ≤ ||v⃗||^{2} − \frac{⟨v⃗, u⃗⟩}{||u⃗||^{2}}⟨u⃗, v⃗⟩ = ||v⃗||^{2} − \frac{|⟨v⃗, u⃗⟩|^{2}}{||u⃗||^{2}} ,

by sesquilinearity. Rearranging gives the result. □

What does this buy us? Let’s say that v⃗ is a \underline{unit vector} if ||v⃗|| = 1. If ||v⃗|| ≠ 0, then we can always divide v⃗ by its length to get a unit vector. Thus we can restate the previous result as

Proposition 3. If v⃗ and u⃗ are unit vectors, then

|⟨v⃗, u⃗⟩| ≤ 1.

Thus we can find a unique θ ∈ [0, π] such that

cos(θ) = ⟨v⃗, u⃗⟩.

Definition 3. The angle between v⃗ and u⃗ is that θ such that

⟨v⃗, u⃗⟩ = ||v⃗||||u⃗|| cos(θ).

With this, we can get a refined triangle inequality.

Proposition 4 (Triangle Inequality). For all v⃗ and u⃗, we have

||v⃗ + u⃗|| ≤ ||v⃗|| + ||u⃗||.

Proof. We show the square of the inequality.

||v⃗ + u⃗||^{2} = ⟨v⃗ + u⃗, v⃗ + u⃗⟩ = ||v⃗||^{2} + ||u⃗||^{2} + ⟨u⃗, v⃗⟩ + ⟨v⃗, u⃗⟩.

Since ⟨v⃗, u⃗⟩ + ⟨u⃗, v⃗⟩ is twice the real part of ⟨v⃗, u⃗⟩, we know

⟨v⃗, u⃗⟩ + ⟨u⃗, v⃗⟩ ≤ 2|⟨v⃗, u⃗⟩| ≤ 2||v⃗|| · ||u⃗||.

LECTURE 22: INNER PRODUCTS 3

This gives

||v⃗ + u⃗||^{2} ≤ ||v⃗||^{2} + 2||v⃗|| · ||u⃗|| + ||u⃗||^{2} = (||v⃗|| + ||u⃗||)^{2}.  □

We can do better, if we are working over ℝ: we can rederive the law of cosines. Recalling that

⟨v⃗, u⃗⟩ = ⟨u⃗, v⃗⟩,

we can simplify the earlier formula and derive the following. Proposition 5. For any u⃗ and v⃗, we have

||u⃗ − v⃗||^{2} = ||u⃗||^{2} + ||v⃗||^{2} − 2||u⃗|| · ||v⃗|| cos(θ),

where θ is the angle between u⃗ and v⃗.