LECTURE 16: TENSOR PRODUCTS

We address an aesthetic concern raised by bilinear forms: the source of a bilinear function is not a vector space. When doing linear algebra, we want to be able to bring all of that machinery to bear. That means we need to consider only vector spaces and linear maps. Our constructions today will answer that point. We begin with a generalization of a bilinear form.

Definition. A function f: U × V → W is \underline{bilinear} if

f(au⃗_{1} + bu⃗_{2}, v⃗) = af(u⃗_{1}, v⃗) + bf(u⃗_{2}, v⃗) and
f(u⃗, av⃗_{1} + bv⃗_{2}) = af(u⃗, v⃗_{1}) + bf(u⃗, v⃗_{2}).

Let Bilin(U, V; W) denote the set of all bilinear functions U × V → W. This is clearly a vector space with the usual operations.

Thus we have for each fixed u⃗ a linear map fu⃗ = f(u⃗, −): V → W, and for each v⃗ ∈ V, a linear map fv⃗ = f(−, v⃗): U → W. This already is a kind of generalization of our Reisz map we saw last time, and we will return to this later.

The tensor product is the unique (up to isomorphism) vector space such that for any W

L(U ⊗ V, W) = Bilin(U, V; W).

In some sense, this is our defining principal. We’ll give two different constructions, and then show they are the same. We first review somewhat a “universal” property. We’ve actually already seen this notion twice before:

(1) The free vector space on a set X, 𝔽{X}, had the defining property that to give a map of sets X → V was the same as to give a linear transformation 𝔽{X} → V, and that we recovered any map of sets by composing the linear map with a canonical map of sets X → 𝔽{X} that was the inclusion of a basis.

(2) The quotient space V/W had the defining property that given a map L: V → U such that W ⊂ ker(L), we can find a unique map V/W → U such that the composite V → V/W → U is L.

In both of these cases, we had two pieces of data: the vector space we wanted and a distinguished map to our vector space that factors any map of a certain kind (in the first example, it was just maps of sets, and in the second, it was linear maps with a kernel bigger than W). We can do the same for the tensor product by specifying its universal property.

Definition. Let U and V be vector spaces. Then the tensor product is vector space U ⊗ V such that

(1) There is a bilinear map i: U × V → U ⊗ V and

(2) Given any bilinear map f: U × V → W, there is a unique linear map L_{f}: U ⊗ V → W such that L_{f} ∘ i = f.

Of course, this definition might be defining nothing. On the other hand, if we have two vector spaces that satisfy the two conditions of the definition, then they are isomorphic (and if we work slightly harder and ensure that the map U ×V → U ⊗V is remembered, then the isomorphism is also unique).

Proposition. If (U ⊗ V)_{1} and (U ⊗ V)_{2}, together with their bilinear maps i_{1} and i_{2} respectively, both satisfy the conditions of the definition, then there is a unique isomorphism that takes i_{1} to i_{2} and vice-versa.

Proof. Setting W in the definition equal to (U ⊗ V)_{2}, we have a unique map

L_{i_{2}} : (U ⊗ V)_{1} → (U ⊗ V)_{2}

such that L_{i_{2}} ∘ i_{1} = i_{2}. On the other hand, if we swap the roles of (U ⊗ V)_{1} and (U ⊗ V)_{2}, then we get a map

L_{i_{1}} : (U ⊗ V)_{2} → (U ⊗ V)_{1}.

Now the composite L_{2} = L_{i_{2}} ∘ L_{i_{1}} : (U ⊗ V)_{2} → (U ⊗ V)_{2} is a linear map such that L_{2} ∘ i_{2} = i_{2}. Thus if we apply our defining property when W = (U ⊗ V)_{2} and factoring through (U ⊗ V)_{2}, then we see that L_{2} is the unique map such that L_{2} ∘ i_{2} = i_{2} (this was the uniqueness part of the definition). Of course, the identity map is also such a map, so by uniqueness, L_{2} is the identity map. The same argument shows that L_{1} = L_{i_{1}} ∘L_{i_{2}} is the identity, and we have constructed inverse isomorphisms. □

We’ll now focus on two constructions of very different flavors: an almost unuseable one that has very good formal properties and an unnaturally one (with a choice [boo!] of basis) that is easier to compute with.

Construction #1. In this construction, we will produce something that obviously satisfies the conditions of the definition. We begin by thinking again about what we want. We want a vector space U⊗V, together with a bilinear map i: U×V → U⊗V, such that for any W,

L(U ⊗ V, W) ≅ Bilin(U, V; W)

and the isomorphism from L is given by L ↦ L ∘ i. We will build U ⊗ V and i by first considering all functions (not just bilinear ones). The free vector space 𝔽{−} has the desired universal property here:

L(𝔽{U × V }, W) = Map(U × V, W),

and the isomorphism is again given by composing with the inclusion i: U × V → 𝔽{U ×V }. Every bilinear function is in particular a map of sets U ×V → W, so we need to simply impose restrictions that make our functions bilinear. We’ll consider linearity in the first factor, as linearity in the second is an obvious extension.

A function f: U × V → W is linear iff

f(au⃗_{1} + bu⃗_{2}, v⃗) = af(u⃗_{1}, v⃗) + bf(u⃗_{2}, v⃗), or equivalently
f(au⃗_{1} + bu⃗_{2}, v⃗) − af(u⃗_{1}, v⃗) − bf(u⃗_{2}, v⃗) = 0.

This is a linear combination of values of f on elements of U × V, so we can record the same information with L_{f}, the linear transformation 𝔽{U × V } → W encoding f. Our identification now becomes: f is linear in the first factor iff

L_{f}((au⃗_{1} + bu⃗_{2}, v⃗) − a(u⃗_{1}, v⃗) − b(u⃗_{2}, v⃗)) = 0,

since L_{f} is linear and L_{f}((u⃗, v⃗)) = f(u⃗, v⃗).

This is a much nicer condition. We consider now the subspace I of 𝔽{U × V } spanned by all possible vectors of this form

I = ⟨(au⃗_{1} + bu⃗_{2}, v⃗) − a(u⃗_{1}, v⃗) − b(u⃗_{2}, v⃗), (u⃗, av⃗_{1} + bv⃗_{2}) − a(u⃗, v⃗_{1}) − b(u⃗, v⃗_{2})⟩.

Thus f is bilinear iff I ⊂ ker(L_{f}). The universal property of the quotient tells us then that f is bilinear iff L_{f} extends uniquely over 𝔽{U × V }/I. Here is a diagram that records much of this

\xymatrix{U × V \ar[d]_{f} \ar[r]^{i} & 𝔽\{U × V \} \ar[dl]_{L_{f}} \ar[r]^{π_{I}} & 𝔽\{U × V \}/I \ar[dll]^{L̃_{f}} \\ W} .

The first diagonal map exists and is unique no matter what f is. The second diagonal map exists and is unique whenever f is bilinear. Thus we define the tensor product to be 𝔽{U × V }/I, and the structure map is the composite π_{I} ∘ i. We must check that this is bilinear. This is easy, however. The element (au⃗_{1}+bu⃗_{2}, v⃗) ∈ U ×V maps under i to the basis vector by the same name. However, since

(au⃗_{1} + bu⃗_{2}, v⃗) − a(u⃗_{1}, v⃗) − b(u⃗_{2}, v⃗) ∈ I,

we know that

π_{I}((au⃗_{1} + bu⃗_{2}, v⃗)) = aπ_{I}(u⃗_{1}, v⃗) + bπ_{I}(u⃗_{2}, v⃗),

and thus π_{I} ∘ i is bilinear.

That this construction gives us the right universal property is immediate. There are a few draw-backs:

(1) If |𝔽| is infinite, then 𝔽{U × V } is HUGE (the dimension is |𝔽|^{2} · dim U · dim V). Additionally, I is also HUGE (and it has essentially the same dimension).

(2) It’s not immediately clear that U ⊗V is not the zero space. In other words,

that I ⊊ 𝔽{U × V }. Additionally, at the end of the day, we don’t normally specific a linear transformation by specifying where EVERY element goes. We just work with a basis. That brings us to construction 2.

Construction 2. Now assume that U has a basis {u⃗_{1}, …, u⃗_{n}} and V has a basis {v⃗_{1}, …, v⃗_{m}} (and here infinite bases work no differently). We start with an easy proposition.

Proposition. Let f be a bilinear function U × V → W then f is uniquely determined by its values on (u⃗_{i}, v⃗_{j}), and any choice of values on these points determines a bilinear function.

Proof. Let u⃗ = ∑ a_{i}u⃗_{i} and let v⃗ = ∑ b_{j}v⃗_{j}. Then by linearity in the first factor

f(u⃗, v⃗) = ∑_{i} a_{i}f(u⃗_{i}, v⃗).

By linearity in the second factor, this is

= ∑_{i} a_{i} ∑_{j} b_{j}f(u⃗_{i}, v⃗_{j}) = ∑_{i,j} a_{i}b_{j}f(u⃗_{i}, v⃗_{j}).

Thus specifying the values on (u⃗_{i}, v⃗_{j}) defines f. Since these form a basis, there are no additional restrictions and hence any collection of values gives a bilinear function. □

This gives us our second definition:

U ⊗ V = 𝔽{(u⃗_{i}, v⃗_{j})}.

From the above proposition, we know that bilinear functions U × V are in 1-1 correspondence with linear maps 𝔽{(u⃗_{i}, v⃗_{j})}, and therefore we have almost verified the universal property. The only remaining point is to produce a bilinear map U × V → U ⊗ V. However, using our 1-1 correspondence, we just have to produce a linear transformation U ⊗ V → U ⊗ V. The identity works very nicely, and the corresponding bilinear map is the last piece of structure for the universal property.

This gives us a very nice, easy consequence.

Proposition. The dimension of U ⊗ V is dim(U) · dim(V).

Moving On. We already showed that any two things that satisfy the universal property are uniquely isomorphic. Thus our two definitions of ⊗ are the same, and we will feel free to use both depending on context. In all cases, we will denote the image of (u⃗, v⃗) in U ⊗ V by u⃗ ⊗ v⃗.

We turn now to maps. Let’s consider S: U → U′ and T: V → V′. Then we want to produce a new linear map S ⊗ T: U ⊗ V → U′ ⊗ V′. To produce a linear map like this is the same as producing a bilinear map U × V → U′ ⊗ V′. This, however, is essentially formal (one of the advantages of a universal property).

Proposition. If i: U′ × V′ → U′ ⊗ V′ is the bilinear structure map, then

i ∘ (S × T): U × V → U′ ⊗ V′

is bilinear. Here S × T: U × V → U′ × V′ is given by (S × T)(u⃗, v⃗) = (S(u⃗), T (v⃗)).

Proof. We first check linearity in the first factor. Our map is a composite, so we break it into two pieces and analyze them separately.

(S × T)(au⃗_{1} + bu⃗_{2}, v⃗) = (S(au⃗_{1} + bu⃗_{2}), T (v⃗)) = (aS(u⃗_{1}) + bS(u⃗_{2}), T (v⃗)).

The map i is bilinear, so we know that

i((aS(u⃗_{1}) + bS(u⃗_{2}), T (v⃗))) = ai(S(u⃗_{1}), T (v⃗)) + bi(S(u⃗_{2}), T (v⃗))

This is exactly what expresses linearity in the first factor. The second factor is handled identically. □

Thus we get a linear map S ⊗ T: U ⊗ V → U′ ⊗ V′ by

(S ⊗ T)(u⃗ ⊗ v⃗) = S(u⃗) ⊗ T (v⃗)

and extending by linearity.

Proposition. The assignment (S, T) ↦ S ⊗ T is a bilinear map

L(U, U′) × L(V, V′) → L(U ⊗ V, U′ ⊗ V′),

and thus descends to a linear map

L(U, U′) ⊗ L(V, V′) → L(U ⊗ V, U′ ⊗ V′).

When all spaces are finite dimensional, this is an isomorphism.

We can drop the dimensionality restrictions, but we won’t explore that here.

Proof. This kind of argument is the same as the ones we have been giving. We check directly that S ⊗ T is linear in each factor, and then we conclude we have a map.

For this, consider (aS_{1} + bS_{2}) ⊗ T. Equality of functions means they have the same values at each point, and so it suffices to show that on a generic test point (u⃗ ⊗ v⃗),

(aS_{1} + bS_{2}) ⊗ T (u⃗ ⊗ v⃗) = aS_{1} ⊗ T (u⃗ ⊗ v⃗) + bS_{2} ⊗ T (u⃗ ⊗ v⃗).

We write out the left most term:

(aS_{1} + bS_{2} ⊗ T)(u⃗ ⊗ v⃗) = ((aS_{1} + bS_{2})(u⃗)) ⊗ T (v⃗) = (aS_{1}(u⃗) + bS_{2}(u⃗)) ⊗ T (v⃗).

Since the tensor product in U′ ⊗ V′ (where the last term in the equality lives) is bilinear, we conclude that this is the right-hand side of the desired equality.

Showing that this is an isomorphism is slightly tricker. Careful details will be left as an exercise. For now, note that if we choose a basis for all of our vector spaces, then we get a basis for the linear transformations and for the tensor product by mirroring the “dual basis” construction. With this basis, it is immediate that basis vectors map to basis vectors in a bijective way, and hence that the map is an isomorphism. □