LECTURE 17: PROPERTIES OF TENSOR PRODUCTS

Last time we showed how to build a universal vector space for bilinear forms. Today we will look more at what this does for maps and what happens in more general cases.

We ended last time with the following theorem.

Theorem. If U, U′, V, V′ are finite dimensional vector spaces, then the tensor product map gives an isomorphism

L(U, V) ⊗ L(U′, V′) → L(U ⊗ U′, V ⊗ V′).

We then used a very simple proposition to get a very nice corollary.

Proposition. For any vector space V,

V ⊗ 𝔽 ≅ V ≅ 𝔽 ⊗ V,

with all isomorphisms being induced by the natural basis {1} of 𝔽.

For any vector space V,

L(𝔽, V) → V

given by f ↦ f(1) is an isomorphism.

Corollary. We have a natural isomorphism for any finite dimensional V and W

V^{∗} ⊗ W ≅ L(V, 𝔽) ⊗ L(𝔽, W) → L(V ⊗ 𝔽, 𝔽 ⊗ W) ≅ L(V, W).

This gives us another way to think about linear transformations as matrices. After choosing a basis, we identify elements of V^{∗} with row vectors and elements of W^{∗} with column vectors. This means we can identify elements of V^{∗} ⊗ W with row vectors times column vectors, or matrices.

This identification drives home the point that elements of V^{∗} ⊗ W are not simple elements of the form f ⊗ w⃗ but rather linear combinations of these. Since dim ker(f) ≥ dim(V) − 1, we know that the rank of the matrix associated to f ⊗ w⃗ is at most 1. So we must have sums of these to produce matrices of larger rank.

Now assume we are given ordered bases B = {u⃗_{1}, … } and B′ = {u′_{1}, … } of U and U′. Then we can get an ordered basis of U ⊗ U′ as follows.

Definition. Let I and J be linearly ordered sets. Then we define the lexigraphic ordering on I × J by saying

(i, j) < (i′, j′) if \begin{cases}i < i′ & or \\ i = i′, j < j′. \end{cases}

We know that the vectors u⃗_{i} ⊗ u′_{j} form a basis for U ⊗ U′. Thus the basis is indexed by I × J, so we have an ordered basis using the lexigraphic ordering. We can then ask what happens with matrices associated to linear transformations. The general form is this:

( {u⃗_{1} ⊗ u′_{1} … u⃗_{1} ⊗ u′_{n}} … {u⃗_{m} ⊗ u′_{1} … u⃗_{m} ⊗ u′_{n}} ; v⃗_{1} ⊗ v′_{1} ; ⋮ ; v⃗_{1} ⊗ v′_{ℓ} ; ⋮ ; v⃗_{k} ⊗ v′_{1} ; ⋮ ; v⃗_{k} ⊗ v′_{ℓ} )

As written, we see a block decomposition: the blocks are n×ℓ-blocks corresponding to u⃗_{i} ⊗ U′ and v⃗_{j} ⊗ V′. Understanding the matrix for L ⊗ L′ follows from this simple observation. We know that the matrix for L ⊗ L′ is found by expressing L(u⃗_{i}) ⊗ L′(u′_{j}) in terms of v⃗_{k} ⊗ v′_{ℓ}. Let A be the matrix for L and B the matrix for L′.

Proposition. With the lexigraphically ordered basis, the matrix for L ⊗ L′, A ⊗ B is the block matrix with m × k-blocks of size n × ℓ, and where the (i, j)^{th} block is given by a_{i,j}B.

Proof. Our discussion above showed it has this block form. The (i, j)^{th} block is the one corresponding to expressing L ⊗ L′ on ⟨u⃗_{i}⟩ ⊗ U′ in terms of ⟨v⃗_{j}⟩ ⊗ V′. Since we compute L ⊗ L′ by just evaluating the linear transformations on the corresponding factors, we see that the coefficient for the L part is a_{i,j}. The actual block is given by expressing L′ in the corresponding bases, and so we get that the block is a_{i,j} from the first factor times B from the second, as required. □

Remark. We can make this block decomposition more natural. Choosing a basis for U and V gives us identifications

U ≅ ⟨u⃗_{1}⟩ ⊕ ⋯ ⊕ ⟨u⃗_{m}⟩, and V ≅ ⟨v⃗_{1}⟩ ⊕ ⋯ ⊕ ⟨v⃗_{k}⟩.

This gives us a similar splitting of the tensor products (more on this in a little bit):

U ⊗ U′ ≅ ⨁_{i=1}^{m}⟨u⃗_{i}⟩ ⊗ U′, and V ⊗ V′ ≅ ⨁_{j=1}^{k}⟨v⃗_{j}⟩ ⊗ V′.

From the second Homework, we know that the direct sum plays nicely with L(−, −). Thus we can, by induction, conclude that

L(U ⊗ U′, V ⊗ V′) ≅ ⨁_{i=1}^{m} ⨁_{j=1}^{k} L(⟨u⃗_{i}⟩ ⊗ U′, ⟨v⃗_{j}⟩ ⊗ V′).

This identifies a linear transformation U ⊗ U′ to V ⊗ V′ with an m × k-array of linear transformations. In our situation, the piece corresponding to the (i, j) factor is the tensor product (where we have used the notation from the problem set)

(π_{⟨v⃗_{j}⟩} ∘ L ∘ ι_{⟨u⃗_{i}⟩}) ⊗ L′.

By definition of A, the first tensor factor is just multiplication by a_{i,j}. The second part is just L′, so its matrix is B, and we have reproduced a second way the earlier result.

This gives us a new operation on matrices: tensor product.

Definition. If A ∈ M_{m,k} and B ∈ M_{n,ℓ}, then A⊗B is the block matrix with m×k blocks of size n × ℓ and where the i, j block is a_{i,j}B.

That this is a nice operation will follow from our properties of tensor products. We list some here, and some will occur as exercises.

Proposition. Let U, V, W be finite dimensional vector spaces.

(1) (U ⊗ V) ⊗ W is naturally isomorphic to U ⊗ (V ⊗ W).

(2) U ⊗ V is naturally isomorphic to V ⊗ U.

(3) U ⊗ (V ⊕ W) is naturally isomorphic to (U ⊗ V) ⊕ (U ⊗ W).

(4) If W ∈ I is a sequence of vector spaces, then U ⊗ ( ⨁_{i∈I} W_{i}) is naturally isomorphic to ⨁_{i∈I}(U ⊗ W_{i}).

Many of these are most easily proved by showing that both sides satisfy the same universal property. We’ll need an example for the first one more in later lecture, so we’ll spell it out now.

Definition. Let U_{1}, …, U_{n} and V be vector spaces. A function

f: U_{1} × ⋯ × U_{n} → V

is \underline{multilinear} if it is linear in each factor:

f(u⃗_{1}, …, au⃗_{i} + bu′_{i}, …, u⃗_{n}) = af(u⃗_{1}, …, u⃗_{i}, …, u⃗_{n}) + bf(u⃗_{1}, …, u′_{i}, …, u⃗_{n}).

Just as with bilinear functions, we can ask for the universal space for multlinear functions, and we can define it in exactly the same way as the tensor product: the universal space for U_{1}, …, U_{n} and multilinear functions is the tensor product U_{1} ⊗ ⋯ ⊗ U_{n}. This is not the same as U_{1} ⊗ ( ⋯ ⊗ (U_{n−1} ⊗ U_{n}) …). It is, however, naturally isomorphic to it.

The reason is slightly subtle. Both do satisfy the universal property (hence the natural isomorphism part). The reason equality fails is that the sets U_{1} × ⋯ × U_{n} and U_{1} × (U_{2} × ⋯ × U_{n}) (and more generally for other associations) are not the same. The former is ordered n-tuples, the latter is an ordered pair, where the second term is an ordered (n − 1)-tuple. Not the same, but naturally bijective. All of our earlier statements for tensor products of two factors hold for tensor products of arbitrarily many factors, and we will use them without comment in future lectures.