\documentclass{amsart}
%\usepackage{kbordermatrix}

\newtheorem*{defn}{Defintion}
\newtheorem*{prop}{Proposition}
\newtheorem*{thm}{Theorem}
\newtheorem*{cor}{Corollary}
%\theoremstyle{definition}
\newtheorem*{defin}{Definition}
\newtheorem*{example}{Example}
\newtheorem*{remark}{Remark}

\newcommand{\F}{\mathbb F}
\renewcommand{\L}{\mathcal L}
\newcommand{\bilin}[1]{\langle#1\rangle}

\title{Lecture 17: Properties of Tensor Products}

\begin{document}
\maketitle

Last time we showed how to build a universal vector space for bilinear forms. Today we will look more at what this does for maps and what happens in more general cases.

We ended last time with the following theorem.

\begin{thm}
If $U,U',V,V'$ are finite dimensional vector spaces, then the tensor product map gives an isomorphism
\[
\L(U,V)\otimes\L(U',V')\to\L(U\otimes U',V\otimes V').
\]
\end{thm}

We then used a very simple proposition to get a very nice corollary.

\begin{prop}
For any vector space $V$,
\[
V\otimes\F\cong V\cong \F\otimes V,
\]
with all isomorphisms being induced by the natural basis $\{1\}$ of $\F$.

For any vector space $V$,
\[
\L(\F,V)\to V
\]
given by $f\mapsto f(1)$ is an isomorphism.
\end{prop}

\begin{cor}
We have a natural isomorphism for any finite dimensional $V$ and $W$
\[
V^\ast\otimes W\cong\L(V,\F)\otimes\L(\F,W)\to\L(V\otimes\F,\F\otimes W)\cong\L(V,W).
\]
\end{cor}

This gives us another way to think about linear transformations as matrices. After choosing a basis, we identify elements of $V^\ast$ with row vectors and elements of $W^\ast$ with column vectors. This means we can identify elements of $V^\ast\otimes W$ with row vectors times column vectors, or matrices.

This identification drives home the point that elements of $V^\ast\otimes W$ are {\emph{not}} simple elements of the form $f\otimes \vec{w}$ but rather linear combinations of these. Since $dim \ker(f)\geq \dim(V)-1$, we know that the rank of the matrix associated to $f\otimes\vec{w}$ is at most $1$. So we must have sums of these to produce matrices of larger rank.

Now assume we are given ordered bases $\mathcal B=\{\vec{u}_1,\dots\}$ and $\mathcal B'=\{\vec{u}'_1,\dots\}$ of $U$ and $U'$. Then we can get an ordered basis of $U\otimes U'$ as follows.

\begin{defin}
Let $I$ and $J$ be linearly ordered sets. Then we define the \underline{lexigraphic ordering} on $I\times J$ by saying
\[
(i,j)<(i',j')\text{ if }\begin{cases}
i<i' & \text{or} \\
i=i', j<j'.
\end{cases}
\]
\end{defin}

We know that the vectors $\vec{u}_i\otimes\vec{u}_j'$ form a basis for $U\otimes U'$. Thus the basis is indexed by $I\times J$, so we have an ordered basis using the lexigraphic ordering. We can then ask what happens with matrices associated to linear transformations. The general form is this:

\[
\bordermatrix{{}& \{\vec{u}_1\otimes\vec{u}'_1 & \dots & \vec{u}_1\otimes\vec{u}'_n\} & \dots & \{\vec{u}_m\otimes\vec{u}'_1 & \dots &
\vec{u}_m\otimes\vec{u}'_n\} \cr
\vec{v}_1\otimes\vec{v}'_1 & \cr
\vdots & \cr
\vec{v}_1\otimes\vec{v}'_\ell & \cr
\vdots & \cr
\vec{v}_k\otimes\vec{v}'_1 & \cr
\vdots & \cr
\vec{v}_k\otimes\vec{v}'_\ell &}
\]
As written, we see a block decomposition: the blocks are $n\times \ell$-blocks corresponding to $\vec{u}_i\otimes U'$ and $\vec{v}_j\otimes V'$. Understanding the matrix for $L\otimes L'$ follows from this simple observation. We know that the matrix for $L\otimes L'$ is found by expressing $L(\vec{u}_i)\otimes L'(\vec{u}'_j)$ in terms of $\vec{v}_k\otimes\vec{v}_\ell'$. Let $A$ be the matrix for $L$ and $B$ the matrix for $L'$. 
\begin{prop}
With the lexigraphically ordered basis, the matrix for $L\otimes L'$, $A\otimes B$ is the block matrix with $m\times k$-blocks of size $n\times\ell$, and where the $(i,j)$\textsuperscript{th} block is given by $a_{i,j} B$.
\end{prop}
\begin{proof}
Our discussion above showed it has this block form. The $(i,j)^{\text{th}}$ block is the one corresponding to expressing $L\otimes L'$ on $\langle\vec{u}_i\rangle\otimes U'$ in terms of $\langle\vec{v}_j\rangle\otimes V'$. Since we compute $L\otimes L'$ by just evaluating the linear transformations on the corresponding factors, we see that the coefficient for the $L$ part is $a_{i,j}$. The actual block is given by expressing $L'$ in the corresponding bases, and so we get that the block is $a_{i,j}$ from the first factor times $B$ from the second, as required.
\end{proof}

\begin{remark}
We can make this block decomposition more natural. Choosing a basis for $U$ and $V$ gives us identifications 
\[
U\cong \bilin{\vec{u}_1}\oplus\dots\oplus\bilin{\vec{u}_m},\text{ and } V\cong\bilin{\vec{v}_1}\oplus\dots\oplus\bilin{\vec{v}_k}.
\]
This gives us a similar splitting of the tensor products (more on this in a little bit):
\[
U\otimes U'\cong\bigoplus_{i=1}^m \bilin{\vec{u}_i}\otimes U',\text{ and } V\otimes V'\cong\bigoplus_{j=1}^{k}\bilin{\vec{v}_j}\otimes V'.
\]

From the second Homework, we know that the direct sum plays nicely with $\L(-,-)$. Thus we can, by induction, conclude that
\[
\L(U\otimes U', V\otimes V')\cong \bigoplus_{i=1}^m\bigoplus_{j=1}^k \L\big(\bilin{\vec{u}_i}\otimes U',\bilin{\vec{v}_j}\otimes V'\big).
\]

This identifies a linear transformation $U\otimes U'$ to $V\otimes V'$ with an $m\times k$-array of linear transformations. In our situation, the piece corresponding to the $(i,j)$ factor is the tensor product (where we have used the notation from the problem set)
\[
\left(\pi_{\bilin{\vec{v}_j}}\circ L\circ \iota_{\bilin{\vec{u}_i}}\right)\otimes L'.
\]
By definition of $A$, the first tensor factor is just multiplication by $a_{i,j}$. The second part is just $L'$, so its matrix is $B$, and we have reproduced a second way the earlier result.
\end{remark}

This gives us a new operation on matrices: tensor product.
\begin{defin}
If $A\in M_{m,k}$ and $B\in M_{n,\ell}$, then $A\otimes B$ is the block matrix with $m\times k$ blocks of size $n\times \ell$ and where the $i,j$ block is $a_{i,j}B$.
\end{defin}

That this is a nice operation will follow from our properties of tensor products. We list some here, and some will occur as exercises.

\begin{prop}
Let $U$, $V$, $W$ be finite dimensional vector spaces.
\begin{enumerate}
\item $(U\otimes V)\otimes W$ is naturally isomorphic to $U\otimes(V\otimes W)$.
\item $U\otimes V$ is naturally isomorphic to $V\otimes U$.
\item $U\otimes (V\oplus W)$ is naturally isomorphic to $(U\otimes V)\oplus (U\otimes W)$.
\item If $W\in I$ is a sequence of vector spaces, then $U\otimes\big(\bigoplus_{i\in I} W_i\big)$ is naturally isomorphic to $\bigoplus_{i\in I} (U\otimes W_i)$.
\end{enumerate}
\end{prop}

Many of these are most easily proved by showing that both sides satisfy the same universal property. We'll need an example for the first one more in later lecture, so we'll spell it out now.

\begin{defin}
Let $U_1,\dots,U_n$ and $V$ be vector spaces. A function
\[
f\colon U_1\times\dots\times U_n\to V
\]
is \underline{multilinear} if it is linear in each factor:
\[
f(\vec{u}_1,\dots,a\vec{u}_i+b\vec{u}_i',\dots,\vec{u}_n)=af(\vec{u}_1,\dots,\vec{u}_i,\dots,\vec{u}_n)+bf(\vec{u}_1,\dots,\vec{u}_i',\dots,\vec{u}_n).
\]
\end{defin}

Just as with bilinear functions, we can ask for the universal space for multlinear functions, and we can define it in exactly the same way as the tensor product: the universal space for $U_1,\dots,U_n$ and multilinear functions is the tensor product $U_1\otimes \dots\otimes U_n$. This is not the same as $U_1\otimes\big(\dots\otimes(U_{n-1}\otimes U_n)\dots\big)$. It is, however, naturally isomorphic to it.

The reason is slightly subtle. Both do satisfy the universal property (hence the natural isomorphism part). The reason equality fails is that the sets $U_1\times\dots\times U_n$ and $U_1\times\big(U_2\times\dots\times U_n\big)$ (and more generally for other associations) are not the same. The former is ordered $n$-tuples, the latter is an ordered pair, where the second term is an ordered $(n-1)$-tuple. Not the same, but naturally bijective. All of our earlier statements for tensor products of two factors hold for tensor products of arbitrarily many factors, and we will use them without comment in future lectures.


\end{document}