1. The sum-of-squares combined error

To be more concrete, we should ask for $x$ and $y$ that make the equations as nearly true as possible. In other words, we should try to find $x$ and $y$ that minimize the errors

$$\begin{array}{lcl} \epsilon _ 1 & = & x+2y\\ \epsilon _ 2 ... ...\epsilon _ 3 & = & x-2y\\ \epsilon _ 4 & = & 2x-y \end{array}$$

There is still a problem, though: It is not clear what it means to minimize several errors simultaneously. We must choose some single combined measure of the errors and then minimize that.

To combine the errors we should not use $\epsilon _ 1 + \epsilon _ 2 + \epsilon _ 3 + \epsilon _ 4$, because negative errors could cancel out positive ones. It is more reasonable, but still not good, to use $\vert \epsilon _ 1 \vert + \vert \epsilon _ 2 \vert + \vert \epsilon _ 3 \vert + \vert \epsilon _ 4 \vert$. The use of absolute values is undesirable both because the absolute value function is not differentiable and because there might be many $(x,y)$ pairs giving the same minimum value (as it turns out).

Instead, a nice measure of the combined error is the sum of squares of the individual errors. Because the combined error depends on the choice of $x$ and $y$, let's write

$E(x,y) = \epsilon _ 1 ^ 2 + \epsilon _ 2 ^ 2 + \epsilon _ 3 ^ 2 + \epsilon _ 4 ^ 2$.

Just trying some different choices for $x$ and $y$ gives

$E(1,1) = 5$ (since $0 ^ 2 + 0 ^ 2 + 2 ^ 2 + 1 ^ 2 = 5$),

$E(1,0) = 5$,

$E(0,1) = 23$ (worse); but the least-squares method will reveal

$E(1.4,0.5) = 0.9$ (fantastic, and the lowest possible error).

It is worth noting that the least squares method will allow several small errors in preference to one large one; for example, $1 ^ 2 + 1 ^ 2 + 1 ^ 2 + 1 ^ 2$ is smaller than $4 ^ 2 + 0 ^ 2 + 0 ^ 2 + 0 ^ 2$.