This is just a quick explanation of problem 32.2 concerning 
Toeplitz matrix computations.
Solution sketch by Saunders, 12/99.


Toeplitz matrices are square matrices with the property that 
each entry (except those in the first row and column) is equal to 
the entry immediately the northwest (previous col and previous row).
This implies that the matrix has a diagonally striped pattern.

It follows that an n by n Toeplitz matrix corresponds to a sequence
of 2n - 1 values (t_0, ... , t_{2n-2}) by the definition
A = ( a_{i,j}), where a_{i,j} = t_{n-1-i+j}.  
(i.e. place t_0 at the bottom of the first column, then proceed first up 
the column then over the first row)

Part a:  Is the sum of Toeplitz matrices Toeplitz? product?
Solution: sum yes, product no.
That sum is Toeplitz
easily follows because if C = A + B, then 
c_{i,j} = a_{i,j} + b_{i,j}         // by def of matrix sum
	= a_{i-1,j-1} + b+{i-1,j-1} // by Toeplitz property of A,B
	= c{i-1,j-1}                // by def of matrix sum.

But is the product of Toeplitz matrices Toeplitz?
No.  Usually not.

             (1 1 1)            (2 2 3)
For instance (0 1 1) squared is (1 1 2), not Toeplitz.
             (1 0 1)            (2 1 2)

Part b:  Represent so that Toeplitz matrix addition takes O(n) time.

Solution: Represent by sequence of the entries in the first row and col
in the order described above.
To add the matrices just add the sequence elements componentwise.
Since the first row,col of C is the componentwise sum of the first row,cols
of A,B, and since C is Toeplitz as shown above, we see this gives the 
correct representation of the matrix sum.

Part c:  Multiply Toeplitz matrix times vector in O(nlg(n)) time.
Solution:
The product to compute is v = Au.  Let the indexing in u be in 
reverse order, that is let u_i be defined so that 
u = (u_n, ... , u_1).  But let v be indexed as v = (v_1 .. v_n) 
in the usual way.  Then

v_i = sum_{j = 1 to n}  a_{i,j}u_{n+1-j}
    = sum_{j = 1 to n}  t_{n-1-i+j}u_{n+1-j}
Which is just the coefficient of x^{2n-i} in 
the polynomial product ( sum_i t_i x^i ) times ( sum_i u_i x^i )
So treat both the sequence t and vector u as polynomials (pad with 
zeros to degree 2n-1), multiply using FFT, then extract the n desired
coefficients to be the components of v.

This costs O(n) + FFT cost on 2n degree polys. The latter dominates for
for a total which is  O(nlg(n)).

Part d: Multiply 2 Toeplitz matrices fast.

Solution:  Treat the second matrix as a sequence of n vectors and 
apply part c to each.  Cost is n * part c cost = O(n^2 lg(n)).

This is much faster than general matrix multiplication.  But this solution
treats the columns of the second matrix individually, ignoring the 
"shifted" relationships that exist among them.  Is there a way to 
exploit these relationships?  I don't know.