Assignment #2

Due Tuesday, Oct. 1 by midnight

Implement a planar SSD template tracker.  Using the Kalman 

filter, estimate the 3 x 3 homography H_{t} at each time step

t over an image sequence which maps a template T to its location 

in the current image I_{t}.  You will manually initialize the tracker 

in the first frame.

Here's a source for the template image.  You should rectify and 

crop it to create the actual T (the sign is 30" tall and 24" wide).

Here's the AVI containing a 91-frame sequence taken from a 

car driven past this sign.

I've already written the outline of a program that handles AVI file reading and

writing:  hw2.m.  You need to fill in the numbered steps for Kalman filter

initialization, extracting a measurement and updating the filter with it, and 

displaying the current state.  The program uses a Kalman filter toolbox 

written by Kevin Murphy at UC Berkeley (reference web page here).  

Here's the toolbox: kalman_toolbox.tar.gz.  Unzip it, untar it, and put it 

in your Matlab path before running hw2.m.

The three numbered steps to be done are explained below.

(1) Initialize the Kalman filter.  Choose a state representation

(i.e., the homography matrix at a minimum--possibly velocity terms

as well), various noise parameters, deduce the initial state from

the corners the user has selected, and then call kalman_update.

(2) Extract a measurement.  This is the most involved part of the

assignment.  The most straightforward approach is to do a nonlinear 

minimization on the SSD function as described in the paper "Video 

Mosaics for Virtual Environments," by R. Szeliski.  For this you need to 

use the Matlab function lsqnonlin.  A coarse-to-fine search using 

a Gaussian pyramid may be helpful depending on the image speed of the 

tracked object.  Or you may want to try an edge-based approach like 

chamfer matching to get an initial estimate for the nonlinear minimizer.

Experiment and see what works.

Use bilinear interpolation to predict the appearance of the template in 

the image before matching.  

(3) Record and draw the current state.  Write the current state and 

state covariance to a file, and draw an outline around the tracked rectangle 

based on the current homography.

Turn in your Matlab code modifying the hw2.m file, an AVI file

of your best tracking run, the corresponding state estimate log file,

and a short description of your design choices (i.e., Kalman filter

parameters, measurement extraction method, etc.).  Please submit 

a single tarball/zipfile of a directory containing your files named

yourlastname_yourfirstname_hw2.tar.gz.

.