An Interactive System for Correcting Distortions in Multiperspective Images

                                  By Liang Wei           APR, 4th, 2006

Multi-perspective Images capture multiple perspectives into a single image, thus it is inherent to deviations from single-perspective views. Here I propose an interactive system for correcting multi-perspective distortions, such that portions of those multiperspective image will be rectified to appear 'perspective'. This system can be useful for producing cell animations, in that a large multiperspective image is known and we only wish to correct a small window of it to make it locally appear ' perspective'. The key concepts behind the system is that every multiperspective image can be described by a collection of rays (ie. Plenoptic Function), with fixed rays, the final image we see is a specific sampling of those rays on retina plane, so by changing the orientation and shape of the sampling plane, we can find an optimal specification of the sampling plane that will yield a 'least distorted' image similar to those obtained by central projection model.

For implementation, I build a cost function based on [1] that measures the error between user-specified image point that need to be corrected and the corrected position of those image points(both of them are specified via user interaction). The following images are the results produced by our system.

The first two images are a catacaustics produced by reflection seen on a spherical mirror. On the left is the source image. An rectangular window is specified by user indicating which region needs to be corrected. This region is shown in the middle view which is composed of user defined feature point for later correction. The red point denotes the source position of  distorted point, while blue point is a one-to-one correspondence of red point denoting where the user want the red point to be in an undistorted image. The rightmost view shows the resampled image after our optimization procedure.

Figure1 Distortion Correction on a Catacaustic Mirror.

The following image shows distortion produced by a reflecting planar mirror with randomly perturbed normal.

Figure 2 Distortion Correction on Planar Mirror with Perturbed Normal

The following two images show the flexibility of our distortion correction system. It can be applied to reflection distortions produced by any arbitrary mesh surface. In this example, we used a horse mesh model to produce reflection distortion, and corrected portion of this distorted image via our optimization procedure.

Figure 3 Distortion Correction on arbitrary mesh models

[1] Jingyi Yu and Leonard McMillan, “Multiperspective Projection and Collineation”, 10th IEEE International Conference on Computer Vision (ICCV), Beijing, P. R. China, Oct., 2005