We present a novel stereo imaging technique called Dual Focus Stereo Imaging or DFSI. DFSI uses a pair of images captured from different viewpoints, and at different focuses but with identical wide aperture size. Each image in an DFSI pair exhibits different defocus blur and the two images form a defocused stereo pair. To model defocus blur, we introduce a defocus kernel map (DKM) that computes the size of the blur disk at each pixel. We derive a novel disparity defocus constraint for computing the DKM in DFSI, and integrate DKM estimation with disparity map estimation to simultaneously recover both maps. We show that the recovered DKMs provide useful guidance for segmenting the in-focus regions. We demonstrate using DFSI in a variety of imaging applications including low-light imaging, automatic defocus matting, and multi-focus photomontage.

Estimating Defocus Kernel Map
Our DFSI method uses a pair of images captured at different viewpoints and focuses but with the same wide aperture size. We call these images dual focus stereo pair or DFSP. Each image in DFSP has shallow depth of field, and the region that appears in focus in one image will appear blurry in the other image. As we know, an in-focus point maps to a single point, and an out-of-focus point focuses either in front of or behind the image plane and therefore splats a circle of confusion on the image plane. We use DKM to model this defocus blur kernel for each pixel on the image plane.
To estimate the DKMs, we first extract the salient features in each image and use them to estimate camera parameters. We then integrate the defocus kernel map estimation with the disparity map estimation and refine the salient maps. Once we obtain the initial estimation to the DKM, we interactively refine the salient features, the camera parameters, and the DKM.

Application : Low-light Imaging
Here we show the pipeline of our low-light imaging. We first recover the DKM and the disparity map using the DFSP pair, and then use the DKM to segment the in-focus regions. The last step is to warp and fuse an all-focus image. We model the foreground segmentation as a markov random field. We can automatically segment out the in-focus object using DKM, since DKM models the blurriness of each pixel in the image. Because DKM is not accurate enough for direct warping, we design a snake-based method to warp the in-focus regions to the other image. The details of our algorithm can be found in the proposal.

Application : Automatic Defocus Matting
Next, we demonstrate how to use the recovered DKMs from DFSP pairs to automatically extract alpha mattes. For trimap-based matting schemes, the quality of the alpha matte relies heavily on the estimation accuracy of the unknown region. Here we present a new scheme for generating the trimap by exploiting edge continuities. The key observation here is that, for fuzzy objects, strong edges along the foreground boundary should be considered as part of the unknown region. Therefore, we grow the unknown region along the edges directions. Once we obtain the trimap, we use Robust Matting [41] to estimate the alpha matte, the foreground, and the background.

Application : Multi-focus Photomontage
Finally, we apply DFSI for creating multi-focus photomontage which virtually focuses at multiple scene depths. Given a DFSP pair, we start by segmenting the in-focus region using the recovered DKMs. In low-light imaging, we directly warp the segmented in-focus region from one image to the other. We assume that scene objects lie on different depth layers and rely on Snake for locating the occlusion boundary. For multi-focus photomontage, scene depth may vary smoothly. Therefore, directly warping the foreground region will lead to sudden changes of blurriness across the warping boundary. To resolve this issue, we compute the DKM for each image in the DFSP pair to determine the defocus blur kernel size near the warped boundaries. We then blur the warped in-focus boundary region accordingly to maintain smooth transitions.


PDF