Feng Li, Yu Ji and Jingyi Yu |
ABSTRACT | SETUP | PIPELINE | ALGORITHM | RESULTS |
|
Our goal is to combine the benefits of different types of cameras (with respect to aperture, shutter, resolution, and spectrum) to construct a low-light imaging system. Figure shows our proposed hybrid camera system: it consists of one Pointgrey Grasshopper high speed monochrome (HS-M) cameras (top-left), one Flea2 high resolution color (HR-C) camera (top-right), and two Pointgrey Flea2 high resolution monochrome (HRM) cameras (bottom). All cameras are equipped with the same Rainbow 16mm Cmount F1.4 lens. We mount the four cameras on a T-slotted aluminum grid, which is then mounted on two conventional tripods for indoor applications. To deal with the long working range of outdoor applications, we also build a giant “tripod” from a 6 foot step ladder to hold the camera array grid, as shown in Figure(right). |
Pipeline |
Our system pipeline. We first
preprocess the HS-M images to improve the
brightness of dark regions, and then apply BM3D
denoising algorithm to partially remove sensor
noise. Next we find low-noise patch priors from
HR-M images for full-spectrum denoising of HS-M
and HRC images. Finally, we design an
alternating minimization algorithm to denoise
the local contrast enhanced HS-M images and
reconstruct high quality color images from the
HR-C inputs. |
Hybrid Denoising |
we model image noise as white, zero-mean Gaussian noise, with a standard deviation σ, and assume the noisy image I can be decomposed into a latent image and noise η: | ||
(1) | ||
where x is the pixel index. | ||
Multi-view Block Matching | ||
we present a multi-view block matching (MVBM) technique for patch-based image denoising. The goal of MVBM is to find similar patches from HR-M image pairs for each patch in HS-M (or HR-C) images. | ||
Patch-Based Denoising | ||
Based on the image noise formulation Eq.(1), we design a new full-spectrum spatial regularization that uses patches from the HR-M images to add back the high frequency details for HS-M images. We formulate the full-spectrum denoising as: | ||
(2) | ||
where and are two positive weights used to control the strength of TV and spatial prior regularization respectively; || · ||1 represents the ℓ1-norm. is obtained from our MVBM method. We choose ℓ2-norm for the data fidelity term since noise is assumed to be additive and Gaussian. We also add a Total Variation(TV) regularization term to the data fidelity term to make the reconstruction process more reliable. | ||
Iterative Optimization | ||
Compared with Tikhonov-like regularization, the minimization problem formulated by Eq.(2) is computationally expensive to solve due to nonlinearity and non-differentiability of the regularization terms. we present an effective solution: we introduce two auxiliary vectors u, v ∈ to the objective function in Eq.(2) and then reformulate this denoising problem equivalently as an optimization problem with linear constraints | ||
(3) | ||
To solve for the Eq.(3), we apply the alternating minimization method to the augmented Lagrangian function of Eq.(3) using the following iterative scheme: | ||
(4a) (4b) (4c) (4d) (4e) |
||
where is the augmented Lagrangian function of Eq.(3) defined by | ||
(5) | ||
where and are two vectors of Lagrange multipliers and , ∈ , and are penalty parameters and γ is the step length for updating and . By iteratively updating the Lagrange multipliers and , the solution to Eq.(4a)-(4e) will eventually converge to the one to Eq.(3). | ||
Results |
Synthetic Scene Denoising Results | ||||||||
Input Image | BM3D Result | Our Result | ||||||
Synthetic Scene Debluring Results | ||||||||
Indoor Scene Denoising Results | ||||||||
Input Image | High-Resolution Left | High-Resolution Right | ||||||
Pre-Processed Image | BM3D Result | Our Result | ||||||
Indoor Scene Debluring results | ||||||||
Outdoor Scene Denoising Results | ||||||||
Input Image | BM3D Result | Our Result | ||||||
Video 1: Indoor Scene Denoising | |
Left: Pre-Processed Input | Right: Our Denoised Result |
Video 2: Outdoor Scene Denoising | |
Left: Pre-Processed Input | Right: Our Denoised Result |
|
This project is supported by the Air Force Office of Scientific Research (AFOSR) Young Investigator Program (YIP). We would like to thank Dr. Sjogren, our Program Director, for his invaluable guidance on this project. |