Angular Domain Reconstruction of Dynamic 3D Fluid Surface

We present a novel and simple computational imaging solution to robustly and accurately recover 3D dynamic fluid surfaces. Traditional specular surface reconstruction schemes place special patterns (checkerboard or color patterns) beneath the fluid surface to establish point-pixel correspondences. However, point-pixel correspondences alone are insufficient to recover surface normal or height and they rely on additional constraints to resolve the ambiguity. In this paper, we exploit using Bokode - a computational optical device that emulates a pinhole projector - for capturing ray-ray correspondences which can then be used to directly recover the surface normals. We further develop a robust feature matching algorithm based on the Active-Appearance Model to robustly establishing ray-ray correspondences. Our solution results in an angularly sampled normal field and we derive a new angular-domain surface integration scheme to recover the surface from the normal fields. Specifically, we reformulate the problem as an over-constrained linear system under spherical coordinate and solve it using Singular Value Decomposition. Experiments results on real and synthetic surfaces demonstrate that our approach is robust and accurate, and is easier to implement than state-of-the-art multi-camera based approaches.

Bokode-based Acquisition System




Top: a schematic showing the Incident-Exit-Ray (IER) correspondences. Each point on the Bokode pattern maps to a beam of parallel rays. These rays are refracted by fluid surface and gathered by the viewing camera at a pixel. Bottom: our experimental setup. We construct a Bokode using a flashlight, a diffuser, a high-resolution pattern and webcam lens. We also use an auxiliary bi-convex spherical lens to collect more lights refracted by the fluid surface.


A block diagram that shows the pipeline of our Bokode based fluid surface reconstruction framework.


Results on a synthetic sinusoid wave (top two rows) and on a Helmholtz wave (bottom two rows). We show the cropped Bokode pattern used in reconstruction. In particular, column 2 are sampled surface normals under spherical coordinate in respect to and and we further compute normal maps in Cartesian coordinate and compare with the ground truth ones to demonstrate the accuracy of our method as shown in column 6 and 7 (Click the image for full resolution image). The complete sequences can be found in the video below.


Results on two sets of real data (a perturbed wave and a ring wave). From left to right: we show the captured image with matched features, the sampled normal under spherical coordinate, the reconstructed surface, and the surface normal field computed from the reconstructed surface (Click the image for full resolution image). The complete sequences can be found in the video below.

Video 1: Synthetic waves result


Video 2: Real data results