submitted by Liang Wei May, 26th, 2006
In my final project, I implemented a real-time stereo matching system on programmable graphics hardware, using both window-based and dynamic programming approaches. For window based I approach, I used a real-time stereo-pair capture device -- Bumblebee Camera designed by PointGrey Research, to produce a dense real-time disparity map. The following sections contain the source code and resulting images of my program.
1) Window-Based Approach
For window-based approach, I used the method proposed by Ruigang et al.[1]. For implementation, I first computed the absolute difference cost of stereo pairs under different disparity hypothesis using fragment shader, then aggregate the cost over 4 by 4 window by enabling bilinear interpolation and sampling the subpixel of the computed texture image from the previous step. Finally assign this cost as the Depth in Z-buffer, and assign disparity as the pixel color, so OpenGL will automatically draw those disparities associated with the smallest cost for every pixel in the original image.
The following are some results for different data sets found in Middlebury Stereo Vision web site.
For all these images the left is ground truth, the right is disparity computed by my program.
The tsukuba data sets
The Cone data sets:
The Saw data sets:
The running time of my program is similar as that reported in Ruigang's paper. For Tsukuba data sets, if the search range is 0-15, one frame of disparity computation takes only 15ms on a Pentium 4 3.20GHz computer with NVidia Geforce 6800 GPU.
This is the link to my source code: RealStereo.rar (Note: the program uses NVidia SDK and only NVidia GPU is supported)
2) Dynamic Programming Approach
I have also implemented a Dynamic Programming based stereo matching program on GPU. Because of the searching and table setup process in Dynamic Programming can not be easily implemented on GPU, which is designed for parallel computing. I only implement cost computation part of this method on GPU, which is the same as window based approach. For dynamic programming, I read back from GPU the computed costs, and run dynamic programing algorithm on CPU. Though it is slower than windows-based method, The overall disparity quality is better. But there are still obvious
streaking effects that can be seen in horizontal direction. as the following image shows
The following pictures shows DP's result on Saw data sets
References:
[1] Ruigang Yang and Marc Pollefeys, Multi-Resolution Real-Time Stereo On Commodity Graphics Hardware, CVPR 2003.