Non-rigid structure from motion

People: Rohith MV, Xiaolong Wang and Chandra Kambhamettu

Estimation of non-rigid structure from motion (NRSFM) has often been performed as a linear combination of basis shapes. However, when dealing with scenes containing human articulated motion (especially in presence of clothing), the number of basis shapes precludes accurate results. We model deformation as a combination of articulated and non-rigid surface deformation. We propose a novel algorithm for segmenting motion to find articulated components and a hierarchical NRSFM algorithm which estimates articulated and non-rigid structure separately. Our method attempts to remove the articulated motion from the observation matrix leaving behind only the non-rigid component which can be represented with fewer bases. Results show that our method successfully segments motion in a variety of cases and structure reconstruction is improved using the hierarchical framework.

Illustration of one reconstructed face based on 2D points

Related publications

• Rohith MV, Chandra Kambhamettu. Estimation and utilization of articulations in recovering non-rigid structure from motion using motion subspaces, Joint ACM Workshop on Human Gesture and Behavior Understanding (J-HGBU'11), ACM Multimedia 2011.
• T. Srinark, C. Kambhamettu and M. Stone, "A Hierarchical Method for 3D Rigid Motion Estimation", Asian Conference on Computer Vision, pp. 791-800, Vol. 2, Hyderabad, India 2006. (Acceptance Rate: 128/500=25%)
• Dimitris Metaxas, Chandra Kambhamettu and Dmitry Goldgof, editorial, special issue on Articulated and nonrigid motion, "Image and Vision Computing (IVC)", 2006.
• Min Li, Chandra Kambhamettu and Maureen Stone, "Nonrigid motion recovery for 3D surfaces", "Image and Vision Computing (IVC)", pp. 250-261, Vol. 25, Issue 3, 2006.
• Min Li, Chandra Kambhamettu and Maureen Stone, "Spline-based Motion Recovery for 3D Surfaces Using Nonrigid Shape Properties", IEEE workshop on Articulated and Nonrigid Motion 2004, in conjunction with CVPR'04, Washington, DC, USA, June 2004.
• Pavel Laskov and Chandra Kambhamettu, "Curvature-based Algorithms for Non-rigid Motion and Correspondence Estimation", IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 25, No. 11, pp. 1349-1354, Nov. 2003.
• Chandra Kambhamettu, Dmitry Goldgof, Matthew He, and Pavel Laskov, "3D Nonrigid Motion Analysis Under Small Deformations", "Image and Vision Computing (IVC)", Vol. 21, No. 3, pp. 229-245, March 2003. (12th in the 25 most downloaded articles of IVC: January - December 2003) [ref]
• Pavel Laskov and Chandra Kambhamettu, "Curvature-Based Algorithms for Non-Rigid Motion and Correspondence Estimation", Fifth Asian Conference on Computer Vision (ACCV2002), Vol I, pp. 19-26, Melbourne, Australia, January, 2002.
• Pavel Laskov, Chandra Kambhamettu, "Comparison of 3D Algorithms for Non-rigid Motion and Correspondence Estimation", Proceedings of the 12th British Machine Vision Conference (BMVC2001), pp. 273-282, Manchester, UK, September 2001.
• Lin Zhou and Chandra Kambhamettu, "Hierarchical Structure and Nonrigid Motion Recovery from 2D Monocular Views", IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR2000), pp. II-752-759, Hilton Head Island, South Carolina, June 2000.
• Lin Zhou, Chandra Kambhamettu and Dmitry B. Goldgof, "Fluid Structure and Motion Analysis from Multi-spectrum 2D Cloud Image Sequences", IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR2000), pp. II-744-751, Hilton Head Island, South Carolina, June 2000.
• Lin Zhou, Chandra Kambhamettu, and Dmitry B. Goldgof, "Extracting Nonrigid Motion and 3D Structure of Hurricanes from Satellite Image Sequences without Correspondences", IEEE Conference on Computer Vision and Pattern Recognition (CVPR1999), Vol. II, pp. 280-285, Fort Collins, CO, June 1999.
• Lin Zhou, Chandra Kambhamettu, and Dmitry B. Goldgof, "Structure and Nonrigid Motion Analysis of Satellite Cloud Images", Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP1998), pp. 285-291, New Delhi, India, Dec. 21-23, 1998.
• Ram Balasubramanian, Dmitry B. Goldgof and Chandra Kambhamettu, "Tracking of Nonrigid Motion and 3D Structure from 2D Image Sequences without Correspondences", International Conference on Image Processing, pp. 933-937, vol 1, Chicago, IL, October 4-7, 1998.
• Chandra Kambhamettu, Dmitry B. Goldgof, Demetri Terzopoulos and Thomas S. Huang, "Nonrigid Motion Analysis", Handbook of PRIP: Computer vision (edited by Tzay Young), Vol 2, pp. 405-430, Academic Press, San Diego, CA 1994, also re-printed in "Deformable Models in Medical Image Analysis", edited by Ajit Singh, Dmitry Goldgof and Demetri Terzopoulos, pp. 270-284, 1998.
• Sanjoy K. Mishra, Chandra Kambhamettu, Dmitry B. Goldgof and Thomas S. Huang, "Curvature-Based Nonrigid Motion Analysis from Three-Dimensional Correspondences", International Journal of Imaging Systems and Technology, Vol 4, pp. 214-225, 1993.
• Chandra Kambhamettu and Dmitry B. Goldgof, "Point Correspondence Recovery in Nonrigid Motion", Proceedings of IEEE conference on Computer Vision and Pattern Recognition (CVPR1992), pp. 222-227, 1992.