Face Recognition in Videos

Aim

To perform effective face recognition in a still-to-video based face recognition scenario. We aim to use the temporal and spatial characteristics of the face images from the video frames.

Challenges

  • Matching high resolution still images with low resolution video frames.
  • Pose, illumination changes in the face, and occlusion of the face region.

Proposes Solution

  • Capture spatial information of face images using graphs constructed using the feature points of the face and labeled using feature descriptors.
  • Capture temporal information of face images using Adaptive Graph Appearance Models (AGAM) using the graph set of a subject.

Abstract

In this work, we present a novel graph, sub-graph and super-graph based face representation which captures the facial shape changes and deformations caused due to pose changes and use it in the construction of an adaptive appearance model. A sub-graph and super-graph is extracted for each pair of training graphs of an individual and added to the graph model set and used in the construction of appearance mode. The spatial properties of the feature points are effectively captured using the graph model set. the adaptive graph appearance model constructed using the graph model set captures the temporal characteristics of the video frames by adapting the model with the results of recognition from each frame during the testing stage. The graph model set and the adaptive appearance model are used in the two stage matching process, and are updated with the sub-graphs and super-graphs constructed using the graph of the previous frame and the training graphs of an individual. The results indicate that the performance of the system is improved by using sub-graphs and super-graphs in the appearance model.

High resolution images used for training purposes

Close range video frames used in our experiments

Moderate range video frames used in our experiments

Related Publications