Bidirectional relighting for 3D-aided 2D face recognition

Introduction Face recognition is one of the most widely researched topics in computer vision because of a wide variety of applications that require identity management. Most existing face recognition studies are focused on two-dimensional (2D) images with nearly frontal-view faces and constrained illumination. However, 2D facial images are strongly affected by varying illumination conditions and changes in pose. Thus, although existing methods are able to provide satisfactory performance under constrained conditions, they are challenged by unconstrained pose and illumination conditions. FRVT 2006 explored the feasibility of using three-dimensional (3D) data for both enrollment and authentication (Phillips et al. 2007). The algorithms using 3D data have demonstrated their ability to provide good recognition rates. For practical purposes, however, it is unlikely that large scale deployments of 3D systems will take place in the near future because of the high cost of the hardware. Nevertheless, it is not unreasonable to assume that an institution may want to invest in a limited number of 3D scanners, if having 3D data for enrollment can yield higher accuracy for 2D face authentication/identification. In this respect we have developed a face recognition method that makes use of 3D face data for enrollment while requiring only 2D data for authentication. During enrollment, different from the existing methods (e.g., Blanz and Vetter 2003) that use a 2D image to infer a 3D model in the gallery, we use 2D+3D data (2D texture plus 3D shape) to build subject-specific annotated 3D models.