3D-2D face recognition with pose and illumination normalization

Ioannis A. Kakadiaris, George Toderici, Georgios Evangelopoulos, Georgios Passalis, Dat Chu, Xi Zhao, Shishir K. Shah, Theoharis Theoharis

Research output: Contribution to journalArticlepeer-review

54 Scopus citations


In this paper, we propose a 3D-2D framework for face recognition that is more practical than 3D-3D, yet more accurate than 2D-2D. For 3D-2D face recognition, the gallery data comprises of 3D shape and 2D texture data and the probes are arbitrary 2D images. A 3D-2D system (UR2D) is presented that is based on a 3D deformable face model that allows registration of 3D and 2D data, face alignment, and normalization of pose and illumination. During enrollment, subject-specific 3D models are constructed using 3D+2D data. For recognition, 2D images are represented in a normalized image space using the gallery 3D models and landmark-based 3D-2D projection estimation. A method for bidirectional relighting is applied for non-linear, local illumination normalization between probe and gallery textures, and a global orientation-based correlation metric is used for pairwise similarity scoring. The generated, personalized, pose- and light- normalized signatures can be used for one-to-one verification or one-to-many identification. Results for 3D-2D face recognition on the UHDB11 3D-2D database with 2D images under large illumination and pose variations support our hypothesis that, in challenging datasets, 3D-2D outperforms 2D-2D and decreases the performance gap against 3D-3D face recognition. Evaluations on FRGC v2.0 3D-2D data with frontal facial images, demonstrate that the method can generalize to databases with different and diverse illumination conditions.

Original languageEnglish (US)
Pages (from-to)137-151
Number of pages15
JournalComputer Vision and Image Understanding
StatePublished - Jan 1 2017


  • 3D-2D face recognition
  • 3D-2D model fitting
  • Biometrics
  • Computer vision
  • Face and gesture recognition
  • Illumination normalization
  • Model-based face recognition
  • Object recognition
  • Physically-based modeling

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition


Dive into the research topics of '3D-2D face recognition with pose and illumination normalization'. Together they form a unique fingerprint.

Cite this