TY - JOUR
T1 - 3D-2D face recognition with pose and illumination normalization
AU - Kakadiaris, Ioannis A.
AU - Toderici, George
AU - Evangelopoulos, Georgios
AU - Passalis, Georgios
AU - Chu, Dat
AU - Zhao, Xi
AU - Shah, Shishir K.
AU - Theoharis, Theoharis
N1 - Funding Information:
This research was funded in part by the Office of the Director of National Intelligence (ODNI) and by the Intelligence Advanced Research Projects Activity (IARPA) through the Army Research Laboratory (ARL) and by the University of Houston (UH) Eckhard Pfeiffer Endowment Fund. All statements of fact, opinion or conclusions contained herein are those of the authors and should not be construed as representative of the official views or policies of IARPA, the ODNI, the U.S. Government, or UH. The authors would like to thank (i) S. Zafeiriou and G. Tzimiropoulos for providing the score normalization code, (ii) L. Chen and LIRIS Lab for sharing their results on UHDB11, and (iii) P. Dou for performing selected experiments in the paper.
Publisher Copyright:
© 2016
PY - 2017/1/1
Y1 - 2017/1/1
N2 - In this paper, we propose a 3D-2D framework for face recognition that is more practical than 3D-3D, yet more accurate than 2D-2D. For 3D-2D face recognition, the gallery data comprises of 3D shape and 2D texture data and the probes are arbitrary 2D images. A 3D-2D system (UR2D) is presented that is based on a 3D deformable face model that allows registration of 3D and 2D data, face alignment, and normalization of pose and illumination. During enrollment, subject-specific 3D models are constructed using 3D+2D data. For recognition, 2D images are represented in a normalized image space using the gallery 3D models and landmark-based 3D-2D projection estimation. A method for bidirectional relighting is applied for non-linear, local illumination normalization between probe and gallery textures, and a global orientation-based correlation metric is used for pairwise similarity scoring. The generated, personalized, pose- and light- normalized signatures can be used for one-to-one verification or one-to-many identification. Results for 3D-2D face recognition on the UHDB11 3D-2D database with 2D images under large illumination and pose variations support our hypothesis that, in challenging datasets, 3D-2D outperforms 2D-2D and decreases the performance gap against 3D-3D face recognition. Evaluations on FRGC v2.0 3D-2D data with frontal facial images, demonstrate that the method can generalize to databases with different and diverse illumination conditions.
AB - In this paper, we propose a 3D-2D framework for face recognition that is more practical than 3D-3D, yet more accurate than 2D-2D. For 3D-2D face recognition, the gallery data comprises of 3D shape and 2D texture data and the probes are arbitrary 2D images. A 3D-2D system (UR2D) is presented that is based on a 3D deformable face model that allows registration of 3D and 2D data, face alignment, and normalization of pose and illumination. During enrollment, subject-specific 3D models are constructed using 3D+2D data. For recognition, 2D images are represented in a normalized image space using the gallery 3D models and landmark-based 3D-2D projection estimation. A method for bidirectional relighting is applied for non-linear, local illumination normalization between probe and gallery textures, and a global orientation-based correlation metric is used for pairwise similarity scoring. The generated, personalized, pose- and light- normalized signatures can be used for one-to-one verification or one-to-many identification. Results for 3D-2D face recognition on the UHDB11 3D-2D database with 2D images under large illumination and pose variations support our hypothesis that, in challenging datasets, 3D-2D outperforms 2D-2D and decreases the performance gap against 3D-3D face recognition. Evaluations on FRGC v2.0 3D-2D data with frontal facial images, demonstrate that the method can generalize to databases with different and diverse illumination conditions.
KW - 3D-2D face recognition
KW - 3D-2D model fitting
KW - Biometrics
KW - Computer vision
KW - Face and gesture recognition
KW - Illumination normalization
KW - Model-based face recognition
KW - Object recognition
KW - Physically-based modeling
UR - http://www.scopus.com/inward/record.url?scp=84979671321&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84979671321&partnerID=8YFLogxK
U2 - 10.1016/j.cviu.2016.04.012
DO - 10.1016/j.cviu.2016.04.012
M3 - Article
AN - SCOPUS:84979671321
VL - 154
SP - 137
EP - 151
JO - Computer Vision and Image Understanding
JF - Computer Vision and Image Understanding
SN - 1077-3142
ER -