Multi-view 3D face reconstruction with deep recurrent neural networks

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations


Image-based 3D face reconstruction has great potential in different areas, such as facial recognition, facial analysis, and facial animation. Due to the variations in image quality, single-image-based 3D face reconstruction might not be sufficient to accurately reconstruct a 3D face. To overcome this limitation, multi-view 3D face reconstruction uses multiple images of the same subject and aggregates complementary information for better accuracy. Though theoretically appealing, there are multiple challenges in practice. Among these challenges, the most significant is that it is difficult to establish coherent and accurate correspondence among a set of images, especially when these images are captured in different conditions. In this paper, we propose a method, Deep Recurrent 3D FAce Reconstruction (DRFAR), to solve the task ofmulti-view 3D face reconstruction using a subspace representation of the 3D facial shape and a deep recurrent neural network that consists of both a deep con-volutional neural network (DCNN) and a recurrent neural network (RNN). The DCNN disentangles the facial identity and the facial expression components for each single image independently, while the RNN fuses identity-related features from the DCNN and aggregates the identity specific contextual information, or the identity signal, from the whole set of images to predict the facial identity parameter, which is robust to variations in image quality and is consistent over the whole set of images. Through extensive experiments, we evaluate our proposed method and demonstrate its superiority over existing methods.

Original languageEnglish (US)
Title of host publicationIEEE International Joint Conference on Biometrics, IJCB 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages10
ISBN (Electronic)9781538611241
StatePublished - Jan 29 2018
Event2017 IEEE International Joint Conference on Biometrics, IJCB 2017 - Denver, United States
Duration: Oct 1 2017Oct 4 2017

Publication series

NameIEEE International Joint Conference on Biometrics, IJCB 2017


Conference2017 IEEE International Joint Conference on Biometrics, IJCB 2017
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Instrumentation
  • Signal Processing
  • Biomedical Engineering


Dive into the research topics of 'Multi-view 3D face reconstruction with deep recurrent neural networks'. Together they form a unique fingerprint.

Cite this