Joint prototype and metric learning for set-to-set matching: Application to biometrics

In this paper, we focus on the problem of image set classification. Since existing methods utilize all available samples to model each image set, the corresponding time and storage requirements are high. Such methods are also susceptible to outliers. To address these challenges, we propose a method that jointly learns prototypes and a Mahalanobis distance. The prototypes learned represent the gallery image sets using fewer samples, while the classification accuracy is maintained or improved. The distance learned ensures that the notion of similarity between sets of images is reflected more accurately. Specifically, each gallery set is modeled as a hull spanned by the learned prototypes. The prototypes and distance metric are alternately updated using an iterative scheme. Experimental results using the YouTube Face, ETH-80, and Cambridge Hand Gesture datasets illustrate the improvements obtained.