A Groupwise Multilinear Correspondence Optimization for 3D Faces

ICCV 2015

Timo Bolkart Stefanie Wuhrer
Saarland University, MMCI INRIA Rhône-Alpes

Multilinear face models are widely used to model the space of human faces with expressions. For databases of 3D human faces of different identities performing multiple expressions, these statistical shape models decouple identity and expression variations. To compute a high-quality multilinear face model, the quality of the registration of the database of 3D face scans used for training is essential. Meanwhile, a multilinear face model can be used as an effective prior to register 3D face scans, which are typically noisy and incomplete. Inspired by the minimum description length approach, we propose the first method to jointly optimize a multilinear model and the registration of the 3D scans used for training. Given an initial registration, our approach fully automatically improves the registration by optimizing an objective function that measures the compactness of the multilinear model, resulting in a sparse model. We choose a continuous representation for each face shape that allows to use a quasi-Newton method in parameter space for optimization. We show that our approach is computationally significantly more efficient and leads to correspondences of higher quality than existing methods based on linear statistical models. This allows us to evaluate our approach on large standard 3D face databases and in the presence of noisy initializations.

Paper Gradient Video
PDF (5 MB) PDF (1 MB) AVI (20 MB)

To facilitate evaluating the model for different applications, we make the optimized multilinear face models from our submission available for non-commercial research purposes. We ask that you respect the conditions of using the models, which are detailed in the readme.pdf files provided with the models.
We provide
  • the optimized multilinear face models for
    • the Bosphorus subset (65 identities in 7 expressions) here1,
    • the Bosphorus subset (39 identities in 26 action units) here1,
    • the BU-3DFE subset (50 identities in 7 expressions) here2, and
    • the full BU-3DFE database (100 identities in 25 expression) here2.
  • an example framework that shows how to use a multilinear face model to reconstruct unregistered face scans here, and
  • the multilinear correspondence optimization code here.

Utilized Face Data
1 This face model was computed using the Bosphorus face database. If you use this statistical model in your publications, please also reference the following work.
  • A. Savran, N. Alyüz, H. Dibeklioglu, O. Çeliktutan, B. Gökberk, B. Sankur, and L. Akarun
    Bosphorus Database for 3D Face Analysis
    The First COST 2101 Workshop on Biometrics and Identity Management (BIOID), 2008
2 This face model was computed using the BU-3DFE face database. We wish to thank Lijun Yin for making this database available and for giving us permission to make the statistical models available for non-commerical research purposes. If you use this statistical model in your publications, please also reference the following work..
  • L. Yin, X. Wei, Y. Sun, J. Wang, M. Rosato
    A 3D Facial Expression Database For Facial Behavior Research
    International Conference on Automatic Face and Gesture Recognition, 2006, pages 211-216

We thank Arnur Nigmetov for help with the comparison of the different tensor decompositions, and Alan Brunton, and Michael Wand for helpful discussions. This work has been partially funded by the German Research Foundation (WU 786/1-1, Cluster of Excellence MMCI).