Self-calibrated, multi-spectral photometric stereo for 3D face capture


This paper addresses the problem of obtaining 3d detailed reconstructions of human faces in real-time and with inexpensive hardware. We present an algorithm based on a monocular multi-spectral photometric-stereo setup. This system is known to capture high-detailed deforming 3d surfaces at high frame rates and without having to use any expensive hardware or synchronized light stage. However, the main challenge of such a setup is the calibration stage, which depends on the lights setup and how they interact with the specific material being captured, in this case, human faces. For this purpose we develop a self-calibration technique where the person being captured is asked to perform a rigid motion in front of the camera, maintaining a neutral expression. Rigidity constrains are then used to compute the head's motion with a structure-from-motion algorithm. Once the motion is obtained, a multi-view stereo algorithm reconstructs a coarse 3d model of the face. This coarse model is then used to estimate the lighting parameters with a stratified approach: In the first step we use a RANSAC search to identify purely diffuse points on the face and to simultaneously estimate this diffuse reflectance model. In the second step we apply non-linear optimization to fit a non-Lambertian reflectance model to the outliers of the previous step. The calibration procedure is validated with synthetic and real data.

Publication DOI:
Divisions: ?? 50811700Jl ??
Additional Information: The original publication is available at
Uncontrolled Keywords: Software,Artificial Intelligence,Computer Vision and Pattern Recognition
Publication ISSN: 1573-1405
Last Modified: 27 May 2024 07:10
Date Deposited: 20 Dec 2012 10:57
Full Text Link: http://www.spri ... 66603qv30j4823/
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
PURE Output Type: Article
Published Date: 2012-03
Authors: Vogiatzis, George (ORCID Profile 0000-0002-3226-0603)
Hernández, Carlos



Version: Accepted Version

Export / Share Citation


Additional statistics for this record