Facial reconstruction is a highly active field of research, and is applied frequently in the areas of facial rehabilitation [1] and forensic facial reconstruction [2]. Academic efforts in the field have been geared towards improving the accuracy and efficiency of the 3D models, which are generated from the input 2D images containing distinguishable human facial features with enough intra- variety. Such images allow researchers to generalise an average layout of features that is attributed to a generic human face. However, research in this field is invariably having to contend with the limitations inherent to 2D images, which slow down research progress. These limitations are largely due to: occlusions, the restricted variety in the features of a human face, different facial poses, and even partial views.
This project offers a system, outlined in Figure 1, consisting of the creation of specific regression trees for homogenizing human faces, and then applying the regressed face to an input 2D image to be reconstructed in 3D. These trees contain placements of a pre-defined set of landmarks that are found within the input 2D face, where applicable. A 3D morphable model is a collection of 3D surfaces, in which all vertices in every surface map correspond to the vertices of the different surfaces found within that same morphable model [3]. These points are then applied to the 3D morphable model which, after a series of transformations, outputs a reconstructed 3D model that would be adjusted to the specific features found in the input image.
The evaluation process used here involves determining mathematically how one could compare different models from the same person. This was achieved through the use of mesh-analysis metrics such as the 75th percentile, root mean square and geometric mean [5]. The result is a comprehensive 3D reconstruction system as shown in Figure 2, which generates a 3D model with its corresponding texture through landmark extraction from a single 2D portrait image.
References/Bibliography:
[1] G. Storey, R. Jiang and A. Bouridane, “Role for 2D image generated 3D face models in the rehabilitation of facial palsy,” Healthcare Technology Letters, vol. 4, (4), pp. 145-148, 2017.
[2] Y. Hu et al, “Automatic 3D reconstruction for face recognition,” in Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings. 2004, pp. 843-848.
[3] V. Blanz and T. Vetter, “A morphable model for the synthesis of 3D faces,” in Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 1999.
[4] C. Cao et al, “FaceWarehouse: A 3D Facial Expression Database for Visual Computing,” IEEE Transactions on Visualization and Computer Graphics, vol. 20, (3),2014, pp. 413-425.
[5] F. Zuzana, “Comparison and Analysis of Multiple 3D Shapes”, MSc, Masarykova Univerzita, Brno, Czech Republic. 2015.
Student: Matthew Caruana
Course: B.Sc. IT (Hons.) Software Development
Supervisor: Dr. Joseph Vella