面向识别的人脸三维信息估计

doi:10.7523/j.ucas.2021.0058

摘要/Abstract

摘要： 三维人脸识别相较当前流行的二维人脸识别具有识别精度高、防伪强度高的优势，代表了人脸识别的发展方向。因三维人脸的采集成本高昂，目前三维人脸识别无法像二维人脸识别方法那样直接借助海量人脸数据建立、优化识别算法。如何准确、高效地获取三维人脸的增广训练数据是推动三维人脸识别应用发展中最为迫切的问题。当前学术界的大量文献主要聚焦在如何得到更好的三维人脸重建可视化效果，而没有对后续的识别任务多加考虑，以至于用这些重建图像训练的三维人脸识别算法难以达到令人满意的效果。针对这一问题，提出一种面向识别的人脸三维信息估计方法。与一般方法不同，该方法直接在信息估计与后续识别之间建立互动桥梁：在人脸三维信息估计的训练过程中直接借助相应的识别网络来督促提高三维人脸信息的估计效果。为此，先构造一种三维人脸信息表示——深度-表面法向量图（DN图），然后通过真实三维数据集训练一个人脸CycleGAN模型，该模型用以学习一种从二维人脸到DN图的保留身份信息的映射并用U-Net网络的形式对其进行表示。在5个数据集上与其他方法进行了比较实验，其中在ND-2006数据集中的提升尤为显著，提高31.8%。此外，进行了数据增广下性能提升的对比实验，在同样条件下做数据增广，基于人脸CycleGAN的增广方法性能上的提升更加明显，在CASIA 3D数据集上的提高幅度达14.9%。

关键词: 人脸三维信息估计, 循环生成对抗网络, 人脸DN图, 身份保留损失, 参数预训练

Abstract: 3D face recognition has the advantages of recognition accuracy and anti-counterfeiting strength over the popular 2D face recognition, and represents the development direction of face recognition. Due to the high cost of 3D facial acquisition, 3D facial recognition can not establish and optimize face recognition algorithms directly rely on massive face data as 2D facial recognition methods. How to obtain augmented training data for 3D faces accurately and efficiently is the most pressing problem in driving the development of 3D face recognition applications. A large amount of related research focuses on how to get better 3D face reconstruction visualization,but does not give much consideration to the subsequent recognition task,so that the recognition accuracy of 3D face recognition algorithms trained with these reconstructed images is much lower than expected. To address this problem, a recognition-oriented method for estimating facial 3D information is proposed. Different from the general method, this method directly builds an interactive bridge between information estimation and subsequent recognition:during the training process of 3D face information estimation, it directly bases on the corresponding recognition network to supervise and improve the estimation effect of 3D face information. For this purpose, we first construct a 3D face information representation, the depth-surface normal vector map (DN map), and then train a facial CycleGAN model with a real 3D dataset to learn a mapping from 2D face to DN map with preserved identity information and represent it in the form of U-Net network. Experiments are conducted on five datasets to compare with other methods, and the improvement is particularly significant in the ND-2006 dataset, with an improvement of 31.8%. In addition, experiments on performance improvement under data augmentation are conducted, here the performance improvement of the augmentation method based on the facial CycleGAN is more obvious under the same conditions of data augmentation, with a maximum improvement of 14.9% on the CASIA 3D dataset.

Key words: facial 3D information estimation, cycle generative adversarial networks, facial DN map, identity preserving loss, parameter pre-train

中图分类号:

TP391

陈汉钦, 秦进, 赵彤, 阎瑶. 面向识别的人脸三维信息估计[J]. 中国科学院大学学报, 2023, 40(2): 268-279.

CHEN Hanqin, QIN Jin, ZHAO Tong, YAN Yao. Recognition-oriented facial 3D information estimation[J]. Journal of University of Chinese Academy of Sciences, 2023, 40(2): 268-279.

参考文献

[1] Bowyer K W, Chang K, Flynn P. A survey of approaches and challenges in 3D and multi-modal 3D +2D face recognition[J]. Computer Vision and Image Understanding, 2006, 101(1):1-15.
[2] Deng J K, Guo J, Xue N N, et al. ArcFace:additive angular margin loss for deep face recognition[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:4685-4694.
[3] Parkhi O M, Vedaldi A, Zisserman A. Deep face recognition[C]//Procedings of the British Machine Vision Conference 2015. Swansea. British Machine Vision Association, 2015:1-12.
[4] Schroff F, Kalenichenko D, Philbin J. FaceNet:a unified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 2015:815-823.
[5] Taigman Y, Yang M, Ranzato M, et al. DeepFace:closing the gap to human-level performance in face verification[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. June 23-28, 2014, Columbus, OH, USA. IEEE, 2014:1701-1708.
[6] Faltemier T C, Bowyer K W, Flynn P J. Using a multi-instance enrollment representation to improve 3D face recognition[C]//2007 First IEEE International Conference on Biometrics:Theory, Applications, and Systems. September 27-29, 2007, Crystal City, VA, USA. IEEE, 2007:1-6.
[7] Dou P F, Shah S K, Kakadiaris I A. End-to-end 3D face reconstruction with deep neural networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI. IEEE, 2017:1503-1512.
[8] Feng Y, Wu F, Shao X H, et al. Joint 3D face reconstruction and dense alignment with position map regression network[C]//Computer Vision-ECCV 2018, 2018:557-574. DOI:10.1007/978-3-030-01264-9_33.
[9] Richardson E, Sela M T, Kimmel R. 3D face reconstruction by learning from synthetic data[C]//2016 Fourth International Conference on 3D Vision (3DV). October 25-28, 2016, Stanford, CA, USA. IEEE, 2016:460-469.
[10] Zulqarnain Gilani S, Mian A. Learning from millions of 3D scans for large-scale 3D face recognition[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018:1896-1905.
[11] Mu G D, Huang D, Hu G S, et al. Led3D:a lightweight and efficient deep approach to recognizing low-quality 3D faces[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:5766-5775.
[12] Gilani S Z, Mian A, Shafait F, et al. Dense 3D face correspondence[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(7):1584-1598.
[13] Gilani S Z, Mian A. Towards large-scale 3D face recognition[C]//2016 International Conference on Digital Image Computing:Techniques and Applications (DICTA). November 30-December 2, 2016, Gold Coast, QLD, Australia. IEEE, 2016:1-8.
[14] Gilani S Z, Mian A, Eastwood P. Deep, dense and accurate 3D face correspondence for generating population specific deformable models[J]. Pattern Recognition, 2017, 69:238-250.
[15] Isola P, Zhu J Y, Zhou T H, et al. Image-to-image translation with conditional adversarial networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:5967-5976.
[16] Wang W X, Fu Y W, Qian X L, et al. FM2u-net:face morphological multi-branch network for makeup-invariant face verification[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020:5729-5739.
[17] Johnson J, Alahi A, Li F F. Perceptual losses for real-time style transfer and super-resolution[C]//Computer Vision-ECCV 2016, 2016:694-711. DOI:10.1007/978-3-319-46475-6_43.
[18] Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision (ICCV). October 22-29, 2017, Venice, Italy. IEEE, 2017:2242-2251.
[19] Zhang K P, Zhang Z P, Li Z F, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10):1499-1503.
[20] Ronneberger O, Fischer P, Brox T. U-net:convolutional networks for biomedical image segmentation[M]//Lecture Notes in Computer Science. Cham:Springer International Publishing, 2015:234-241.
[21] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016:770-778.
[22] Yi D, Lei Z, Liao S C, et al. Learning face representation from scratch[EB/OL]. arXiv:1411.7923. (2014-11-28)[2021-01-20]. https://arxiv.org/pdf/1411.7923.pdf.
[23] Zhu X Y, Lei Z, Liu X M, et al. Face alignment across large poses:a 3D solution[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016:146-155.
[24] Tran A T, Hassner T, Masi I, et al. Regressing robust and discriminative 3D morphable models with a very deep neural network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:1493-1502.
[25] Guo J Z, Zhu X Y, Yang Y, et al. Towards fast, accurate and stable 3D dense face alignment[M]//Computer Vision-ECCV 2020. Cham:Springer International Publishing, 2020:152-168.
[26] Vijayan V, Bowyer K W, Flynn P J, et al. Twins 3D face recognition challenge[C]//2011 International Joint Conference on Biometrics (IJCB). October 11-13, 2011, Washington, DC, USA. IEEE, 2011:1-7.
[27] Zhong C, Sun Z N, Tan T N. Learning efficient codes for 3D face recognition[C]//200815th IEEE International Conference on Image Processing. October 12-15, 2008, San Diego, CA, USA. IEEE, 2008:1928-1931.
[28] Savran A, Alyüz N, Dibeklioğlu H, et al. Bosphorus database for 3D face analysis[M]//Lecture Notes in Computer Science. Berlin, Heidelberg:Springer Berlin Heidelberg, 2008:47-56.
[29] Gupta S, Markey M K, Bovik A C. Anthropometric 3D face recognition[J]. International Journal of Computer Vision, 2010, 90(3):331-349.