基于级联卷积网络的面部关键点定位算法

doi:10.7523/j.issn.2095-6134.2020.04.017

中国科学院大学学报 ›› 2020, Vol. 37 ›› Issue (4): 562-569.DOI: 10.7523/j.issn.2095-6134.2020.04.017

基于级联卷积网络的面部关键点定位算法

孙铭堃^1,2, 梁令羽^1,2, 汪涵¹, 何为¹, 赵鲁阳¹

1. 中国科学院上海微系统与信息技术研究所宽带无线移动通信研究室, 上海 201800;
2. 中国科学院大学, 北京 100049

收稿日期:2019-01-03 修回日期:2019-03-22 发布日期:2020-07-15
通讯作者: 何为
基金资助:
国家重点研发计划（2018YFC1505204）和中国科学院青年创新促进会（2015186）资助

Facial landmark detection based on cascade convolutional neural network

SUN Mingkun^1,2, LIANG Lingy^1,2, WANG Han¹, HE Wei¹, ZHAO Luyang¹

1. Broadband Wireless Mobile Communications Research Lab, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai 201800, China;
2. University of Chinese Academy of Sciences, Beijing 100049, China

Received:2019-01-03 Revised:2019-03-22 Published:2020-07-15
Supported by:

摘要/Abstract

摘要： 目前，人的面部关键点定位算法在限定环境下已达到很高的识别率，但在非限定环境下，仍易受到环境光线不均、测试角度范围广、检测目标姿态多样及遮挡模糊等因素的影响。提出一种级联卷积网络以提高关键点定位的精度与鲁棒性。在进行人脸检测时，该算法在Light-VGGNet的基础上提出一种DPM-CNN网络结构，引入五官可变形部件，将人脸检测与五官定位同时进行，提高人脸检测精度并降低人脸检测对面部关键点定位的影响。在进行内部关键点定位时，采用由粗到细的算法思想，将两层不同的网络级联实现对内外关键点的定位。利用FDDB数据集进行测试，无论在人脸检测，还是面部关键点定位上，所提出的卷积网络结构准确度和检测速度均高于其他算法，在非限定环境下表现出很好的鲁棒性。

关键词: 非限定环境, 级联卷积网络, Light-VGGNet, DPM-CNN, 人脸检测, 面部关键点定位

Abstract: Current facial landmark detection algorithm has achieved promising recognition rates in constrained environment, but in unconstrained environment it is still susceptible to various factors such as non-uniform ambient illumination, wide range of angles, variations in pose, occlusion, and blur. To deal with these problems, we propose a cascade convolutional network to improve the accuracy and robustness of the landmark detection. In face detection, we propose a DPM-CNN model based on Light-VGGNet, which introduces the location information of the facial features. In this way, the detection accuracy is improved and the impact of face detection on the positioning of landmarks is reduced. In facial landmark detection, this algorithm adopts two different layers of cascade networks progressively to complete the positioning of the internal and external landmarks. Finally, on the FDDB dataset, this new algorithm is proved to have higher accuracy rate and detection speed than other algorithms in face detection or in facial landmark location. This algorithm is also robust in an unconstrained environment.

Key words: unconstrained environment, cascade convolutional neural network, Light-VGGNet, DPM-CNN, face detection, facial landmark location

中图分类号:

TP391.4

孙铭堃, 梁令羽, 汪涵, 何为, 赵鲁阳. 基于级联卷积网络的面部关键点定位算法[J]. 中国科学院大学学报, 2020, 37(4): 562-569.

SUN Mingkun, LIANG Lingy, WANG Han, HE Wei, ZHAO Luyang. Facial landmark detection based on cascade convolutional neural network[J]. , 2020, 37(4): 562-569.

参考文献

[1] Singh R, Om H. An overview of face recognition in an unconstrained environment[C]//2013 IEEE Second International Conference on Image Information Processing(ICIIP). Shimla, India:IEEE, 2013:672-677.
[2] Reisfeld D, Yeshurun Y. Robust-detection of facial features by generalized symmetry[C]//Iapr International Conference on Pattern Recognition. Hague, Netherlands:IEEE, 1992:117-120.
[3] Cootes T, Taylor C, Cooper D, et al. Active shape models:their training and application[J]. Computer Vision and Image Understanding, 1995, 61(1):38-59.
[4] Cristinacce D, Cootes T. Automatic feature localization with constrained local models[J]. Pattern Recognition, 2008, 41(10):3054-3067.
[5] Asthana A, Zafeirious S, Cheng S, et al. Robust discriminative response map fitting with constrained local models[C]//Computer Vision and Pattern Recognition(CVPR). Portland, OR, USA:IEEE, 2013:3444-3451.
[6] Valstar M, Martinez B, Binefa X, et al. Facial point detection using boosted regression and graph models[C]//2010 IEEE Computer Societ Computer Vision and Pattern Recognition(CVPR). San Francisco, California, USA:IEEE, 2010:2729-2736.
[7] Dantone M, Gall J, Fanelli G, et al. Real-time facial feature detection using conditional regression forests[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Providence, RI, USA:IEEE, 2012:2578-2585.
[8] Dollaár P, Welinder P, Perona P. Cascaded pose regression[C]//2010 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). San Francisco, California, USA:IEEE, 2010:1078-1085.
[9] Martinez B, Valstar M, Binefa X, et al. Local evidence aggregation for regression-based facial point detection[J]. IEEE Transations on Pattern Analysis and Machine Intelligence, 2013:35(5):1149-1163.
[10] Sun Y, Wang X, Tang X. Deep convolutional network cascade for facial point detection[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Portland, OR, USA:IEEE, 2013:3476-3483.
[11] Zhang J, Shan S, Kan M, et al. Coarse-to-fine auto-encoder networks(CFAN) for real-time face alignment[C]//European Conference on Computer Vision(ECCV). Springer, Cham:2014:1-16.
[12] Li H, Lin Z, Shen X, et al. A convolutional neural network cascade for face detection[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Boston, MA, USA:IEEE, 2015:5325-5334.
[13] Kowalski M, Naruniec J, Trzcinski T. Deep alignment network:a convolutional neural network for robust face alignment[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu, Hawaii, USA:IEEE, 2017:2034-2043.
[14] Zhou E, Fan H, Cao Z, et al. Extensive facial landmark localization with coarse-to-fine convolutional network cascade[C]//2013 IEEE International Conference on Computer Vision Workshops(ICCVW). Sydney, Australia:IEEE, 2013:386-391.
[15] Huang R, Xie X, Feng Z, et al. Face recognition by landmark pooling-based cnn with concentrate loss[C]//2017 IEEE International Conference on Image Processing (ICIP). Beijing, China:IEEE, 2017:1582-1586.
[16] Goodfellow I, Bengio Y, Courville A. Deep learning[M]. Massachusetts:MIT Press, 2016:326.
[17] Krizhevsky A, Sutskever I, Hinton G. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[18] Dalal N, Triggs B. Histograms of oriented gradients for human detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR). San Diego:IEEE, 2005:886-893.
[19] Viola P, Jones M. Robust real-time face detection[J]. International Journal of Computer Vision, 2004, 57(2):137-154.
[20] Girshick R,Donahue J,Darrell T,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Columbus, OH:IEEE, 2014:580-587
[21] Ren S, He K, Girshick R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems, 2015:1049-5258.
[22] Peng K, Chen T. A framework of extracting multi-scale features using multiple convolutional neural networks[C]//IEEE International Conference on Multimedia and Expo(ICME). Turin, Italy:IEEE, 2015:1-6.
[23] Sun Y, Wang X, Tang X. Deep convolutional network cascade for facial point detection[C]//Computer Vision and Pattern Recognition(CVPR). Portland, OR, USA:IEEE, 2013:3476-3483.
[24] 井长兴,章东平,杨力.级联神经网络人脸关键点定位研究[J].中国计量大学学报,2018,29(2):187-193.

基于级联卷积网络的面部关键点定位算法

Facial landmark detection based on cascade convolutional neural network

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价

访问统计

联系我们