Facial landmark detection based on cascade convolutional neural network

doi:10.7523/j.issn.2095-6134.2020.04.017

Abstract

Abstract: Current facial landmark detection algorithm has achieved promising recognition rates in constrained environment, but in unconstrained environment it is still susceptible to various factors such as non-uniform ambient illumination, wide range of angles, variations in pose, occlusion, and blur. To deal with these problems, we propose a cascade convolutional network to improve the accuracy and robustness of the landmark detection. In face detection, we propose a DPM-CNN model based on Light-VGGNet, which introduces the location information of the facial features. In this way, the detection accuracy is improved and the impact of face detection on the positioning of landmarks is reduced. In facial landmark detection, this algorithm adopts two different layers of cascade networks progressively to complete the positioning of the internal and external landmarks. Finally, on the FDDB dataset, this new algorithm is proved to have higher accuracy rate and detection speed than other algorithms in face detection or in facial landmark location. This algorithm is also robust in an unconstrained environment.

Key words: unconstrained environment, cascade convolutional neural network, Light-VGGNet, DPM-CNN, face detection, facial landmark location

CLC Number:

TP391.4

SUN Mingkun, LIANG Lingy, WANG Han, HE Wei, ZHAO Luyang. Facial landmark detection based on cascade convolutional neural network[J]. , 2020, 37(4): 562-569.

References

[1] Singh R, Om H. An overview of face recognition in an unconstrained environment[C]//2013 IEEE Second International Conference on Image Information Processing(ICIIP). Shimla, India:IEEE, 2013:672-677.
[2] Reisfeld D, Yeshurun Y. Robust-detection of facial features by generalized symmetry[C]//Iapr International Conference on Pattern Recognition. Hague, Netherlands:IEEE, 1992:117-120.
[3] Cootes T, Taylor C, Cooper D, et al. Active shape models:their training and application[J]. Computer Vision and Image Understanding, 1995, 61(1):38-59.
[4] Cristinacce D, Cootes T. Automatic feature localization with constrained local models[J]. Pattern Recognition, 2008, 41(10):3054-3067.
[5] Asthana A, Zafeirious S, Cheng S, et al. Robust discriminative response map fitting with constrained local models[C]//Computer Vision and Pattern Recognition(CVPR). Portland, OR, USA:IEEE, 2013:3444-3451.
[6] Valstar M, Martinez B, Binefa X, et al. Facial point detection using boosted regression and graph models[C]//2010 IEEE Computer Societ Computer Vision and Pattern Recognition(CVPR). San Francisco, California, USA:IEEE, 2010:2729-2736.
[7] Dantone M, Gall J, Fanelli G, et al. Real-time facial feature detection using conditional regression forests[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Providence, RI, USA:IEEE, 2012:2578-2585.
[8] Dollaár P, Welinder P, Perona P. Cascaded pose regression[C]//2010 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). San Francisco, California, USA:IEEE, 2010:1078-1085.
[9] Martinez B, Valstar M, Binefa X, et al. Local evidence aggregation for regression-based facial point detection[J]. IEEE Transations on Pattern Analysis and Machine Intelligence, 2013:35(5):1149-1163.
[10] Sun Y, Wang X, Tang X. Deep convolutional network cascade for facial point detection[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Portland, OR, USA:IEEE, 2013:3476-3483.
[11] Zhang J, Shan S, Kan M, et al. Coarse-to-fine auto-encoder networks(CFAN) for real-time face alignment[C]//European Conference on Computer Vision(ECCV). Springer, Cham:2014:1-16.
[12] Li H, Lin Z, Shen X, et al. A convolutional neural network cascade for face detection[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Boston, MA, USA:IEEE, 2015:5325-5334.
[13] Kowalski M, Naruniec J, Trzcinski T. Deep alignment network:a convolutional neural network for robust face alignment[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu, Hawaii, USA:IEEE, 2017:2034-2043.
[14] Zhou E, Fan H, Cao Z, et al. Extensive facial landmark localization with coarse-to-fine convolutional network cascade[C]//2013 IEEE International Conference on Computer Vision Workshops(ICCVW). Sydney, Australia:IEEE, 2013:386-391.
[15] Huang R, Xie X, Feng Z, et al. Face recognition by landmark pooling-based cnn with concentrate loss[C]//2017 IEEE International Conference on Image Processing (ICIP). Beijing, China:IEEE, 2017:1582-1586.
[16] Goodfellow I, Bengio Y, Courville A. Deep learning[M]. Massachusetts:MIT Press, 2016:326.
[17] Krizhevsky A, Sutskever I, Hinton G. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[18] Dalal N, Triggs B. Histograms of oriented gradients for human detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR). San Diego:IEEE, 2005:886-893.
[19] Viola P, Jones M. Robust real-time face detection[J]. International Journal of Computer Vision, 2004, 57(2):137-154.
[20] Girshick R,Donahue J,Darrell T,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Columbus, OH:IEEE, 2014:580-587
[21] Ren S, He K, Girshick R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems, 2015:1049-5258.
[22] Peng K, Chen T. A framework of extracting multi-scale features using multiple convolutional neural networks[C]//IEEE International Conference on Multimedia and Expo(ICME). Turin, Italy:IEEE, 2015:1-6.
[23] Sun Y, Wang X, Tang X. Deep convolutional network cascade for facial point detection[C]//Computer Vision and Pattern Recognition(CVPR). Portland, OR, USA:IEEE, 2013:3476-3483.
[24] 井长兴,章东平,杨力.级联神经网络人脸关键点定位研究[J].中国计量大学学报,2018,29(2):187-193.