图像中的强光在一定程度上会降低图像的质量,本文致力于从受到强光影响的图像中去除强光并生成清晰图像。为解决这个问题,提出一种带有注意力辅助模块的生成对抗网络。它主要由加入压缩-激励模块的卷积长短期记忆网络和注意力矩阵辅助模块组成,注意力辅助模块可以指导自动编码器生成清晰的图像。该方法可以轻松地移植处理其他类似的图像恢复问题。实验证明,改进后的网络体系结构是有效的并且有一定的意义。
The highlights in the image will degrade the image quality to some extent. In this paper, we focus on visually removing the highlights from degraded images and generating clean images. In order to solve this problem, we present an attention-auxiliary generative adversarial networks. It mainly consists of the convolutional long short term memory network with squeeze-and-excitation (SE) block and the map-auxiliary module. Map-auxiliary can instruct the autoencoder to generate clean images. The injection of SE block and map-auxiliary module to the generator is the main contribution of this paper. And our proposed deep learning-based approach can be easily ported to handle other similar image recovery problems. Experiments prove that the network architecture is effective and makes a lot of sense.
[1] Tan P, Lin S, Quan L, et al. Highlight removal by illumination-constrained inpainting//Proceedings of the 9th IEEE International Conference on Computer Vision. October 13-16, 2003, Nice, France. IEEE, 2003:164-169. DOI:10.1109/ICCV.2003.1238333.
[2] Yang Q X, Wang S N, Ahuja N. Real-time specular highlight removal using bilateral filtering//European Conference on Computer Vision-ECCV 2010. Springer, Berlin, Heidelberg, 2010:87-100. DOI:10.1007/978-3-642-15561-1_7.
[3] Kurihata H, Takahashi T, Ide I, et al. Rainy weather recognition from in-vehicle camera images for driver assistance//IEEE Proceedings of Intelligent Vehicles Symposium, June 6-8, 2005, Las Vegas, NV, USA. IEEE, 2005:205-210. DOI:10.1109/IVS.2005.1505103.
[4] Roser M, Geiger A. Video-based raindrop detection for improved image registration//2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops. September 27-October 4, 2009, Kyoto, Japan. IEEE, 2009:570-577. DOI:10.1109/ICCVW.2009.5457650.
[5] Roser M, Kurz J, Geiger A. Realistic modeling of water droplets for monocular adherent raindrop recognition using bézier curves//Asian Conference on Computer Vision-ACCV 2010. Springer, Berlin, Heidelberg, 2010:235-244. DOI:10.1007/978-3-642-22819-3_24.
[6] Tanaka Y, Yamashita A, Kaneko T, et al. Removal of adherent waterdrops from images acquired with a stereo camera system[J]. IEICE Transactions on Information and Systems, 2006, E89-D (7):2021-2027. DOI:10.1093/ietisy/e89-d.7.2021.
[7] Yamashita A, Fukuchi I, Kaneko T. Noises removal from image sequences acquired with moving camera by estimating camera motion from spatio-temporal information//2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. October 10-15, 2009, St. Louis, MO, USA. IEEE, 2009:3794-3801. DOI:10.1109/IROS.2009.5354639.
[8] You S D, Tan R T, Kawakami R, et al. Adherent raindrop modeling, detection and removal in video[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(9):1721-1733. DOI:10.1109/TPAMI.2015.2491937.
[9] Hara T, Saito H, Kanade T. Removal of glare caused by water droplets//2009 Conference for Visual Media Production. November 12-13, 2009, London, UK. IEEE, 2009:144-151. DOI:10.1109/CVMP.2009.17.
[10] Qian R, Tan R T, Yang W H, et al. Attentive generative adversarial network for raindrop removal from a single image//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. 2018:2482-2491. DOI:10.1109/CVPR.2018.00263.
[11] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Advances in Neural Information Processing Systems, 2014:2672-2680.
[12] Hu J, Shen L, Sun G. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. 2018:7132-7141. DOI:10.1109/CVPR.2018.00745.
[13] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition. June 27-30, 2016, Las Vegas, NV, USA. 2016:770-778. DOI:10.1109/CVPR.2016.90.
[14] Shi X J, Chen Z R, Wang H, et al. Convolutional LSTM network:a machine learning approach for precipitation nowcasting//Advances in Neural Information Processing Systems, 2015:802-810.
[15] Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines//Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010:807-814.
[16] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780. DOI:10.1162/neco.1997.9.8.1735.
[17] Johnson J, Alahi A, Li F F. Perceptual losses for real-time style transfer and super-resolution//European Conference on Computer Vision-ECCV 2016. Springer, Cham, 2016:694-711. DOI:10.1007/978-3-319-46475-6_43.
[18] Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks//2017 IEEE International Conference on Computer Vision. October 22-29, 2017, Venice, Italy. IEEE, 2017:2242-2251. DOI:10.1109/ICCV.2017.244.
