基于注意力生成对抗网络的图像强光去除

doi:10.7523/j.ucas.2020.0018

中国科学院大学学报 ›› 2022, Vol. 39 ›› Issue (4): 524-531.DOI: 10.7523/j.ucas.2020.0018

• 电子信息与计算机科学 • 上一篇下一篇

基于注意力生成对抗网络的图像强光去除

赵心驰^1,2,3, 姜策^1,2, 何为¹

1. 中国科学院上海微系统与信息技术研究所中国科学院无线传感网与通信重点实验室, 上海 201800;
2. 中国科学院大学, 北京 100049;
3. 成都中科微信息技术研究院有限公司, 成都 610000

收稿日期:2020-02-13 修回日期:2020-06-10 发布日期:2021-05-31
通讯作者: 何为
基金资助:
Supported by the National Key Research and Development Program of China (2018YFC1505204-2), Key Deployment Project of Chinese Academy of Sciences(KFZD-SW-431), Chengdu’s Major Scientific and Technological Innovation Projects(2019-YF08-00082-GX)

Removing highlights from single image via an attention-auxiliary generative adversarial network

ZHAO Xinchi^1,2,3, JIANG Ce^1,2, HE Wei¹

1. Key Lab of Wireless Sensor Network and Communication, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai 201800, China;
2. University of Chinese Academy of Sciences, Beijing 100049, China;
3. Chengdu ZhongKeWei Information Technology Research Institute Co Ltd, Chengdu 610000, China)

Received:2020-02-13 Revised:2020-06-10 Published:2021-05-31
Supported by:
Supported by the National Key Research and Development Program of China (2018YFC1505204-2), Key Deployment Project of Chinese Academy of Sciences(KFZD-SW-431), Chengdu’s Major Scientific and Technological Innovation Projects(2019-YF08-00082-GX)

摘要/Abstract

摘要： 图像中的强光在一定程度上会降低图像的质量,本文致力于从受到强光影响的图像中去除强光并生成清晰图像。为解决这个问题,提出一种带有注意力辅助模块的生成对抗网络。它主要由加入压缩-激励模块的卷积长短期记忆网络和注意力矩阵辅助模块组成,注意力辅助模块可以指导自动编码器生成清晰的图像。该方法可以轻松地移植处理其他类似的图像恢复问题。实验证明,改进后的网络体系结构是有效的并且有一定的意义。

关键词: 生成对抗网络, 注意力矩阵辅助, 压缩-激励模块, 图像恢复, 强光去除

Abstract: The highlights in the image will degrade the image quality to some extent. In this paper, we focus on visually removing the highlights from degraded images and generating clean images. In order to solve this problem, we present an attention-auxiliary generative adversarial networks. It mainly consists of the convolutional long short term memory network with squeeze-and-excitation (SE) block and the map-auxiliary module. Map-auxiliary can instruct the autoencoder to generate clean images. The injection of SE block and map-auxiliary module to the generator is the main contribution of this paper. And our proposed deep learning-based approach can be easily ported to handle other similar image recovery problems. Experiments prove that the network architecture is effective and makes a lot of sense.

Key words: GAN (generative adversarial networks), attention map-auxiliary, squeeze-andexcitation block, image restoration, highlights-removal

中图分类号:

TP391.4

赵心驰, 姜策, 何为. 基于注意力生成对抗网络的图像强光去除[J]. 中国科学院大学学报, 2022, 39(4): 524-531.

ZHAO Xinchi, JIANG Ce, HE Wei. Removing highlights from single image via an attention-auxiliary generative adversarial network[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(4): 524-531.

参考文献

[1] Tan P, Lin S, Quan L, et al. Highlight removal by illumination-constrained inpainting//Proceedings of the 9th IEEE International Conference on Computer Vision. October 13-16, 2003, Nice, France. IEEE, 2003:164-169. DOI:10.1109/ICCV.2003.1238333.
[2] Yang Q X, Wang S N, Ahuja N. Real-time specular highlight removal using bilateral filtering//European Conference on Computer Vision-ECCV 2010. Springer, Berlin, Heidelberg, 2010:87-100. DOI:10.1007/978-3-642-15561-1_7.
[3] Kurihata H, Takahashi T, Ide I, et al. Rainy weather recognition from in-vehicle camera images for driver assistance//IEEE Proceedings of Intelligent Vehicles Symposium, June 6-8, 2005, Las Vegas, NV, USA. IEEE, 2005:205-210. DOI:10.1109/IVS.2005.1505103.
[4] Roser M, Geiger A. Video-based raindrop detection for improved image registration//2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops. September 27-October 4, 2009, Kyoto, Japan. IEEE, 2009:570-577. DOI:10.1109/ICCVW.2009.5457650.
[5] Roser M, Kurz J, Geiger A. Realistic modeling of water droplets for monocular adherent raindrop recognition using bézier curves//Asian Conference on Computer Vision-ACCV 2010. Springer, Berlin, Heidelberg, 2010:235-244. DOI:10.1007/978-3-642-22819-3_24.
[6] Tanaka Y, Yamashita A, Kaneko T, et al. Removal of adherent waterdrops from images acquired with a stereo camera system[J]. IEICE Transactions on Information and Systems, 2006, E89-D (7):2021-2027. DOI:10.1093/ietisy/e89-d.7.2021.
[7] Yamashita A, Fukuchi I, Kaneko T. Noises removal from image sequences acquired with moving camera by estimating camera motion from spatio-temporal information//2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. October 10-15, 2009, St. Louis, MO, USA. IEEE, 2009:3794-3801. DOI:10.1109/IROS.2009.5354639.
[8] You S D, Tan R T, Kawakami R, et al. Adherent raindrop modeling, detection and removal in video[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(9):1721-1733. DOI:10.1109/TPAMI.2015.2491937.
[9] Hara T, Saito H, Kanade T. Removal of glare caused by water droplets//2009 Conference for Visual Media Production. November 12-13, 2009, London, UK. IEEE, 2009:144-151. DOI:10.1109/CVMP.2009.17.
[10] Qian R, Tan R T, Yang W H, et al. Attentive generative adversarial network for raindrop removal from a single image//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. 2018:2482-2491. DOI:10.1109/CVPR.2018.00263.
[11] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Advances in Neural Information Processing Systems, 2014:2672-2680.
[12] Hu J, Shen L, Sun G. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. 2018:7132-7141. DOI:10.1109/CVPR.2018.00745.
[13] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition. June 27-30, 2016, Las Vegas, NV, USA. 2016:770-778. DOI:10.1109/CVPR.2016.90.
[14] Shi X J, Chen Z R, Wang H, et al. Convolutional LSTM network:a machine learning approach for precipitation nowcasting//Advances in Neural Information Processing Systems, 2015:802-810.
[15] Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines//Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010:807-814.
[16] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780. DOI:10.1162/neco.1997.9.8.1735.
[17] Johnson J, Alahi A, Li F F. Perceptual losses for real-time style transfer and super-resolution//European Conference on Computer Vision-ECCV 2016. Springer, Cham, 2016:694-711. DOI:10.1007/978-3-319-46475-6_43.
[18] Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks//2017 IEEE International Conference on Computer Vision. October 22-29, 2017, Venice, Italy. IEEE, 2017:2242-2251. DOI:10.1109/ICCV.2017.244.

基于注意力生成对抗网络的图像强光去除

Removing highlights from single image via an attention-auxiliary generative adversarial network

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	胡兵兵, 唐华, 吴幼龙. 基于互信息约束的生成对抗网络分类模型[J]. 中国科学院大学学报, 2022, 39(4): 551-560.
[2]	陈国炜, 刘磊, 郭嘉逸, 潘宗序, 胡文龙. 基于生成对抗网络的半监督遥感图像飞机检测[J]. 中国科学院大学学报, 2020, 37(4): 539-546.