基于互信息约束的生成对抗网络分类模型

doi:10.7523/j.ucas.2020.0037

中国科学院大学学报 ›› 2022, Vol. 39 ›› Issue (4): 551-560.DOI: 10.7523/j.ucas.2020.0037

• 电子信息与计算机科学 • 上一篇下一篇

基于互信息约束的生成对抗网络分类模型

胡兵兵^1,2,3, 唐华^1,2,3, 吴幼龙¹

1. 上海科技大学信息科学与技术学院, 上海 201210;
2. 中国科学院上海微系统与信息技术研究所, 上海 200050;
3. 中国科学院大学, 北京 100049

收稿日期:2020-04-27 修回日期:2020-08-10 发布日期:2021-06-01
通讯作者: 胡兵兵
基金资助:
国家自然科学基金(61901267)和上海市浦江人才计划(18PJ1408500)资助

Classification models based on generative adversarial networks with mutual information regularization

HU Bingbing^1,2,3, TANG Hua^1,2,3, WU Youlong¹

1. School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China;
2. Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Science, Shanghai 200050, China;
3. University of Chinese Academy of Science, Beijing 100049, China

Received:2020-04-27 Revised:2020-08-10 Published:2021-06-01

摘要/Abstract

摘要： 传统的机器学习方法需要大量的含标注数据集来训练模型,并且容易引发过拟合,而生成对抗网络可以无监督地进行训练。此外,互信息约束能够让模型生成指定类别的数据,可用于扩充数据集。提出InfoCatGAN和C-InfoGAN两种模型,前者在CatGAN的基础上增加了互信息约束,使得生成的图片更加逼真;后者使用InfoGAN模型中的辅助网络Q做分类,能够在生成高质量图片的同时,达到较好的分类准确率。二者均能通过隐变量控制生成图片的类别,这对数据增强具有一定意义。另外,在加入少量标签信息之后,模型的准确率能有所提升。

关键词: 生成对抗网络, 无监督学习, 半监督学习, 互信息

Abstract: This paper studies classification models based on generative adversarial networks with mutual information regularization. Traditional machine learning methods rely on a large number of labeled datasets, which are scarce in practice, to train the model and can easily overfit to spurious correlations in the data; while generating adversarial networks can be trained in an unsupervised manner. In addition, mutual information constraint allows the model to generate data of a specified through latent variables, which has a certain significance for data augmentation. Moreover, after adding a small amount of label information, the accuracy of the model can be improved. category, which can be used to expand the data set. This paper proposes the InfoCatGAN and CInfoGAN classification models. The former adds the mutual information term to CatGAN model in order to generate images of higher visual fidelity; the latter uses the InfoGAN model for classification, which can ensure the quality of the generated images and provide a mentionable classification accuracy. Additionally, both two models can control the category of generated images

Key words: GANs, unsupervised learning, semi-supervised learning, mutual information

中图分类号:

U283.4

胡兵兵, 唐华, 吴幼龙. 基于互信息约束的生成对抗网络分类模型[J]. 中国科学院大学学报, 2022, 39(4): 551-560.

HU Bingbing, TANG Hua, WU Youlong. Classification models based on generative adversarial networks with mutual information regularization[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(4): 551-560.

参考文献

[1] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J].Communications of the ACM, 2017,60(6):84-90.DOI:10.1145/3065386.
[2] Taigman Y, Yang M, Ranzato M, et al. DeepFace:closing the gap to human-level performance in face verification//2014 IEEE Conference on Computer Vision and Pattern Recognition. June 23-28, 2014, Columbus, OH, USA. IEEE, 2014:1701-1708.DOI:10.1109/CVPR.2014.220.
[3] 江璐,赵彤,吴敏.基于深度卷积神经网络的指纹纹型分类算法[J].中国科学院大学学报, 2016, 33(6):808-814.DOI:10.7523/j.issn.2095-6134.2016.06.013.
[4] Xu L L, Neufeld J, Larson B, et al. Maximum margin clustering//Advances in Neural Information Processing Systems 17(NIPS 2004). 2005:1537-1544.
[5] Krause A, Perona P, Gomes R G. Discriminative clustering by regularized information maximization//Advances in Neural Information Processing Systems 23(NIPS 2010). 2010:775-783.
[6] Springenberg J T. Unsupervised and semi-supervised learning with categorical generative adversarial networks. ArXiv preprint, arXiv:1511.06390.(2016-04-30). https://arxiv.org/abs/1511.06390.
[7] 宋旭鸣,沈逸飞,石远明.基于深度学习的智能移动边缘网络缓存[J].中国科学院大学学报, 2020,37(1):128-135.DOI:10.7523/j.issn.2095-6134.2020.01.015.
[8] 田玮,朱廷劭.基于深度学习的微博用户自杀风险预测[J].中国科学院大学学报, 2018, 35(1):131-136.DOI:10.7523/j.issn.2095-6134.2018.01.018.
[9] 杨建斌,张卫强,刘加.深度神经网络自适应中基于身份认证向量的归一化方法[J].中国科学院大学学报, 2017, 34(5):633-639.DOI:10.7523/j.issn.2095-6134.2017.05.014.
[10] Salakhutdinov R, Hinton G. Deep boltzmann machines[J].Journal of Machine Learning Research, 2009,5:448-455.
[11] Goodfellow I, Mirza M, Courville A, et al. Multi-prediction deep Boltzmann machines//Advances in Neural Information Processing Systems 26(NIPS 2013). 2013:548-556.
[12] Bengio Y, Laufer E, Alain G, et al. Deep generative stochastic networks trainable by backprop//Proceedings of the 31st International Conference on Machine Learning, PMLR, 2014,32(2):226-234.
[13] Kingma D P, Mohamed S, Rezende D J, et al. Semi-supervised learning with deep generative models//Advances in Neural Information Processing Systems 27(NIPS 2014). 2014:3581-3589.
[14] Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786):504-507.DOI:10.1126/science.1127647.
[15] Vincent P, Larochelle H, Bengio Y, et al. Extracting and composing robust features with denoising autoencoders//Proceedings of the 25th International Conference on Machine Learning (ICML). 2008:1096-1103.DOI:10.1145/1390156.1390294.
[16] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Advances in Neural Information Processing Systems 27(NIPS 2014). 2014:2672-2680.
[17] Chen X, Duan Y, Houthooft R, et al. InfoGAN:interpretable representation learning by information maximizing generative adversarial nets//Advances in Neural Information Processing Systems 29(NIPS 2016). 2016:2172-2180.
[18] LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation, 1989, 1(4):541-551.DOI:10.1162/neco.1989.1.4.541.
[19] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images. Computer Science Department, University of Toronto, Tech. 2009.
[20] Li C X, Xu T, Zhu J, et al. Triple generative adversarial nets//Advances in Neural Information Processing Systems 30(NIPS 2017). 2017:4088-4098.
[21] Wu S, Deng G C, Li J C, et al. Enhancing TripleGAN for semi-supervised conditional instance synthesis and classification//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:10083-10092.DOI:10.1109/CVPR.2019.01033.
[22] Poole B, Ozair S, van der Oord A, et al. On variational bounds of mutual information//Proceedings of the 36th International Conference on Machine Learning, PMLR.2019,97:5171-5180.
[23] Grandvalet Y, Bengio Y. Semi-supervised learning by entropy minimization//Advances in Neural Information Processing Systems 17(NIPS 2004). 2005:529-536.
[24] Spurr A, Aksan E, Hilliges O. Guiding InfoGAN with semi-supervision//Joint European Conference on Machine Learning and Knowledge Discovery in Databases. ECML PKDD, 2017:119134.DOI:10.1007/978-3-319-71249-9_8.
[25] Heusel M, Ramsauer H, Unterthiner T, et al. Gans trained by a two time-scale update rule converge to a local nash equilibrium//Advances in Neural Information Processing Systems 30(NIPS 2017). 2017:6626-6637.
[26] Salimans T, Goodfellow I, Zaremba W, et al. Improved techniques for training GANs//Advances in Neural Information Processing Systems 29(NIPS). 2016:2234-2242.
[27] Xiao H, Rasul K, Vollgraf R. Fashion-MNIST:a novel image dataset for benchmarking machine learning algorithms. ArXiv Preprint, arXiv:1708.07747.(2017-09-15). https://arxiv.org/abs/1708.07747.

基于互信息约束的生成对抗网络分类模型

Classification models based on generative adversarial networks with mutual information regularization

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 4

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	赵心驰, 姜策, 何为. 基于注意力生成对抗网络的图像强光去除[J]. 中国科学院大学学报, 2022, 39(4): 524-531.
[2]	杨雨龙, 郭田德, 韩丛英. 基于原型学习改进的伪标签半监督学习算法[J]. 中国科学院大学学报, 2021, 38(6): 841-851.
[3]	陈国炜, 刘磊, 郭嘉逸, 潘宗序, 胡文龙. 基于生成对抗网络的半监督遥感图像飞机检测[J]. 中国科学院大学学报, 2020, 37(4): 539-546.
[4]	黄晓英, 郑委, 滕吉红. 选择生成器序列概率模型信息论分析[J]. 中国科学院大学学报, 2006, 23(6): 775-781.