Classification models based on generative adversarial networks with mutual information regularization

doi:10.7523/j.ucas.2020.0037

Abstract

Abstract: This paper studies classification models based on generative adversarial networks with mutual information regularization. Traditional machine learning methods rely on a large number of labeled datasets, which are scarce in practice, to train the model and can easily overfit to spurious correlations in the data; while generating adversarial networks can be trained in an unsupervised manner. In addition, mutual information constraint allows the model to generate data of a specified through latent variables, which has a certain significance for data augmentation. Moreover, after adding a small amount of label information, the accuracy of the model can be improved. category, which can be used to expand the data set. This paper proposes the InfoCatGAN and CInfoGAN classification models. The former adds the mutual information term to CatGAN model in order to generate images of higher visual fidelity; the latter uses the InfoGAN model for classification, which can ensure the quality of the generated images and provide a mentionable classification accuracy. Additionally, both two models can control the category of generated images

Key words: GANs, unsupervised learning, semi-supervised learning, mutual information

CLC Number:

U283.4

HU Bingbing, TANG Hua, WU Youlong. Classification models based on generative adversarial networks with mutual information regularization[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(4): 551-560.

References

[1] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J].Communications of the ACM, 2017,60(6):84-90.DOI:10.1145/3065386.
[2] Taigman Y, Yang M, Ranzato M, et al. DeepFace:closing the gap to human-level performance in face verification//2014 IEEE Conference on Computer Vision and Pattern Recognition. June 23-28, 2014, Columbus, OH, USA. IEEE, 2014:1701-1708.DOI:10.1109/CVPR.2014.220.
[3] 江璐,赵彤,吴敏.基于深度卷积神经网络的指纹纹型分类算法[J].中国科学院大学学报, 2016, 33(6):808-814.DOI:10.7523/j.issn.2095-6134.2016.06.013.
[4] Xu L L, Neufeld J, Larson B, et al. Maximum margin clustering//Advances in Neural Information Processing Systems 17(NIPS 2004). 2005:1537-1544.
[5] Krause A, Perona P, Gomes R G. Discriminative clustering by regularized information maximization//Advances in Neural Information Processing Systems 23(NIPS 2010). 2010:775-783.
[6] Springenberg J T. Unsupervised and semi-supervised learning with categorical generative adversarial networks. ArXiv preprint, arXiv:1511.06390.(2016-04-30). https://arxiv.org/abs/1511.06390.
[7] 宋旭鸣,沈逸飞,石远明.基于深度学习的智能移动边缘网络缓存[J].中国科学院大学学报, 2020,37(1):128-135.DOI:10.7523/j.issn.2095-6134.2020.01.015.
[8] 田玮,朱廷劭.基于深度学习的微博用户自杀风险预测[J].中国科学院大学学报, 2018, 35(1):131-136.DOI:10.7523/j.issn.2095-6134.2018.01.018.
[9] 杨建斌,张卫强,刘加.深度神经网络自适应中基于身份认证向量的归一化方法[J].中国科学院大学学报, 2017, 34(5):633-639.DOI:10.7523/j.issn.2095-6134.2017.05.014.
[10] Salakhutdinov R, Hinton G. Deep boltzmann machines[J].Journal of Machine Learning Research, 2009,5:448-455.
[11] Goodfellow I, Mirza M, Courville A, et al. Multi-prediction deep Boltzmann machines//Advances in Neural Information Processing Systems 26(NIPS 2013). 2013:548-556.
[12] Bengio Y, Laufer E, Alain G, et al. Deep generative stochastic networks trainable by backprop//Proceedings of the 31st International Conference on Machine Learning, PMLR, 2014,32(2):226-234.
[13] Kingma D P, Mohamed S, Rezende D J, et al. Semi-supervised learning with deep generative models//Advances in Neural Information Processing Systems 27(NIPS 2014). 2014:3581-3589.
[14] Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786):504-507.DOI:10.1126/science.1127647.
[15] Vincent P, Larochelle H, Bengio Y, et al. Extracting and composing robust features with denoising autoencoders//Proceedings of the 25th International Conference on Machine Learning (ICML). 2008:1096-1103.DOI:10.1145/1390156.1390294.
[16] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Advances in Neural Information Processing Systems 27(NIPS 2014). 2014:2672-2680.
[17] Chen X, Duan Y, Houthooft R, et al. InfoGAN:interpretable representation learning by information maximizing generative adversarial nets//Advances in Neural Information Processing Systems 29(NIPS 2016). 2016:2172-2180.
[18] LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation, 1989, 1(4):541-551.DOI:10.1162/neco.1989.1.4.541.
[19] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images. Computer Science Department, University of Toronto, Tech. 2009.
[20] Li C X, Xu T, Zhu J, et al. Triple generative adversarial nets//Advances in Neural Information Processing Systems 30(NIPS 2017). 2017:4088-4098.
[21] Wu S, Deng G C, Li J C, et al. Enhancing TripleGAN for semi-supervised conditional instance synthesis and classification//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:10083-10092.DOI:10.1109/CVPR.2019.01033.
[22] Poole B, Ozair S, van der Oord A, et al. On variational bounds of mutual information//Proceedings of the 36th International Conference on Machine Learning, PMLR.2019,97:5171-5180.
[23] Grandvalet Y, Bengio Y. Semi-supervised learning by entropy minimization//Advances in Neural Information Processing Systems 17(NIPS 2004). 2005:529-536.
[24] Spurr A, Aksan E, Hilliges O. Guiding InfoGAN with semi-supervision//Joint European Conference on Machine Learning and Knowledge Discovery in Databases. ECML PKDD, 2017:119134.DOI:10.1007/978-3-319-71249-9_8.
[25] Heusel M, Ramsauer H, Unterthiner T, et al. Gans trained by a two time-scale update rule converge to a local nash equilibrium//Advances in Neural Information Processing Systems 30(NIPS 2017). 2017:6626-6637.
[26] Salimans T, Goodfellow I, Zaremba W, et al. Improved techniques for training GANs//Advances in Neural Information Processing Systems 29(NIPS). 2016:2234-2242.
[27] Xiao H, Rasul K, Vollgraf R. Fashion-MNIST:a novel image dataset for benchmarking machine learning algorithms. ArXiv Preprint, arXiv:1708.07747.(2017-09-15). https://arxiv.org/abs/1708.07747.