基于伽马散度的半监督自训练图像分类*

doi:10.7523/j.ucas.2024.042

中国科学院大学学报

基于伽马散度的半监督自训练图像分类^*

毛紫茜, 张三国^†

中国科学院大学数学科学学院,北京 100049;
中国科学院大数据挖掘与知识管理重点实验室,北京 100049

收稿日期:2023-10-18 修回日期:2024-04-22 发布日期:2024-06-04
通讯作者: ^†E-mail：sgzhang@ucas.ac.cn
基金资助:
^*国家自然科学基金（批准号：12171454 ,U19B2940）和中央高校基本科研业务费专项资金资助

A semi-supervised self-training image classification method based on gamma divergence

MAO Ziqian, ZHANG Sanguo

School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China;
Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing 100049, China

Received:2023-10-18 Revised:2024-04-22 Published:2024-06-04

摘要/Abstract

摘要： 近年来,各种半监督自训练方法在解决图像分类问题时取得了巨大进展,并受到人们广泛关注。然而,即使是主流前沿的FixMatch方法,在模型训练过程中也会出现伪标签错误累积导致的训练不稳定和类别间极端性能失衡问题。为了解决该一问题,本文在现有的FixMatch方法的基础上引入具有稳健性质的伽马散度,并提出基于伽马散度的半监督自训练方法。在该方法中,我们将伽马散度以正则项的形式添加到训练模型的损失函数中,对错误记忆的伪标签进行纠偏处理。同时,我们在人工添加污染的模拟数据集和CIFAR-10数据集上进行不同方法的对比实验,验证了GammaFixMatch方法处理伪标签错误累积问题的优越性质。

关键词: 半监督, 自训练, 伽马散度

Abstract: In recent years, various semi-supervised self-training methods have made significant progress in solving image classification problems and have attracted widespread attention. However, even the state-of-the-art methods like FixMatch can suffer from training instability and extreme performance imbalance between classes due to the accumulation of pseudo-labeling errors. To address this issue, this paper introduces the concept of robust gamma divergence on top of the FixMatch method and proposes a semi-supervised self-training approach based on gamma divergence. In this method, we incorporate gamma divergence as a regularization term to the loss function of the training model to correct the mislabeled instances caused by error accumulation. Additionally, we conduct comparative experiments on artificially polluted simulated datasets and the CIFAR-10 dataset to demonstrate the superiority of the GammaFixMatch method in handling the problem of pseudo-labeling error accumulation.

Key words: Semi-supervised, Self-training, Gamma Divergence

中图分类号:

毛紫茜, 张三国. 基于伽马散度的半监督自训练图像分类^*[J]. 中国科学院大学学报, DOI: 10.7523/j.ucas.2024.042.

MAO Ziqian, ZHANG Sanguo. A semi-supervised self-training image classification method based on gamma divergence[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.042.

参考文献

[1] Krizhevsky A, Sutskever I, Hinton G E.ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.DOI:10.1145/3065386.
[2] Grandvalet Y, Bengio Y.Semi-supervised learning by entropy minimization[C]//Proceedings of the 17th International Conference on Neural Information Processing Systems. December 13-18, 2004, Vancouver, BC, Canada. ACM, 2004: 529-536.
[3] Chen T, Kornblith S, Swersky K, et al.Big self-supervised models are strong semi-supervised learners[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. December 6-12, 2020, Vancouver, BC, Canada. ACM, 2020: 22243-22255. DOI: 10.5555/3495724.3497589.
[4] Lee D H.Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks[C]//Workshop on challenges in representation learning, ICML. 2013, 3(2): 896.
[5] Sohn K, Berthelot D,Li C L, et al.FixMatch: Simplifying semi-supervised learning with consistency and confidence[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. December 6-12, 2020, Vancouver, BC, Canada. ACM, 2020: 596-608. DOI: 10.5555/3495724.3495775.
[6] Kirkpatrick J, Pascanu R, Rabinowitz N, et al.Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114(13): 3521-3526. DOI: 10.1073/pnas.1611835114.
[7] Chen B, Jiang J, Wang X, et al.Debiased self-training for semi-supervised learning[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. DOI:10.48550/2202.07136.
[8] Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. 2018: arXiv: 1810.04805. (2018-10-11)[2024-04-10].http://arxiv.org/abs/1810.04805.pdf.
[9] Zhou B L, Lapedriza A, Khosla A, et al.Places: A 10 million image database for scene recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6): 1452-1464. DOI:10.1109/TPAMI.2017.2723009
[10] Fujisawa H, Eguchi S.Robust parameter estimation with a small bias against heavy contamination[J]. Journal of Multivariate Analysis, 2008, 99(9): 2053-2081. DOI: 10.1016/j.jmva.2008.02.004.
[11] Ren M Y, Zhang S G, Ma S G, et al.Gene-environment interaction identification via penalized robust divergence[J]. Biometrical Journal. Biometrische Zeitschrift, 2022, 64(3): 461-480. DOI: 10.1002/bimj.202000157.
[12] Hung H, Jou Z Y, Huang S Y.Robust mislabel logistic regression without modeling mislabel probabilities[J]. Biometrics, 2018, 74(1): 145-154.DOI:10.1111/biom.12726.
[13] Jones M C, Hjort N L, Harris I R, et al.A comparison of related density‐based minimum divergence estimators[J]. Biometrika, 2001, 88(3): 865-873. DOI: 10.1093/biomet/88.3.865.
[14] Zhang Z L, Sabuncu M R.Generalized cross entropy loss for training deep neural networks with noisy labels[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. December 3-8, 2018, Montréal, Canada. ACM, 2018: 8792-8802. DOI: 10.5555/3327546.3327555.
[15] Amini M R, Feofanov V, Pauletto L, et al. Self-training: a survey[EB/OL].2022: arXiv: 2202.12040.(2022-02-24)[2024-04-10].https://doi.org/10.48550/arXiv.2202.12040.
[16] Zou Y, Yu Z D, Liu X F, et al.Confidence regularized self-training[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South). IEEE, 2019: 5981-5990. DOI: 10.1109/ICCV.2019.00608.
[17] Huber P J.Robust statistics[M]. Wiley, 1982. DOI: 10.1002/0471725250.
[18] Ouali Y, Hudelot C, Tami M.Semi-supervised semantic segmentation with cross-consistency training[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. IEEE, 2020: 12671-12681. DOI: 10.1109/CVPR42600.2020.01269.
[19] Lai X, Tian Z T, Jiang L, et al.Semi-supervised semantic segmentation with directional context-aware consistency[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA. IEEE, 2021: 1205-1214. DOI: 10.1109/CVPR46437.2021.00126.
[20] Cubuk E D, Zoph B, Shlens J, et al.Randaugment: Practical automated data augmentation with a reduced search space[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, WA, USA. IEEE, 2020: 3008-3017. DOI: 10.1109/CVPRW50498.2020.00359.
[21] Berthelot D, Carlini N, Cubuk E D, et al.ReMixMatch: semi-supervised learning with distribution alignment and augmentation anchoring[EB/OL]. 2019: arXiv: 1911.09785.(2019-11-21)[2024-04-10]. https://doi.org/10.48550/arXiv.1911.09785.
[22] Berthelot D, Carlini N, Goodfellow I, et al.MixMatch: a holistic approach to semi-supervised learning[EB/OL]. 2019: arXiv: 1905.02249.(2019-05-06)[2024-04-10].https://doi.org/10.48550/arXiv.1905.02249

基于伽马散度的半监督自训练图像分类^*

A semi-supervised self-training image classification method based on gamma divergence

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	胡兵兵, 唐华, 吴幼龙. 基于互信息约束的生成对抗网络分类模型[J]. 中国科学院大学学报, 2022, 39(4): 551-560.
[2]	杨雨龙, 郭田德, 韩丛英. 基于原型学习改进的伪标签半监督学习算法[J]. 中国科学院大学学报, 2021, 38(6): 841-851.
[3]	陈国炜, 刘磊, 郭嘉逸, 潘宗序, 胡文龙. 基于生成对抗网络的半监督遥感图像飞机检测[J]. 中国科学院大学学报, 2020, 37(4): 539-546.