基于伽马散度的半监督自训练图像分类
收稿日期: 2023-10-18
修回日期: 2024-04-22
网络出版日期: 2024-06-04
基金资助
国家自然科学基金(12171454);国家自然科学基金(U19B2940);中央高校基本科研业务费专项资助
Semi-supervised self-training image classification method based on gamma divergence
Received date: 2023-10-18
Revised date: 2024-04-22
Online published: 2024-06-04
近年来，各种半监督自训练方法在解决图像分类问题时取得了巨大进展，受到人们广泛关注。然而，即使是主流前沿的FixMatch方法，在模型训练过程中也会出现伪标签错误累积导致的训练不稳定和类别间极端性能失衡问题。为解决该问题，在现有FixMatch方法的基础上引入具有稳健性质的伽马散度，并提出基于伽马散度的半监督自训练方法。该方法将伽马散度以正则项的形式添加到训练模型的损失函数中，对错误记忆的伪标签进行纠偏处理。在人工添加污染的模拟数据集和CIFAR-10数据集上进行不同方法的对比实验，验证了GammaFixMatch方法处理伪标签错误累积问题的优越性。
毛紫茜 , 张三国 . 基于伽马散度的半监督自训练图像分类[J]. 中国科学院大学学报, 2026 , 43(4) : 433 -443 . DOI: 10.7523/j.ucas.2024.042
In recent years， various semi-supervised self-training methods have made significant progress in solving image classification problems and have attracted widespread attention. However， even the state-of-the-art methods like FixMatch can suffer from training instability and extreme performance imbalance between classes due to the accumulation of pseudo-labeling errors. To address this issue， this paper introduces the concept of robust gamma divergence on top of the FixMatch method and proposes a semi-supervised self-training approach based on robust gamma divergence. In this method， we incorporate gamma divergence as a regularization term to the loss function of the training model to correct the mislabeled instances caused by error accumulation. Additionally， we conduct comparative experiments on artificially polluted simulated datasets and the CIFAR-10 dataset to demonstrate the superiority of the GammaFixMatch method in handling the problem of pseudo-label error accumulation.
| [1] | Krizhevsky A， Sutskever I， Hinton G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM， 2017， 60（6）： 84-90.DOI：10.1145/3065386 . |
| [2] | Grandvalet Y， Bengio Y. Semi-supervised learning by entropy minimization［C］//Proceedings of the 17th International Conference on Neural Information Processing Systems. December 13-18， 2004， Vancouver， BC， Canada. ACM， 2004： 529 - 536. |
| [3] | Chen T， Kornblith S， Swersky K， et al. Big self-supervised models are strong semi-supervised learners［C］//Proceedings of the 34th International Conference on Neural Information Processing Systems. December 6 - 12， 2020， Vancouver， BC， Canada. ACM， 2020： 22243 -22255. DOI： 10.5555/3495724.3497589 . |
| [4] | Lee D H. Pseudo-label： the simple and efficient semi-supervised learning method for deep neural networks［C］//Workshop on challenges in representation learning， ICML. 2013， 3（2）： 896. |
| [5] | Sohn K， Berthelot D， Li C L， et al. FixMatch： simplifying semi-supervised learning with consistency and confidence［C］//Proceedings of the 34th International Conference on Neural Information Processing Systems. December 6 - 12， 2020， Vancouver， BC， Canada. ACM， 2020： 596 - 608. DOI： 10.5555/3495724.3495775 . |
| [6] | Kirkpatrick J， Pascanu R， Rabinowitz N， et al. Overcoming catastrophic forgetting in neural networks［J］. Proceedings of the National Academy of Sciences of the United States of America， 2017， 114（13）： 3521-3526. DOI： 10.1073/pnas.1611835114 . |
| [7] | Chen B， Jiang J， Wang X， et al. Debiased self-training for semi-supervised learning［C］//Proceedings of the 36th International Conference on Neural Information Processing Systems. DOI：10.48550/2202.07136 . |
| [8] | Devlin J， Chang M W， Lee K， et al. BERT： pre-training of deep bidirectional transformers for language understanding［EB/OL］. arXiv 2018： 1810.04805. （2018-10-11）［2024-04-10］.. |
| [9] | Zhou B L， Lapedriza A， Khosla A， et al. Places： a 10 million image database for scene recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018， 40（6）： 1452-1464. DOI：10.1109/TPAMI.2017.2723009 . |
| [10] | Fujisawa H， Eguchi S. Robust parameter estimation with a small bias against heavy contamination［J］. Journal of Multivariate Analysis， 2008， 99（9）： 2053-2081. DOI： 10.1016/j.jmva.2008.02.004 . |
| [11] | Ren M Y， Zhang S G， Ma S G， et al. Gene–environment interaction identification via penalized robust divergence［J］. Biometrical Journal. Biometrische Zeitschrift， 2022， 64（3）： 461-480. DOI： 10.1002/bimj.202000157 . |
| [12] | Hung H， Jou Z Y， Huang S Y. Robust mislabel logistic regression without modeling mislabel probabilities［J］. Biometrics， 2018， 74（1）： 145-154.DOI：10.1111/biom.12726 . |
| [13] | Jones M C， Hjort N L， Harris I R， et al. A comparison of related density‐based minimum divergence estimators［J］. Biometrika， 2001， 88（3）： 865-873. DOI： 10.1093/biomet/88.3.865 . |
| [14] | Zhang Z L， Sabuncu M R. Generalized cross entropy loss for training deep neural networks with noisy labels［C］//Proceedings of the 32nd International Conference on Neural Information Processing Systems. December 3 - 8， 2018， Montréal， Canada. ACM， 2018： 8792–8802. DOI： 10.5555/3327546.3327555 . |
| [15] | Amini M R， Feofanov V， Pauletto L， et al. Self-training： a survey［EB/OL］. arXiv 2022： 2202.12040.（2022-02-24）［2024-04-10］.. |
| [16] | Zou Y， Yu Z D， Liu X F， et al. Confidence regularized self-training［C］//2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Seoul， Korea （South）. IEEE， 2019： 5981-5990. DOI： 10.1109/ICCV.2019.00608 . |
| [17] | Huber P J. Robust statistics［M］. Wiley， 1982. DOI： 10.1002/0471725250 . |
| [18] | Ouali Y， Hudelot C， Tami M. Semi-supervised semantic segmentation with cross-consistency training［C］//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle， WA， USA. IEEE， 2020： 12671-12681. DOI： 10.1109/CVPR42600.2020.01269 . |
| [19] | Lai X， Tian Z T， Jiang L， et al. Semi-supervised semantic segmentation with directional context-aware consistency［C］//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Nashville， TN， USA. IEEE， 2021： 1205-1214. DOI： 10.1109/CVPR46437.2021.00126 . |
| [20] | Cubuk E D， Zoph B， Shlens J， et al. Randaugment： practical automated data augmentation with a reduced search space［C］//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Seattle， WA， USA. IEEE， 2020： 3008-3017. DOI： 10.1109/CVPRW50498.2020.00359 . |
| [21] | Berthelot D， Carlini N， Cubuk E D， et al. ReMixMatch： semi-supervised learning with distribution alignment and augmentation anchoring［EB/OL］. arXiv 2019： 1911.09785.（2019-11-21）［2024-04-10］. . |
| [22] | Berthelot D， Carlini N， Goodfellow I， et al. MixMatch： a holistic approach to semi-supervised learning［EB/OL］. arXiv 2019： 1905.02249.（2019-05-06）［2024-04-10］.. |
/
| 〈 |
|
〉 |
