Journal of University of Chinese Academy of Sciences >
Semi-supervised self-training image classification method based on gamma divergence
Received date: 2023-10-18
Revised date: 2024-04-22
Online published: 2024-06-04
In recent years， various semi-supervised self-training methods have made significant progress in solving image classification problems and have attracted widespread attention. However， even the state-of-the-art methods like FixMatch can suffer from training instability and extreme performance imbalance between classes due to the accumulation of pseudo-labeling errors. To address this issue， this paper introduces the concept of robust gamma divergence on top of the FixMatch method and proposes a semi-supervised self-training approach based on robust gamma divergence. In this method， we incorporate gamma divergence as a regularization term to the loss function of the training model to correct the mislabeled instances caused by error accumulation. Additionally， we conduct comparative experiments on artificially polluted simulated datasets and the CIFAR-10 dataset to demonstrate the superiority of the GammaFixMatch method in handling the problem of pseudo-label error accumulation.
Ziqian MAO , Sanguo ZHANG . Semi-supervised self-training image classification method based on gamma divergence[J]. Journal of University of Chinese Academy of Sciences, 2026 , 43(4) : 433 -443 . DOI: 10.7523/j.ucas.2024.042
| [1] | Krizhevsky A， Sutskever I， Hinton G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM， 2017， 60（6）： 84-90.DOI：10.1145/3065386 . |
| [2] | Grandvalet Y， Bengio Y. Semi-supervised learning by entropy minimization［C］//Proceedings of the 17th International Conference on Neural Information Processing Systems. December 13-18， 2004， Vancouver， BC， Canada. ACM， 2004： 529 - 536. |
| [3] | Chen T， Kornblith S， Swersky K， et al. Big self-supervised models are strong semi-supervised learners［C］//Proceedings of the 34th International Conference on Neural Information Processing Systems. December 6 - 12， 2020， Vancouver， BC， Canada. ACM， 2020： 22243 -22255. DOI： 10.5555/3495724.3497589 . |
| [4] | Lee D H. Pseudo-label： the simple and efficient semi-supervised learning method for deep neural networks［C］//Workshop on challenges in representation learning， ICML. 2013， 3（2）： 896. |
| [5] | Sohn K， Berthelot D， Li C L， et al. FixMatch： simplifying semi-supervised learning with consistency and confidence［C］//Proceedings of the 34th International Conference on Neural Information Processing Systems. December 6 - 12， 2020， Vancouver， BC， Canada. ACM， 2020： 596 - 608. DOI： 10.5555/3495724.3495775 . |
| [6] | Kirkpatrick J， Pascanu R， Rabinowitz N， et al. Overcoming catastrophic forgetting in neural networks［J］. Proceedings of the National Academy of Sciences of the United States of America， 2017， 114（13）： 3521-3526. DOI： 10.1073/pnas.1611835114 . |
| [7] | Chen B， Jiang J， Wang X， et al. Debiased self-training for semi-supervised learning［C］//Proceedings of the 36th International Conference on Neural Information Processing Systems. DOI：10.48550/2202.07136 . |
| [8] | Devlin J， Chang M W， Lee K， et al. BERT： pre-training of deep bidirectional transformers for language understanding［EB/OL］. arXiv 2018： 1810.04805. （2018-10-11）［2024-04-10］.. |
| [9] | Zhou B L， Lapedriza A， Khosla A， et al. Places： a 10 million image database for scene recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018， 40（6）： 1452-1464. DOI：10.1109/TPAMI.2017.2723009 . |
| [10] | Fujisawa H， Eguchi S. Robust parameter estimation with a small bias against heavy contamination［J］. Journal of Multivariate Analysis， 2008， 99（9）： 2053-2081. DOI： 10.1016/j.jmva.2008.02.004 . |
| [11] | Ren M Y， Zhang S G， Ma S G， et al. Gene–environment interaction identification via penalized robust divergence［J］. Biometrical Journal. Biometrische Zeitschrift， 2022， 64（3）： 461-480. DOI： 10.1002/bimj.202000157 . |
| [12] | Hung H， Jou Z Y， Huang S Y. Robust mislabel logistic regression without modeling mislabel probabilities［J］. Biometrics， 2018， 74（1）： 145-154.DOI：10.1111/biom.12726 . |
| [13] | Jones M C， Hjort N L， Harris I R， et al. A comparison of related density‐based minimum divergence estimators［J］. Biometrika， 2001， 88（3）： 865-873. DOI： 10.1093/biomet/88.3.865 . |
| [14] | Zhang Z L， Sabuncu M R. Generalized cross entropy loss for training deep neural networks with noisy labels［C］//Proceedings of the 32nd International Conference on Neural Information Processing Systems. December 3 - 8， 2018， Montréal， Canada. ACM， 2018： 8792–8802. DOI： 10.5555/3327546.3327555 . |
| [15] | Amini M R， Feofanov V， Pauletto L， et al. Self-training： a survey［EB/OL］. arXiv 2022： 2202.12040.（2022-02-24）［2024-04-10］.. |
| [16] | Zou Y， Yu Z D， Liu X F， et al. Confidence regularized self-training［C］//2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Seoul， Korea （South）. IEEE， 2019： 5981-5990. DOI： 10.1109/ICCV.2019.00608 . |
| [17] | Huber P J. Robust statistics［M］. Wiley， 1982. DOI： 10.1002/0471725250 . |
| [18] | Ouali Y， Hudelot C， Tami M. Semi-supervised semantic segmentation with cross-consistency training［C］//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle， WA， USA. IEEE， 2020： 12671-12681. DOI： 10.1109/CVPR42600.2020.01269 . |
| [19] | Lai X， Tian Z T， Jiang L， et al. Semi-supervised semantic segmentation with directional context-aware consistency［C］//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Nashville， TN， USA. IEEE， 2021： 1205-1214. DOI： 10.1109/CVPR46437.2021.00126 . |
| [20] | Cubuk E D， Zoph B， Shlens J， et al. Randaugment： practical automated data augmentation with a reduced search space［C］//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Seattle， WA， USA. IEEE， 2020： 3008-3017. DOI： 10.1109/CVPRW50498.2020.00359 . |
| [21] | Berthelot D， Carlini N， Cubuk E D， et al. ReMixMatch： semi-supervised learning with distribution alignment and augmentation anchoring［EB/OL］. arXiv 2019： 1911.09785.（2019-11-21）［2024-04-10］. . |
| [22] | Berthelot D， Carlini N， Goodfellow I， et al. MixMatch： a holistic approach to semi-supervised learning［EB/OL］. arXiv 2019： 1905.02249.（2019-05-06）［2024-04-10］.. |
/
| 〈 |
|
〉 |