Robust semi-supervised learning model based on model averaging and γ-divergence

doi:10.7523/j.ucas.2024.026

Abstract

Abstract: Semi-supervised learning is a key research problem in the field of pattern recognition and machine learning, and has been widely used in various fields in recent years. In practical problems, labeled samples are costly to obtain, while unlabeled samples are easier to obtain despite the lack of labeling information. Semi-supervised learning uses a large amount of unlabeled data and a small amount of labeled data at the same time to perform pattern recognition work. In this paper, we propose a robust semi-supervised approach based on model averaging and γ-divergence: on the one hand, the problem of low quality of unlabeled data is addressed by introducing model averaging method; on the other hand, the problem of mislabeling of labeled data is addressed by introducing logistic regression based on γ-divergence. One of the advantages of the proposed model is that we are able to process the data by exploiting the predictive differences of the different models to effectively utilize the information of the unlabeled data while minimizing the harmful information in it. And by introducing γ-divergence to reduce the effect of mislabeled data in labeled data on the fitting effect, we ultimately obtain a model that is robust for both unlabeled and labeled data. Simulation studies and applications of Breast Cancer Wisconsin Dataset show that compared with existing semi-supervised learning methods, the new method proposed in this paper has a significant improvement in prediction performance when the data quality is low.

Key words: semi-supervised learning, model averaging, γ-divergence, robustness

CLC Number:

O212.1

WU Huizhen, ZHANG Sanguo. Robust semi-supervised learning model based on model averaging and γ-divergence[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.026.

References

[1] Bennett K, Demiriz A.Semi-supervised support vector machines[C]//Advances in Neural Information processing systems. Cambridge, MA, USA. MIT Press, 1998:368-374.DOI: 10.5555/340534.340671.
[2] Joachims T.Transductive inference for text classification using support vector machines[C]// Proceedings of the Sixteenth International Conference on Machine Learning 99. San Francisco, CA, USA. Morgan Kaufmann Publishers Inc,1999:200-209. DOI: 10.5555/645528.657646.
[3] Pan D W, Nie L Q, Kang W X, et al.UAV anomaly detection using active learning and improved S3VM model[C]//2020 International Conference on Sensing, Measurement & Data Analytics in the era of Artificial Intelligence(ICSMD). Xi' an, China. IEEE,2020:253-258. DOI:10.1109/ICSMD50554.2020.9261709.
[4] Tao J T, Zhang N N, Chang J Y, et al.Unlabeled sample selection for mineral prospectivity mapping by semi-supervised support vector machine[J]. Natural Resources Research, 2022,31(5): 2247-2269. DOI:10.1007/s11053-022-10093-0.
[5] Kawashima T, Fujisawa H.Robust and sparse regression in generalized linear model by stochastic optimization[J]. Japanese Journal of Statistics and Data Science, 2019,2(2): 465-489. DOI:10.1007/s42081-019-00049-9.
[6] Hung H, Jou Z Y, Huang S Y.Robust mislabel logistic regression without modeling mislabel probabilities[J]. Biometrics, 2018, 74(1): 145-154. DOI:10.1111/biom.12726.
[7] Kawashima T, Fujisawa H.Robust and Sparse Regression via γ-Divergence[J]. Entropy, 2017,19(11):608. DOI:10.3390/e19110608.
[8] Laine S, Aila T. Temporal ensembling for semi-supervised learning[EB/OL]. arXiv preprint arXiv:1610.02242, (2016-10-7)[2024-4-9]. https://arxiv.org/abs/1610.02242.
[9] Tarvainen A, Valpola H.Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[C]//Advances in neural information processing systems. Red Hook, NY, USA. Curran Associates Inc, 2017:1195-1204. DOI: 10.5555/3294771.3294885.
[10] Al-Azzam N, Shatnawi I.Comparing supervised and semi-supervised Machine Learning Models on Diagnosing Breast Cancer[J]. Annals of Medicine and Surgery, 2021, 62: 53-64. DOI:10.1016/j.amsu.2020.12.043.