Welcome to Journal of University of Chinese Academy of Sciences,Today is
Mathematics & Physics

Robust semi-supervised learning model based on model averaging and γ-divergence

  • Huizhen WU ,
  • Sanguo ZHANG
Expand
  • CAS Key Laboratory of Big Data Mining and Knowledge Management,School of Mathematical Sciences,University of Chinese Academy of Sciences,Beijing 100049,China

Received date: 2024-01-03

  Revised date: 2024-04-18

  Online published: 2024-05-22

Abstract

Semi-supervised learning is a key research problem in the field of pattern recognition and machine learning, and has been widely used in various fields in recent years. In practical problems, labeled samples are costly to obtain, while unlabeled samples are easier to obtain despite the lack of labeling information. Semi-supervised learning uses a large amount of unlabeled data and a small amount of labeled data at the same time to perform pattern recognition work. In this paper, we propose a robust semi-supervised approach based on model averaging and γ-divergence: on the one hand, the problem of low quality of unlabeled data is addressed by introducing model averaging method; on the other hand, the problem of mislabeling of labeled data is addressed by introducing logistic regression based on γ-divergence. One of the advantages of the proposed model is that we can process the data by exploiting the predictive differences among the different models to effectively utilize the information of the unlabeled data while minimizing the harmful information in it. By introducing γ-divergence to reduce the effect of mislabeled data in labeled data on the fitting effect, we ultimately obtain a model that is robust for both unlabeled and labeled data. Simulation studies and applications of Breast Cancer Wisconsin Dataset show that compared with existing semi-supervised learning methods, the new method proposed in this paper has a significant improvement in prediction performance when the data quality is low.

Cite this article

Huizhen WU , Sanguo ZHANG . Robust semi-supervised learning model based on model averaging and γ-divergence[J]. Journal of University of Chinese Academy of Sciences, 2026 , 43(1) : 14 -22 . DOI: 10.7523/j.ucas.2024.026

References

[1] Bennett K, Demiriz A. Semi-supervised support vector machines[C]//Advances in Neural Information processing systems. Cambridge, MA, USA. MIT Press, 1998:368-374.DOI: 10.5555/340534.340671 .
[2] Joachims T. Transductive inference for text classification using support vector machines[C]// Proceedings of the Sixteenth International Conference on Machine Learning 99. San Francisco, CA, USA. Morgan Kaufmann Publishers Inc,1999:200-209. DOI: 10.5555/645528.657646 .
[3] Pan D W, Nie L Q, Kang W X, et al. UAV anomaly detection using active learning and improved S3VM model[C]//2020 International Conference on Sensing, Measurement & Data Analytics in the era of Artificial Intelligence(ICSMD). Xi’ an, China. IEEE,2020:253-258. DOI:10.1109/ICSMD50554.2020.9261709 .
[4] Tao J T, Zhang N N, Chang J Y, et al. Unlabeled sample selection for mineral prospectivity mapping by semi-supervised support vector machine[J]. Natural Resources Research202231(5): 2247-2269. DOI:10.1007/s11053-022-10093-0 .
[5] Kawashima T, Fujisawa H. Robust and sparse regression in generalized linear model by stochastic optimization[J]. Japanese Journal of Statistics and Data Science20192(2): 465-489. DOI:10.1007/s42081-019-00049-9 .
[6] Hung H, Jou Z Y, Huang S Y. Robust mislabel logistic regression without modeling mislabel probabilities[J]. Biometrics201874(1): 145-154. DOI:10.1111/biom.12726 .
[7] Kawashima T, Fujisawa H. Robust and sparse regression via γ-divergence [J]. Entropy201719(11):608. DOI:10.3390/e19110608 .
[8] Laine S, Aila T. Temporal ensembling for semi-supervised learning[EB/OL]. arXiv 2016:1610.02242. (2016-10-07)[2024-04-09]. .
[9] Tarvainen A, Valpola H. Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results[C]//Advances in neural information processing systems. Red Hook, NY, USA. Curran Associates Inc, 2017:1195-1204. DOI: 10.5555/3294771.3294885 .
[10] Al-Azzam N, Shatnawi I. Comparing supervised and semi-supervised machine learning models on diagnosing breast cancer[J]. Annals of Medicine and Surgery202162: 53-64. DOI:10.1016/j.amsu.2020.12.043 .
Outlines

/