欢迎访问中国科学院大学学报,今天是
数学与物理学

基于模型平均与γ-散度的稳健半监督学习方法

  • 吴慧桢 ,
  • 张三国
展开
  • 中国科学院大学数学科学学院 中国科学院大数据挖掘与知识管理重点实验室,北京 100049

收稿日期: 2024-01-03

  修回日期: 2024-04-18

  网络出版日期: 2024-05-22

基金资助

国家自然科学基金(12171454);国家自然科学基金(U19B2940);中央高校基本科研业务费专项资助

Robust semi-supervised learning model based on model averaging and γ-divergence

  • Huizhen WU ,
  • Sanguo ZHANG
Expand
  • CAS Key Laboratory of Big Data Mining and Knowledge Management,School of Mathematical Sciences,University of Chinese Academy of Sciences,Beijing 100049,China

Received date: 2024-01-03

  Revised date: 2024-04-18

  Online published: 2024-05-22

摘要

提出一种基于模型平均与γ-散度的稳健半监督方法:一方面,通过引入模型平均方法解决无标签数据质量不高的问题;另一方面,通过引入基于γ-散度的逻辑回归解决有标签数据存在误标签的问题。该模型的优点在于,它能够利用不同模型的预测差异来处理数据,有效利用无标签数据的信息,同时尽可能减少其中的有害信息;并通过引入γ-散度减少有标签数据中误标签数据对拟合效果的影响,最终得到对于无标签数据和有标签数据均稳健的模型。模拟研究和Breast Cancer数据应用表明,与现有半监督学习方法相比,当数据质量较低时,所提出的新方法在预测性能上有明显的提升。

本文引用格式

吴慧桢 , 张三国 . 基于模型平均与γ-散度的稳健半监督学习方法[J]. 中国科学院大学学报, 2026 , 43(1) : 14 -22 . DOI: 10.7523/j.ucas.2024.026

Abstract

Semi-supervised learning is a key research problem in the field of pattern recognition and machine learning, and has been widely used in various fields in recent years. In practical problems, labeled samples are costly to obtain, while unlabeled samples are easier to obtain despite the lack of labeling information. Semi-supervised learning uses a large amount of unlabeled data and a small amount of labeled data at the same time to perform pattern recognition work. In this paper, we propose a robust semi-supervised approach based on model averaging and γ-divergence: on the one hand, the problem of low quality of unlabeled data is addressed by introducing model averaging method; on the other hand, the problem of mislabeling of labeled data is addressed by introducing logistic regression based on γ-divergence. One of the advantages of the proposed model is that we can process the data by exploiting the predictive differences among the different models to effectively utilize the information of the unlabeled data while minimizing the harmful information in it. By introducing γ-divergence to reduce the effect of mislabeled data in labeled data on the fitting effect, we ultimately obtain a model that is robust for both unlabeled and labeled data. Simulation studies and applications of Breast Cancer Wisconsin Dataset show that compared with existing semi-supervised learning methods, the new method proposed in this paper has a significant improvement in prediction performance when the data quality is low.

参考文献

[1] Bennett K, Demiriz A. Semi-supervised support vector machines[C]//Advances in Neural Information processing systems. Cambridge, MA, USA. MIT Press, 1998:368-374.DOI: 10.5555/340534.340671 .
[2] Joachims T. Transductive inference for text classification using support vector machines[C]// Proceedings of the Sixteenth International Conference on Machine Learning 99. San Francisco, CA, USA. Morgan Kaufmann Publishers Inc,1999:200-209. DOI: 10.5555/645528.657646 .
[3] Pan D W, Nie L Q, Kang W X, et al. UAV anomaly detection using active learning and improved S3VM model[C]//2020 International Conference on Sensing, Measurement & Data Analytics in the era of Artificial Intelligence(ICSMD). Xi’ an, China. IEEE,2020:253-258. DOI:10.1109/ICSMD50554.2020.9261709 .
[4] Tao J T, Zhang N N, Chang J Y, et al. Unlabeled sample selection for mineral prospectivity mapping by semi-supervised support vector machine[J]. Natural Resources Research202231(5): 2247-2269. DOI:10.1007/s11053-022-10093-0 .
[5] Kawashima T, Fujisawa H. Robust and sparse regression in generalized linear model by stochastic optimization[J]. Japanese Journal of Statistics and Data Science20192(2): 465-489. DOI:10.1007/s42081-019-00049-9 .
[6] Hung H, Jou Z Y, Huang S Y. Robust mislabel logistic regression without modeling mislabel probabilities[J]. Biometrics201874(1): 145-154. DOI:10.1111/biom.12726 .
[7] Kawashima T, Fujisawa H. Robust and sparse regression via γ-divergence [J]. Entropy201719(11):608. DOI:10.3390/e19110608 .
[8] Laine S, Aila T. Temporal ensembling for semi-supervised learning[EB/OL]. arXiv 2016:1610.02242. (2016-10-07)[2024-04-09]. .
[9] Tarvainen A, Valpola H. Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results[C]//Advances in neural information processing systems. Red Hook, NY, USA. Curran Associates Inc, 2017:1195-1204. DOI: 10.5555/3294771.3294885 .
[10] Al-Azzam N, Shatnawi I. Comparing supervised and semi-supervised machine learning models on diagnosing breast cancer[J]. Annals of Medicine and Surgery202162: 53-64. DOI:10.1016/j.amsu.2020.12.043 .
文章导航

/