基于序贯算法的重构密度估计*

doi:10.7523/j.ucas.2024.018

中国科学院大学学报

基于序贯算法的重构密度估计^*

黄思源¹, 谢田法¹, 熊世峰^2,3,†

1 北京工业大学数学统计学与力学学院,北京100124;
2 中国科学院大学数学科学学院,北京 101408;
3 中国科学院数学与系统科学研究院,北京 100190

收稿日期:2023-10-20 修回日期:2024-04-03 发布日期:2024-05-09
通讯作者: † E-mail: xiong@amss.ac.cn
基金资助:
* 国家自然科学基金项目 (12171462)和国家重点研发计划项目(2022YFF0609903)资助

Reconstruction density estimation based on sequential algorithms

HUANG Siyuan¹, XIE Tianfa¹, XIONG Shifeng^2,3,†

1 School of Mathematics, Statistics and Mechanics, Beijing University of Technology, Beijing 100124, China;
2 School of Mathematical Sciences, University of Chinese Academy and Science, Beijing 101408, China;
3 Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China

Received:2023-10-20 Revised:2024-04-03 Published:2024-05-09

摘要/Abstract

摘要： 本文对于重构方法给出的密度估计,提出了一种基于序贯思想来解决重构密度估计中节点选择问题的算法。由于密度估计可视为一个无监督学习问题,即没有响应变量y,因此针对回归的节点序贯选择方法在这里不适用。我们将节点视为一个参数,通过最小化损失函数选出下一个节点,利用贪心算法找出整个节点集。该算法操作简单,进一步提升了估计效果,且可以减小由于节点选择不同而对密度估计产生的影响。此外,本文还根据重构方法中参数的实际含义给出了先验,利用Metropolis算法得到后验分布的样本,通过样本分位数近似总体分位数构造了密度函数逐点的区间估计。最后,我们在几个数据集上验证了序贯重构密度估计及其区间估计的效果。

关键词: 重构方法, 密度估计, 序贯算法, 区间估计

Abstract: In this paper, for the density estimation given by the reconstruction approach, an algorithm based on the sequential idea is proposed to solve the node selection problem in the reconstructed density estimation. Since density estimation can be regarded as an unsupervised learning problem, i.e., there is no response variable y, the node sequential selection approach for regression is not applicable here. We regard the node as a parameter and select the next node by minimising the loss function, then find out the entire set of nodes using a greedy algorithm. This algorithm is simple to operate, further improves the estimation effect, and can reduce the impact on density estimation due to different node selection. In addition, in this paper, the prior is given according to the actual meanings of the parameters in the reconstruction approach, the samples of the posterior distribution are obtained using the Metropolis algorithm, so that the interval estimation of the density function point by point is constructed by approximating the overall quartile through the sample quartiles. Finally, we validate the sequential reconstruction density estimation and its interval estimation on several datasets.

Key words: Key Words Reconstruction approach, Density estimation, Sequential algorithm, Interval estimation

中图分类号:

O212.7

黄思源, 谢田法, 熊世峰. 基于序贯算法的重构密度估计^*[J]. 中国科学院大学学报, DOI: 10.7523/j.ucas.2024.018.

HUANG Siyuan, XIE Tianfa, XIONG Shifeng. Reconstruction density estimation based on sequential algorithms[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.018.

参考文献

[1] 周建, 徐海芹. 一种基于核密度估计的图像边缘检测方法[J]. 计算机科学, 2018, 45(S1): 239-241.
[2] Harvey A, Oryshchenko V.Kernel density estimation for time series data[J]. International Journal of Forecasting, 2012, 28(1): 3-14. DOI: 10.1016/j.ijforecast.2011.02.016.
[3] Tran T N, Wehrens R, Buydens L M C. KNN-kernel density-based clustering for high-dimensional multivariate data[J]. Computational Statistics & Data Analysis, 2006, 51(2): 513-525. DOI: 10.1016/j.csda.2005.10.001.
[4] 李存华, 孙志挥, 陈耿, 等. 核密度估计及其在聚类算法构造中的应用[J]. 计算机研究与发展, 2004, 41(10): 1712-1719.
[5] Parzen E.On estimation of a probability density function and mode[J]. The Annals of Mathematical Statistics, 1962, 33(3): 1065-1076. DOI: 10.1214/aoms/1177704472.
[6] Rosenblatt M.Remarks on some nonparametric estimates of a density function[J]. The Annals of Mathematical Statistics, 1956, 27(3): 832-837. DOI: 10.1214/aoms/1177728190.
[7] Silverman B W.Density estimation for statistics and data analysis[M]. London: Chapman and Hall, 1986.
[8] Härdle W, Werwatz A, Müller M, et al.Nonparametric and Semiparametric Models[M]. Berlin, Heidelberg: Springer Berlin Heidelberg, 2004. DOI: 10.1007/978-3-642-17146-8.
[9] Xiong S F.The reconstruction approach: from interpolation to regression[J]. Technometrics, 2021, 63(2): 225-235. DOI: 10.1080/00401706.2020.1764869.
[10] Santner T J, Williams B J, Notz W I.The Design and Analysis of Computer Experiments[M]. New York, NY: Springer New York, 2018. DOI: 10.1007/978-1-4939-8847-1.
[11] Mu W Y, Xiong S F.A class of space-filling designs and their projection properties[J]. Statistics & Probability Letters, 2018, 141: 129-134. DOI: 10.1016/j.spl.2018.06.002.
[12] Wu Y, Xiong S F.Kernel reconstruction learning[J]. Neurocomputing, 2023, 522: 1-10. DOI: 10.1016/j.neucom.2022.12.015.
[13] Dick J.On quasi-monte Carlo rules achieving higher order convergence[C]//L' Ecuyer P, Owen A. Monte Carlo and Quasi-Monte Carlo Methods 2008. Berlin, Heidelberg: Springer, 2009: 73-96. DOI:10.1007/978-3-642-04107-5_5
[14] Tseng P.Convergence of a block coordinate descent method for nondifferentiable minimization[J]. Journal of Optimization Theory and Applications, 2001, 109(3): 475-494. DOI: 10.1023/A: 1017501703105.
[15] Kumar A, Vembu S, Menon A K, et al.Beam search algorithms for multilabel learning[J]. Machine Learning, 2013, 92(1): 65-89. DOI: 10.1007/s10994-013-5371-6.
[16] Neal R M.MCMC using Hamiltonian dynamics[M]//Handbook of Markov Chain Monte Carlo. New York: Chapman and Hall/CRC, 2011: 113-162. DOI: 10.1201/b10905-6.
[17] Sherlock C, Roberts G.Optimal scaling of the random walk Metropolis on elliptically symmetric unimodal targets[J]. Bernoulli, 2009, 15(3): 774-798. DOI: 10.3150/08-BEJ176.
[18] Gelman A, Gilks W R, Roberts G O.Weak convergence and optimal scaling of random walk Metropolis algorithms[J]. The Annals of Applied Probability, 1997, 7(1): 110-120. DOI: 10.1214/aoap/1034625254.
[19] Ram P, Gray A G.Density estimation trees[C]//Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. August 21 - 24, 2011, San Diego, California, USA. ACM, 2011: 627-635. DOI: 10.1145/2020408.2020507.
[20] Kullback S, Leibler R A.On information and sufficiency[J]. The Annals of Mathematical Statistics, 1951, 22(1): 79-86. DOI: 10.1214/aoms/1177729694.
[21] Hall P.Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density[J]. The Annals of Statistics, 1992, 20(2): 675-694. DOI: 10.1214/aos/1176348651.
[22] Phillips S J, Anderson R P, Schapire R E.Maximum entropy modeling of species geographic distributions[J]. Ecological Modelling, 2006, 190(3-4): 231-259. DOI: 10.1016/j.ecolmodel.2005.03.026.

基于序贯算法的重构密度估计^*

Reconstruction density estimation based on sequential algorithms

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 4

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	李娅, 刘亚岚, 任玉环, 王智灏, 曲畅. 城市功能区语义信息挖掘与遥感分类[J]. 中国科学院大学学报, 2019, 36(1): 56-63.
[2]	蒋自然, 王万荣, 朱华友, 吴威. 中国商品交易市场的综合实力及其时空分异——基于省域单元的实证分析[J]. 中国科学院大学学报, 2016, 33(3): 380-386.
[3]	常楠, 张三国. 基于核方法和高斯混合模型的手机上网时长统计分析及应用[J]. 中国科学院大学学报, 2015, 32(1): 136-139.
[4]	卢晓鹏殷学民邹谋炎. 基于核密度相关度量的视频目标跟踪[J]. 中国科学院大学学报, 2007, 24(4): 501-505.