针对重构方法给出的密度估计，提出一种基于序贯思想来解决重构密度估计中节点选择问题的算法。将节点视为一个参数，通过最小化损失函数选出下一个节点，利用贪心算法找出整个节点集。该算法操作简单，不仅可以提升估计效果，而且可以减小由于节点选择不同而对密度估计产生的影响。此外，还根据重构方法中参数的实际含义给出了先验分布，利用Metropolis算法得到后验分布的样本，通过样本分位数近似总体分位数构造了密度函数逐点的区间估计。最后，在几个数据集上验证了序贯重构密度估计及其区间估计的效果。
In this paper, for the density estimation given by the reconstruction approach, an algorithm based on the sequential idea is proposed to solve the node selection problem in the reconstructed density estimation. Since density estimation can be regarded as an unsupervised learning problem, i.e., there is no response variable y, the node sequential selection approach for regression is not applicable here. We regard the node as a parameter and select the next node by minimising the loss function, then determine the entire set of nodes using a greedy algorithm. This algorithm is simple to operate, further improves the estimation effect, and can reduce the impact on density estimation due to different node selection. In addition, in this paper, the prior is given according to the actual meanings of the parameters in the reconstruction approach, the samples of the posterior distribution are obtained using the Metropolis algorithm, so that the interval estimation of the density function point by point is constructed by approximating the overall quartile through the sample quartiles. Finally, we validate the sequential reconstruction density estimation and its interval estimation on several datasets.
[1] 周建, 徐海芹. 一种基于核密度估计的图像边缘检测方法[J]. 计算机科学, 2018, 45(S1): 239-241.
[2] Harvey A, Oryshchenko V. Kernel density estimation for time series data[J]. International Journal of Forecasting, 2012, 28(1): 3-14. DOI: 10.1016/j.ijforecast.2011.02.016.
[3] Tran T N, Wehrens R, Buydens L M C. KNN-kernel density-based clustering for high-dimensional multivariate data[J]. Computational Statistics & Data Analysis, 2006, 51(2): 513-525. DOI: 10.1016/j.csda.2005.10.001.
[4] 李存华, 孙志挥, 陈耿, 等. 核密度估计及其在聚类算法构造中的应用[J]. 计算机研究与发展, 2004, 41(10): 1712-1719.
[5] Parzen E. On estimation of a probability density function and mode[J]. The Annals of Mathematical Statistics, 1962, 33(3): 1065-1076. DOI: 10.1214/aoms/1177704472.
[6] Rosenblatt M. Remarks on some nonparametric estimates of a density function[J]. The Annals of Mathematical Statistics, 1956, 27(3): 832-837. DOI: 10.1214/aoms/1177728190.
[7] Silverman B W. Density estimation for statistics and data analysis[M]. London: Chapman and Hall, 1986.
[8] Härdle W, Werwatz A, Müller M, et al. Nonparametric and Semiparametric Models[M]. Berlin, Heidelberg: Springer Berlin Heidelberg, 2004. DOI: 10.1007/978-3-642-17146-8.
[9] Xiong S F. The reconstruction approach: from interpolation to regression[J]. Technometrics, 2021, 63(2): 225-235. DOI: 10.1080/00401706.2020.1764869.
[10] Santner T J, Williams B J, Notz W I. The design and analysis of computer experiments[M]. New York, NY: Springer New York, 2018. DOI: 10.1007/978-1-4939-8847-1.
[11] Mu W Y, Xiong S F. A class of space-filling designs and their projection properties[J]. Statistics & Probability Letters, 2018, 141: 129-134. DOI: 10.1016/j.spl.2018.06.002.
[12] Wu Y, Xiong S F. Kernel reconstruction learning[J]. Neurocomputing, 2023, 522: 1-10. DOI: 10.1016/j.neucom.2022.12.015.
[13] Dick J. On quasi-monte Carlo rules achieving higher order convergence[C]//L'Ecuyer P, Owen A. Monte Carlo and Quasi-Monte Carlo Methods 2008. Berlin, Heidelberg: Springer, 2009: 73-96. DOI:10.1007/978-3-642-04107-5_5.
[14] Tseng P. Convergence of a block coordinate descent method for nondifferentiable minimization[J]. Journal of Optimization Theory and Applications, 2001, 109(3): 475-494. DOI: 10.1023/A: 1017501703105.
[15] Kumar A, Vembu S, Menon A K, et al. Beam search algorithms for multilabel learning[J]. Machine Learning, 2013, 92(1): 65-89. DOI: 10.1007/s10994-013-5371-6.
[16] Neal R M. MCMC using Hamiltonian dynamics[M]//Handbook of Markov Chain Monte Carlo. New York: Chapman and Hall/CRC, 2011: 113-162. DOI: 10.1201/b10905-6.
[17] Sherlock C, Roberts G. Optimal scaling of the random walk Metropolis on elliptically symmetric unimodal targets[J]. Bernoulli, 2009, 15(3): 774-798. DOI: 10.3150/08-BEJ176.
[18] Gelman A, Gilks W R, Roberts G O. Weak convergence and optimal scaling of random walk Metropolis algorithms[J]. The Annals of Applied Probability, 1997, 7(1): 110-120. DOI: 10.1214/aoap/1034625254.
[19] Ram P, Gray A G. Density estimation trees[C]//Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. August 21-24, 2011, San Diego, California, USA. ACM, 2011: 627-635. DOI: 10.1145/2020408.2020507.
[20] Kullback S, Leibler R A. On information and sufficiency[J]. The Annals of Mathematical Statistics, 1951, 22(1): 79-86. DOI: 10.1214/aoms/1177729694.
[21] Hall P. Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density[J]. The Annals of Statistics, 1992, 20(2): 675-694. DOI: 10.1214/aos/1176348651.
[22] Phillips S J, Anderson R P, Schapire R E. Maximum entropy modeling of species geographic distributions[J]. Ecological Modelling, 2006, 190(3/4): 231-259. DOI: 10.1016/j.ecolmodel.2005.03.026.
