Reconstruction density estimation based on sequential algorithms

doi:10.7523/j.ucas.2024.018

Abstract

Abstract: In this paper, for the density estimation given by the reconstruction approach, an algorithm based on the sequential idea is proposed to solve the node selection problem in the reconstructed density estimation. Since density estimation can be regarded as an unsupervised learning problem, i.e., there is no response variable y, the node sequential selection approach for regression is not applicable here. We regard the node as a parameter and select the next node by minimising the loss function, then find out the entire set of nodes using a greedy algorithm. This algorithm is simple to operate, further improves the estimation effect, and can reduce the impact on density estimation due to different node selection. In addition, in this paper, the prior is given according to the actual meanings of the parameters in the reconstruction approach, the samples of the posterior distribution are obtained using the Metropolis algorithm, so that the interval estimation of the density function point by point is constructed by approximating the overall quartile through the sample quartiles. Finally, we validate the sequential reconstruction density estimation and its interval estimation on several datasets.

Key words: Key Words Reconstruction approach, Density estimation, Sequential algorithm, Interval estimation

CLC Number:

O212.7

HUANG Siyuan, XIE Tianfa, XIONG Shifeng. Reconstruction density estimation based on sequential algorithms[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.018.

References

[1] 周建, 徐海芹. 一种基于核密度估计的图像边缘检测方法[J]. 计算机科学, 2018, 45(S1): 239-241.
[2] Harvey A, Oryshchenko V.Kernel density estimation for time series data[J]. International Journal of Forecasting, 2012, 28(1): 3-14. DOI: 10.1016/j.ijforecast.2011.02.016.
[3] Tran T N, Wehrens R, Buydens L M C. KNN-kernel density-based clustering for high-dimensional multivariate data[J]. Computational Statistics & Data Analysis, 2006, 51(2): 513-525. DOI: 10.1016/j.csda.2005.10.001.
[4] 李存华, 孙志挥, 陈耿, 等. 核密度估计及其在聚类算法构造中的应用[J]. 计算机研究与发展, 2004, 41(10): 1712-1719.
[5] Parzen E.On estimation of a probability density function and mode[J]. The Annals of Mathematical Statistics, 1962, 33(3): 1065-1076. DOI: 10.1214/aoms/1177704472.
[6] Rosenblatt M.Remarks on some nonparametric estimates of a density function[J]. The Annals of Mathematical Statistics, 1956, 27(3): 832-837. DOI: 10.1214/aoms/1177728190.
[7] Silverman B W.Density estimation for statistics and data analysis[M]. London: Chapman and Hall, 1986.
[8] Härdle W, Werwatz A, Müller M, et al.Nonparametric and Semiparametric Models[M]. Berlin, Heidelberg: Springer Berlin Heidelberg, 2004. DOI: 10.1007/978-3-642-17146-8.
[9] Xiong S F.The reconstruction approach: from interpolation to regression[J]. Technometrics, 2021, 63(2): 225-235. DOI: 10.1080/00401706.2020.1764869.
[10] Santner T J, Williams B J, Notz W I.The Design and Analysis of Computer Experiments[M]. New York, NY: Springer New York, 2018. DOI: 10.1007/978-1-4939-8847-1.
[11] Mu W Y, Xiong S F.A class of space-filling designs and their projection properties[J]. Statistics & Probability Letters, 2018, 141: 129-134. DOI: 10.1016/j.spl.2018.06.002.
[12] Wu Y, Xiong S F.Kernel reconstruction learning[J]. Neurocomputing, 2023, 522: 1-10. DOI: 10.1016/j.neucom.2022.12.015.
[13] Dick J.On quasi-monte Carlo rules achieving higher order convergence[C]//L' Ecuyer P, Owen A. Monte Carlo and Quasi-Monte Carlo Methods 2008. Berlin, Heidelberg: Springer, 2009: 73-96. DOI:10.1007/978-3-642-04107-5_5
[14] Tseng P.Convergence of a block coordinate descent method for nondifferentiable minimization[J]. Journal of Optimization Theory and Applications, 2001, 109(3): 475-494. DOI: 10.1023/A: 1017501703105.
[15] Kumar A, Vembu S, Menon A K, et al.Beam search algorithms for multilabel learning[J]. Machine Learning, 2013, 92(1): 65-89. DOI: 10.1007/s10994-013-5371-6.
[16] Neal R M.MCMC using Hamiltonian dynamics[M]//Handbook of Markov Chain Monte Carlo. New York: Chapman and Hall/CRC, 2011: 113-162. DOI: 10.1201/b10905-6.
[17] Sherlock C, Roberts G.Optimal scaling of the random walk Metropolis on elliptically symmetric unimodal targets[J]. Bernoulli, 2009, 15(3): 774-798. DOI: 10.3150/08-BEJ176.
[18] Gelman A, Gilks W R, Roberts G O.Weak convergence and optimal scaling of random walk Metropolis algorithms[J]. The Annals of Applied Probability, 1997, 7(1): 110-120. DOI: 10.1214/aoap/1034625254.
[19] Ram P, Gray A G.Density estimation trees[C]//Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. August 21 - 24, 2011, San Diego, California, USA. ACM, 2011: 627-635. DOI: 10.1145/2020408.2020507.
[20] Kullback S, Leibler R A.On information and sufficiency[J]. The Annals of Mathematical Statistics, 1951, 22(1): 79-86. DOI: 10.1214/aoms/1177729694.
[21] Hall P.Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density[J]. The Annals of Statistics, 1992, 20(2): 675-694. DOI: 10.1214/aos/1176348651.
[22] Phillips S J, Anderson R P, Schapire R E.Maximum entropy modeling of species geographic distributions[J]. Ecological Modelling, 2006, 190(3-4): 231-259. DOI: 10.1016/j.ecolmodel.2005.03.026.