Calibration kernel-based ratio estimation for likelihood-free posterior inference

doi:10.7523/j.ucas.2024.023

Abstract

Abstract:

Bayesian inference often faces challenges where the likelihood function is difficult to evaluate or lacks explicit expression， known as likelihood-free Bayesian problems， where posterior distributions can only be inferred indirectly through samples generated under specific parameters. Existing methods such as approximate Bayesian computation， synthetic likelihood， and Bayesian optimization focus on addressing these issues. This paper extends Miller et al.’s （2022） sequential neural ratio estimation method for likelihood-free Bayesian problems by transforming likelihood-to-evidence ratio estimation into a multi-class problem for efficient posterior estimation. We introduce a new calibration kernel-based ratio estimation method （CKRE）， to enhance the original method’s training efficiency and performance. The convergence of our proposed method is proven， and numerical experiments demonstrate its significant improvement in accurately estimating posterior distributions under limited sample generation conditions.

Key words: likelihood-free Bayesian inference, sequential neural ratio estimation, calibration kernel-based ratio estimation, contrastive learning

CLC Number:

O212.1

Yifei XIONG, Sanguo ZHANG. Calibration kernel-based ratio estimation for likelihood-free posterior inference[J]. Journal of University of Chinese Academy of Sciences, 2026, 43(4): 444-452.

Figures/Tables 2

References 29

[1]	Paninski L， Cunningham J P. Neural data science： accelerating the experiment-analysis-theory cycle in large-scale neuroscience［J］. Current Opinion in Neurobiology， 2018， 50： 232-241. DOI： 10.1016/j.conb.2018.04.007 .
[2]	Brehmer J， Louppe G， Pavez J， et al. Mining gold from implicit models to improve likelihood-free inference［J］. PNAS， 2020， 117（10）： 5242-5249. DOI： 10.1073/pnas.1915980117 .
[3]	Gonçalves P J， Lueckmann J M， Deistler M， et al. Training deep neural density estimators to identify mechanistic models of neural dynamics［J］. eLife， 2020， 9： e56261. DOI： 10.7554/eLife.56261 .
[4]	Hashemi M， Vattikonda A N， Jha J， et al. Amortized Bayesian inference on generative dynamical network models of epilepsy using deep neural density estimators［J］. Neural Networks， 2023， 163： 178-194. DOI： 10.1016/j.neunet.2023.03.040 .
[5]	Clarke R， Ressom H W， Wang A T， et al. The properties of high-dimensional data spaces： implications for exploring gene and protein expression data［J］. Nature Reviews. Cancer， 2008， 8（1）： 37-49. DOI： 10.1038/nrc2294 .
[6]	Romaszko L， Williams C K I， Moreno P， et al. Vision-as-inverse-graphics： obtaining a rich 3D explanation of a scene from a single image［C］//2017 IEEE International Conference on Computer Vision Workshops （ICCVW）. Venice， Italy. IEEE， 2017： 940-948. DOI： 10.1109/ICCVW.2017.115 .
[7]	Beaumont M A， Zhang W Y， Balding D J. Approximate Bayesian computation in population genetics［J］. Genetics， 2002， 162（4）： 2025-2035. DOI： 10.1093/genetics/162.4.2025 .
[8]	Marin J M， Pudlo P， Robert C P， et al. Approximate Bayesian computational methods［J］. Statistics and Computing， 2012， 22（6）： 1167-1180. DOI： 10.1007/s11222-011-9288-2 .
[9]	Wood S N. Statistical inference for noisy nonlinear ecological dynamic systems［J］. Nature， 2010， 466（7310）： 1102-1104. DOI： 10.1038/nature09319 .
[10]	Price L F， Drovandi C C， Lee A， et al. Bayesian synthetic likelihood［J］. Journal of Computational and Graphical Statistics， 2018， 27（1）： 1-11. DOI： 10.1080/10618600.2017.1302882 .
[11]	Gutmann M U， Corander J. Bayesian optimization for likelihood-free inference of simulator-based statistical models［J］. Journal of Machine Learning Research， 2016， 17（1）： 4256-4302.DOI：10.48550/arXiv.1501.03291 .
[12]	Andrieu C， Roberts G O. The pseudo-marginal approach for efficient Monte Carlo computations［J］. The Annals of Statistics， 2009， 37（2）： 697-725. DOI： 10.1214/ 07-aos574 .
[13]	Andrieu C， Doucet A， Holenstein R. Particle Markov chain Monte Carlo methods［J］. Journal of the Royal Statistical Society Series B： Statistical Methodology， 2010， 72（3）： 269-342. DOI： 10.1111/j.1467-9868.2009.00736.x .
[14]	Barthelmé S， Chopin N. Expectation propagation for likelihood-free inference［J］. Journal of the American Statistical Association， 2014， 109（505）： 315-333. DOI： 10.1080/01621459.2013.864178 .
[15]	Kobyzev I， Prince S J D， Brubaker M A. Normalizing flows： an introduction and review of current methods［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（11）： 3964-3979. DOI： 10.1109/TPAMI.2020.2992934 .
[16]	Papamakarios G， Sterratt D C， Murray I. Sequential neural likelihood： fast likelihood-free inference with autoregressive flows［EB/OL］. arXiv 2018： 1805.07226. （2019-01-21）［2024-04-02］. .
[17]	Papamakarios G， Murray I. Fast ε-free inference of simulation models with Bayesian conditional density estimation［C/OL］//Proceedings of the 30th International Conference on Neural Information Processing Systems. December 5 - 10， 2016， Barcelona， Spain. ACM， 2016： 1036-1044. （2016-12-10）［2024-04-02］. .
[18]	Lueckmann J M， Goncalves P J， Bassetto G， et al. Flexible statistical inference for mechanistic models of neural dynamics［C/OL］//Proceedings of the 31st International Conference on Neural Information Processing Systems. December 4-9， 2017， Long Beach， California， USA. ACM， 2017： 1289-1299.（2017-12-10）［2024-04-02］..
[19]	Greenberg D， Nonnenmacher M， Macke J. Automatic posterior transformation for likelihood-free inference［C/OL］// International Conference on Machine Learning. PMLR 2019： 2404-2414. （2019-11-03）［2024-04-02］. .
[20]	Hermans J， Begy V， Louppe G. Likelihood-free MCMC with amortized approximate ratio estimators［C/OL］//Proceedings of the 37th International Conference on Machine Learning. ACM， 2020： 4239-4248. （2020-12-27）［2024-04-02］. .
[21]	Durkan C， Murray I， Papamakarios G. On contrastive learning for likelihood-free inference［C/OL］//Proceedings of the 37th International Conference on Machine Learning. ACM， 2020： 2771-2781. （2020-12-27）［2024-04-02］. .
[22]	Miller B K， Weniger C， Forre P. Contrastive neural ratio estimation［C/OL］. Advances in Neural Information Processing Systems，2022，35：3262-3278.（2022-09-21）［2024-04-02］. .
[23]	Delaunoy A， Hermans J， Rozet F， et al. Towards reliable simulation-based inference with balanced neural ratio estimation［C/OL］//Advances in Neural Information Processing Systems， 2022， 35： 20025-20037. （2022-09-21）［2024-04-02］. .
[24]	Cranmer K， Brehmer J， Louppe G. The frontier of simulation-based inference［J］. Proceedings of the National Academy of Sciences of the United States of America， 2020， 117（48）： 30055-30062. DOI： 10.1073/pnas.1912789117 .
[25]	Lueckmann J M， Boelts J， Greenberg D S， et al. Benchmarking simulation-based inference［EB/OL］. arXiv 2021： 2101.04653. （2021-01-31）［2024-04-02］. .
[26]	Blum M G B， François O. Non-linear regression models for approximate Bayesian computation［J］. Statistics and Computing， 2010， 20（1）： 63-73. DOI： 10.1007/s11222-009-9116-0 .
[27]	Gutmann M， Hyvarinen A. Noise-contrastive estimation of unnormalized statistical models， with applications to natural image statistics［J］. Journal of Machine Learning Research， 2012， 13： 307-361. （2012-11-09）［2024-04-02］. .
[28]	He Z J， Xu Z H， Wang X Q. Unbiased MLMC-based variational Bayes for likelihood-free inference［J］. SIAM Journal on Scientific Computing， 2022， 44（4）： A1884-A1910. DOI： 10.1137/21M1449051 .
[29]	Gretton A， Borgwardt K M， Rasch M J， et al. A kernel two-sample test［J］. Journal of Machine Learning Research， 2012， 13： 723-773.