Learning stochastic Hamiltonian systems via neural network and numerical quadrature formulae*

doi:10.7523/j.ucas.2024.074

Abstract

Abstract: Detecting and predicting the behavior of Hamiltonian systems via machine learning has been drawing increasing attentions in recent years. In this paper, we propose a data-driven neural network learning approach for stochastic Hamiltonian systems based on using numerical quadrature in the moments of solutions to build up the network loss functions. Good long-term predictions are then achieved utilizing symplectic integrators. Numerical experiments on two models show effectiveness of the proposed method.

Key words: stochastic Hamiltonian systems, neural networks, numerical quadrature, Simpson’s formula, symplectic integrators

CLC Number:

O241.8

CHENG Xupeng, WANG Lijin. Learning stochastic Hamiltonian systems via neural network and numerical quadrature formulae^*[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.074.

References

[1] E Weinan.A proposal on machine learning via dynamical systems[J]. Communications in Mathematics and Statistics, 2017, 5(1): 1-11. DOI:10.1007/s40304-017-0103-z.
[2] Lu Y P, Zhong A X, Li Q Z, et al. Beyond finite layer neural networks: bridging deep architectures and numerical differential equations[EB/OL]. arXiv: 1710.01021v3. (2017-10-27)[2024-04-10]. http://arxiv.org/abs/1710.01021v3.
[3] Chen T Q, Rubanova Y, Bettencourt J, et al. Neural ordinary differential equations[EB/OL]. 2018: 1806.07366. (2018-06-19)[2024-04-10]. http://arxiv.org/abs/1806.07366v5.
[4] Greydanus S, Dzamba M, Yosinski J. “Hamiltonian Neural Networks[EB/OL]. 2019: 1906.01563. (2019-06-04)[2024-04-10]. http://arxiv.org/abs/1906.01563v3.
[5] Chen Z D, Zhang J Y, Arjovsky M, et al. Symplectic Recurrent Neural Networks[EB/OL]. 2019: 1909.13334. (2019-09-29)[2024-04-10]. http://arxiv.org/abs/1909.13334v2.
[6] Zhu A Q, Jin P Z, Tang Y F. Deep Hamiltonian networks based on symplectic integrators[EB/OL]. arXiv:2004.13830. (2020-04-23)[2024-04-10]. https://arxiv.org/abs/2004.13830
[7] Jin P Z, Zhang Z, Zhu A Q, et al.SympNets: Intrinsic structure preserving symplectic networks for identifying Hamiltonian systems[J]. Neural Networks, 2020, 132: 166-179. DOI:10.1016/j.neunet.2020.08.017.
[8] Tong Y J, Xiong S Y, He X Z, et al.Symplectic neural networks in Taylor series form for Hamiltonian systems[J]. Journal of Computational Physics, 2021, 437: 110325. DOI:10.1016/j.jcp.2021.110325.
[9] Xiong S Y, Tong Y J, He X Z, et al. Nonseparable Symplectic Neural Net- works[EB/OL]. arXiv: 2010.12636. (2020-10-23)[2024-04-10]. http://arxiv.org/abs/2010.12636v3.
[10] David M, Méhats F.Symplectic learning for Hamiltonian neural networks[J]. Journal of Computational Physics, 2023, 494: 112495. DOI:10.1016/j.jcp.2023.112495.
[11] Liu X Q, Xiao T S, Si S, et al. Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise[EB/OL]. arXiv:1906.02355. (2019-06-05)[2024-04-10]. https://arxiv.org/abs/1906.02355v1.
[12] Kong L K, Sun J M, Zhang C. SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates[EB/OL]. arXiv: 2008.10546. (2020-08-24)[2024-04-10]. http://arxiv.org/abs/2008.10546v1.
[13] Jia J T, Benson Austin R. Neural Jump Stochastic Differential Equations[EB/OL]. arXiv: 1905.10403. (2019-05-24)[2024-04-10]. http://arxiv.org/abs/1905.10403v3.
[14] Dietrich F, Makeev A, Kevrekidis G, et al.Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning[J]. Chaos: an Interdisciplinary Journal of Nonlinear Science, 2023, 33(2): 1-19. DOI:10.1063/5.0113632.
[15] Tzen B, Raginsky M. Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit[EB/OL]. arXiv: 1905.09883. (2019-05-23)[2024-04-10]. http://arxiv.org/abs/1905.09883v2.
[16] Ryder T, Golightly A, McGough A S, et al. Black-box variational inference for stochastic differential equations[EB/OL]. arXiv: 1802.03335. (2018-02-09)[2024-04-10]. http://arxiv.org/abs/1802.03335v3
[17] Opper M.Variational inference for stochastic differential equations[J]. Annalen der Physik. 2019, 531(3): 1-9. DOI:10.1002/andp.201800233.
[18] Dai M, Duan J Q, Hu J Y, et al.Variational inference of the drift function for stochastic differential equations driven by Lévy processes[J]. Chaos: an Interdisciplinary Journal of Nonlinear Science, 2022, 32(6): 1-10. DOI:10.1063/5.0095605.
[19] Wang Z P, Wang L J, Learning parameters of a class of stochastic Lotka-Volterra systems with neural networks[EB/OL]. 2023: (2023-03-21)[2024-04-10]. http://journal.ucas.ac.cn/CN/10.7523/j.ucas.2023.012.
[20] Fang C, Lu Y B, Gao T, et al.An end-to-end deep learning approach for extracting stochastic dynamical systems with α -stable Lévy noise[J]. Chaos: an Interdisciplinary Journal of Nonlinear Science, 2022, 32(6): 1-26. DOI:10.1063/5.0089832.
[21] Yang L X, Gao T, Lu Y B, et al.Neural network stochastic differential equation models with applications to financial data forecasting[J]. Applied Mathematical Modelling, 2023, 115: 279-299. DOI:10.1016/j.apm.2022.11.001.
[22] Deng R Z, Chang B, Brubaker M A, et al. Modeling continuous stochastic processes with dynamic normalizing flows[EB/OL]. arXiv: 2002.10516. (2020-02-24)[2024-04-10]. http://arxiv.org/abs/2002.105161v4.
[23] Urain J, Ginesi M, Tateo D, et al.Imitationflow: Learning deep stable stochastic dynamic systems by normalizing flows[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas, NV, USA. IEEE, 2020: 5231-5237. DOI:10.1109/IROS45743.2020.9341035.
[24] Papamakarios G, Nalisnick E, Rezende D J, et al. Normalizing flows for probabilistic modeling and inference[EB/OL]. arXiv: 1912.02762. (2019-12-05)[2024-04-10]. http://arxiv.org/abs/1912.02762v2
[25] Chen X L, Yang L, Duan J Q, et al.Solving inverse stochastic problems from discrete particle observations using the Fokker-Planck equation and physics-informed neural networks[J]. SIAM Journal on Scientific Computing, 2021, 43(3): B811-B830. DOI:10.1137/20m1360153.
[26] Li Y, Duan J Q.A data-driven approach for discovering stochastic dynamical systems with non-Gaussian Lévy noise[J]. Physica D: Nonlinear Phenomena, 2021, 417: 132830. DOI:10.1016/j.physd.2020.132830.
[27] Solin A, Tamir E, Verma P. Scalable inference in SDEs by direct matching of the Fokker-Planck-Kolmogorov equation[EB/OL]. arXiv: 2110.15739. (2021-10-29)[2024-04-10]. http://arxiv.org/abs/2110.15739v1.
[28] Lu Y B, Li Y, Duan J Q.Extracting stochastic governing laws by non-local Kramers-Moyal formulae[J]. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 2022, 380(2229): 20210195. DOI:10.1098/rsta.2021.0195.
[29] Chen X L, Wang H, Duan J Q.Detecting stochastic governing laws with observation on stationary distributions[J]. Physica D: Nonlinear Phenomena, 2023, 448: 133691. DOI:10.1016/j.physd.2023.133691.
[30] Chen Y, Xiu D B.Learning stochastic dynamical system via flow map operator[J]. Journal of Computational Physics, 2024, 508: 112984. DOI:10.1016/j.jcp.2024.112984.
[31] Milstein G N, Repin Y M, Tretyakov M V.Numerical methods for stochastic systems preserving symplectic structure[J]. SIAM Journal on Numerical Analysis. 2002, 40(4): 1583-1604. DOI:10.1137/s0036142901395588.
[32] Higham D.J. An algorithmic introduction to numerical simulation of stochastic differential equations[J]. SIAM Review, 2001, 43(3): 525-546. DOI:10.1137/S0036144500378302.

Learning stochastic Hamiltonian systems via neural network and numerical quadrature formulae^*

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 9

Recommended Articles

Metrics

Comments

[1]	YAO Mufeng, ZAN Luyang, LI Baipeng, LI Qingting, CHEN Zhengchao. Building change detection from remote sensing images using CAR-Siamese net [J]. Journal of University of Chinese Academy of Sciences, 2023, 40(3): 380-387.
[2]	HU Lina, GAO Shenghua. Hierarchical label-guided human parsing [J]. , 2020, 37(6): 820-827.
[3]	ZHU Sijie, LEI Bin, WU Yirong. Automatic color correction for remote sensing optical image based on dense convolutional networks [J]. , 2019, 36(1): 93-100.
[4]	RUAN Jialin, WANG Lijin. A Lie algebraic approach for a class of highly oscillatory stochastic Hamiltonian systems [J]. , 2019, 36(1): 5-10.
[5]	LI Song, WEI Zhonghao, ZHANG Bingchen, HONG Wen. Target recognition using the transfer learning-based deep convolutional neural networks for SAR images [J]. , 2018, 35(1): 75-83.
[6]	JIANG Lu, ZHAO Tong, WU Min. Fingerprint pattern classification algorithm based on deep convolutional neural networks [J]. , 2016, 33(6): 808-814.
[7]	LEI Yu, CAI Hongbing, ZHAO Danning. Prediction of length-of-day variation using grey relational analysis and extreme learning machine [J]. Journal of University of Chinese Academy of Sciences, 2015, 32(5): 588-593.
[8]	LAN Run-Rong, CHENG Xi-Jun. A new RBF neural network and its application on individual credit rating in banks [J]. , 2013, 30(3): 298-303.
[9]	Jin Hui-bin, Zhang Can, Qin Lei. Prediction method of the ionospheric F2 layer critical frequency based on neural networks [J]. Journal of University of Chinese Academy of Sciences, 2008, 25(3): 403-407.