Application of neural network modeling methods in solving initial boundary value problems for partial differential equations

About the authors:

Vladimir E. Vershinin, Chief Specialist, Tyumen Petroleum Research Center, Tyumen, Russia; Associate Professor, Department of Physical Processes and Systems Modeling, School of Natural Sciences, University of Tyumen, Tyumen, Russia; ve_vershinin2@tnnc.rosneft.ru

Roman Yu. Ponomarev, Manager, Tyumen Petroleum Research Center, Tyumen, Russia; ryponomarev@tnnc.rosneft.ru

Abstract:

Machine learning allows you to solve a variety of data analysis problems, but its use for solving differential equations has appeared relatively recently. The approximation of the solution of the boundary value problem for differential equations (ordinary and partial derivatives) is constructed using neural network functions. The selection of weighting coefficients is carried out during the training of the neural network. The criteria for the quality of training in this case are inconsistencies in the equation and boundary-initial conditions. This approach makes it possible, instead of grid solutions, to find solutions defined on the entire feasible region of the boundary value problem. Specific examples show the features of the application of physics-informed neural networks to the solution of boundary value problems for differential equations of various types. Physics-informed neural networks training methods can be used in the tasks of retraining intelligent control systems on incomplete sets of input data.

References:

Vasiliev, A. N., Tarhov, D. A., & Shemyakina, T. A. (2015). Neural network approach to problems of mathematical physics. Nestor-History. [In Russian]

Zrelova, D. P., & Ulyanov, S. V. (2022). Physics-informed classical Lagrange / Hamilton neural networks in deep learning. Modern Information Technologies and IT-Education, 18(2), 310–325. https://doi.org/10.25559/SITITO.18.202202.310-325 [In Russian]

Kovalenko, A. N., Chernomorets, A. A., & Petina, M. A. (2017). On the neural networks application for solving of partial differential equations. Scientific Bulletin of Belgorod State University. Series: Economics. Informatics, (9), 103–110. [In Russian]

Kolmogorov, A. N. (1956). On the representation of continuous functions of several variables by superpositions of continuous functions of a smaller number of variables. Proceedings of the USSR Academy of Sciences, 108(2), 179–182. [In Russian]

Tarhov, D. A. (2014). Neural network models and algorithms. Radiotekhnika. [In Russian]

Haykin, S. (2019). Neural networks. Dialektika. [In Russian]

Cai, S., Wang, Z., Wang, S., Perdikaris, P., & Karniadakis, G. E. (2021). Physics-informed neural networks for heat transfer problems. Journal of Heat Transfer, 143(6), Article 060801. https://doi.org/10.1115/1.4050542

Carleo, G., Cirac, I., Cranmer, K., Daudet, L., Schuld, M., Tishby, N., Vogt-Maranto, L., & Zdeborová, L. (2019). Machine learning and the physical sciences. Reviews of Modern Physics, 91(4), Article 045002. https://doi.org/10.1103/RevModPhys.91.045002

Galperin, E. A., Pan, Z., & Zheng, Q. (1993). Application of global optimization to implicit solution of partial differential equations. Computers & Mathematics with Applications, 25(10–11), 119–124. https://doi.org/10.1016/0898-1221(93)90287-6

Galperin, E. A., & Zheng, Q. (1993). Solution and control of PDE via global optimization methods. Computers & Mathematics with Applications, 25(10–11), 103–118. https://doi.org/10.1016/0898-1221(93)90286-5

Kansa, E. J. (1990a). Multiquadrics — A scattered data approximation scheme with applications to computational fluid-dynamics — I surface approximations and partial derivative estimates. Computers & Mathematics with Applications, 19(8), 127–145. https://doi.org/10.1016/0898-1221(90)90270-T

Kansa, E. J. (1990b). Multiquadrics — A scattered data approximation scheme with applications to computational fluid-dynamics — II solutions to parabolic, hyperbolic and elliptic partial differential equations. Computers & Mathematics with Applications, 19(8), 147–161. https://doi.org/10.1016/0898-1221(90)90271-K

Kansa, E. J. (1999). Motivation for using radial basis functions to solve PDEs. Lawrence Livermore National Laboratory; Embry-Riddle Aeronatical University.

Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 422–440. https://doi.org/10.1038/s42254-021-00314-5

Kingma, D. P., & Ba, J. (2015, May 7–9). Adam: A method for stochastic optimization [Confe­rence paper]. The 3rd International Conference for Learning Representations, San Diego, CA, USA. https://doi.org/10.48550/arXiv.1412.6980

Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998). Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5), 987–1000. https://doi.org/10.1109/72.712178

Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2017). Machine learning of linear differential equations using Gaussian processes. Journal of Computational Physics, 348, 683–693. https://doi.org/10.1016/j.jcp.2017.07.050

Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686–707. https://doi.org/10.1016/j.jcp.2018.10.045

Rasmussen, C. E., & Williams, C. K. I. (2005). Gaussian processes for machine learning. The MIT Press. https://doi.org/10.7551/mitpress/3206.001.0001

Sharan, M., Kansa, E. J., & Gupta, S. (1997). Application of the multiquadric method for nume­rical solution of elliptic partial differential equations. Applied Mathematics and Computation, 84(2), 275–302. https://doi.org/10.1016/S0096-3003(96)00109-9

Thuerey, N., Holl, Ph., Mueller, M., Schnell, P., Trost, F., & Um, K. (2022). Physics-based deep learning. https://doi.org/10.48550/arXiv.2109.05237