Derivation of parametric activation functions of a neural network for effective approximation of solutions to boundary value problems of differential equations of parabolic type

About the author:

Roman Yu. Ponomarev, Manager, Tyumen Petroleum Research Center, Tyumen, Russia; Postgraduate Student, Department of Modeling of Physical Processes and Systems, School of Natural Sciences, University of Tyumen, Tyumen, Russia; ryponomarev@tnnc.rosneft.ru

Abstract:

Recently, the use of neural networks for approximating solutions to boundary value problems of differential equations has become popular. The method allows to obtain a grid-free approximation of the solution under predefined initial boundary conditions. However, for training a neural network with an arbitrary activation function, it is necessary to use control points in the solution area, where the fulfillment of the differential equation and the specified initial boundary conditions is checked. The quality of the final neural network approximation consists of two factors: the accuracy of the initial boundary conditions and the accuracy of the approximation of the differential equation. This approach has limitations: performing the differential equation at control points does not guarantee performing the differential equation at arbitrary solution points other than the control points. The limitation can be offset by cross-checking the quality of convergence of the neural network on test points that are not included in the training set, but it is impossible to completely eliminate this effect using this method.

A new approach using parametric activation functions is proposed, which makes it possible to efficiently approximate solutions to boundary value problems of linear differential equations of parabolic type. The approach is described on the example of approximation of solutions to the boundary value problem of the diffusivity equation.

References:

Basniev, K. S., Kochina, I. N., & Maksimov, V. M. (1993). Underground hydromechanics. Nedra. Pp. 227–252. [In Russian]

Barenblatt, G. I., Kostov, V. M., & Ryzhik, V. M. (1984). Movement of liquids and gases in natural formations. Nedra. 211 p. [In Russian]

Butyrskiy, E. Yu., Kuvaldin, I. A., & Chalkin, V. P. (2010). Approximation of multidimensional functions. Scientific Device Development, 20(2), 82–92. [In Russian]

Vershinin, V. E., Ponomarev, R. Y. (2023). Application of neural network modeling methods in solving initial boundary value problems for partial differential equations. Tyumen State University Herald. Physical and Mathematical Modeling. Oil, Gas, Energy, 9(3), 132–147. [In Russian]

Vasiliev, A. N., Tarkhov, D. A., & Shemyakina, T. A. (2015). Neural network approach to mathematical physics problems. Nestor-Istoria. 260 p. [In Russian]

Vasiliev, A. N., Tarkhov, D. A., & Shemyakina, T. A. (2016) Approximate analytical solutions of ordinary differential equations. Modern Information Technologies and IT Education, 12(2–3), 188–195. [In Russian]

Gorban, A. N. (1998). Generalized approximation theorem and computational capabilities of neural networks. Siberian Journal of Computer Engineering, 1(1), 12–24. [In Russian]

Zatrelova, D. P., & Ulyanov, S. V. (2022). Models of physically informed classical Lagrangian/Hamiltonian neural networks in deep learning. Modern Information Technologies and IT Education, 18(2), 310–325. [In Russian]

Kovalenko, A. N., Chernomorets, A. A., & Petina, M. A. (2017). On the use of neural networks for solving partial differential equations. Scientific Bulletin of Belgorod State University. Series: Economics. Informatics, 9(258), 103–110. [In Russian]

Kolmogorov, A. N. (1957). On the representation of continuous functions of several variables as a superposition of continuous functions of one variable. Doklad AN SSSR = Report of the USSR Academy of Sciences, 114(5), 953–956. [In Russian]

Khaykin, S. (2019). Neural networks. Full course. Williams. 1103 p. [In Russian]

Aziz, K., Settari, A. (1979). Petroleum reservoir simulation. Chapman and Hall, Boca Raton

Cybenko, G. V. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control Signals and Systems, 2(4), 303–314.

Fraces, C. G., Tchelepi, H. (2021). Physics informed deep learning for flow and transport in porous media. arXiv:2104.02629. https://doi.org/10.48550/arXiv.2104.02629

Fraces, C. G., Tchelepi, H. (2022). Uncertainty quantification for transport in porous media using parameterized physics informed neural networks. arXiv:2205.12730. https://doi.org/10.48550/arXiv.2205.12730

Fuks, O., Tchelepi, H. A. (2020). Limitations of physics informed machine learning for nonlinear two-phase transport in porous media. Journal of Machine Learning for Modeling and Computing, 1(1), 19–37.