Is there a connection between the modulation of Laplace’s Tidal Equation (LTE) solutions and the highly nonlinear fits of neural networks?

“Neural tensor networks have been widely used in a large number of natural language processing tasks such as conversational sentiment analysis, named entity recognition and knowledge base completion. However, the mathematical explanation of neural tensor networks remains a challenging problem, due to the bilinear term. According to Taylor’s theorem, a kth order differentiable function can be approximated by a kth order Taylor polynomial around a given point. Therefore, we provide a mathematical explanation of neural tensor networks and also reveal the inner link between them and feedforward neural networks from the perspective of Taylor’s theorem. In addition, we unify two forms of neural tensor networks into a single framework and present factorization methods to make the neural tensor networks parameter-efficient. Experimental results bring some valuable insights into neural tensor networks.”

https://w.sentic.net/new-perspective-for-neural-tensor-networks.pdf

The connection is via Taylor’s series expansion whereby neural nets try to resolve tight inflection points that occur naturally in the numerical flow of potentially turbulent fluid dynamics.

- Li, Wei, Luyao Zhu, and Erik Cambria. “Taylor’s theorem: A new perspective for neural tensor networks.”
*Knowledge-Based Systems*228 (2021): 107258. - Zhao, H., Chen, Y., Sun, D., Hu, Y., Liang, K., Mao, Y., … & Shao, H. “TaylorNet: A Taylor-Driven Generic Neural Architecture”. submitted to ICLR 2023

Also wavelets can maneuver tight inflections.

“In this study, the applicability of physics informed neural networks using wavelets as an activation function is discussed to solve non-linear differential equations. One of the prominent equations arising in fluid dynamics namely Blasius viscous flow problem is solved. A linear coupled differential equation, a non-linear coupled differential equation, and partial differential equations are also solved in order to demonstrate the method’s versatility. As the neural network’s optimum design is important and is problem-specific, the influence of some of the key factors on the model’s accuracy is also investigated. To confirm the approach’s efficacy, the outcomes of the suggested method were compared with those of the existing approaches. The suggested method was observed to be both efficient and accurate.”

https://www.nature.com/articles/s41598-023-29806-3

- Uddin, Z., Ganga, S., Asthana, R.
*et al.*Wavelets based physics informed neural networks to solve non-linear differential equations.*Sci Rep***13**, 2882 (2023). https://doi.org/10.1038/s41598-023-29806-3

Given the fact that NN results are so difficult to reverse engineer to match a physical understanding could this be a missing link?

Continue reading