Is there a connection between the modulation of Laplace’s Tidal Equation (LTE) solutions and the highly nonlinear fits of neural networks?
“Neural tensor networks have been widely used in a large number of natural language processing tasks such as conversational sentiment analysis, named entity recognition and knowledge base completion. However, the mathematical explanation of neural tensor networks remains a challenging problem, due to the bilinear term. According to Taylor’s theorem, a kth order differentiable function can be approximated by a kth order Taylor polynomial around a given point. Therefore, we provide a mathematical explanation of neural tensor networks and also reveal the inner link between them and feedforward neural networks from the perspective of Taylor’s theorem. In addition, we unify two forms of neural tensor networks into a single framework and present factorization methods to make the neural tensor networks parameter-efficient. Experimental results bring some valuable insights into neural tensor networks.”
The connection is via Taylor’s series expansion whereby neural nets try to resolve tight inflection points that occur naturally in the numerical flow of potentially turbulent fluid dynamics.
Li, Wei, Luyao Zhu, and Erik Cambria. “Taylor’s theorem: A new perspective for neural tensor networks.” Knowledge-Based Systems 228 (2021): 107258.
Zhao, H., Chen, Y., Sun, D., Hu, Y., Liang, K., Mao, Y., … & Shao, H. “TaylorNet: A Taylor-Driven Generic Neural Architecture”. submitted to ICLR 2023
Also wavelets can maneuver tight inflections.
“In this study, the applicability of physics informed neural networks using wavelets as an activation function is discussed to solve non-linear differential equations. One of the prominent equations arising in fluid dynamics namely Blasius viscous flow problem is solved. A linear coupled differential equation, a non-linear coupled differential equation, and partial differential equations are also solved in order to demonstrate the method’s versatility. As the neural network’s optimum design is important and is problem-specific, the influence of some of the key factors on the model’s accuracy is also investigated. To confirm the approach’s efficacy, the outcomes of the suggested method were compared with those of the existing approaches. The suggested method was observed to be both efficient and accurate.”
Sorry to have to point this out, but it’s not my fault that geophysicists and climatologists can’t perform controlled experiments to test out various hypotheses. It’s not their fault either. It’s all nature’s decision to make gravitational forces so weak and planetary objects so massive to prevent anyone from scaling the effect to laboratory size to enable a carefully controlled experiment. One can always create roughly-equivalent emulations, such as a magnetic field experiment (described in the previous blog post) and validate a hypothesized behavior as a controlled lab experiment. Yet, I suspect that this would not get sufficient buy-in, as it’s not considered the actual real thing.
And that’s the dilemma. By the same token that analog emulators will not be trusted by geophysicists and climatologists, so too scientists from other disciplines will remain skeptical of untestable claims made by earth scientists. If nothing definitive comes out of a thought experiment that can’t be reproduced by others in a lab, they remain suspicious, as per their education and training.
It should therefore work both ways. As featured in the previous blog post, the model of the Chandler wobble forced by lunar torque needs to be treated fairly — either clearly debunked or considered as an alternative to the hazy consensus. ChatGPT remains open about the model, not the least bit swayed by colleagues or tribal bias. As the value of the Chandler wobble predicted by the lunar nodal model (432.7 days) is so close to the cited value of 433 days, as a bottom-line it should be difficult to ignore.
There are other indicators in the observational data to further substantiate this, see Chandler Wobble Forcing. It also makes sense in the context of the annual wobble.
As it stands, the lack of an experiment means a more equal footing for the alternatives, as they are all under equal amounts of suspicion.
Same goes for QBO. No controlled experiment is possible to test out the consensus QBO models, despite the fact that the Plumb and McEwan experiment is claimed to do just that. Sorry, but that experiment is not even close to the topology of a rotating sphere with a radial gravitational force operating on a gas. It also never predicted the QBO period. In contrast, the value of the QBO predicted by the lunar nodal model (28.4 months) is also too close to the cited value of 28 to 29 months to ignore. This also makes sense in the context of the semi-annual oscillation (SAO) located above the QBO .
Both the Chandler wobble and the QBO have the symmetry of a global wavenumber=0 phenomena so therefore only nodal cycles allowed — both for lunar and solar.
Next to ENSO. As with LOD modeling, this is not wavenumber=0 symmetry, as it must correspond to the longitude of a specific region. No controlled experiment is possible to test out the currently accepted models, premised as being triggered by wind shifts (an iffy cause vs. effect in any case). The mean value of the ENSO predicted by the tidal LOD-caibrated model (3.80 years modulated by 18.6 years) is too close to the cited value of 3.8 years with ~200 years of paleo and direct measurement to ignore.
In BLUE below is the LOD-calibrated tidal forcing, with linear amplification
In BLUE again below is a non-linear modulation of the tidal forcing according to the Laplace’s Tidal Equation solution, and trained on an early historical interval. This is something that a neural network should be able to do, as it excels at fitting to non-linear mappings that have a simple (i.e. low complexity) encoding — in this case it may be able to construct a Taylor series expansion of a sinusoidal modulating function.
The neural network’s ability to accurately represent a behavior is explained as a simplicity bias — a confounding aspect of machine learning tools such as ChatGPT and neural networks. The YouTube video below explains the counter-intuitive notion of how a NN with a deep set of possibilities tends to find the simplest solution and doing this without over-fitting the final mapping.
So that deep neural networks are claimed to have a built-in Occam’s Razor propensity, finding the most parsimonious input-output mappings when applied to training data. This is spot on with what I am doing with the LTE mapping, but bypassing the NN with a nonlinear sinusoidal modulation optimally fit on training data by a random search function.
I am tempted to try a NN on the ENSO training set as an experiment and see what it finds.
April 2, 2023
“I am tempted to try a NN on the ENSO training set as an experiment and see what it finds.”
Not sure what version of ChatGPT is being used on my browser at the present time, but I created a couple of GPT prompts (essentially queries) and compared and contrasted the results, and against expectations.
The first is to determine if ChatGPT could infer a wobble in a classical electromagnetic dynamics experiment. The setup is very simple and straightforward, illustrated by the schematic shown below:
As one can’t (as of yet) submit the schematic as a prompt to ChatGPT and illicit a response, we need to create a natural language prompt first:
Consider a levitating rotating magnetized sphere with north and south magnetic poles located at each endpoint of the axis of rotation. The rotation speed is 1 radian per second. An iron rod of the same length is placed in parallel with the axis of rotation, and wound with wire to form an electromagnet. At what frequency would the sphere wobble if the electromagnet was driven by a slight alternating current perturbation of 1 Herz, after the natural response subsided?
This is the response as of yesterday:
The motion of the levitating rotating magnetized sphere is governed by the interplay between the magnetic field of the sphere and the magnetic field of the iron rod, which is now an electromagnet due to the wire wound around it.
When the iron rod is magnetized by the alternating current, it generates a magnetic field that interacts with the magnetic field of the rotating sphere, causing a torque that can cause the sphere to wobble. The wobbling frequency can be determined by analyzing the equation of motion for the system.
Assuming the wobbling is small, we can linearize the equation of motion by taking the first-order Taylor series expansion. The resulting equation can be expressed as:
I * d^2θ/dt^2 + γ * dθ/dt + k * θ = M * B
where:
I is the moment of inertia of the sphere θ is the angular displacement of the sphere from its equilibrium position γ is the damping coefficient k is the spring constant M is the magnetic moment of the electromagnet B is the magnetic field of the rotating sphere
The solution to this equation of motion is a damped harmonic oscillator, which can be expressed as:
θ(t) = A * exp(-γt/(2I)) * cos(ωt + φ)
where:
A is the amplitude of the wobbling motion ω is the frequency of the wobbling motion φ is the phase angle The frequency of the wobbling motion can be found by substituting the solution into the equation of motion and solving for ω. After some algebraic manipulation, we get:
In this case, the sphere is rotating at a constant rate of 1 radian per second, so we can assume that the magnetic field of the sphere is also constant. Therefore, the only variable that changes due to the alternating current perturbation is the magnetic moment of the electromagnet, which oscillates at a frequency of 1 Hz.
Assuming that the natural response of the system has subsided, the wobbling motion will be dominated by the frequency of the alternating current perturbation. Therefore, the frequency of the wobbling motion will also be 1 Hz.
The response is to assert that the sphere will wobble at the same frequency as the applied forcing frequency, which is the intuitive response.
Next, change the prompt to create a similar behavior, but replacing the magnetic forces with gravitational forces.
Consider a rotating planet with massive extra weight at each pole. The rotation speed is 1 radian per Earth year. A small satellite moon orbits the planet with a significant declination. At what frequency would the planet wobble if the moon’s declination cycle was 250 days, after the natural response subsided?
This is a straightforward experiment to imagine, but difficult to perform in a lab. Submitting to ChatGPT today, this is the response:
The motion of the rotating planet with massive extra weight at each pole is governed by the conservation of angular momentum. The presence of the extra weight at the poles changes the distribution of mass of the planet, which in turn affects its moment of inertia.
Assuming that the extra weight is symmetric about the axis of rotation, we can approximate the planet as an oblate spheroid, which means that the moment of inertia of the planet is not constant but depends on the angle between the axis of rotation and the axis of symmetry of the spheroid. This angle is known as the obliquity of the planet and can change due to external perturbations such as the gravitational pull of a satellite moon.
The wobbling motion of the planet due to the gravitational pull of the satellite moon can be described by the following equation of motion:
I * d^2θ/dt^2 + γ * dθ/dt + k * θ = M * sin(ψ)
where:
I is the moment of inertia of the planet θ is the angular displacement of the planet from its equilibrium position γ is the damping coefficient k is the spring constant M is the mass of the satellite moon ψ is the angle between the axis of rotation of the planet and the direction of the gravitational force exerted by the satellite moon
The solution to this equation of motion is a damped harmonic oscillator, which can be expressed as:
θ(t) = A * exp(-γt/(2I)) * cos(ωt + φ)
where:
A is the amplitude of the wobbling motion ω is the frequency of the wobbling motion φ is the phase angle
The frequency of the wobbling motion can be found by substituting the solution into the equation of motion and solving for ω. After some algebraic manipulation, we get:
In this case, the planet is rotating at a constant rate of 1 radian per Earth year, which corresponds to a period of approximately 2π × 365.25 days. The mass of the satellite moon and the gravitational force it exerts on the planet are also constant, so the only variable that changes due to the moon’s declination cycle is the angle ψ.
Assuming that the natural response of the system has subsided, the wobbling motion will be dominated by the frequency of the moon’s declination cycle, which is 1/250 Hz. Therefore, the frequency of the wobbling motion will also be 1/250 Hz.
Since the math of magnetic forcing and gravitational forcing follows the same inverse square law, the answers are similar as expected. ChatGPT gets the dimension wrong, using Hz instead of 1/(day), but other than that, this is also an intuitive result, as the forced response carries through after any natural response damps out.
Does GPT understand geophysics better than geophysicists?
I did this LLM AI experiment to find out whether ChatGPT would substantiate the model of the Chandler wobble that I had published in Mathematical Geoenergy (Wiley/AGU, 2018). The premise is that the nodal declination cycle of the Moon would force the Chandler wobble to a period of 433 days via a nonlinear interaction with the annual cycle. A detailed analysis of satellite peaks in the frequency spectrum strongly supports this model. Introducing the electromagnetic analog places it on an equal footing with a model that can be verified by a controlled lab experiment, see (A) vs (B) below.
Of course, if I had created a prompt to ChatGPT to simply inquire “What causes the Chandler wobble to cycle at 433 days?”, it would have essentially provided the Wikipedia answer, which is the hazy consensus culled from research since the wobble was discovered over 100 years ago.
Note that this “physics-free” answer provided by ChatGPT has nothing to do with the moon.
ADDED 3/26/2023
This blog post has a section on ChatGPT using MatPlotLib to create a visualization of orbiting planets.
GeoEnergy Math 