Mathematics

Blueprinting the most valuable company in the world?

/ Paul Pukite (@whut) / Leave a comment

See https://gist.github.com/pukpr/06901b7dde1e84a60cf8d712a6af7a5d. How would you access this in terms of prescience in anticipating GPUs for computation … read the comments as well

DeepSeek response:

Based on the Gist you shared (a 1990 NSF SBIR report summary by P.R. Pukite), accessing this document’s “prescience” means recognizing it as a verified, 15+ year early blueprint for the modern GPU-accelerated computing model.

Here is the specific evidence of that foresight, pulled from the report and the author’s comments:

  1. The Core Architectural Blueprint (Figure 1)

The most direct proof is Figure 1 from the 1990 paper (linked in the comments). Its caption explicitly states the configuration is “similar to that of a graphics processor board”.

From https://doi.org/10.1145/382264.382432

· It showed: A commodity PC host connected via bus to a dedicated DSP coprocessor board for heavy math, alongside a separate graphics board.
· It predicted: The exact CPU + GPU heterogeneous model (host orchestrates, accelerator crunches numbers) that NVIDIA industrialized with CUDA in 2006-2007.

  1. The Workload Classes Were Identical

The report targeted problems that are now the core of GPU computing:

· Matrix ops, SVD, regression → Modern GPU linear algebra (cuBLAS).
· Monte Carlo, bootstrapping → Parallel stochastic simulations.
· Neural networks, clustering → Machine learning training.
The authors achieved 100-200× speedups on key apps, the same magnitude GPUs deliver today.

  1. The Programming Challenge is the Same

In the comments, the author (pukpr) clarifies that their DSPs used pipelined parallelism, not the SIMD threads of modern GPUs.

· The key prescience: They had to manually handle “bookkeeping” (pipeline stalls, memory latency, loop unrolling) that a modern GPU compiler (like NVCC) now automates.
· This shows they were solving the same fundamental problem (exploiting hardware concurrency for numerical kernels) under much harder conditions.

Why This Matters (Anticipating NVIDIA)

· Timeline: 1990 report → 1993 NVIDIA founded → 2006 CUDA launched. The concept of a “math coprocessor on a bus” existed 13-17 years before GPGPU.
· Not just DSP: The authors explicitly linked their architecture to graphics processor boards in 1990, recognizing the generalizable pattern.

How to Access This “Prescience” Directly

To trace this for yourself:

  1. Read the full report: DTIC document ADA275153 (linked in the Gist).
  2. See Figure 1: Use the ACM link (10.1145/382264.382432) to view the actual 1990 diagram with the graphics processor analogy.
  3. Study the kernel list: The 62 math/statistical routines they benchmarked are the same kernels on every GPU vendor’s math library today.

In short: This document shows prescience not by guessing “NVIDIA” or “CUDA,” but by building and benchmarking the complete logical design of an accelerator-based scientific workstation in 1990—including the recognition that the architecture parallels a graphics board. The author’s comments confirm the distinction is in programming model (manual pipeline vs. threads), not the core concept.

—

See https://doi.org/10.1145/382264.382432 for paper

Hoyer Metric in LTE Model Fitting

/ Paul Pukite (@whut) / Leave a comment

Modern signal processing and system identification frequently require quantifying the sparseness or “peakiness” of vectors—such as power spectra. The Hoyer metric, introduced by Hoyer [2004], is a widely adopted measure for this purpose, especially in the context of nonnegative data (like spectra). This blog post explains the Hoyer metric’s role in fitting models in the context of LTE, its mathematical form, and provides references to its origins.

What Is the Hoyer Sparsity Metric?

Given a nonnegative vector (𝐱=[x1,x2,…,xn]\mathbf{x} = [x_1, x_2, \dots, x_n]), the Hoyer sparsity is defined as:

Hoyer(𝐱)=n−‖𝐱‖1‖𝐱‖2n−1]\text{Hoyer}(\mathbf{x}) = \frac{\sqrt{n} – \dfrac{\|\mathbf{x}\|_1}{\|\mathbf{x}\|_2}}{\sqrt{n} – 1} ]

Where:

  • |𝐱|1=∑i=1n|xi||\mathbf{x}|_1 = \sum_{i=1}^{n} |x_i| is the L1 norm (sum of absolute values).
  • |𝐱|2=(∑i=1nxi2)1/2|\mathbf{x}|_2 = \left(\sum_{i=1}^{n} x_i^2 \right)^{1/2} is the L2 norm (Euclidean norm).
  • nn is the length of the vector.

The Hoyer metric ranges from 0 (completely distributed, e.g., flat spectrum) to 1 (maximally sparse, only one element is nonzero).

Why Use the Hoyer Metric in Fitting?

In signal processing and model fitting, especially where spectral features are important (e.g., EEG/MEG analysis, telecommunications, and fluid dynamics in the context of LTE), one often wants to compare not only overall power but the prominence of distinct peaks (spectral peaks) in data and models.

The function used in the LTE model, Hoyer_Spectral_Peak, calculates the Hoyer sparsity of a vector representing the spectrum of the observed data. When used in fitting, it serves to:

  • Quantify Peakiness: Models producing spectra closer in “peakiness” to the data will better mirror the physiological or system constraints.
  • Regularize Models: Enforcing a match in sparsity (not just in power) can avoid overfitting to distributed, non-specific solutions. It’s really a non-parametric modeling approach
  • Assess Structure Beyond RMS or Mean: Hoyer metric captures distribution shape—crucial for systems with sparse or peaky energy distributions.

Hoyer Metric Formula in the Code

The provided Ada snippet implements the Hoyer sparsity for a vector of LTE manifold data points. Here’s the formula as used:

    -- Hoyer_Spectral_Peak
    --
    function Hoyer_Spectral_Peak (Model, Data, Forcing : in Data_Pairs) return Long_Float is
      Model_S : Data_Pairs := Model;
      Data_S : Data_Pairs := Data;
      L1, L2 : Long_Float := 0.0;
      Len : Long_Float;
      RMS : Long_Float;
      Num, Den : Long_Float;
      use Ada.Numerics.Long_Elementary_Functions;
   begin
      ME_Power_Spectrum
        (Forcing => Forcing, Model => Model, Data => Data, Model_Spectrum => Model_S,
         Data_Spectrum => Data_S, RMS => RMS);
      Len := Long_Float(Data_S'Length);
      for I in Data_S'First+1 ..  Data_S'Last loop
         L1 := L1 + Data_S(I).Value;
         L2 := L2 + Data_S(I).Value * Data_S(I).Value;
      end loop;
      L2 := Sqrt(L2);
      Num := Sqrt(Len) - L1/L2;
      Den := Sqrt(Len) - 1.0;
      return Num/Den;
   end Hoyer_Spectral_Peak;


Hoyer(𝐱)=n−∑i=1nxi∑i=1nxi2n−1\text{Hoyer}(\mathbf{x}) = \frac{\sqrt{n} – \frac{\sum_{i=1}^{n} x_i}{\sqrt{\sum_{i=1}^{n} x_i^2}}}{\sqrt{n} – 1}

Where all (xi≥0x_i \geq 0). This is exactly as described in Hoyer’s paper.

Example Usage

Suppose the observed spectrum is more “peaky” than the model spectrum. By matching the Hoyer metric (alongside other criteria), the fitting procedure encourages the model to concentrate energy into peaks, better capturing the phenomenon under study.

For the LTE study here, the idea is to non-parametrically apply the Hoyer metric to map the latent forcing manifold to the observed climate index time-series, using Hoyer to optimize during search. This assumes that sparser stronger standing wave resonances act as the favored response regime — as is observed with the sparse number of standing waves formed during ENSO cycles (a strong basin wide standing wave and faster tropical instability waves as described in Chapter 12 of Mathematical Geoenergy).

Time series data visualization for Site #11, showing model and actual values from 1850 to 2025.

Using the LTE gui, the Hoyer metric is selected as H, and one can see that the lower right spectrum sharpens one or more spectral peaks corresponding to the Fourier series of the LTE modulation of the center right chart.

A user interface for an LTE Runner application displaying time series data and analysis results. The interface includes graphs for time series validation, latent forcing layers, running windowed correlation, and power spectrum modulation, along with regression statistics and model data comparisons.

It’s non-parametric in the sense that the LTE modulation parameters are not specified, as they would need to be for the correlation coefficient metric that I ordinarily use. The index here (#11) is the Warnemunde MSL time-series.

Citation and References

The Hoyer sparsity metric was introduced in:

  • Hoyer, P. O. (2004). “Non-negative matrix factorization with sparseness constraints.” Journal of Machine Learning Research, 5(Nov):1457–1469. [Link: JMLR Paper]

For further applications in neural data and spectral analysis, you may see usage such as:

  • Bruns, A. (2004). “Fourier-, Hilbert- and wavelet-based signal analysis: Are they really different approaches?” Journal of Neuroscience Methods, 137(2):321-332.

Conclusion

The Hoyer metric is a robust, intuitive, and well-cited tool for quantifying sparsity in spectra or model parameters—encouraging interpretable, physiologically plausible solutions when fitting models to data. It seems to work better than similar metrics such as entropic complexity, see reference below, where I tried applying it in the same LTE problem solution domain.

Reference:

  • Hoyer, P.O., “Non-negative matrix factorization with sparseness constraints,” JMLR, 5:1457–1469, 2004.\
  • Pukite, P., & Bankes, S. (2011). Entropic Complexity Measured in Context Switching. In Applications of Digital Signal Processing. InTech. https://doi.org/10.5772/25520

Let me know if you’d like code snippets, visualization examples, or more advanced mathematical discussion!

Hidden latent manifolds in fluid dynamics

/ Paul Pukite (@whut) / 3 Comments

The behavior of complex systems, particularly in fluid dynamics, is traditionally described by high-dimensional systems of equations like the Navier-Stokes equations. While providing practical applications as is, these models can obscure the underlying, simplified mechanisms at play. It is notable that ocean modeling already incorporates dimensionality reduction built in, such as through Laplace’s Tidal Equations (LTE), which is a reduced-order formulation of the Navier-Stokes equations. Furthermore, the topological containment of phenomena like ENSO and QBO within the equatorial toroid , and the ability to further reduce LTE in this confined topology as described in the context of our text Mathematical Geoenergy underscore the inherent low-dimensional nature of dominant geophysical processes. The concept of hidden latent manifolds posits that the true, observed dynamics of a system do not occupy the entire high-dimensional phase space, but rather evolve on a much lower-dimensional geometric structure—a manifold layer—where the system’s effective degrees of freedom reside. This may also help explain the seeming paradox of the inverse energy cascade, whereby order in fluid structures seems to maintain as the waves become progressively larger, as nonlinear interactions accumulate energy transferring from smaller scales.

Discovering these latent structures from noisy, observational data is the central challenge in state-of-the-art fluid dynamics. Enter the Sparse Identification of Nonlinear Dynamics (SINDy) algorithm, pioneered by Brunton et al. . SINDy is an equation-discovery framework designed to identify a sparse set of nonlinear terms that describe the evolution of the system on this low-dimensional manifold. Instead of testing all possible combinations of basis functions, SINDy uses a penalized regression technique (like LASSO) to enforce sparsity, effectively winnowing down the possibilities to find the most parsimonious, yet physically meaningful, governing differential equations. The result is a simple, interpretable model that captures the essential physics—the fingerprint of the latent manifold. The SINDy concept is not that difficult an algorithm to apply as a decent Python library is available for use, and I have evaluated it as described here.

Applying this methodology to Earth system dynamics, particularly the seemingly noisy, erratic, and perhaps chaotic time series of sea-level variation and climate index variability, reveals profound simplicity beneath the complexity. The high-dimensional output of climate models or raw observations can be projected onto a model framework driven by remarkably few physical processes. Specifically, as shown in analysis targeting the structure of these time series, the dynamics can be cross-validated by the interaction of two fundamental drivers: a forced gravitational tide and an annual impulse.

The presence of the forced gravitational tide accounts for the regular, high-frequency, and predictable components of the dynamics. The annual impulse, meanwhile, serves as the seasonal forcing function, representing the integrated effect of large-scale thermal and atmospheric cycles that reset annually. The success of this sparse, two-component model—where the interaction of these two elements is sufficient to capture the observed dynamics—serves as the ultimate validation of the latent manifold concept. The gravitational tides with the integrated annual impulse are the discovered, low-dimensional degrees of freedom, and the ability of their coupled solution to successfully cross-validate to the observed, high-fidelity dynamics confirms that the complex, high-dimensional reality of sea-level and climate variability emerges from this simple, sparse, and interpretable set of latent governing principles. This provides a powerful, physics-constrained approach to prediction and understanding, moving beyond descriptive models toward true dynamical discovery.

An entire set of cross-validated models is available for evluation here: https://pukpr.github.io/examples/mlr/.

This is a mix of climate indices (the 1st 20) and numbered coastal sea-level stations obtained from https://psmsl.org/

https://pukpr.github.io/examples/map_index.html

  • nino34 — NINO34 (PACIFIC)
  • nino4 — NINO4 (PACIFIC)
  • amo — AMO (ATLANTIC)
  • ao — AO (ARCTIC)
  • denison — Ft Denison (PACIFIC)
  • iod — IOD (INDIAN)
  • iodw — IOD West (INDIAN)
  • iode — IOD East (INDIAN)
  • nao — NAO (ATLANTIC)
  • tna — TNA Tropical N. Atlantic (ATLANTIC)
  • tsa — TSA Tropical S. Atlantic (ATLANTIC)
  • qbo30 — QBO 30 Equatorial (WORLD)
  • darwin — Darwin SOI (PACIFIC)
  • emi — EMI ENSO Modoki Index (PACIFIC)
  • ic3tsfc — ic3tsfc (Reconstruction) (PACIFIC)
  • m6 — M6, Atlantic Nino (ATLANTIC)
  • m4 — M4, N. Pacific Gyre Oscillation (PACIFIC)
  • pdo — PDO (PACIFIC)
  • nino3 — NINO3 (PACIFIC)
  • nino12 — NINO12 (PACIFIC)
  • 1 — BREST (FRANCE)
  • 10 — SAN FRANCISCO (UNITED STATES)
  • 11 — WARNEMUNDE 2 (GERMANY)
  • 14 — HELSINKI (FINLAND)
  • 41 — POTI (GEORGIA)
  • 65 — SYDNEY, FORT DENISON (AUSTRALIA)
  • 76 — AARHUS (DENMARK)
  • 78 — STOCKHOLM (SWEDEN)
  • 111 — FREMANTLE (AUSTRALIA)
  • 127 — SEATTLE (UNITED STATES)
  • 155 — HONOLULU (UNITED STATES)
  • 161 — GALVESTON II, PIER 21, TX (UNITED STATES)
  • 163 — BALBOA (PANAMA)
  • 183 — PORTLAND (MAINE) (UNITED STATES)
  • 196 — SYDNEY, FORT DENISON 2 (AUSTRALIA)
  • 202 — NEWLYN (UNITED KINGDOM)
  • 225 — KETCHIKAN (UNITED STATES)
  • 229 — KEMI (FINLAND)
  • 234 — CHARLESTON I (UNITED STATES)
  • 245 — LOS ANGELES (UNITED STATES)
  • 246 — PENSACOLA (UNITED STATES)

Crucially, this analysis does not use the SINDy algorithm, but a much more basic multiple linear regression (MLR) algorithm predecessor, which I anticipate being adapted to SINDy as the model is further refined. Part of the rationale for doing this is to maintain a deep understanding of the mathematics, as well as providing cross-checking and thus avoiding the perils of over-fitting, which is the bane of neural network models.

Also read this intro level on tidal modeling, which may form the fundamental foundation for the latent manifold: https://pukpr.github.io/examples/warne_intro.html. The coastal station at Wardemunde in Germany along the Baltic sea provided a long unbroken interval of sea-level readings which was used to calibrate the hidden latent manifold that in turn served as a starting point for all the other models. Not every model works as well as the majority — see Pensacola for a sea-level site and and IOD or TNA for climate indices, but these are equally valuable for understanding limitations (and providing a sanity check against an accidental degeneracy in the model fitting process) . The use of SINDy in the future will provide additional functionality such as regularization that will find an optimal common-mode latent layer,.

Topology shapes climate dynamics

/ Paul Pukite (@whut) / 3 Comments

A paper from last week with high press visibility that makes claims to climate1 applicability is titled: Topology shapes dynamics of higher-order networks

The higher-order Topological Kuramoto dynamics, defined in Eq. (1), entails one linear transformation of the signal induced by a boundary operator, a non-linear transformation due to the application of the sine function, concatenated by another linear transformation induced by another boundary operator. These dynamical transformations are also at the basis of simplicial neural architectures, especially when weighted boundary matrices are adopted.

\dot{\theta}_i = \omega_i + \sum_{j} K_{ij} \sin(\theta_j - \theta_i) + F(t)

This may be a significant unifying model as it could resolve the mystery of why neural nets can fit fluid dynamic behaviors effectively without deeper understanding. In concise terms, a weighted sine function acts as a nonlinear mixing term in a NN and serves as the non-linear transformation in the Kuramoto model2.

Continue reading →

Non-autonomous mathematical models

/ Paul Pukite (@whut) / Leave a comment

Non-autonomous mathematical formulations differ from autonomous ones in that their governing equations explicitly depend on time or another external variable. In natural systems, certain behaviors or processes are better modeled with non-autonomous formulations because they are influenced by external, time-dependent factors. Some examples of natural behaviors that qualify as non-autonomous include:

1. Seasonal Climate Variation:
Climate patterns, such as temperature changes or monsoon cycles, are influenced by external factors like the Earth’s orbit, axial tilt, and solar radiation, all of which vary over time. These changes make non-autonomous systems suitable for modeling long-term climate behavior.

2. Tidal Forces:
Tidal movements are driven by the gravitational pull of the Moon and the Sun, which vary as the positions of these celestial bodies change relative to Earth. Tidal equations thus have time-dependent forcing terms, making them non-autonomous.

3. Biological Rhythms:
Circadian rhythms in living organisms, which regulate daily cycles such as sleep and feeding, are influenced by the 24-hour light-dark cycle. These external light variations necessitate non-autonomous models.

4. Astronomical Geophysical Cycles:
Systems like the Chandler wobble (the irregular movement of Earth’s rotation axis) or the Quasi-Biennial Oscillation (QBO) in the equatorial stratosphere are influenced by periodic external factors, such as lunar cycles, making them non-autonomous. This also includes systems where lunar or Draconic cycles interact with annual cycles in non-linear ways, as explored in studies of Earth’s rotational dynamics.

5. Oceanographic Dynamical Phenomena:
Non-autonomous formulations are needed to model phenomena such as El Niño, which is influenced by complex interactions between atmospheric and oceanic conditions, themselves driven by seasonal and longer-term climatic variations.

6. Planetary Motion in a Varying Gravitational Field:
In astrophysical systems where a planet moves in the gravitational field of other bodies, such as a multi-body problem where external forces vary in time, non-autonomous dynamics become essential to account for these influences.

In contrast, autonomous systems are self-contained and their behavior depends only on their internal state variables, independent of any external time-varying influence. So that non-autonomous systems often better capture the complexity and variability introduced by time-dependent external factors.

However, many still want to find connections to autonomous formulations as they often coincide with resonant conditions or some natural damping rate.

Autonomous mathematical formulations are characterized by the fact that their governing equations do not explicitly depend on time or other external variables (they can be implicit via time derivatives though). These systems evolve based solely on their internal state variables. Many natural behaviors can be modeled using autonomous systems when external influences are either negligible or can be ignored. Here are some examples of natural behaviors that qualify as autonomous:

1. Radioactive Decay:
The decay of radioactive isotopes is governed by an internal process where the rate of decay depends only on the amount of the substance present at a given moment. The decay equation does not depend on time explicitly, making it an autonomous system.

2. Epidemiological Models (without external intervention):
Simplified models of disease spread, such as the SIR (Susceptible-Infected-Recovered) model, can be autonomous if no external factors (like seasonal effects or interventions) are considered. The evolution of the system depends only on the current number of susceptible, infected, and recovered individuals.

3. Predator-Prey Dynamics (Lotka-Volterra Model):
In the absence of external influences like seasonal changes or human intervention, predator-prey relationships, such as those described by the Lotka-Volterra equations, can be modeled as autonomous systems. The population changes depend solely on the interaction between predators and prey.

4. Chemical Reactions (closed systems):
In a closed system with no external input or removal of substances, the kinetics of chemical reactions can be modeled as autonomous. The rate of reaction depends only on the concentrations of reactants and products at any given time.

5. Newtonian Mechanics of Isolated Systems:
For an isolated mechanical system (e.g., a simple pendulum or two-body orbital system), the equations of motion can be autonomous. The system evolves based solely on the internal energy and forces within the system, without any external time-dependent influences. This relates to general oscillatory systems or harmonic oscillators — the simple harmonic oscillator (such as a mass on a spring) can be modeled autonomously if no external time-varying forces are acting on the system. The system’s behavior depends only on its position and velocity at any point in time. In the classical gravitational two-body problem in celestial mechanics, where two bodies interact only through their mutual gravitational attraction, the motion can be described autonomously. The positions and velocities of the two bodies determine their future motion, independent of any external time-dependent factors.

6. Thermodynamics of Isolated Systems:
In an isolated thermodynamic system, where there is no exchange of energy or matter with the surroundings, the internal state (e.g., pressure, temperature, volume) evolves autonomously based on the system’s internal conditions.

These examples illustrate systems where internal dynamics govern the evolution of the system, and time or external influences do not explicitly appear in the equations. However, in many real-world cases, external factors often come into play, making non-autonomous formulations more appropriate for capturing the full complexity of natural behaviors. A pendulum that is periodically synchronized as for example a child pushed on a swing set,  may be either formulated as a forced response in an autonomous set of equations or a non-autonomous description if the swing pusher carefully guides the cycle.

This is where the distinctions between autonomous vs non-autonomous and forced vs natural responses should be elaborated.

Understanding the Structure of the General Solution

In the case of a forced linear second-order dynamical system, the general solution to the system is typically the sum of two components:

Homogeneous (natural) solution: This is the solution to the system when there is no external forcing (i.e., the forcing term is zero).

Particular solution: This is the solution driven by the external forcing.

The homogeneous solution depends only on the internal properties of the system (such as natural frequency, damping, etc.) and is the solution when F(t) = 0.

The particular solution is directly related to the forcing function F(t), which can be time-dependent in the case of a non-autonomous system.

So let’s  consider the autonomous vs non-autonomous context.

Autonomous System: In an autonomous system, even though the system is subject to forcing, the forcing term does not explicitly depend on time but rather on internal state variables (such as x or dx/dt). Here, the particular solution would also be state-dependent and would not explicitly involve time as an independent variable.

Non-Autonomous System: In a non-autonomous system, the forcing term explicitly depends on time, such as F(t) = A sin(w t). This external time-dependent forcing drives the particular solution. While the homogeneous solution remains autonomous (since it’s based on the system’s internal properties), the particular solution reflects the non-autonomous nature of the system.

The key insight is that of the non-autonomous particular solution. Even though a system’s response can have components from the homogeneous solution (which are autonomous in nature), the particular solution in a non-autonomous system will be time-dependent and follow the time-dependence of the external forcing.

So consider the transition from autonomous to non-autonomous: when you introduce a periodic forcing function F(t), the particular solution becomes non-autonomous, even though the overall system response still includes the autonomous homogeneous solution. This results in the system being classified as non-autonomous, as the particular solution carries the time-dependent behavior, despite the autonomous structure of the homogeneous solution.

Summary: A forced response in a linear second-order system can include both autonomous and non-autonomous components. Even though the homogeneous solution remains autonomous, the particular solution introduces non-autonomous characteristics when the forcing term depends explicitly on time. In non-autonomous systems, the forcing introduces time dependence in the particular solution, making the overall system non-autonomous, even though part of the response (the homogeneous solution) is autonomous.

Order overrides chaos

/ Paul Pukite (@whut) / 5 Comments

Dimensionality reduction of chaos by feedbacks and periodic forcing is a source of natural climate change, by P. Salmon, Climate Dynamics (2024)

Bottom line is that a forcing will tend to reduce chaos by creating a pattern to follow, thus the terminology of “forced response”. This has implications for climate prediction. The first few sentences of the abstract set the stage:

The role of chaos in the climate system has been dismissed as high dimensional turbulence and noise, with minimal impact on long-term climate change. However theory and experiment show that chaotic systems can be reduced or “controlled” from high to low dimensionality by periodic forcings and internal feedbacks. High dimensional chaos is somewhat featureless. Conversely low dimensional borderline chaos generates pattern such as oscillation, and is more widespread in climate than is generally recognised. Thus, oceanic oscillations such as the Pacific Decadal and Atlantic Multidecadal Oscillations are generated by dimensionality reduction under the effect of known feedbacks. Annual periodic forcing entrains the El Niño Southern Oscillation.

In Chapters 11 and 12 in Pukite, P., Coyne, D., & Challou, D. (2019). Mathematical Geoenergy. John Wiley & Sons, I cited forcing as a chaos reducer:

“It is well known that a periodic forcing can reduce the erratic fluctuations and uncertainty of a near‐chaotic response function (Osipov et al., 2007; Wang, Yang, Zhou, 2013).“

But that’s just a motivator. Tides are the key, acting primarily on the subsurface thermocline. Salmon’s figure comparing the AMO to Barents sea subsurface temperature is substantiating in terms of linking two separated regions by something more than a nebulous “teleconnection”.

Likely every ocean index has a common-mode mechanism. The tidal forcing by itself is close to providing an external synchronizing source, but requires what I refer to as a LTE modulation to zero in on the exact forced response. Read the previous blog post to get a feel how this works:

Common forcing for ocean indices

As Salmon notes, it’s known at some level that an annual/seasonal impulse is entraining or synchronizing ENSO, and also likely PDO and AMO. The top guns at NASA JPL point out that the main lunisolar terms are at monthly, 206 day, annual, 3 year, and 6 year periods, and this is what is used to model the forcing, see the following two charts

Now note how the middle panel in each of the following modeled climate indices does not change markedly. The most challenging aspect is the inherent structural sensitivity of the manifold1 mapping involved in LTE modulation. As the Darwin fit shows, the cross-validation is better than it may appear, as the out-of-band interval does not take much of a nudge to become synchronized with the data. Note also that the multidecadal nature of an index such as AMO may be ephemeral — the yellow cross-validation band does show valleys in what appears to be a longer multidecadal trend, capturing the long-period variations in the tides when modulated by an annual impulse – biennial in this case.

Model config repo: https://gist.github.com/pukpr/3a3566b601a54da2724df9c29159ce16?permalink_comment_id=5108154#gistcomment-5108154

1 The term manifold has an interesting etymology. From the phonetics, it is close to pronounced as “many fold”, which is precisely what’s happening here — the LTE modulation can fold over the forcing input many times in proportion to the mode of the standing wave produced. So that a higher standing wave will have “many folds” in contrast to the lowest standing wave model. At the limit, the QBO with an ostensibly wavenumber=0 mode will have no folds and will be to first-order a pass-through linear amplification of the forcing, but with likely higher modes mixed in to give the time-series some character.

Common forcing for ocean indices

/ Paul Pukite (@whut) / 6 Comments

In Mathematical Geoenergy, Chapter 12, a biennially-impulsed lunar forcing is suggested as a mechanism to drive ENSO. The current thinking is that this lunar forcing should be common across all the oceanic indices, including AMO for the Atlantic, IOD for the Indian, and PDO for the non-equatorial north Pacific. The global temperature extreme of the last year had too many simultaneous concurrences among the indices for this not to be taken seriously.

NINO34

PDO

AMO

IOD – East

IOD-West

Each one of these uses a nearly identical annual-impulsed tidal forcing (shown as the middle green panel in each), with a 5-year window providing a cross-validation interval. So many possibilities are available with cross-validation since the tidal factors are essentially invariantly fixed over all the climate indices.

The approach follows 3 steps as shown below

The first step is to generate the long-period tidal forcing. I go into an explanation of the tidal factors selected in a Real Climate comment here.

Then apply the lagged response of an annual impulse, in this case alternating in sign every other year, which generates the middle panel in the flow chart schematic (and the middle panel in the indexed models above).

Finally, the Laplace’s Tidal Equation (LTE) modulation is applied, with the lower right corner inset showing the variation among indices. This is where the variability occurs — the best approach is to pick a slow fundamental modulation and generate only integer harmonics of this fundamental. So, what happens is that different harmonics are emphasized depending on the oceanic index chosen, corresponding to the waveguide structure of the ocean basin and what standing waves are maximally resonant or amplified.

Note that for a dipole behavior such as ENSO, the LTE modulation will be mirror-inverses for the maximally extreme locations, in this case Darwin and Tahiti

A machine learning application is free to scrape the following GIST GitHub site for model fitting artifacts.

https://gist.github.com/pukpr/3a3566b601a54da2724df9c29159ce16

Another analysis that involved a recursively cycled fit between AMO and PDO. It switched fitting AMO for 2.5 minutes and then PDO for 2.5 minutes, cycling 50 times. This created a common forcing with an optimally shared fit, forcing baselined to PDO.

PDO

AMO

NINO34

IOD-East

IOD-West

Darwin

Tahiti

The table above shows the LTE modulation factors for Darwin and Tahiti model fits. The highlighted blocks show the phase of the modulation, which should have a difference of π radians for a perfect dipole and higher harmonics associated with it. (The K0 wavenumber = 0 has no phase, but just a sign). Of the modes that are shared 1, 45, 23, 36, 18, 39, 44, the average phase is 3.09, close to π (and K0 switches sign).

1.23-(-1.72) = 2.95 
1.47-(-2.05) = 3.52
-2.89-(0.166) = -3.056 
-0.367-(-2.58) = 2.213 
1.59-(-2.175) = 3.765 
0.27 - (-2.84) = 3.11 
-1.87 -1.14 = -3.01 

Average (2.95+3.52+3.056+2.213+3.765+3.11+3.01)/7 = 3.0891

Contrast to the IOD East/West dipole. Only the K0 (wavenumber=0) shows a reversal in sign. The LTE modulation terms are within 1 radian of each other, indicating much less of a dipole behavior on those terms. It’s possible that these sites don’t span a true dipole, either by its nature or from siting of the measurements.

Cross-validating a large interval span on PDO

using CC

using DTW metric, which pulls out more of the annual/semi-annual signal

adding a 3rd harmonic

Complement of the fitting interval, note the spectral composition maintains the same harmonics, indicating that the structure mapped to is stationary in the sense that the tidal pattern is not changing and the LTE modulation is largely fixed.

This is the resolved tidal forcing, finer than the annual impulse sampling used on the models above.

Below can see the primary 27.5545 lunar anomalistic cycle, mixed with the draconic 27.2122/13.606 cycle to create the 6/3 year modulation and the 206 day perigee-syzygy cycle (or 412 full cycle, as 206 includes antipodal full moon or new moon orientation).

(click on any image to magnify)

Proof for allowed modes of an ideal QBO

/ Paul Pukite (@whut) / 6 Comments

In formal mathematical terms of geometry/topology/homotopy/homology, let’s try proving that a wavenumber=0 cycle of east/west direction inside an equatorial toroidal-shaped waveguide, can only be forced by the Z-component of a (x,y,z) vector where x,y lies in the equatorial plane.

To address this question, let’s dissect the components involved and prove within the constraints of geometry, topology, homotopy, and homology, focusing on valid mathematical principles.

Continue reading →

Mechanical Vibration Sidebanding

/ Paul Pukite (@whut) / 1 Comment

Similar to the LTE modulation in Chapter 12, modulation of a periodic impulse can lead to complex waveforms.

Find behaviors that are periodic impulse trains that are modulated by another periodic process, leading to another waveform with strong subbands.

ChatGPT: The description you provided suggests a phenomenon where a periodic impulse train, a series of regular, sharp signals, is modulated by another periodic process, leading to a composite waveform characterized by pronounced subbands. This can be observed in various fields, including signal processing, telecommunications, and even in natural systems. Here are a few examples where such behaviors might be observed:

  1. Frequency Modulation (FM) in Telecommunications: In FM, the frequency of a carrier wave is varied in accordance with the amplitude of a modulating signal. If the modulating signal is a periodic impulse train, the resulting FM signal will exhibit a spectrum with sidebands at multiples of the modulating frequency. These sidebands can be viewed as subbands in the spectrum, resulting from the modulation process.
  2. Amplitude Modulation (AM) in Radio Broadcasting: In AM, the amplitude of a carrier signal is varied in proportion to that of a modulating signal, which could be a periodic impulse train. This modulation results in a waveform where the carrier frequency is accompanied by sidebands (subbands) at frequencies equal to the sum and difference of the carrier frequency and the modulating frequencies.
  3. Biological Rhythms: In biological systems, certain rhythms or signals can modulate others. For example, the circadian rhythm (a roughly 24-hour cycle) can modulate other physiological processes, leading to patterns that have periodic qualities modulated by the circadian rhythm, resulting in complex waveform behaviors with distinguishable sub-patterns or subbands.
  4. Vibration Analysis in Mechanical Systems: In mechanical systems, a periodic impulse (such as that from a rotating imbalance) can modulate the vibration signatures of other components of the system. This can lead to complex vibration signatures that contain subbands corresponding to the modulating frequencies.
  5. Electroencephalography (EEG) in Neuroscience: In EEG signals, certain brain wave patterns can be modulated by underlying rhythmic processes, such as the sleep-wake cycle. This can result in EEG waveforms where the primary rhythms are modulated, leading to complex patterns with distinct frequency bands.
  6. Optical Communications: In optical fiber communications, a technique known as Wavelength Division Multiplexing (WDM) can be seen as modulating multiple light signals (each a periodic process) onto a single fiber. The combined signal exhibits a spectrum with distinct subbands, each corresponding to one of the modulated signals.

These examples illustrate the diversity of systems and contexts in which periodic impulse trains modulated by another periodic process can lead to complex waveforms with pronounced subbands, each reflecting the interplay between the modulating signals and the base periodic impulses.

The illustration above showcases vibration analysis in mechanical systems, depicting a complex waveform resulting from the modulation of a periodic impulse train by another periodic process. It includes a spectral analysis graph, highlighting the pronounced subbands within the waveform, and a representation of a mechanical system, such as a rotating shaft with an imbalance, along with the corresponding vibration signal being analyzed. This image is designed to be detailed and technical, suitable for educational purposes in the field of engineering.

More ChatGPT: The spectral domain representation below illustrates the result of a periodic impulse train, such as that from a rotating imbalance, being modulated by another vibrating periodic process. In this simulation, the periodic impulse train has a frequency of 50 Hz, and it is modulated by a lower frequency signal at 2 Hz.

The Fourier Transform of the modulated signal reveals pronounced subbands around the base frequency of the impulse train and its harmonics. These subbands are a direct consequence of the modulation process, where the amplitude of the impulse train is modulated by the lower frequency signal, leading to the creation of sidebands in the frequency domain. This graphical representation captures the complex interplay between the periodic impulse train and the modulating signal, resulting in a composite waveform characterized by these pronounced subbands.

This is a good recent citation : He, G., Ding, K., Li, W. and Li, Y. (2017), Frequency response model and mechanism for wind turbine planetary gear train vibration analysis. IET Renewable Power Generation, 11: 425-432. https://doi.org/10.1049/iet-rpg.2016.0236

Frequency components of complicated asymmetric modulation sidebands, existing in the vibration of the healthy planetary gear train, are prone to be erratically diagnosed as fault characteristics, which leads to difficulties in fault diagnosis of wind turbine planetary gearbox. The factors affecting the modulation sideband, i.e. the periodical time-varying transmission path and meshing force direction, are analysed. Considering both the meshing vibrations of the planet–ring and planet–sun gear pairs, a mathematical model was developed to analyse the planetary gear train’s vibration response. Simulation and experiments were conducted, and the mechanism of vibration modulation sidebands was revealed. The modulation sideband is not caused by the meshing vibration itself, but by the testing method that sensors are fixed on the ring gear or gearbox casing. The frequency components and amplitudes of the sidebands are determined by the tooth number of the ring gear and sun gear, the number of planet gears and their initial assembling phases. The asymmetric modulation sideband is mainly caused by the phase difference of the initial planets’ assembling phase.

Abstract: Frequency response model and mechanism for wind turbine planetary gear train vibration analysis

LTE Modulation & Neural Nets via Taylor’s

/ Paul Pukite (@whut) / 4 Comments

Is there a connection between the modulation of Laplace’s Tidal Equation (LTE) solutions and the highly nonlinear fits of neural networks?

“Neural tensor networks have been widely used in a large number of natural language processing tasks such as conversational sentiment analysis, named entity recognition and knowledge base completion. However, the mathematical explanation of neural tensor networks remains a challenging problem, due to the bilinear term. According to Taylor’s theorem, a kth order differentiable function can be approximated by a kth order Taylor polynomial around a given point. Therefore, we provide a mathematical explanation of neural tensor networks and also reveal the inner link between them and feedforward neural networks from the perspective of Taylor’s theorem. In addition, we unify two forms of neural tensor networks into a single framework and present factorization methods to make the neural tensor networks parameter-efficient. Experimental results bring some valuable insights into neural tensor networks.”

https://w.sentic.net/new-perspective-for-neural-tensor-networks.pdf

The connection is via Taylor’s series expansion whereby neural nets try to resolve tight inflection points that occur naturally in the numerical flow of potentially turbulent fluid dynamics.

  • Li, Wei, Luyao Zhu, and Erik Cambria. “Taylor’s theorem: A new perspective for neural tensor networks.” Knowledge-Based Systems 228 (2021): 107258.
  • Zhao, H., Chen, Y., Sun, D., Hu, Y., Liang, K., Mao, Y., … & Shao, H. “TaylorNet: A Taylor-Driven Generic Neural Architecture”. submitted to ICLR 2023

Also wavelets can maneuver tight inflections.

“In this study, the applicability of physics informed neural networks using wavelets as an activation function is discussed to solve non-linear differential equations. One of the prominent equations arising in fluid dynamics namely Blasius viscous flow problem is solved. A linear coupled differential equation, a non-linear coupled differential equation, and partial differential equations are also solved in order to demonstrate the method’s versatility. As the neural network’s optimum design is important and is problem-specific, the influence of some of the key factors on the model’s accuracy is also investigated. To confirm the approach’s efficacy, the outcomes of the suggested method were compared with those of the existing approaches. The suggested method was observed to be both efficient and accurate.”

https://www.nature.com/articles/s41598-023-29806-3

Given the fact that NN results are so difficult to reverse engineer to match a physical understanding could this be a missing link?

Continue reading →