The idea of a digital twin is relatively new in terms of coinage of terms, but the essential idea has been around for decades. In the past, a digital twin was referred to as a virtual simulation of a specific system, encoded via a programming language. In the case of a system that was previously built, the virtual simulation emulated all the behaviors and characteristics of that system, only operated on a computer, with any necessary interactive controls and displays provided on a console, either real or virtual. A widely known example of a VS is that of a flight simulator, which in historical terms was the industrial forerunner to today’s virtual reality. A virtual simulation could also be used during the design of the system, with the finished digital twin providing a blueprint for the actual synthesis of the end-product. This approach has also been practiced for decades, both in the electronics industry via logic synthesis of integrated circuits from a hardware description language and with physical products via 3D printing from CAD models.
Continue readingClimate
Models and Simulation of the climate
Gist Evaluation
The Gist site on GitHub allows you to comment on posts very easily. For example, images of charts can be pasted in the discussion area. Also snippets of code can be added and updated, which is useful for neural net evaluation. The following is a link to an initial Gist area for evaluating LTE models.
Comparison of LTE models applying slight variations of tidal forcing but larger allowance of basin tidal modulation
AMO
NINO4
IOD-East
IOD-West
PDO
Comparison of forcing
Sub(Surface)Stack
I signed up for a SubStack account awhile ago and recently published two articles on this account (SubSurface) in the last week.
- https://pukite.substack.com/p/machine-learning-validates-the-enso
- https://pukite.substack.com/p/machine-learning-validates-the-amo
The SubStack authoring interface has good math equation mark-up, convenient graphics embedding, and an excellent footnoting system. On first pass, it only lacks control over font color.
The articles are focused on applying neural network cross-validation to ENSO and AMO modeling, as suggested previously. I haven’t completely explored the configuration space but one aspect that may becoming clear is the value of wavelet neural networks (WNN) for time-series analysis. The WNN approach seems much more amenable to extracting sinusoidal modulation of the input-to-output mapping — trained on a rather short interval and then cross-validated out-of-band. The Mexican hat wavelet (2nd derivative of a Gaussian) as an activation function in particular locks in quickly to an LTE modulation that took longer to find with the custom search software I have developed at GitHub. I think the reason for the efficiency is that it’s optimizing to a Taylor’s series expansion of the input terms, a classic nonlinear expansion that NN’s excel at.
The following training run using the Mexican hat activation and ADAM optimizer is an eye-opener, as it achieved an admirable fit within a minute of computation.
The GREEN on BLUE is training on NINO4 data over two end-point intervals, with the RED cross-validation over the out-of-band region. The correlation coefficient is 0.34, which is impressive considering the nature of the waveform. Clearly there is similarity.
Moreover, if we compare the model fit to data via the WNN against the LTE harmonics approach, you can also see where the two fare equally poorly. Below in the outer frame is the NINO4 LTE fit with the YELLOW arrow pointing downward at a discrepancy (a peak in the data not resolved in the fit). In comparison the yellow-bordered inset shows the same discrepancy on the WNN training run. So the fingerprints essentially match with no coaching.
The neural net chain is somewhat deep with 6 layers, but I think this is needed to expand to the higher-order terms in the Taylor’s series. In the directed graph below, L01 is the input tidal forcing and L02 is the time axis (with an initial very low weighting).
It also appears temporally stationary across the entire time-span, so that the WNN temporal contribution appears minimal.
In a previous fit the horizontal striations (indicating modulation factor at a forcing level) matched with the LTE model, providing further evidence that the the WNN was mapping to an optimal modulation.
The other Sub(Surface)Stack article is on the AMO, which also reveals promising results. This is a video of the training in action
Intersectionality and No Regrets
The issue is Intersectionality and the solution is a No Regrets Strategy, which means that you don’t regret the choice you make because at least some of the bases are covered.
After filling in the blanks of the Venn diagram:
Quick YT explainer of No Regrets
Interesting to ask GPT-4 what No Regrets means in the context of the earth, i.e. ContextEarth.com
What does the “No Regrets Strategy” mean in terms of mitigating climate change, fossil fuel depletion, and air pollution?
The “No Regrets Strategy” in the context of climate change mitigation, fossil fuel depletion, and air pollution refers to a set of policies, measures, and actions that provide multiple benefits, regardless of the uncertainties associated with the future development of these issues. These measures are seen as advantageous and worthwhile, even if the worst predictions of climate change, fossil fuel depletion, or air pollution do not materialize.
The No Regrets Strategy typically includes:
- Energy efficiency improvements: By improving energy efficiency, we can reduce energy consumption, decrease greenhouse gas emissions, reduce dependence on fossil fuels, and lower air pollution. These improvements have economic benefits, as they lead to cost savings for businesses and consumers.
- Renewable energy development: Investing in renewable energy sources, such as solar, wind, and hydro, helps to diversify energy supply, reduce greenhouse gas emissions, and decrease reliance on fossil fuels. This transition also has the potential to create new jobs and stimulate economic growth.
- Afforestation and reforestation: Planting trees and restoring forests can help to sequester carbon dioxide, enhance biodiversity, and improve air and water quality. This strategy also helps to protect ecosystems and provides valuable recreational spaces for communities.
- Sustainable transportation: Encouraging public transportation, walking, cycling, and the use of electric vehicles reduces fossil fuel consumption, greenhouse gas emissions, and air pollution. These measures can also help to alleviate traffic congestion and improve public health.
- Waste reduction and recycling: Reducing waste generation and promoting recycling conserves resources, reduces greenhouse gas emissions, and lowers air and water pollution. These actions can also create jobs and generate economic benefits.
By implementing these no-regrets measures, societies can mitigate climate change, reduce fossil fuel dependence, and improve air quality, while also realizing economic, social, and environmental benefits.
The bible of the No Regrets Strategy is Mathematical GeoEnergy
LTE Modulation & Neural Nets via Taylor’s
Is there a connection between the modulation of Laplace’s Tidal Equation (LTE) solutions and the highly nonlinear fits of neural networks?
“Neural tensor networks have been widely used in a large number of natural language processing tasks such as conversational sentiment analysis, named entity recognition and knowledge base completion. However, the mathematical explanation of neural tensor networks remains a challenging problem, due to the bilinear term. According to Taylor’s theorem, a kth order differentiable function can be approximated by a kth order Taylor polynomial around a given point. Therefore, we provide a mathematical explanation of neural tensor networks and also reveal the inner link between them and feedforward neural networks from the perspective of Taylor’s theorem. In addition, we unify two forms of neural tensor networks into a single framework and present factorization methods to make the neural tensor networks parameter-efficient. Experimental results bring some valuable insights into neural tensor networks.”
https://w.sentic.net/new-perspective-for-neural-tensor-networks.pdf
The connection is via Taylor’s series expansion whereby neural nets try to resolve tight inflection points that occur naturally in the numerical flow of potentially turbulent fluid dynamics.
- Li, Wei, Luyao Zhu, and Erik Cambria. “Taylor’s theorem: A new perspective for neural tensor networks.” Knowledge-Based Systems 228 (2021): 107258.
- Zhao, H., Chen, Y., Sun, D., Hu, Y., Liang, K., Mao, Y., … & Shao, H. “TaylorNet: A Taylor-Driven Generic Neural Architecture”. submitted to ICLR 2023
Also wavelets can maneuver tight inflections.
“In this study, the applicability of physics informed neural networks using wavelets as an activation function is discussed to solve non-linear differential equations. One of the prominent equations arising in fluid dynamics namely Blasius viscous flow problem is solved. A linear coupled differential equation, a non-linear coupled differential equation, and partial differential equations are also solved in order to demonstrate the method’s versatility. As the neural network’s optimum design is important and is problem-specific, the influence of some of the key factors on the model’s accuracy is also investigated. To confirm the approach’s efficacy, the outcomes of the suggested method were compared with those of the existing approaches. The suggested method was observed to be both efficient and accurate.”
https://www.nature.com/articles/s41598-023-29806-3
- Uddin, Z., Ganga, S., Asthana, R. et al. Wavelets based physics informed neural networks to solve non-linear differential equations. Sci Rep 13, 2882 (2023). https://doi.org/10.1038/s41598-023-29806-3
Given the fact that NN results are so difficult to reverse engineer to match a physical understanding could this be a missing link?
Continue readingLimits of Predictability?
A decade-old research article on modeling equatorial waves includes this introductory passage:
“Nonlinear aspects plays a major role in the understanding of fluid flows. The distinctive fact that in nonlinear problems cause and effect are not proportional opens up the possibility that a small variation in an input quantity causes a considerable change in the response of the system. Often this type of complication causes nonlinear problems to elude exact treatment. “
https://doi.org/10.1029/2012JC007879
From my experience if it is relatively easy to generate a fit to data via a nonlinear model then it also may be easy to diverge from the fit with a small structural perturbation, or to come up with an alternative fit with a different set of parameters. This makes it difficult to establish an iron-clad cross-validation.
This doesn’t mean we don’t keep trying. Applying the dLOD calibration approach to an applied forcing, we can model ENSO via the NINO34 climate index across the available data range (in YELLOW) in the figure below (parameters here)
The lower right box is a modulo-2π reduction of the tidal forcing as an input to the sinusoidal LTE modulation, using the decline rate (per month) as the divisor. Why this works so well per month in contrast to per year (where an annual cycle would make sense) is not clear. It is also fascinating in that this is a form of amplitude aliasing analogous to the frequency aliasing that also applies a modulo-2π folding reduction to the tidal periods less than the Nyquist monthly sampling criteria. There may be a time-amplitude duality or Lagrangian particle-relabeling in operation that has at its central core the trivial solutions of Navier-Stokes or Euler differential equations when all segments of forcing are flat or have a linear slope. Trivial in the sense that when a forcing is flat or has a 1st-order slope, the 2nd derivatives due to divergence in the differential equations vanish (quasi-static). This means that only the discontinuities, which occur concurrently with the annual ENSO predictability barrier, need to be treated carefully (the modulo-2π folding could be a topological Berry phase jump?). Yet, if these transitions are enhanced by metastable interface instabilities as during thermocline turn-over then the differential equation conditions could be transiently relaxed via a vanishing density difference. Much happens during a turn-over, but it doesn’t last long, perhaps indicating a geometric phase. MV Berry also discusses phase changes in the context of amphidromic tidal singularities here.
Suffice to say that the topological properties of reduced dimension volumes and at interfaces remain mysterious. The main takeaway is that a working NINO34-fitted ENSO model is produced, and if not here then somewhere else a machine-learning algorithm will discover it.
The key next step is to apply the same tidal forcing to an AMO model, taking care not to change the tidal factors enough to produce a highly sensitive nonlinear response in the LTE model. So we retain an excluded interval from training (in YELLOW below) and only adjust the LTE parameters for the region surrounding this zone during the fitting process (parameters here).
The cross-validation agreement is breathtakingly good in the excluded (out-of-band) training interval. There is zero cross-correlation between the NINO34 and AMO time-series to begin with so that this is likely revealing the true emergent characteristics of a tidally forced mechanism.
As usual all the introductory work is covered in Mathematical Geoenergy
- https://agupubs.onlinelibrary.wiley.com/doi/10.1002/9781119434351.ch11 (wind)
- https://agupubs.onlinelibrary.wiley.com/doi/10.1002/9781119434351.ch12 (wave)
A community peer-review contributed to a recent QBO article is here and PDF here. The same question applies to QBO as ENSO or AMO: is it possible to predict future behavior? Is the QBO model less sensitive to input since the nonlinear aspect is weaker?
Added several weeks later: This monograph PDF available “Introduction to Geophysical Fluid Dynamics: Physical and Numerical Aspects”. Ignoring higher-order time derivatives is key to solving LTE.
Note the cite to Billy Kessler
Gerstner waves
An exact solution for equatorially trapped waves
Nonlinear aspects plays a major role in the understanding of fluid flows. The distinctive fact that in nonlinear problems cause and effect are not proportional opens up the possibility that a small variation in an input quantity causes a considerable change in the response of the system. Often this type of complication causes nonlinear problems to elude exact treatment. A good illustration of this feature is the fact that there is only one known explicit exact solution of the (nonlinear) governing equations for periodic two-dimensional traveling gravity water waves. This solution was first found in a homogeneous fluid by Gerstner
Adrian Constantin, JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 117, C05029, doi:10.1029/2012JC007879, 2012
These are trochoidal waves
Even within the context of gravity waves explored in the references mentioned above, a vertical wall is not allowable. This drawback is of special relevance in a geophysical context since [cf. Fedorov and Brown, 2009] the Equator works like a natural boundary and equatorially trapped waves, eastward propagating and symmetric about the Equator, are known to exist. By the 1980s, the scientific community came to realize that these waves are one of the key factors in explaining the El Niño phenomenon (see also the discussion in Cushman-Roisin and Beckers [2011]).
modulo-2π and Berry phase
Darwin
It turns out that the Darwin location of the Southern Oscillation Index (SOI) dipole is brilliantly easy to behaviorally model on it’s own.
The input forcing is calibrated to the differential length-of-day (LOD) with a correlation coefficient of 0.9997, and only a few terms are required to capture the standing-wave modes corresponding to the ENSO dipole.
So which curve below is the time-series data of atmospheric pressure at Darwin and which is the Laplace’s Tidal Equation (LTE) model calibrated from dLOD measurements?
- (bottom, red) = ?
- (top, blue) = ??
As a bonus, the couple of years outside of the training interval are extrapolated from the model. This shouldn’t be hard for climate scientists, …. or is it still too difficult?
If that isn’t enough to discriminate between the two, the power spectra of the LTE mapping to model and to data is shown below. This identifies a couple of the lower frequency modulations as strong peaks and a few weaker higher harmonic peaks that sharpen the model’s detail. This shows that the data’s behavior possesses a high amount of order not apparent in the time series.
Poll on Twitter =>
Why isn’t the Tahiti time-series included since that would provide additional signal discrimination via a differential measurement as one should be the complement of the other? It should accentuate the signal and remove noise (and any common-mode behavior) if the Darwin and Tahiti are perfect anti-nodes for all standing-wave modes. However, it appears that only the main ENSO standing-wave mode is balanced in all modes.
In that case, the Darwin set alone works well. Mastodon
Cross-validation
Cross-validation is essentially the ability to predict the characteristics of an unexplored region based on a model of an explored region. The explored region is often used as a training interval to test or validate model applicability on the unexplored interval. If some fraction of the expected characteristics appears in the unexplored region when the model is extrapolated to that interval, some degree of validation is granted to the model.
This is a powerful technique on its own as it is used frequently (and depended on) in machine learning models to eliminate poorly performing trials. But it gains even more importance when new data for validation will take years to collect. In particular, consider the arduous process of collecting fresh data for El Nino Southern Oscillation, which will take decades to generate sufficient statistical significance for validation.
So, what’s necessary in the short term is substantiation of a model’s potential validity. Nothing else will work as a substitute, as controlled experiments are not possible for domains as large as the Earth’s climate. Cross-validation remains the best bet.
Continue readingLunar Eclipse Unremarkable
NdGT has a point — you do see the earth’s shadow moving across the moon, but once covered, a #lunarEclipse just looks like a duller moon (similar “new moons” are also observed like clockwork and thus take the excitement out of it). Yet the alignment of tidal forces does a number on the Earth’s climate that is totally cryptic and thus overlooked. Perhaps old Dr. Neil would find more interesting tying lunar cycles to climate indices such as ENSO and the Indian Ocean Dipole? It’s all based on geophysical fluid dynamics. Oh, and a bonus — discriminate on the variability of IOD and there’s the underlying AGW trend!
BTW, a key to this IOD model fit is to apply dual annual impulses, one for each monsoon season, summer and winter. Whereas, ENSO only has the spring predictability barrier.
Continue reading
GeoEnergy Math 