# NAO

The challenge of validating the models of climate oscillations such as ENSO and QBO, rests primarily in our inability to perform controlled experiments. Because of this shortcoming, we can either do (1) predictions of future behavior and validate via the wait-and-see process, or (2) creatively apply techniques such as cross-validation on currently available data. The first is a non-starter because it’s obviously pointless to wait decades for validation results to confirm a model, when it’s entirely possible to do something today via the second approach.

There are a variety of ways to perform model cross-validation on measured data.

In its original and conventional formulation, cross-validation works by checking one interval of time-series against another, typically by training on one interval and then validating on an orthogonal interval.

Another way to cross-validate is to compare two sets of time-series data collected on behaviors that are potentially related. For example, in the case of ocean tidal data that can be collected and compared across spatially separated geographic regions, the sea-level-height (SLH) time-series data will not necessarily be correlated, but the underlying lunar and solar forcing factors will be closely aligned give or take a phase factor. This is intuitively understandable since the two locations share a common-mode signal forcing due to the gravitational pull of the moon and sun, with the differences in response due to the geographic location and local spatial topology and boundary conditions. For tides, this is a consensus understanding and tidal prediction algorithms have stood the test of time.

In the previous post, cross-validation on distinct data sets was evaluated assuming common-mode lunisolar forcing. One cross-validation was done between the ENSO time-series and the AMO time-series. Another cross-validation was performed for ENSO against PDO. The underlying common-mode lunisolar forcings were highly correlated as shown in the featured figure.  The LTE spatial wave-number weightings were the primary discriminator for the model fit. This model is described in detail in the book Mathematical GeoEnergy to be published at the end of the year by Wiley.

Another common-mode cross-validation possible is between ENSO and QBO, but in this case it is primarily in the Draconic nodal lunar factor — the cyclic forcing that appears to govern the regular oscillations of QBO.  Below is the Draconic constituent comparison for QBO and the ENSO.

The QBO and ENSO models only show a common-mode correlated response with respect to the Draconic forcing. The Draconic forcing drives the quasi-periodicity of the QBO cycles, as can be seen in the lower right panel, with a small training window.

This cross-correlation technique can be extended to what appears to be an extremely erratic measure, the North Atlantic Oscillation (NAO).

Like the SOI measure for ENSO, the NAO is originally derived from a pressure dipole measured at two separate locations — but in this case north of the equator.  From the high-frequency of the oscillations, a good assumption is that the spatial wavenumber factors are much higher than is required to fit ENSO. And that was the case as evidenced by the figure below.

ENSO vs NAO cross-validation

Both SOI and NAO are noisy time-series with the NAO appearing very noisy, yet the lunisolar constituent forcings are highly synchronized as shown by correlations in the lower pane. In particular, summing the Anomalistic and Solar constituent factors together improves the correlation markedly, which is because each of those has influence on the other via the lunar-solar mutual gravitational attraction. The iterative fitting process adjusts each of the factors independently, yet the net result compensates the counteracting amplitudes so the net common-mode factor is essentially the same for ENSO and NAO (see lower-right correlation labelled Anomalistic+Solar).

Since the NAO has high-frequency components, we can also perform a conventional cross-validation across orthogonal intervals. The validation interval below is for the years between 1960 and 1990, and even though the training intervals were aggressively over-fit, the correlation between the model and data is still visible in those 30 years.

NAO model fit with validation spanning 1960 to 1990

Over the course of time spent modeling ENSO, the effort that went into fitting to NAO was a fraction of the original time. This is largely due to the fact that the temporal lunisolar forcing only needed to be tweaked to match other climate indices, and the iteration over the topological spatial factors quickly converges.

Many more cross-validation techniques are available for NAO, since there are different flavors of NAO indices available corresponding to different Atlantic locations, and spanning back to the 1800’s.

# ENSO, AMO, PDO and common-mode mechanisms

The basis of the ENSO model is the forcing derived from the long-period cyclic lunisolar gravitational pull of the moon and sun. There is some thought that ENSO shows teleconnections to other oceanic behaviors. The primary oceanic dipoles are ENSO and AMO for the Pacific and Atlantic. There is also the PDO for the mid-northern-latitude of the Pacific, which has a pattern distinct from ENSO. So the question is: Are these connected through interactions or do they possibly share a common-mode mechanism through the same lunisolar forcing mechanism?

Based on tidal behaviors, it is known that the gravitational pull varies geographically, so it would be understandable that ENSO, AMO, and PDO would demonstrate distinct time-series signatures. In checking this, you will find that the correlation coefficient between any two of these series is essentially zero, regardless of applied leads or lags. Yet the underlying component factors (the lunar Draconic, lunar Anomalistic, and solar modified terms) may potentially emerge with only slight variations in shape, with differences only in relative amplitude. This is straightforward to test by fitting the basic ENSO model to AMO and PDO by allowing the parameters to vary.

The following figure is the result of fitting the model to ENSO, AMO, and PDO and then comparing the constituent factors.

First, note that the same parametric model fits each of the time series arguably well. The Draconic factor underling both the ENSO and AMO model is almost perfectly aligned, indicated by the red starred graph, with excursions showing a CC above 0.99. All of the rest of the CC’s in fact are above 0.6.

The upshot of this analysis is two-fold. First to consider how difficult it is to fit any one of these time series to a minimal set of periodically-forced signals. Secondly that the underlying signals are not that different in character, only that the combination in terms of a Laplace’s tidal equation weighting are what couples them together via a common-mode mechanism. Thus, the teleconnection between these oceanic indices is likely an underlying common lunisolar tidal forcing, just as one would suspect from conventional tidal analysis.

# ENSO model verification via Fourier analysis infill

Because the ENSO model generates precise temporal harmonics via a non-linear solution to Laplace’s Tidal Equations, it may in practice be trivially easy to verify. By only using higher-frequency harmonics (T<1.25y) during spectral training (with a small window of low-frequency signal to stabilize the solution, T>11y), the model essentially fills in the missing bulk of the signal frequency spectrum, 1.25y < T < 11y.  This is shown below in Figure 1.

Fig. 1: Bottom panel of amplitude ENSO SOI spectra shows the training windows.  A primarily low-amplitude spectral signal is used to fit the model (using least-squares on the error signal). Upper spectra shows the expanded view of the out-of-band fit. This rich spectra is all due to the non-linear harmonic solution of the ENSO Laplace’s Tidal Equation solution.

This agreement is statistically unlikely (nee impossible) to occur unless the out-of-band signal had knowledge of the fundamental harmonics (i.e. the highest amplitude terms in the meat of the spectra) that are contributing to the higher harmonics.

Figure 2 is the underlying temporal fit. Although not as good a fit as what we can achieve using more of the primary Fourier terms, it is still striking.

Fig. 2: Temporal model fit using only Fourier frequency terms shorter than 1.25 years and longer than 11 years. The correlation coefficient is 0.7 here

The consensus claim is that ENSO is a chaotic process with no long-term coherence. Yet, this shows excellent agreement with a forced lunisolar model showing very long-term coherence.   An issue to raise is: why has the obvious deterministic forcing model been abandoned as a plausible physical mechanism so long ago?

# Amateur Hour

Three science philosophy papers and two recent findings seemed to fit together for this post.

Divergent Perspectives on Expert Disagreement: Preliminary Evidence from Climate Science, Climate Policy, Astrophysics, and Public Opinion :
James R. Beebe (University at Buffalo), Maria Baghramian (University College Dublin), Luke Drury (Dublin Institute for Advanced Studies), Finnur Dellsén (Inland Norway University of Applied Sciences)

“We found that, as compared to educated non-experts, climate experts believe (i) that there is less disagreement within climate science about climate change, (ii) that more of the disagreement that does exist concerns public policy questions rather than the science itself, (iii) that methodological factors play less of a role in generating existing disagreement among experts about climate science, (iv) that fewer personal and institutional biases influence the nature and direction of climate science research, (v) that there is more agreement among scientists about which methods or theoretical perspectives should be used to examine and explain the relevant phenomena, (vi) that disagreements about climate change should not lead people to conclude that the scientific methods being employed today are unreliable or incapable of revealing the truth, and (vii) that climate science is more settled than ideological pundits would have us believe and settled enough to base public policy on it. In addition, we observed that the uniquely American political context predicted participants’ judgments about many of these factors. We also found that, commensurate with the greater inherent uncertainty and data lacunae in their field, astrophysicists working on cosmic rays were generally more willing to acknowledge expert disagreement, more open to the idea that a set of data can have multiple valid interpretations, and generally less quick to dismiss someone articulating a non-standard view as non-expert, than climate scientists. ”

Any scientific discipline that lends itself to verification via controlled experiments is more open to non-standard views. It really is amazing how fast a new idea can be accepted when an experiment can be repeated by others. In climate science, no such control is possible, as the verification process can take years, while the non-standard views continue to proliferate. No wonder climate scientists get hardened to outsider views, as they have no way of immediately dismissing alternate interpretations.

On the other hand, astrophysics has a long history of being open to outsider opinion, with the amateur astronomer often given equal attention to a new finding. Two very recent cases come to mind.

These are two very concrete and objective findings that can be verified easily by others. So kudos to these two amateur sleuths for their persistence.

However, it’s not so easy to verify that one is achieving verifiable progress in areas such as hydrology, as an ongoing debate series reveals that “Hydrology is a hard subject” (with a not to Feynman). The following paper tries to argue for a more open interpretation to the scientific process.

Every scientist borrows techniques from each column, but it is certainly true that the lack of being able to devise controlled experiments in climatology and hydrology places those researchers at a disadvantage compared to the lab-based researchers, or to incremental event-based discoveries (such as with astronomy).  Baker is almost suggesting  that a  better qualitative measure of holistic progress (uberty?) should take the place of a complete quantitative understanding.

And even if one could make sense of a hypothesized behavior, one still has to navigate the landmines of prediction versus a probabilistic forecast.

Stigma in science: the case of earthquake prediction
Helene Joffe, Tiziana Rossetto, Caroline Bradley, and Cliodhna O’Connor

Earthquake prediction is at a cross-roads, with a rather obvious debate going on at the USGS (and spilling over to other research groups) on whether lunisolar gravitational forcing can provide a significant trigger to the timing of an earthquake. Again, because of a lack of controlled experimentation, the argument can only take place in statistical  terms, and will take time and more observational data to resolve.

Yet, bottom-line, the question remains, who owns disciplines such as astrophysics, climate science, hydrology, and seismology?  To take as an example, the entire data set for the ENSO climate behavior can be reduced to the monthly time series of barometric pressure measured at only two locations, one in Tahiti and one in Darwin. This is as open to public interpretation as the amateur astronomers that scan the night-time skies for fresh discoveries. The three papers all say that experts disagree on how to solve or model the big geophysics problems.  I’d suggest that we allow the educated non-experts take a crack and listen to what they have to say.

click to enlarge

# GC41B-1022: Biennial-Aligned Lunisolar-Forcing of ENSO: Implications for Simplified Climate Models

In the last month, two of the great citizen scientists that I will be forever personally grateful for have passed away. If anyone has followed climate science discussions on blogs and social media, you probably have seen their contributions.

Keith Pickering was an expert on computer science, astrophysics, energy, and history from my neck of the woods in Minnesota. He helped me so much in working out orbital calculations when I was first looking at lunar correlations. He provided source code that he developed and it was a great help to get up to speed. He was always there to tweet any progress made. Thanks Keith

Kevin O’Neill was a metrologist and an analysis whiz from Wisconsin. In the weeks before he passed, he told me that he had extra free time to help out with ENSO analysis. He wanted to use his remaining time to help out with the solver computations. I could not believe the effort he put in to his spreadsheet, and it really motivated me to spending more time in validating the model. He was up all the time working on it because he was unable to lay down. Kevin was also there to promote the research on other blogs, right to the end. Thanks Kevin.

There really aren’t too many people willing to spend time working analysis on a scientific forum, and these two exemplified what it takes to really contribute to the advancement of ideas. Like us, they were not climate science insiders and so will only get credit if we remember them.

# The ENSO Forcing Potential – Cheaper, Faster, and Better

Following up on the last post on the ENSO forcing, this note elaborates on the math.  The tidal gravitational forcing function used follows an inverse power-law dependence, where a(t) is the anomalistic lunar distance and d(t) is the draconic or nodal perturbation to the distance.

$F(t) propto frac{1}{(R_0 + a(t) + d(t))^2}'$

Note the prime indicating that the forcing applied is the derivative of the conventional inverse squared Newtonian attraction. This generates an inverse cubic formulation corresponding to the consensus analysis describing a differential tidal force:

$F(t) propto -frac{a'(t)+d'(t)}{(R_0 + a(t) + d(t))^3}$

For a combination of monthly and fortnightly sinusoidal terms for a(t) and d(t) (suitably modified for nonlinear nodal and perigean corrections due to the synodic/tropical cycle)   the search routine rapidly converges to an optimal ENSO fit.  It does this more quickly than the harmonic analysis, which requires at least double the unknowns for the additional higher-order factors needed to capture the tidally forced response waveform. One of the keys is to collect the chain rule terms a'(t) and d'(t) in the numerator; without these, the necessary mixed terms which multiply the anomalistic and draconic signals do not emerge strongly.

As before, a strictly biennial modulation needs to be applied to this forcing to capture the measured ENSO dynamics — this is a period-doubling pattern observed in hydrodynamic systems with a strong fundamental (in this case annual) and is climatologically explained by a persistent year-to-year regenerative feedback in the SLP and SST anomalies.

Here is the model fit for training from 1880-1980, with the extrapolated test region post-1980 showing a good correlation.

The geophysics is now canonically formulated, providing (1) a simpler and more concise expression, leading to (2) a more efficient computational solution, (3) less possibility of over-fitting, and (4) ultimately generating a much better correlation. Alternatively, stated in modeling terms, the resultant information metric is improved by reducing the complexity and improving the correlation — the vaunted  cheaper, faster, and better solution. Or, in other words: get the physics right, and all else follows.

# Approximating the ENSO Forcing Potential

From the last post, we tried to estimate the lunar tidal forcing potential from the fitted harmonics of the ENSO model. Two observations resulted from that exercise: (1) the possibility of over-fitting to the expanded Taylor series, and (2) the potential of fitting to the ENSO data directly from the inverse power law.

The Taylor’s series of the forcing potential is a power-law polynomial corresponding to the lunar harmonic terms. The chief characteristic of the polynomial is the alternating sign for each successive power (see here), which has implications for convergence under certain regimes. What happens with the alternating sign is that each of the added harmonics will highly compensate the previous underlying harmonics, giving the impression that pulling one signal out will scramble the fit. This is conceptually no different than eliminating any one term from a sine or cosine Taylor’s series, which are also all compensating with alternating sign.

The specific conditions that we need to be concerned with respect to series convergence is when r (perturbations to the lunar orbit) is a substantial fraction of R (distance from earth to moon) :

$F(r) = frac{1}{(R+r)^3}$

Because we need to keep those terms for high precision modeling, we also need to be wary of possible over-fitting of these terms — as the solver does not realize that the values for those terms have the constraint that they derive from the original Taylor’s series. It’s not really a problem for conventional tidal analysis, as the signals are so clean, but for the noisy ENSO time-series, this is an issue.

Of course the solution to this predicament is not to do the Taylor series harmonic fitting at all, but leave it in the form of the inverse power law. That makes a lot of sense — and the only reason for not doing this until now is probably due to the inertia of conventional wisdom, in that it wasn’t necessary for tidal analysis where harmonics work adequately.

So this alternate and more fundamental formulation is what we show here.

# Reverse Engineering the Moon’s Orbit from ENSO Behavior

[mathjax]With an ideal tidal analysis, one should be able to apply the gravitational forcing of the lunar orbit1 and use that as input to solve Laplace’s tidal equations. This would generate tidal heights directly. But due to aleatory uncertainty with respect to other factors, it becomes much more practical to perform a harmonic analysis on the constituent tidal frequencies. This essentially allows an empirical fit to measured tidal heights over a training interval, which is then used to extrapolate the behavior over other intervals.  This works very well for conventional tidal analysis.

For ENSO, we need to make the same decision: Do we attempt to work the detailed lunar forcing into the formulation or do we resort to an empirical bottoms-up harmonic analysis? What we have being do so far is a variation of a harmonic analysis that we verified here. This is an expansion of the lunar long-period tidal periods into their harmonic factors. So that works well. But could a geophysical model work too?

# Interface-Inflection Geophysics

This paper that a couple of people alerted me to is likely one of the most radical research findings that has been published in the climate science field for quite a while:

Topological origin of equatorial waves
Delplace, Pierre, J. B. Marston, and Antoine Venaille. Science (2017): eaan8819.

An earlier version on ARXIV was titled Topological Origin of Geophysical Waves, which is less targeted to the equator.

The scientific press releases are all interesting

What the science writers make of the research is clearly subjective and filtered through what they understand.

# CW

Now that we have strong evidence that AMO and PDO follows the biennial modulated lunar forcing found for ENSO, we can try modeling the Chandler wobble in detail. Most geophysicists argue that the Chandler wobble frequency is a resonant mode with a high-Q factor, and that random perturbations drive the wobble into its characteristic oscillation. This then interferes against the yearly wobble, generating the CW beat pattern.

But it has really not been clearly established that the measure CW period is a resonant frequency.  I have a detailed rationale for a lunar forcing of CW in this post, and Robert Grumbine of NASA has a related view here.

The key to applying a lunar forcing is to multiply it by a extremely regular seasonal pulse, which introduces enough of a non-linearity to create a physically-aliased modulation of the lunar monthly signal (similar as what is done for ENSO, QBO, AMO, and PDO).