Pulling a signal out of the noise

This is an older chart I made that was part of a thread at the Azimuth Forum.

If you were to take a cursory look at the unfiltered signals on the right, you would never imagine that the correlation on the left would be that good. The ENSO signal is the difference between the Tahiti and Darwin signals, and that correlates with the tidal gauge in Sydney, Australia.  The key is to apply the right kind of filtering to the signals; in this case removing the noisy sub-year fluctuations.


Model of the Quasi-Biennial Oscillation

(update here)

The idea is as described earlier : Find the lunar forcing on the earth and then alias the forcing to a seasonal (yearly) period. This becomes the forcing for the QBO. The rationale is that the faster lunar cycles will not cause the stratospheric winds to change direction, but if these cycles are provoked with a seasonal peak in energy, then a longer-term multiyear period will emerge. This is a well-known mechanism that occurs in many different natural phenomena.

There are two steps to the model. (1) Determine the lunar gravitational potential as a function of time, and (2) plot the potential in units of 1 month or 1 year. The last part is critical, as that emulates the aliasing required to remove the sub-monthly cycles in the lunar forcing.

If one then matches this plot against the QBO time-series, you will find a high correlation coefficient. If the lunar potential is tweaked away from its stationary set of parameters, the fit degrades rapidly. So it becomes essentially a binary match. If it didn’t fit, then the lunar gravitational potential hypothesis would be invalidated. But since it does fit precisely, then it remains a highly plausible model.

Figure 1 shows the mean-square potential of the lunar gravitational pull, also known as the tidal generating potential (applied in the context of predicting tides). On the left, the scale is expanded.

Fig. 1 : Mean-square potential of the lunar gravitational pull (from [1]). On the left, the scale is expanded.

As background, I originally discovered the connection of QBO to the lunar potential via machine learning (Eureqa), see Figure 2.

Fig 2: The original connection from QBO to a tide generating potential was discovered by machine learning — upper right, panel A. The fitted signal was unaliased, squared and shown to align in panel B. Panel C shows the details with the fully unaliased signal at a finer scale.

This fit worked remarkably well considering that it is very difficult to dig out the aliased periods. Letting the machine learning run for a day helped considerably.

Yet it is also useful to reverse the direction of the fitting process. Instead of deducing the model from a sinusoidal decomposition, let us estimate the tidal generating potential as shown in Figure 1 and described by Ray [1]. We then inductively proceed  forward and see how well it fits to the QBO time-series.

Fig 3 : Empirical fit to the tidal generating potential of Figure 1.

This empirical fit uses only three factors — the lunar cycles corresponding to the Draconic month, the Anomalistic month, and the Tropical month. Those are known to a high precision, along with a value for the Tropical year. The composition of these factors is then squared to generate the empirical model of the potential.

If we lay the empirical model on top of Ray’s diagram, it looks like Figure 4.

Fig 4 : Alignment of empirical model with Ray chart. Note the long-term 18.6 year (diurnal) beat period and the shorter 4.425 year (semidiurnal) beat period. Also a rapid bi-annual component.

On the expanded scale, the sub-monthly periods appear, as shown in Figure 5.

Fig 5 : Expanded scale showing the sub-monthly variations in the tidal generating potential.

These higher frequency components disappear when the alias is introduced.

I did not do a complete ephemeris-based empirical model for the tidal generating potential as Ray did, since the basic pattern is fairly easy to deduce from the three lunar cycles.

The final step is to un-square and then alias the tidal generating potential and compare to the QBO time-series. This is shown in Figure 6.

Fig 6: Fit of the unaliased tidal generating potential to the QBO

There is nothing at all complicated about the recipe for fitting the tidal generating potential to the QBO. It is a mechanical process since none of the lunar cycles parameters can be changed.

As a next step I will submit this finding to Physical Review Letters.  From what I have seen in the literature search, there is no consideration of applying a straightforward forcing of the lunar gravitational pull to model QBO.  It appears that most QBO models derive from what Richard Lindzen originally proposed some 40+ years ago — but since many mainstream climate scientists do not consider Lindzen (an AGW denier) very trustworthy or even competent (e.g. a trail of retracted papers and debunked theories), it’s likely that his original model was simply wrong, or at best, incomplete. What the new model does is provide a concise recipe and a highly plausible geophysical context for understanding the origin of QBO.

The further significance of all this is that the same lunar forcing that applies to QBO also likely applies to the phenomena of El Nino and modeling the ENSO time-series, see the ENSO sloshing paper and some more recent work.


[1] R. D. Ray, “Decadal climate variability: Is there a tidal connection?,” Journal of climate, vol. 20, no. 14, pp. 3542–3560, 2007.

QBO is a lunar-solar forced system

This is a followup to the QBOM part 2 post. I showed the machine learning (Eureqa) chart of Figure 1 earlier. The ML exercise mapped the QBOM time series from 1953 to the present time in terms of a set of sinusoidal factors.

Fig. 1: The machine learning fit. The highest complexity solution is shown below.

In the previous post, I focused on the two primary sinusoidal factors and how close they match the aliased Draconic and Synodic (or Tropical as 13 Tropical months = 12 Synodic months) lunar month cycles. The table is reproduced below :

strength  aliased freq   period      in days          actual        % error
78         2.66341033   2.359075219  27.20894362  27.212=draconic  0.011
35         2.29753386   2.734751989  29.53743558  29.531=synodic  -0.021
35         2.29753386   2.734751989  27.32677375  27.322=tropical -0.019

There are a couple more sinusoidal factors involved in the machine learning fit that are not aliased:

strength   frequency    in days       candidate                       % error
30         77.8811187   27.26730124   27.2669=avg(draconic+tropical)  0.0013
26         72.1900786   31.78927972   31.8119=lunar evection cycle   -0.07

These are as well very close to predicted values, if that is what the machine learning is trying to match to. So the temptation is to unalias all the sinusoidal factors and see how well it matches to the harmonic beating of orbital parameters.

Continue reading

Raising the Bar on ENSO Model Validation

I have been using the Azimuth Project Forum as a sounding board for the ENSO Model [1,2,3,4,5,6,7,8].  The audience there is very science-savvy so are not easily convinced of the worth of any particular finding (and whether it is correct in the first place). They also tend to prefer pure math because that can be sufficiently detached from the muddy world of applied physics such that one can avoid being labeled as “right” or “wrong”.  With math one can always come up with a formulation that can exist on its own terms, separate from a practical application.

So trying to convince those folks in the validity of the ENSO model is difficult at best.

Recently the advice has been to do statistical validation on the model. One participant recommended I try an experimental approach

“I still don’t have the spare cycles to address this fully, but given that one of the two terms of an AIC or BIC is the log likelihood and there is not a closed form representation of the likelihood in this case, I’d probably explore either the empirical likelihood work of Art Owen and his students, for one thing as packaged in the emplik R package, or possibly Approximate Bayesian Computation (ABC; see also and here).”

I am not going to go to the trouble of “exploring” some unaccepted statistical validation procedure, when I am having enough of a challenge defending the ENSO model physics. What am I supposed to do — defend someone else’s empirical statistical research in addition to defending my own work? No thanks.

It seems to be always about #RaisingTheBar to see what someone will do to defend their results.

Fair enough.

So I will in this post show an overwhelming piece of evidence that the modeling work is on the right track.

Continue reading