[mathjax]A short piece that ties together the analysis of ENSO and QBO over the last year.
The premise has been that periodic changes in angular momentum applied to the earth’s rotation is enough of a forcing to steer the behavior of the El Nino Southern Oscillation (ENSO) in the equatorial Pacific ocean and of the Quasi-Biennial Oscillation (QBO) in upper atmospheric winds. Whoever would have you believe that these behaviors could be spontaneously generated is clearly not thinking straight. For every action there is a reaction, and both QBO and ENSO are likely reactions to the same forcing action.
Both this forum (and the Azimuth Project forum) has provided plenty of analysis to show exactly how that comes about, but in retrospect, it’s the machine learning (ML) experiments via Eureqa that has provided the most eye-opening evidence. Robots find what they find and since they are free from the vagaries of human nature, they can’t lie about what they discover.
The first two for QBO have a primary sinusoidal factor that are nearly identical, 2.66341033 and 2.663161 rads/year, and the ENSO has a value 2.64123448 rads/year. If the first two values are averaged and then that is averaged with the ENSO value, the result is 2.65226007 rads/year (the significant figures are as reported by Eureqa). That value is equivalent to a seasonally aliased 2.65226007 +13$$cdot$$2$$pi$$ rads/year, which is a period of 27.21195913 days — while the Draconic lunar month is 27.21222082 days. That’s an error of 0.00096%.
So the primary ENSO forcing period as determined by ML was a tiny bit shorter than the Draconic and the primary QBO forcing period was a wee bit longer than the Draconic period. Given that is partly due to noise in the fit, it’s reassuring to see that the average would get even closer to a plausible forcing value.
The entire premise of the lunar forcing driving both QBO and ENSO hinges on the precision of the modeled values; as the cycles of a lunisolar model can quickly get out of sync with the data unless enough precision is available to span 60 to 100 years.
Recall again these words by the professional contrarian scientist Richard Lindzen:
” 5. Lunar semidiurnal tide : One rationale for studying tides is that they are motion systems for which we know the periods perfectly, and the forcing almost as well (this is certainly the case for gravitational tides). Thus, it is relatively easy to isolate tidal phenomena in the data, to calculate tidal responses in the atmosphere, and to compare the two. Briefly, conditions for comparing theory and observation are relatively ideal. Moreover, if theory is incapable of explaining observations for such a simple system, we may plausibly be concerned with our ability to explain more complicated systems. Lunar tides are especially well suited to such studies since it is unlikely that lunar periods could be produced by anything other than the lunar tidal potential. The only drawback in observing lunar tidal phenomena in the atmosphere is their weak amplitude, but with sufficiently long records this problem can be overcome [viz. discussion in Chapman and Lindaen (1970)] at least in analyses of the surface pressure oscillation. ” — from Lindzen, Richard S., and Siu-Shung Hong. “Effects of mean winds and horizontal temperature gradients on solar and lunar semidiurnal tides in the atmosphere.” Journal of the Atmospheric Sciences 31.5 (1974): 1421-1446.
That bolded part is the monetary payoff. If Lindzen, who is known as the father of QBO theory, asserts that if measured periods aligning with lunar periods is a sufficient comparison, then he would be forced into agreeing with this current analysis. Nothing else will come close to the precision required.
And the payoff turns into the daily double as it also works for explaining ENSO. The combination of parsimony and plausibility is hard to argue with.
GeoEnergy Math 
Another way to put it is the ENSO model is 0.012% too high and the QBO model is 0.014% too low when compared to the Draconic period. Those are very close by themselves, but the average is as spot on as you can get.
There’s an amplification effect that occurs with aliasing which tends to exaggerate small differences. In some sense it acts like a great discriminator, but it also makes analysis less straightforward. I will likely discuss this in the next article.
LikeLike
Hi paul,
There are cases where you claim there are no degrees of freedom in your model. You are proposing a physical model where you expect certain parameters will have a frequency of 2.65 rads per year. Even a simple linear regression has N-2 degrees of freedom where N is the number of data points, for more complex multiple linear regression (ordinary least squares) with many independent variables (k of them) the degrees of freedom are N-k-1.
The problem with using Eureka is that the program spits out a relationship and then we try to make physical sense of the result.
This as a little different from traditional methods where a theory was proposed and then data was collected to confirm the theory.
Possibly if you think of 2.65 rads/year being your hypothesis and then you can use statistics to confirm your hypothesis.
There are a number of issues with time series data and there are statistical methods for testing the robustness of your result.
LikeLike
Dennis,
I only said that applies to QBO. In that model, I have no freedom to select the values of the periods. Of course there is freedom to select the scaling of the parameters. But that is no different than what is done to predict the occurrence and strength of tides.
All the fine detail lines up and that will not happen if the periods are wrong. In the figure below, the top is used as a training to generate the fit below.
LikeLike
Hi Paul,
Ok, got it. Possibly you could reduce the number of different periods that you use. For example the default for your QBO model calls for 19 different sinusoidal functions in entroplet, when using only the 1953 to 1980 QBO data, using only 3 parameters gets a correlation coefficient almost as good.
I think in some sense this shows that only a couple of these parameters are significant. Where can I find the QBO data?
I can play with some simple regressions to see if I can replicate your work. I will also need to read up in the aliasing to get a clearer idea of how this is done. I will skip taking the second derivative for now.
LikeLike
Found QBO data at
http://www.esrl.noaa.gov/psd/data/correlation/qbo.data
LikeLike
Dennis, Yes, that’s the right QBO data. I use the 70, 30,15 hPa sets but other altitudes are there as well.
Granted 19 parameters is too much, but I am exploring the intricate detail in the curves as well. All the details in the shoulders of the peaks are the result of the harmonics of the fundamental period. These are aliased as well.
The analysis problem of course is that those fine details are not strong contributors to the correl coeff, so you have to look for incremental improvements and then observe to see if those details are being replicated in both the model and the data.
Did you know that tidal analysis programs use hundreds or even thousands of these coefficients? No one seems to complain that qualifies as over-fitting. I am trying to achieve that kind of agreement. This is still exploratory so I am on the lookout for something I may have missed.
LikeLike
“I can play with some simple regressions to see if I can replicate your work. I will also need to read up in the aliasing to get a clearer idea of how this is done. I will skip taking the second derivative for now.”
Good luck! The aliasing math is in this page
http://contextearth.com/2015/11/17/the-math-of-seasonal-aliasing/
Taking the 2nd derivative is only important if you want to track the fine detail — i.e the periods that are at 0.71 years, etc. What it does is tend to magnify those higher frequencies and suppress the fundamental. Be careful in that what looks like high frequency noise not may not be. Remember that lunar tides are at periods of <0.08 year and so can add potentially lots of detail depending on the seasonal effects of the aliasing.
This is an interesting project in that the more you peel the onion, the more details get revealed but still seem to make sense.
LikeLike
Found data at
https://climatedataguide.ucar.edu/climate-data/qbo-quasi-biennial-oscillation
Also read following paper:
Click to access baldwin0101.pdf
It is not clear what the connection between moon periods and QBO would be?
Can you explain a proposed physical connection?
The connection to ENSO I can see, but how QBO fits in is unclear to me.
Finding a connection in the data by itself is not enough, a proposed mechanism is needed.
LikeLike
Dennis,
The connection is very simple, and even Richard Lindzen proposed it years ago.
The underlying premise is that no physical mechanism such as QBO will spontaneously oscillate with such precision unless there is a forcing behind it. A resonant system might fit the bill but these usually damp out. So the QBO must be a forced oscillation.
So what candidates for a periodic forcing are there? The pickings are pretty slim. There is the seasonal signal and the lunar signals (plus planetary signals, Jupiter=11.86 year beat). There is also the Chandler wobble and perhaps quasi-periodic sunspot data (the latter not regular enough though).
I already quoted Lindzen above about the applicability of Lunar forcing, but he evidently did not consider seasonally aliased lunar forcing. So that upper atmospheric air that is seasonally energized is pulled by gravitational forces that start the wind in motion (i.e. gravity waves). And of course this can be assisted by the ocean being forced by the same lunar factor, and these gravity waves can reinforce the effect as they are convected upward.
LikeLike
Ok, That makes sense.
So the periodicity that appears in the temperature data (that is not related to CO2, aerosols, length of day, solar output, or the southern oscillation index.) ould be explained by these gravitational forces from the Sun. moon and Jupiter?
It seems the variation in both length of day and the southern oscillation index might also be explained by these gravitational forces. If so the CSALT model would probably be better without the S and L ( the CAT model as it were) and then include a 60 year cycle (in place of length of day) and some of the other periods that Eureqa pulled out in your ENSO and QBO runs, maybe try to limit this to the 4 or 5 highest statistically significant periods. This is something I can do, but I need to nail down the seasonal aliasing a little better first.
LikeLike
Regarding the possibility of ML overfitting, I think this is a valid concern. I might suggest that you apply the Akaike Information Criterion (AIC) to the Eureqa models as they are generated, and cut it off when you’ve begun to go backwards. AIC is available in the stats package in R.
LikeLike
Keith, An option for selecting an Akaike criteria is already part of Eureqa. In fact that is what Eureqa is essentially all about as it is always forcing one to make a decision regarding goodness of fit versus complexity with their Pareto view.
I think careful over-fitting is a way of life during the exploratory phase. But even later on, it may not be a big deal. I keep reading about the fact that tidal analysis programs can include 100’s or even 1000’s of Fourier series terms. Nobody uses Akaike on this because the physics says to use all those terms. In other words, it’s not a statistical experiment anymore.
Consider a five factor ENSO fit that only trains on a 15 year interval starting in 1942. But it gets the part shaded almost dead on.
There is something very deterministic about the ENSO phenomenon that has been impossible to crack. The only way that this will get figured out is by pushing the boundaries.
LikeLike
I’m not at all surprised by the longer period for Enso as there are continents blocking a straightforward response of the oceans. That the QBO is faster than necessary for a free-flowing fluid could be a response to the ENSO irregularities? Could be very intersting to just force the values you’ve presented here to a GCM and see what comes about? Could be the desired improvement on enso predictability could show up even in the smaller models.
LikeLike
Good thoughts. I’m not sure how important the geography is. Once a standing wave is set, such as is the case for the SOI dipole, the spatial dependence is separated out of the wave equation. But it does have some effect on the sensitivity to different forcing frequencies. If that’s what you are suggesting, I may tend to agree.
LikeLike
Umm, this is above my pay grade so I’m possibly using wrong words. As an El Nino hitting the South America coastline has atmospheric effects, could these influence the qbo? Does QBO behave oddly in El Nino years? After reading your and Grumbines work, and passing much of it as above my pay grade, I’ve come to think El Ninos as some sort of release of tension between the atmospheric and ocean response to Chandler wobble. On a water planet these would be totally in sync, I think, but the presence of continents would force the oceanic rythm to give the atmospheric response a push at (possibly) regular intervals. Both would still be initially driven by celestial mechanics, I guess.
One phase shift on a rapidly warming planet won’t a bad model make (attempting humor), weather and ocean currents are still pretty chaotic. Say f.e. Agulhas retroflexion has failed dramatically letting the heat escape to Atlantic. And possibly there are other reasons for it, no volcanos back then though.
LikeLike
Thanks for the comment. I intuitively think that the events known as Sudden Stratospheric Warmings (SSW) may have more to do with deviations of the QBO.
Since the QBO is very structured via the aliased draconic tide, these SSW events may be convenient points to look for deviations.
I would rate my confidence highest in models for QBO and the Chandler wobble, and the ENSO is not quite there yet but making progress.
LikeLike
Ah, those events… Is this a fairly accurate representation of these? http://neven1.typepad.com/blog/2013/04/sudden-stratospheric-warmings-causes-effects.html
The location identified in the article that would be involved in the making of an ssw event isn’t directly involved with enso variations though, but rather with monsoon. Uh, some complex stuff. The record on ssw is likely too short to draw any connections between enso, ssw and qbo. But still sounding like a good theory/hypothesis to me.
LikeLike
An SSW event provides an energy release point that could impact jet stream flow. Yup that is only a hypothesis that an SSW event will impact QBO temporarily.
Hopefully the writer of that post on SSW, RG can comment. I sent him a DM and he said he would like to look into the QBO model.
LikeLike