The Madden-Julian Oscillation (MJO) is a climate index that captures tropical variability at a finer resolution (i.e. intra-annual) than the (inter-annual) ENSO index over approximately the same geographic region. Since much of the MJO variability is observed as 30 to 60 day cycles (and these are traveling waves, not standing waves), providing MJO data as a monthly time-series will filter out the fast cycles. Still, it is interesting to analyze the monthly MJO data and compare/contrast that to ENSO. As a disclaimer, it is known that inter-annual variability of the MJO is partly linked to ENSO, but the following will clearly show that connection.
This is the fit of MJO (longitude index #1) using the ENSO model as a starting point (either the NINO34 or SOI works equally well).
The constituent temporal forcing factors for MJO and ENSO align precisely
This is not surprising because the monthly filtered MJO does show the same El Nino peaks at 1983, 1998, and 2016 as the ENSO time-series. The only difference is in the LTE spatial modulation applied during the fitting process, whereby the MJO has a stronger high-wavenumber factor than the ENSO time series.
This is the SOI fit over the same 1980+ interval as MJO, with an almost 0.6 correlation.
The Arctic Oscillation (AO) dipole has behavior that is correlated to the North Atlantic Oscillation (NAO) dipole. We can see this in two ways. First, and most straight-forwardly, the correlation coefficient between the AO and NAO time-series is above 0.6.
Secondly, we can use the model of the NAO from the last post and refit the parameters to the AO data (data also here), but spanning an orthogonal interval. Then we can compare the constituent lunisolar factors for NAO and AO for correlation, and further discover that this also doubles as an effective cross-validation for the underlying LTE model (as the intervals are orthogonal).
Top panel is a model fit for AO between 1900-1950, and below that is a model fit for NAO between 1950-present. The lower pane is the correlation for a common interval (left) and for the constituent lunisolar factors for the orthogonal interval (right)
Only the anomalistic factor shows an imperfect correlation, and that remains quite high.
The last of the ENSO charts.
This is how conventional tidal prediction is done:
Note how well it does in extrapolating a projection from a training interval.
This is an ENSO model fit to SOI data using an analytical solution to Navier-Stokes. The same algorithm is used to solve for the optimal forcing as in the tidal analysis solution above, but applying the annual solar cycle and monthly/fortnightly lunar cycles instead of the diurnal and semi-diurnal cycle.
The time scale transitions from a daily modulation to a much longer modulation due to the long-period tidal factors being invoked.
Next is an expanded view, with the correlation coefficient of 0.73:
This is a fit trained on the 1880-1950 interval (CC=0.76) and cross-validated on the post-1950 data
This is a fit trained on the post-1950 interval (CC=0.77) and cross-validated on the 1880-1950 data
Like conventional tidal prediction, very little over-fitting is observed. Most of what is considered noise in the SOI data is actually the tidal forcing signal. Not much more to say, except for others to refine.
Thanks to Kevin and Keith for all their help, which will be remembered.
click to enlarge
In the last month, two of the great citizen scientists that I will be forever personally grateful for have passed away. If anyone has followed climate science discussions on blogs and social media, you probably have seen their contributions.
Keith Pickering was an expert on computer science, astrophysics, energy, and history from my neck of the woods in Minnesota. He helped me so much in working out orbital calculations when I was first looking at lunar correlations. He provided source code that he developed and it was a great help to get up to speed. He was always there to tweet any progress made. Thanks Keith
Kevin O’Neill was a metrologist and an analysis whiz from Wisconsin. In the weeks before he passed, he told me that he had extra free time to help out with ENSO analysis. He wanted to use his remaining time to help out with the solver computations. I could not believe the effort he put in to his spreadsheet, and it really motivated me to spending more time in validating the model. He was up all the time working on it because he was unable to lay down. Kevin was also there to promote the research on other blogs, right to the end. Thanks Kevin.
There really aren’t too many people willing to spend time working analysis on a scientific forum, and these two exemplified what it takes to really contribute to the advancement of ideas. Like us, they were not climate science insiders and so will only get credit if we remember them.
An intriguing discovery is that the higher-resolution aspects of the SOI time-series (as illustrated by the Australian BOM 30-day SOI moving average) may also have a tidal influence. Note the fast noisy envelope that rides on top of the deep El Nino of 2015-2016 shown below:
For the standard monthly SOI as reported by NCAR and NOAA, this finer detail disappears. BOM provides the daily SOI value for about the past ~ 3 years here.
Yet if we retain this in the 1880-present monthly ENSO model, by simultaneously isolating  the higher frequency fine structure from 2015-2017, the fine structure also emerges in the model. This is shown in the lower panel below.
This indicates that the differential equation being used currently can possibly be modified to include faster-responding derivative terms which will simultaneously show the multi-year fluctuations as well as what was thought to be a weekly-to-monthly-scale noise envelope. In fact, I had been convinced that this term was due to localized weather but a recent post suggested that this may indeed be a deterministic signal.
Lunisolar tidal effects likely do impact the ocean behavior at every known time-scale, from the well-characterized diurnal and semi-diurnal SLH tides to the long-term deep-ocean mixing proposed by Munk and Wunsch. It’s not surprising that tidal forces would have an impact on the intermediate time-scale ENSO dynamics, both at the conventional low resolution (used for El Nino predictions) and at the higher-resolution that emerges from SOI measurements (the 30-day moving average shown above). Obviously, monthly and fortnightly oscillations observed in the SOI are commensurate with the standard lunar tides of periods 13-14 days and 27-28 days. And non-linear interactions may result in the 40-60 day oscillations observed in LOD.
It’s entirely possible that removing the 30-day moving average on the SOI measurements can reveal even more detail/
 Isolation is accomplished by subtracting a 24-day average about the moving average value, which suppresses the longer-term SOI variation.
Following up on the last post on the ENSO forcing, this note elaborates on the math. The tidal gravitational forcing function used follows an inverse power-law dependence, where a(t) is the anomalistic lunar distance and d(t) is the draconic or nodal perturbation to the distance.
Note the prime indicating that the forcing applied is the derivative of the conventional inverse squared Newtonian attraction. This generates an inverse cubic formulation corresponding to the consensus analysis describing a differential tidal force:
For a combination of monthly and fortnightly sinusoidal terms for a(t) and d(t) (suitably modified for nonlinear nodal and perigean corrections due to the synodic/tropical cycle) the search routine rapidly converges to an optimal ENSO fit. It does this more quickly than the harmonic analysis, which requires at least double the unknowns for the additional higher-order factors needed to capture the tidally forced response waveform. One of the keys is to collect the chain rule terms a'(t) and d'(t) in the numerator; without these, the necessary mixed terms which multiply the anomalistic and draconic signals do not emerge strongly.
As before, a strictly biennial modulation needs to be applied to this forcing to capture the measured ENSO dynamics — this is a period-doubling pattern observed in hydrodynamic systems with a strong fundamental (in this case annual) and is climatologically explained by a persistent year-to-year regenerative feedback in the SLP and SST anomalies.
Here is the model fit for training from 1880-1980, with the extrapolated test region post-1980 showing a good correlation.
The geophysics is now canonically formulated, providing (1) a simpler and more concise expression, leading to (2) a more efficient computational solution, (3) less possibility of over-fitting, and (4) ultimately generating a much better correlation. Alternatively, stated in modeling terms, the resultant information metric is improved by reducing the complexity and improving the correlation — the vaunted cheaper, faster, and better solution. Or, in other words: get the physics right, and all else follows.
From the last post, we tried to estimate the lunar tidal forcing potential from the fitted harmonics of the ENSO model. Two observations resulted from that exercise: (1) the possibility of over-fitting to the expanded Taylor series, and (2) the potential of fitting to the ENSO data directly from the inverse power law.
The Taylor’s series of the forcing potential is a power-law polynomial corresponding to the lunar harmonic terms. The chief characteristic of the polynomial is the alternating sign for each successive power (see here), which has implications for convergence under certain regimes. What happens with the alternating sign is that each of the added harmonics will highly compensate the previous underlying harmonics, giving the impression that pulling one signal out will scramble the fit. This is conceptually no different than eliminating any one term from a sine or cosine Taylor’s series, which are also all compensating with alternating sign.
The specific conditions that we need to be concerned with respect to series convergence is when r (perturbations to the lunar orbit) is a substantial fraction of R (distance from earth to moon) :
Because we need to keep those terms for high precision modeling, we also need to be wary of possible over-fitting of these terms — as the solver does not realize that the values for those terms have the constraint that they derive from the original Taylor’s series. It’s not really a problem for conventional tidal analysis, as the signals are so clean, but for the noisy ENSO time-series, this is an issue.
Of course the solution to this predicament is not to do the Taylor series harmonic fitting at all, but leave it in the form of the inverse power law. That makes a lot of sense — and the only reason for not doing this until now is probably due to the inertia of conventional wisdom, in that it wasn’t necessary for tidal analysis where harmonics work adequately.
So this alternate and more fundamental formulation is what we show here.
[mathjax]With an ideal tidal analysis, one should be able to apply the gravitational forcing of the lunar orbit1 and use that as input to solve Laplace’s tidal equations. This would generate tidal heights directly. But due to aleatory uncertainty with respect to other factors, it becomes much more practical to perform a harmonic analysis on the constituent tidal frequencies. This essentially allows an empirical fit to measured tidal heights over a training interval, which is then used to extrapolate the behavior over other intervals. This works very well for conventional tidal analysis.
For ENSO, we need to make the same decision: Do we attempt to work the detailed lunar forcing into the formulation or do we resort to an empirical bottoms-up harmonic analysis? What we have being do so far is a variation of a harmonic analysis that we verified here. This is an expansion of the lunar long-period tidal periods into their harmonic factors. So that works well. But could a geophysical model work too?
[mathjax]After spending several years on formulating a model of ENSO (then and now) and then spending a day or two on the AMO model, it’s obvious to try the other well-known standing wave oscillation — specifically, the Pacific Decadal Oscillation (PDO). Again, all the optimization infrastructure was in place, with the tidal factors fully parameterized for automated model fitting.
This fit is for the entire PDO interval:
What’s interesting about the PDO fit is that I used the AMO forcing directly as a seeding input. I didn’t expect this to work very well since the AMO waveform is not similar to the PDO shape except for a vague sense with respect to a decadal fluctuation (whereas ENSO has no decadal variation to speak of).
Yet, by applying the AMO seed, the convergence to a more-than-adequate fit was rapid. And when we look at the primary lunar tidal parameters, they all match up closely. In fact, only a few of the secondary parameters don’t align and these are related to the synodic/tropical/nodal related 18.6 year modulation and the Ms* series indexed tidal factors, in particular the Msf factor (the long-period lunisolar synodic fortnightly). This is rationalized by the fact that the Pacific and Atlantic will experience maximum nodal declination at different times in the 18.6 year cycle.
After spending several years (edit: part-time) on formulating a model of ENSO (then and now), I decided to test out the formulation on another standing wave oscillation — specifically, the Atlantic Multidecadal Oscillation (AMO). All the optimization infrastructure was in place, with the tidal factors fully parameterized for automated model fitting.
This fit is for a training interval 1900-1980:
The ~60 year oscillation is a hallmark of AMO, and according to the results, this arises primarily from the anomalistic lunar forcing cycle modulated by a biennial seasonal modulation. Because of the spiked biennial modulation, we do not get a single long-period cycle but one that is also modulated by the forcing monthly tidal periods. As with ENSO, second-order effects in the anomalistic cycle described by lunar evection and variation is critical.
Outside of the training interval, the cross-validated test interval matches the AMO data arguably well. Since AMO is based on SST anomalies, it’s possible that strong ENSO episodes and volcanic perturbations (e.g. post 1991 Pinatubo eruption) can have an impact on the AMO measure.
This is a typical fit over the entire interval.
This is the day after I started working on the AMO model, so these results are preliminary but also promising. AMO has a completely different character than ENSO and is more of an upper latitude phenomenon, which means that the tidal forces have a different impact than the equatorial ENSO cycle. Some more work may reveal whether the volcanic or ENSO forcing overrides the tidal forcing in certain intervals.