I signed up for a SubStack account awhile ago and recently published two articles on this account (SubSurface) in the last week.

• https://pukite.substack.com/p/machine-learning-validates-the-enso
• https://pukite.substack.com/p/machine-learning-validates-the-amo

The SubStack authoring interface has good math equation mark-up, convenient graphics embedding, and an excellent footnoting system. On first pass, it only lacks control over font color.

The articles are focused on applying neural network cross-validation to ENSO and AMO modeling, as suggested previously. I haven’t completely explored the configuration space but one aspect that may becoming clear is the value of wavelet neural networks (WNN) for time-series analysis. The WNN approach seems much more amenable to extracting sinusoidal modulation of the input-to-output mapping — trained on a rather short interval and then cross-validated out-of-band. The Mexican hat wavelet (2nd derivative of a Gaussian) as an activation function in particular locks in quickly to an LTE modulation that took longer to find with the custom search software I have developed at GitHub. I think the reason for the efficiency is that it’s optimizing to a Taylor’s series expansion of the input terms, a classic nonlinear expansion that NN’s excel at.

The following training run using the Mexican hat activation and ADAM optimizer is an eye-opener, as it achieved an admirable fit within a minute of computation.

The GREEN on BLUE is training on NINO4 data over two end-point intervals, with the RED cross-validation over the out-of-band region. The correlation coefficient is 0.34, which is impressive considering the nature of the waveform. Clearly there is similarity.

Moreover, if we compare the model fit to data via the WNN against the LTE harmonics approach, you can also see where the two fare equally poorly. Below in the outer frame is the NINO4 LTE fit with the YELLOW arrow pointing downward at a discrepancy (a peak in the data not resolved in the fit). In comparison the yellow-bordered inset shows the same discrepancy on the WNN training run. So the fingerprints essentially match with no coaching.

The neural net chain is somewhat deep with 6 layers, but I think this is needed to expand to the higher-order terms in the Taylor’s series. In the directed graph below, L01 is the input tidal forcing and L02 is the time axis (with an initial very low weighting).

It also appears temporally stationary across the entire time-span, so that the WNN temporal contribution appears minimal.

In a previous fit the horizontal striations (indicating modulation factor at a forcing level) matched with the LTE model, providing further evidence that the the WNN was mapping to an optimal modulation.

The other Sub(Surface)Stack article is on the AMO, which also reveals promising results. This is a video of the training in action