What really is the color GREEN? What does it mean to different people? We actually have no idea. Now, consider The Dress on its 10 year anniversary. One can either argue with others whether it is BLUE and BLACK or WHITE and GOLD, or objectively look at it’s composition from a technical standpoint. Below is a cross-section of the dress image using the online image-picker application showing a palette from selected pixels in the image
By applying an annual impulse sample-and-hold on a common-mode basis set of tidal factors, a wide range of climate indices can be modeled and cross-validated. Whether it is a biennial impulse or annual impulse, the slowly modulating envelope is roughly the same, thus models of multidecadal indices such as AMO and PDO show similar skill — with cross validation results evaluated here for a biennial impulse. Now we will evaluate for annual impulse.
Continue readingFirst evaluation of Gemini Advanced 1.5 Pro with Deep Research. Logged in with trial subscription and gave this as an initial prompt. Saved the results to a Google Docs file, and then created the following PDF. Note the top-level focus on this blog and published citations in the scope of Chapter 12 of Mathematical Geoenergy, even though the chapter was not directly cited — only via an embedded cite in a submitted ESD Ideas article, reference 1. Interesting that the Lin & Qian paper not cited.
Prompt: Explain tidal forcing behind ENSO using derivations based on reduced effective gravity on equatorial thermocline.
A climate science breakthrough likely won’t be on some massive computation but on a novel formulation that exposes some fundamental pattern (perhaps discovered by deep mining during a machine learning exercise). Over 10 years ago, I wrote on a blog post on how one can extract the ENSO signal by doing simple signal processing on a sea-level height (SLH) tidal time-series — in this case, at Fort Denison located in Sydney harbor.
The formulation/trick is to take the difference between the SLH reading and that from 2 years (24 months) prior, described here
Check the recent blog post Lunar Torque Controls All for context of how it fits in to the unified model.
The rationale for this 24 month difference is likely related to the sloshing of the ocean triggered on an annual basis. I think this is a pattern that any ML exercise would find with very little effort. After all, it didn’t take me that long to find it. But the point is that the ML configuration has to be open and flexible enough to be able to search, generate, and test for the same formulation. IOW, it may not find it if the configuration, perhaps focused on computationally massive PDEs, is too narrow. That was my comment to a RC post on applying machine learning to climate science, see the following link and subsequent quote:
Nick McGreivy commented on:
“ML-based parameterizations have to work well for thousands of years of simulations, and thus need to be very stable (no random glitches or periodic blow-ups) (harder than you might think). Bias corrections based on historical observations might not generalize correctly in the future.”
This same issue arises when using ML to simulate PDEs. The solution is to analytically calculate what the stability condition(s) is (are), then at each timestep to add some numerical diffusion that nudges the solution towards satisfying the stability condition(s). I imagine this same technique could be used for ML-based parametrizations.
In a #Geophysics feed on Bsky.app, the latest “post” after 2 days is still this:
Instead, this is what they should be discussing [1,2]:
Also discussed at https://www.realclimate.org/index.php/archives/2024/11/twenty-years-of-blogging-in-hindsight/
yet responses are not on point
CITES
In addition to the standard correlation coefficient (CC) and RMS error, non-standard metrics that have beneficial cross-validation properties include dynamic time warp (DTW), complexity invariant-distance (CID) see [2], and a CID-modified DTW. The link above describes my implementation of the DTW metric but I have yet to describe the CID metric. It’s essentially the CC multiplied by a factor that empirically adjusts the embedded summed distance between data points (i.e. the stretched length) of the time-series so that the signature or look of two time-series visually match in complexity.
CID = CC * min(Length(Model, Data))/ max(Length(Model, Data))The authors of the CID suggest that it’s a metric based on “an invariance that the community seems to have missed”.
And a CID-modified DTW is thus:
CID = DTW * min(Length(Model, Data))/ max(Length(Model, Data))
I have tried this on the QBO model with good cross-validation results featuring up to-data data from https://www.atmohub.kit.edu/data/qbo.dat
These have similar tidal factor compositions and differ mainly in the LTE modulation and phase delay. As discussed earlier, any anomalies in the QBO behavior are likely the outcome of an erratic periodicity caused by incommensurate annual and draconic cycles and exaggerated by LTE.
from https://gist.github.com/pukpr/e562138af3a9da937a3fb6955685c98f
REFERENCES
[1] Batista, Gustavo EAPA, et al. “CID: an efficient complexity-invariant distance for time series.” Data Mining and Knowledge Discovery 28 (2014): 634-669.R
https://link.springer.com/article/10.1007/s10618-013-0312-3
“A key enabling assumption, sometimes called the manifold hypothesis [14], is that the data lie on or near a low-dimensional manifold; for physical systems with dissipation, such manifolds can often be rigorously shown to exist [15–18]. These manifolds enable a low-dimensional latent state representation, and hence, low-dimensional dynamical models. Linear manifold learning techniques, such as principal component analysis, cannot learn the nonlinear manifolds that represent most systems in nature. To do so, we require nonlinear methods, some of which are developed in [19–25] and reviewed in [26].“
Floryan, Daniel, and Michael D. Graham. “Data-driven discovery of intrinsic dynamics.” Nature Machine Intelligence 4.12 (2022): 1113-1120.
From <https://www.nature.com/articles/s42256-022-00575-4>
GC22A-04 Can ML beats chaos?
Abstract
“Chaos is typically blamed for the lack of predictability beyond a forecasting time window, which is on the order of 10 days for weather forecasting. However, on the one hand, most turbulent and chaotic systems exhibit strong coherence in the flow, such as synoptic events in weather or coherent structures in turbulence. On the other hand, most physical model might have additional structural errors that limit their capacity to correctly forecast beyond a certain time horizon, independent of chaos.
We will show in this presentation that most chaotic and turbulent flows can be predicted on relatively long range, at least longer than expected with both physical models and standard deep learning, using a combination of a reduced order model (that captures the low-dimensional coherent structures in the flow) and generative AI to obtain the crisp results and details of the flow. We will conclude stating that current AI-based weather models might not have achieved a plateau in performance yet, especially at longer time scales, and that physics-based weather model still have room for improvements. Reduced order models might not be able to beat chaos but can lead to much longer-range prediction than currently expected.“
https://agu.confex.com/agu/agu24/meetingapp.cgi/Paper/1522150
Reminded by a 20-year anniversary post at RealClimate.org, that I’ve been blogging for 20 years + 6 months on topics of fossil fuel depletion + climate change. The starting point was at a BlogSpot blog I created in May 2004, where the first post set the stage:
Click on the above to go to the complete archives (almost daily posts) until I transitioned to WordPress and what became the present blog. After 2011, my blogging pace slowed down considerably as I started to write in more in more technical terms. Eventually the most interesting and novel posts were filtered down to a set that would eventually become the contents of Mathematical Geoenergy : Discovery, Depletion, and Renewal, published in late 2018/early 2019 by Wiley with an AGU imprint.
The arc that my BlogSpot/WordPress blogging activity followed occupies somewhat of a mirror universe to that of RealClimate. I initially started out with an oil depletion focus and by incrementally understanding the massive inertia that our FF-dependent society had developed, it placed the climate science aspect into a different perspective and context. After realizing that CO2 did not like to sequester, it became obvious that not much could be done to mitigate the impact of gradually increasing GHG levels, and that it would evolve into a slow-moving train wreck. That’s part of the reason why I focused more on research into natural climate variability. In contrast, RealClimate (and all the other climate blogs) continued to concentrate on man-made climate change. At this point, my climate fluid dynamics understanding is at some alternate reality level, see the last post, still very interesting but lacking any critical acceptance (no debunking either, which keeps it alive and potentially valid).
The oil depletion aspect more-or-less spun off into the PeakOilBarrel.com blog [*] maintained by my co-author Dennis Coyne. That’s like watching a slow-moving train wreck as well, but Dennis does an excellent job of keeping the suspense up with all the details in the technical modeling. Most of the predictions regarding peak oil that we published in 2018 are panning out.
As a parting thought, the RealClimate hindsight post touched on how AI will impact information flow going forward. Having worked on AI knowledgebases for environmental modeling during the LLM-precursor stage circa 2010-2013, I can attest that it will only get better. At the time, we were under the impression that knowledge used for modeling should be semantically correct and unambiguous (with potentially a formal representation and organization, see figure below), and so developed approaches for that here and here (long report form).
As it turned out, lack of correctness is just a stage, and AI users/customers are satisfied to get close-enough for many tasks. Eventually, the LLM robots will gradually clean up the sources of knowledge and converge more to semantic correctness. Same will happen with climate models as machine learning by the big guns at Google, NVIDIA, and Huawei will eventually discover what we have found in this blog over the course of 20+ years.
Note:
[*] In some ways the PeakOilBarrel.com blog is a continuation of the shuttered TheOilDrum.com blog, which closed shop in 2013 for mysterious reasons.
The truly massive scale in the motion of fluids and solids on Earth arises from orbital interactions with our spinning planet. The most obvious of these, such as the daily and seasonal cycles, are taken for granted. Others, such as ocean tides, have more complicated mechanisms than the ordinary person realizes (e.g. ask someone to explain why there are 2 tidal cycles per day). There are also less well-known motions, such as the variation in the Earth’s rotation rate of nominally 360° per day, which is called the delta in Length of Day (LOD), and in the slight annual wobble in the Earth’s rotation axis. Nevertheless, each one of these is technically well-characterized and models of the motion include a quantitative mapping to the orbital cycles of the Sun, Moon, and Earth. This is represented in the directed graph below, where the BLUE ovals indicate behaviors that are fundamentally understood and modeled via tables of orbital factors.
However, those ovals highlighted in GRAY are nowhere near being well-understood in spite of being at least empirically well-characterized via years of measurements. Further, what is (IMO) astonishing is the lack of research interest in modeling these massive behaviors as a result of the same orbital mechanisms as that which causes tides, seasons, and the variations in LOD. In fact, everything tagged in the chart is essentially a behavior relating to an inertial response to something. That something, as reported in the Earth sciences literature, is only vaguely described — and never as a tidal or tidal/annual interaction.
I don’t see how it’s possible to overlook such an obvious causal connection. Why would the forcing that causes a massive behavior such as tides suddenly stop having a connection to other related inertial behaviors? The answers I find in the research literature are essentially that “someone looked in the past and found no correlation” [1].
Continue readingGeophysics is a challenging endeavor — one part geology as historical evidence and the other part current measurements revealing how the forces of nature are continuously shaping the behavior of the Earth. Yet, how much faith can we place on any one interpretation, or on any past consensus?
This goes double for the study of geophysical fluid dynamics — volumes and forces so large that controlled experiments are impossible. So that when a semi-controlled impulse comes about, say from the calving of a glacier, the empirical results may be surprising:
The subsequent mega-tsunami — one of the highest in recent history — set off a wave which became trapped in the bendy, narrow fjord for more than a week, sloshing back and forth every 90 seconds.
The phenomenon, called a “seiche,” refers to the rhythmic movement of a wave in an enclosed space, similar to water splashing backwards and forwards in a bathtub or cup. One of the scientists even tried (and failed) to recreate the impact in their own bathtub.
While seiches are well-known, scientists previously had no idea they could last so long.
“Had I suggested a year ago that a seiche could persist for nine days, people would shake their heads and say that’s impossible,” said Svennevig, who likened the discovery to suddenly finding a new colour in a rainbow. — http://www.msn.com/en-in/news/techandscience/what-caused-the-mysterious-9-day-earth-vibration-scientists-point-to-650-ft-mega-tsunami-that-rocked-greenland/ar-AA1qwIqk
This took place in a large Greenland fjord — massive but not as massive as the ocean. And that’s where massive geophysical forces also take place, such as from tidal pull.
So what about the Atlantic Multidecadal Oscillation (AMO)? The Mt long-period tide is an interference between the Mf nodal tidal cycle (13.66 days) and the Mm anomalistic tidal cycle (27.55 days). When this is synchronized to an annual impulse, a multidecadal response results.
It is much too easy to model the 50-60 year AMO cycle via nonlinear triad frequency-doubling mechanisms, and also simulate the faster cycling from the predominate Mf and Mm interactions with the annual cycle. This is essentially a sloshing behavior operating on the ocean’s thermocline as described in Chapter 12 of Mathematical Geoenergy, applying the nonlinear geophysical dynamics embedded within Laplace’s Tidal Equations.
New colors of the rainbow are waiting to be revealed. No excuse for others to reproduce the tidal forcing models as described and perform cross-validated model fits to replicate the results above. Work smart not hard.
