The challenge of explaining climate phenomenon such as ENSO leads to an interesting conundrum. Do we want to understand the physics behind the phenomenon, or do we want to optimize our ability to forecast?
Take an example of the output of a crude power supply. Consider that all one has is one cycle of output.
- The forecaster thinks that it is fair to use only one half of that cycle, because then he can use that to forecast the other half of the cycle.
- The problem solver wants the whole cycle.
Why is the problem solver in better shape?
- The forecaster looks at the half of a cycle and extrapolates it to a complete cycle. See the dotted line below.
- The problem solver looks at the continuation and discovers that it is a full-wave rectified signal. See the solid line below
In this case, the problem solver is right because the power supply happens to be a full-wave rectifier needed to create a DC supply voltage. The forecaster happened to make a guess that would have been correct only if it was an AC power supply.
Lose your generality and that is what can happen. As Dara says, the key is to look for structures or patterns in the data — while reducing the noise — and if that means to use as much of the data as possible, so be it.