There is a moment in the development of every trading model where adding complexity stops helping and starts hurting. Most developers never notice it, because the number they are watching keeps improving.
That moment has a name, and understanding it explains more about why AI underperforms in markets than any discussion of algorithms or architectures.
Two ways to be wrong
A model can fail in two directions, and they pull against each other.
It can be too simple. A model that assumes gold moves in a straight line with real yields will miss the periods when that relationship weakens, when the dollar dominates, or when official sector demand absorbs supply the model expected to push prices down. It is wrong because it is not capturing structure that genuinely exists. This is bias.
It can be too complex. A model with enough parameters can fit every wobble in its training data, including the wobbles that were random. It looks superb on the data it learned from and falls apart on anything else. This is variance.
The uncomfortable part is that reducing one usually increases the other. Simplify to avoid memorising noise and you start missing real relationships. Add flexibility to capture those relationships and you start absorbing noise along with them.
There is no setting that eliminates both. There is only a point where their sum is smallest.
Why the error curves diverge
Plot two lines against model complexity and the whole problem becomes visible.
In-sample error — how well the model fits data it has already seen — falls continuously. It essentially always improves as you add parameters, because a sufficiently flexible model can eventually fit anything, including pure randomness.
Out-of-sample error — how well it performs on data it has never seen — falls at first, then turns and rises.
The turn is the entire story. Up to that point, added complexity is capturing genuine structure. Past it, added complexity is capturing noise, and because noise does not repeat, everything learned after the turn is worse than useless: it is confident nonsense.
The tell is the gap. When in-sample performance keeps improving while out-of-sample performance stalls or deteriorates, the model has crossed over. It is no longer learning the market. It is learning that specific slice of history.
Why markets make this worse than most fields
The trade-off exists everywhere in machine learning. Financial markets aggravate it in ways that are structural rather than technical.
The signal-to-noise ratio is brutal. In image recognition, the pattern is overwhelmingly stronger than the noise. In markets, most short-term movement is noise, and the genuine signal is faint. A model given freedom to fit will mostly fit randomness, because randomness is most of what is there.
The data is not stationary. Image recognition benefits from the fact that a cat looked the same in 2015 and looks the same today. Markets change. A relationship learned during one policy regime can weaken or invert in the next. The model did not break — the world moved.
You cannot generate more data. In most fields, more examples reduce variance and permit greater complexity safely. Markets produce data at a fixed rate, and the periods that matter most — crises, policy turns, structural breaks — are the rarest of all.
The observations are not independent. Consecutive days are correlated, so a dataset of ten thousand daily bars contains far less independent information than the count implies. Models systematically overestimate how much evidence they have.
What overfitting looks like in practice
You rarely see it directly. You see symptoms.
An unusually smooth equity curve. Real edges are noisy and uncomfortable to hold. A backtest that climbs cleanly with shallow drawdowns is usually a curve that learned the answers rather than the method.
Fragility to small changes. Shift one parameter slightly and performance collapses. A genuine edge degrades gracefully; a fitted artefact falls off a cliff. This is one of the fastest diagnostics available, and it takes minutes.
Performance concentrated in one period. Strip out the best six months and the strategy is flat. That is not a strategy; it is a description of six months.
Improvement that arrives only after many attempts. If a model became profitable on the fortieth configuration tested, the fortieth was not better. It was luckier. Testing enough variants guarantees one will look excellent on any dataset, including random ones.
Where the honest line sits
There is no formula, but there are disciplines that keep you near the right side of the turn.
Hold out data you never touch. Reserve a period and do not look at it during development. The moment you tune anything after seeing it, it stops being out-of-sample and becomes part of your training set.
Walk forward. Train on one window, test on the next, roll, repeat. Slower than a single historical run and far more informative, because it approximates how the model would actually have been used.
Prefer fewer parameters, deliberately. A simple model that works moderately across many conditions is more valuable than a complex one that works brilliantly in one. Simplicity here is not a limitation; it is a defence against variance.
Judge stability, not peak performance. The question is not which configuration scored highest, but which scored consistently across regimes. Peak scores are usually where the luck accumulated.
Count your attempts. Every additional variant tested raises the probability that your best result is noise. If you have tested hundreds, your best result almost certainly is.
What this means for using AI in markets
This trade-off is the mathematical reason behind a distinction we return to repeatedly: AI processes markets well and forecasts them poorly.
Processing tasks — monitoring many variables, flagging when a correlation is decaying, screening for unusual positioning — tolerate complexity comfortably, because they describe what is happening rather than extrapolating what happens next. Getting them slightly wrong costs you a false alert.
Forecasting sits on the wrong side of the curve. It demands extrapolation from limited, noisy, non-stationary data, which is precisely the environment where added complexity buys memorisation rather than understanding.
The practical rule: if a model's performance improves every time you make it more complicated, you have not found a better model. You have found the point where it started memorising, and you have walked straight past it.
This article is for general market information and education. It does not constitute financial or investment advice. Trading carries risk, and past performance is not indicative of future results.
0 Comments