How we avoid overfitting
A backtest with spectacular returns and no out-of-sample test tells you very little. These are the checks a strategy has to pass before we trust it.
QuantFX2 min read
The problem
Fitting a curve to past data is easy. The RSI crossed 30 exactly when price bounced, and the golden cross caught every rally. A model like that has memorised noise, and the market will not repeat it.
How we validate
Walk-forward testing
We train on two years of data and test on the following six months, which the model has not seen. Then we roll the window forward and repeat, across as many market regimes as the history allows.
Across instruments
The same logic runs on related instruments. If a strategy works on EURUSD but fails on GBPUSD and USDJPY, it is probably curve-fit.
Monte Carlo resampling
We shuffle the trade order 10,000 times and look at the spread of maximum drawdown. If it varies by more than 50%, the strategy is fragile.
Warning signs
Any of these sends a strategy back for a closer look:
- A backtest Sharpe ratio above 3.
- No losing months across five or more years.
- Parameters tuned to three or more decimal places.
- Results on one pair that do not carry over to similar pairs.
- Performance that degrades as soon as it trades live.
What a sound result looks like
- Similar performance in sample and out of sample.
- Consistent results across instruments.
- A believable Sharpe ratio, roughly 1.0 to 2.0.
- Simple logic we can explain.
- Results that hold when the parameters move a little.
Live validation
Every new strategy then trades for three months on a demo account with live pricing before it is offered to licensees. If it cannot hold up there, it goes back to research.