How we avoid overfitting
A backtest that shows 200% annual returns is probably worthless. Here's how we validate strategies properly.
The Overfitting Problem
It's easy to fit a curve to past data. The RSI crossed 30 exactly when price bounced. The moving average golden cross predicted every rally. In reality, you've just memorized noise.
Our Validation Process
1. Walk-Forward Testing
- Train on 2 years of data
- Test on the next 6 months (out-of-sample)
- Roll the window forward
- Repeat across multiple market regimes
2. Cross-Validation
We test across multiple instruments. If a strategy only works on EURUSD but fails on GBPUSD and USDJPY, it's probably curve-fit.
3. Monte Carlo Simulation
Randomize trade order 10,000 times. If max drawdown varies by more than 50%, the strategy is fragile.
Red Flags
- ✗ Sharpe ratio > 3 in backtest
- ✗ Zero losing months across 5+ years
- ✗ Parameters optimized to 3+ decimal places
- ✗ Works on one pair but not similar pairs
- ✗ Performance degrades immediately in live trading
What Good Looks Like
- Similar performance in-sample and out-of-sample
- Consistent across multiple instruments
- Reasonable Sharpe ratio (1.0-2.0)
- Simple, explainable logic
- Robust to parameter changes
Live Validation
We paper trade every new strategy for 3 months before risking real capital. If it can't survive that period, it goes back to the drawing board.
Conclusion
The goal isn't to build the best backtest. It's to build a strategy that works in conditions it's never seen before. That's the only kind that survives real markets.