Advanced
Why Most Backtests Lie
Every strategy looks genius in a backtest. The equity curve goes up and to the right. The Sharpe ratio is 2.5. Then you deploy it and lose money in the first week. Here’s why — and how to build backtests that actually predict live performance.
The 5 biases that ruin backtests
Survivorship Bias
You backtest using today's S&P 500 constituents. But Enron, Lehman, and WorldCom were in the S&P too — before they went to zero. Your backtest never bought them, so your results are artificially inflated.
Fix: Use point-in-time constituent data. AlgoThesis includes delisted tickers in historical analysis.
Look-Ahead Bias
Your strategy uses data that wasn't available at the time. Example: buying a stock the day before earnings because you "know" the results. In a backtest, future data is just another column — it's easy to accidentally peek at it.
Fix: Strict chronological ordering. Never reference any data point before its actual release date.
Overfitting
You tweak your moving average from 20 to 23 days because that works better on AAPL from 2020-2024. Congratulations: you've found the noise, not the signal. The parameter will fail on new data.
Fix: Use out-of-sample testing. If you optimize on 2020-2023, validate on 2024. If it falls apart, it was overfit.
Transaction Cost Neglect
A strategy that trades 50 times per day looks great with zero commissions. Add realistic spreads ($0.01-0.05 per share), slippage, and market impact — and the strategy is underwater.
Fix: Always include realistic transaction costs. For small accounts, assume 0.1% round-trip minimum.
Selection Bias
You tested 100 strategies and found 5 that work. But with 100 tries, you'd expect 5 to work by pure chance at a 5% significance level. You've mined the data, not discovered alpha.
Fix: Apply Bonferroni correction or use a holdout period that none of your strategies have ever seen.
The red flags in a backtest report
- Sharpe ratio above 3.0 on a sample period under 2 years
- Win rate above 70% with no losing months
- Equity curve with no drawdowns — real markets have drawdowns
- Strategy was "optimized" on the same data it was tested on
- No transaction costs or slippage modeled
How to build a backtest you can trust
- Use at least 2 years of data — Ideally covering both bull and bear markets
- Split into train/test periods — Optimize on one period, validate on another
- Include transaction costs — Commissions, spreads, and slippage
- Paper trade before live — Run with simulated money for 2–4 weeks
- Track live vs backtest divergence — If live performance is more than 30% worse, something is wrong
How AlgoThesis reduces backtest bias
AlgoThesis mitigates these pitfalls by design. The backtesting engine uses point-in-time data, includes transaction cost estimates, and flags strategies with suspiciously high Sharpe ratios. The catalyst layer ensures your strategy is tested against real event timing — not future knowledge of what earnings results were. And the paper trading mode lets you validate before deploying a single dollar.
Backtests that account for reality
Catalyst-aware. Point-in-time data. No look-ahead bias.
Try AlgoThesis Free →