What Is Backtesting — and How to Read the Results Honestly

Backtesting is the single most powerful tool a retail investor has for checking a strategy before real capital is at risk. It is also the tool with the highest ratio of confidence created to confidence deserved. A clean-looking equity curve is easy to produce and easy to believe. The interesting work is knowing what the curve actually shows, what it can’t show, and which numbers in the report deserve your attention.

This post walks through what a backtest is, the metrics that matter (and the order to read them in), the warning signs of an overfit result, and how to use a backtest honestly — as a way to falsify a strategy, not to confirm one you already like.

What a backtest actually is

A backtest is a simulation. You take a set of rules — “hold this ETF when the Fear Index is below X, rotate to cash above Y” — and you replay them day by day against real historical price data. The simulation tracks what you would have owned each day, what you would have paid to switch, and what your portfolio would have been worth at the end. The output is a hypothetical equity curve plus a set of summary statistics. By construction, a well-built backtest processes each day using only the information that would have been available on that day — no peeking at future prices. That property has a technical name (no look-ahead bias), but the idea is simple: on any simulated day, the strategy is only allowed to see what a real investor could have seen at the time.

Two things are worth being very clear about from the start. First, the equity curve is hypothetical. It is what would have happened if you had run these exact rules on these exact prices — nothing more. Second, historical prices are not a preview of future ones. As the line often attributed to Mark Twain goes, history doesn’t repeat itself, but it often rhymes — and that rhyme is exactly the point of a backtest. A backtest tells you whether a strategy was internally coherent given the conditions of the past, and gives you an honest sense of what has worked, what hasn’t, and under which conditions. It does not, and cannot, tell you what will happen next.

Taken with those two caveats, a backtest is still enormously valuable. It’s the closest thing to a rehearsal you can get before the real performance.

What a good backtest tells you (and what it doesn’t)

A useful backtest answers three questions.

• Was the idea coherent? Given the rules you wrote, did the strategy behave the way you’d hoped in each regime? Did it actually reduce exposure when it was supposed to? Did it get back in when it was supposed to?
• How bad does it get? The deepest drawdown, the longest underwater stretch, the worst year. This is the emotional test — would you actually hold this strategy through the pain the backtest is showing you?
• How does it compare? Not just to buy-and-hold in return, but in risk-adjusted terms. A strategy that beats buy-and-hold on paper by taking twice the risk is not the same thing as one that beats it at similar risk.

A backtest cannot answer whether this will keep working. It cannot predict a new regime the market has never shown you. It cannot tell you whether you’ll hold through a drawdown that lasted two years on the chart but felt like ten while you were living it. It’s a rehearsal, not a forecast.

The metrics worth reading — in the right order

Most backtest reports show a wall of numbers. The order in which you read them matters more than which ones are technically “best”.

Maximum drawdown, first. The deepest peak-to-trough loss the strategy took during the backtest. This is the single most important number in the report — it tells you whether the strategy is one you would actually execute. A backtest with an amazing return and a 90% drawdown is not a strategy you own; it’s a strategy you sell out of somewhere on the way down. If the max drawdown is more than you’d genuinely tolerate, nothing else on the page matters yet.

Then the underwater curve. How long the strategy spent below its previous high. A 30% drawdown that recovers in six months is very different from a 30% drawdown that stays underwater for four years. The second one tests your patience far more than the first.

Then CAGR (compound annual growth rate). The one-number summary of what the strategy returned per year, on average, compounding. Only after you’ve decided the risk is acceptable does the return number mean anything. A great CAGR at unacceptable drawdown is not an option you can take.

Sharpe ratio. Return per unit of volatility. It is the single most useful number for comparing two strategies on a risk/reward basis: a strategy with double the return but triple the volatility isn’t strictly better, and the Sharpe ratio makes that obvious in one figure. As a rough sense of scale, a Sharpe above 1 is generally considered good, above 2 is exceptional, and below 0.5 is unimpressive. Read it against the same-period buy-and-hold Sharpe rather than in isolation — the number only means something in context.

Three warning signs of an overfit backtest

Overfitting is what happens when you tune a strategy so tightly to the historical data that it captures noise as if it were signal. The result is a beautiful backtest and a disappointing real portfolio. Three signs to watch for.

• Too many rules. Every additional parameter — threshold, overlay, weighting — is another degree of freedom that can be quietly tuned to the past. A strategy with two rules and one threshold is honest. A strategy with fifteen rules and eleven thresholds has almost certainly been curve-fit.
• Too short a history. A strategy tested only on the calm 2013–2019 period has never seen a real crisis. One tested only on 2020–2024 has seen exactly one crisis and one rate-hike cycle. A backtest that doesn’t include 2020 and 2022 is a backtest that hasn’t been stress-tested.
• Suspiciously smooth curve. Real strategies have ugly stretches — months when the rule was wrong and the alternative would have looked smarter. A backtest whose equity curve rises in a nearly-straight line is almost certainly overfit, not brilliant. Reality is bumpier than that.

The most useful discipline against overfitting is negative: try hard to break your own strategy in the backtest, and only trust it if you can’t. Change the thresholds a little. Extend the period. Change the ETFs. If the result falls apart on tiny changes, the original was fragile, not robust.

Using a backtest honestly

The best mental model for a backtest is that it exists to falsify a strategy, not to confirm one you already like. Its job is to surface the failure modes, not to advertise the successes. Two practical habits make this real.

First, before you look at the equity curve, write down the conditions you think would make the strategy struggle. A rate-hike cycle. A slow bear. A single-day flash crash. Then go check whether the backtest has actually been through those conditions, and how it behaved. If the answer is “I don’t know”, the backtest hasn’t given you enough to trust the strategy yet.

Second, once a backtest passes your own honest tests, treat that as the beginning of the validation, not the end. Paper-trade the strategy for a few weeks or months. Follow the signals on paper. Watch whether you actually do what the rule says when it fires. A clean backtest tells you the strategy is coherent. A paper run tells you whether you can run it. Both are needed before real capital, and we set that out in more detail in how to build your first strategy in PortfolioLab.

There is one more layer, and it is not inside any backtest. A clean simulation on past data tells you whether the strategy was coherent under conditions the market has already shown. It cannot tell you whether the same logic will still make sense under conditions the market hasn’t shown yet. That is a thinking exercise, not a computing one: ask yourself, honestly, why the strategy worked — is it a durable market truth, or a coincidence of the specific decade you tested? That judgement call is part of building and testing a strategy. The backtest is the input to it, not the answer.

The takeaway

A backtest is honest arithmetic. The dishonesty, when it appears, is in how humans read them — accepting the confirming numbers, ignoring the drawdowns, forgetting the caveats about the future. Read the risk numbers first, treat any suspiciously smooth curve as a warning instead of a promise, and use the backtest to look for reasons the strategy won’t work rather than reasons it will. That single discipline separates a rehearsal that helps you from one that flatters you.

If you’re weighing whether the rule-based approach itself is worth the trouble before you even open a backtest, the buy-and-hold vs tactical comparison sets out both sides honestly. And if hedging is part of what your strategy is meant to do, when hedging actually pays off covers what genuinely works and what just costs returns.