Backtesting vs Market Replay
Choose the method according to the question. Testing a rule and rehearsing a decision can support the same project, but their results should be labelled separately.
Two questions, two outputs
| Comparison | Backtesting | Market replay |
|---|---|---|
| Main question | How did defined rules behave in the selected history? | How did I make decisions as the sequence unfolded? |
| Before starting | Write rules, sample and cost assumptions. | Write the exercise and visible information. |
| Output | A reproducible sample of rule outcomes. | A record of decisions and execution errors. |
| Main trap | Tuning repeatedly to the same data. | Using familiarity with later prices. |
A replay session can contribute observations to a manual test, provided the rules and record are sufficiently clear. Conversely, a backtest can identify scenarios worth rehearsing in replay.
Build the rule sheet first
Identify the eligible instrument, interval, session and data source. Define entry, initial risk, invalidation and exit. State how an ambiguous bar, missing data or overlapping position is handled. This prevents later results from quietly changing the original question.
If a change is necessary, give the revised rule a new version. Keep the earlier result rather than replacing it with the best-looking interpretation.
Separate first-pass and repeated sessions
A first-pass replay lets you record decisions without knowing the next part of the sequence. Repeating a familiar day can train a procedure, but familiarity makes it a different exercise. Label the repeat instead of adding it to an unseen evaluation sample.
Use a journal template to preserve the exercise type, rule version and review notes.
Review both types of evidence
For backtesting, review the sample, costs, average outcome and drawdown. For market replay, review timing, rule adherence and missed decisions. Neither a favourable historical average nor a well-executed replay guarantees a live result.
Questions about this topic
Can replay replace backtesting?
It can supply observations to a clearly defined manual test, but execution practice alone does not replace evaluating a rule across a suitable sample.