Jim DudasAug 7, 2026Why does a futures backtest look excellent and then disappoint in practice? I first check whether the test used only information, prices, costs, and timing that were genuinely available to the trader. In 2020, the National Futures Association’s Interpretive Notice 9025 required a hypothetical-results disclaimer immediately before simulated results when a covered member had less than 12 months of actual results, and it specifically warned about hindsight, liquidity, slippage, and financial-risk limits. I use the same standard in research: a smooth equity curve tells me something about the model, not what could necessarily have been traded.
The working example is a CRT reclaim filtered by session VWAP, or volume-weighted average price, deviation. CRT is not a standardized exchange term; here it means a defined session range, a sweep beyond one boundary, then a later close reclaiming that boundary. I am not trying to prove that setup works. I want CRT-only, VWAP-only, and combined versions encoded so they compete under exactly the same data, fills, exits, and validation rules.
I treat a credible futures backtest as a controlled experiment: define the rule, preserve time order, charge realistic costs, then challenge it on unseen data. Costs alone can overturn an attractive result. As verified in 2026, TradingView’s strategy documentation gives an illustrative backtest that moves from 17.61% equity growth before fees to 1.42% net profit after a 1% commission. That is a simulated, assumption-bound example, not a forecast of future performance.
Before I test a signal, I make the research environment reproducible, secure exchange-quality historical data, and document the session and contract rules. In 2026, the Python Software Foundation released Python 3.14.7; the official pandas release notes listed pandas 3.0.5, and the NumPy project listed NumPy 2.5.1. Those exact versions are not mandatory. Pinning whatever versions you use is. In practice, the hard work is usually verifying and cleaning the feed, not installing the software.
I want timestamped open, high, low, close, and volume bars, plus the contract symbol and timezone. Exchange specifications stay beside the data so multiplier, tick size, trading hours, and expiration rules cannot become hidden assumptions. Define the London and New York windows in Eastern Time, then perform timezone-aware conversion from the source timestamps. Otherwise daylight-saving changes can quietly shift the very sessions you think you are testing.
| Approach | Best fit | Main tradeoff |
|---|---|---|
| Custom Python | Maximum control over sessions, resampling, state machines, and trade-level diagnostics. | You own data cleaning, execution assumptions, and test correctness. |
| Platform backtester | Fast iteration when the strategy maps cleanly to the platform’s bar and order model. | Session logic, intrabar sequencing, and export detail may be constrained by the engine. |
| Specialist research | Useful when the hard part is data lineage, validation design, or a reproducible handoff. | You still need explicit rules; outsourcing cannot rescue an ambiguous strategy. |
I consider this step finished only when another developer can implement the strategy without asking what “sweep,” “reclaim,” or “near VWAP” means. The NFA’s 2020 hypothetical-performance guidance identified hindsight and execution limitations. The practical consequence is simple: assumptions need to be explicit before simulated performance deserves much weight.
Write events, not chart impressions. Range set: freeze the session high and low only after the designated range-building window ends. Sweep detected: price must move beyond a frozen boundary under the stated rule. Reclaim confirmed: the confirmation bar must close back through that boundary. Entry eligible: only then can an order become legal; it cannot receive an earlier price from inside the confirmation bar.
Next I lock down invalidation, stop placement, targets, position sizing, and no-trade conditions. A phrase such as “skip weak volume” or “avoid choppy conditions” is not a rule until the field and threshold are defined. Keep sizing and exits identical across variants. Otherwise you are testing several changes at once instead of isolating signal quality.
Before the full run, I audit sample sessions trade by trade. I record which bar finalized the range, which created the sweep, which confirmed the reclaim, what VWAP was available at that moment, and which later bar could legally execute the order. If I cannot reconstruct that chain from stored data, I do not regard the backtest as auditable.
Before deriving a higher-timeframe signal, preserve the exact contract, session, and timestamp behind every one-minute bar. Contract substitution is not harmless. As verified in 2026, CME Group’s Micro E-mini Equity Index Futures FAQ lists the Micro E-mini Dow multiplier at 10% of the E-mini Dow multiplier; CME’s Micro Metals futures reference lists Micro Gold at 10% of standard Gold size and Micro Silver at 20% of standard Silver size. YM/MYM, GC/MGC, and SI/Micro Silver therefore need to remain distinct contracts.
I first convert every source timestamp to a timezone-aware index, then assign sessions in Eastern Time. Each session is built independently; rolling statistics, forward-fills, and resample buckets must not spill across the boundary. Daylight-saving transition weeks deserve a direct inspection of raw timestamps and final labels. A fixed UTC offset is convenient, but convenience is not evidence that the session stayed correct.
A continuous contract joins successive expiries into one historical series. The question is how that join was made, because the rule can alter measured price changes. As verified in 2026, CME Group’s Continuous Price Series methodology distinguishes an Active Contract driven around liquidity migration from a Front Contract that switches two business days before expiry. Pick one active-contract rule, retain the selected contract on every bar, and resist changing the roll after you have seen the result.
Missing bars require classification, not cosmetic repair. I separate genuine exchange closures from feed gaps and never fabricate OHLC values merely to create a uniform index. When the strategy needs volume or sequence information that is missing, I invalidate that session and preserve the exclusion in the audit log.
Resampling comes after session labels, and it stays inside those sessions. More importantly, each derived bar receives the timestamp when the full bar actually became known. The signal may use a completed aggregate bar; it cannot read the final high, low, close, or volume while that bar is still developing. That one rule removes a surprising amount of accidental look-ahead.
A session VWAP is only valid if it resets where intended and every observation uses present or earlier data. No later bars. As verified in 2026, TradingView’s Volume Weighted Average Price documentation defines typical price as (high + low + close) / 3 and VWAP as cumulative typical-price-times-volume divided by cumulative volume. The formula is simple; getting the session state and timestamp discipline right is the real test.
VWAP, or volume-weighted average price, is cumulative, which makes the reset rule part of the strategy definition. Here I calculate it separately for every defined session. At the boundary, both running totals restart; typical price times volume accumulates alongside volume, and their ratio produces VWAP. Values from the prior session must never carry forward.
A z-score measures distance from a reference mean in standard-deviation units. If it becomes the VWAP-deviation filter, its dispersion estimate must also be time-honest. Decide before testing whether dispersion means session-to-date data or a trailing window. If you make that choice after studying the equity curve, you are tuning the parameter, not validating an idea.
I hand-check several consecutive bars before trusting a full strategy run. Starting from the raw rows, I recalculate typical price, cumulative price-volume, cumulative volume, VWAP, and the deviation value, then compare them with the research engine. A chart platform can provide a useful second opinion. The hand calculation, however, is what verifies the reset rule and the timing.
I model CRT sweep-and-reclaim logic as a small state machine: establish the range, observe a sweep, wait for a confirmed reclaim, then permit a later entry. That sequencing matters. In 2020, the NFA explicitly identified hindsight as an inherent limitation of hypothetical performance. A same-bar “confirmation” cannot legitimately borrow information from the unfinished future of that bar.
Once the range-building window closes, freeze the session range. From then onward, its high and low are reference levels; they do not keep expanding while the algorithm searches for a sweep. I store both values and the timestamp when they became final. That gives the audit trail enough evidence to show that the signal never consulted a still-forming range.
The sweep needs a mechanical test. For a long setup, for example, the market may have to trade below the frozen session low before any reclaim becomes valid; the short setup mirrors this above the high. If a minimum excursion matters, define it in contract-aware units such as ticks. “Far enough below” may look obvious on a chart, but it cannot be tested consistently.
A reclaim exists only after the selected confirmation bar closes back through the frozen boundary. That immediately raises the practical question: when could I actually enter? The earliest legal point is after that close, at the next executable price supported by the simulator. If one bar contains both stop and target but the intrabar path is unknown, use a documented conservative convention or finer data. Never award yourself the favorable sequence trade by trade.
Now the three variants can compete fairly. I run CRT-only, VWAP-only, and combined tests with the same sizing, exits, data, and fill rules, then charge costs a live order would have faced. As verified in 2026, TradingView’s documented strategy example moved from 17.61% equity growth before commission to 1.42% net profit after a 1% commission. That simulated result is assumption-bound and does not predict future performance, but it illustrates how quickly cost can change the conclusion.
Change the signal condition and nothing else. I keep the date range, contract-selection rule, session labels, stops, targets, sizing, slippage, and execution timing fixed across all three variants. That converts the comparison into a genuine component test. If the combined rule improves the evidence, you can ask whether VWAP adds useful information beyond the CRT event instead of merely reducing or increasing trade count.
Commission is only one leak between a theoretical fill and my trading result. As verified in 2026, NinjaTrader’s pricing page listed per-side brokerage commissions from $0.09 to $0.39 for Micro contracts and $0.59 to $1.29 for Standard contracts, with exchange, clearing, and NFA fees additional. Use the schedule appropriate to the modeled account and contract, then add slippage that can make fills worse. Assuming every order receives the bar’s most convenient price is not conservative research.
Timing can matter as much as the explicit fee. If a signal becomes known at bar close, I do not let it fill at an earlier intrabar price unless finer data and the execution model prove that sequence. Stop and limit orders also need a fixed convention: does a touch fill, is trade-through required, or does another condition apply? Choose once and enforce it everywhere.
I write every trade to CSV so the result can be challenged later. The record includes contract, session, direction, signal variant, signal and eligible-entry timestamps, entry and exit reason, gross and net result, costs, VWAP deviation, CRT state, holding time, MAE (maximum adverse excursion), and MFE (maximum favorable excursion). This is where impossible fills usually become obvious, and it makes later segmentation reproducible.
Our product measures CRT reclaim signals against session VWAP deviation with chronological out-of-sample tests.
The validation question is straightforward: did later data influence the rules I am now evaluating? It should not. Parameters are selected on earlier observations, frozen, then measured on a later period that played no role in tuning. In 2026, Mroziewicz and Ślepaczuk tested a walk-forward design across six intraday frequencies and 81 window combinations, using a 19-month training period before applying selected parameters to a distinct 21-month out-of-sample period.
I do not randomly shuffle market bars into training and test sets. Markets arrive in time order, and a random split can let later regimes influence earlier decisions. Instead, define the research period, freeze rules and parameters, and run the out-of-sample period once. Walk-forward testing applies the same discipline repeatedly by advancing training and evaluation windows under a schedule fixed in advance.
What happens if the VWAP-deviation threshold or sweep setting moves slightly? If performance immediately collapses, I regard the strategy as fragile. Test a modest neighborhood around the selected parameters and study the surface, not merely the winning coordinate. A broad stable area is more persuasive than one isolated peak. Most importantly, freeze that grid before viewing out-of-sample results or the supposed validation data becomes another tuning set.
A pooled equity curve can conceal which part of the system actually works. I report YM or MYM separately from Gold and Silver products, then segment by session, direction, time of day, signal variant, and VWAP-deviation bucket. Within each slice, compare drawdown, expectancy, MAE, MFE, holding time, and return-to-VWAP behavior. If nearly all performance comes from one market or session, that fact belongs in the conclusion rather than behind an aggregate curve.
When a backtest looks suspiciously good, I usually investigate three things first: what the simulator knew, how generously it filled, and whether unlike data were pooled together. In 2020, the NFA highlighted hindsight, liquidity, slippage, and financial-risk limitations and required prominent hypothetical disclosure below the 12-month actual-results threshold. Those warnings map closely to the practical failure modes worth auditing.
Look-ahead bias does not require an obvious peek into tomorrow. It can come from reading the completed high of a bar that was still forming, using a range that later expanded, consuming a revised indicator value, or selecting a contract with knowledge unavailable at the time. My fix is mechanical: record when each feature becomes available, then prevent the decision engine from accessing newer state.
Suppose one bar contains both the entry and the target. Did price visit them in that favorable order? The bar alone cannot tell you. Start orders from the first legally executable moment, include trading costs, and handle ambiguous same-bar paths conservatively unless finer data resolves the sequence.
A perfect threshold can be the most dangerous result in the table. If a small neighboring change destroys performance, the model may be fitting noise rather than a durable relationship. Keep the complete optimization surface, not just its best cell, and do not inspect unseen data until the search rules are fixed.
Pooling markets and sessions can make a narrow edge look broad and impressive. I want each instrument and each session reported independently before I look at the portfolio total. The combined equity curve is useful, but it should be the final aggregation, not the only evidence presented.
I do not judge a backtest from one equity curve or one attractive ratio. A trustworthy result is a set of diagnostics that can be audited from several angles. As verified in 2026, TradingView’s Profit Factor documentation defines profit factor as the absolute value of winning-trade results divided by the absolute value of losing-trade results, with values above 1 indicating profitability under that metric. Useful information, certainly, but still only one view of the system.
For every market and session, report trade count, win rate, profit factor, net expectancy, average win and loss, maximum drawdown, MAE, MFE, holding time, monthly results, long/short results, time-of-day and VWAP-deviation buckets, return-to-VWAP rate, and the CRT-only versus VWAP-only versus combined comparison. I put drawdown and signal ablation beside returns because a profitable headline without its risk and component evidence is incomplete.
I prefer plain evidence labels over promotional language. Supported means the behavior stays directionally consistent out of sample, survives reasonable costs, and remains intact across nearby parameters. Preliminary means the pattern is interesting but lacks enough depth or stability. Inconclusive means different slices conflict or uncertainty overwhelms the signal. Not supported means the original claim fails once timing and costs are modeled honestly. Historical performance is not a promise of future results.
What do I want from a useful backtest? Reproducibility, realistic costs, honest timing, and reasonable stability when nearby assumptions change. In 2026, Mroziewicz and Ślepaczuk separated a 19-month training period from a 21-month out-of-sample period. That concrete discipline matters far more than making one in-sample equity curve look polished.
For the CRT-and-VWAP example, the strongest evidence comes from comparing the component signals while keeping trade-level details visible. Traadence’s backtest futures trading strategies product applies that same narrow method to CRT reclaim signals, session VWAP deviation, and chronological out-of-sample testing. That is the standard I want on trading and strategy research: past results are historical evidence, never a guarantee of future performance.
Backtest an algorithmic futures strategy by translating the idea into codeable rules, cleaning contract and session data, calculating features only from information available at each timestamp, modeling realistic fills and costs, and validating frozen rules on later unseen data. In 2026, the cited walk-forward study used a 19-month training period and a separate 21-month out-of-sample period, illustrating the key principle: keep tuning and evaluation chronologically separate. Save a trade-level audit trail so every signal and fill can be reconstructed.
Jim Dudas is the Trading Strategist & Signals Lead at Traadence. He backtests strategies before they go live, runs the signals desk, and writes about walk-forward testing, track-record transparency, and honest trading education.
Learn how to build a futures trading bot in NinjaTrader 8 by validating strategies, backtesting data, and preparing automated intraday execution.
Alex HodgeSep 6, 2026
Use this BNB trading bot buyer’s guide to compare execution, data speed, risk controls, backtesting, API security, and maintenance before choosing a setup.
Alex HodgeSep 4, 2026
Compare what matters in an automated futures trading platform: fill logic, broker API support, MES contract rollover, testing, reliability, and risk controls.
Alex HodgeAug 7, 2026