The Proof Is In The Pudding

BarroMetrics Views: The Proof Is In The Pudding

It’s my belief that our trading plan ‘setups’ ought to be vetted for an edge. This is true even for discretionary rule-based (DRB) traders. Sure, the testing for DRB setups will be tested in the absence of context; and context is critical for the DRB trader. Nevertheless, the testing will, at least, provide an indication if there is validity to the pattern.

We need to appreciate what backtesting does and does not provide. It does tell us that the pattern probably has an edge. It does not tell us that the equity curve produced will be replicated.

A good example is the study I posted yesterday by InvestiQuant in yesterday’s blog, ‘System Research, for You?‘ If you watched the video, you’d have seen that the study suggested:

  1. A downside bias.
  2. The better setup: an open-gap up to sell into. 
  3. The secondary setup: an open-gap down. The strategy tested sell the open, cover at the close. 

So, the study outlines the edge. When it comes to trading, there is no way, I’ll just sell the open. When I trade, I look for bias, zone, setup, trigger (entry), stop and trade management.

Figure 1 shows the way I implemented the edge.

  • Bias: established
  • Zone: the 2nd and 3rd std of the linear regression bands.
  • Setup: last night, I entered twice. In each case, the setup was a neutral bar in the zone (green arrows)
  • Initial stop: above the 3rd std (black line)
  • Trade Management: I used a Market Profile idea to manage the trade, bringing the stop down above the ledges (Figure 2).
  • Exited all positions on the big bar’s close after the red arrow 

Backtesting is a tool – that’s all it is. The fact that a result shows a fabulous edge does not mean that on this trade, the edge will play out. It is over the large sample size that the proof is found in the pudding.


FIGURE 1 5-min ES


FIGURE 2 30-min Market Profile ES

5 thoughts on “The Proof Is In The Pudding”

  1. Hi Chris

    I liked it. In fact, I have subscribed to the newsletter – thanks for the nudge.

    I especially like the comments:

    1) “typically highly path-dependent”

    2) The only way to properly evaluate this, I advised, was to backtest the strategy over many hundreds of thousands of test-runs using Monte Carlo simulation. That would reveal all too clearly that the risk of ruin was far larger than might appear from a single backtest.

    Glad my tester wasn’t smoking weed when he told me that MC testing was a must.

Leave a Reply

Your email address will not be published. Required fields are marked *