Backtesting, lets talk about it!

[roar]

@TipsyWisdom said

ah ok, accounting for what has been said so far, I could also put all closing conditions, under a once per minute and it would also solve all this?

Yes, it helps. Basically, if you do your trading actions on open/close prices, everything is fine because metatrader usually has the open & close prices for every candle. Problems arise the moment you make trading actions at any fixed level (pending order, fixed SL, fixed TP, trailing stop) because metatrader might not have data exactly for the level you want.

[MrDaisyBates]

@roar Weird! So, if you do 1 OHLC in strategy tester, and you have a fixed TP at whatever price level, whenever that TP it hit by a candle, the tester will give you the High or Close (assuming bullish) price for your TP?

[roar]

@MrDaisyBates tester will take Open, High, Low or Close price, whatever is closest. And remember these are always 1-min OHLC, so if you use 1-hour chart the effect is less noticeable because there are 60*4 data points available.

But yes, fixed levels are a fundamental issue because they don't (can't) respect the limited nature of history data

[MrDaisyBates]

@roar Ah, okay! That makes sense. Thanks for the clarification. I need to keep this in mind!

[roar]

Actually, I think the issue is a bit more nuanced than simply taking the closest available data point. Lets say the SL and TP are both inside the H and L of a history candle. Now the tester has to decide, did the trade hit TP before it hit SL, because they both were possible with certain intra-candle movement patterns. Now, I think the tester favors the profitable side there as well - it always acts like TP was hit before SL.

[MrDaisyBates]

@roar Hahaha. Oof. Yeah. I imagine this is extremely problematic if you're dealing with very low timeframe algos and stuff especially. I suppose other than some large news candles periodically, if you were testing higher timeframe stuff, like you said, it becomes less and less discernible.

[cmuphyfx]

This thread has been incredibly helpful - thank you to all who contributed. I too have run into this problem where on 1M OHLC I have an incredible EA and then live it drastically underperforms. The challenge here is that in order to follow 'good' backtesting procedures we need to use 2+ years of data, and 1-2 years of out of sample data for verification. As we all know those tests take hours and hours to run every tick.

Here's a quick solution, which I think embodies the essence of @roar's comments for a once-per-bar TP and SL. The challenge here is how to optimize trailing stops - I don't think you can. I suspect we'll have to use every tick data for that?

[cmuphyfx]

Forgot to add details on the solution.

1. once per bar loop through open trades (my EA only has 1 trade open at a time)
2. Get the SL size in pips and multiply it by a factor, for example, if you want a 2RR trade you multiply the SL by 2
3. Check if we're a buy or sell order
4. calculate the TP price by adding or subtracting the pips you just calculated to the open price
5. Check close conditions for SL - Was the low of the last candle lower than our SL price? Was the high of the last candle higher than our SL price - if yes, close
6. Same as above but for a TP price.

[cmuphyfx]

@roar When you run an EA live, do you then replace all trading actions (open, close) from once per bar to per tick (via pass block?)

[Julianrob]

@TipsyWisdom When backtesting on M1 bar data, the ticks generated to build that candle are interpolated. That means, there are ticks that make up the OHLC of the candle, but everything in between is made up of random ticks that travel somewhat up and down.

Therefore, let's assume a candle has a high and low wick. Has it reached the high first before the close, or the low? This question is the key and holds a critical answer, at which most scalpers will be given false hopes that their system will yield real results after the backtesting looks good.

The only real way to test your strategy properly is forward testing, on a demo account. This will be very accurate and will be as close to live trading results as you're likely to get