Testing and DATA

[andyjames]

Hello,

I just want to mention something very important to those of you that may not know.
Fxdreema is an amazing tool to create your expert advisors, but it doesn't end there. If your testing is not done with as much accuracy as possible then how do you really know how it will perform. There are many methods to test your EA but the testing quality itself all boils down to one thing... your data.

MT4 comes complete with data which you can easily download from MetaQuotes. Unfortunately this data is very often full of 'holes' which affects your results. Not only that but the data from MetaQuotes only uses the 1 minute time frame in the MT4 strategy tester.

Plus, MT4 does not store the real ticks from your broker’s server. MT4 only has the open, high, low, close and number of ticks for each minute, so it generates fake ticks. If you use a trailing stop for example, or have an EA take profit at 10 - 20 pips then testing on MT4 standard data is going to give you completely different results compared to, if you had actually run your EA for that period in real life.

Essentially the answer is to use actual tick data which you can download from the web, I personally use Dukascopy tick data. Then there's the problem of importing that into MT4 and using it correctly. You can do it manually but it's a real pain in the arse to be honest.

So for testing I use a certain tool which isn't free. You can find it here at http://eareview.net/tick-data-suite

I have no affiliation at all with this product and at around a hundred bucks it's not cheap. But, I've been using it for many months, the level of support you get is excellent just like here at fxdreema.

I won't go into exactly how you use it but I can say it simplifies the whole process and your testing becomes much, much more accurate and you can use real spreads, and easily generate the GMT offsets needed for visual testing.

Happy testing,
Andy

[fxDreema]

Good information, Andy, thank you for it.

Ticks data is a "must" when trying strategies with small stop-loss and take-profit, and especially when using techiques like trailing stop. It will be lose of time if you backtest scalping with reguar ticks data.

Far away from 1M, when we work with bigger size of stop-loss and take-profit, it's also iportant that we have good quality of lower timeframe data. 1M can be good enough, but note that M1 data normally exists in MetaTrader's history for only a few months ago. Well, it can be downloaded for longer period (this requires more PC resources).

What I actually like is that "Open prices only" model in MetaTrader's Tester. The good part is that this model is much more faster that "Every tick" model. But it requires special strategies to be tested with it. Trailing stop is not a good choice here. If our strategy is to keep single trade per bar, it's ok to be backtested using this model. In fxDreema you can use "Once per bar" block at the very top, and this will make things to happen only when a new bar comes.

[andyjames]

__Good information, Andy, thank you for it.

Ticks data is a "must" when trying strategies with small stop-loss and take-profit, and especially when using techiques like trailing stop. It will be lose of time if you backtest scalping with reguar ticks data.

Far away from 1M, when we work with bigger size of stop-loss and take-profit, it's also iportant that we have good quality of lower timeframe data. 1M can be good enough, but note that M1 data normally exists in MetaTrader's history for only a few months ago. Well, it can be downloaded for longer period (this requires more PC resources).

What I actually like is that "Open prices only" model in MetaTrader's Tester. The good part is that this model is much more faster that "Every tick" model. But it requires special strategies to be tested with it. Trailing stop is not a good choice here. If our strategy is to keep single trade per bar, it's ok to be backtested using this model. In fxDreema you can use "Once per bar" block at the very top, and this will make things to happen only when a new bar comes.[/quote:3p74f2ox]

I agree with you. Most of my projects use the "once per bar" block to start trade entry logic trees, ie EMA 12 > EMA 24 and set it to the last completed candle (candle id 1), then use the 'Start trades loop' to monitor any trades, trailing stops etc.

But one big problem with MT4 is the data, using the 'Open prices' model is great to give you a fast approximation of how your logic will work, then if you like the results I'd recommend setting the model to ticks only to get a much more accurate result.

By the way, the software I mentioned in my original post comes with a 7 day trial, the functionality in the trail is full. Before you download a trial I'd first get the tick data downloaded from Dukascopy.. create demo account (14 days only) and download the jForex script, run that and download the data. If you want lots of currency pairs over several years it can take a couple of days to download. So best to get the tick data files downloaded first as they are several Giga bytes each.

Another good reason not to use MT4 Metaquotes data is that the strategy tester only uses fixed spreads, the Tick Data Suite interacts with MT4 and uses the actual spreads as most brokers now have variable spreads. This alone can make a huge difference in your testing.

[seanmcgu]

__Good information, Andy, thank you for it.

In fxDreema you can use "Once per bar" block at the very top, and this will make things to happen only when a new bar comes.[/quote:u0mx76jm]
Hello, All

First of all, fxDreema, thank you for the wonderful program you have created! I just started using it two days ago and have already created an EA. Thanks!

I just wanted to share this with anyone that might be having the same issue that I had.

When replacing the “Buy now” block (I was using the “No trade is running” block at the beginning) for a “Draw Arrow” block, I noticed that many, Many, MANY arrows were being printed:

http://img338.imageshack.us/img338/4254/manyarrowserrortofxdree.png

So I started reading the posts in the forums looking for an answer. I found it here after reading about using the “Once per bar” block. Nice!!

This is my code now and it’s working great!

http://img824.imageshack.us/img824/7307/codeafterfixingmultiple.png

And this is how the screen looks now:

http://img850.imageshack.us/img850/5272/fixedarrowproblem.png

Thanks again for such a wonderful program!

 .....Sean

[fxDreema]

seanmcgu, thanks for your opinion

When you are working under "Tick", blocks will be executed again and again for every new tick that comes - those blocks on the top level basically (connected from output to others and nothing connected to them). All blocks below are depending on the blocks above.

That is what an EA is all about - something is repeated again and again and again, and every time the EA checks some conditions (many times per second), and depending on those conditions it takes some actions. When it take new action, things changes and on the next moment conditions are different.

So, it's normal for some condition to be there on top level - it needs to be executed as frequently as possible (well, it depends on the condition, but there is always one on the top). "Once per bar" is something like condition+action, because it has internal memory that is changin over time. "No trade is running" is a condition. "Buy now" and "Draw arrow" are actions, and without conditions above to stop them, they will be executed as many times as many ticks arrive.

[seanmcgu]

__seanmcgu, thanks for your opinion
When you are working under "Tick", blocks will be executed again and again...[/quote:pcidlcrb]
Thanks. Makes sense!