Question for experienced backtesters
-
Hi All,
So I backtested my EA on GBP/USD for period 01/01/16 to 05/08/20. I just use the historic data that comes with IG's MT4 terminal.
I was wondering if any of the forum members with extensive experience could take a look over the results and advise:
a) Is this a reliable way to backtest an EA?
b) What is your opinion of the performance of this EA?
c) I have read about some 3rd party backtesting services/software that say they are super accurate and reliable. Would you recommend any of these and what sort of fees do they charge?
-
How to make a 99.9% of quality backtest https://www.mql5.com/en/blogs/post/688162
-
@ambrogio Thanks. I have seen software like this doing my own searches. I was hoping for a slightly more informative response rather than a simple link to a product I have already seen advertised and do not - with no further information or reccomendation - trust at this point (certainly not enough to pay for it).
-
@sirboyce The modelling quality is terrible (n/a), so simply forget it. Do not trust any modelling quality below 99%.
-
@l-andorrà Thanks. I've run it at 90% with same results but will go for higher.
-
@l-andorrà Oh yeah...what do you use for backtesting?
-
@sirboyce I use TDS2 for data quality above 99%. It is a bit expensive, but it is certainly worth it.
-
-
@sirboyce Edit: Period of above backtest is 1st Jan 2012 to 8th August 2020
-
@sirboyce said in Question for experienced backtesters:
@ambrogio Thanks. I have seen software like this doing my own searches. I was hoping for a slightly more informative response rather than a simple link to a product I have already seen advertised and do not - with no further information or reccomendation - trust at this point (certainly not enough to pay for it).
The link I gave you is for tickstory.com, if you had visited the site you would have seen that there is a free version with some limitations, but enough to produce and understand how to perform the 99.9% quality backtest
After that you can decide whether or not to pay tickstory full version, or TDS2 -
@ambrogio Yes. I initially downloaded Tickstory Lite...but was unhappy as my virus protection identified and removed a virus from their software and wouldn't allow it to run. I can assess TDS for a month free and it is way better.
-
@sirboyce Impressive results. Can you please share the EA?
(English) I will try to help everyone in these fxDreema forums. But if you want to learn how to use the platform in depth or more quickly, I can help you with my introductory fxDreema course in English at https://www.theandorraninvestor.eu.
(Català) Miraré d’ajudar tothom en aquests fòrums d’fxDreema. Tanmateix, si vols aprendre a fer servir la plataforma amb més profunditat o més de pressa, t’hi puc ajudar amb el meu curs d’introducció a fxDeema en català a https://www.theandorraninvestor.eu/ca.
(Español) Intentaré ayudar a todo el mundo en estos foros de fxDreema. Sin embargo, si quieres aprender a usar la plataforma en profundidad o más deprisa, te puedo ayudar con mi curso de introducción a fxDreema en español en https://www.theandorraninvestor.eu/es.
-
@l-andorrà I do not think people share really good ea,s... maybe just boasting or maybe I am totally wrong here ... lets wait and see.... my apologies a forehand if I am mistaken, just my experience talking here.
-
@alphaomega See your point. But the truth is...I am quite new to all of this and might well look at these results and think everything is absolutely fine with the EA...but somebody more experienced might have said something like "take a look at your drawdown...that really needs work before hou consider going live" or something like that. I was also open to someone who is experienced in EA development and has paid for top-notch backtesting software to take the initiative and offer to backtest for me (in which case I would have been happy to share the ex4) and perhaps help me to optimise the EA to filter out some of those losses etc.
-
@sirboyce ... All good brother... Like I said, and I am sticking to my guns... hope you get rich, stay the person you are and grow in yourself!! and I mean that from the bottom of my heart. I wish I had a very good EA but I have not yet..Your EA looks like a martingale if I look at the results. I also have a few and I just give you a word of caution, be very careful.... happy green pip hunting!
-
@alphaomega Not a problem and thanks for your wishes. Same to you! BTW...not martingale...lot size is 2% of Equity each time
-
I always look at Profit factor, profit, drawdown, and number of trades as a starting point to evaluating any of my EAs. Typically a high profit factor with low drawdown is more consistent at making money. However, profit factor and drawdown on their own can be misleading which is why I also look at the number of trades. You could have a really high profit factor and low drawdown but the EA only made a few trades. On the flip side, I personally would also not want my EA making too many trades unless it had a nearly straight equity curve. Just my personal opinion but everyone has their own preferences.
-
Strategy optimizer always tries to "trick" you by finding profitable trades from totally random set of possibilities, i.e. profit in history but random result in future.
With this mindset, the goal in optimization is to find results that are least likely random coincidences, i.e. results with high statistical significance. This significance increases when you have
- longer test periods (5-10 years)
- more trades (200-1000 per year)
- better profit/drawdown ratio (recovery factor)
Drawdown number itself is nothing to worry about, just change your lot size to adjust it.
-
Many thanks for your comments guys. Thought I would share the results of a test I did from Jan 2004 - Jul 2020:
While I would take £10K to £5M any day of the week, and the report stats are mainly pretty stellar and everything worked out fine with the glory of hindsight...fact remains that if I had gone live with this EA beginning of 2009 I would have seen an utterly miserable 2 and a half year run of abject failure which would have been close to wiping a 10K account!
This is really frustrating to see as any other period is a steady (occasionally rapid) uptrend in profit.
Through looking at these results and analysing the visual trades in the tester it is clear that I need the following:
-
Additional indicator(s) to filter out trades that were destined to fail by never taking them in the first place. I am thinking really good volume and baseline confirmation indicator conditions. Suggestions welcome!
-
Money Management: I am happy where the EA places the initial SL, but in this EA test there were no trailing SL's, break evens, partial profit taking etc. Basically, when trades lose they hit the original trade SL and those trades can be really hefty...or crippling (e.g. that 2009-2011 period!). The EA gets way more wins than loses which is what held it up over the test period. Suggestions welcome!
-
Exit indicator(s): The trade exit conditions as is performs...OK. But I noticed that plenty of times I would take a 80 pip win when the trade moved to e.g. 200 pips at the peak. Also need to let winners run more. I need to consider accurate and reliable exit indicators to maximise wins, and just cut losses quickly on trades that slipped though the net and are never going to win.
Suggestions welcome regarding all the above (entry filters, trade management, exits). Any suggestions that resolve these issues and really improve the performance win a free EA!
-
-
I would say the most important factor is your test methodology. I assume you used optimization algorithm for the whole period up to July 2020. Now you must live test the EA in order to confirm the profitability.
A better way would have been to optimize for, say, up to 2019. Once you get a similar nice curve from optimization, you can "forward" run the rest of history (2019-2020). Much easier way to confirm profitability.
