Code Performance / Backtesting
-
Hello,
I have 2 EAs that have about 200+ Blocks each. Its 1 indicator used in the EA and I remember at early backtests the EA was running at a good speed.
Now, there are a lot of condition blocks, connected in a row or parallel with AND/OR connectors, lots of formulas and a bunch of variables and constants used.So I wonder if there is a good "style" of cnnecting condition blocks or some rules in general that affect performance.
Like...is it better to create solutions with conditions block set parallel and connected with AND to summarize the result of all conditions, or is it better to put them in a row one after another ? Well maybe that doesnt make any difference at all. Thanks to the flexibility of fxdreema, its easy to achieve things on very different ways.
But whats best practice ?
I saw it does not even make that huge difference if I let it run on every M15 Bar for example instead of running it every tick!? Using timer event with increased time settings did also not result in faster testing.
Cheers
-
I don't recommend using "AND" and "OR" blocks if not really needed. Especially "AND" block is not needed in most cases. There is a map of connections (an array in MQL code) and when these blocks are used they always search into this map to know what is happening. And that mapping system is very primitive, because MQL lacks of good array functions. Well, I can't explaing it very good, but the point is that when these blocks are used - more resources are used. And that should be no problem in demo/real working because even the slowest EA is super-fast in this situation, but it's important in backtesting. But this is how AND and OR are working today... maybe someday I will optimize them more, who knows. Currently I'm optimizing the website itself, MQL follows.
Formula blocks and variables are fine, there is no problem of using as many variables as needed. I mean, if you define 10000 variables it will be the same. But if there are calculations that sucks energy, like indicators, I can advice to connect them in a way that the one that gives rare signals is above. Also, these blocks like "No trade...", "Check trades count"... when they are more they can slow down the EA.
To be hones, there was no priority for EA's builded with fxDreema to be as fast as possible. The most important was the functionality and all the options. But I think that I will create functionality to track the speed of the blocks, at least this is something I want to see, so I hope that i will do something about it. The small problem is that it's not so easy for me to modify the currently existing blocks, compatibility issues
-
thanks, that answers my questions. I might look at my AND blocks, which I have a ton of ^^
Might be an issue then. Is it possible you point me in some direction regarding the array and map of connections so I could better understand ? A link to some mql4 site would be enoug to look into it. Not sure what you exactly mean. Because actually, not sure how to do my functions without AND.
I use both true and false outputs connected over many condition blocks with ANDs to cover all possible scenarios.
I could use Turn OFF/ON or Toggle block instead. Or I could modify variables with 0 or 1 if condition true / false (doing that seems to be even more slow), or I could set condition blocks in a row, like the one being less active at top, rest below. I mean its same result in row or with AND block.
Just set parallel or in a row. And I assume in a row, as it fails to get to the end of the row more often, it should be faster. Will try this.
And maybe MT4 and Strategy Tester is an issue aswell, as it seem even slower than Excel calculating
Cheers
-
Everything is in the output code, find this function named "ListOfIDsConnectedToMe". This function is automatically generated and depends on the blocks structure. Every block is represented in MQL4 as a normal function, so you have network of functions at the end. "AND" block needs to know which blocks are connected to it, so it can wait for all of them to reach it in order to pass at the end. So, every time when "AND" is executed, it searches for data in "ListOfIDsConnectedToMe", but this searching is very primitive as I already said, and it's probably slow.
"AND" block is to be used when 2 or more blocks are connected to it, but those blocks belongs to different groups of blocks. Imagine that you have group of 5 blocks on the left and group of 7 block on the right, and you want to check if you reached a point somewhere in the left group, and somewhere in the right group. Obviously it will be hard to connect some blocks one after another when they belongs to different groups of blocks, but in this case "AND" can help.
In all the normal situations I recommend to just connect blocks one after another. In this way when a block pass it immediately and directly calls the next one.
"OR" is useful when you have 2 parallel conditions and there is a chance for them to be true at the same time. If you have two blocks connected in parallel, and they both are true, each one of them will pass and will execute the next block(s) twice.
But there is a way to connect blocks in order to acheive OR functionality by using the yellow outputs. -
ok so I will adapt my EA completely and check the outcome of it
Let you know if it resulted in some breakthrough increase of the performance
-
__To be hones, there was no priority for EA's builded with fxDreema to be as fast as possible. The most important was the functionality and all the options. But I think that I will create functionality to track the speed of the blocks, at least this is something I want to see, so I hope that i will do something about it. The small problem is that it's not so easy for me to modify the currently existing blocks, compatibility issues :)[/quote:2xutqupd]
2 questions:
- Is there a way to test where the bottleneck in my ea is? Which block or blocks are slowing down the performance of my ea?
- If a block is turned off, does it affect the speed of the ea?
Thanks
-
I had this idea, to add some option to collect the time each block uses to work. Well, it was not that simple when I started doing it. I think there was some problem with the speed of execution of a single block and the precision of the function that returns time (GetTickCount()). As I remember, a single block passes too fast to be detected by this function.
-
ok, do you have any general suggestions for optimizing ea speed? Do you thinking keeping inactive blocks (turned off) slows down the ea?
-
No, I don't think so, it depends on the blocks before the one that is disabled - if they are some conditions that check something only to reach one disabled block below, then they are parasite.
Well, it's true that EAs made with fxDreema are not so fast, this is because of the universal nature of fxDreema - most of the blocks has different input parameters/options that are included in the end product, and these are also parasite.
Maybe there are things I can do to make it faster, but I've tried it before and the result was that I need to sacrifice something else
Otherwise I can suggest when you have multiple conditions to put those that are easy and simple to check at the top. Basically indicators, especially custom indicators are better to be below. Filters like "Time filter", "Once per bar", "No trade..." - on top. Also, connect blocks one after another instead of using "AND" block. -
great, thanks for the tips!
-
I can suggest something fancy, and limited, at least to speed up the initial ticks creation process
If you have enough RAM, create a RAM disk and put MetaTrader there. RAM disk is like you have HDD, but it's from the RAM and it's super fast, and it does not wear like SSD's, and it's easy to create/delete one. I personally use SoftPerfect RAM Disk, but there are other options.
I don't know if this speeds up the backtest itself, but if it reads from the disk while backtesting, then most probably it will be faster.
I tried this yesterday with 1GB fom the RAM, I was able to backtest 1 year. But if you have some more RAM it will be really niceThere is even a way to redirect some local folder to another folder - to trick Windows to think that it uses one folder, but to use another. Well, I've tried this once without success
-
that sound promising
