Optimization speed improvements
-
Hi everyone,
I've wrote and been using this EA successfully for the past year, but large optimizations take v long time, even on a 24 cores, 32 threads i9 new laptop. Although it's a huge improvement (over 10 times faster, so now I can do 24h instead of 10 days!), I'm looking for ways to improve the code.
I saw a recommendation to edit the code and take out of the onTick stuff like once a bar conditions, do labels instead of comments, etc. I'm not sure about what goes where in the code, that's why I'm using this awesome fxdreema platform.
So I'm wondering if I could change around some pieces of my project and place them under different events (tabs) other than onTick.
Can you advise on what could go under onInit or onTimer or onTrade, etc...? Would that make a difference, or, since they are under Once per bar / Once a day blocks, they would not be called every single tick?
Thank you!
image url) -
@Bogdaiki
Can't really say what is going on without having access to inside the blocks but first, there is no display during optimization. Prints or draws are not executed thus make no optimization difference.
The number of cores on CPUs is limited compared to GPUs. You can split the code of the EA to be executed on several GPU cores but it is not much worthy as it will be much time consuming to code. Moreover on large sets of parameters it is the optimizer algorithm that is the most time(power) consuming, not the code of the EA itself. And it doesn't allow for GPU calculation. Still there are other solutions like the MQL5 cloud network.
Placing the EA under other tabs will make no speed difference in this case. Nor declaring variables on "on Init" really it is more about code cleanliness even though fxdreema code is far to be optimized. -
@seb-0 you are right FX is not clean or optimised, to me this is the trade-off to manually coding on MT, I think it has a lot to do with being able to add different lines of code quickly.
-
@Bogdaiki I'm not a programmer, so I cannot be any helpful here except for recommending what I do when I I find a successful strategy on an FXDreema bot. I simply hire a real programmer to make it rebuilt from scratch manually.
That makes all the difference. Some of my bots in fxDreema are using more than 60000 lines. Once they are manually rebuilt, they need less than 2000. And trust me, optimizing 2000 lines code is FAR faster than optimizing 60000.
-
@l-andorrà
Interesting, especially if these 60000 lines are executed every call. How much faster ? Have you compared the optimization time of both versions ? -
You are probably totally aware of the following but the easiest way to reduce optimization time is to reduce the amount of optimization passes.
If there are 5 parameters to optimize and each parameter has 100 steps it will produce 100 x 100 x 100 x 100 x 100 = 10,000,000,000 passes when using the slow complete algorithm method.
If it can be reduced to 4 parameters then it will produce 100 x 100 x 100 x 100 = 100,000,000 passes.
If it can be reduced to 50 steps then it will produce 50 x 50 x 50 x 50 = 6,250,000 passes.
So, the original example has 1,600 times more passes than the last.
This is a very exaggerated example but it highlights the effect of reduced (efficient?) parameter selection. -
@albyxx
Indeed, the number of combinations is : ((stop-start)/step+1)^parameters.
Increasing the step size decreases exponentially the number of combinations, and at the same time it reduces the over-fitting(which is good) but also the finesse.
So too thin steps induces too much computational costs and high over-fitting probability, while too thick steps induces a lack of finesse(accuracy).
Better using the fast genetic algorithm though. -
The EA doesn't call all lines at the same time it will call the lines depending on where they are in the code, many like the code on init only ruuns on start-up, on tick will run on every tick, unless you have something to stop it. That being said though there is still a lot that will run every tick.
-
@seb-0 The fxDReema bot needed one week to do 1,200 iterations. The manually built bot using the same input parameters, same testing period, etc. hit 115,000 iterations in the same period. Simply no contest.
-
Thank you all for your inputs!
@l-andorrà how much would one charge to rebuild something like I show in the screenshot? Everything is based on an indicator native to MT5, with some candle conditions and calculates the balance every day, taking profits or stopping losses using a variable set percentage. -
@Bogdaiki That will depend on the programmer doing the job. I cannot provide an approximate price because I'm not a programmer so I have no idea. I strongly recommend you asking for several quotes to different programmers before hiring one. That way you will get an approximate idea of the average price.
