Australian (ASX) Stock Market Forum

Optimizing optimization in Amibroker

Jump to new
T

telstrareg

Joined
24 April 2007
Posts
68
Reactions
0
I’m just flying by the seat of my pants here as system development infant. To me it seems sensible to run a coarse optimization of the variables in order to identify ranges of interest, and then use smaller variable intervals inside the ranges that look sexy. Making sense so far?

However the problem is that the optimizer only increments linearly. That’s not a big deal if you were say optimizing a variable range 1000 to 1100 in increments of 5, but it’s very inefficient if your variable is range 0 to 80 inc 5. ie in the sequence 0, 5, 10…..75, 80, the 75 to 80 step is proportionately much smaller than the 5 to 10. So my coarse optimization would be too course for the low numbers and needlessly fine for the high numbers.

So have any of you developed a way around this in Ami? Conceptually I understand that you would have to create a function that spread the steps in some kind of geometric progression, but being mathematically challenged I don’t have clue how to approach it, nor am I sure it’s even possible given the constraints of the optimize function.

Given that optimization is the most time consuming thing in Ami, I think it’s well worth sorting this out early on.
 
I think you'd really have to run the optimize in multiple passes, each time narrowing your 'step' parameter and it's min/max range. Sounds like a lot of trouble - as you say, the optimize function will only accept a simple number as an incrementor, nothing you can do to change that.

I'd just give it more time to run - I've left my laptop crunching away overnight sometimes for this. But if the parameters you are using need such precise values you'll get 3d optimization maps which have very sharp peaks, something to avoid, ie they'll be overly curve-fitted.
 
mrWoodo said:
the optimize function will only accept a simple number as an incrementor, nothing you can do to change that.
Click to expand...

Yeah unless there is some way to do what optimize does without using the optimize function, I can't see a way round it.

Although I spotted the following in the code ref and I'm wondering if this has anything to do with what I'm talking about (don't really understand it)

------------------------------------------
You can Optimize parameters with custom number series by using the numbers generated by the Optimize() function as an index to access numbers in a custom array. Here is an example using a custom array FB[] of Fibonacci numbers:

FB[0] = 0.0; FB[1] = 23.6; FB[2] = 38.2; FB[3] = 50.0; FB[4] = 61.8; FB[5] = 100; FB[6] = 161.8; FB[7] = 261.8; FB[8] = 423.6;
FBindex = Optimize("FBindex",0,0,8,1);
FibNum = FB[FBindex];
... place your Code using FibNum here ...
 
------------------------------------------
You can Optimize parameters with custom number series by using the numbers generated by the Optimize() function as an index to access numbers in a custom array. Here is an example using a custom array FB[] of Fibonacci numbers:

FB[0] = 0.0; FB[1] = 23.6; FB[2] = 38.2; FB[3] = 50.0; FB[4] = 61.8; FB[5] = 100; FB[6] = 161.8; FB[7] = 261.8; FB[8] = 423.6;
FBindex = Optimize("FBindex",0,0,8,1);
FibNum = FB[FBindex];
... place your Code using FibNum here ...
Click to expand...

I just saw that in the manual :) Pretty neat !
So if you had an indicator that was 0-100, and you wanted to narrow in on say 70-80% you could have something like :

setting[0] = 0;
setting[1] = 10;
setting[2] = 20;
setting[3] = 30;
setting[4] = 40;
setting[5] = 50;
setting[6] = 60;
setting[7] = 70; //narrowing in from here
setting[8] = 72;
setting[9] = 74;
setting[10] = 76;
setting[11] = 78;
setting[12] = 80; //going back to steps of 10
setting[13] = 90;
setting[14] = 100;

index = Optimize("Settings",0,0,14,1);
IndicatorValue = setting[index];
 
mrWoodo said:
I just saw that in the manual :) Pretty neat !
So if you had an indicator that was 0-100, and you wanted to narrow in on say 70-80% you could have something like :

setting[0] = 0;
setting[1] = 10;
setting[2] = 20;
setting[3] = 30;
setting[4] = 40;
setting[5] = 50;
setting[6] = 60;
setting[7] = 70; //narrowing in from here
setting[8] = 72;
setting[9] = 74;
setting[10] = 76;
setting[11] = 78;
setting[12] = 80; //going back to steps of 10
setting[13] = 90;
setting[14] = 100;

index = Optimize("Settings",0,0,14,1);
IndicatorValue = setting[index];
Click to expand...

You mean by using the comment lines to narrow the range?
 
telstrareg said:
Fvck, my ADD is really bad today. Could you explain conceptually what's going on here, please?
Click to expand...
:D K, will try - What he's doing is using the optimizer to pull figures out from a predefined list. (In the original, it was fibonacci numbers).

In my prev post, I'm declaring an array that has 15 values in it (tho it's numbered 0 to 14, all this AFL stuff is zero-based). Now it doesn't matter what the values in the array are, could be linear or not.

index = Optimize("Settings",0,0,14,1);

See how the min and max are 0 and 14? The optimizer is being used merely as a counter which corresponds to the size of the array. It's this line :

IndicatorValue = setting[index];

that fetches the real 'test-value'. It refers back to what 'run' we are in (from the Optimize statement) and gets the corresponding value. Eg The first time round the Optimize loop, we are at Setting '0', so we fetch '0%'. Second time round, we are trying Setting '1', which has a value of '10%'.

setting[0] = 0;
setting[1] = 10;
...
...

All the way up to our last setting[14], which has a value of 100%.
God this is difficult to explain :banghead: Hope this makes sense!
The thing to note here tho, is that the list of what values to try and run had to be predefined by the user. I _think_ this is what you're after, ie if you don't want to waste time making ami go through very granular steps from say 0-70 and 80-100 when you already know that 70-80 is the zone of most interest.
 
EDIT, sorry I see youve already posted an answer

I think the Ritalin is starting to kick in. If I understand right, the optimize function is now stepping through some kind of custom array and then sending the results to the indicator variable in question, and then on to automatic analysis? What do square brackets mean?

EDIT
Ok the brackets are for the array indexes, sorry for all the dumbass questions :D
 
CanOz said:
You mean by using the comment lines to narrow the range?
Click to expand...
Hi CanOz, nope - that was just to try and make a bit clearer - Hopefully my prev post explains it a bit better. Been reading your mechanical system thread, good stuff ! - Trying to make the time to get more into it but god it's difficult with a full time job eh :)
 
mrWoodo said:
Trying to make the time to get more into it but god it's difficult with a full time job eh :)
Click to expand...

Yes indeed, i know the feeling...full time job, descretionary trading system to manage, GF to manage:D, Amibroker to learn, ridiculous threads to read on ASF....i could go on!

Got two weeks off soon Mr.Woodo, hopefully you can drop bythe thread one i get it going again.

Cheers,
 
OK, MrWoodo’s code works like a charm. To make this method of optimization practical, I’m proposing how I think it might work conceptually. Any help with the code would be appreciated.

Rather than working with predefined arrays which need to be manually changed each time, we should just calculate the optimal geometric progression on the fly from the low and high values and the desired increments.

A function which:

1. We enter the number of increments we want to have for a particular variable
2. Performs a calculation on the low number and high number of that range to determine the range values (based on the desired number of increments) which create the most favourable distribution.
3. Rounds the range values to the desired number of decimal places.
4. I guess we’ll need some kind of loop to create the array required.

The mathematicians among you might have some understanding of what is the best geometric progression eg. if we were looking at lower and upper values of 2 and 128, and 5 increments we would want to generate an array something like this:

2, 4, 8, 16, 32, 64, 128

But that’s an easy example using powers of 2, I would have no idea what kind of formula to use if the low and high values were 17 and 130 and we wanted 19 increments.

Given the number of clock cycles one could save over the years, I really think this is worth it.
 
Top