Amibroker and Monte Carlo backtesting - I think I've cracked it

[Harro26]

I've only had Amibroker for about a month and have found it priceless in developing and testing trading strategies. The one down fall that I have found is that you cannot do any form of Monte Carlo backtesting. To date I have been setting my positionscore to random, running numerous back tests, logging the data in an excel spreadsheet and comparing results. this has been very time consuming.
I think I have found a simple way to do some pretty good Monte Carlo back testing in Amibroker, I caveat this info with the following:
1. Like I said, I have only had Amibroker for a month so am no expert.
2. I may be re-inventing the wheel but I didn't know that you could do this so thought i'd post it to help others like myself that are new to Amibroker
3. There have been posts on this before but this seems to be a simpler way to do it than I have seen before.
4. I've never seen Monte Carlo testing before so this may not give you all of the bells and whistles of full up Monte Carlo testing.

Anyways here it is:
Write this line of code into the top of your formula:

PS=Optimize("Postion Score",1,1,y,1);
PositionScore = Random()*PS;

Where y is equal to the number of Monte Carlo tests you want to run. I.e if you want to simulate 1000 set of trades enter 1000 etc.

Now hit the analysis button and instead of clicking on Back test click on Optimize, this will give you the results of all the tests. You can copy this into excel and create graphs etc.

Harro

 

[Chorlton]

I've only had Amibroker for about a month and have found it priceless in developing and testing trading strategies. The one down fall that I have found is that you cannot do any form of Monte Carlo backtesting. To date I have been setting my positionscore to random, running numerous back tests, logging the data in an excel spreadsheet and comparing results. this has been very time consuming.
I think I have found a simple way to do some pretty good Monte Carlo back testing in Amibroker, I caveat this info with the following:
1. Like I said, I have only had Amibroker for a month so am no expert.
2. I may be re-inventing the wheel but I didn't know that you could do this so thought i'd post it to help others like myself that are new to Amibroker
3. There have been posts on this before but this seems to be a simpler way to do it than I have seen before.
4. I've never seen Monte Carlo testing before so this may not give you all of the bells and whistles of full up Monte Carlo testing.

Anyways here it is:
Write this line of code into the top of your formula:

PS=Optimize("Postion Score",1,1,y,1);
PositionScore = Random()*PS;

Where y is equal to the number of Monte Carlo tests you want to run. I.e if you want to simulate 1000 set of trades enter 1000 etc.

Now hit the analysis button and instead of clicking on Back test click on Optimize, this will give you the results of all the tests. You can copy this into excel and create graphs etc.

Harro

Hi Harro,

May I ask a few questions:

1) Firstly, why do you want to run Monte Carlo in the first place? What are you actually trying to assess & test for? For example, is it a real-life scenario?

2) If it is a possible "real-life" scenario (eg. do not have enough current funds to take all new trade signals) then does your approach really mimic this scenario and accurately tests for it?

regards,

Chorlton

 

[Harro26]

Hi Harro,

May I ask a few questions:

1) Firstly, why do you want to run Monte Carlo in the first place? What are you actually trying to assess & test for? For example, is it a real-life scenario?

2) If it is a possible "real-life" scenario (eg. do not have enough current funds to take all new trade signals) then does your approach really mimic this scenario and accurately tests for it?

regards,

Chorlton

G'day Chorlton,

I am testing a few new trading strategies and want to know how they would have performed in the past. The problem is that I have $100000 to trade and am allowing myself to commit 10% of total equity/trade. On any one day I will recieve numerous entry signals and depending on which of these I choose to take will drastically alter the final outcome. For example, I may have 100% of my equity commited to trades, I than sell my shares in 1 company and the next day I run my system and it gives me 5 different buy signals but I can only take 1 of these, the trade that I choose to take will have a butterfly effect on all future trades. Conducting Monte Carlo backtesting (As I understand it) allows me to examine all of the different possible iterations. If I run the back test 10000 times using this method I will get a good indication on the best, worst and average performance of my method over the testing period. The results vary drastically (From negative to 2000% returns in some cases).

The backtesting will never mimic real life trading as I will alway use my discretion and buy what I think is the best trade. What it will tell me is if I nail it and make all the right trades what will my return be, If I completeley balls it up and make all of the wrong trades (More likely) what will my returns be and if i make 50% of the correct trades what will my returns be.

My 2 main objectives/rules in creating a mechanical trading system are as follows:
1. The system must make sense. It is possible to make any system make money with backtesting and optimization, if the system does not make sense but it is still making money in the back testing you have more than likely over optimized it.
2. The system must outperform the market 100% of the time (Or as close to as is possible) during back testing.
This is where the Monte Carlo backtesting comes in. I have run my current system thousands of time over different time periods and it outperforms the market >99% of the time (I won't say 100%).

 

[Chorlton]

Harro,

Firstly, I totally understand why you would want to use Monte Carlo in your backtesting. I also use it and believe it is a vital exercise for assessing the effectiveness of a system which will trade a portfolio of stocks. In such circumstances the chance of taking every signal is relatively low (although as a caveat using leverage may improve this), so one needs some level of confidence as to how it would perform regardless of which stocks were taken.

However, the reason for my questions in my original post is that although there are many ways to carry out Monte Carlo analysis, the results are not aways that useful, as the way it is performed needs to mimic as closely as possible real trading scenarios.

This link should hopefully explain in more detail the issue that I am referring to.
http://www.compuvision.com.au/Examples/PortofolioOptimizationMyth.htm

As a side note, I am not associated with this product / Company (although I have personally purchased it and use it in my own system development) nor am I suggesting that you should buy it.
However, I thought I would post the link as I believe you may find the information useful ....

 

[Harro26]

Hi Chorlton,

Thanks for the link. I'm not fully swept up on Tradesim and I assume it is a great product because I know alot of people are using it, the method of backtesting in Amibroker that I have discussed simply ads a random variable to which stock it selects. I don't believe it is swaying the results through "time series shuffling or synthesis of trades using random generators and price distributions" (From the link). Each run simply selects a different set of trades each time if there are more available than your position sizing and equity allow.
No optimisation takes place, the optimisation function is simply used to create a random number generator, this determines which trades are entered.
From my limited experience I believe optimization can be a deadly weapon if not used correctly, whenever I optimize a paramater I run it numerous times and set my position score to random such that I get different results everytime. I than look at the group of results that give me the most consistent profits.
I believe that after Optimizing a system you should be able to change the paramaters to sensible figures around what the optimization gave you and still outperform the index. If this does not occur I go back to the drawing board.

 

[Wysiwyg]

Anyways here it is:
Write this line of code into the top of your formula:

PS=Optimize("Postion Score",1,1,y,1);
PositionScore = Random()*PS;

Where y is equal to the number of Monte Carlo tests you want to run. I.e if you want to simulate 1000 set of trades enter 1000 etc.

Now hit the analysis button and instead of clicking on Back test click on Optimize, this will give you the results of all the tests. You can copy this into excel and create graphs etc.

Harro

Harro, the results are repeated from every run. Does this happen for you?

 

[Harro26]

Wysiwyg,

no, every run gives me different results. All I can think of is it is something to do with you position sizing, if you always buy into every trade. Perhaps your system doesn't give you more entry signals on each day than your position sizing allows. Might be something to do with settings that I am not aware of.
Not sure. If you set positionscore=random(); everytime you hit Back Test it should give you different results.