ASX 200 Monthly Momentum Strategy - Review and Insights

[vharik831]

Hi All,

I have attached the strategy backtest report from Amibroker for a monthly strategy I am working on.

Whilst the results look promising, I wanted to get feedback on the stats, where you see issues.

I am concerned about the Monte carlo analysis.
I donot fully understand on how to interpret Monte carlo analysis results. But i do see a lot of zeroes in there
So, would like some help from the experienced. Is it a risk to trade this system as is?

PS: I had to attach the report in zip format as HTML attachments are not accepted. Let me know if you are concerned with opening zip files. I will attach image files.

Regards,
Hari

 

[MovingAverage]

nothing attached

 

[vharik831]

nothing attached

attached now.

 

[MovingAverage]

First, MC analysis is very important as it allow you to fully understand how your system performs with selection bias. Second, the way MC is done in AB is pretty poor and rudimentary at best. Best way to do MC analysis is add the code "mtRandom() > 0.7" to your "buy" line. Insert a dummy parameter and optimize it over say 1000 or more runs. Export the optimization output into a CSV file and then import into Excel. Then using the Data Analysis plugin select an output parameter (say net profit or system draw down) and put that into a histogram. What you will then get is a complete picture of how your system performs with selection bias. This is much more insightful than the built in MC of AB. Also the way AB does MC is a little questionable. Here is a few pics of a system I recently back-tested using the same technique. For simplicity I'm only including net profit and system drawdown but you can pretty much review any back-test output stat from AB to get a complete picture of how your system performs with selection bias. Like I said, MC testing is extremely important if you really want to understand the behavior of your system. It is very hard to judge a system's performance based on a single run.

 

[vharik831]

First, MC analysis is very important as it allow you to fully understand how your system performs with selection bias. Second, the way MC is done in AB is pretty poor and rudimentary at best. Best way to do MC analysis is add the code "mtRandom() > 0.7" to your "buy" line. Insert a dummy parameter and optimize it over say 1000 or more runs. Export the optimization output into a CSV file and then import into Excel. Then using the Data Analysis plugin select an output parameter (say net profit or system draw down) and put that into a histogram. What you will then get is a complete picture of how your system performs with selection bias. This is much more insightful than the built in MC of AB. Also the way AB does MC is a little questionable. Here is a few pics of a system I recently back-tested using the same technique. For simplicity I'm only including net profit and system drawdown but you can pretty much review any back-test output stat from AB to get a complete picture of how your system performs with selection bias. Like I said, MC testing is extremely important if you really want to understand the behavior of your system. It is very hard to judge a system's performance based on a single run.

View attachment 120222

View attachment 120223

Thank you! Appreciate your response and I will give this a go.
After I get the buy setups, I use position score to rank the stocks. Then i pick the top 10 stocks to start with and as I exit positions, i take the top stock, ranked by the position score, every month.

So, what I am trying to figure out is which variable Monte carlo is trying to simulate? Is it just the timing of which month I start trading my system? I believe picking of the stocks from a bucket of stocks is no longer random due to ranking the stocks.

Regards,
Hari

 

[Warr87]

if its a rotational monthly system then it hinges on your ranking method. improving that will change the stats.

from whats presented it appears to be a good system, though I'm not sure how you got such a low MDD from 13yrs of backtesting. seems unusually low.

 

[MovingAverage]

Thank you! Appreciate your response and I will give this a go.
After I get the buy setups, I use position score to rank the stocks. Then i pick the top 10 stocks to start with and as I exit positions, i take the top stock, ranked by the position score, every month.

So, what I am trying to figure out is which variable Monte carlo is trying to simulate? Is it just the timing of which month I start trading my system? I believe picking of the stocks from a bucket of stocks is no longer random due to ranking the stocks.

Regards,
Hari

Without getting too complex, the MC in AB is not simulating any variables. For want of a better description it just does a shuffle of the order in which stocks are taken. In the mtRandom() example I did your simulation will make a binary decision as to whether an entry should be taken so what you get with the dummy optimize is a a bunch of different stock selections if that makes sense.

 

[vharik831]

if its a rotational monthly system then it hinges on your ranking method. improving that will change the stats.

from whats presented it appears to be a good system, though I'm not sure how you got such a low MDD from 13yrs of backtesting. seems unusually low.

Thanks Warr87. MDD is fluctuating based on the year I start. The back test report started on Jan 2008. But if I had started on Jan 2007, MDD became -33%. This was due to the outlier year - 2008.

 

[Warr87]

start time is definitely a factor. even sharpe ratio (if you like that metric) will drastically change if you happen to backtest right before a 'black swan' event. but that's also why the MDD being less than 20% over such a long period of time seems low. but i don't know what kind of filters you have in place. and it by no knows your results aren't valid.

looking at an average DD could also be important to have a realistic understanding of how often, or to what degree, you are in a DD. MDD is obviously the worst case scenario.

im currently really likely my own monthly momentum system, though mine is 'riskier' than yours (I removed index filters), though my return is higher. it has been trading pretty well right now.

my own system, in my testing, appeared to fair better with more positions as well. something for you to consider. (I currently run it with 10 positions, but even up to 40 positions it has reduced MDD likely due to a spread risk amongst more positions.)

 

[vharik831]

start time is definitely a factor. even sharpe ratio (if you like that metric) will drastically change if you happen to backtest right before a 'black swan' event. but that's also why the MDD being less than 20% over such a long period of time seems low. but i don't know what kind of filters you have in place. and it by no knows your results aren't valid.

looking at an average DD could also be important to have a realistic understanding of how often, or to what degree, you are in a DD. MDD is obviously the worst case scenario.

im currently really likely my own monthly momentum system, though mine is 'riskier' than yours (I removed index filters), though my return is higher. it has been trading pretty well right now.

my own system, in my testing, appeared to fair better with more positions as well. something for you to consider. (I currently run it with 10 positions, but even up to 40 positions it has reduced MDD likely due to a spread risk amongst more positions.)

great inputs. Thanks. Yes, I played with the number of positions and 10 is the sweet spot for my system as well. I dont have an index filter, but my exit is tight. I get out on the first sign of weakness. However, due to the system being monthly, first sign of weakness is often too late!

Tried adding an index filter to entry, but it reduces the returns drastically, which could be due to, missing out on the initial part of the recovery cycle.

Yes, I look at the sharpe ration and you are right. When you start the system is critical. Irrespective of when I start the first year is either a looser or is marginally profitable.

I am not yet in that frame of mind to trade monthly, which requires a lot of patience. I trade a daily system and I feel a lot more in control.

 

[Warr87]

I noted in my own trading that monthly can be great, but as you say, when there is weakness it can be too late. I've entered into positions and a week later, they all dropped dramatically. but that is part of the game really. there is a trade off for it requiring less time to manage.

 

[vharik831]

First, MC analysis is very important as it allow you to fully understand how your system performs with selection bias. Second, the way MC is done in AB is pretty poor and rudimentary at best. Best way to do MC analysis is add the code "mtRandom() > 0.7" to your "buy" line. Insert a dummy parameter and optimize it over say 1000 or more runs. Export the optimization output into a CSV file and then import into Excel. Then using the Data Analysis plugin select an output parameter (say net profit or system draw down) and put that into a histogram. What you will then get is a complete picture of how your system performs with selection bias. This is much more insightful than the built in MC of AB. Also the way AB does MC is a little questionable. Here is a few pics of a system I recently back-tested using the same technique. For simplicity I'm only including net profit and system drawdown but you can pretty much review any back-test output stat from AB to get a complete picture of how your system performs with selection bias. Like I said, MC testing is extremely important if you really want to understand the behavior of your system. It is very hard to judge a system's performance based on a single run.

View attachment 120222

View attachment 120223

I now have these for my system. Troubling part is the net profit that I get when I run the system for the same time frame(as the optimization) is not even reported in the optimization reports!

If I am interpreting the net profit % graph correctly, does that say that the system is more likely to generate profit between 382% and 800%?

 

[MovingAverage]

I now have these for my system. Troubling part is the net profit that I get when I run the system for the same time frame(as the optimization) is not even reported in the optimization reports!

If I am interpreting the net profit % graph correctly, does that say that the system is more likely to generate profit between 382% and 800%?

View attachment 120236

View attachment 120237

While the Net Profit % is not a gaussian distribution I reckon your estimate of 382 to 800 is probably on the money. Your Sys Drawdown looks to be pretty much gaussian so so I reckon you're on the money to expect 2SD of 17% DD. That's a pretty good looking Sys Drawdown for a monthly system. As a general rule the lower the Sys Drawdown the lower the net profit. I must admit for 13 years of trades a net profit of 392 to 800 is a little on the low side, but hey you have an impressive DD so that is to be expected.

 

[MovingAverage]

BTW, how many trades did your system do in that 13 year period? The other reason I was reluctant to go live with my monthly system is I felt the number of trades wasn't statistically relevant so my level of confidence in the back testing was low

 

[vharik831]

BTW, how many trades did your system do in that 13 year period? The other reason I was reluctant to go live with my monthly system is I felt the number of trades wasn't statistically relevant so my level of confidence in the back testing was low

Number of trades are about 350 over 13 years, that is up to 2 trades on an average per month, with 10 open positions most of the time.

Thanks for your analysis of the histograms.