What do you reckon about these results?

[Harro26]

Hi all,

I am currently trading a system on the ASX that I have developed with the following backtested returns since 1/1/95:

ASX: Annual Return = 31%, Max drawdown = 27%
NYSE: Annual Return = 18%, Max drawdown = 32%
NASDAQ: Annual Return = 22%, Max drawdown = 32%
AMEX: Annual Return = 5%, Max drawdown = 32%
SGX: Annual Return = 16%, Max drawdown = 38%

The system was created against the ASX from 2000 - 2010 and I used the other markets as out of sample data to back test against. I was initially pretty happy with these results (Aside from the AMEX) but the problem is that since I began trading this on 1/1/2010 the system has not performed particularly well. It is down 13% since I began trading, and when I backtest since the start date (1/1/2010) it gives me a return of -16% I.e greater than the market decline since then and slippage/missed trades has obvioulsy worked in my favour. Also if I backtest from '95, the max drawdown (i.e -27%) occurs now with the peak occuring in October 09.
The system is a trend trading system and the market since October 09 has not been a trand traders market.
I have since set to work reducing the max drawdown and have ammended the system to give the following results since '95:

ASX: Annual Return - 33%, Max drawdown - 15% (Does not occur now)
NYSE: Annual Return - 15%, Max drawdown - 20%
NASDAQ: Annual Return - 25%, Max drawdown - 24%
AMEX: Annual Return - 3%, Max drawdown - 32%
SGX: Annual Return - 19%, Max drawdown - 31%

I am interested to hear peoples opinions on this. What do you think of the initial and new system? What would you do? Is the system over optimized to the market conditions (Even though tested against different markets and the SGX has been so different to other markets since 95 i.e. It has basically gone nowhere)? Should I pull my money and paper trade for a while? Should I stick with it and hope it turns (This is what I think I should do)? What do you think about the poor results against the AMEX?

Any feedback would be appreciated.

Cheers,

Harro

 

[tech/a]

Ok
Pretty common results for a trend trading method.
Have you been able to run a montecarlo analysis. To get an indication of minimum and maximum expectancies.

You may be unlucky enough to be forward trading your system in less than ideal markets or really unlucky and in a market not seen in testing.

You can keep trading until your actual results improve or fall out of the parameters of those returned in testing.
You can add a filter and see if that helps.
You can gain more info from Montecarlo testing.
Experiment with stops--IE width
Experiment with position sizing.


Long trading systems often have a few great winners and lots of small losses---the open profit holds the system and the psychology together.
If you have no open profit then chances are that you are trading in a period which is close to or greater than the worst experienced in the testing.

Dont give in --great time to learn.

 

[yonnie]

no surprise about your result seeing that the market has been ranging until May this year.

what you could try out is to always paper trade when the results are starting to get negative and to jump into the market when the results improve.

would that be helpful?

 

[Harro26]

Thanks Tech,

I haven't been able to run montecarlo analysis as I don't have Trade Sim, I have my positionscore set to give only one solution/trade. I have removed this and set positionscore = random()*PS and done a pseudo montecalo backtest by optimizing PS (Using Amibroker) and this gives similar annual returns but +/- a few percent. I will have another look at it and look at the drawdowns given.

I think I will continue trading the new system until the max DD reaches that reached in backtesting and then reasess what I am doing. I am still experimenting with the other things you mentioned to see if it will help.

You're right, I have learned a massive amount over the past few months that I didn't consider in system design. Namely that trading a system that suffers relatively large draw downs is hard work mentally. This has forced me to go back and have a look at reducing the max DD, when I first created the system all I was interested in was the annual return and did not worry about any of the other paramaters given (i.e max DD). I have since adjusted my setup and sell paramaters (Not my buy entry) to reduce the max DD and would have been happy to sacrifice some of the annual returns to reduce this and make it an easier system to trade. As it turned out doing this actually increased my annual returns (On the ASX) and reduced my exposure from 53% to 40% which is also a good thing (Due interest rates on margin loan etc.).


Harro


Harro

 

[yonnie]

OR

maybe look at the ASX200 index for guidance.

when this correction ends and the index is up again for a few days you might look at jumping in.

whatever you decide, there will always be cons and pros to every idea.

 

[Wysiwyg]

ASX: Annual Return - 33%, Max drawdown - 15%

I am obviously on the wrong path. After hundreds of back tests with multiple strategies I have never produced an annual return like that for 3/1/2000 to . Not even close. The best back test annual return I can get is 7% with a draw down of 9%. Maybe I should jig the strategy. Dunno what I'm doing wrong.

Strange how your system it is not working now when similar conditions prevailed in the past.

??

 

[tech/a]

Perhaps a few more tests then "W"

 

[Wysiwyg]

Perhaps a few more tests then "W"

I assume it is long only with appropriate commission and allowance for slippage so could you produce the results from 1/11/2007 to 3/3/2009 please.

 

[Harro26]

I assume it is long only with appropriate commission and allowance for slippage so could you produce the results from 1/11/2007 to 3/3/2009 please.

G'day Wysiwyg,

My system is long only with commisions applied. Slippage has not been applied.
From 1/1/2000 my system gives an annual return of 35.5%
From 1/11/2007 to 3/3/2009 my system give an annual return of 22.5%

I am no expert at trading (Given I have only recently started) but have been reasonably successful with system design and backtesting. The best advice I can give you is the following:

Of all the different systems I tried trend following was by far the most successful. There is plenty of articles on the internet regarding the Turtles trading method and I reccommend these as a start.

I don't use any of the indicators available (Stochastics, CCI etc.) except for moving averages, all of the information you need is in price and volume (Which is what creates all of the indicators anyway)

Filter your buys using the index you are trading against. I.E. buy = ma(xao,10)> ma(xao,30). Use XAO = Foreign("xao","Close") in amibroker to do this. This is why I still had a 22.5% profit during the crash as I was out of the maket for the majority of it getting back in for some quick profits during the April - May bounce.

Like I said earlier all of this is good in theory but when the market isn't moving as it hasn't since October the trend trading strategy probably isn't going to work. I am sticking with it in the hope that the market will make up it's mind and eventually move in the correct direction, up.


Hope this helps,

Harro

 

[Wysiwyg]

G'day Wysiwyg,

I am no expert at trading (Given I have only recently started) but have been reasonably successful with system design and backtesting. The best advice I can give you is the following:

Of all the different systems I tried trend following was by far the most successful. There is plenty of articles on the internet regarding the Turtles trading method and I reccommend these as a start.

Oh I have tested many trend following systems too but the consecutive losses, consistently low percentage winners (in the 30's) and excessive draw down are not good results.

Filter your buys using the index you are trading against. I.E. buy = ma(xao,10)> ma(xao,30). Use XAO = Foreign("xao","Close") in amibroker to do this. This is why I still had a 22.5% profit during the crash as I was out of the maket for the majority of it getting back in for some quick profits during the April - May bounce.

I already use an index filter on my testing. Below I have tried your MA > and the results are not good.

Typically, a moving average cross over, a moving average cross or a trailing stop is the best way I know how to exit if the trend changes!!!!!!

Like I said earlier all of this is good in theory but when the market isn't moving as it hasn't since October the trend trading strategy probably isn't going to work. I am sticking with it in the hope that the market will make up it's mind and eventually move in the correct direction, up.

If the strategy averaged 22% through the GFC then this recent market trajectory is but a blip. But that is the whole idea of back testing a strategy. We want to be as honest to ourselves as the software allows and with Amibroker you can code in complete honesty.

This is a basic trend following strategy.

Start capital 100k (set in tester). The maximum stop loss (set in tester) is set at 20% per trade. Commission is $30 (set in tester). Trailing ATR stop set at 5 * ATR(50).

PosQty = 10; 
SetOption("MaxOpenPositions", PosQty );
PositionSize = -100/PosQty; 
PositionScore = Random();
SetTradeDelays(1,1,0,0); 
V1 = EMA(C*V, 50) > 25000;

ATRMultiplier = Param("ATR Multiplier", 5, 1.5, 10, 0.5);
ATRRange = Param("ATR Range", 50, 1, 100, 1);
TrailStop = ATRMultiplier * ATR(ATRRange);
ApplyStop(2, 2, TrailStop, 1);

Call = Foreign("^AORD", "C", True);
Check = MA(Call, 10) > MA(Call, 30);

Buy = Cross(MA(C,20),MA(C,50)) & Check & V1;
Sell = 0;

 

[Harro26]

G'day Wysiwyg,

I added the following to your strategy for the purposes of comparing backtest results PositionScore = RelStrength("xao")/Close; (Just a pluck).
I then baktested your strategy against the ASX since 2000 and it gave the following results

Annual return - 3.71%, Max DD - 40%, Winners - 41%, Exposure - 63%

I then added the following sell signal Sell = Cross(MA(C,50),MA(C,20)); to give the following results:

Annual return - 6%, Max DD - 33%, Winners - 38%, Exposure - 64%

Next I added a volume indicator to the buy signal to give
Buy = Cross(MA(C,20),MA(C,50)) & Check & V1
AND Volume > MA(Volume,100);

Results returned are now:

Annual return - 7%, Max DD - 42%, Winners - 41%, Exposure - 64%

Next I looked to reduce the max DD by ammending the sell signal to the following:

Sell = Cross(MA(C,50),MA(C,20))
OR MA(Call,10)<MA(Call,30);

Reullts:

Annual return - 12%, Max DD - 25%, Winners - 45%, Exposure - 43%

Now I ammended the trailing stop as follows purely because it looked complicated:
ApplyStop(2,1,30,1);

Results:

Annual return - 18%, Max DD - 24%, Winners - 45%, Exposure - 43%

The final formula is:

PosQty = 10;
SetOption("MaxOpenPositions", PosQty );
PositionSize = -100/PosQty;
PositionScore = RelStrength("xao")/Close;
SetTradeDelays(1,1,0,0);
V1 = EMA(C*V, 50) > 25000;

ApplyStop(2,1,30,1);

Call = Foreign("xao", "C", True);
Check = MA(Call, 10) > MA(Call, 30);

Buy = Cross(MA(C,20),MA(C,50)) & Check & V1
AND Volume > MA(Volume,100);

Sell = Cross(MA(C,50),MA(C,20))
OR MA(Call,10)<MA(Call,30);

Results on other markets are as follows:

NYSE: Annual return - 6%, Max DD - 24%, Winners - 48%, Exposure - 56% (Not awesome)
NASQAQ: Annual return - 17%, Max DD - 23%, Winners - 44%, Exposure - 48% (Pretty good as the NASDAQ is down about 45% since 2000)
AMEX: Annual return - 13%, Max DD - 25%, Winners - 48%, Exposure - 41% (Much better than my system)
SGX: Annual return - 17%, Max DD - 23%, Winners - 45%, Exposure - 43%
(Again, pretty good as the SGX has basically gone nowhere since 2000)

In no way do I advocate that anyone trades this system as I only spent about 15 minutes changing it but it's looking like it has legs.

It is quite a bit different from my system but I like it's simplicity. For info I had trades set to buy and sell at the days average price. I try to mimick this in Interactive Brokers by submitting my buy at 1130 as a Relative order with 0 offset, I'm still tweeking the best way to do this (Any tips on getting the best price in IB would be appreciated).

Harro

 

[Wysiwyg]

G'day Wysiwyg,

Results:

Annual return - 18%, Max DD - 24%, Winners - 45%, Exposure - 43%

Good Day Harro.

Nuh, On the ASX (more than 2000 stocks) from 1/1/2000 to 1/1/2010 I got these results using that formula.

Annual return = 3.65%, Max DD = 20.16%, Winners = 41%, All trades = 335

Even more than before.

For info I had trades set to buy and sell at the days average price.
Harro

It is not possible except by looking into the future!! You cannot know the daily average price until the EOD. The back tester is using data already known to buy/sell a position before the day is over. In reality you would have to buy/sell next day once the average price is known.

Please inform on how to use a price before the EOD that is not known until the EOD. Thanks.

*P.S. Oh and that trailing stop had 1 winner.

 

[Wysiwyg]

Of all the different systems I tried trend following was by far the most successful. There is plenty of articles on the internet regarding the Turtles trading method and I reccommend these as a start.

The Turtle Trading System is a Breakout System based upon a Donchian Channel breakout.

"The Turtle System enters on breakouts. Most breakouts do not result in trends. This means that most of the trades that the Turtles made resulted in losses."

I recommend the Turtle Trading System if one is trading futures (which are the markets they traded) and one has deep pockets as well as strong mental stability.

Thank you.

 

[Harro26]

Wysiwyg,

That is puzzling. I have checked all of my settings etc. and they all appear to be fine, I have attached them if you would like to check them we may be able to help each other out (Note - You will have to change the file extension back to .ABS, I changed it to .DOC to allow it to be uploaded, not sure if it will still work but give it a go).

Are you including slippage in your results?

Are you backtesting historical asx data or current day asx data, I am backtesting current day data? (Obvious limitations involved that I am exploring).

Did you convert XAO (Under call and Positionscore) back to ^AORD, I forgot to mention this.

The only other thing I can think of is that we are testing different data. I have included my list of ASX FPO, If you could run a backtest against it and see what happens perhaps we can get to the bottom of this, you've got me worried now that the error may be from my side.

As for trade entry timing, if i set it to by in at Open vice average it actually gives about a 1% better annual return. The problem is that getting in at open is very difficult as I work during the day, so if I just set a limit order I may miss the trade or end up paying a premium on this if it was to open below what I was expecting. The idea behind me submitting a relative order at 1130 with 0 offset is that I will "on average" get in at the average price. Sometimes I will pay more and sometimes I will pay less but it will all come out in the wash. It is a work in progress and I am currently experimenting with this to see what the actual outcome is.

Let me know how your results go,

Harro

 

[Wysiwyg]

Wysiwyg,

That is puzzling. I have checked all of my settings etc. and they all appear to be fine, I have attached them if you would like to check them we may be able to help each other out (Note - You will have to change the file extension back to .ABS, I changed it to .DOC to allow it to be uploaded, not sure if it will still work but give it a go).

Tried the .abs extension and a few programs to open it but illegible. ASCII format?? Sidestep that one.

Are you including slippage in your results?

I add extra onto the brokerage.

Are you backtesting historical asx data or current day asx data, I am backtesting current day data? (Obvious limitations involved that I am exploring).

Yes

Did you convert XAO (Under call and Positionscore) back to ^AORD, I forgot to mention this.

Yes

The only other thing I can think of is that we are testing different data. I have included my list of ASX FPO, If you could run a backtest against it and see what happens perhaps we can get to the bottom of this, you've got me worried now that the error may be from my side.

I have downloaded data from 1985 to now from the list you sent except for the five letter ones. I think they are options or warrants on shares.

As for trade entry timing, if i set it to by in at Open vice average it actually gives about a 1% better annual return. The problem is that getting in at open is very difficult as I work during the day, so if I just set a limit order I may miss the trade or end up paying a premium on this if it was to open below what I was expecting.

I see. Good luck with that idea but for back testing I can't see any benefit from trying to buy/sell prior to knowing what the price is.

Let me know how your results go,

Harro

This is the results page using the Relative Strength position score. Slightly varies with the random position score.

 

[Harro26]

I've got no idea. My results are attached. Even if I add .5% slippage/buy and sell I still get 15% annual return. .ABS = amibroker backtest settings, you just open the settings window, click load and select the file.

What results are others getting when they backtest the given formula?

 

[Wysiwyg]

I've got no idea. My results are attached. Even if I add .5% slippage/buy and sell I still get 15% annual return. .ABS = amibroker backtest settings, you just open the settings window, click load and select the file.
What results are others getting when they backtest the given formula?

Good Evening.

I have established a difference with the "Trades" data you posted.

None of our trades match and that is because I have Yahoo data and you have Premium data. That should be okay because the results should be at least nearer. On your big winner, $110,000 for SDL, my tester did not take the trade and that would be because other trades were open at that time which comes back to dissimilar data.

With SDL, I checked if my MA's crossed on the 29th March 2007 and they did indeed so the trigger was there but the back tester did not take it.
Actually my back tester appears to take the worst of trades. Strangely the largest test winner was only $8000. I had commission set at 0.08% too.

It is most likely the different data but surely it would not be that detracting on results. I am going to hang onto some of those strategies I rejected due to mediocre results.

Harro, my thanks for helping sort this huge difference in results out.

 

[Wysiwyg]

Good Evening.

I have established a difference with the "Trades" data you posted.

On your big winner, $110,000 for SDL, my tester did not take the trade and that would be because other trades were open at that time which comes back to dissimilar data.

With SDL, I checked if my MA's crossed on the 29th March 2007 and they did indeed so the trigger was there but the back tester did not take it.

Aha. You are not getting away that easy. On further investigation of why SDL was not taken. I ran a back test from the period in which SDL was taken by your tester. Still, with a full account SDL was not traded. I thought what could this be.

Your additional criterion, the Volume > MA(Volume,100); was not met on the 29th March 2009 according to my 100 Period MA of the volume.

An independent 100% reliable source shows me that on the day of the signal, the 29th. March 2007, the volume was low. Additionally, your results show open price of 13c on the 30th. when the open was 12.5c. The signal day had an open & high of 13c. No 13c on trade day the 30th.

This is critical to know. It is the defining factor on data difference.

Was the volume on this day, the 29th. March 2007, above your 100 MA?

 

[RazzaDazzla]

Wysiwyg,


As for trade entry timing, if i set it to by in at Open vice average it actually gives about a 1% better annual return. The problem is that getting in at open is very difficult as I work during the day, so if I just set a limit order I may miss the trade or end up paying a premium on this if it was to open below what I was expecting. The idea behind me submitting a relative order at 1130 with 0 offset is that I will "on average" get in at the average price. Sometimes I will pay more and sometimes I will pay less but it will all come out in the wash. It is a work in progress and I am currently experimenting with this to see what the actual outcome is.

Let me know how your results go,

Harro

Not sure if I understand correctly, but why wouldn't you just have your system buy in the opening auction? If you put a limit order in, you would just be filled at the opening price in the auction up to your limit price would you not?

Or am I completely missing something?

 

[Harro26]

Aha. You are not getting away that easy. On further investigation of why SDL was not taken. I ran a back test from the period in which SDL was taken by your tester. Still, with a full account SDL was not traded. I thought what could this be.

Your additional criterion, the Volume > MA(Volume,100); was not met on the 29th March 2009 according to my 100 Period MA of the volume.

An independent 100% reliable source shows me that on the day of the signal, the 29th. March 2007, the volume was low. Additionally, your results show open price of 13c on the 30th. when the open was 12.5c. The signal day had an open & high of 13c. No 13c on trade day the 30th.

This is critical to know. It is the defining factor on data difference.

Was the volume on this day, the 29th. March 2007, above your 100 MA?

That's interesting, my data has a volume on the 29th of 20 650 000 with a 100 day MA of 11 748 000. I'd like to think premium data is more accurate than Yahoo data but perhaps not. My results show an open and buy prcie of 12.5c on the 30th which is obviously what my data says also, not sure what you were saying WRT that.

Regardless, with SDL excluded from the backtest the annual return is still 16% and with position score set to random and 10 backtests run I get the following results:

Max Net % profit - 676%
Min Net % profit - 213%
Average Net % profit - 405%

So I don't think we have got to the bottom of this.

Again, if anyone has run the formula above I'd be interested to hear what your results were.

Thanks.