This is a mobile optimized page that loads fast, if you want to load the real page, click this text.

System Robustness

For those who have Amibroker and are interested some code for computing the e-ratio live at:

http://theasxgorilla.blogspot.com/2007/09/e-ratio-40w-moving-average-system-code.html

If anyone reviews this and finds errors please let me know as I've developed a winning system based on it and I'm in the final stages of re-mortgaging and going 98% LVR on my house to raise funds for trading...joking, but seriously, if you find errors please post a comment and I'll revise the code.
 
If anyone reviews this and finds errors please let me know as I've developed a winning system based on it and I'm in the final stages of re-mortgaging and going 98% LVR on my house to raise funds for trading...joking,

LOL you almost gave me a heart attack!!
 

Hi Howard.

Firstly, thanks for contributing another top post, i like especially the discussion on exits.
The points 2 and 3 on exits is something that im going to start working on shortly.

Just one comment on what you said about trend following systems (Iv highlighted this section in your post).

Dont you think that if you have a good entry, like that used in the study by Blackstar, when a stock makes an all-time high, that this would still perform well in sideways or bearish periods? (Not outstanding, but can beat the market perhaps?)

A stock making an all time high in a bearmarket or sideways market is a rare find, but one that does so is likely to be a champion. Say if in a bullmarket you have 50 stocks a year making all time highs, and in a bearmarket/sideways market you have only 3-4, if your entry is good enough to identify these winners, and you set your system to enable pyramiding, then theres a good chance that you will find your portfolio full of large positions in very few (champion) stocks.

The above is just my opinion, alot of it still hasnt been tested.

Comments and further discussion very much appreciated.
 
Should have been more clear - by 'Black Box' I was refering to purely mechanical systems.
 
"Black Box" means undisclosed mechanics of a system.
A Purely mechanical system need not be a Black Box.
 
Ok i did a walk-forward analysis on my system and it performed well, as was expected, because of the bullish market conditions.

The initial period for testing and optimisation was 01-01-1998 until 31-12-2003 (6 years).

The 3.5 year period from 01-01-2004 until 30-06-2007 was left untouched (out-of-sampel) until this morning when the walk-forward analysis was conducted.

Now the system is essentially a long-term trend following system with average holding time for winners to be about 200 days (6-7 months). Note that several of the big winners usually last for a year or longer.

To be able to compare the returns from my system to that of the market (XAO) on an annual basis, I told TradeSim to close out ALL TRADES at the end of the year (using the OR SetTriggerAtDate function as part of the ExitTrigger).

Now while this gave me an idea of yearly performance, what it also does is underestimate the results, because many of the winners are cut prematurely. This is shown because when i test it over 2-year blocks, the outperformance of the market is much more substantial.

These are the tests that i did to try to increase system robustness:

*Delisted stocks are always included. In fact, excluding these results in slightly decreased performance!

*Run monte carlo simulations (20,000) on a year by year basis and in 2-year blocks in both the in-sample and out-of-sample. This reduces start and end date biases.
- In 2001, average return was 16.16%, with 99.93% of portfolio tests being profitable.
- In 2002, average return was 12.17% with 99.97% of portfolio tests being profitable
- In any 2-year block, the returns were always greater than the sum of the single years, and every portfolio being profitable.

*Run the system over a period I like to call, THE WORST, from 01-07-2001 until 01-03-2003. Pretty much peak to trough of the closest thing we've had to a bearmarket the last 15 years. XAO was -18% over this period. My system on average gained 8%, 82.4% of the portfolios in the green.

*Remove the best 3 and worst 3 trades from the trade database. Difference in results compared to when these are included is negligible.

*Model in worst case slippage. Profitability drops by 40-50%. Is this acceptable?

Any other further ideas as to how to increase system robustness would be appreciated (before i get too excited!).

Thanks,
Nizar.
 
Nizar
One comment that I would make, aside from a pretty thorough testing approach, is that closing out the trades prematurely on a long term system is introducing another exit criteria into the strategy.

I replicated the test period you used for some ideas that I had recently, closing the trades as you did, as well as letting them run to conclusion. So buy orders were taken during the test period (2001-2002), but in one test the trades were closed at the end, whilst with the other test let them run until the system exit criteria were met.

I found that results, in this test and over this time period, were improved slightly when trades were automatically closed at the end of the test period. The results from this approach would be different from actual trading and would be dependent on the end date chosen.

I attached results from 2 tests. "ATR stop 2001 2002" closes trades at the end of the test period, whilst the other file doesn't. I guess that the differences are not all that big but, my thoughts are that just closing the trades at the end of the test period, especially for long term systems, will give a slightly distorted picture. With shorter term systems this may not be an issue.

Also if you step through 2 year periods, moving the start date forward 3 months at a time then the results will show the variability due to different startup times. Startup can be tough with a long term system because sometimes you won't see any decent closed trades profits for quite some time.

regards
stevo
 

Attachments

  • Monte Carlo Report (Tradesim - ATR stop 1 Open buy sell).pdf
    69.2 KB · Views: 14
  • Monte Carlo Report (Tradesim - ATR stop 2001 2002).pdf
    69.1 KB · Views: 8

Thank you for your thoughts, Stevo.
 
Cookies are required to use this site. You must accept them to continue using the site. Learn more...