Breakout systems

[tech/a]

I just wanted to clarify the definition of a 'break-out' system...apples with apples, etc

To be honest, I haven't spent much time testing long term systems on equities.

I'm more inclined toward higher frequency models - just a personal preference.

And nothing wrong with that either.

 

[stevo]

I just wanted to clarify the definition of a 'break-out' system...apples with apples, etc

To be honest, I haven't spent much time testing long term systems on equities.

I'm more inclined toward higher frequency models - just a personal preference.

Sorry for being a little cryptic with my previous post on vegetables.

One trader can buy on a breakout for his / her system, whilst another sees the stock in a trading range. It all depends on timeframe.

There are many possibilities (infinite??) in terms of breakout systems when possible strategies (including exit criteria), markets traded and timeframes are taken into account.

Some people were suggesting that the entry (breakout or otherwise) is irrelevant and that it is the exit that is important. The previous link I posted showed that a random entry underperformed a simple breakout entry using a random exit. Think of a random exit as a randomly timed exit (random nbars exit in Amibroker) within certain range parameters.

Following from this concept it is possible to design a very tradeable system that has a fixed trade length and a breakout entry. For example enter on price breaking predefined timeframe dependent criteria and sell 6 hours, 6 days, or 6 weeks later - depending on the time frame you want to trade. Add in a relatively wide trailing stop and you have a potential system. It could also be a fairly easy system to trade, depending on how good your broker is, because the timed exit is clearly defined on entry.

There are so many possibilities when it comes to trading systems - think it, code it, test it, do it!

 

[tech/a]

And nothing wrong with that either.

Was going to add why does any breakout method have to be longterm?

Stevo however has covered that above.

 

[MichaelD]

Some people were suggesting that the entry (breakout or otherwise) is irrelevant and that it is the exit that is important. The previous link I posted showed that a random entry underperformed a simple breakout entry using a random exit.

You'd be referring to me, here. I don't understand how you can come to the conclusions above - your blog entries do NOT support your conclusions as you are changing TWO variables, not one.

System 1: ROC Breakout Entry + Random Exit
System 2: Random Entry + MA Exit

These two systems have NOTHING in common and cannot be compared.

If you really wanted to compare systems, you'd compare;

System 1: Random Entry + Random Exit
System 2: ROC Breakout Entry + Random Exit

You'd then be able to find out if the entry added or subtracted value.

 

[stevo]

You'd be referring to me, here. I don't understand how you can come to the conclusions above - your blog entries do NOT support your conclusions as you are changing TWO variables, not one.

System 1: ROC Breakout Entry + Random Exit
System 2: Random Entry + MA Exit

These two systems have NOTHING in common and cannot be compared.

If you really wanted to compare systems, you'd compare;

System 1: Random Entry + Random Exit
System 2: ROC Breakout Entry + Random Exit

You'd then be able to find out if the entry added or subtracted value.

I can't disagree with this rational, although anything can be compared. System 1 beat system 2 - why is another issue

Have a look at http://drawdown.blogspot.com/2006/03/comparing-systems-using-monte-carlo.html

Whilst the post doesn't answer your point in the timeframe that I trade I have found that the entry / exit combo is very important. The exit isn't the only aspect of the system that is adding to performance.

I know because I tested random entry versus breakout entry and the same exit criteria for both systems.

Obviously it is worthwhile for every trader to objectively define what they mean by "beat" and do their own work on determining what works for them.

 

[MichaelD]

Have a look at http://drawdown.blogspot.com/2006/03/comparing-systems-using-monte-carlo.html

...I tested random entry versus breakout entry and the same exit criteria for both systems.

I'm confused. Your blog text refers to four systems in the graphs;

1. Random Buy / Random Sell
2. Weekly MACD
3. RS III
4. S6

There doesn't seem to be any system other than system 1 there that uses Random Sell as the exit.

There also doesn't seem to be a Monte Carlo distribution comparing systems 3/4 with the same systems with random entry.

 

[MichaelD]

Time for some specifics. Here is a comparison between a random entry system and a breakout entry system. The two systems are identical apart from their entry;

Universe: ASX200
Test Period: 10 years back from today
Exit: 6.5 ATR
Position Sizing: 2% risk
No pyramiding
Monte Carlo runs: 1000

System 1: RANDOM ENTRY (EntryTrigger:=1)
Total Profit: 657%
Winners: 44%
Drawdown: 12%

System 2: BREAKOUT ENTRY - ROC(CLOSE,1,percent) >= 5 AND VOLUME >= (Mov(VOLUME,50,EXPONENTIAL)*1.5)
Total Profit: 623%
Winners: 44%
Drawdown: 15%


System 2 performs significantly worse than System 1 - it is less profitable, but far more importantly, the drawdown is significantly higher for this lower profit.

Entering on a breakout does NOT add value to long term trend following systems. In fact, it does the opposite.

 

[theasxgorilla]

Entering on a breakout does NOT add value to long term trend following systems. In fact, it does the opposite.

IMO, you need to observe the distribution of results. The average doesn't tell anywhere near as much as the way results are distributed, skewed etc.

Also, if you start both tests from the same date on every run through there is a fair chance you are introducing start-date bias. For example, the point where you start your test may have had historically good returns, but been poor at producing breakout triggers. The breakout system will always miss this period, and as a kind of tipping point, can account for anomalies between your results.

In all my random testing I randomise that start-date by up to 3 months to try and avoid this as a factor.

In addition, you haven't stated the total number of trades each system is making. Assuming the holding times are comparible this will tell you the relative exposure each system has to the market. In an upward trending bullmarket, like the one we've just had, just being in the market more often, can often lead to better returns. For this reason, when I do random testing I also randomise a wait period between trades, to simulate the time when a real system would be waiting for it's next signal.

The difference in those results is not conclusive. I've done a fair bit of testing with random systems and the truth IMO is that nothing really is conclusive...but I'd be well wary of drawing conclusions from those results.

You should also consider some MAE/MFE analysis if you want to measure the effectiveness (or prove the ineffectiveness) of an entry of n-periods.

Maybe also consider introducing a random function into the breakout test results to generate even more paths through the data than a simple Monte Carlo will manage.

 

[MichaelD]

IMO, you need to observe the distribution of results. The average doesn't tell anywhere near as much as the way results are distributed, skewed etc.

Also, if you start both tests from the same date on every run through there is a fair chance you are introducing start-date bias. For example, the point where you start your test may have had historically good returns, but been poor at producing breakout triggers. The breakout system will always miss this period, and as a kind of tipping point, can account for anomalies between your results.

In all my random testing I randomise that start-date by up to 3 months to try and avoid this as a factor.

In addition, you haven't stated the total number of trades each system is making. Assuming the holding times are comparible this will tell you the relative exposure each system has to the market. In an upward trending bullmarket, like the one we've just had, just being in the market more often, can often lead to better returns. For this reason, when I do random testing I also randomise a wait period between trades, to simulate the time when a real system would be waiting for it's next signal.

The difference in those results is not conclusive. I've done a fair bit of testing with random systems and the truth IMO is that nothing really is conclusive...but I'd be well wary of drawing conclusions from those results.

I'll answer these objections, but let's leave it at that. As far as my trading goes, I have long gone beyond the issue of entry for long term trend following and have no reason to revisit it - my view is that the real money for long term trend following is elsewhere (optimizing money management, which I have spent much time working on). I no longer have the psychological need nor energy to expend on converting others to my view.


1. The standard deviation and distribution of the results for both systems was much the same.

2. You can start the simulation on any date - the results come out basically the same. I remind you, however, that I used a 10 year date range to include a range of markets in the test.

3. Total number of trades was much the same, around 550 or so.

4. You're also missing the point that a breakout entry was NOT clearly and obviously superior to random entry.


The big problem with trend followers is that we remain convinced that we can add value to our systems by what is simply curve fitting and we will spend endless hours curve fitting the entry to give the best historical results (which paradoxically will likely give the worst results going forwards).

Long term trend following derives its edge purely and simply by cutting out the big losers whilst letting the big winners run. All the rest is window dressing to make us feel as if we have some degree of control over our results.

Accepting the fact that entry is mostly insignificant in long term trend following is highly liberating, and allows you to move beyond the "5%". I have come to accept, however, that the vast bulk of long term trend followers will never accept this, but as long as they honour their stops, they'll still be profitable.

 

[theasxgorilla]

MichaelD, with all due respect, it's your study. If you didn't want it to be critiqued, why post it?

In addition to my previous comments, using today's ASX200 as your universe, with 10 years of past data is NOT a way to compare a breakout system to a random entry system that has "entry=1" as the trigger.

The latter system will always be in the market...and when those markets are today's ASX200 constituents and we know where they're going to end up, you wouldn't do yourself any favours waiting for a breakout would you?

Anyway, this isn't my point.

Entries become important when you have initial stoplosses. Your study did not have any initial stoplosses. My own research into random entry/exit testing has suggested that adding a stop to such a system reduces CAGR, but vastly improves Max DD.

Working on entries can help keep the system up off it's initial stops, if that makes sense.

My study:

http://theasxgorilla.blogspot.com/2007/08/2500.html

 

[tech/a]

Michael.

I really cant believe that your making your point with such weak arguement.
Youve basically classified all breakouts as conforming to your singular entry criteria.

There are literally 100s of ways to define a breakout.

I could point out Techtraders Breakout criteria as a positive example but that has no validity as a singular entity.

To argue that Random entry is more profitable than all breakout entrys is crazy in my view.

EVERY trend in EVERY timeframe begins with a breakout from something.

Add another 10 or more valid breakout criteria to filter entry against random.

Add to that criteria which place a POTENTIAL breakout in to a higher probability such as that in T/T (Which is very basic) and you have a far more potent entry that the canned version you have supplied.

Add to that your exit which you ignore as relevant.
Failure is deamed relative to longterm result only???

TH works in 60 second timeframes and Ill bet there is a breakout of some sort
in his timeframe.

Out of interest,do you believe ANY entry is better than Random or are all entries in your view a delusional "Feel good" method of entering a trade?

 

[bingk6]

If you really wanted to compare systems, you'd compare;

System 1: Random Entry + Random Exit
System 2: ROC Breakout Entry + Random Exit

You'd then be able to find out if the entry added or subtracted value.

I fail to see how even these two systems can be compared. At the end of the day, random is exactly that, random, and for system 1 which incorporates two random factors, the results returned could potentially range from -100K to 100K, depending on the iteration chosen. For the results of a singular iteration from system 1 to be compared to a singular iteration from system 2, which also incorporates a random exit, is to my line of thinking, a meaningless exercise, akin to comparing two "stabs in the dark".

I am of the opinion that even for long term trending systems, some entries are consistently better than others. ASX has documented some code in Amibroker in his blog for testing the effectiveness of entries alone, using the Edge ratio (derived from the Turtles Trading book ??) which is I believe gives a very good indication as to the edge that your entry provides.

 

[MichaelD]

Entries become important when you have initial stoplosses. Your study did not have any initial stoplosses.

An initial stop loss increases trade frequency (and decreases win %). It thus increases trade costs. All the studies of initial stop loss that I've seen (including yours - systems 2 and 3) fail to take trade costs into account - if they do include trade costs, all of a sudden the initial stop doesn't seem quite so attractive, as the drawdown and CAGR then end up much the same as a system that doesn't use an initial stop. Having said that, further investigation of an initial stop loss is the one entry-related item on my to do list as the work I did a while back suggested there might be something a bit worthwhile there.

In regards to universes - you get much the same comparative results when using a delisted universe as you do when you use the current ASX200. It was just a convenient folder I had lying around.

It is my view that a long term trend following system should always be fully invested in the market. This is consistent with my views that the entry has no positive predictive value and that the edge in these systems is the upwards bias in the market, which is omnipresent in the time frame we are trading in. This is also consistent with the observations that I have never been able to develop and have never seen an index filter which improves results, with all filters hurting results.

Anyhow, if you want to muck about with a few % difference in outcome, then keep working on the entry. If you want to change your results by orders of magnitude, then work on money management.

 

[MichaelD]

Out of interest,do you believe ANY entry is better than Random or are all entries in your view a delusional "Feel good" method of entering a trade?

All components of a trading system are important, including the entry, but the relative importance of each component is dependent on the edge you are exploiting.

For long term trend following, the entry is practically insignificant - it's the risk & money management where the bang for buck is.

ps I don't have any problems at all in not being believed and being laughed at - the continuing belief of traders in breakouts is one of the edges I exploit. For that particular edge, the entry is THE most important aspect of the trade.

 

[Boggo]

Aside from the 'mine is bigger than yours' cycle, lets put up some trades.

Mine, BOL - Boom Logistics.

.

 

[stevo]

I fail to see how even these two systems can be compared. At the end of the day, random is exactly that, random, and for system 1 which incorporates two random factors, the results returned could potentially range from -100K to 100K, depending on the iteration chosen. For the results of a singular iteration from system 1 to be compared to a singular iteration from system 2, which also incorporates a random exit, is to my line of thinking, a meaningless exercise, akin to comparing two "stabs in the dark".

The entry and exit is random, but the market tends to have a bias. If I was flipping a perfect coin I would agree that the results would be ranged evenly around an average. But a random entry / exit test will show a profit in the majority of cases, at least in the Aussie market over the last 10 years.

I guess that point to the exercise is that if a system can't soundly beat random entry and exit then it's not worth trading it.

MichaelD's argument is along the lines of the discussion in the book "Fooled by Randomness". It goes along the lines of - "I have examined 2000 swans and none of them are black. Therefore there are no black swans." Then someone discovered Australia and found black swans.

So - "I have examined a couple of thousand breakout systems and the entry doesn't help the results. Therefore breakout entries don't work." Maybe if 10000 different entries were tested one of them would work. But of course there are time limitations to such work.

 

[MichaelD]

I guess that point to the exercise is that if a system can't soundly beat random entry and exit then it's not worth trading it.

100% agree with this.

I'm also amused by a quote I read somewhere that goes along the lines;
"If you torture a given dataset for long enough, it will admit to anything."

It really all boils down to our inability to accept that fact that we cannot control the markets.

 

[julius]

Michael,

I agree with you on a number of points, particularly the impact of the money management on system performance. You can make most systems profitable if you get it right...

However I'd argue that certain breakout entries provide an edge because of the risk/reward r'ship they present. Essentially, the entry may as well be random but the pattern provides a risk/reward 'filter'.

Not neccesarily the 'best' R:R but the most robust protective stop which minimises the chances of getting 'whipped' out of the position, while also minimising the loss if the pattern does fail.

I'm just not sure if you could effectively program these types of entries.

 

[bingk6]

The entry and exit is random, but the market tends to have a bias. If I was flipping a perfect coin I would agree that the results would be ranged evenly around an average. But a random entry / exit test will show a profit in the majority of cases, at least in the Aussie market over the last 10 years.

I guess that point to the exercise is that if a system can't soundly beat random entry and exit then it's not worth trading it.

I agree that the market has a bias and that the random entry/exit would tend to skew more towards profitability than not. Indeed, I would say that if sufficient iterations were put through the random entry/exit tests and there were no skewing in the conditions within which these tests were run, that the results would tend to towards the performance of the index.

Therefore, if one wanted to see whether a system was worth trading, won;t it be a simplier exercise just to compare its performance to the index? I just don't see what random entry/exit offers above comparing to index performance.

 

[bingk6]

I don't have any problems at all in not being believed and being laughed at - the continuing belief of traders in breakouts is one of the edges I exploit. For that particular edge, the entry is THE most important aspect of the trade.

Michael,

Are you trading Turtlesoup ??