Outside the "Blueprint"

[nizar]

Ok, if you forward trade a system and you come to a point where your system underperforms, and falls below the minimum ranges obtained from montecarlo analysis of the same system during historical backtesting.

What do you do?

For the purposes of this thread, lets say your system is robust and optimised (not overoptimised ).

Say, if you worked out from backtesting that you could have 18 losses in a row, and you've just suffered your 19th?
So you exit the system and go back to the drawing board (because you are trading "blind" and that your system is not tradeable because market conditions have changed??

Or do you keep going?

How about in terms of drawdowns?
If maxDD is supposed to be 31% but now it just became a touch higher?
What now??

In Way of the Turtle, Curtis Faith says that he was trading a $20million account, and overnight after the October 1987 crash, he lost $11million?
He did not mention changing the system after this event.

Surely Richard Dennis wouldnt have anticipated a drop of this magnitude through backtesting and his previous years of trading??

Any discussion and feedback would be seriously appreciated.
Thank you.

 

[tech/a]

Ok, if you forward trade a system and you come to a point where your system underperforms, and falls below the minimum ranges obtained from montecarlo analysis of the same system during historical backtesting.

What do you do?

Presuming that the Monte Carlo analysis returns 100% winning trades and the method has been tested over a reasonable amount of data---then I would exit all positions.

For the purposes of this thread, lets say your system is robust and optimised (not overoptimised ).

Explain optimised? Optimising a portfolio trading method is crazy! There is a place in singular stocks/indexes commodities and futures though---well my belief.

Say, if you worked out from backtesting that you could have 18 losses in a row, and you've just suffered your 19th?
So you exit the system and go back to the drawing board (because you are trading "blind" and that your system is not tradeable because market conditions have changed??

Or do you keep going?

Why would you keep going.
Whats the point then of testing?

How about in terms of drawdowns?
If maxDD is supposed to be 31% but now it just became a touch higher?
What now??

See above.

In Way of the Turtle, Curtis Faith says that he was trading a $20million account, and overnight after the October 1987 crash, he lost $11million?
He did not mention changing the system after this event.

Surely Richard Dennis wouldnt have anticipated a drop of this magnitude through backtesting and his previous years of trading??

Any discussion and feedback would be seriously appreciated.
Thank you.

No he certainly wouldn't have.Nor would everyone else left in the market on the day of the crash.
The basic formula of the Turtle method is still as it was.(Although thats not what the books about).
However the data could be used in any testing after it had occured.
A new Blueprint would then be produced and possibly alter the results of Monte Carlo testing on most systems.

Systems trading isnt a guarentee you wont get belted by an out of sample event.It will however give you a road map for your trading based upon the Conditions and Parameters set in your trading method.
Something most traders DONT have.

 

[theasxgorilla]

In Way of the Turtle, Curtis Faith says that he was trading a $20million account, and overnight after the October 1987 crash, he lost $11million?
He did not mention changing the system after this event.

Surely Richard Dennis wouldnt have anticipated a drop of this magnitude through backtesting and his previous years of trading??

In the book he talks about the special money management techniques that the Turtles used which limited the units of risk they could expose to highly correlated, lesser correlated and supposed non-correlated markets. From what I can understand it seems like this was a bit like a weighted and volatility adjusted method of measuring portfolio heat.

Faith believes Dennis designed it this way to avoid a single shock-event from wiping out his business. He also believes that Dennis may have learned this lesson via silver back in the 70s. As Faith says, all markets have a tendency to become highly correlated during a shock-event.

The Turtles program was wound down in 1988 so you could losely draw the conclusion that Faith stopped trading after (due to??) that event. This quote from Dennis at http://www.streetstories.com/rd_finan_trader.html might answer the question whether he changed anything after 1987:

"I am no longer a trader, I am now a researcher," he says. "I am constantly performing research to determine how I can tweak the system and create new parameters to allow it to adapt to the changes that are constantly occurring in the market."

 

[Nick Radge]

The losses incurred on that day were from a move in the Eurodollars. These track US interest rates and normally move 4 - 6 ticks a session. When the Fed pumped primed that October morning, Eurodollars gapped 250 ticks. That was what did the damage to Faith and many others. 1987 was a Black Swan, an aberration. It will never appear in testing. QBE on Sep 11, 2001 was an aberration. It created the largest maxDD for my Signal Service system. I kept on trading using my "common sense approach" that I have discussed herein before. Okay. We got hammered from an event. But does a single event, especially a Black Swan event, make a system worthless? I say no. My Signal Service has done nothing but make money ever since without any parameters being changed. The scope of the maxDD was indeed changed but that does not make the system worthless.

I cannot say it enough, Black Swan or not, if you know WHY the system will make money over the longer term, then there is no reason to change. Nasty events will happen. They always will because people are emotional. But a robust system will live long afterward. I have hammered this home to my subscribers over the last few weeks with may trend following systems. They will ALWAYS be ahead of the pack. They will ALWAYS be in the 10%...unless you turn them off when in doubt...like now. In 2002 a number of my clients turned it off...only to see the 2003 bull market roar into action. Their error was not understanding. You must understand. I admit that I find it hard that Faith misunderstood. I have spoken with him on a few occasions. He is possibly one of the nicest guys around. I just wonder if it was more his personality speaking rather than the system. I trade the Turtle system in 2001 for 9-months and I must say it was very tough.

Of course, if you don't understand WHY your system makes money, then yes, you have a problem.

 

[nizar]

Thats guys for the responses.
Pretty much what i was looking for from those i was hoping would respond.

Much appreciated

 

[tech/a]

Nick

I cannot say it enough, Black Swan or not, if you know WHY the system will make money over the longer term, then there is no reason to change. Nasty events will happen. They always will because people are emotional. But a robust system will live long afterward. I have hammered this home to my subscribers over the last few weeks with may trend following systems. They will ALWAYS be ahead of the pack. They will ALWAYS be in the 10%...unless you turn them off when in doubt...like now. In 2002 a number of my clients turned it off...only to see the 2003 bull market roar into action. Their error was not understanding. You must understand. I admit that I find it hard that Faith misunderstood. I have spoken with him on a few occasions. He is possibly one of the nicest guys around. I just wonder if it was more his personality speaking rather than the system. I trade the Turtle system in 2001 for 9-months and I must say it was very tough.

Of course, if you don't understand WHY your system makes money, then yes, you have a problem.

Well I have to say that it appears from the above that if you understand WHY your system will make profit then there is no need to even test it.
Even if it traded out of the test results just keep trading it.

That could mean that some systems trade their owner to ruin.
Do you believe that in all cases that if you know WHY your system is profitable that this wont/couldnt or cant happen?

Black Swan or not,

Black Swan I understand the not I cant get my head around.

Question.
You have a system you've traded very profitably for years,oneday it trades outside its parameters and drawdowns are below accepted during testing.
You are convinced that the basic principals behind the system which made it profitable for years still apply.However all open trades have been either stopped or exited.

Do you.
(1) Keep buying on any new buy signal generated.
(2) Stop buying and wait for the method to show profit again.(equity Curve)
(3) Include the market conditions into future testing and if acceptable continue to trade---if not ignore and trade anyway.

Is there ANYTIME you would stop trading a system that even though you knew WHY it SHOULD/WAS making profit it clearly wasnt and was now outside extensive test results.

 

[nizar]

Question.
You have a system you've traded very profitably for years,oneday it trades outside its parameters and drawdowns are below accepted during testing.
You are convinced that the basic principals behind the system which made it profitable for years still apply.However all open trades have been either stopped or exited.

Do you.
(1) Keep buying on any new buy signal generated.
(2) Stop buying and wait for the method to show profit again.(equity Curve)
(3) Include the market conditions into future testing and if acceptable continue to trade---if not ignore and trade anyway.

Is there ANYTIME you would stop trading a system that even though you knew WHY it SHOULD/WAS making profit it clearly wasnt and was now outside extensive test results.

Great post tech. Good questions.
I, too, wait in anticipation of Nick's response.

 

[Temjin]

Agreed, great questions.

One thing I've learnt very recently from Perry Kaufman is that all historic testing should take price shock (totally unexpected events) into account and that you need to make sure your system does not "bias" toward making profits in such "unexpected events".

From his own belief, all well-tested system should only provide profit/lost in all price shock cases.

So there is a chance that your system may be trading outside your previously tested parameters but is largely due to a price shock event that was not anticipated. (or rather impossible to)

Don't we all have the beliefs that market do change and systems will eventually fail? If we understood how our system was profitable in previously timeframe, then we should know immediately why it wasn't profitable in the current timeframe due to a market change. What I am saying is that there has to be a reason why the system is trading outside parameters and our job is to find out exactly why.

 

[tech/a]

The WHY is the key here.

I'll suggest that the why is either.

Positive expectancy due to---
(1) More winning trades than losing trades.
OR
(2) Greater winning trades in value than losing trades.
OR
(3) A Combination of BOTH.

I maybe wrong but I think Nick means that after an event regardless of timeframe for that event then chances are that the system that was winning before will again now.All methods go through drawdown even the best.
A trading of a system which answers the question of WHY should not be thrown away if it at sometime trades outside its blueprint.

(My addition to this is)
That the system be evaluated with the conditions which placed it outside its blueprint INCLUDED.
If the WHY is STILL evident then trade away with the NEW Blueprint.
(Again for me personally)

I'd stop trading the System as I dont know
(1) when the event which caused the system to trade outside the blueprint will end.
(2) How the WHY will be effected when the conditions which caused the trading to be outside the BP are included in new testing.

Nick?

 

[It's Snake Pliskin]

The WHY is the key here.

I'll suggest that the why is either.

Positive expectancy due to---
(1) More winning trades than losing trades.
OR
(2) Greater winning trades in value than losing trades.
OR
(3) A Combination of BOTH.

Tech,

I suggest the WHY is the assumptions based off knowledge and tested. PE is surely the result.

Why does it have this MA and not this one? Becuase I know WHY.

 

[tech/a]

Tech,

I suggest the WHY is the assumptions based off knowledge and tested. PE is surely the result.

Why does it have this MA and not this one? Becuase I know WHY.

Snake.
I maybe wrong but Ive been around Radge long enough to believe this is what he means.(My assumptions).
The analysis isnt remotely close to the why,fundamental/or technical or both.

 

[It's Snake Pliskin]

Snake.
I maybe wrong but Ive been around Radge long enough to believe this is what he means.(My assumptions).
The analysis isnt remotely close to the why.

Ok so you are looking at it from the result and how the result is essential to survival angle.

I was looking at it from the angle of I know what market, stage, liquidity, etc... etc... more so What components do what and WHY - then the results come.

 

[Nick Radge]

Black Swan events cannot be tested, because they can come from anywhere at anytime and they always take a different shape. Usually a trader will blow up due to the combination of a Black Swan + leverage. LTCM is an example. Victor Niederhoffer is an example. There are many examples, but the common theme is an unforeseen event combined with leverage. The question though is whether or not their core strategies are now worthless? I say no. Let's use the Faith example and ask, "Is trend following now worthless?" Of course not. Is selling high option volatility now worthless? No. John Merriwether still does what he did at LTCM and he still has billions under management. He may not take the same leverage he used to and he may stick with more liquid paper, but he's essentially doing what he did before. Perhaps the difference between him and Faith/Niederhoffer is self belief rather than strategy belief.

I used to trade strangles in the SPI for many years but stopped in 2001. It became apparent that option volatility had dropped so much that it was not worth pursuing that strategy in that market. I was able to sell premium at 23 points each month, but eventually I could only get 12 or so points. The market conditions had changed so the strategy was binned. I could have pursued other markets where the strategy would continue to work, but the point is the strategy remained valid, but the market had changed.

Various systems may need turning off for various reasons. Volatility is a goos reason for many short term systems. The Nikkei from 1999 through 2004 displayed very low volatility and short term systems fell over. Since volatility has resumed those short term systems can be switched back on.

If your system steps outside its boundaries, and I'd suggest does so with some kind of buffer, then one would need to ask what and why. The factors causing the excursion can be exhaustive and are not generic, but usually it will have something to do with the numbers being skewed back against you. You would need to investigate those and see what has changed in their makeup. Has the win rate dropped off? Has my win/loss ratio changed? Changes in these factors will point to system based issues or market based issues. As I said above, its not a single generic element.

 

[tech/a]

Thanks
Understood.

 

[It's Snake Pliskin]

Black Swan events cannot be tested, because they can come from anywhere at anytime and they always take a different shape. Usually a trader will blow up due to the combination of a Black Swan + leverage. LTCM is an example. Victor Niederhoffer is an example. There are many examples, but the common theme is an unforeseen event combined with leverage. The question though is whether or not their core strategies are now worthless? I say no. Let's use the Faith example and ask, "Is trend following now worthless?" Of course not. Is selling high option volatility now worthless? No. John Merriwether still does what he did at LTCM and he still has billions under management. He may not take the same leverage he used to and he may stick with more liquid paper, but he's essentially doing what he did before. Perhaps the difference between him and Faith/Niederhoffer is self belief rather than strategy belief.

I used to trade strangles in the SPI for many years but stopped in 2001. It became apparent that option volatility had dropped so much that it was not worth pursuing that strategy in that market. I was able to sell premium at 23 points each month, but eventually I could only get 12 or so points. The market conditions had changed so the strategy was binned. I could have pursued other markets where the strategy would continue to work, but the point is the strategy remained valid, but the market had changed.
Various systems may need turning off for various reasons. Volatility is a good reason for many short term systems. The Nikkei from 1999 through 2004 displayed very low volatility and short term systems fell over. Since volatility has resumed those short term systems can be switched back on.

If your system steps outside its boundaries, and I'd suggest does so with some kind of buffer, then one would need to ask what and why. The factors causing the excursion can be exhaustive and are not generic, but usually it will have something to do with the numbers being skewed back against you. You would need to investigate those and see what has changed in their makeup. Has the win rate dropped off? Has my win/loss ratio changed? Changes in these factors will point to system based issues or market based issues. As I said above, its not a single generic element.

Yes Nick you have given a good answer here.

I am sometimes baffled as to why a system if traded in one market should be consistant and profitable. I think people may be doing it the hard way when they focus on one market only and then change the system as the market changes. Is this a point worth discussion? I don't know, just thinking aloud.

 

[theasxgorilla]

Snake,

I would think if you focus on one market you are probably better positioned than many to adapt your system to it. What are most of us going to do when the ASX stops handing out profits like it has???

ASX.G

 

[It's Snake Pliskin]

What are most of us going to do when the ASX stops handing out profits like it has???
ASX.G

Adapt or give up. Hard but true.

 

[stevo]

Optimising a portfolio trading method is crazy!

Maybe I misunderstand this point. Surely optimising only one stock is crazier! The beauty of portfolio testing and optimising over a large universe of stocks is that the the large volume of data gives the system more credibility.

When optimising a couple of parameters in a portfolio test it is very useful to look at the 3D charts that Amibroker produces. If it looks like the Swami's nail bed then don't rush in and choose the optimum parameters.

It's ok to turn a system off it that is part of the system's rules and it's also nice if there is an ON switch. My main system switched itself of in early June, flickered on briefly in late June and has been off ever since.

stevo