Should redundant raw trading signals be included in backtesting?

[Mark17]

Suppose I have a long-only trading system for one equity. In live trading, my plan would be to be long the equity with 100% of my position size or out of the market. If the buy criteria are met on consecutive bars then I would ignore the redundant signals.

For backtesting purposes, should I include trades from even the "redundant" buy signals? I could imagine a backtest where results looked good taking just the first signals but horrible if all signals were taken. Would the latter change my opinion of the system? Most definitely. I would suspect the impressive first-signal-only performance might have been a [lucky] result of chance alone rather than a robust statement about the system.

I'm interested to hear thoughts on this subject.

Thanks,
Mark

 

[tech/a]

Suppose I have a long-only trading system for one equity. In live trading, my plan would be to be long the equity with 100% of my position size or out of the market. If the buy criteria are met on consecutive bars then I would ignore the redundant signals.

For backtesting purposes, should I include trades from even the "redundant" buy signals? I could imagine a backtest where results looked good taking just the first signals but horrible if all signals were taken. Would the latter change my opinion of the system? Most definitely. I would suspect the impressive first-signal-only performance might have been a [lucky] result of chance alone rather than a robust statement about the system.

I'm interested to hear thoughts on this subject.

Thanks,
Mark

First signal systems can require a " latch" in their formula to lock in the first occurrence and not trigger a buy until the exit or stop has been triggered on that trade.

For example trading the highest high for 10 days can trigger a new trade the very next bar and then again the next day----

 

[Synergy]

My thought is that if they are not going to be taken in live trading, then they shouldn't be included in the backtest. But I do understand your concerns...

I'd look to try and single out these secondary signals and test them. You could put a condition on your entry that the previous day also be a sigal, or that there has to be a signal within the previous 5 days, or whatever you are interested in.

If you have a large enough set of results from this then you can learn something about your system. You can even code them out if the results are poor. In reality though, they were never going to be taken by the system anyway.

 

[lindsayf]

Its a question I have asked myself recently. On a low timeframe trend following method that I am back and fwd testing I have included all signals to produce a psuedo equity curve and set of statisitics. This data set obviously has more trades in it than what I would take if trading live. But nevertheless each signal meets my criteria and so what I am taking away from the analysis is an estimate of the expectancy per trade . At this stage IMO this is a valid and useful thing to do. Be interested to hear people views on this.

 

[Gringotts Bank]

Suppose I have a long-only trading system for one equity. In live trading, my plan would be to be long the equity with 100% of my position size or out of the market. If the buy criteria are met on consecutive bars then I would ignore the redundant signals.

For backtesting purposes, should I include trades from even the "redundant" buy signals? I could imagine a backtest where results looked good taking just the first signals but horrible if all signals were taken. Would the latter change my opinion of the system? Most definitely. I would suspect the impressive first-signal-only performance might have been a [lucky] result of chance alone rather than a robust statement about the system.

I'm interested to hear thoughts on this subject.

Thanks,
Mark

Start your backtest on a range of different dates. That will give you a quick idea.
If your start date is 6th Jun. Try 5th and 7th Jun.

If you're using AB, there are ways to take every trade in your backtest.

 

[sinner]

Suppose I have a long-only trading system for one equity. In live trading, my plan would be to be long the equity with 100% of my position size or out of the market. If the buy criteria are met on consecutive bars then I would ignore the redundant signals.

For backtesting purposes, should I include trades from even the "redundant" buy signals? I could imagine a backtest where results looked good taking just the first signals but horrible if all signals were taken. Would the latter change my opinion of the system? Most definitely. I would suspect the impressive first-signal-only performance might have been a [lucky] result of chance alone rather than a robust statement about the system.

I'm interested to hear thoughts on this subject.

Thanks,
Mark

Obviously if you are only taking the original signal then only backtest against the original signal (include a "latch" as tech describes, such as "if numOpenPositions < 1").

However, many systems can be traded using overlapping signals as a method to pyramid trades. Depending on the system, overlapping signals can be a good thing. I found a lot of systems which had low return but with pyamiding showed lots of promise.

Here is a simple trend following system which uses overlapping signals to increase the original stake:

ENTER LONG IF CLOSE = HIGHEST[100](Close), EXIT IN 10 DAYS
ENTER SHORT IF CLOSE = LOWEST[100](Close), EXIT IN 10 DAYS
(h/t myforexdot for the idea)
As you can see in this system, the maximum inherent leverage is 10:1, and we can always enter a new position on any given day a new breakout occurs.

 

[Mark17]

I'm using Amibroker so I would use the RegularRawMulti mode to do this.

I think part of this is a question about whether the redundant signals are independent or not, and that is a tough question to answer. In other words, suppose I get buy signals on three consecutive bars. Is there anything special about the first one? If not then the results of trades generated on all three should be considered whether or not you would actually trade that way. The reason is because sure, for that one instance the first trade may end up well but in live trading, who's to say it will end up that way?

Consider two different scenarios: an OB/OS indicator and the last trading day of the month. In the first case, suppose you get three consecutive buy signals--what's so special about the first one? The second or third one is more ripe for correction--maybe? In some cases the indicator stays OB/OS for a long time so all signals should be heeded. In the second case, consider three consecutive buy signals beginning on the last trading day of the month. If a definitive seasonality exists and the last trading day of the month tends to outperform the first couple trading days of the month then doesn't there seem to be more dependence here and reason to not include all trades in the backtest?

I'm really not sure.

The only thing I still learn strongly toward is that I'd feel much differently about profitable results from a first-signal-only backtest if inclusion of redundant trade results significantly decreased profitability.

 

[lindsayf]

It will depend on the entry criteria IMO - If the criteria for a trade are met then inclusion of that trade in the dataset will be useful wont it? If the criteria necessarily excludes setups that occur if a trade is already open then the start point of the backtest is important and mulitple tests with different start points will tell you something about the (in)dependence of the trades. For the method I am looking at it is quite possible that I miss the first signal but I would not hesitate to take the second one as I cannot see any significant dependence , nor any reason for there to be dependence....so I look at all setups.......but all methods are not equal.