# Would you trade this system?



## nizar (13 September 2007)

A very LAZY End of Day system, between 5-7 trades triggered a year.
Average holding time for a winner over 400 days.

Id be very grateful to get any feedback and construstive criticisms.
If you wouldnt trade it -- why not?
And in which ways can it be improved?

There is a 2-year period in which the system made no money at all (1995-1997 see monthly return chart)  -- Is this acceptable?

The testing period is 10 years, 1992 until 2002, over the entire ASX market.


----------



## nizar (13 September 2007)

Monte Carlo Report	

Trade Database Filename	
C:\TradeSimData\BlueprintMachinewithOoooomphLATEST.trb	

Simulation Summary	
Simulation Date:	13/09/2007
Simulation Time:	2:29:09 PM
Simulation Duration:	193.33 seconds

Trade Parameters	
Initial Capital:	$30,000.00
Portfolio Limit:	100.00%
Maximum number of open positions:	100
Position Size Model:	Fixed Percent Risk
Percentage of capital risked per trade:	2.00%
Position size limit:	100.00%
Portfolio Heat:	100.00%
Pyramid profits:	Yes
Transaction cost (Trade Entry):	$44.00
Transaction cost (Trade Exit):	$44.00
Margin Requirement:	100.00%
Magnify Position Size(& Risk) according to Margin Req:	No
Margin Requirement Daily Interest Rate (Long Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Long Trades):	0.0000%
Margin Requirement Daily Interest Rate (Short Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Short Trades):	0.0000%

Trade Preferences	
Trading Instrument:	Stocks
Break Even Trades:	Process separately
Trade Position Type:	Process all trades
Entry Order Type:	Default Order
Exit Order Type:	Default Order
Minimum Trade Size:	$500.00
Accept Partial Trades:	No
Volume Filter:	Reject Trades if Position Size is greater than
	10.00% of the maximum traded volume
Pyramid Trades:	Yes
Favour Trade Pyramid:	No
Start Pyramid at any level up to level:	N/A
Maximum Pyramid Level Limited to:	N/A
Maximum Pyramid Count Limited to:	N/A

Simulation Stats	
Number of trade simulations:	50000
Trades processed per simulation:	668
Maximum Number of Trades Executed:	73
Average Number of Trades Executed:	65
Minimum Number of Trades Executed:	57
Standard Deviation:	8.00

Profit Stats	
Maximum Profit:	$532,904.37 (1776.35%)
Average Profit:	$507,873.68 (1692.91%)
Minimum Profit:	$481,811.58 (1606.04%)
Standard Deviation:	$23,488.87 (78.30%)
Probability of Profit:	100.00%
Probability of Loss:	0.00%

Percent Winning Trade Stats	
Maximum percentage of winning trades:	45.61%
Average percentage of winning trades:	40.61%
Minimum percentage of winning trades:	35.62%
Standard Deviation:	5.00%

Percent Losing Trade Stats	
Maximum percentage of losing trades:	64.38%
Average percentage of losing Trades:	59.39%
Minimum percentage of losing trades:	54.39%
Standard Deviation:	5.00%

Average Relative Dollar Drawdown Stats	
Maximum of the Average Relative Dollar Drawdown:	$6,256.97
Average of the Average Relative Dollar Drawdown:	$5,457.29
Minimum of the Average Relative Dollar Drawdown:	$4,614.18
Standard Deviation:	$794.30

Average Relative Percent Drawdown Stats	
Maximum of the Average Relative Percent Drawdown:	3.5568%
Average of the Average Relative Percent Drawdown:	3.0507%
Minimum of the Average Relative Percent Drawdown:	2.5184%
Standard Deviation:	0.5034%

Maximum Peak-to-Valley Dollar Drawdown Stats	
Maximum Absolute Dollar Drawdown:	$49,609.25
Average Absolute Dollar Drawdown:	$39,053.76
Minimum Absolute Dollar Drawdown:	$28,120.89
Standard Deviation:	$10,491.17

Maximum Peak-to-Valley Percent Drawdown Stats	
Maximum Absolute Percent Drawdown:	16.0116%
Average Absolute Percent Drawdown:	15.2180%
Minimum Absolute Percent Drawdown:	14.2264%
Standard Deviation:	0.8054%


----------



## nizar (13 September 2007)

Profit Summary			
Profit Status:	PROFITABLE		
Starting Capital:	$30,000.00		
Finishing Capital:	$559,826.44		
Maximum Equity/(Date):	$538,872.46 (10/09/2002)		
Minimum Equity/(Date):	-$2,827.01 (9/03/1993)		
Gross Trade Profit:	$630,743.44 (2102.48%)		
Gross Trade Loss:	-$100,916.99 (-336.39%)		
Total Net Profit:	$529,826.44 (1766.09%)		
Average Profit per Trade:	$9,295.20		
Profit Factor:	6.2501		
Profit Index:	84.00%		
Total Transaction Cost:	$5,016.00		
Total Slippage:	$0.00		
Total Trade Interest:	$0.00		
Daily Compound Interest Rate:	0.0806%		
Annualized Compound Interest Rate:	34.2018%		

Trade Statistics			
Trades Processed:	668		
Trades Taken:	57		
Partial Trades Taken:	0		
Trades Rejected:	411		
Winning Trades:	26 (45.61%)		
Losing Trades:	31 (54.39%)		
Breakeven Trades:	0 (0.00%)		

Largest Winning Trade/(Date):	$122,650.19 (3/07/2002)		
Largest Losing Trade/(Date):	-$9,675.20 (28/06/2002)		
Average Winning Trade:	$24,259.36		
Average Losing Trade:	-$3,255.39		
Average Win/Average Loss:	7.4521		

Trade Breakdown	Long and Short Trades	Long Trades	Short Trades
Normal Exit:	25 (43.86%)	25 (43.86%)	0 (0.00%)
Protective Stop:	29 (50.88%)	29 (50.88%)	0 (0.00%)
Open Trade:	3 (5.26%)	3 (5.26%)	0 (0.00%)

Total Trades:	57 (100.00%)	57 (100.00%)	0 (0.00%)

Trade Duration Statistics	Winning and Losing Trades	Winning Trades	Losing Trades
Maximum Trade Duration:	1167 (days)	1167 (days)	175 (days)
Minimum Trade Duration:	2 (days)	10 (days)	2 (days)
Average Trade Duration:	217.00 (days)	425.19 (days)	42.39 (days)

Consecutive Trade Statistics			
Maximum consecutive winning trades:	4		
Maximum consecutive losing trades:	8		
Average consecutive winning trades:	2.00		
Average consecutive losing trades:	2.21		

Trade Expectation Statistics			
Normalized Expectation per dollar risked:	$4.36		
Maximum Reward/Risk ratio:	44.01		
Minimum Reward/Risk ratio:	-1.93		
Average Positive Reward/Risk ratio:	$10.73		
Average Negative Reward/Risk ratio:	-$0.98		

Relative Drawdown			
Maximum Dollar Drawdown/(Date):	$28,881.95 (18/04/2000)		
Maximum Percentage Drawdown/(Date):	14.6100% (18/04/2000)		

Absolute (Peak-to-Valley) Dollar Drawdown			
Maximum Dollar Drawdown:	$28,881.95 (14.6100%)		
Capital Peak/(Date):	$197,642.89 (3/03/2000)		
Capital Valley/(Date):	$168,760.95 (18/04/2000)		

Absolute (Peak-to-Valley) Percent Drawdown			
Maximum Percentage Drawdown:	14.6100% ($28,881.95)		
Capital Peak/(Date):	$197,642.89 (3/03/2000)		
Capital Valley/(Date):	$168,760.95 (18/04/2000)		


Performance Summary Report			

Trade Database Filename			
C:\TradeSimData\BlueprintMachinewithOoooomphLATEST.trb			

Trade Statistics	Long and Short Trades	Long Trades	Short Trades
Trades Taken:	57	57	0
Total Net Profit:	$529,826.44	$529,826.44	N/A
Average Trade Profit:	$9,295.20	$9,295.20	N/A
Maximum Trade Profit:	$122,650.19	$122,650.19	N/A
Minimum Trade Profit:	-$9,675.20	-$9,675.20	N/A
Break Even Trades:	0	0	0
Winning Trades:	26	26	0
Losing Trades:	31	31	0
Profitable Trades:	45.61%	45.61%	N/A
Losing Trades:	54.39%	54.39%	N/A
Average Winning Trade Profit:	$24,259.36	$24,259.36	N/A
Average Losing Trade Profit:	-$3,255.39	-$3,255.39	N/A


----------



## nizar (13 September 2007)

Charts attached.


----------



## mb1 (13 September 2007)

Winning trades 26. Losing trades 31.
how is this a good system?


----------



## Shane Baker (13 September 2007)

Hi Nizar

Thanks for posting. Personally I wouldn't trade this system for a number of reasons. 

1. Too few trades to be considered statistically robust.
2. The largest winning trade is about 20% of profits from one trade. I prefer to keep the largest winner to a smaller percentage of the total profits.
3. The closed trade (MaxDD) drawdown is  higher then I'd prefer personally for a low frequency trading system. I prefer a MaxDD under 10% for my weekly based systems which trade about 25-50 times per year. I like low drawdown system so I can use leverage to bump up the returns.
4. Long periods of no profit or poor returns may make the system hard to stick with. Capital is tied up and being unproductive for long periods.

Things I like are

1. Frequency does not equal profitability as your results show.
2. Low frequency means low commissions
3. Is it a long or short side system?

It's a start and ceratinly something to keep working on. Good luck.

Cheers

Shane


----------



## nizar (13 September 2007)

Shane Baker said:


> Hi Nizar
> 
> Thanks for posting. Personally I wouldn't trade this system for a number of reasons.
> 
> ...




Thanks for your feedback Shane.
Yes points 1,2 and 4 are of concern to me as well.

Its a long only system.


----------



## nizar (13 September 2007)

mb1 said:


> Winning trades 26. Losing trades 31.
> how is this a good system?




You have a lot to learn, my friend.

Take a look at average win/average loss, and also see the expectancy of the system.

I didnt ask if it was a good or bad system, rather, i wanted to know if you would trade this system, and also i asked for some constructive criticism.

If you are looking for 100% winners before you trade a system then you will be looking for a long time.


----------



## nizar (13 September 2007)

Another thing iv noticed with this method is that im getting stopped out way too much, with 51% of trades hitting the protective stop.

Any ideas why this is the case?

The stop i thought was quite loose at 3*ATR(10).

The exit is exactly the same as tech/a.
180 day ema of the lows.


----------



## wayneL (13 September 2007)

nizar said:


> Another thing iv noticed with this method is that im getting stopped out way too much, with 51% of trades hitting the protective stop.
> 
> Any ideas why this is the case?
> 
> ...



Depends on the entry.

3 ATR's is conceptually much tighter for a breakout system than it is for a pull back system.


----------



## theasxgorilla (13 September 2007)

nizar said:


> The stop i thought was quite loose at 3*ATR(10).




I actually thought this was quite tight.  What happens if you use a 10% of position size fixed initial stop, just for comparison sake?


----------



## stevo (13 September 2007)

nizar
Besides the reasons that Shane has outlined above I wouldn't trade it because it is based on daily charts - I cannot risk not making a trade because I am tied up in a meeting, out sailing etc. Also a daily system cannot handle as much money as a weekly system - I can buy / sell over 5 days rather than 5 hours. But enough of my biases!

Why have maximum trades equal to 100? Drawdown is quite high since it is closed trade drawdown. What does the equity curve look like?

On a daily chart I would think that 3*ATR as a maximum stop loss is quite tight for a long term system. I might use that for a weekly time frame. Why not optimise the multiplier and see what the 3D chart looks like - use a dummy parameter to get the 3D chart and also do a mini monte carlo.

regards
stevo


----------



## stevo (13 September 2007)

Nizar
Doh - I found the equity curve! 

There are some big winners - it looks like 5 or 6 trades make up most of the dollar profit.

stevo


----------



## nizar (13 September 2007)

theasxgorilla said:


> I actually thought this was quite tight.  What happens if you use a 10% of position size fixed initial stop, just for comparison sake?




ASX.G.

What do you mean exactly?

Allocate 10% of equity to each position?
Or use 10% of the purchase price as a stop-loss (like T/T) instead of the 3*ATR stop?

Stevo thanks for your thoughts.
Yes it needs alot of work.


----------



## Shane Baker (13 September 2007)

Hi Nizar

Why not run the system with no initial stop and have a look at the results? Try plotting the R multiple return against the initial stop loss (use the trailing stop loss as the initial stop) as a percentage in a scatter graph (see example) and look at whether there are clusters that might suggest a logical stop placement in % terms. I still am concerned that your entry criteria are too tight and there aren't enough trades for a proper analysis.



Cheers

Shane


----------



## rnr (13 September 2007)

Nizar,

Out of interest would you post the Trade Log in an Excel spreadsheet?

The reason for the request is to have a gander at the value of the Initial Stops in the various trades.

Also, have you coded intra-day or next day exits (for both Initial and Profit Stops)?

Cheers,
rnr


----------



## howardbandy (13 September 2007)

Greetings --

Are the results being discussed in-sample or out-of-sample?

Only out-of-sample results have any value in predicting the future behavior and profitability of the system.

Howard
www.quantitativetradingsystems.com


----------



## nizar (13 September 2007)

Shane -- good idea, im working on it.

rnr -- the trade log is coming.

Howard -- in-sample. I have left aside 5years of data untouched to be used as an out-of-sample data, but im waiting until iv completed your book before i touch it. By the way, i received your book on Tuesday. Thanks. Its next on my list, after Iv finished the book by your best friend Bob Pardo LOL.


----------



## Nick Radge (14 September 2007)

> Only out-of-sample results have any value in predicting the future behavior and profitability of the system.




Howard,
Whilst I respect that out of sample testing makes sense in certain scenarios, I really don't think its the _*only *_way to do things as you portend. As an example, a system that has not been optmised, has not been curve fitted and operates with the same parameters on every symbol, why will out of sample be any better than in sample in this case?


----------



## howardbandy (14 September 2007)

Nick Radge said:


> Howard,
> Whilst I respect that out of sample testing makes sense in certain scenarios, I really don't think its the _*only *_way to do things as you portend. As an example, a system that has not been optmised, has not been curve fitted and operates with the same parameters on every symbol, why will out of sample be any better than in sample in this case?




Hi Nick --

Sorry to disagree, but I think every system must be tested out-of-sample.  I believe that it is the only way.  

In-sample results are always good.  We do not stop designing until they are good.

At any rate, if a person develops a system without out-of-sample testing, then out-of-sample testing begins tomorrow, with real money.  

I prefer to do the testing without the exposure to losing real money if I have made a mistake in the development process.

Thanks,
Howard


----------



## Chorlton (14 September 2007)

nizar said:


> Thanks. Its next on my list, after Iv finished the book by your best friend Bob Pardo LOL.





Hi Nizar,

Now that you have read it would you recommend Pardo's book on System Development / Testing??


----------



## tech/a (14 September 2007)

Howard.

In your experience would you suggest 
Fixed origin evaluation 
OR
Rolling origin evaluation and why?
How do you evelute the period ahead that you should use in the evaluation?
If optimising at what point do you re calibrate?
How many out of series test periods are in your veiw enough to give confidence and at what point do you gain that confidence?

How do you do your out of sample testing,what software do you use?
Or are you simply walking forward through a data set?
How do you extrapolate single special events that fall within the out of sample test period/s.

I would argue that out of sample testing of models offers no guarentee that the in sample test results will perform any better or worse than the results returned within the parameters of those results obtained through multiple or Montecarlo testing going forward in realtime.


----------



## nizar (14 September 2007)

Howard.
I agree with Nick.

From my understanding, if the system had not been optimised and curve-fitted, then in-sample is just the same as out-sample data.

What i mean by the same is that you can draw the same conclusions.

For example, if i have an idea to test.
And i test it ONCE over a certain data set -- and i dont change anything thereafter, then the results from this initial run can be considered as an out-of-sample data.
Correct?

Chorlton -- I been busy with school (and testing). I havent finished Pardo's book yet. I will PM you when i have,


----------



## nizar (14 September 2007)

OK so the improvements that were required to be made to the system were:

*Increasing the number of trades
(I personally would like 20-30 per year)
*Loosening the stop -- 50% being exited by hitting the protective stop is far too high. We aim for 10-15%. 
*Increased trade frequency as there were long periods of time without trading -- Opportunity cost issue and goes hand-in-hand with the first point.

Drawdowns at 15%ish were okay for me, as this system will be traded unleveraged.

Stevo -- OK i will try fixing the maximum number of open positions to 5,6,7,10, see if that improves things.

Shane -- i thought that scatter chart was standard in tradesim, apparently not, im not really that good at using Excel.

Open equity im having some problems getting it up on TradeSim. Iv been in communication with David (the developer) through email and he's going to get back to me.

I'll get back tonight with the results after changes were made.


----------



## rnr (14 September 2007)

Nizar,



> rnr -- the trade log is coming.




No mention of this in your latest post - any chance of a look?



> Maximum number of open positions: 100



 Whilst this may be causing you a minor problem perhaps you need to look at the problems that may be caused by such a high percentage here. 







> Position size limit: *100.00%*




According to one of your charts the maximium number of trades at any point in time is 11 and quite often it's sitting around 6.

Cheers,
rnr


----------



## nizar (14 September 2007)

rnr.

I have made significant changes to the system.

The trade log of the original is attached.

Position size limit in the past when used it hasnt helped the results.


----------



## nizar (14 September 2007)

Changes made:
*Position sizing model changed to $1,600 risk per trade
*Initial stop changed to 20% of purchase price
*Exit changed to 250d EMA.	
*Chose to favour pyramid.


Monte Carlo Report	

Trade Database Filename	
C:\TradeSimData\BlueprintMachinewithOoooomph20pcstop250.trb	

Simulation Summary	
Simulation Date:	14/09/2007
Simulation Time:	5:57:29 PM
Simulation Duration:	2567.08 seconds

Trade Parameters	
Initial Capital:	$30,000.00
Portfolio Limit:	100.00%
Maximum number of open positions:	100
Position Size Model:	Fixed Dollar Risk
Capital risked per trade:	$1,600.00
Position size limit:	100.00%
Portfolio Heat:	100.00%
Pyramid profits:	Yes
Transaction cost (Trade Entry):	$44.00
Transaction cost (Trade Exit):	$44.00
Margin Requirement:	100.00%
Magnify Position Size(& Risk) according to Margin Req:	No
Margin Requirement Daily Interest Rate (Long Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Long Trades):	0.0000%
Margin Requirement Daily Interest Rate (Short Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Short Trades):	0.0000%

Trade Preferences	
Trading Instrument:	Stocks
Break Even Trades:	Process separately
Trade Position Type:	Process all trades
Entry Order Type:	Default Order
Exit Order Type:	Default Order
Minimum Trade Size:	$500.00
Accept Partial Trades:	No
Volume Filter:	Reject Trades if Position Size is greater than
	10.00% of the maximum traded volume
Pyramid Trades:	Yes
Favour Trade Pyramid:	Yes
Start Pyramid at any level up to level:	N/A
Maximum Pyramid Level Limited to:	N/A
Maximum Pyramid Count Limited to:	N/A

Simulation Stats	
Number of trade simulations:	20000
Trades processed per simulation:	4111
Maximum Number of Trades Executed:	228
Average Number of Trades Executed:	215
Minimum Number of Trades Executed:	195
Standard Deviation:	10.45

Profit Stats	
Maximum Profit:	$689,298.81 (2297.66%)
Average Profit:	$593,946.77 (1979.82%)
Minimum Profit:	$450,261.55 (1500.87%)
Standard Deviation:	$82,742.05 (275.81%)
Probability of Profit:	100.00%
Probability of Loss:	0.00%

Percent Winning Trade Stats	
Maximum percentage of winning trades:	46.26%
Average percentage of winning trades:	44.60%
Minimum percentage of winning trades:	41.36%
Standard Deviation:	1.68%

Percent Losing Trade Stats	
Maximum percentage of losing trades:	58.64%
Average percentage of losing Trades:	55.40%
Minimum percentage of losing trades:	53.74%
Standard Deviation:	1.68%

Average Relative Dollar Drawdown Stats	
Maximum of the Average Relative Dollar Drawdown:	$3,026.46
Average of the Average Relative Dollar Drawdown:	$2,457.74
Minimum of the Average Relative Dollar Drawdown:	$2,106.96
Standard Deviation:	$315.17

Average Relative Percent Drawdown Stats	
Maximum of the Average Relative Percent Drawdown:	1.3266%
Average of the Average Relative Percent Drawdown:	1.0801%
Minimum of the Average Relative Percent Drawdown:	0.8874%
Standard Deviation:	0.1309%

Maximum Peak-to-Valley Dollar Drawdown Stats	
Maximum Absolute Dollar Drawdown:	$28,562.26
Average Absolute Dollar Drawdown:	$24,614.07
Minimum Absolute Dollar Drawdown:	$18,405.32
Standard Deviation:	$3,295.07

Maximum Peak-to-Valley Percent Drawdown Stats	
Maximum Absolute Percent Drawdown:	15.3609%
Average Absolute Percent Drawdown:	15.3609%
Minimum Absolute Percent Drawdown:	15.3609%
Standard Deviation:	0.0000%


----------



## nizar (14 September 2007)

Profit Summary			
Profit Status:	PROFITABLE		
Starting Capital:	$30,000.00		
Finishing Capital:	$677,739.52		
Maximum Equity/(Date):	$647,739.52 (12/10/2005)		
Minimum Equity/(Date):	-$4,608.27 (8/10/1992)		
Gross Trade Profit:	$750,738.03 (2502.46%)		
Gross Trade Loss:	-$102,998.51 (-343.33%)		
Total Net Profit:	$647,739.52 (2159.13%)		
Average Profit per Trade:	$3,114.13		
Profit Factor:	7.2888		
Profit Index:	86.28%		
Total Transaction Cost:	$18,304.00		
Total Slippage:	$0.00		
Total Trade Interest:	$0.00		
Daily Compound Interest Rate:	0.0643%		
Annualized Compound Interest Rate:	26.4368%		

Trade Statistics			
Trades Processed:	4111		
Trades Taken:	208		
Partial Trades Taken:	0		
Trades Rejected:	986		
Winning Trades:	96 (46.15%)		
Losing Trades:	112 (53.85%)		
Breakeven Trades:	0 (0.00%)		

Largest Winning Trade/(Date):	$32,575.40 (12/10/2005)		
Largest Losing Trade/(Date):	-$2,734.90 (11/08/1999)		
Average Winning Trade:	$7,820.19		
Average Losing Trade:	-$919.63		
Average Win/Average Loss:	8.5036		

Trade Breakdown	Long and Short Trades	Long Trades	Short Trades
Normal Exit:	176 (84.62%)	176 (84.62%)	0 (0.00%)
*Protective Stop:	21 (10.10%)	21 (10.10%)	0 (0.00%)*
Open Trade:	11 (5.29%)	11 (5.29%)	0 (0.00%)

Total Trades:	208 (100.00%)	208 (100.00%)	0 (0.00%)

Trade Duration Statistics	Winning and Losing Trades	Winning Trades	Losing Trades
Maximum Trade Duration:	1784 (days)	1784 (days)	492 (days)
Minimum Trade Duration:	2 (days)	26 (days)	2 (days)
Average Trade Duration:	365.39 (days)	626.31 (days)	141.74 (days)

Consecutive Trade Statistics			
*Maximum consecutive winning trades:	24		
Maximum consecutive losing trades:	18* 
Average consecutive winning trades:	4.17		
Average consecutive losing trades:	4.87		

Trade Expectation Statistics			
Normalized Expectation per dollar risked:	$1.89		
Maximum Reward/Risk ratio:	19.82		
Minimum Reward/Risk ratio:	-1.67		
Average Positive Reward/Risk ratio:	$4.76		
Average Negative Reward/Risk ratio:	-$0.56


----------



## nizar (14 September 2007)

rnr -- position size limit using 100% hasnt made the simulations any worse (I have tried 20,25,30).

I figured if theres only a few stocks breaking out no harm in holding only these stocks and pyramiding into them like a champion.

Is the large consecutive losers a concern for anybody?
$1,600 risk per trade i realise is VERY HIGH initially.

A long string of losses to begin with and its goodbye.
Though this didnt happen in 20,000 simulations.

Maximum number of open positions, because of the money management model chosen, at the peak reached more than 40.

Is this a concern for anybody?

But we are on the right track so good news i guess as we:
*Managed to increase the number of trades
*Decrease number of exits from the maximum stop loss (only 10%)
*Managed to still report good figures for profit and max.DD.

Some more feedback would be appreciated.


----------



## nizar (14 September 2007)

Some charts.

Note: The exploration was run from 01-07-1992 until 30-09-2002.

MS/Tradesim seems to take this as STOP taking entries after 30-09-2002.
BUT the trades that are open, are left to run their course until the exit is hit.

So thats why we have upto 2005 on the yearly profit charts and on the closed equity chart, even though no new entries were taken after the stop date in 2002.

Even if you cut off at 2002, thats 400k, so 30%pa.

Looking at the Trade Database Manager in TradeSim, it seems in this period UTB, BLD, ORG, GNS were the standouts. The system pyramided into them several times and each parcel made >100% profit. Small parcels though. But thats what happens when you fix your risk to a dollar amount.


----------



## Shane Baker (14 September 2007)

Hi Nizar

I wouldn't suggest using a  fixed $ risk. Basically you need a money management algorithm that increases the dollars risked as your equity grows and decreases it as you lose. This basically leaves you with fixed fractional percentage risk based (ie risk 2% of equity per trade) or fixed percentage allocation money management models (ie allocate 10% of capital per trade). 

I might suggest trying something like 15 % portfolio heat, 0.5% risk  for a pyramiding system and see how you go.  I'd also consider for the asx 200/300suggesting a closer initial stop such as 15% and then pyramid every 15% rather than using  a trigger as a pyramid point. If aggressively pyramiding doesn't work then probably the system has a poor dependency and pyramiding should be dispensed with.

Hope this helps

Cheers

Shane


----------



## theasxgorilla (14 September 2007)

nizar said:


> Is this a concern for anybody?




Re: 40 concurrent open positions...

I think the standard reponse here is that beyond a certain point you start to move toward having too great of a representative sample of the index, and your results will become anchored to the index rather than outperforming it.

I have personally found via some Amibroker optimisations that 8-15 is an effective range.  Having fewer seemed to improve CAGR but tended to increase Max DD.  Having more improved Max DD, without disproportionately affecting CAGR...this is what I was after.


----------



## Sir Burr (14 September 2007)

Would you trade this system?

No. 5.5 trades/per year doesn't seem enough trades (backtested) to make me feel confident with this system. Don't ask me how many would though, maybe 5-10x that!


----------



## howardbandy (14 September 2007)

Greetings --

Let me try a slightly different explanation of why I think it is necessary to perform an out-of-sample test of any trading system.

Imagine that all the possible trading systems have been described and there is a piece of paper with the computer code and description for each one.  

One system might be "buy when two moving averages cross" and another might be "buy a specific number of days before options expiration" and so forth -- there are thousands.  

Every system has formal parameters associated with it.  The moving average crossover system has several that we can identify easily.  Which type of moving average?  Simple, exponential, weighted, adaptive, ...  How many bars in the first moving average?  1, 2, 3, ...  How many bars in the second moving average?  1, 2, 3, ...  And so forth.  Every characteristic of the system that can be changed is a parameter.  When the design and testing of the system has been completed, there will be one value chosen for each of the parameters.  Those values represent the actual arguments that will be used when the system is run.  

Each system description is put into a separate drawer in a large room.  In the drawer, along with the description of each system are a large number of marbles.  Each marble represents one of the possible sets of arguments that go with that trading system.  Pretend this is the drawer for the moving average crossover system.  One marble has "simple, 2, 20"  another "exponential, 3, 45" another "weighted, 7, 2" and so forth.  Every possible combination has its own marble.  The drawer has Every possible combination -- those that encapsulate some characteristic of the market that will trade well, along with those that are curve-fit.  There may be many that trade well, or there may be none.  There are certainly many that are curve-fit.  We cannot tell which are which yet, because they must be applied to a specific time series and evaluated.

In order to trade any of these systems, I need the description and code for the system, and the values for the arguments.  I will be applying that code and that set of arguments to a specific ticker and a specific time period. 

As a system designer, I walk into the room and go to the drawer marked "Moving Average Crossover."  I pull the drawer open, read the description of the system and pick out one marble.  There is nothing hidden -- I can read what is on the marble before I pick it.  If I wish, I can specify which marble I want -- "Exponential, 5, 40" -- and pick it.

With my code and arguments in hand, I turn to my computer, load the set of data I want to use, and run the system.  This is the in-sample test.  If I like the results, I can either run another computer test, or begin trading live.  Either way, this is an out-of-sample test.

If I think there might be another combination that will work better, I can go back to the drawer and select another marble -- either by asking for a specific one or by picking one at random -- and run another in-sample test.  I can choose which of the two results I like better, according to my objective function.  I have an objective function, whether I have formally defined it or not.  Whenever I make a choice from among alternatives, I am applying my objective function and ranking the alternatives.

Whether I pick one marble, or two, or four, or take every marble in the drawer -- I have no way of knowing whether I have picked a curve-fit marble or a good marble.  And if it is a good marble, I have no way of know whether it is the best marble until I run a test using each marble on the ticker and data I want to trade.  So I run all those tests and choose the best, according to my objective function.  If I have run the tests over all the data I have, I have test results, but no way of judging whether the marble is a curve-fit marble or a good marble.  The out-of-sample test gives me a way to evaluate that.  

Whether the out-of-sample test is run on data that I have reserved for that purpose, or whether the out-of-sample test is run on live data using real money, there IS an out-of-sample test.  

My point is that I can perform the out-of-sample test myself for the cost of a little electricity, or the market can perform it for me using real money.

You are each free to use whichever method of performing your out-of-sample testing you wish.  But do not be fooled into thinking that since you picked just one marble out of the drawer that that marble is any more likely to be a good marble than if you had searched through all of them.  

Thanks for listening,
Howard


----------



## nizar (14 September 2007)

Shane Baker said:


> Hi Nizar
> 
> *I wouldn't suggest using a  fixed $ risk. Basically you need a money management algorithm that increases the dollars risked as your equity grows and decreases it as you lose. This basically leaves you with fixed fractional percentage risk based (ie risk 2% of equity per trade) or fixed percentage allocation money management models (ie allocate 10% of capital per trade). *
> 
> ...




Thats what i thought to.
So much so that i totally ignored the other options.

I never thought the results would be enhanced with a fixed $ stop.


----------



## nizar (14 September 2007)

Sir Burr said:


> Would you trade this system?
> 
> No. 5.5 trades/per year doesn't seem enough trades (backtested) to make me feel confident with this system. Don't ask me how many would though, maybe 5-10x that!




How about 2nd set of results with 208 trades?


----------



## Shane Baker (15 September 2007)

Hi Nizar

Can I ask what is your universe of shares that you are testing over?  If it is a group of shares such as asx 200 then there may only be  a small number of those shares that were trading in 1992 :-(

Cheers

Shane


----------



## Shane Baker (15 September 2007)

Hi ASX G



> Re: 40 concurrent open positions...
> 
> I think the standard reponse here is that beyond a certain point you start to move toward having too great of a representative sample of the index, and your results will become anchored to the index rather than outperforming it.
> 
> I have personally found via some Amibroker optimisations that 8-15 is an effective range. Having fewer seemed to improve CAGR but tended to increase Max DD. Having more improved Max DD, without disproportionately affecting CAGR...this is what I was after.




This depends how the system entry selection is constructed. If you have a single entry and exit than the index may be reflected unless you have  a filter eg price that  restricts the entries to a particular group of shares. Then it would probably be indicative of that sub set of shares movements.

If you have a system that pyramids aggressively into a trend then you may have forty open positions but in only five shares as an example.

Cheers

Shane


----------



## theasxgorilla (15 September 2007)

Shane Baker said:


> This depends how the system entry selection is constructed. If you have a single entry and exit than the index may be reflected unless you have  a filter eg price that  restricts the entries to a particular group of shares. Then it would probably be indicative of that sub set of shares movements.
> 
> If you have a system that pyramids aggressively into a trend then you may have forty open positions but in only five shares as an example.




I knew someone was going to say that!  Agree 110%.

Interesting re: the pyramiding, I never thought of it that way.  Seems then that it comes down to whether Nizar is pyramiding in this instance.

ASX.G


----------



## Shane Baker (15 September 2007)

I might be mistaken but I thought Nizar was pyramiding.



> I figured if theres only a few stocks breaking out no harm in holding only these stocks and pyramiding into them like a champion.


----------



## tech/a (15 September 2007)

Howard
Thank you for your rsponse and I like the analogy.

Can I pose the following to add perhaps something which is possibly not considered in the above.

The database which you are applying your conditions,arguements and variables to---your system.
We choose a database as this is from where we will extract our model to best take advantage of the characteristic we know is within that dataset.
Lets say that this is our dataset and since the beginning of time it has looked like this.



We also have our own objectives which we require in the results which we see when applying our system and its constituents to the data.
The out of sample data needs to be long enough to have any meaning full result on the out of sample testing.

To select an out of data sample which looks like this 


Will have obviously negative results as the system wasnt designed for these conditions.
You will find similar conditions across most datasets.

So while your system has been designed across a large set of data your out of sample dataset is much less (Generally) if taken in isolation it can be misleading.You could throw away a perfectly viable method which suits your objectives.Hence the question of how many out of sample test results give rise to confidence.

My arguement is that in the end you want to outperform the dataset your designing your system to take part in.Wether it outperforms it at the very highest level,you will never determine as the data will be dynamic.Wether it fits perfectly your criteria ALL the time is also a pointless exercise. At times it will at times it wont. The destination is in my view more important than the journey as each time its applied through time the journey (Catastrophic events avoided as best we can) will be different to the in sample *AND* the out of sample testing.

Lets say this is my system with all the objectives,variables and conditions applicable to my own requirements and those that work well against the data base its designed for.

Here is my system--I like it!
I've selected all of my objectives and this is what I have come up with.




Continued in next post due to number of images.


----------



## tech/a (15 September 2007)

Continued

Ive tested it over many thousands of tests tweeking and improving over my database which is.Its dynamic everytime I go out on it it alters.I never know exactly what it will look like.I can and have tested it all and on just part sections (out of sample).



And


And



In the end I found I get to my destination,sometimes quicker and safer than other times.
At times in my testing I come across this and could select this as an out of sample period even randomly.



My system isnt designed to perform at all well in this,although many have tried and failed.



For me the bottomline is that while we can NEVER be certain we have the very best system which will cope with EVERY condition the market throws at us,we can have confidence through even in sample testing that out system can and will perform WELL ENOUGH to out perfom most in those conditions it was/is designed for.

I would argue that even better results given the basic system will come from outside improvements like leverage and or re investments of profit back into the system,rather than endlessly looking for the perfectly straight road--or more to the point a system that makes the road endlessly straight.


----------



## rnr (15 September 2007)

Nizar,



> rnr -- position size limit using 100% hasnt made the simulations any worse (I have tried 20,25,30).




I was thinking more along the lines of 10%, 12.5% & 15%.



> Changes made:
> *Position sizing model changed to $1,600 risk per trade
> *Initial stop changed to 20% of purchase price
> *Exit changed to 250d EMA.
> *Chose to favour pyramid.




Did you make all these changes in 1 hit or did you stage and test the changes 1 at a time in an attempt to optimise each element of the trade?

Did you change the InitialStop from 3*ATR(10) to say 6*ATR(10) and what affect did it have on the Protective Stop Exits.

I don't quite understand why the results of the second simulation have run on to 2005?

Cheers,
rnr


----------



## nizar (15 September 2007)

rnr said:


> Nizar,
> 
> 
> 
> ...





rnr.

No, not all at one go. I did it bit by bit so i could find out which element was adding the edge. 

6ATR i did try it, as with 4 and 5ATR as well. None of them worked as well as the 20% stop. The wider stops took on a less number of trades (which is low enough as it is!!) and i couldnt get the # of stocks hitting the protective stop exits better than it is now with the 20% stop.

As to why the results have run into 2005. It was never my intention. See my post #30.
In both cases the metastock exploration was set to:

ExtFml( "TradeSim.SetStartRecordDate",01,06,1992); 
ExtFml( "TradeSim.SetStopRecordDate",30,09,2002); 

With position size limit set to anywhere below 30% the results get worse, in terms of profit. Im not keen on this because since my entry criteria is quite selective, when the market is going a bit sideways, not many stocks are going to be making all time highs. But the ones that do, i want to buy them again and again and again. And due to this pyramiding, i feel that its okay to have a bit of flexibility and leave the position size open.

But i think one drawback is that open profit drawdowns are likely to be increased if im too concentrated in a few stocks, and im gonna have to give back alot of profit at the end of the trend.

If only i could get my open equity chart on tradeSim to start working, then ill have a proper look at it, and see if its worth the pain or not.

Shane.

The whole market is my universe, with liquidity filter $500k per day.

Thanks for the feedback guys, keep it coming


----------



## tech/a (15 September 2007)

Tradesim Open Equity.

Go to trade database log.
Right click - click plot open equity-Daily
Then load data base from drop down.
Run it --tick plot closed equity when looking at the open equity chart.


----------



## howardbandy (15 September 2007)

Hi Tech/a --

Nice car.

The post I made earlier did not describe the walk-forward process that goes with the out-of-sample testing.  But I thought I posted that already?

Thanks,
Howard


----------



## nizar (15 September 2007)

tech/a said:


> Tradesim Open Equity.
> 
> Go to trade database log.
> Right click - click plot open equity-Daily
> ...




Thanks.

Tech do you get a warning/error when plotting open equity for weekly systems?

Im getting this warning:

"There are trades with non-daily periodicity.
It is recommended that Open Equity chart be generated only using trades with daily periodicity.
You may receive price invalidation warnings if Entry and Exit price checking has been enabled.
Do you wish to proceed to plot the Open Equity?"

Any ideas?

(Iv got it working fine for EOD systems -- and Iv got to say, the style of the open equity chart looks awesome!!!)


----------



## tech/a (15 September 2007)

howardbandy said:


> Hi Tech/a --
> 
> Nice car.
> 
> ...




And your point is?

At some point you'll trade your system and it to will be walk forward only realtime.
Upon completion of a period that period of out of sample trading would have become in sample.

All your doing is testing snippets of data in isolation.
How long do you test out of sample data?
When is enough enough?
You tweek and twiddle from that out of sample test/s then test another---ad infinitum.

When is a system ready to trade?
At what point do you say yes I'm happy that my objectives have been met?
How do you know you have the optimum conditions and parameters?
*When do you stop Tweeking and Twiddling.?*

Your books on the way---Very interested in its contents.--Seriously!


----------



## kaleon (15 September 2007)

Hi Nizar,

I have attached a metastock file with the actual constituents of the ASX300 back in 1/1/2001.

Run tradesim on this data for the year 2001 only. Make sure all your exits occur on the last trading day of 2001. That way we can see what the actual profit or loss there was for the system.

To make tradesim exit on the last trading day, you have to find what the last trading day was on the daily chart or the weekly chart depending what chart your system uses.

Use this 'date filter'

StDay:=Input(" Start day",1,31,28);
StMnth:=Input("Start month",1,12,12);
StYear:=Input("Start year",1980,2020,2001);
EnDay:=Input(" End day",1,31,28);
EnMnth:=Input(" End month",1,12,12);
EnYear:=Input(" End year",1980,2020,2001);
If((Year() > StYear OR (Year()=StYear AND ((Month() >
StMnth) OR (Month() = StMnth AND DayOfMonth() >=
StDay)))) AND (Year() < EnYear OR (Year()=EnYear AND
((Month() < EnMnth) OR (Month() = EnMnth AND
DayOfMonth() <= EnDay)))) ,1,0);


and then in the tradesim formulae do this to your exit trigger


ExtFml("TradeSim.SetStartRecordDate",1 ,1 ,2001);
ExtFml("TradeSim.SetStopRecordDate",31 ,12 , 2001);

ExitTrigger:=If(Fml("date filter"),Fml("date filter"),Fml("your exit trigger"));

ExitPrice:=C;

Then post the results here so we can see how in performed in this bear period. Gives you a good idea how robust the system is.


Tradesim parameters should include

Transaction Cost rate:0.75%
Fraction risked capital 1.5%
max positions 10
position size limit:10%
portfolio heat limit 20%
portfolio limit 100%
margin requirement 100%
entry order control: stop order,unconditionally enter trade
exit order control: default
limit position size to a maximum of : 10%
include breakeven trades with losing trades

Now that should really test your system


----------



## nizar (15 September 2007)

kaleon said:


> Hi Nizar,
> 
> I have attached a metastock file with the actual constituents of the ASX300 back in 1/1/2001.
> 
> ...




Hi kaleon.

Thanks for the date filter!
Good work. And thanks for the feedback.

I still havent got the (2001) ASX300 constituent list form you.

Im more than happy to run the test, and as a matter of fact, I'll go one better.

How about from 01-07-2001 until 01-03-2003?
Thats just about peak to trough of the "bear market". XAO lost 18% in this period.

2001 the market actually ended positive +6.98% for the year.

Also another question, where do you put the "0.75%" as a transaction cost percentage in TradeSim?

Thanks.


----------



## rnr (15 September 2007)

Hi Nizar,



> I don't quite understand why the results of the second simulation have run on to 2005?




I must be losing the plot as it was only when I read kaleon's post that it twigged and I remembered reading a post about this on the TradeSim forum (TradeSim > Ideas and Suggestions > "Closing" prices of open trades) which may well negate the "date filter" to which kaleon makes reference.

You could try this perhaps - it should close out all OPEN TRADES at the StopRecordDate (or the day after) at the CLOSING price of the day!

```
ExitPrice:= If(ExitTrigger,OPEN,CLOSE);
```

Cheers,
rnr


----------



## Sir Burr (15 September 2007)

tech/a said:


> And your point is?




Tech, a question that I have been pondering:

Why is Monte Carlo Analysis better than Optimising/Walk Forward?

We've been lead down the Monte Carlo path with Tradesim but is it the better way to go?

What's the difference?

Cheers SB


----------



## bingk6 (15 September 2007)

Sir Burr said:


> Why is Monte Carlo Analysis better than Optimising/Walk Forward?
> 
> We've been lead down the Monte Carlo path with Tradesim but is it the better way to go?
> 
> ...




Hi SB,

Monte Carlo Analysis and Optimising/Walk Forward are not mutually exclusive. I see Optimising/Walk Forward as a single facet process whereas Optimising/Monte Carlo is a multi-facet process and is therefore more thorough.


----------



## nizar (15 September 2007)

Sir Burr said:


> Tech, a question that I have been pondering:
> 
> Why is Monte Carlo Analysis better than Optimising/Walk Forward?
> 
> ...




Hi SB.

Monte Carlo simulations are an exhaustive type of analysis.
I would do a monte carlo analysis (20,000-100,000 simulations) on both in-sample and out-sample data (ie. when doing a walk-forward analysis).

I probably havent understood your question properly


----------



## nizar (15 September 2007)

rnr.
Thanks for the code.

kaleon.
Results below.

System didnt do too flash -- in fact i have left this system and have gone for a weekly approach.

As I suspect both the EOD and Weekly systems did poorly in this period.

Also note that 1-2 years is not enough time to trade systems in which average holding time for a winner is 300-600 days.

The systems -- or in fact any system im working on -- is not designed to do well in bearmarkets. But still it should beat the overall market.

*For Year 2001* (XAO +6.98%)


Monte Carlo Report	

Trade Database Filename	
C:\TradeSimData\EODSystemv2Kaleon.trb	

Simulation Summary	
Simulation Date:	15/09/2007
Simulation Time:	7:42:23 PM
Simulation Duration:	16.09 seconds

Trade Parameters	
Initial Capital:	$30,000.00
Portfolio Limit:	100.00%
Maximum number of open positions:	10
Position Size Model:	Fixed Percent Risk
Percentage of capital risked per trade:	1.50%
Position size limit:	10.00%
Portfolio Heat:	20.00%
Pyramid profits:	Yes
Transaction cost (Trade Entry):	$44.00
Transaction cost (Trade Exit):	$44.00
Margin Requirement:	100.00%
Magnify Position Size(& Risk) according to Margin Req:	No
Margin Requirement Daily Interest Rate (Long Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Long Trades):	0.0000%
Margin Requirement Daily Interest Rate (Short Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Short Trades):	0.0000%

Trade Preferences	
Trading Instrument:	Stocks
Break Even Trades:	Include with losing trades
Trade Position Type:	Process all trades
Exit Order Type:	Default Order
Minimum Trade Size:	$500.00
Accept Partial Trades:	No
Volume Filter:	Reject Trades if Position Size is greater than
	10.00% of the maximum traded volume
Pyramid Trades:	Yes
Favour Trade Pyramid:	Yes
Start Pyramid at any level up to level:	N/A
Maximum Pyramid Level Limited to:	N/A
Maximum Pyramid Count Limited to:	N/A

Simulation Stats	
Number of trade simulations:	5000
Trades processed per simulation:	741
Maximum Number of Trades Executed:	24
Average Number of Trades Executed:	18
Minimum Number of Trades Executed:	15
Standard Deviation:	1.21

Profit Stats	
Maximum Profit:	$3,371.95 (11.24%)
Average Profit:	$1,064.02 (3.55%)
Minimum Profit:	-$3,379.61 (-11.27%)
Standard Deviation:	$1,003.91 (3.35%)
Probability of Profit:	84.28%
Probability of Loss:	15.72%

Percent Winning Trade Stats	
Maximum percentage of winning trades:	68.75%
Average percentage of winning trades:	50.99%
Minimum percentage of winning trades:	26.09%
Standard Deviation:	6.23%

Percent Losing Trade Stats	
Maximum percentage of losing trades:	73.91%
Average percentage of losing Trades:	49.01%
Minimum percentage of losing trades:	31.25%
Standard Deviation:	6.23%

Average Relative Dollar Drawdown Stats	
Maximum of the Average Relative Dollar Drawdown:	$3,183.46
Average of the Average Relative Dollar Drawdown:	$516.52
Minimum of the Average Relative Dollar Drawdown:	$87.16
Standard Deviation:	$380.12

Average Relative Percent Drawdown Stats	
Maximum of the Average Relative Percent Drawdown:	10.6115%
Average of the Average Relative Percent Drawdown:	1.7572%
Minimum of the Average Relative Percent Drawdown:	0.2924%
Standard Deviation:	1.2707%

Maximum Peak-to-Valley Dollar Drawdown Stats	
Maximum Absolute Dollar Drawdown:	$6,532.00
Average Absolute Dollar Drawdown:	$2,781.43
Minimum Absolute Dollar Drawdown:	$934.02
Standard Deviation:	$846.15

Maximum Peak-to-Valley Percent Drawdown Stats	
Maximum Absolute Percent Drawdown:	21.7733%
Average Absolute Percent Drawdown:	9.2714%
Minimum Absolute Percent Drawdown:	3.1134%
Standard Deviation:	2.8205%


----------



## nizar (15 September 2007)

*For 01-07-2001 to 01-03-07* (XAO -18%)




Monte Carlo Report	

Trade Database Filename	
C:\TradeSimData\EODSystemv2KaleonWorst.trb	

Simulation Summary	
Simulation Date:	15/09/2007
Simulation Time:	10:33:06 PM
Simulation Duration:	21.63 seconds

Trade Parameters	
Initial Capital:	$30,000.00
Portfolio Limit:	100.00%
Maximum number of open positions:	10
Position Size Model:	Fixed Percent Risk
Percentage of capital risked per trade:	1.50%
Position size limit:	10.00%
Portfolio Heat:	20.00%
Pyramid profits:	Yes
Transaction cost (Trade Entry):	$44.00
Transaction cost (Trade Exit):	$44.00
Margin Requirement:	100.00%
Magnify Position Size(& Risk) according to Margin Req:	No
Margin Requirement Daily Interest Rate (Long Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Long Trades):	0.0000%
Margin Requirement Daily Interest Rate (Short Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Short Trades):	0.0000%

Trade Preferences	
Trading Instrument:	Stocks
Break Even Trades:	Include with losing trades
Trade Position Type:	Process all trades
Exit Order Type:	Default Order
Minimum Trade Size:	$500.00
Accept Partial Trades:	No
Volume Filter:	Reject Trades if Position Size is greater than
	10.00% of the maximum traded volume
Pyramid Trades:	Yes
Favour Trade Pyramid:	Yes
Start Pyramid at any level up to level:	N/A
Maximum Pyramid Level Limited to:	N/A
Maximum Pyramid Count Limited to:	N/A

Simulation Stats	
Number of trade simulations:	5000
Trades processed per simulation:	987
Maximum Number of Trades Executed:	43
Average Number of Trades Executed:	31
Minimum Number of Trades Executed:	21
Standard Deviation:	3.08

Profit Stats	
Maximum Profit:	$15,886.86 (52.96%)
Average Profit:	$2,462.05 (8.21%)
Minimum Profit:	-$8,992.86 (-29.98%)
Standard Deviation:	$3,556.86 (11.86%)
Probability of Profit:	75.54%
Probability of Loss:	24.46%

Percent Winning Trade Stats	
Maximum percentage of winning trades:	50.00%
Average percentage of winning trades:	21.19%
Minimum percentage of winning trades:	3.23%
Standard Deviation:	6.14%

Percent Losing Trade Stats	
Maximum percentage of losing trades:	96.77%
Average percentage of losing Trades:	78.81%
Minimum percentage of losing trades:	50.00%
Standard Deviation:	6.14%

Average Relative Dollar Drawdown Stats	
Maximum of the Average Relative Dollar Drawdown:	$2,869.04
Average of the Average Relative Dollar Drawdown:	$1,259.00
Minimum of the Average Relative Dollar Drawdown:	$370.87
Standard Deviation:	$395.45

Average Relative Percent Drawdown Stats	
Maximum of the Average Relative Percent Drawdown:	10.5273%
Average of the Average Relative Percent Drawdown:	4.4158%
Minimum of the Average Relative Percent Drawdown:	1.2599%
Standard Deviation:	1.4398%

Maximum Peak-to-Valley Dollar Drawdown Stats	
Maximum Absolute Dollar Drawdown:	$10,297.40
Average Absolute Dollar Drawdown:	$5,451.14
Minimum Absolute Dollar Drawdown:	$1,766.96
Standard Deviation:	$1,385.42

Maximum Peak-to-Valley Percent Drawdown Stats	
Maximum Absolute Percent Drawdown:	34.3247%
Average Absolute Percent Drawdown:	18.1696%
Minimum Absolute Percent Drawdown:	5.8899%
Standard Deviation:	4.6200%


----------



## howardbandy (15 September 2007)

Greetings --

Walk forward is the process of selecting a series of time periods, each a combination of an in-sample period and a following out-of-sample period, performing a system building operation on the in-sample period, recording the results on the out-of-sample period, and analyzing the combined out-of-sample performance.

Monte Carlo is a general term describing an analysis method where pseudo-random changes are made to inputs to a system and the resulting outputs are analyzed.  Monte Carlo can be used to study trading systems in several ways, including:
Perturbing the values of the arguments to a system to test for sensitivity.
Perturbing the data processed to test for sensitivity.
Reordering the sequence of trades to test for equity curve behavior.
Drawing simulated trades from a distribution to test for equity curve behavior.

It is possible to combine walk forward and Monte Carlo.

Thanks,
Howard


----------



## howardbandy (15 September 2007)

Hi Tech/a --



tech/a said:


> And your point is?
> 
> At some point you'll trade your system and it to will be walk forward only realtime.
> Upon completion of a period that period of out of sample trading would have become in sample.
> ...




Sorry for my flip answer -- I misunderstood your post.

I made two posts in the thread on System Robustness -- numbers 120 and 126, both on August 23.  Those describe the process of validating a trading system.

There is no certainty that any given trading system will be profitable when traded with real money.  All we can do is perform the validation steps, with the goal of increasing our confidence that the system will be profitable.  

To me, that means in-sample and out-of-sample testing, walk forward processing, sensitivity analysis, and so forth.

Thanks,
Howard


----------



## julius (15 September 2007)

RE: Out of sample testing

Are the nature of the signals we select for a trading system subject to a natural bias toward performance? ie. a non-systematic process of optimization by the trader who develops the system.

If so, does this support the use of out of sample testing to validate the objectiveness of the design?


----------



## nizar (16 September 2007)

rnr said:


> Hi Nizar,
> 
> 
> 
> ...




Actually Kaleons code and yours both didnt seem to work.
My latest exit is still in 2005 sometime.

I'll email David and see what he says.


----------



## kaleon (16 September 2007)

Nizar,

My file for the ASX 2001 is 17MB. Have you got an email address that would take that size a file. I found there is a big difference testing the current ASX300 back in 2001 compared to testing the actual constituents of the ASX300 as it was back then. My file includes those shares delisted and those with name changes. Was a painstaking task doing it but worth it as I eliminate the survivorship bias.

Now my date filter was for the last date of 2001 on the weekly chart. The last date if you are trading from a daily chart would be 31/12/2001.

So it should be

StDay:=Input(" Start day",1,31,31);
StMnth:=Input("Start month",1,12,12);
StYear:=Input("Start year",1980,2020,2001);
EnDay:=Input(" End day",1,31,31);
EnMnth:=Input(" End month",1,12,12);
EnYear:=Input(" End year",1980,2020,2001);
If((Year() > StYear OR (Year()=StYear AND ((Month() >
StMnth) OR (Month() = StMnth AND DayOfMonth() >=
StDay)))) AND (Year() < EnYear OR (Year()=EnYear AND
((Month() < EnMnth) OR (Month() = EnMnth AND
DayOfMonth() <= EnDay)))) ,1,0);


Also you might have to use the forum.dll latch function in your exit code to help you get exits on the 31/12/2001

eg

le:=Fml("Your entry trigger"); 

lx:=Fml("Your exit trigger");

SE:=0;
SX:=0;

B:=ExtFml("forum.Latch",LE,LX,SE,SX);

B = 0 AND Ref(B,-1) = 1



Also add this to your Entry code. It is from Roy Larsen


Date Filter1
StDay:=Input(" Start day",1,31,1);
StMnth:=Input("Start month",1,12,1);
StYear:=Input("Start year",1980,2020,2001);
EnDay:=Input(" End day",1,31,31);
EnMnth:=Input(" End month",1,12,12);
EnYear:=Input(" End year",1980,2020,2001);
If((Year() > StYear OR (Year()=StYear AND ((Month() >
StMnth) OR (Month() = StMnth AND DayOfMonth() >=
StDay)))) AND (Year() < EnYear OR (Year()=EnYear AND
((Month() < EnMnth) OR (Month() = EnMnth AND
DayOfMonth() <= EnDay)))) ,1,0);


Note this date filter is used with a different purpose from the other date filter I gave earlier and that is why I named this date filter1. This date filter when incorporated with your entry code forces only entries that occur between 1/1/2001 and 31/1/2001

So for example

date:=fml("date filter1");
entry:=cross(c,mov(c,21,s));
entry and date

Hope that helps

Also to get the 0.75% go to the preference section where you set your parameters. You have transaction costs:Fixed costs, fixed costs per share/contract,fractional costs. Choose the fractional cost and this will enable you to choose your transaction cost as a fraction of your cost for entry into a trade.

I try to stress my system as much as possible to see how it performs.


Also I chose 2001 as opposed to 2002 because of the unexpected events of sep 11. A lot of systems would have been unprepared for that and most systems perform worse in 2001 than 2002 even though 2002 was a down year for the XAO.


----------



## kaleon (16 September 2007)

Here is my system using my ASX300 list from 2001

It is a weekly system


Detailed Report
(Kaleon asx300 2001list)

Simulation Summary
Simulation Date:                                            15/09/2007
Simulation Time:                                           11:35:34 PM
Simulation Duration:                                      0.13 seconds

Trade Summary
Earliest Entry Date in the Trade Database:                  25/01/2001
Latest Entry Date in the Trade Database:                    21/12/2001
Earliest Exit Date in the Trade Database:                   16/02/2001
Latest Exit Date in the Trade Database:                     28/12/2001

Start Trade Entry Date:                                     25/01/2001
Stop Trade Entry Date:                                      21/12/2001
First Entry Date:                                           25/01/2001
Last Entry Date:                                            21/12/2001
First Exit Date:                                            16/02/2001
Last Exit Date:                                             28/12/2001

Total Trading duration:                                       337 days

Profit Summary
Profit Status:                                              PROFITABLE
Starting Capital:                                           $30,000.00
Finishing Capital:                                          $30,393.37
Maximum Equity/(Date):                            $393.37 (28/12/2001)
Minimum Equity/(Date):                         -$2,899.20 (21/09/2001)
Gross Trade Profit:                                 $3,728.24 (12.43%)
Gross Trade Loss:                                 -$3,334.87 (-11.12%)
Total Net Profit:                                      $393.37 (1.31%)
Average Profit per Trade:                                       $19.67
Profit Factor:                                                  1.1180
Profit Index:                                                   10.55%
Total Transaction Cost:                                        $744.76
Total Slippage:                                                $938.41
Daily Compound Interest Rate:                                  0.0039%
Annualized Compound Interest Rate:                             1.4210%

Trade Statistics
Trades Processed:                                                   23
Trades Taken:                                                       20
Partial Trades Taken:                                                0
Trades Rejected:                                                     3
Winning Trades:                                             9 (45.00%)
Losing Trades:                                             11 (55.00%)
Breakeven Trades:                                            0 (0.00%)

Normal Exit Trades:                                        16 (80.00%)
Delayed Normal Exit Trades:                                  0 (0.00%)
Open Trades:                                                 0 (0.00%)
Protective Stop Exit Trades:                                4 (20.00%)
Time Stop Exit Trades:                                       0 (0.00%)
Profit Stop Exit Trades:                                     0 (0.00%)

Largest Winning Trade/(Date):                   $1,708.29 (28/12/2001)
Largest Losing Trade/(Date):                     -$629.55 (21/09/2001)
Average Winning Trade:                                         $414.25
Average Losing Trade:                                         -$303.17
Average Win/Average Loss:                                       1.3664

Trade Duration Statistics
(All Trades)
Maximum Trade Duration:                                     239 (days)
Minimum Trade Duration:                                       7 (days)
Average Trade Duration:                                      89 (days)
(Winning Trades)
Maximum Trade Duration:                                     239 (days)
Minimum Trade Duration:                                       7 (days)
Average Trade Duration:                                     121 (days)
(Losing Trades)
Maximum Trade Duration:                                     140 (days)
Minimum Trade Duration:                                       7 (days)
Average Trade Duration:                                      64 (days)

Consecutive Trade Statistics
Maximum consecutive winning trades:                                  3
Maximum consecutive losing trades:                                   8
Average consecutive winning trades:                               2.25
Average consecutive losing trades:                                2.75

Trade Expectation Statistics
Normalized Expectation per dollar risked:                      $0.0910
Maximum Reward/Risk ratio:                                        5.04
Minimum Reward/Risk ratio:                                       -1.37
Average Positive Reward/Risk ratio:                               1.07
Average Negative Reward/Risk ratio:                              -0.71

Relative Drawdown
Maximum Dollar Drawdown/(Date):                 $2,978.82 (21/09/2001)
Maximum Percentage Drawdown/(Date):               9.9290% (21/09/2001)

Absolute (Peak-to-Valley) Dollar Drawdown
Maximum Dollar Drawdown:                           $2,978.82 (9.9290%)
Capital Peak/(Date):                             $30,000.00 (18000101)
Capital Valley/(Date):                         $27,021.18 (21/09/2001)

Absolute (Peak-to-Valley) Percent Drawdown
Maximum Percentage Drawdown:                       9.9290% ($2,978.82)
Capital Peak/(Date):                             $30,000.00 (18000101)
Capital Valley/(Date):                         $27,021.18 (21/09/2001)


----------



## kaleon (16 September 2007)

And now using the current ASX300 list


Detailed Report
(Kaleon asx300 2007 list)

Simulation Summary
Simulation Date:                                            15/09/2007
Simulation Time:                                           11:39:19 PM
Simulation Duration:                                      0.14 seconds

Trade Summary
Earliest Entry Date in the Trade Database:                  25/01/2001
Latest Entry Date in the Trade Database:                     7/12/2001
Earliest Exit Date in the Trade Database:                   16/03/2001
Latest Exit Date in the Trade Database:                     28/12/2001

Start Trade Entry Date:                                     25/01/2001
Stop Trade Entry Date:                                       7/12/2001
First Entry Date:                                           25/01/2001
Last Entry Date:                                             7/12/2001
First Exit Date:                                            16/03/2001
Last Exit Date:                                             28/12/2001

Total Trading duration:                                       337 days

Profit Summary
Profit Status:                                              PROFITABLE
Starting Capital:                                           $30,000.00
Finishing Capital:                                          $35,642.40
Maximum Equity/(Date):                          $5,642.40 (28/12/2001)
Minimum Equity/(Date):                         -$1,892.83 (28/12/2001)
Gross Trade Profit:                                 $8,250.36 (27.50%)
Gross Trade Loss:                                  -$2,607.96 (-8.69%)
Total Net Profit:                                   $5,642.40 (18.81%)
Average Profit per Trade:                                      $331.91
Profit Factor:                                                  3.1635
Profit Index:                                                   68.39%
Total Transaction Cost:                                        $640.34
Total Slippage:                                              $1,498.51
Daily Compound Interest Rate:                                  0.0512%
Annualized Compound Interest Rate:                            20.5214%

Trade Statistics
Trades Processed:                                                   18
Trades Taken:                                                       17
Partial Trades Taken:                                                0
Trades Rejected:                                                     1
Winning Trades:                                             8 (47.06%)
Losing Trades:                                              9 (52.94%)
Breakeven Trades:                                            0 (0.00%)

Normal Exit Trades:                                        15 (88.24%)
Delayed Normal Exit Trades:                                  0 (0.00%)
Open Trades:                                                 0 (0.00%)
Protective Stop Exit Trades:                                2 (11.76%)
Time Stop Exit Trades:                                       0 (0.00%)
Profit Stop Exit Trades:                                     0 (0.00%)

Largest Winning Trade/(Date):                   $3,242.89 (28/12/2001)
Largest Losing Trade/(Date):                     -$734.03 (21/09/2001)
Average Winning Trade:                                       $1,031.29
Average Losing Trade:                                         -$289.77
Average Win/Average Loss:                                       3.5590

Trade Duration Statistics
(All Trades)
Maximum Trade Duration:                                     337 (days)
Minimum Trade Duration:                                      21 (days)
Average Trade Duration:                                     148 (days)
(Winning Trades)
Maximum Trade Duration:                                     337 (days)
Minimum Trade Duration:                                      50 (days)
Average Trade Duration:                                     217 (days)
(Losing Trades)
Maximum Trade Duration:                                     196 (days)
Minimum Trade Duration:                                      21 (days)
Average Trade Duration:                                      86 (days)

Consecutive Trade Statistics
Maximum consecutive winning trades:                                  4
Maximum consecutive losing trades:                                   3
Average consecutive winning trades:                               1.60
Average consecutive losing trades:                                2.25

Trade Expectation Statistics
Normalized Expectation per dollar risked:                      $0.7200
Maximum Reward/Risk ratio:                                        7.12
Minimum Reward/Risk ratio:                                       -1.55
Average Positive Reward/Risk ratio:                               2.24
Average Negative Reward/Risk ratio:                              -0.63

Relative Drawdown
Maximum Dollar Drawdown/(Date):                 $1,187.06 (17/08/2001)
Maximum Percentage Drawdown/(Date):               3.9250% (17/08/2001)

Absolute (Peak-to-Valley) Dollar Drawdown
Maximum Dollar Drawdown:                           $2,248.87 (7.4360%)
Capital Peak/(Date):                           $30,241.17 (16/03/2001)
Capital Valley/(Date):                         $27,992.30 (28/12/2001)

Absolute (Peak-to-Valley) Percent Drawdown
Maximum Percentage Drawdown:                       7.4360% ($2,248.87)
Capital Peak/(Date):                           $30,241.17 (16/03/2001)
Capital Valley/(Date):                         $27,992.30 (28/12/2001)


Quite a big difference.


I think you will find your system deep in the red when you use my ASX300 list.


----------



## tech/a (16 September 2007)

Sir Burr said:


> Tech, a question that I have been pondering:
> 
> Why is Monte Carlo Analysis better than Optimising/Walk Forward?
> 
> ...




Off to run (Well actually shuffle) the City Bay.Will make some comments this afternoon.
Bingk Julius and Howard all make some points Id like to add to.


----------



## theasxgorilla (16 September 2007)

Sir Burr said:


> Why is Monte Carlo Analysis better than Optimising/Walk Forward?




An extention of this question, at risk of digressing the thread, I wonder about the assertion Curtis Faith makes that regular Monte Carlo testing breaks up the sequencing of Black Swan events and hence the distribution of results from a series of Monte Carlo might be misleading.  He claims that his TradingBlox software has parameters to keep these events properly grouped.  I haven't investigated this and it's of course entirely possible that my interpretation of what he was saying is also incorrect...but since we're calling Monte Carlo into question we might as well take it the whole 9 yards.

For this reason I think GPs idea of randomly dropping a proportion of trades has a lot of merit.

ASX.G


----------



## howardbandy (16 September 2007)

Hi Julius --



julius said:


> RE: Out of sample testing
> 
> Are the nature of the signals we select for a trading system subject to a natural bias toward performance? ie. a non-systematic process of optimization by the trader who develops the system.
> 
> If so, does this support the use of out of sample testing to validate the objectiveness of the design?




I'm not sure you meant this for me, but I'll comment.

In my opinion, the following is a reasonable outline for the design, test, and validation of a trading system.  Of course there are personal biases -- they should be identified and incorporated into the objective function if it is possible to do that, or decided upon in one of the early steps before system design and testing starts.

Step 1 is definition of the objective function.  The objective function assigns scores to features that the system designer wants to reward or punish.  Over any given ticker and time period, alternative systems are ranked by objective function -- higher values are better.

Step 2 is selection of frequency of trading, frequency of data collection, when to enter, style of orders, followed by account size, issues to trade, number of issues to hold, and statement of expectations.

Step 3 is design of the system -- the entries and exits.

Step 4 is selection of the data to use to develop the system.  What period of time, how long is each in-sample period, how long is each out-of-sample period, how many walk forward steps, what gets optimized, and so forth.  

Step 5 are the optimization and automatic walk forward runs.  If the objective function has been well designed, there are no judgments made during the optimization and walk forward -- everything is automatic. When all walk forward steps have been completed, evaluate the concatenated out-of-sample results.  (Always ignore all in-sample results -- they are always good and have no predictive value.)  Followed by a one-time decision whether to trust the system and trade it, or go back to the drawing board.  There is no guarantee that any system will be profitable -- the best we can hope for is a high level of confidence.  And that level of confidence is directly related to the rigor with which the system design, test, and validation was carried out.

Step 6 -- only necessary if the system is being traded live.  Take every trade without exception!  Monitor the performance of the system to evaluate whether it is continuing to work or not.   Periodically, typically after the time of an in-sample period has passed, reoptimize and continue trading using the new argument values.    

Thanks,
Howard


----------



## howardbandy (16 September 2007)

theasxgorilla said:


> An extention of this question, at risk of digressing the thread, I wonder about the assertion Curtis Faith makes that regular Monte Carlo testing breaks up the sequencing of Black Swan events and hence the distribution of results from a series of Monte Carlo might be misleading.  He claims that his TradingBlox software has parameters to keep these events properly grouped.  I haven't investigated this and it's of course entirely possible that my interpretation of what he was saying is also incorrect...but since we're calling Monte Carlo into question we might as well take it the whole 9 yards.
> 
> For this reason I think GPs idea of randomly dropping a proportion of trades has a lot of merit.
> 
> ASX.G




Hi Gorilla --

First, let me state my opinion that Monte Carlo techniques can be very valuable in helping evaluate the performance of a trading system, and help decide whether a trading system is likely to be profitable in the future.  

Curtis Faith is one of the original Turtles.  Their systems were classical breakouts, with low percentage of winning trades and high ratio of winning amount to losing amount.  They traded commodities and looked for favorable Black Swans.  They were almost always in either a long position or a short position hoping for the big move or big trend.

Monte Carlo techniques can be applied in many ways.  One way is to reorder observed actual trades.  Another is to first create a statistical distribution from the observed actual trades, then make repeated random selections from that distribution.  

Both of these techniques create simulated equity curves, with the hope that the distribution of these simulated equity curves gives some insight into the possible behavior of the trading system's equity curve, including how Black Swan events might affect the system. 

If the question is "Can either of these techniques help describe the effect of Black Swan events and help prepare the trader for them," I think the answer is clearly "yes."

If the question is "Can either of these techniques help avoid future Black Swans," I think the answer is clearly "no."

Thanks,
Howard


----------



## theasxgorilla (16 September 2007)

howardbandy said:


> Monte Carlo techniques can be applied in many ways.  One way is to reorder observed actual trades.  Another is to first create a statistical distribution from the observed actual trades, then make repeated random selections from that distribution.
> 
> Both of these techniques create simulated equity curves, with the hope that the distribution of these simulated equity curves gives some insight into the possible behavior of the trading system's equity curve, including how Black Swan events might affect the system.




Thanks for the quick response Howard.  _Boy have I got a lot to learn_.

His point was that if you don't keep the observed actual trades together in time blocks sufficiently large enough to group the sequence of trades in the Black Swan events then you won't encounter them as intact events in MC analysis often enough to give an accurate indication of their impact on the distribution of results.  That was my understanding.  It made sense to me.

I agree that doing it his way won't help you get around these events...but I would expect that the distribution of results might better represent the probability that you are going to encounter, say for example, the severe drawdowns often associated with these events.

I actually don't understand what the blue version of MC analysis does, so I'm not sure if it addresses the issue.  I'd be curious to know which version of the above TradeSim employs.

Thanks again for the reply.

ASX.G


----------



## Shane Baker (16 September 2007)

Hi ASXG

My understanding is that TradeSim MC randomises the trades but keeps the date sequence intact. So if there was capital available for one trade, but six signals occurred, then normally TradeSim would take the first share alphabetically. 

In MC it shuffles the sequence of trades to give a  range of possible portfolios but maintains the date sequence of entries.

Hope this helps

Cheers

Shane


----------



## julius (16 September 2007)

Thanks for your response Howard.

I thought I might try to better articulate the original question, given the interesting points of difference in regards to out of sample testing:

The steps Howard lists above serve as systematic alternative to an unstructured, or 'natural' development process. The purpose behind this is to minimise possible subjective influence on the system; which is ultimately the goal of all systems development, regardless of the approach. In this way, we maximise the validity of the system out of sample.

Most technical traders tend to employ a number of different 'indicators' which are mixed and match based on the current situation or in other words the percieved market state. It follows then that the selection of the indicators chosen to be used in a system are subject to some subjective judgement. Note that I refer to indicators in the general sense.

The degree to which this may influence live trading performance depends, I think, on the nature of the system (ie. trend/swing, short term/long term, etc) and also the indicators chosen - is it the combination of the indicators or the specific parameters which are contributing to performance?

Somewhere else Howard posted something about system decay, however I haven't heard Nick Radge address this issue and I suspect that it is not a major part of his methodology - not that either method is better, just different, and perhaps points of difference may arise from the nature of the systems being developed. "There is more than one way to skin a cat"

Interested to hear others thoughts on these issues.

Apologies Nick & Howard if I have misinterpreted anything, feel free to correct me.


----------



## nizar (16 September 2007)

Shane Baker said:


> Hi ASXG
> 
> My understanding is that TradeSim MC randomises the trades but keeps the date sequence intact. So if there was capital available for one trade, but six signals occurred, then normally TradeSim would take the first share alphabetically.
> 
> ...




Yes thats my understanding as well, and its how David explains it in the TradeSim manual.


----------



## nizar (16 September 2007)

kaleon said:


> Nizar,
> 
> My file for the ASX 2001 is 17MB. Have you got an email address that would take that size a file. I found there is a big difference testing the current ASX300 back in 2001 compared to testing the actual constituents of the ASX300 as it was back then. My file includes those shares delisted and those with name changes. Was a painstaking task doing it but worth it as I eliminate the survivorship bias.
> 
> ...




Kaleon.
I found a much easier way to exit all open trades at the date which you want.

In the MS exploration, as part of your exit trigger put your exit and then add this:

ExitTrigger:=("your exit") OR ExtFml( "TradeSim.SetTriggerAtDate",dd,mm,yyyy);

Hope this helps.

By the way, it seems your a gun at MetaStock code.
Maybe you can help me with this thread:
https://www.aussiestockforums.com/forums/showthread.php?p=200922#post200922


----------



## nizar (16 September 2007)

Ok heres mine over the year 2001 over the whole market with a liquidity filter (its included in my entry criteria).

Detailed Report			

Trade Database Filename			
C:\TradeSimData\WeeklyMaster07for2001.trb			

Simulation Summary			
Simulation Date:	16/09/2007		
Simulation Time:	2:01:15 PM		
Simulation Duration:	1.45 seconds		

Trade Summary			
Earliest Entry Date in the Trade Database:	4/01/2001		
Latest Entry Date in the Trade Database:	28/12/2001		
Earliest Exit Date in the Trade Database:	25/01/2001		
Latest Exit Date in the Trade Database:	11/01/2002		

Start Trade Entry Date:	4/01/2001		
Stop Trade Entry Date:	28/12/2001		
First Entry Date:	4/01/2001		
Last Entry Date:	28/12/2001		
First Exit Date:	25/01/2001		
Last Exit Date:	11/01/2002		

Total Trading duration:	372 days		

Profit Summary			
Profit Status:	PROFITABLE		
Starting Capital:	$30,000.00		
Finishing Capital:	$31,413.62		
Maximum Equity/(Date):	$1,413.62 (11/01/2002)		
Minimum Equity/(Date):	-$1,757.07 (23/02/2001)		
Gross Trade Profit:	$3,600.25 (12.00%)		
Gross Trade Loss:	-$2,186.63 (-7.29%)		
Total Net Profit:	$1,413.62 (4.71%)		
Average Profit per Trade:	$128.51		
Profit Factor:	1.6465		
Profit Index:	39.26%		
Total Transaction Cost:	$484.00		
Total Slippage:	$0.00		
Total Trade Interest:	$0.00		
Daily Compound Interest Rate:	0.0124%		
Annualized Compound Interest Rate:	4.6214%		

Trade Statistics			
Trades Processed:	761		
Trades Taken:	11		
Partial Trades Taken:	0		
Trades Rejected:	185		
Winning Trades:	7 (63.64%)		
Losing Trades:	4 (36.36%)		
Breakeven Trades:	0 (0.00%)		

Largest Winning Trade/(Date):	$944.46 (21/12/2001)		
Largest Losing Trade/(Date):	-$1,069.00 (16/02/2001)		
Average Winning Trade:	$514.32		
Average Losing Trade:	-$546.66		
Average Win/Average Loss:	0.9408		

Trade Breakdown	Long and Short Trades	Long Trades	Short Trades
Normal Exit:	11 (100.00%)	11 (100.00%)	0 (0.00%)

Total Trades:	11 (100.00%)	11 (100.00%)	0 (0.00%)

Trade Duration Statistics	Winning and Losing Trades	Winning Trades	Losing Trades
Maximum Trade Duration:	350 (days)	350 (days)	98 (days)
Minimum Trade Duration:	14 (days)	14 (days)	20 (days)
Average Trade Duration:	158.45 (days)	219.14 (days)	52.25 (days)

Consecutive Trade Statistics			
Maximum consecutive winning trades:	5		
Maximum consecutive losing trades:	3		
Average consecutive winning trades:	3.50		
Average consecutive losing trades:	2.00		

Relative Drawdown			
Maximum Dollar Drawdown/(Date):	$1,845.07 (23/02/2001)		
Maximum Percentage Drawdown/(Date):	6.1500% (23/02/2001)		

Absolute (Peak-to-Valley) Dollar Drawdown			
Maximum Dollar Drawdown:	$1,845.07 (6.1500%)		
Capital Peak/(Date):	$30,000.00 (18000101)		
Capital Valley/(Date):	$28,154.93 (23/02/2001)		

Absolute (Peak-to-Valley) Percent Drawdown			
Maximum Percentage Drawdown:	6.1500% ($1,845.07)		
Capital Peak/(Date):	$30,000.00 (18000101)		
Capital Valley/(Date):	$28,154.93 (23/02/2001)


----------



## nizar (16 September 2007)

I should also say the system i am running these tests are is weekly.

This is testing over the current ASX300.

Kaleon, would you be able to tell me which stocks are part of the 2001 list for ASX300.
Then i can create my own custom folder as i have the current ASX300 and also a folder of delisted securities.

Detailed Report			

Trade Database Filename			
C:\TradeSimData\WeeklyMaster07for2001CurrentASX300.trb			

Simulation Summary			
Simulation Date:	16/09/2007		
Simulation Time:	2:11:28 PM		
Simulation Duration:	0.20 seconds		

Trade Summary			
Earliest Entry Date in the Trade Database:	4/01/2001		
Latest Entry Date in the Trade Database:	28/12/2001		
Earliest Exit Date in the Trade Database:	25/01/2001		
Latest Exit Date in the Trade Database:	11/01/2002		

Start Trade Entry Date:	4/01/2001		
Stop Trade Entry Date:	28/12/2001		
First Entry Date:	4/01/2001		
Last Entry Date:	21/09/2001		
First Exit Date:	25/01/2001		
Last Exit Date:	11/01/2002		

Total Trading duration:	372 days		

Profit Summary			
Profit Status:	PROFITABLE		
Starting Capital:	$30,000.00		
Finishing Capital:	$34,041.46		
Maximum Equity/(Date):	$4,041.46 (11/01/2002)		
Minimum Equity/(Date):	-$1,241.00 (16/02/2001)		
Gross Trade Profit:	$5,712.02 (19.04%)		
Gross Trade Loss:	-$1,670.56 (-5.57%)		
Total Net Profit:	$4,041.46 (13.47%)		
Average Profit per Trade:	$449.05		
Profit Factor:	3.4192		
Profit Index:	70.75%		
Total Transaction Cost:	$396.00		
Total Slippage:	$0.00		
Total Trade Interest:	$0.00		
Daily Compound Interest Rate:	0.0340%		
Annualized Compound Interest Rate:	13.2020%		

Trade Statistics			
Trades Processed:	425		
Trades Taken:	9		
Partial Trades Taken:	0		
Trades Rejected:	106		
Winning Trades:	6 (66.67%)		
Losing Trades:	3 (33.33%)		
Breakeven Trades:	0 (0.00%)		

Largest Winning Trade/(Date):	$2,788.46 (11/01/2002)		
Largest Losing Trade/(Date):	-$1,069.00 (16/02/2001)		
Average Winning Trade:	$952.00		
Average Losing Trade:	-$556.85		
Average Win/Average Loss:	1.7096		

Trade Breakdown	Long and Short Trades	Long Trades	Short Trades
Normal Exit:	9 (100.00%)	9 (100.00%)	0 (0.00%)

Total Trades:	9 (100.00%)	9 (100.00%)	0 (0.00%)

Trade Duration Statistics	Winning and Losing Trades	Winning Trades	Losing Trades
Maximum Trade Duration:	371 (days)	371 (days)	98 (days)
Minimum Trade Duration:	20 (days)	112 (days)	20 (days)
Average Trade Duration:	195.22 (days)	266.17 (days)	53.33 (days)

Consecutive Trade Statistics			
Maximum consecutive winning trades:	4		
Maximum consecutive losing trades:	2		
Average consecutive winning trades:	3.00		
Average consecutive losing trades:	1.50		

Relative Drawdown			
Maximum Dollar Drawdown/(Date):	$1,329.00 (16/02/2001)		
Maximum Percentage Drawdown/(Date):	4.4300% (16/02/2001)		

Absolute (Peak-to-Valley) Dollar Drawdown			
Maximum Dollar Drawdown:	$1,329.00 (4.4300%)		
Capital Peak/(Date):	$30,000.00 (18000101)		
Capital Valley/(Date):	$28,671.00 (16/02/2001)		

Absolute (Peak-to-Valley) Percent Drawdown			
Maximum Percentage Drawdown:	4.4300% ($1,329.00)		
Capital Peak/(Date):	$30,000.00 (18000101)		
Capital Valley/(Date):	$28,671.00 (16/02/2001)


----------



## nizar (16 September 2007)

julius said:


> Somewhere else Howard posted something about system decay, however I haven't heard Nick Radge address this issue and I suspect that it is not a major part of his methodology - not that either method is better, just different, and perhaps points of difference may arise from the nature of the systems being developed. "There is more than one way to skin a cat"
> 
> Interested to hear others thoughts on these issues.




Well i am nowhere near as qualified as Nick Radge or Howard to speak on this matter, but will put forward my view nonetheless.

Howard did say that systems do decay over time, and he gave the Donchian Style breakout systems that worked so well in the 70s and 80s -- and in his view do not work well now -- as an example.

Now many of us here who have successfully traded trend following systems in the last few years would beg to differ.

In my point of view, mechanical systems have a far sooner expiry date when traded on the futures market than on the stockmarket.

Why? Because as a % of total market participants, i would suspect that mechanical systems traders make up a far greater proportion in the futures markets than they do in the stockmarket. As a result of this, there is much more research and development going on into designing mechanical systems in the futures market compared to in the stockmarket.

BlackStar's paper in 2005 suggested in its introduction that it was not commonplace for trend following systems to be traded in the stockmarket, even though it had been practiced in the futures market for decades.

Also, In the stockmarket, you have buy-and-hold types, and of course, how can we forget "believers" (look at the BMN thread to know what i mean). I dont think there are many of those types in the futures markets (?).

Any mistakes/misunderstanding in the above is probably from total ignorance. Iv never traded futures, and im only just getting started in systems testing and design. Its pretty much based on what Iv read and from speaking to others.


----------



## kaleon (16 September 2007)

Nizar,

This is the list of ASX300 constituents as of 31 Dec 2000

AAT
AAU
ABC
ADA
ADB
ADP
ADZ
AFI
AFT
AGG
AGH
AGK
AHD
AHX
AIP
AIX
AJR
ALL
ALS
ALU
AMC
AMM
AMP
ANM
ANN
ANZ
AOG
AOR
APA
APL
APN
AQP
ARG
ART
ASC
ASX
AUD
AUN
AXA
AXN
BAM
BBG
BDL
BEN
BHP
BIR
BLD
BOQ
BRL
BRS
BRY
BTA
BWA
BWP
BXB
CAA
CAG
CAT
CBA
CCL
CDO
CEP
CEQ
CEW
CFT
CFX
CGJ
CIR
CLI
CLT
CND
CNP
COA
COH
CPA
CPU
CRG
CSL
CSR
CTL
CTR
CTX
CWO
CXP
DDF
DGD
DID
DIT
DJS
DJW
DRT
ECP
EDI
EML
ENE
ENG
ENV
ERG
ETR
EZY
FCL
FEA
FFL
FGL
FHF
FLX
FOA
FXJ
GDM
GGL
GHG
GMF
GPT
GTP
GUD
GWT
HFY
HIH
HLY
HRP
HSL
HSN
HTA
HVN
HWE
HYO
IAG
IAM
IBA
IBC
ICC
IDT
IFM
IHG
IIF
ILU
IOF
IPG
IPH
ITG
IXL
JBM
JDV
JHX
JUP
KAZ
KYC
LEI
LFE
LHG
LLC
LMS
LNN
LOK
MAQ
MBL
MCW
MGI
MGR
MIA
MIG
MIM
MIS
MLB
MLE
MME
MOF
MRE
MRZ
MTS
MYO
NAB
NCM
NDY
NEV
NFD
NLX
NMB
NRT
NUF
NVS
NWL
NWS
OEC
OIL
OML
ONE
ONX
OPS
ORG
ORI
OSH
OST
OTT
PAO
PAS
PBB
PBL
PGL
PHY
PLF
PLT
PMC
PMM
PMP
POF
PPT
PPX
PRG
PRK
PRT
PTD
PTZ
PWR
PWT
QAN
QBE
QCH
QTK
RDF
RGS
RIC
RIO
RKN
RMD
ROC
RSG
SBC
SCP
SEN
SEV
SFH
SFL
SGB
SGM
SGP
SGS
SGW
SHL
SIP
SKE
SLX
SMI
SMX
SNX
SOH
SPP
SPT
SRA
SRP
SRV
SSX
STG
STO
SUN
SWS
SYB
TAB
TAH
TAP
TCN
TEM
TEN
TGG
THG
TIF
TIM
TLA
TLS
TMN
TNE
TOL
TOR
UCL
UEL
UGL
USC
UTB
UXC
VCM
VEA
VRL
VSL
WAN
WBC
WES
WFA
WFT
WMR
WMT
WOW
WPL
WSF
WYL
ZTL


----------



## kaleon (16 September 2007)

Nizar

With your results using the current ASX300 list you have not include slippage and transaction costs. Have you set the entry order control as a stop order and to unconditionally enter a trade?

If I followed your set up my results are:




Detailed Report
(Kaleon asx300list no transaction/slippage, default entry)

Simulation Summary
Simulation Date:                                            16/09/2007
Simulation Time:                                            1:16:29 PM
Simulation Duration:                                      0.14 seconds

Trade Summary
Earliest Entry Date in the Trade Database:                  25/01/2001
Latest Entry Date in the Trade Database:                     7/12/2001
Earliest Exit Date in the Trade Database:                   16/03/2001
Latest Exit Date in the Trade Database:                     28/12/2001

Start Trade Entry Date:                                     25/01/2001
Stop Trade Entry Date:                                       7/12/2001
First Entry Date:                                           25/01/2001
Last Entry Date:                                             7/12/2001
First Exit Date:                                            16/03/2001
Last Exit Date:                                             28/12/2001

Total Trading duration:                                       337 days

Profit Summary
Profit Status:                                              PROFITABLE
Starting Capital:                                           $30,000.00
Finishing Capital:                                          $37,850.58
Maximum Equity/(Date):                          $7,850.58 (28/12/2001)
Minimum Equity/(Date):                           -$954.20 (21/09/2001)
Gross Trade Profit:                                 $9,547.27 (31.82%)
Gross Trade Loss:                                  -$1,696.69 (-5.66%)
Total Net Profit:                                   $7,850.58 (26.17%)
Average Profit per Trade:                                      $461.80
Profit Factor:                                                  5.6270
Profit Index:                                                   82.23%
Total Transaction Cost:                                          $0.00
Total Slippage:                                                  $0.00
Daily Compound Interest Rate:                                  0.0690%
Annualized Compound Interest Rate:                            28.6290%

Trade Statistics
Trades Processed:                                                   18
Trades Taken:                                                       17
Partial Trades Taken:                                                0
Trades Rejected:                                                     1
Winning Trades:                                            11 (64.71%)
Losing Trades:                                              6 (35.29%)
Breakeven Trades:                                            0 (0.00%)

Normal Exit Trades:                                        15 (88.24%)
Delayed Normal Exit Trades:                                  0 (0.00%)
Open Trades:                                                 0 (0.00%)
Protective Stop Exit Trades:                                2 (11.76%)
Time Stop Exit Trades:                                       0 (0.00%)
Profit Stop Exit Trades:                                     0 (0.00%)

Largest Winning Trade/(Date):                   $3,411.60 (28/12/2001)
Largest Losing Trade/(Date):                     -$564.06 (21/09/2001)
Average Winning Trade:                                         $867.93
Average Losing Trade:                                         -$282.78
Average Win/Average Loss:                                       3.0693

Trade Duration Statistics
(All Trades)
Maximum Trade Duration:                                     337 (days)
Minimum Trade Duration:                                      21 (days)
Average Trade Duration:                                     148 (days)
(Winning Trades)
Maximum Trade Duration:                                     337 (days)
Minimum Trade Duration:                                      28 (days)
Average Trade Duration:                                     177 (days)
(Losing Trades)
Maximum Trade Duration:                                     196 (days)
Minimum Trade Duration:                                      21 (days)
Average Trade Duration:                                      95 (days)

Consecutive Trade Statistics
Maximum consecutive winning trades:                                  4
Maximum consecutive losing trades:                                   2
Average consecutive winning trades:                               2.20
Average consecutive losing trades:                                1.50

Trade Expectation Statistics
Normalized Expectation per dollar risked:                      $1.0000
Maximum Reward/Risk ratio:                                        7.74
Minimum Reward/Risk ratio:                                       -1.23
Average Positive Reward/Risk ratio:                               1.95
Average Negative Reward/Risk ratio:                              -0.63

Relative Drawdown
Maximum Dollar Drawdown/(Date):                   $909.46 (17/08/2001)
Maximum Percentage Drawdown/(Date):               2.9850% (17/08/2001)

Absolute (Peak-to-Valley) Dollar Drawdown
Maximum Dollar Drawdown:                           $1,416.92 (4.6510%)
Capital Peak/(Date):                           $30,462.72 (16/03/2001)
Capital Valley/(Date):                         $29,045.80 (21/09/2001)

Absolute (Peak-to-Valley) Percent Drawdown
Maximum Percentage Drawdown:                       4.6510% ($1,416.92)
Capital Peak/(Date):                           $30,462.72 (16/03/2001)
Capital Valley/(Date):                         $29,045.80 (21/09/2001)


----------



## kaleon (16 September 2007)

That last tradesim test was using fixed % risk. 

Now using equal dollar units of $5000 per trade with no slippage and no transaction costs and default entry which is what I think you have used for your results I get this:

Detailed Report
(Kaleon asx300 current list no transaction/slippage, default entry, fixed dollar units)

Simulation Summary
Simulation Date:                                            16/09/2007
Simulation Time:                                            1:20:25 PM
Simulation Duration:                                      0.09 seconds

Trade Summary
Earliest Entry Date in the Trade Database:                  25/01/2001
Latest Entry Date in the Trade Database:                     7/12/2001
Earliest Exit Date in the Trade Database:                   16/03/2001
Latest Exit Date in the Trade Database:                     28/12/2001

Start Trade Entry Date:                                     25/01/2001
Stop Trade Entry Date:                                       7/12/2001
First Entry Date:                                           25/01/2001
Last Entry Date:                                            16/11/2001
First Exit Date:                                            16/03/2001
Last Exit Date:                                             28/12/2001

Total Trading duration:                                       337 days

Profit Summary
Profit Status:                                              PROFITABLE
Starting Capital:                                           $30,000.00
Finishing Capital:                                          $42,368.62
Maximum Equity/(Date):                         $12,368.62 (28/12/2001)
Minimum Equity/(Date):                             $88.31 (17/08/2001)
Gross Trade Profit:                                $14,432.18 (48.11%)
Gross Trade Loss:                                  -$2,063.57 (-6.88%)
Total Net Profit:                                  $12,368.62 (41.23%)
Average Profit per Trade:                                    $1,124.42
Profit Factor:                                                  6.9938
Profit Index:                                                   85.70%
Total Transaction Cost:                                          $0.00
Total Slippage:                                                  $0.00
Daily Compound Interest Rate:                                  0.1025%
Annualized Compound Interest Rate:                            45.3381%

Trade Statistics
Trades Processed:                                                   18
Trades Taken:                                                       11
Partial Trades Taken:                                                0
Trades Rejected:                                                     7
Winning Trades:                                             8 (72.73%)
Losing Trades:                                              3 (27.27%)
Breakeven Trades:                                            0 (0.00%)

Normal Exit Trades:                                        10 (90.91%)
Delayed Normal Exit Trades:                                  0 (0.00%)
Open Trades:                                                 0 (0.00%)
Protective Stop Exit Trades:                                 1 (9.09%)
Time Stop Exit Trades:                                       0 (0.00%)
Profit Stop Exit Trades:                                     0 (0.00%)

Largest Winning Trade/(Date):                   $7,469.03 (28/12/2001)
Largest Losing Trade/(Date):                   -$1,195.62 (17/08/2001)
Average Winning Trade:                                       $1,804.02
Average Losing Trade:                                         -$687.86
Average Win/Average Loss:                                       2.6227

Trade Duration Statistics
(All Trades)
Maximum Trade Duration:                                     337 (days)
Minimum Trade Duration:                                      42 (days)
Average Trade Duration:                                     162 (days)
(Winning Trades)
Maximum Trade Duration:                                     337 (days)
Minimum Trade Duration:                                      42 (days)
Average Trade Duration:                                     184 (days)
(Losing Trades)
Maximum Trade Duration:                                     196 (days)
Minimum Trade Duration:                                      42 (days)
Average Trade Duration:                                     103 (days)

Consecutive Trade Statistics
Maximum consecutive winning trades:                                  4
Maximum consecutive losing trades:                                   2
Average consecutive winning trades:                               2.67
Average consecutive losing trades:                                1.50

Relative Drawdown
Maximum Dollar Drawdown/(Date):                 $2,028.77 (17/08/2001)
Maximum Percentage Drawdown/(Date):               6.3170% (17/08/2001)

Absolute (Peak-to-Valley) Dollar Drawdown
Maximum Dollar Drawdown:                           $2,028.77 (6.3170%)
Capital Peak/(Date):                           $32,117.07 (16/03/2001)
Capital Valley/(Date):                         $30,088.31 (17/08/2001)

Absolute (Peak-to-Valley) Percent Drawdown
Maximum Percentage Drawdown:                       6.3170% ($2,028.77)
Capital Peak/(Date):                           $32,117.07 (16/03/2001)
Capital Valley/(Date):                         $30,088.31 (17/08/2001)


----------



## kaleon (16 September 2007)

Sorry Nizar,

I noticed you had transaction costs but no slippage. I think slippage would eat away a large portion of potential profit. Did you used fixed dollar units of fixed % risk in your simulations?


----------



## debono (16 September 2007)

Great thread to read guys.Well done for sharing so much of your research.
I have a question if you don't mind.
When you use a back testing method to review your systems, do you then forward test for a period of time and compare actual results vs historical theorectical?
The reason I ask this is some years back when I was playing with metastock I would find systems that appear to work on back test well however when the 'go live' test was undertaken the profitability dropped dramatically and mostly became unprofitable.
I think this is because back testing assumes a get out at price target. This is not always the case (especially with derivatives) because the volume does not allow the trade to be completely exited.
You guys have obviusly spent a great deal more time than I did so wondering if this figures at all in your equations?

thanks and congrats again on a great thread

debono


----------



## nizar (16 September 2007)

kaleon said:


> Sorry Nizar,
> 
> I noticed you had transaction costs but no slippage. I think slippage would eat away a large portion of potential profit. Did you used fixed dollar units of fixed % risk in your simulations?




Yes, you are correct.
Looks like your system is much better than mine!
Well done.

Seems like I have a long way to go, but thats to be expected as Iv just started on this journey.

Its gonna be alot of work to get a 2001 ASX300 list. Thanks for the list.


----------



## nizar (16 September 2007)

debono said:


> Great thread to read guys.Well done for sharing so much of your research.
> I have a question if you don't mind.
> *When you use a back testing method to review your systems, do you then forward test for a period of time and compare actual results vs historical theorectical?*
> The reason I ask this is some years back when I was playing with metastock I would find systems that appear to work on back test well however when the 'go live' test was undertaken the profitability dropped dramatically and mostly became unprofitable.
> ...




Well the answer to the part in bold is YES!
We have to do this in order to test the robustness of the system and to validate the system.

There is alot of discussion about walk-forward analysis in one of the other threads i started, titled "System Robustness".


----------



## bingk6 (16 September 2007)

nizar said:


> In my point of view, mechanical systems have a far sooner expiry date when traded on the futures market than on the stockmarket.
> 
> Why? Because as a % of total market participants, i would suspect that mechanical systems traders make up a far greater proportion in the futures markets than they do in the stockmarket. As a result of this, there is much more research and development going on into designing mechanical systems in the futures market compared to in the stockmarket.





Hi Nizar,

I would have thought that it would be the other way round. Futures are based on indices (whose value are constructed from the SPs of many individual stocks). Therefore the only way to fade the futures and have any sort of impact on its value would be to impact all its constitutent stocks, which makes it far less controllable than individual stocks. 

Interesting topic all the same regarding system decay. I guess once performance does not measure up to expectations that it would be time to reoptimize and try again.


----------



## howardbandy (16 September 2007)

Hi Gorilla --

You asked about this statement in my response to a Monte Carlo question:
"Another is to first create a statistical distribution from the observed actual trades, then make repeated random selections from that distribution. "

This technique begins with a number of observations, with the intent of generating several other sets of observations that have the same statistical characteristics, but are not necessarily the same values, and not necessarily in the same time order.

The following example uses 15 because I can list 15 without filling up too much space.  I would probably have many more than 15.

Say I have run a trading system simulation and my out-of-sample results show 15 closed trades.  The percent gain for them, as they occur in time order, is:
4
-1
3
2
-1
-1
1
0
3
1
2
-3
0
-2
6

I want to simulate what the equity curve would look like if I had other sequences of trades that came from the same distribution, but not necessarily those exact same numbers and not necessarily in that order.

I begin by creating a Cumulative Distribution Function using those observations.  (If that statement did not make sense, go to either my book, Quantitative Trading Systems, Chapter 22, or a text for statistics, econometrics, or simulation.)  If desired, I smooth the curve.

To simulate a single equity curve, I make 15 random draws from this distribution (with replacement, for those who care), each draw representing a closed trade, and create the equity curve for those 15.

I do this many times -- say 100 times -- and evaluate the results.  I can ask questions such as:
What is the probability that this system has a drawdown greater than 10% at any time?
What is the probability that there will be a string of 3 or more losing trades in a row?
What is the average terminal relative wealth?
What is the standard deviation terminal relative wealth?

Terminal relative wealth, trw, is the ratio of the final account balance to the initial account balance.  If the account begins with a balance of $10,000 and ends with a balance of $25,000, then terminal relative wealth is 2.5.

If I have been compounding, that is investing 100% of the account in every trade, then there is a simple relationship between the average percentage gain per trade, p, and the number of trades, n.   

trw = (1+p) ^ n

For example, a system that makes 2 trades in a year and gains 10% on each has trw of (1.10)^2 or 1.210 -- a gain of 21.0%.
A system that makes 10 trades a year and gains 2% on each has trw of (1.02)^10 or 1.219 -- a gain of 21.9%.

Thanks,
Howard


----------



## howardbandy (16 September 2007)

debono said:


> Great thread to read guys.Well done for sharing so much of your research.
> I have a question if you don't mind.
> When you use a back testing method to review your systems, do you then forward test for a period of time and compare actual results vs historical theorectical?
> The reason I ask this is some years back when I was playing with metastock I would find systems that appear to work on back test well however when the 'go live' test was undertaken the profitability dropped dramatically and mostly became unprofitable.
> ...




Hi debono --

My short answer is "no."  

First, there are no historical theoretical return for comparison.

More importantly, the in-sample results have no value in predicting the future profitability or performance of a trading system.  Only out-of-sample results count.

There has been a lot of discussion of this in this thread and in the thread on Robustness.

Thanks,
Howard


----------



## howardbandy (16 September 2007)

bingk6 said:


> Hi Nizar,
> 
> 
> Interesting topic all the same regarding system decay. I guess once performance does not measure up to expectations that it would be time to reoptimize and try again.




Hi Bibgk6 --

Right!

Thanks,
Howard


----------



## nizar (16 September 2007)

bingk6 said:


> Hi Nizar,
> 
> I would have thought that it would be the other way round. Futures are based on indices (whose value are constructed from the SPs of many individual stocks). Therefore the only way to fade the futures and have any sort of impact on its value would be to impact all its constitutent stocks, which makes it far less controllable than individual stocks.
> 
> Interesting topic all the same regarding system decay. I guess once performance does not measure up to expectations that it would be time to reoptimize and try again.




I wasnt referring to only indices, but commodities as well.


----------



## tech/a (17 September 2007)

howardbandy said:


> Hi Bibgk6 --
> 
> Right!
> 
> ...





Howard.
Are you suggesting optimisation for all trading systems?

Or only single entities--Futures/Commodities/Indexes?


----------



## tech/a (17 September 2007)

Sir Burr said:


> Tech, a question that I have been pondering:
> 
> Why is Monte Carlo Analysis better than Optimising/Walk Forward?
> 
> ...





S/B its just another analytic tool.
Infact Montecarlo analysis as it is used by the software most have is far from ideal.true Montecarlo looks at EVERY possible variable.
There is a great deal of difference and each can and should co exist.The opinions on Optimising/Forward testing and Walk Forward testing are in some part different to mine.

My own opinion and research differ somewhat to that expressed here.

While I have attempted to express and question the thinking of others I'm afraid it falls on deaf ears answers are obtuse or questions completely ignored.
Fortunately I'm working on systems design with my son who is a Doctor of Physics and has powerful systems design and modelling software and analytical capability.I have my own tutor.
As he says "I can make any mathamatical arguement tell you what ever you want".


----------



## howardbandy (17 September 2007)

tech/a said:


> Howard.
> Are you suggesting optimisation for all trading systems?
> 
> Or only single entities--Futures/Commodities/Indexes?




Hi Tech/a --

Yes, optimize everything.  Optimize just means make a thorough search -- nothing more than that.  If I am going to choose from among two or three or twelve alternatives, why stop there?  I'll look at thousands.  You read my post using the "marbles" analogy, yes?  Post Number 34 in this thread.  I explain why in it.

Thanks,
Howard


----------



## howardbandy (17 September 2007)

tech/a said:


> S/B its just another analytic tool.
> Infact Montecarlo analysis as it is used by the software most have is far from ideal.true Montecarlo looks at EVERY possible variable.
> There is a great deal of difference and each can and should co exist.The opinions on Optimising/Forward testing and Walk Forward testing are in some part different to mine.
> 
> ...




Hi Tech/a --

If there is question that you have asked me that I have not answered, or an answer that I have given that is obtuse, please ask again.

Thanks,
Howard


----------



## tech/a (17 September 2007)

tech/a said:


> Howard.
> 
> In your experience would you suggest
> Fixed origin evaluation
> ...





For Starters.


----------



## howardbandy (17 September 2007)

tech/a said:


> For Starters.




Hi Tech/a --

I'll start at the top.  I read that post and got confused -- the questions asked seemed to be rhetorical, leading up to your statement at the end of the quoted text.  May I suggest one question per post?  And if you specifically want my comments, please ask me by name.

About anchored or rolling periods --

In my opinion, rolling.  If anchored is used, the in-sample period for the last walk forward step during development, and then for all the "bring up to date" steps during actual use of the system, is "all the data."  The system is being given more and more data representing more and more market conditions and is less and less likely to be tuned to the current conditions.  One of the benefits of using the walk forward process is being able to tune the system to the current conditions.

Thanks,
Howard


----------



## howardbandy (17 September 2007)

Hi Tech/a --

I do not understand this one:

How do you evelute the period ahead that you should use in the evaluation?

Thanks,
Howard


----------



## howardbandy (17 September 2007)

Hi Tech/a --

You wrote: "If optimising at what point do you re calibrate?"

Assuming that you mean re-optimize --

I'll define re-optimize to mean the following:
1.  Move the in-sample period forward in time so that it includes the most recent data, or nearly so.
2.  Run an optimization to determine the values for the arguments that give the highest objective function score for that in-sample period.
3.  Begin using the new values of the arguments to test the out-of-sample data or to begin trading using the new values.

There are a couple of possibilities:

1.  Always re-optimize after a set number of out-of-sample bars.  This is what would have been done during the walk forward phase of development of the model, so it makes sense to continue doing that.

2.  Re-optimize whenever the out-of-sample results, or trading results, indicate that the system is performing at a lower level than anticipated.  To answer this question requires some statistical analysis.  The implications of re-optimizing after a variable number of out-of-sample bars is that you might continue to trade a system beyond the normal walk-forward period.  If the system is performing well, feel free to do that.

There is no single answer.  It depends on the characteristics of the market being modeled and the model.     

Thanks,
Howard


----------



## howardbandy (17 September 2007)

Hi Tech/a --

You wrote: "How many out of series test periods are in your veiw enough to give confidence and at what point do you gain that confidence?"

There is no one-size-fits-all answer to this question.

The goal of the system development process -- whether you are using methods similar to those I describe or some other method -- is to develop confidence that the system will trade profitable in the future.  The emphasis is on "develop confidence."  The characteristics of a trading system that give you confidence may not be the same as the characteristics that give me confidence.  

I believe that the "goodness" of a trading system is defined by each individual (or trading company) well before system testing begins -- at the time when the objective function was being defined.

If the combined out-of-sample results of a walk forward process produce results that are satisfactory in terms of the definition of the objective function, then that should give the level of confidence needed.    

Thanks,
Howard


----------



## howardbandy (17 September 2007)

Hi Tech/a --

You wrote: "How do you do your out of sample testing,what software do you use?"

I use AmiBroker for most of my development work.  In chapter 20 of my book, I describe using AmiBroker to perform a walk forward analysis.  

At present, it is necessary to set the dates for the in-sample and out-of-sample periods "manually," but I would not be surprised to see AmiBroker extended to have automatic walk forward capabilities in the near future.

Thanks,
Howard


----------



## howardbandy (17 September 2007)

Hi Tech/a --

You wrote: "Or are you simply walking forward through a data set?"

I do not understand the question.

Thanks,
Howard


----------



## howardbandy (17 September 2007)

Hi Tech/a --

You wrote: "How do you extrapolate single special events that fall within the out of sample test period/s."

I do not understand the question.

Thanks,
Howard


----------



## nizar (17 September 2007)

Hi Howard,

Amibroker has been around for about 5 years i think.
You mention having written a paper in 1969 and so I assume you mustve already been a top trader for a number of years before amibroker was released?

What did you use before amibroker was around?


----------



## howardbandy (17 September 2007)

Hi Tech/a --

You wrote: "I would argue that out of sample testing of models offers no guarentee that the in sample test results will perform any better or worse than the results returned within the parameters of those results obtained through multiple or Montecarlo testing going forward in realtime."

I may not be interpreting your statement correctly, but I'll make two comments that might apply.

1.  The walk forward process and Monte Carlo analysis are complimentary, not exclusive.  It is very common for Monte Carlo techniques to be applied during a system development that is using the walk forward process.

2.  As I wrote in a post just a few minutes ago, there are no guarantees.  The best that we can hope for is to develop confidence that our system will be profitable.  

The points I am making and the techniques that I am describing throughout my book, my speeches, my workshops, and my posts to this forum are intended to help system developers and traders produce systems that are likely to trade profitably. 

I hear comments about the difficulty of doing what I suggest, and I hear suggestions to the effect that reliable systems can be developed without out-of-sample testing.  But there will be an out-of-sample test.  My point is that using the techniques I describe, I can make a series of confidence building out-of-sample tests without risking real money before the marketplace makes its out-of-sample test.

Thanks for listening,
Howard


----------



## stevo (17 September 2007)

howardbandy said:


> The system is being given more and more data representing more and more market conditions and is less and less likely to be tuned to the current conditions.  One of the benefits of using the walk forward process is being able to tune the system to the current conditions.




Wouldn't more data and more market conditions help ensure that the system isn't optimised to only the recent market conditions and potentially build a more robust system? If the test data universe is in a strong uptrend then the system may not handle a correction all that well if it is only tuned to recent market data.

There is a big difference optimising on a single stock with a year's worth of data versus optimising on 500 stocks over a 3 year period - I know what testing approach I prefer - more data and more market conditions.

Is there some sort of balance required in terms of testing - optimised to current conditions, but not too optimised. It is probably not possible to design a system that will always work, but it is possible to turn a system off / on if market conditions change.

Having said all that I do like rolling walk forward testing. Are there any rules in terms of what sort of walk forward period to use - if a system has a 3 month average trade length I would think that the rolling test period would have to be longer than 3 months and probably closer to at least 1 year.


----------



## howardbandy (17 September 2007)

stevo said:


> Wouldn't more data and more market conditions help ensure that the system isn't optimised to only the recent market conditions and potentially build a more robust system?




Hi Stevo --

In my opinion, No.

The market being modeled is dynamic -- the model of that market is static.  As long as the market remains in or near the state it was in in when the model was built, the model will produce profitable buy and sell signals.  When the market changes, the signals will degrade.

It is very difficult (impossible?) to build a model that works over all market conditions.  The purpose of having shorter rather than longer in-sample periods is so the model can be tuned more precisely to the state of the market.  The purpose of the walk forward process is to keep the model in tune with the then-current state of the market.

Thanks,
Howard


----------



## howardbandy (17 September 2007)

Greetings all --

I have made quite a few posts to the thread on "System Robustness" as well as to this one.  I am starting to repeat myself.  

No offense intended, but please forgive me if I do not make new postings where one of my earlier postings has already expressed my view.

Thanks,
Howard


----------



## It's Snake Pliskin (17 September 2007)

howardbandy said:


> Greetings all --
> 
> I have made quite a few posts to the thread on "System Robustness" as well as to this one.  I am starting to repeat myself.
> 
> ...




Howard,

I am impressed with your patience so far. I thought the same.
Thanks for your thoughts and opinions. Maybe some should read your book then comment later.
Regards
Snake


----------



## howardbandy (17 September 2007)

nizar said:


> Hi Howard,
> 
> Amibroker has been around for about 5 years i think.
> You mention having written a paper in 1969
> ...




Hi Nizar --

Yes, I have been studying and applying modeling and simulation techniques to financial time series for a long time.

In my opinion, there is nothing currently available to the masses of us (who are not employed by companies that have proprietary development platforms) developing trading systems that is better than AmiBroker.  Everything I have used in the past, including ones that I wrote myself, is obsolete. 

Thanks,
Howard


----------



## nizar (17 September 2007)

howardbandy said:


> The points I am making and the techniques that I am describing throughout my book, my speeches, my workshops, and my posts to this forum are intended to help system developers and traders produce systems that are likely to trade profitably.
> 
> I hear comments about the difficulty of doing what I suggest, and I hear suggestions to the effect that reliable systems can be developed without out-of-sample testing.  But there will be an out-of-sample test.  My point is that using the techniques I describe, I can make a series of confidence building out-of-sample tests without risking real money before the marketplace makes its out-of-sample test.
> 
> ...




Howard.

Thanks for sharing your knowledge and experience with us.
You have taught me a great deal and for that Im very grateful.
Everything that you have said has made sense to me (often after further questioning by myself) and has been taken aboard for my own testing.


----------



## tech/a (18 September 2007)

howardbandy said:


> Hi Tech/a --
> 
> I do not understand this one:
> 
> ...




That which you chose for out of sample testing.
Ill do as Snake suggests though.

Howard.
What features in your opinion make Amibroker so good.
Have you heard of Tradesim---which can be added to Amibroker.
http://compuvision.com.au/


----------



## howardbandy (18 September 2007)

tech/a said:


> That which you chose for out of sample testing.




Hi Tech/a -- 

The out-of-sample period immediately follows the in-sample period.  It is as long as you wish -- as long as the results continue to be acceptable.  (Theoretically, there can be time gap between the in-sample period and the out-of-sample period, but I have never seen that benefit the system.  The out-of-sample period can never be earlier in time -- longer ago -- than the in-sample period.)

If the system works (is profitable, scores high on the objective function, etc) because it identifies some inefficiency in the market, as long as the market remains stable relative to that component, the system will continue to work.  When the system degrades, it is because the underlying market has changed, and you will need to re-optimize.

If the walk forward process has been followed, then you already know what to expect when you re-optimize.  That is, you have several observations of the out-of-sample performance following a re-optimization.  If those results are consistently good, that gives you confidence in the walk forward process itself.  You will know whether or not you can re-optimize whenever you want to -- the answer is probably that you can -- in fact many very good systems re-optimize after every bar.

Gaining confidence in the walk forward process is a very important part of the designing, testing, and validating trading systems.

Thanks for listening,
Howard


----------



## tech/a (18 September 2007)

*Howard.*

Thanks for your time and effort.

I have a number of questions but will start with the *Practical Application of Optimisation*.

While I and I'm sure most here understand the idea behind optimisation of variables and possibly conditions. I would like your comments on the following.

We can choose anytime to re optimise.
That can be daily,weekly,at the beginning of a trade,at the end of a trade,really anytime.
*Question 1* When is the "Optimum time to re calibrate?"

Which brings me to *practical application*.

Ive tested a system every which way Forward/backward/montecarlo,in and out of sample and I have an optimised variable set on 3 conditions of 10,25,8

I'm ready and confident to trade.

*Scenario 1*
A week later all my conditions and variables are met on a trade.
So I just run the optimisation again.---now its 15,20 and 6.Nothing like a week ago.

*Scenario 2*
A week later All the conditions are met so I enter the trade.My exit criteria are 10,20 and 4.
5 days later all are hit,but when I optimise the variables again the exit criteria is 8,25,8 and that happened 2 days ago.Now what?
OR
The optimum variables for exit are now 12,25,6 which hasnt now been hit although the original ones have. Now what?

How can you then trust that the optimised variables which you have set against the objectives of your trading are infact optimum.
Infact Id argue that after a long enough period the entries and exits would appear random but between maximum and minimum values.

Why would the trading results be any better than a fixed selection of variables (perhaps your first optimisation results). the results provided the logic behind the system is one of positive expectancy are going to fall between the extremes of the values chosen for optimisation the maximum and the minimum. They will never fall at the maximum or minimum as like the market the dynamics will continually move the possible result.

Therfore I cant see how/why optimisation will improve result over the longterm.

Infact if you were to re calibrate the variables 12 mths later they wouldnt (normally) resemble the initial variables selected at the time of completion of systems design. How can you have confidence when infact the best time for entry or exit may be found to have passed or be changing to such an extent that it ends up impacting on expectancy.

For me its *Practical application*rather than the *thoery* of how and why it should be better or indeed a necessity.

Can you show a Practical application of your arguement?
Perhaps a chart showing in dynamic realtime the difference between optimised and static variables of a method and the improvement.
I dont know---what do you suggest.


----------



## nizar (18 September 2007)

Tech.
Some really good points in there.

Im looking forward to Howard's response.

I do understand the practical application of optimisation, but why we need to re-optimise after having traded the system is beyond me.


----------



## howardbandy (18 September 2007)

Greetings all --

About the values of the variable moving around a lot.

What is your objective function?  Some do not work well.  For example, optimizing for total profit often does not trade well out-of-sample.  

If the optimal values are changing wildly over short time periods, it sounds like either the trading model or the market is unstable.  If the market is changed to something that is fairly smooth -- say XLE, one the S&P sector ETFs -- does the model seem to behave better?  If so, then it is the market that is unstable.  If not, then it is the model that is unstable.  We all have difficulty showing confidence in an unstable system.  (System == model plus market.)

If you want to experiment further:

Pick a set of variables that are optimal for an in-sample time period.  Keep the ticker and time period fixed.  

Make several test runs with each run having slightly different values for the variables.

For example, a moving average crossover system has two parameters -- the moving average lengths.  Say the optimal values for the arguments are 20 and 7.  Try nine runs -- 18 and 5, 18 and 7, 18 and 9, 20 and 5, 20 and 7, 20 and 9, 22 and 5, 22 and 7, 22 and 9. 

If the results are very similar, then the system is robust relative to perturbation of the arguments.  If the results vary widely, then the optimizer found an undesirable "peak" surrounded by steep cliffs, rather than a desirable fairly flat "plateau."

Thanks,
Howard


----------



## howardbandy (18 September 2007)

Hi Nizar --

You wrote: "I do understand the practical application of optimisation, but why we need to re-optimise after having traded the system is beyond me."

You do not need to re-optimize as long as the system is performing to your satisfaction when actually traded.

The key points are:

The walk forward process is important in and of itself.  By making several walk forward steps, you will get some experience with how the system behaves during and after optimization.

The walk forward process establishes a length of time that is known to be a practical period of time after which a re-optimization can be done.  (Re-optimization can almost always be done more frequently, if desired.)

Any system that is being traded in real-time trades and is satisfactory can be left alone.  But -- eventually it will degrade.  Every system eventually fails.  Once a system has failed it seldom (never?) recovers.  It must be either abandoned or re-optimized.    

--------------

I think I need to make one major point again.  

If I have used all of the data I have available to develop a trading system, and I make any choices at all, I am performing an optimization.  (An optimization is just another name for an organized search, usually over a larger parameter space than is or even can be done by setting the values one at a time.)

If I optimize over all but, say, the last year of data, then look at the results from the last year of data and subsequently change the model, I no longer have any out-of-sample data left -- all the data is now in-sample.

Following one or more in-sample tests, there will be an out-of-sample test.  I have a choice -- to follow good system design, test, and validation procedures and make that (or those) out out-of-sample test myself, or to let the market make the out-of-sample test after I have placed actual trades.

I sure am getting tired of saying that over and over.  Somebody, please, say they understand.

Thanks for listening,
Howard


----------



## howardbandy (18 September 2007)

Hi Tech/a --

You wrote: "Therfore I cant see how/why optimisation will improve result over the longterm."

My point is -- whenever any of us makes any choice from among alternatives, we are already optimizing.  So we might as well get the most benefit from it.

Thanks,
Howard


----------



## nizar (18 September 2007)

howardbandy said:


> Hi Nizar --
> 
> You wrote: "I do understand the practical application of optimisation, but why we need to re-optimise after having traded the system is beyond me."
> 
> ...




Yes I understand!

The difference and the importance of splitting up your data into in-sample and out-of-sample for walk forward analysis has been explained very succintly here and in your other posts.

Thanks for the explanation of re-optimisation.

Thanks for your time and your continuing contributions.
Much appreciated.


----------



## Sir Burr (18 September 2007)

howardbandy said:


> So we might as well get the most benefit from it.




Thanks for your postings here!

About your book, you mentioned too many variables may result in curve fitting. How many rules/variables would you consider too many?

I'd take a wild guess at 4+ but have no idea as how to "quantify" it. :

Would using variables that have no relation help in preventing curve fitting? Rather than 2 entry and 2 exit variables, for example 1 variable each for position sizing, turnover, entry and exit.

Cheers SB


----------



## theasxgorilla (18 September 2007)

howardbandy said:


> I sure am getting tired of saying that over and over.  Somebody, please, say they understand.




I get it!  Forward testing occurs at the hard right edge the moment you start trading with real money unless you reserved a portion of data to be out-of-sample that you could simulate forward testing with, after testing and optimising on the in-sample data.

I'm in the process of building a test plan and will be reserving said out of sample data.

Howard, thanks for being patient enough to get this important message into our sometimes thick heads! 

ASX.G


----------



## bingk6 (18 September 2007)

Hi Howard,

I understand and I suspect that most of us here at ASF also understand the importance of the walk forward methodology using out of sample data in verifying the robustness of the system going forward. The concept is a very sound one and we thank you for your patience.

The one area that I personally like to see being discussed more is this concept of system decay, and how over time, the system will lose its edge and will ultimately fail. Logic would indicate (to me at least) that as more and more people trade a particular type of system, say a 30 day breakout system, that the system would have a greater chance of succeeding as an ever increasing number of traders initiate the trade upon confirmation of the breakout, to a point where it almost becomes a self-fulfilling prophecy. How then does such a system lose its edge over time ? 

Does a system tend to fail as more traders use it ?? If the answer is yes then can one conclude that the system would more effective if it is used by fewer traders ?

When a system starts to lose its edge (presumably from over-exposure from too many traders using it) the number of traders using it will also tend to decrease over time as they seek out other more profitable strategies. When the number of traders have dropped off sufficiently, will the edge that the system had initially return ?? If the answer is NO, then what residual external factor that had caused the system to initially fail has remained in play to continually impact on the system even though the number of traders using it has dropped off dramatically ?       

I can’t get my head around this system decay concept and hope that you may be able to shed some light on the subject.


----------



## tech/a (18 September 2007)

howardbandy said:


> Hi Tech/a --
> 
> You wrote: "Therfore I cant see how/why optimisation will improve result over the longterm."
> 
> ...




Yes



> If the results are very similar, then the system is robust relative to perturbation of the arguments. If the results vary widely, then the optimizer found an undesirable "peak" surrounded by steep cliffs, rather than a desirable fairly flat "plateau."




An important point with all objectives.
Not trading the best optimised variables but seeing if the system is flatish regardless of best optimisation.
The distribution is within a minor deviation.

Yes I can see this rather than attempting to trade the best optimised parameters.

*On Walk Forward testing*.
With longer term methods which can hold stock for instance 12 mths or more how can we gain a credible walk forward test?

If the Reasons a system will be profitable (Positive expectancy by way of more winners than losers OR greater nett profit gained from all winners than nett losses from losing trades) then how often *Howard *do you see a system breakdown when testing out of sample or walking forward.

*Howard* I also asked the question in regard to your remarks that Trend trading is no longer profitable (or words to that effect).
My question is that without a trend either long or short how is it possible to create a profit. At some point price has to move in a trend in some timeframe or another long enough to register a profit. Other than exotics (Options)
I'm interested in how you can make a profit at all without any trend at all.


----------



## howardbandy (18 September 2007)

Sir Burr said:


> Thanks for your postings here!
> 
> About your book, you mentioned too many variables may result in curve fitting. How many rules/variables would you consider too many?
> 
> ...




Hi Sir Burr --

Let me start with a few general observations.

One -- trading systems are profitable because they recognize some inefficiency in the market they are trading.

Two -- the act of making a profitable trade removes some of the inefficiency and makes further profitable trading more difficult.

Three -- designing trading systems is very much like designing control systems for industrial processes.  Financial data is more complex than most industrial data.  As the systems being managed increase in complexity, the programs that manage those systems must also be more complex.

Four -- simple systems are too easily replicated by many systems developers and potential traders.  

Five -- all systems eventually fail.  A system that has failed will seldom (never?) return to profitability.

Now, on to my point --

One of the balancing acts we try to manage when designing trading systems is putting in enough logic and variables to identify the complex features that will provide the profitable trades without adding so many variables that we allow the system to become curve-fit to the data.  The solution -- you knew this was coming -- is the walk forward out-of-sample test.  If the system has been curve-fit, out-of-sample results will be poor.

So, the answer is to increase the complexity of the system in whatever creative ways you can in order to recognize inefficiencies and give profitable buy and sell signals.  But do not make the system so complex that it becomes curve-fit to the data.

Every rule contained in the trading system adds complexity, removes residual degrees of freedom from the process, and increases the likelihood of curve-fitting.  How many rules and how many variables?  There is no simple answer, but you can test your systems yourself through the validation phase of system development.

Thanks,
Howard


----------



## howardbandy (18 September 2007)

bingk6 said:


> Hi Howard,
> 
> I understand and I suspect that most of us here at ASF also understand the importance of the walk forward methodology using out of sample data in verifying the robustness of the system going forward. The concept is a very sound one and we thank you for your patience.
> 
> ...




Hi Bing --

The post I just made to points Sir Burr raised address some of your question.

To illustrate why having more traders trade a system makes it more difficult for any one of them to make a profit, think about what the order book looks like.  Say a stock is trading at $100 and headed for $105.  The best bid may be for 200 shares at 99.98 and the best offer for 100 shares at 100.02.  There are other offers of 100 shares each at 100.05, 100.10, 100.25, and 100.50.  There are more bids, but they are lower and will not play here.

Say there is one buy signal from a program that is generally correct.  100 shares are bought at the market -- 100.02.   

Instead, say five buys signals arrive.  Those trades take place at 100.02, 100.05, 100.10, 100.25, and 100.50.  

Eventually, the system issues a sell signal.  The same thing happens, but now with prices on the bid side.  One order gets filled at the best bid, the other get worse fills. 

Wait, I hear you say, I use limit orders to avoid that slippage -- I'll buy at 100.06 limit.  If your order came in third, your limit order of 100.05 never got filled and you get no profit at all.

My point is this -- the more systems that want to buy this stock, the worse their average performance will be.

Thanks,
Howard


----------



## howardbandy (18 September 2007)

Hi Tech/a --

You are asking about trends and getting enough out-of-sample results to validate a long-term trading system.

First about trends.  I agree that every trade needs a trend to be profitable.  Buy low and sell high; buy high and sell higher; short high and cover low; short low and cover lower.  Those are the only four profitable cases if we are dealing with prices.

The time frame makes a difference.  What looks like a trading range market on a weekly chart could have nice trends on an hourly chart.

About validating long-term trading.  If the system has long holding periods, or if it trades infrequently, it will be difficult validating it using a single price series.  But the out-of-sample data does not have to come from the same price series.  A system could be developed using one stock, then validated using other stocks.  

But -- beware -- avoid "optimizing the symbol space."  If I develop a system using ABC, and wish to use other stocks to validate the system, then I can test using 100 stocks.  If the system is acceptable (high objective function score) for most of them, that will help validate the system.  If I find it is profitable for only 5 stocks and I pick those 5 to trade, I am at great risk of losing money.  Almost any trading system will show a profit for some stock for some time period when tested on a large number of stocks over a long time period -- just by being lucky.

If the long-term or infrequent system does not validate well, even using this method, do not trade it.

The validation process is one of building confidence.  If we want to build confidence, we need to see lots of examples of success.  Even then, there are no guarantees.  

Thanks,
Howard


----------



## nizar (18 September 2007)

bingk6 said:


> The one area that I personally like to see being discussed more is this concept of system decay, and how over time, the system will lose its edge and will ultimately fail. Logic would indicate (to me at least) that as more and more people trade a particular type of system, say a 30 day breakout system, that the *system would have a greater chance of succeeding as an ever increasing number of traders initiate the trade upon confirmation of the breakout, to a point where it almost becomes a self-fulfilling prophecy. How then does such a system lose its edge over time ? *
> Does a system tend to fail as more traders use it ?? If the answer is yes then can one conclude that the system would more effective if it is used by fewer traders ?




Hi Bingk6.

I notice the question wasnt directed at me, but ill give my view anyway.

If an ever increasing number of traders want to BUY the open, then you will have to pay a higher price.

If an ever increasing number of traders want to SELL the open, when stops/exits are triggered, then you will have to sell at a lower price.


----------



## nizar (19 September 2007)

Anyway, back to the main purpose of this thread.

Iv decided to go with a weekly system instead of EOD.
Easier to run and monitor and gives more flexibility for trade execution.

This new system has no initial stop.
Simply because i could find an initial stop that would improve its performance.

The exit is an ATR type. Initially i went with 3*ATR, but found that 3.5*ATR works better.

This time i have factored in slippage.
TradeSim (for those not familiar) allows for "market orders" which means a random algorithm is used to determine an entry/exit price within the lows and highs of that bar. Im trading weekly, so a bar is 1 week.

This basically means i can enter at any time during the week during testing and also in real-time when the system is traded live.

Starting capital has increased to $45,000 because Iv just got a new client, a family member who wants to put $15k into the system.

Money management is set to $7,500 per trade.
Fixed risk based money management could not be used due to the absence of an initial stop.
I could make the trailing stop as an initial stop but have not yet found out how to do this.

10% of available trading capital was another alternative that was tried, and though I can understand the concept with having bigger positions as your capital base grows and less when it shrinks, its use in this system produces returns and drawdowns were more variable.

And another thing i didnt like was the equity curve, as the capital base grew towards the end of the simulation, most of its profits came from 1-2 big trades.

We didnt see this with the $7,500 model.

We will start with the 10-year simulation from 1992-2002 over the Entire ASX market (Liquidity filter $1mil per week was applied).

As usual, im looking for the same thing as initially.
Feedback! 

Thanks all.


----------



## nizar (19 September 2007)

Monte Carlo Report	

Trade Database Filename	
C:\TradeSimData\WeeklyMaster3.5ATRwithROC.trb	

Simulation Summary	
Simulation Date:	18/09/2007
Simulation Time:	12:50:42 AM
Simulation Duration:	1928.19 seconds

Trade Parameters	
Initial Capital:	$45,000.00
Portfolio Limit:	100.00%
Maximum number of open positions:	40
Position Size Model:	Equal Dollar Units
Trade Size ($ value):	$7,500.00
Pyramid profits:	Yes
Transaction cost (Trade Entry):	$55.00
Transaction cost (Trade Exit):	$55.00
Margin Requirement:	100.00%
Magnify Position Size(& Risk) according to Margin Req:	No
Margin Requirement Daily Interest Rate (Long Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Long Trades):	0.0000%
Margin Requirement Daily Interest Rate (Short Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Short Trades):	0.0000%

Trade Preferences	
Trading Instrument:	Stocks
Break Even Trades:	Process separately
Trade Position Type:	Process all trades
Entry Order Type:	Market Order
Exit Order Type:	Market Order
Minimum Trade Size:	$500.00
Accept Partial Trades:	No
Volume Filter:	Reject Trades if Position Size is greater than
	10.00% of the maximum traded volume
Pyramid Trades:	Yes
Favour Trade Pyramid:	Yes
Start Pyramid at any level up to level:	N/A
Maximum Pyramid Level Limited to:	N/A
Maximum Pyramid Count Limited to:	N/A

Simulation Stats	
Number of trade simulations:	20000
Trades processed per simulation:	3341
Maximum Number of Trades Executed:	340
Average Number of Trades Executed:	319
Minimum Number of Trades Executed:	300
Standard Deviation:	5.17

Profit Stats	
Maximum Profit:	$1,191,944.64 (2648.77%)
Average Profit:	$1,061,127.72 (2358.06%)
Minimum Profit:	$863,892.51 (1919.76%)
Standard Deviation:	$31,838.02 (70.75%)
Probability of Profit:	100.00%
Probability of Loss:	0.00%

Percent Winning Trade Stats	
Maximum percentage of winning trades:	55.76%
Average percentage of winning trades:	51.73%
Minimum percentage of winning trades:	45.85%
Standard Deviation:	1.26%

Percent Losing Trade Stats	
Maximum percentage of losing trades:	54.15%
Average percentage of losing Trades:	48.27%
Minimum percentage of losing trades:	44.24%
Standard Deviation:	1.26%

Average Relative Dollar Drawdown Stats	
Maximum of the Average Relative Dollar Drawdown:	$2,735.05
Average of the Average Relative Dollar Drawdown:	$1,915.68
Minimum of the Average Relative Dollar Drawdown:	$1,410.19
Standard Deviation:	$149.11

Average Relative Percent Drawdown Stats	
Maximum of the Average Relative Percent Drawdown:	0.9390%
Average of the Average Relative Percent Drawdown:	0.6321%
Minimum of the Average Relative Percent Drawdown:	0.4531%
Standard Deviation:	0.0570%

Maximum Peak-to-Valley Dollar Drawdown Stats	
Maximum Absolute Dollar Drawdown:	$36,667.97
Average Absolute Dollar Drawdown:	$25,278.71
Minimum Absolute Dollar Drawdown:	$19,736.76
Standard Deviation:	$1,893.04

Maximum Peak-to-Valley Percent Drawdown Stats	
Maximum Absolute Percent Drawdown:	15.7769%
Average Absolute Percent Drawdown:	12.8032%
Minimum Absolute Percent Drawdown:	9.5921%
Standard Deviation:	0.9085%


----------



## nizar (19 September 2007)

Detailed Report			

Trade Database Filename			
C:\TradeSimData\WeeklyMaster3.5ATRwithROCbug.trb			

Simulation Summary			
Simulation Date:	18/09/2007		
Simulation Time:	11:27:28 PM		
Simulation Duration:	1.00 seconds		

Trade Summary			
Earliest Entry Date in the Trade Database:	3/01/1992		
Latest Entry Date in the Trade Database:	14/12/2001		
Earliest Exit Date in the Trade Database:	3/04/1992		
Latest Exit Date in the Trade Database:	28/12/2001		

Start Trade Entry Date:	3/01/1992		
Stop Trade Entry Date:	14/12/2001		
First Entry Date:	3/01/1992		
Last Entry Date:	30/11/2001		
First Exit Date:	3/04/1992		
Last Exit Date:	28/12/2001		

Total Trading duration:	3647 days		

Profit Summary			
Profit Status:	PROFITABLE		
Starting Capital:	$45,000.00		
Finishing Capital:	$1,138,647.70		
Maximum Equity/(Date):	$1,093,647.70 (28/12/2001)		
Minimum Equity/(Date):	-$2,528.90 (24/09/1992)		
Gross Trade Profit:	$1,258,490.90 (2796.65%)		
Gross Trade Loss:	-$164,843.20 (-366.32%)		
Total Net Profit:	$1,093,647.70 (2430.33%)		
Average Profit per Trade:	$3,439.14		
Profit Factor:	7.6345		
Profit Index:	86.90%		
Total Transaction Cost:	$34,980.00		
Total Slippage:	-$9,099.26		
Total Trade Interest:	$0.00		
Daily Compound Interest Rate:	0.0886%		
Annualized Compound Interest Rate:	38.1762%		

Trade Statistics			
Trades Processed:	3337		
Trades Taken:	318		
Partial Trades Taken:	0		
Trades Rejected:	1537		
Winning Trades:	163 (51.26%)		
Losing Trades:	155 (48.74%)		
Breakeven Trades:	0 (0.00%)		

Largest Winning Trade/(Date):	$55,456.23 (14/04/2000)		
Largest Losing Trade/(Date):	-$5,279.72 (18/07/1997)		
Average Winning Trade:	$7,720.80		
Average Losing Trade:	-$1,063.50		
Average Win/Average Loss:	7.2598		

Trade Breakdown	Long and Short Trades	Long Trades	Short Trades
Normal Exit:	313 (98.43%)	313 (98.43%)	0 (0.00%)
Open Trade:	5 (1.57%)	5 (1.57%)	0 (0.00%)

Total Trades:	318 (100.00%)	318 (100.00%)	0 (0.00%)

Trade Duration Statistics	Winning and Losing Trades	Winning Trades	Losing Trades
Maximum Trade Duration:	1400 (days)	1400 (days)	434 (days)
Minimum Trade Duration:	0 (days)	0 (days)	20 (days)
Average Trade Duration:	258.38 (days)	393.28 (days)	116.52 (days)

Consecutive Trade Statistics			
Maximum consecutive winning trades:	12		
Maximum consecutive losing trades:	15		
Average consecutive winning trades:	3.54		
Average consecutive losing trades:	3.37		

Relative Drawdown			
Maximum Dollar Drawdown/(Date):	$16,382.99 (6/10/2000)		
Maximum Percentage Drawdown/(Date):	6.1090% (24/09/1992)		

Absolute (Peak-to-Valley) Dollar Drawdown			
Maximum Dollar Drawdown:	$26,839.17 (3.3520%)		
Capital Peak/(Date):	$800,604.67 (28/04/2000)		
Capital Valley/(Date):	$773,765.50 (2/03/2001)		

Absolute (Peak-to-Valley) Percent Drawdown			
Maximum Percentage Drawdown:	11.3500% ($22,212.40)		
Capital Peak/(Date):	$195,684.10 (11/03/1994)		
Capital Valley/(Date):	$173,471.70 (20/10/1995)


----------



## nizar (19 September 2007)

Attached are some charts.

Just some notes about the charts created by TradeSim.
The yearly returns charts are on CLOSED equity.
So money in the bank.

What you dont want to see with these is most of the profit coming in 2 or 3 years out of the 10.
Yet thats exactly what it looks like!!

But actually its not the case. I'll explain why.

Going by the chart, id say the system actually made money in 1992 (open profit), and again in 1993 (open profit), as winning trade length is over a year, the big winners would not have been closed out until the exit was hit.

Come 1994, which was a bad year for the market, and what happens, all the exits get hit, and all the open profits from the previous years get locked in and you have a massive year.

Anyway, im in the middle of running single year simulations, and instructing tradeSim to close out all the open trades at the end of the calendar year, so that should give a better picture of the returns.


----------



## nizar (19 September 2007)

Okay attached is an Excel Spreadsheet with my own version of the Yearly Returns of the system.

Note the returns are straight returns, non-compounded.

I basically ran a M/S exploration for every year and then put it through 20,000 Monte Carlo Simulations. Average profit and max. DD values were recorded, as was probability of Profit and average number of trades executed.

For the purposes of comparison, I have included the market index returns.

Earlier on in the test, 1992-1995, $1mil weekly turnover filter that I use was pretty restrictive. Monte Carlo didnt give much variation.

Its not really fair to test a long term system over a 1-year period as most of the big winners run for much longer, but I feel this chart gives a more accurate representation of Annual Returns than the default TradeSim one, as this one reports TOTAL profit in the portfolio at the end of December as all positions are closed.


----------



## theasxgorilla (19 September 2007)

I can't help but wonder if with such a long term system taking so few trades and pyramiding into the longer term winners if such a system wouldn't be best suited to ETFs traded with margin.

If you are back testing on ASX data and you pyramid into winners then in all likelihood you are going to end up with the lion share of your capital in stocks that ended up in the ASX20...or am I missing something?

ASX.G


----------



## Shane Baker (19 September 2007)

Hi ASXG/ Nizar,






> I can't help but wonder if with such a long term system taking so few trades and pyramiding into the longer term winners if such a system wouldn't be best suited to ETFs traded with margin.
> 
> If you are back testing on ASX data and you pyramid into winners then in all likelihood you are going to end up with the lion share of your capital in stocks that ended up in the ASX20...or am I missing something?




I thought I might outline the basic design principles that I might use in such  a system. To my mind is that you are looking for outsize winners. This means that we would need to do research to see where the smooth long running trends may come from. Also as we may wish to use margin we might select a group of share that are marginable as you suggest eg asx 300. The selection of higher cap shares v speccies would also be helpful in exiting large parcels at reasonable slippage.

Once you have narrowed down your likely universe ie marginable shares over the asx 300 as an example, then you need to analyse where the big trends come from. Are they is the higher cap shares or the lower? Are there shares that perhaps are regularly poor trenders because they may yield plays such as property trusts or Telstra? Remember we are looking for 10 baggers here.

When you look at these you may find that the lower priced shares have a greater propensity for long trends. This may give you a ranking criteria or price filter for share selection.

Ultimately you have a filtered universe of shares with a higher probability than the overall market to exhibit some major trends  eg CTX, GNS, GUD, WOR, etc etc.

Now you need to design your system to match the criteria you are trying to capture. You won't need the very beginning of a trend since you hope the trend will last at least a year if not longer. So you could design criteria that improves the winning percentage by entering trends slightly later. You will need a stop that is wide enough to capture the bulk of most long running trends such as a 50 week wma for example.

Once you have this you can look at the placement of your initial stop. I like to use a fixed $ allocation at this point and place an initial stop at the trailing stop. Plotting the R multiple returns against the initial stop percentage usually will show some form of cluster formation that may guide you in placing the level of your stop. If you prefer atr based stops then you may choose shares that teh atr values do not exceed a certain percentage 

eg 2*atr(14)/C<0.15 

would show shares that the 2 atr value was less than 15% of price.

Now we have a price point level at which we can aggressively pyramid. If our initial stop is 15% then we could pyramid every 15% as our original position moves past breakeven.

As this is a long side only system we might also want to incorporate and index filter to keep us from entering trades when the market is being unfavorable.

This is just a small part of design and structure of a system. Hopefully it helps outline some of the ideas in a more coherent fashion.

I had a long look at sector funds a while ago and my research showed that sector rotation strategies seemed to work better in these than aggressive pyramiding strategies. The reasons were related to money management issues. Of course I may revisit that and change my mind as we all do when new ideas or hybrid concepts come to fruition.

Hope this helps

Cheers

Shane


----------



## nizar (19 September 2007)

Shane Baker said:


> As this is a long side only system we might also want to incorporate and index filter to keep us from entering trades when the market is being unfavorable.




Shane.
Thanks for your thoughts.

Just with index filters, have you found them useful in the past?
Do they improve profit/drawdowns?

The feedback Iv been getting is that while they keep you out of the market, you are often late to get back in and miss the often aggressive rebound.


----------



## Shane Baker (19 September 2007)

Hi Nizar

Obviously it depends upon your timeframe for trading and the filter that you choose. Assuming weekly system, I have found that they are very adept at reducing drawdown and flat time but this comes at a price of reducing profits. However IMHO I feel that it is a more robust tradeoff.  This is due to in part to the asymmetrical relationship between percentage drawdown and the amount need to return to the previous equity peak high

Howard Bandy alluded to the fact that it is less reliable to maximise for profit over other parameters. I agree with Howard. I quite enjoyed his book and recommend it for budding system developers.

Cheers

Shane


----------



## stevo (20 September 2007)

> The feedback Iv been getting is that while they keep you out of the market, you are often late to get back in and miss the often aggressive rebound.




If getting in late is considered a problem then consider making the filter very sensitive to index moves - I wouldn't suggest using a 100 week moving average. Something simple like 1 period % change criteria or even just straight price moves  - like close is greater than last weeks high to turn on the system and close is less than last week's low to turn off system. 

Just some suggestions - I am not saying that these ideas work. Remember that the index filter might only turn the system on and off, not necessarily trigger exits, and also that a buy signal also has to be triggered when the index filter is "on".

I agree with Shane - they can reduce profits a little. But you can end up making substantially less trades to get not that much less money. I have been through 3 periods in the last 5 years where I didn't get a buy for 2 to 3 months - I am talking longer term systems and I would still have trades running, just no buy signals. Annual return is not the only criteria to consider - Personally drawdown is just as big an issue.

Also if you try the test with and without trade and / or profit pyramiding what happens to drawdown and profits?

stevo


----------



## nizar (20 September 2007)

stevo said:


> Also if you try the test with and without trade and / or profit pyramiding what happens to drawdown and profits?




Thanks for your points.

Without trade pyramiding, win% decreases, average win/average loss decreases, and consequently profits are decreased.
Drawdown about the same.

Note that when i do pyramid, i do choose the option to "favour pyramid trades" as this produces better results (more profit, less maximum drawdown).

Without profit pyramiding, win% increases, profits are halved, drawdowns are halved, number of trades taken are MUCH less.

I was always planning to pyramid profits from the outset, reason being, the money i put into the system is not money i would be needing again for the forseeable future. So I just want to reinvest everything and let it compound.


----------



## nizar (21 September 2007)

stevo said:


> If getting in late is considered a problem then consider making the filter very sensitive to index moves - I wouldn't suggest using a 100 week moving average. Something simple like 1 period % change criteria or even just straight price moves  - like close is greater than last weeks high to turn on the system and close is less than last week's low to turn off system.
> 
> Just some suggestions - I am not saying that these ideas work. Remember that the index filter might only turn the system on and off, not necessarily trigger exits, and also that a buy signal also has to be triggered when the index filter is "on".
> 
> I agree with Shane - they can reduce profits a little. But you can end up making substantially less trades to get not that much less money. I have been through 3 periods in the last 5 years where I didn't get a buy for 2 to 3 months - I am talking longer term systems and I would still have trades running, just no buy signals. Annual return is not the only criteria to consider - Personally drawdown is just as big an issue.




Stevo.

The index filter that you use in which you only buy entry triggers when the index meets certain conditions, is useful in decreasing CLOSED equity drawdowns yeh?

Because leaving open trades to hit their exit in times the system is effectively turned off I suspect will not do much help in decreasing open equity drawdown as these open trades will get hit by the overall market downturn.

In which ways does your system attempt to decrease open equity drawdowns?

There are 2 methods that come to mind:
1/ If the stock hasnt moved X% in n-period then chop it. Useful for stocks then were trending upwards then flattened.

2/ If using an ATR exit, you could apply a condition where if the placement of the ATR exit hasnt moved in n-periods, then chop it.

How to code them for backtesting is another matter!!

Nizar.


----------



## theasxgorilla (21 September 2007)

nizar said:


> The index filter that you use in which you only buy entry triggers when the index meets certain conditions, is useful in decreasing CLOSED equity drawdowns yeh?




My thoughts...I think this is the case...you are less likely to take a new trade that moves against you and depletes closed equity when said index is in gear.  That is my understanding of the concept.



nizar said:


> In which ways does your system attempt to decrease open equity drawdowns?
> 
> There are 2 methods that come to mind:
> 1/ If the stock hasnt moved X% in n-period then chop it. Useful for stocks then were trending upwards then flattened.
> ...




Would it be fair to assume that most severe open equity drawdowns occur at the end of trends?  This has been my observation.  And the more explosive the blow-off rally at the end the greater the likelihood that you will see a large measured open equity drawdown during the subsequent collapse.  When testing '97 - '07 my worst drawdowns are always May 06 and August 07...pretty much the time when the system also performs at its best.

I think the trick is to have a stop in place that captures as much of a blow off rally as possible, if/when it occurs, whilst not over-tightening and inadvertently stopping you out of a trend too soon.  Because of the way they are pulled up rapidly with price I have found that ATR based stops are most effective at this (or some other volatility derived stop anchored to an element of price).

If scenario 1 occurs ie. volatility contracts and the trend flattens first, rather than collapses, I have found that EMA based stops can be more effective.  Whilst an ATR stop (particularly a non-ratcheting stop) will keep its distance an EMA based stop continues to approach price and 'tighten' as a function of how it works.  So in effect as time passes you give the stock less and less downside room to prove itself.

So the answer may not be a case of one or the other.


----------



## nizar (23 September 2007)

Firstly thanks for everyones thoughts and opinions.
Its all been taken into account.

So now i have the latest version of my weekly system ready.

Just some points to made:
(1) Entry has been changed. Im no longer buying all-time highs primarily because iv only just realised that i cannot accurate backtest this strategy with the amount of data that I have.
(2) Backtesting now always starts from 1998. Starting from 1992 iv decided is too far not only in terms of the accuracy of the data but that its more likely that the market dynamics may have changed.
(3) My ATR trailing exit used to be triggered by the LOW of the price of the bar. So an intra-week spike wouldve stopped me out the following week even if the close was above the stop. This is probably why my stop needed to be so wide and consequently i was giving back alot of profit. This has been changed so that the stop is only triggered by the weekly CLOSE. 
(4) Due to point (3) a tighter stop could then be applied and would still keep my in most of the trends. ie. 2.5*ATR instead of 3.5*ATR.
(5) Iv adopted the idea by Shane:



> If you prefer atr based stops then you may choose shares that teh atr values do not exceed a certain percentage
> 
> eg 2*atr(14)/C<0.15
> 
> ...




So pyramiding only triggered when there is at least 15% profit on the previous pyramid level (including level 0).
Before there was only a trigger function.
Trigger + Percent Profit together does not seem to improve any of the stats compared to Percernt Profit alone.
(6) Initial Stop has been set at the trailing exit level for money management purposes.
I have set this to 1.5% for the following tests.


----------



## nizar (23 September 2007)

Monte Carlo Report	

Trade Database Filename	
C:\TradeSimData\NewWeeklyMASTER 1998-2003 15pc 2ATR Turtle Entry Actual 10.trb	

Simulation Summary	
Simulation Date:	23/09/2007
Simulation Time:	8:37:23 AM
Simulation Duration:	286.52 seconds

Trade Parameters	
Initial Capital:	$45,000.00
Portfolio Limit:	100.00%
Maximum number of open positions:	100
Position Size Model:	Fixed Percent Risk
Percentage of capital risked per trade:	1.50%
Position size limit:	100.00%
Portfolio Heat:	100.00%
Pyramid profits:	Yes
Transaction cost (Trade Entry):	$22.00
Transaction cost (Trade Exit):	$22.00
Margin Requirement:	100.00%
Magnify Position Size(& Risk) according to Margin Req:	No
Margin Requirement Daily Interest Rate (Long Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Long Trades):	0.0000%
Margin Requirement Daily Interest Rate (Short Trades):	0.0000%
Margin Requirement Yearly Interest Rate (Short Trades):	0.0000%

Trade Preferences	
Trading Instrument:	Stocks
Break Even Trades:	Process separately
Trade Position Type:	Process all trades
Entry Order Type:	Market Order
Exit Order Type:	Market Order
Minimum Trade Size:	$500.00
Accept Partial Trades:	No
Volume Filter:	Reject Trades if Position Size is greater than
	10.00% of the maximum traded volume
Pyramid Trades:	Yes
Favour Trade Pyramid:	Yes
Start Pyramid at any level up to level:	N/A
Maximum Pyramid Level Limited to:	N/A
Maximum Pyramid Count Limited to:	N/A

Simulation Stats	
Number of trade simulations:	20000
Trades processed per simulation:	1265
Maximum Number of Trades Executed:	179
Average Number of Trades Executed:	158
Minimum Number of Trades Executed:	138
Standard Deviation:	5.21

Profit Stats	
Maximum Profit:	               $421,785.64 (937.30%)
Average Profit:	                          $206,605.81 (459.12%)
Minimum Profit:	                    $105,442.93 (234.32%)
Standard Deviation:	$34,823.52 (77.39%)
Probability of Profit:	100.00%
Probability of Loss:	0.00%

Percent Winning Trade Stats	
Maximum percentage of winning trades:	62.58%
Average percentage of winning trades:	52.10%
Minimum percentage of winning trades:	41.62%
Standard Deviation:	2.64%

Percent Losing Trade Stats	
Maximum percentage of losing trades:	58.38%
Average percentage of losing Trades:	47.90%
Minimum percentage of losing trades:	37.42%
Standard Deviation:	2.64%

Average Relative Dollar Drawdown Stats	
Maximum of the Average Relative Dollar Drawdown:	$2,503.85
Average of the Average Relative Dollar Drawdown:	$1,119.33
Minimum of the Average Relative Dollar Drawdown:	$584.95
Standard Deviation:	$213.27

Average Relative Percent Drawdown Stats	
Maximum of the Average Relative Percent Drawdown:	1.7230%
Average of the Average Relative Percent Drawdown:	0.9085%
Minimum of the Average Relative Percent Drawdown:	0.5263%
Standard Deviation:	0.1387%

Maximum Peak-to-Valley Dollar Drawdown Stats	
Maximum Absolute Dollar Drawdown:	$60,067.90
Average Absolute Dollar Drawdown:	$16,956.65
Minimum Absolute Dollar Drawdown:	$4,375.91
Standard Deviation:	$6,799.79

Maximum Peak-to-Valley Percent Drawdown Stats	
Maximum Absolute Percent Drawdown:	27.8473%
Average Absolute Percent Drawdown:	9.8650%
Minimum Absolute Percent Drawdown:	5.0345%
Standard Deviation:	2.2823%


----------



## nizar (23 September 2007)

I forgot to add.
Testing period is on the whole ASX ($1million turnover per week liquidity filter) from 01-01-1998 until 31-12-2003 (6 years).


Detailed Report			

Trade Database Filename			
C:\TradeSimData\NewWeeklyMASTER 1998-2003 15pc 2ATR Turtle Entry Actual 10.trb			

Simulation Summary			
Simulation Date:	23/09/2007		
Simulation Time:	9:00:16 AM		
Simulation Duration:	0.45 seconds		

Trade Summary			
Earliest Entry Date in the Trade Database:	2/01/1998		
Latest Entry Date in the Trade Database:	12/12/2003		
Earliest Exit Date in the Trade Database:	20/02/1998		
Latest Exit Date in the Trade Database:	9/01/2004		

Start Trade Entry Date:	2/01/1998		
Stop Trade Entry Date:	12/12/2003		
First Entry Date:	2/01/1998		
Last Entry Date:	28/11/2003		
First Exit Date:	9/04/1998		
Last Exit Date:	24/12/2003		

Total Trading duration:	2182 days		

Profit Summary			
Profit Status:	PROFITABLE		
Starting Capital:	$45,000.00		
Finishing Capital:	$281,900.17		
Maximum Equity/(Date):	$236,900.17 (24/12/2003)		
Minimum Equity/(Date):	$687.26 (9/04/1998)		
Gross Trade Profit:	$328,373.16 (729.72%)		
Gross Trade Loss:	-$91,472.99 (-203.27%)		
Total Net Profit:	$236,900.17 (526.44%)		
Average Profit per Trade:	$1,548.37		
Profit Factor:	3.5898		
Profit Index:	72.14%		
Total Transaction Cost:	$6,732.00		
Total Slippage:	$77,130.18		
Total Trade Interest:	$0.00		
Daily Compound Interest Rate:	0.0841%		
Annualized Compound Interest Rate:	35.9254%		

Trade Statistics			
Trades Processed:	1271		
Trades Taken:	153		
Partial Trades Taken:	0		
Trades Rejected:	739		
Winning Trades:	82 (53.59%)		
Losing Trades:	71 (46.41%)		
Breakeven Trades:	0 (0.00%)		

Largest Winning Trade/(Date):	$27,505.82 (7/12/2001)		
Largest Losing Trade/(Date):	-$4,233.32 (28/06/2002)		
Average Winning Trade:	$4,004.55		
Average Losing Trade:	-$1,288.35		
Average Win/Average Loss:	3.1083		

Trade Breakdown	Long and Short Trades	Long Trades	Short Trades
Normal Exit:	121 (79.08%)	121 (79.08%)	0 (0.00%)
Protective Stop:	18 (11.76%)	18 (11.76%)	0 (0.00%)
Open Trade:	14 (9.15%)	14 (9.15%)	0 (0.00%)

Total Trades:	153 (100.00%)	153 (100.00%)	0 (0.00%)

Trade Duration Statistics	Winning and Losing Trades	Winning Trades	Losing Trades
Maximum Trade Duration:	1239 (days)	1239 (days)	245 (days)
Minimum Trade Duration:	0 (days)	20 (days)	0 (days)
Average Trade Duration:	138.98 (days)	203.99 (days)	63.90 (days)

Consecutive Trade Statistics			
Maximum consecutive winning trades:	8		
Maximum consecutive losing trades:	6		
Average consecutive winning trades:	2.22		
Average consecutive losing trades:	1.97		

Trade Expectation Statistics			
Normalized Expectation per dollar risked:	$1.33		
Maximum Reward/Risk ratio:	35.43		
Minimum Reward/Risk ratio:	-1.23		
Average Positive Reward/Risk ratio:	$3.00		
Average Negative Reward/Risk ratio:	-$0.59		

Relative Drawdown			
Maximum Dollar Drawdown/(Date):	$13,533.74 (5/07/2002)		
Maximum Percentage Drawdown/(Date):	5.9480% (5/07/2002)		

Absolute (Peak-to-Valley) Dollar Drawdown			
Maximum Dollar Drawdown:	$24,202.46 (10.5600%)		
Capital Peak/(Date):	$229,178.79 (5/04/2002)		
Capital Valley/(Date):	$204,976.33 (14/03/2003)		

Absolute (Peak-to-Valley) Percent Drawdown			
Maximum Percentage Drawdown:	10.5600% ($24,202.46)		
Capital Peak/(Date):	$229,178.79 (5/04/2002)		
Capital Valley/(Date):	$204,976.33 (14/03/2003)


----------



## nizar (23 September 2007)

And the charts.


----------



## nizar (23 September 2007)

You will notice that position size limit is set to 100% and portfolio heat is also set to 100%.

I have tried position size limit as 15% and portfolio heat set to 20% and there is no difference to the results.

Anything less than this and the results are affected (not significantly).

But nevertheless, i am happy with these values.

I have removed the best 3 and the worst 3 trades from the trade database -- virtually no difference in the results.

As usual, im looking for comments.

As you can see, Iv already made a great deal of progress so far 

THANK YOU


----------



## tech/a (23 September 2007)

For a longterm system the Curve looks smooth.
All the numbers look within the "expected" parameters.
Wins over 50% unusual.

You can turbo charge it with a 50% margin--just have a look at D/D.
ANZ will probably allow most of your stocks to be traded margin.They have a massive list.

Good work Nizar I know how long and at times difficult this can be.


----------



## Shane Baker (23 September 2007)

Hi Nizar

Have a look at the portfolio heat section in theTradeSim manual. I would suggest setting it to a lower level than 100%....much lower.

Cheers

Shane


----------



## nizar (23 September 2007)

Shane Baker said:


> Hi Nizar
> 
> Have a look at the portfolio heat section in theTradeSim manual. I would suggest setting it to a lower level than 100%....much lower.
> 
> ...




Shane.
See post #141 of this thread.


----------



## nizar (23 September 2007)

tech/a said:


> For a longterm system the Curve looks smooth.




Its because the stop is relatively tight, if you notice average holding time for a winner is quite short for a long term system (if that makes sense!) at about 200 days.

Smoother equity curve is something I always look for.

Something else i taught myself is how important trade frequency is!
Look at these two systems.

System 1:
*42% winners
*Average win/Average loss = 5.4

System 2:
*48% winners
*Average win/Average loss = 3.00


I always thought those were the ONLY 2 important parameters to know to work out expectancy.

But what I realise now is that TRADE FREQUENCY determines how often you can exploit that positive expectancy.

So while it may seem that system 1 is more profitable, in actual fact the annualised compounded annual return for system 1 was 26%, for system 2 it was 32%.

Why?

Because in the test period, system 2 took 247 trades, while system 1 took 109 trades.


----------



## tech/a (23 September 2007)

Shane.

In reality I doubt that Nizars or most longterm systems would have much over 20% Portfolio heat at the one time.


----------



## Shane Baker (23 September 2007)

Hi Guys,

20% portfolio heat means you 20% of capital everytime you re-enter the market, after being fully exited. For example if you run the system and twelve trades come up as per Nizar's 1.5% risk...... and you take them then you immediately have 18% risk in the market. The next day the US invades Iran, oil spikes and the markets tank 20%. Immediately your account is down a minimum of 18%. That is the effect of portfolio heat. It is the amount of equity you are willing to risk at any one single point in time, that you can control. I personally suggest 15% as a max (although the system will show you where the max is in TradeSim on the portfolio heat chart) as it takes nearly 20% to gain that amount back.

I just wanted to make a point regarding risk management in designing the system and understanding what the terms may mean in real time trading experience. I think Nizar has come a long way in a short time and should be congratulated for his efforts. I just wanted to temper enthusiasm with some real time risk control. I have had two periods of losing a full amount of heat and so I believe it can happen to all of us.

Cheers

Shane


----------



## nizar (23 September 2007)

Shane Baker said:


> Hi Guys,
> 
> 20% portfolio heat means you 20% of capital everytime you re-enter the market, after being fully exited. For example if you run the system and twelve trades come up as per Nizar's 1.5% risk...... and you take them then you immediately have 18% risk in the market. The next day the US invades Iran, oil spikes and the markets tank 20%. Immediately your account is down a minimum of 18%. That is the effect of portfolio heat. It is the amount of equity you are willing to risk at any one single point in time, that you can control. I personally suggest 15% as a max (although the system will show you where the max is in TradeSim on the portfolio heat chart) as it takes nearly 20% to gain that amount back.
> 
> ...




Hi Shane.

Thanks for your thoughts.
You do make a good point.
Throughout the testing period the maximum number of positions held in the portfolio was 12.
And Portfolio Heat at its maximum was 17%.

Nizar.


----------

