# Backtest your way to financial security



## It's Snake Pliskin (26 November 2006)

After reading a post in another thread, I thought I would start this one. The title is not sarcasm by the way.



			
				MichaelD said:
			
		

> I'm afraid I'll have to steadfastly disagree with this.
> 
> I agree on one point - that it is very hard or impossible to backtest or prospectively test discretionary strategies with software.
> 
> ...




Michael,

I have some questions:

Why do people backtest?
What constitutes a successful backtest leading to a blueprint?
What biases are there to be aware of?
What issues are there with equipment and software deficiencies?
Why can every aspect of human nature be backtested?
What is one thing that corrupts all backtesting, and what same thing is relevant for trading without backtesting?

Answers to these questions would be greatly appreciated. Others please feel free to comment.

Thanks
Snake


----------



## Bobby (27 November 2006)

*Re: Backtest your way to financial security.*



			
				It's Snake Pliskin said:
			
		

> After reading a post in another thread, I thought I would start this one. The title is not sarcasm by the way.
> 
> 
> 
> ...



Greetings Snake,

Backtesting is just that! Testing the past ~ Many have done the testing to form their trading plans for the future.

I DON'T.
Will be interesting to see what answers you get.

Regards Bob.


----------



## MichaelD (27 November 2006)

*Re: Backtest your way to financial security.*

A very interesting topic indeed.


			
				It's Snake Pliskin said:
			
		

> Why do people backtest?



To work out whether a particular strategy has ever worked in the past. If it hasn't ever worked in the past, it is less likely to work now.


			
				It's Snake Pliskin said:
			
		

> What constitutes a successful backtest leading to a blueprint?



The answer for me is acceptable profitability, acceptable drawdown and acceptable trading characteristics in all test sequences and test timeframes (bear, bull, choppy market and a few others).


			
				It's Snake Pliskin said:
			
		

> What biases are there to be aware of?



You could write a book about these, but in the spirit of Socratic discussion, here are some of the more obvious ones;
1. Inclusion bias - testing the strategy on stocks which are now part of the test universe but wouldn't have been at the time the backtest trade is taken.
2. Survivor bias - not including delisted stocks in the test thus positively influencing the results.
3. Hindsight bias - you can see the right side of the chart if manually backtesting which may bias your decision to exit.


			
				It's Snake Pliskin said:
			
		

> What issues are there with equipment and software deficiencies?



 It's TIME CONSUMING to write and run multiple backtests. There's also quite a steep learning curve involved in learning to drive the software for mechanical backtesting. It's even more time consuming to backtest discretionary trading.


			
				It's Snake Pliskin said:
			
		

> Why can every aspect of human nature be backtested?



I'll presume you mean can't, not can. Backtesting is mechanical. It's human nature to want to "second guess" the machine when actually trading if choosing between entry signals. It's human nature to not want to take a stop when your system says exit because you "know better".


			
				It's Snake Pliskin said:
			
		

> What is one thing that corrupts all backtesting, and what same thing is relevant for trading without backtesting?



You've got me on this one. I'm going to go for human emotion as the answer.


			
				Bobby said:
			
		

> I DON'T .



Why not? (I'm interested as it contrasts to my current approach to the market which is to backtest everything before I trade it.)


----------



## tech/a (27 November 2006)

I need to spend time answering this,Just dont have that time right now.

However even though on the other thread (my post below.) appears to be contradictory to Michaels veiw it *certainly isnt*.



> Michael.
> 
> I certainly used to have your view.
> That everything should be tested and results tabulated to a "blueprint" from which to work.
> ...





I'll re phrase part in that I dont have the expertise to backtest what I do trading short term.If I could I would.I trade in many timeframes in the one move.I dont have software that can test this.
Experience does play a huge part and that experience has been gained from many hrs of backtesting and systems developement.

More when I have time.

*Snake.*

*Survivorship.*

Todays ASX 300 isnt that of 10 yrs ago.So we are backtesting todays as how it would have performed 10 yrs ago.Practically chances are it would not have even been in the list.


----------



## It's Snake Pliskin (27 November 2006)

tech/a said:
			
		

> *Snake.*
> 
> *Survivorship.*
> 
> Todays ASX 300 isnt that of 10 yrs ago.So we are backtesting todays as how it would have performed 10 yrs ago.Practically chances are it would not have even been in the list.




I was talking about time before which is what you are talking about too. 

Regards
Snake


----------



## It's Snake Pliskin (27 November 2006)

*Re: Backtest your way to financial security.*



			
				MichaelD said:
			
		

> You've got me on this one. I'm going to go for human emotion as the answer.




I was hoping you could tell me the answer because I don't think I know. It's definately not emotion, though.


----------



## It's Snake Pliskin (27 November 2006)

*Re: Backtest your way to financial security.*



			
				Bobby said:
			
		

> Greetings Snake,
> 
> Backtesting is just that! Testing the past ~ Many have done the testing to form their trading plans for the future.
> 
> ...




Yes Bobby, I have always had trouble linking the past with the future. Markets change, companies change or fail to survive, new products change trading, so how can the past be representative of the future? How can past bull markets be representative of bullmarkets of late?

I am not criticising backtesting here but would love to have some experts convince me that it is valid and worthy of doing.

Bob have a go at the questions. I like to read what you write!
Regards
Snake :run:

*WayneL,*

Could you please answer the questions too. Bring that avatar along when you do it.


----------



## battiwallah (27 November 2006)

I don’t understand trading – why anyone would want to do this for a living?  As an investor with a buy and hold philosophy, I find it strange that there are some investors who take such extraordinary risks with their wealth.

We should realise that for every buyer there is a seller.  Every time you buy the stock that you think is going to go gangbusters, the seller thinks it is going to go south.  So In order to be a successful trader you have to be consistently a better judge of the stock than everyone else.  You have to be able to outguess others most of the time.  

And remember that not only do you have to do it better than others, but significantly better since you have to cover your brokerage and your capital gains tax when you sell, in addition to a risk premium.  And if you are leveraged then add interest and a further risk premium.  OK, you do get to write off losses against gains, but are you honest in assessing your success rate?  Do your successes lead you to discount or ignore your failures and do you become over-confident?  As they say: “success has a thousand fathers; failure is an orphan”.

The market overall for the last 17 or so years has been very forgiving of investors generally.  It has been the longest bull run in history, except for some minor upsets, and for that reason many people think that this is the status quo – the normal state of affairs in investing.  It ain’t so.  A large number of younger investors have never known a major bear market (a drop of 40% or more) and are in for a very unpleasant surprise when it happens; particularly if they are leveraged to the hilt with margin loans.  A lot of people in that position jumped out of their office windows in 1929.

To those traders who can point to a history of successful trading I ask:  how much of that has been due to your skill and how much to the good fortune of simply being in a sustained bull market?  I suspect that it has been the market that has been carrying most traders and if they were honest they would admit this and realise the risks they are taking.

Traders should realise that they are in a negative sum game.  The universe of traders has to lose money in total – the losses go in brokerage, taxes and interest.  Most of the time traders are merely swapping stocks among themselves (like managed funds).

Admit that trading is an expensive hobby and go and get a real job!!!

Any comments?


----------



## wayneL (27 November 2006)

battiwallah said:
			
		

> Admit that trading is an expensive hobby and go and get a real job!!!
> 
> Any comments?



Oh brother, not another one.

My comment is to get qualified on what you are commenting on before commenting on something you are obviously not qualified to comment on.


----------



## wayneL (27 November 2006)

*Re: Backtest your way to financial security.*



			
				It's Snake Pliskin said:
			
		

> *WayneL,*
> 
> Could you please answer the questions too. Bring that avatar along when you do it.




I'm by no means an expert on backtesting, but will put some thought into this.

later.....


----------



## professor_frink (27 November 2006)

battiwallah said:
			
		

> I don’t understand trading – why anyone would want to do this for a living?  As an investor with a buy and hold philosophy, I find it strange that there are some investors who take such extraordinary risks with their wealth.
> 
> We should realise that for every buyer there is a seller.  Every time you buy the stock that you think is going to go gangbusters, the seller thinks it is going to go south.  So In order to be a successful trader you have to be consistently a better judge of the stock than everyone else.  You have to be able to outguess others most of the time.
> 
> ...




Holy, angry investor Batman!
It's Realist's long lost twin!
Quick, to the batcave, to get our investor repellant spray.
It's the only way to keep us traders safe


----------



## wayneL (27 November 2006)

professor_frink said:
			
		

> Holy, angry investor Batman!
> It's Realist's long lost twin!
> Quick, to the batcave, to get our investor repellant spray.
> It's the only way to keep us traders safe



heheheh  

I might have a bit of fun with this later... unless someone gets in before me  

Right now I have to go and do a deal with a wage slave. back later


----------



## It's Snake Pliskin (27 November 2006)

battiwallah


> I don’t understand trading – why anyone would want to do this for a living?  As an investor with a buy and hold philosophy,




Doing such is RISKY!



> We should realise that for every buyer there is a seller.



Actually there are multiples of sellers and buyers depending on the $ value of transactions. To suggest there is one buyer for every seller, or vice versa, is ridiculous. 



> Every time you buy the stock that you think is going to go gangbusters, the seller thinks it is going to go south.  So In order to be a successful trader you have to be consistently a better judge of the stock than everyone else.  You have to be able to outguess others most of the time.




It isn't about guessing or judging. Businesses are the same.



> And remember that not only do you have to do it better than others, but significantly better since you have to cover your brokerage and your capital gains tax when you sell, in addition to a risk premium.  And if you are leveraged then add interest and a further risk premium.  OK, you do get to write off losses against gains, but are you honest in assessing your success rate?  Do your successes lead you to discount or ignore your failures and do you become over-confident?  As they say: “success has a thousand fathers; failure is an orphan”.




Great advice.  



> The market overall for the last 17 or so years has been very forgiving of investors generally.  It has been the longest bull run in history, except for some minor upsets, and for that reason many people think that this is the status quo – the normal state of affairs in investing.  It ain’t so.  A large number of younger investors have never known a major bear market (a drop of 40% or more) and are in for a very unpleasant surprise when it happens; particularly if they are leveraged to the hilt with margin loans.  A lot of people in that position jumped out of their office windows in 1929.




Yes I remember that back in 1929.  



> To those traders who can point to a history of successful trading I ask:  how much of that has been due to your skill and how much to the good fortune of simply being in a sustained bull market?  I suspect that it has been the market that has been carrying most traders and if they were honest they would admit this and realise the risks they are taking.




From an investor.



> Traders should realise that they are in a negative sum game.  The universe of traders has to lose money in total – the losses go in brokerage, taxes and interest.  Most of the time traders are merely swapping stocks among themselves (like managed funds).



I believe it is a minus some game.



> Admit that trading is an expensive hobby and go and get a real job!



I captured one exclamation mark. But why the vicious tone?



> Any comments?




Have a nice day.


----------



## carmo (27 November 2006)

Not sure if it constitutes back testing or not, but I looked at my losers and most of them had one thing in common, lack of volume. So I consider I learnt from having a good look back.


----------



## MichaelD (27 November 2006)

battiwallah said:
			
		

> So In order to be a successful trader you have to be consistently a better judge of the stock than everyone else.



Totally incorrect, and that's why you can only understand buy/hold. The best traders are the best risk managers. The "best" judges are making money writing newsletters, not trading.


----------



## Bobby (27 November 2006)

*Re: Backtest your way to financial security.*



			
				It's Snake Pliskin said:
			
		

> Yes Bobby, I have always had trouble linking the past with the future. Markets change, companies change or fail to survive, new products change trading, so how can the past be representative of the future? How can past bull markets be representative of bullmarkets of late?
> 
> I am not criticising backtesting here but would love to have some experts convince me that it is valid and worthy of doing.
> 
> ...



Hello Snake,

Hope this answer helps, its for Michael as well ( regarding I DON'T ).

When one backtests they see past performance that was based on EOD trading plans, now today we have realtime online technology that allows us to  see , plan , & trade in an instant.
The key levels used by the EOD traders can be used to advantage by the realtime trader.
Guess what i'm trying to say is ~ Times have changed, new conditions now apply.
Backtesting past performance = Just that .
I may be wrong but think about it   

I like this, what stage are you at ? 

1. *Unconscious incompetence*  = you don't even know you don't know what to do.
2.*Conscious incompetence*  = you now know you don't know what to do.
3.*Conscious competence*  = you know what to do. if only you can do it.

4.*Unconsious competence*  =  { have a guess at this one }

I'm almost at number 3 , I hope *.

Cheers Bob.


----------



## tech/a (27 November 2006)

Wow there is some stuff in this thread to cover.



			
				battiwallah said:
			
		

> I don’t understand trading – why anyone would want to do this for a living?  As an investor with a buy and hold philosophy, I find it strange that there are some investors who take such extraordinary risks with their wealth.




Investors without trading experience have no place "Risking" their wealth attempting to trade.Indeed they will "Risk" their wealth.



> We should realise that for every buyer there is a seller.  Every time you buy the stock that you think is going to go gangbusters, the seller thinks it is going to go south.




Well the seller could also be taking a profit after holding well before you bought,he may also be selling at a breakeven after getting his original trade wrong.  



> So In order to be a successful trader you have to be consistently a better judge of the stock than everyone else.  You have to be able to outguess others most of the time.




Not so you need to understand Reward to Risk how it works and how to be consistantly profitable over the long term even if trading very short timeframes.Your statement is common with those who dont or cannot trade consistently profitably.If you believe its about guessing then you should never attempt to trade.  



> And remember that not only do you have to do it better than others, but significantly better since you have to cover your brokerage and your capital gains tax when you sell, in addition to a risk premium.  And if you are leveraged then add interest and a further risk premium.




Better tha others* only in the Business of trading * 



> OK, you do get to write off losses against gains, but are you honest in assessing your success rate?  Do your successes lead you to discount or ignore your failures and do you become over-confident?  As they say: “success has a thousand fathers; failure is an orphan”.




My success rate is around 2-3/10 my profit not only would make your eyes water but it has mine going as well.Frankly the returns that can be made in the maket are *OBSCENE* Without sounding like a blow hard but using the following as an example---On Fridays trade WMT-- I made the equivalent
to 3 mths pay for my Production Manager in 5 hrs. This morning in a split second on open I made 3 weeks of his wage---*that is OBSCENE*



> The market overall for the last 17 or so years has been very forgiving of investors generally.  It has been the longest bull run in history, except for some minor upsets, and for that reason many people think that this is the status quo – the normal state of affairs in investing.  It ain’t so.  A large number of younger investors have never known a major bear market (a drop of 40% or more) and are in for a very unpleasant surprise when it happens; particularly if they are leveraged to the hilt with margin loans.  A lot of people in that position jumped out of their office windows in 1929.




I love doom and gloom ---very profitable. One of my investment portfolio's is available for public veiw and has been live traded for 4 yrs.Its designed and traded on Margin. If it did drop 50% in a day or so the return over that 5 yrs period would still be 400% on initial capital. IE starting Capital was $30k----- $70K on Margin loan.Balance as at Friday $386,000---50% = $193,000 les the $70k to lender. Balance $126,000.

As for the longest Bullmarket in history you havent looked at a chart of the ASX lately.



> To those traders who can point to a history of successful trading I ask:  how much of that has been due to your skill and how much to the good fortune of simply being in a sustained bull market?  I suspect that it has been the market that has been carrying most traders and if they were honest they would admit this and realise the risks they are taking.




Both the market and skill. It takes skill to outperform the market. Every business has risk--however in all busines including trading I take quantified risks--if you dont take quantified risks---even investors take risk--- then you'll never excell.



> Traders should realise that they are in a negative sum game.  The universe of traders has to lose money in total – the losses go in brokerage, taxes and interest.  Most of the time traders are merely swapping stocks among themselves (like managed funds).




5% of us can and do feed then from the constant supply of new blood who venture into the business of trading.
My Company dominates in its field here and is no different to any business which succeeds---it has a greater market share and some fail.Many competitors have been and gone over the 30 yrs I've been in business. If they cannot do it as well as I or as profitably then they deserve what they sow.



> Admit that trading is an expensive hobby and go and get a real job!!!
> 
> Any comments?




Its not a hobby to me yet I dont trade fulltime,it is simply part of my investment strategy.

Your comments are very general and you take the veiw that all here are in one box. You may well be suprised at how inept your own investment planning may well be in comparison to some here. Agreed many are struggling.
But put in the apprenticeship and the profit will come as will the Business acumen.


----------



## swingstar (27 November 2006)

tech/a said:
			
		

> Every business has risk--however in all busines including trading I take quantified risks--if you dont take quantified risks---even investors take risk--- then you'll never excell.




Just quoting for emphasis. There is no difference between trading and any other type of investing, be it business, property, etc.


----------



## tech/a (27 November 2006)

swingstar said:
			
		

> Just quoting for emphasis. There is no difference between trading and any other type of investing, be it business, property, etc.




Exactly Swing.


----------



## stevo (27 November 2006)

Snake
Wow - good questions. Certainly got me thinking. Trading mechanical systems completely transformed my trading to very average to results that I am very happy with. Developing trading systems is a very rewarding approach to the markets. Back-testing allows me to develop systems.

There have been whole books written on this topic. Beyond Technical Analysis by Tushar Chande springs to hand as a starting point. Although some on these forums are well past this sort of stuff the main points of the book are still reasonable.

My responses are obviously my view of trading and what has worked for me;
*Why do people back-test?*
People back-test in an attempt to develop a winning trading system. For the average trader, like myself, trading a mechanical trading system maximises my chances of success. Without a well thought out, extensively "pressure" tested, and documented set of rules to trade by I am indeed a worse than average trader. With a documented system I can teach nearly anybody to trade successfully. And I am a lousy teacher!

*What constitutes a successful back-test leading to a blueprint?*
One that, over a statistically significant number of trades is profitable.

*What biases are there to be aware of?*
Lots! Back-testing can give insight into these biases. Van Tharpe goes into biases in some detail.

*What issues are there with equipment and software deficiencies?*
Data quality is an issue that needs to be considered. Software is very good these days and is more limited on our abilities to code our ideas and weaknesses than anything inherently wrong with the software. The power of software is quite amazing these days - we can test fundamental data as well as price and volume with AmiBroker and we can use artificial intelligence to optimise. We have the power of Monte Carlo tests at our fingertips. The main limiting factor is ourselves.

*Why can every aspect of human nature be back-tested?*
I doubt that it can - but it is possible to test procrastination and poor execution so it probably possible to test for most issues that a rational person would confront.

*What is one thing that corrupts all back-testing, and what same thing is relevant for trading without back-testing?*
Obviously back-testing can tend to curve fit - which is why you can also forward test. Holding some data outside of the initial sample  is a well known strategy. It is possible to trade without a tested system. But why would someone consciously trade a system if they don't know what it's chances of success are?

I am glad that you didn't ask about optimisation!

Stevo
http://drawdown.blogspot.com/


----------



## MichaelD (27 November 2006)

*Re: Backtest your way to financial security.*



			
				Bobby said:
			
		

> When one backtests they see past performance that was based on EOD trading plans, now today we have realtime online technology that allows us to  see , plan , & trade in an instant.
> The key levels used by the EOD traders can be used to advantage by the realtime trader.



This presumes that you want or need to engage the market intraday. I don't. That would for me defeat the whole point of what I am trying to achieve lifestyle wise by trading.

Backtesting intraday strategies is certainly possible, but is very time consuming (I have access to tick data if I want to utilize it for backtesting, but so far have chosen not to use it as part of a system design).

EOD trader levels do indeed appear to be used by intraday traders to their advantage - 'tis one of the reasons I don't put my stops in market.



			
				Bobby said:
			
		

> Guess what i'm trying to say is ~ Times have changed, new conditions now apply.



"This time it's different."   



			
				Bobby said:
			
		

> Backtesting past performance = Just that .



Yep. In the absence of a crystal ball, it's the next best thing because it can clearly alert you in advance to the following;
1. You're trading a system which has never been profitable so is unlikely to be profitable now.
2. Your system is no longer trading within previously tested limits - why?



			
				Bobby said:
			
		

> what stage are you at ?



Position trading - 3, just following the plan without fear or favour
Short term trading - 2 and steadily working towards 3


----------



## nizar (27 November 2006)

stevo said:
			
		

> *Why can every aspect of human nature be back-tested?*
> I doubt that it can - but it is possible to test procrastination and poor execution so it probably possible to test for most issues that a rational person would confront.




Stevo

How can back-testing, test procrastination and poor execution?


----------



## It's Snake Pliskin (27 November 2006)

*Re: Backtest your way to financial security.*



			
				Bobby said:
			
		

> Hello Snake,
> 
> Hope this answer helps, its for Michael as well ( regarding I DON'T ).
> 
> ...




Bob,

4. Intuitively trading?


----------



## It's Snake Pliskin (27 November 2006)

stevo said:
			
		

> Snake
> Wow - good questions. Certainly got me thinking. Trading mechanical systems completely transformed my trading to very average to results that I am very happy with. Developing trading systems is a very rewarding approach to the markets. Back-testing allows me to develop systems.
> 
> There have been whole books written on this topic. Beyond Technical Analysis by Tushar Chande springs to hand as a starting point. Although some on these forums are well past this sort of stuff the main points of the book are still reasonable.
> ...




Thanks Stevo. 

Yes optimisation could open up a can of worms.

Snake


----------



## It's Snake Pliskin (27 November 2006)

carmo said:
			
		

> Not sure if it constitutes back testing or not, but I looked at my losers and most of them had one thing in common, lack of volume. So I consider I learnt from having a good look back.




And you didn't need software to realise it! It's marvelous how the human brain can learn through experience. It is rather uncomfortable taking the plunge though without some safety blanket so to speak - human nature.


----------



## tech/a (28 November 2006)

nizar said:
			
		

> Stevo
> 
> How can back-testing, test procrastination and poor execution?





It should go a long way to eliminating it.Once you have the trading "Blueprint" .
If you follow exactly entry/exit and stops you'll return similar numbers to those found in backtesting.
Certainly been the case in the 3 I use.
I know Stevo's had the same result with his and Jose' Silva.

The most consistent returns I have found are those returned by traders who have a well tested simple Trading System. *WELL TESTED*


----------



## stevo (28 November 2006)

_"How can back-testing, test procrastination and poor execution?"_

Procrastination could be tested by taking only a % of signals that are triggered. Or random entry factor could be added to the buy signal, such as "buy only if I toss heads or roll a 6 on the dice" type approach. 

Poor execution could be along the lines of buy and sell at the worst possible price, or buy at a random price above the midpoint for the week.

It really comes back to thinking creatively. Try to make the system fail. If you are trading a daily timeframe try delaying the exits for a couple of days and see what happens. I am sure that the time delay could be random - "sell sometime in the week after the exit is given. I can't trade a daily timeframe due to work commitments but it took me some time to work this out. It's much better to learn through back-testing than to learn through experience and real dollars. Experience can be the worst teacher! 

I usually test with a random entry / exit in the week following the signal since I trade on a weekly timeframe. If a system returns a positive result even when the trades are executed at the worst possible price then it is more robust than a system that doesn't. Try all the possible combinations. 

I wouldn't trade a system that relies on me executing the trades at the open or the close price or a very specific trigger price but fails at a random price or worst possible price for the day / week. But I would like to know that the system tends to work better if I get out at the open price. 

Stevo


----------



## tech/a (28 November 2006)

stevo said:
			
		

> _"How can back-testing, test procrastination and poor execution?"_




Good points Steve.



> Procrastination could be tested by taking only a % of signals that are triggered. Or random entry factor could be added to the buy signal, such as "buy only if I toss heads or roll a 6 on the dice" type approach.
> 
> Poor execution could be along the lines of buy and sell at the worst possible price, or buy at a random price above the midpoint for the week.
> 
> ...




Montecarlo testing will turn up a great deal. For those un familiar.

Lets say you test 20000 portfolios.

It's like giving 20000 people your system and your starting capital and telling them to follow it for the test period,then coming back and reporting the results.

Some one off results will return a profit but when Montecarlo tested 80% of portfolio's may return a profit and 20% fail. I and I'm sure steve look for 100% returning profit.I then look at the deviation from the Average to the worst and best performing portfolio. I personally look for around 20% max from best to worst.

Steve's other ways of testing worst cases are very good.

As Steve points out do everything you can to FAIL your system after all some will place $100s of thousands into trading methodologies in the form of Super or hard earned wealth created over many years.
Better to find failure now than when fully invested.


----------



## rub92me (28 November 2006)

I personally would not be comfortable just back testing on recent (say past 24 months) historical data on one exchange. 
I wonder whether it would be better to back test on a variety of data sets you can create with certain characteristics. (Without any pre-conceived bias of what the outcome should be)

1) A number of Bull datasets – a basket of 200 stocks with certain minimum price / volume characteristics with the following criteria:
# 60-80% went up by more than x % over a given time
#10- 30% stayed within x% of the start price over a given time
# 5-20 % went up more than x % over a given time

2) A number of Sideways datasets - a basket of 200 stocks with certain minimum price / volume characteristics with the following criteria:
# 10-30% went up by more than x % over a given time
# 30-50% stayed within x% of the start price over a given time
# 10-30 % went up more than x % over a given time

3) A number of Bear datasets - a basket of 200 stocks with certain minimum price / volume characteristics with the following criteria:
# 5-20 % went up by more than x % over a given time
#10-30% stayed within x% of the start price over a given time
# 60-80% went up more than x % over a given time

I wonder if anyone has attempted to do this with their trading systems, to see how robust they are in certain types of markets, or am I barking up the wrong tree here?


----------



## Bobby (28 November 2006)

*Re: Backtest your way to financial security.*



			
				It's Snake Pliskin said:
			
		

> Bob,
> 
> 4. Intuitively trading?




Yes Snake   
You subconsciously sense when to make that trade .

Michael- Thanks for your reply, you said that you don't place a stop in the market, I understand why, but could you expain in your own words for others.

Like to know more of your thoughts on backtesting  " discretionary " as markets follow volume this may be a challenge.

Cheers Bob.


----------



## It's Snake Pliskin (28 November 2006)

*Re: Backtest your way to financial security.*



			
				Bobby said:
			
		

> Yes Snake
> You subconsciously sense when to make that trade  .Cheers Bob.




The marvelous human brain.


----------



## rub92me (28 November 2006)

rub92me said:
			
		

> 1) A number of Bull datasets – a basket of 200 stocks with certain minimum price / volume characteristics with the following criteria:
> # 60-80% went up by more than x % over a given time
> #10- 30% stayed within x% of the start price over a given time
> # 5-20 % went up more than x % over a given time
> ...



Sorry didn't proofread this properly; should be a-b% went *down* more than x% for the third category in these scenarios


----------



## stevo (28 November 2006)

_"I wonder whether it would be better to back test on a variety of data sets you can create with certain characteristics."_

This is a very good idea. You could also great random data sets or ones just based on the alphabet. You can also test different time periods and do walk forward testing using something like Fred's IO (Intelligent Optimizer). Monte Carlo testing that Tech mentions above also give insights into performance with different stock selections. 

I like to have at least 5 years worth of data. It's interesting to watch the market change over time - companies come and go. The dataset is not static and I cannot guarantee that is is 100% correct. It's great to have the 1999 to 2003 period in the tests. 

*I guess the obvious thing to say about back testing and trading systems are  that there are no guarantees. *I am trading systems that I developed back in 2002 and 2003 using the market data from the period prior to 2003. These same systems test quite well now and have performed to expectations given the market conditions. So they forward tested well! I feel reasonably confident that they will perform going forward, although I have been fortunate, as many have, favourable market conditions have prevailed since 2003.

Tech and some others that use long term mechanical trading systems have shown that systems do work. Some are very surprised at the money they have made using a system (my very trusting brother is one!). At the same time other less well thought out strategies / traders have done a lot of damage. I have seen people post that they put all there money into Telstra and have lost 40% of their equity! A "no strategy" (have a punt, it can't go down anymore) style buy and hold strategy is not for me. But each to their own, what works for one doesn't work for another.

Stevo


----------



## tech/a (28 November 2006)

> *I guess the obvious thing to say about back testing and trading systems are that there are no guarantees.*




Ah yes.

But what you do have a clearly defined parameters that will tell you if the system is no longer working al la turtle system.
OR
If it trades outside the know parameters you have early warning that something is seriosly wrong. IE market conditions havent been seen like those being traded in the test period.
OR
The system is failing.
OR 
Your system testing was/is flawed.

Like Steve I have been trading 3 systems designed using data from 1996-2002 and all have not traded outside those parameters delived during testing.


----------



## MichaelD (28 November 2006)

*Re: Backtest your way to financial security.*



			
				Bobby said:
			
		

> Michael- Thanks for your reply, you said that you don't place a stop in the market, I understand why, but could you expain in your own words for others.
> 
> Like to know more of your thoughts on backtesting  " discretionary " as markets follow volume this may be a challenge.




I don't place a stop in market because backtesting shows me that systems using such a stop have more drawdown and less profit than the exact same systems with an EOD stop not placed in the market. An intraday stop will get hit by extremes of panic selling.

Backtesting discretionary is exactly the same as backtesting mechanical except you have to backtest each trade by hand. Write down the plan. Pick a time to start the backtest. Find your entries. Walk each trade through to its exit. Note the results. Repeat and repeat.


----------



## professor_frink (28 November 2006)

For anyone interested- Amibroker has discretionary back testing capabilities. When you run it, it simply allows you to put in buy,sell,short,cover entries in manually, and will punch out all the same stats as a mechanical test. Quite handy for anyone interested.


----------



## nizar (28 November 2006)

professor_frink said:
			
		

> For anyone interested- Amibroker has discretionary back testing capabilities. When you run it, it simply allows you to put in buy,sell,short,cover entries in manually, and will punch out all the same stats as a mechanical test. Quite handy for anyone interested.




Very nice!


----------



## stevo (29 November 2006)

*Re: Backtest your way to financial security.*



			
				MichaelD said:
			
		

> I don't place a stop in market because backtesting shows me that systems using such a stop have more drawdown and less profit than the exact same systems with an EOD stop not placed in the market. An intraday stop will get hit by extremes of panic selling.



This is the same approach I take - the results test better without a stop placed in the market. Only I use a weekly timeframe because I haven't found an EOD strategy that is worth the effort. This is a much more relaxed approach with lower transaction costs due to less activity (129 trades over nearly 4 years). An ideal week for me is when I don't have to do anything - it makes taking a break much easier.

Obviously not for everyone but it suits me.

Stevo


----------



## swingstar (29 November 2006)

professor_frink said:
			
		

> For anyone interested- Amibroker has discretionary back testing capabilities. When you run it, it simply allows you to put in buy,sell,short,cover entries in manually, and will punch out all the same stats as a mechanical test. Quite handy for anyone interested.




Hi PF, how do you do this (can give me some terms to search the help file)?


----------



## nizar (30 November 2006)

*Re: Backtest your way to financial security.*



			
				MichaelD said:
			
		

> Backtesting discretionary is exactly the same as backtesting mechanical except you have to backtest each trade by hand. Write down the plan. Pick a time to start the backtest. Find your entries. Walk each trade through to its exit. Note the results. Repeat and repeat.




Michael,

Isnt discretionary backtesting the same thing as paper-trading in the sense of what you will get out of it in terms of results ?


----------



## MichaelD (30 November 2006)

*Re: Backtest your way to financial security.*



			
				nizar said:
			
		

> Michael,
> 
> Isnt discretionary backtesting the same thing as paper-trading in the sense of what you will get out of it in terms of results ?



Not quite. When backtesting you can see the right hand side of the chart. This potentially introduces biases into entries and exits despite the very best of intentions.

When paper trading these biases are eliminated since you're already working with the right hand side of the chart. Paper trading also allows you to see if you've excessively curve fitted during backtesting (i.e. you've developed a plan which trades the backtest period(s) perfectly but which doesn't trade the same going forwards).


----------



## professor_frink (30 November 2006)

swingstar said:
			
		

> Hi PF, how do you do this (can give me some terms to search the help file)?



You get the code to do it off Ami's website. It's called discretionary equity, and I found it in the knowledge base section.
Here's the link 
Once you have the code, you just double click on it like you would for any other indicator,the parameters box will have the buy,sell,short,cover options. When you have gone through and done your buying and selling, you can run it through the auto analysis and it will punch out a report for you.


----------



## CanOz (8 December 2006)

Hello all. I've started to get an interest for mech systems after playing around with my stock scanner. Now this is a very simple scan, but so far when i back test it, it seems to do quite well in terms of % gains as measured by the scanner.

It goes like this: Identify a SP that opens below the 13ema, and closes above it on the last day of trading.

While also the open and close are both above the 200 sma.

Only if the volume is above the 21ma.

Surprisingly this seems to backtest really well. Now if i exited when the SP crossed back over the 13ema i might give back some profits, but i might save some too.

So my question is simply, am i on the right path to understanding mech systems and backtesting, and if i apply some risk management, what is stopping me from applying this to my trading?

Perhaps i'm better to test it for 6 months or so.

Cheers,


----------



## It's Snake Pliskin (8 December 2006)

CanOz said:
			
		

> Hello all. I've started to get an interest for mech systems after playing around with my stock scanner. Now this is a very simple scan, but so far when i back test it, it seems to do quite well in terms of % gains as measured by the scanner.
> 
> It goes like this: Identify a SP that opens below the 13ema, and closes above it on the last day of trading.
> 
> ...




As we are all trading/investing for profit/growth, your answer lies in that realisation.


----------



## nizar (9 December 2006)

CanOz said:
			
		

> Hello all. I've started to get an interest for mech systems after playing around with my stock scanner. Now this is a very simple scan, but so far when i back test it, it seems to do quite well in terms of % gains as measured by the scanner.
> 
> It goes like this: Identify a SP that opens below the 13ema, and closes above it on the last day of trading.
> 
> ...




Can,

I like it how you've mentioned exits, as most mechanical systems focus on the perfect entry.

Have you worked out when you will give back profits when the 13ema is crossed? Or is this part of the system (exits) still very much discretionary?

My entry at the moment, is a close above all-time highs, or sometimes even those with the least resistance as possible. As iv said previously, iv never intended to "day-trade" and i would prefer to take a longer term view. But sometimes when a stock gives my entry, i take a postion before the volume confirms that the breakout will be a short-term one. With stocks like MLS, and JMS, i would love them to move ~10% a week and just hold it for a few months ie. ride a longer term trend. But when they move 50-100% in 1-2days, i just say thanks, obviously that sort of move isnt sustainable.

AGM i took a position when it broke through 40c last week, 10years of resistance. This one didnt go crazy, but slow and steady is nice. Im still holding.


----------



## CanOz (9 December 2006)

nizar said:
			
		

> Can,
> 
> I like it how you've mentioned exits, as most mechanical systems focus on the perfect entry.
> 
> ...




The exits i think i will have to leave up to a certain amount of descretion for now. Until i research them a bit more, maybe two closes below the 13ema would be enough to pull the pin and not have it come back and haunt me....not sure yet. I can backtest exits as well, so i might try that.

Allot of this works well in bullish markets and i can't really back test far enough back to test it in bearish markets...but you could also reverse the signals and trade short positions that come up too i suppose. 

I like your system too, buying on the all time highs, and there are some traders in Nick Radge's book "Every Day Traders" that do something very similar, buying on a high and a rising ma.

One thing is for sure, simple seems best and retains robustness. I would like to be in a position one day to be trading this way with the majority of my capital while saving a portion for intraday if the market's really bullish. Then also develop a longer term, bear market resistant system or just take signals from one of Nick's systems to preserve capital and try to beat the banks.

All very interesting stuff, but very basic as i haven't the software to get too deep yet. I will be developing this strategy for a while i think.

Cheers,


----------



## tech/a (9 December 2006)

https://www.aussiestockforums.com/forums/showthread.php?t=5234

There is some discussion on this sister thread which maybe helpful to some.

*Sorry to say but your M/A cross over system is doomed.*
And thats without testing it through Tradesim.

Exits are where you'll make your money.
You have much more to do first before worrying about exits.

Read the stuff in the thread above from Stevo and myself should get the grey matter working.


----------



## CanOz (9 December 2006)

tech/a said:
			
		

> https://www.aussiestockforums.com/forums/showthread.php?t=5234
> 
> There is some discussion on this sister thread which maybe helpful to some.
> 
> ...




Ok teck thanks, i suggested maybe we can get my recent postings on here moved over there.

Doomed......Ahhhhh! Back to the drawing board  


Cheers,


----------



## tech/a (9 December 2006)

Can..


I coded it up and ran it for you.
Afraid its as I expected.                                                                                                  
Detailed Report                                                                                   

Trade Database Filename                                                                           
C:\TradeSimData\Test 13 day simple.trb                                                            

Simulation Summary                                                                                
Simulation Date:                           9/12/2006                                              
Simulation Time:                           2:51:39 PM                                             
Simulation Duration:                       9.91 seconds                                           

Trade Summary                                                                                     
Earliest Entry Date in the Trade Database: 2/09/1998                                              
Latest Entry Date in the Trade Database:   8/12/2006                                              
Earliest Exit Date in the Trade Database:  8/09/1998                                              
Latest Exit Date in the Trade Database:    8/12/2006                                              

Start Trade Entry Date:                    2/09/1998                                              
Stop Trade Entry Date:                     8/12/2006                                              
First Entry Date:                          2/09/1998                                              
Last Entry Date:                           20/08/2004                                             
First Exit Date:                           8/09/1998                                              
Last Exit Date:                            24/08/2004                                             

Total Trading duration:                    2183 days                                              

Profit Summary                                                                                    
Profit Status:                             *BANKRUPT* 
Starting Capital:                          $100,000.00                                            
Finishing Capital:                         -$6.11                                                 
Maximum Equity/(Date):                     $2,682.38 (13/12/1999)                                 
Minimum Equity/(Date):                     -$100,006.11 (24/08/2004)                              
Gross Trade Profit:                        $158,218.24 (158.22%)                                  
Gross Trade Loss:                          -$258,224.34 (-258.22%)                                
Total Net Profit:                          -$100,006.11 (-100.01%)                                
Average Profit per Trade:                  -$54.50                                                
Profit Factor:                             0.6127                                                 
Profit Index:                              -63.21%                                                
Total Transaction Cost:                    $110,100.00                                            
Total Slippage:                            $0.00                                                  
Daily Compound Interest Rate:              100%                                                   
Annualized Compound Interest Rate:         100%                                                   

Trade Statistics                                                                                  
Trades Processed:                          8696                                                   
Trades Taken:                              1835                                                   
Partial Trades Taken:                      0                                                      
Trades Rejected:                           5970                                                   
Winning Trades:                            341 (18.58%)                                           
Losing Trades:                             1494 (81.42%)                                          
Breakeven Trades:                          0 (0.00%)                                              

Largest Winning Trade/(Date):              $18,376.32 (8/12/1999)                                 
Largest Losing Trade/(Date):               -$2,075.00 (5/04/2000)                                 
Average Winning Trade:                     $463.98                                                
Average Losing Trade:                      -$172.84                                               
Average Win/Average Loss:                  2.6845                                                 

Trade Breakdown                            Long and Short Trades     Long Trades    Short Trades  
Normal Exit:                               1796 (97.87%)             1796 (97.87%)  0 (0.00%)     
Protective Stop:                           39 (2.13%)                39 (2.13%)     0 (0.00%)     

Total Trades:                              1835 (100.00%)            1835 (100.00%) 0 (0.00%)     

Trade Duration Statistics                  Winning and Losing Trades Winning Trades Losing Trades 
Maximum Trade Duration:                    79 (days)                 79 (days)      71 (days)     
Minimum Trade Duration:                    1 (days)                  1 (days)       1 (days)      
Average Trade Duration:                    9.00 (days)               22.48 (days)   5.92 (days)   

Consecutive Trade Statistics                                                                      
Maximum consecutive winning trades:        6                                                      
Maximum consecutive losing trades:         106                                                    
Average consecutive winning trades:        1.44                                                   
Average consecutive losing trades:         6.33                                                   

Trade Expectation Statistics                                                                      
Normalized Expectation per dollar risked:  -$0.19                                                 
Maximum Reward/Risk ratio:                 21.03                                                  
Minimum Reward/Risk ratio:                 -2.00                                                  
Average Positive Reward/Risk ratio:        $0.61                                                  
Average Negative Reward/Risk ratio:        -$0.37                                                 

Relative Drawdown                                                                                 
Maximum Dollar Drawdown/(Date):            $6,522.06 (13/04/2000)                                 
Maximum Percentage Drawdown/(Date):        100.2000% (24/08/2004)                                 

Absolute (Peak-to-Valley) Dollar Drawdown                                                         
Maximum Dollar Drawdown:                   $102,418.49 (100.0000%)                                
Capital Peak/(Date):                       $102,412.38 (13/12/1999)                               
Capital Valley/(Date):                     -$6.11 (24/08/2004)                                    

Absolute (Peak-to-Valley) Percent Drawdown                                                        
Maximum Percentage Drawdown:               100.0000% ($102,418.49)                                
Capital Peak/(Date):                       $102,412.38 (13/12/1999)                               
Capital Valley/(Date):                     -$6.11 (24/08/2004)


----------



## tech/a (9 December 2006)

*So then I altered the periodicy to weekly * and while not startling it was profitable.
The first table is the Singular test.                                                                                                     
Detailed Report                                                                                      

Trade Database Filename                                                                              
C:\TradeSimData\Test 13 day simple weekly.trb                                                        

Simulation Summary                                                                                   
Simulation Date:                              9/12/2006                                              
Simulation Time:                              2:57:31 PM                                             
Simulation Duration:                          1.73 seconds                                           

Trade Summary                                                                                        
Earliest Entry Date in the Trade Database:    4/09/1998                                              
Latest Entry Date in the Trade Database:      8/12/2006                                              
Earliest Exit Date in the Trade Database:     11/09/1998                                             
Latest Exit Date in the Trade Database:       8/12/2006                                              

Start Trade Entry Date:                       4/09/1998                                              
Stop Trade Entry Date:                        8/12/2006                                              
First Entry Date:                             4/09/1998                                              
Last Entry Date:                              8/12/2006                                              
First Exit Date:                              11/09/1998                                             
Last Exit Date:                               8/12/2006                                              

Total Trading duration:                       3017 days                                              

Profit Summary                                                                                       
Profit Status:                                *PROFITABLE     * 
Starting Capital:                             $100,000.00                                            
Finishing Capital:                            $165,407.02                                            
Maximum Equity/(Date):                        $65,407.02 (8/12/2006)                                 
Minimum Equity/(Date):                        -$22,701.66 (31/03/2000)                               
Gross Trade Profit:                           $271,037.22 (271.04%)                                  
Gross Trade Loss:                             -$205,630.20 (-205.63%)                                
Total Net Profit:                             $65,407.02 (65.41%)                                    
Average Profit per Trade:                     $122.03                                                
Profit Factor:                                1.3181                                                 
Profit Index:                                 24.13%                                                 
Total Transaction Cost:                       $32,160.00                                             
Total Slippage:                               $0.00                                                  
Daily Compound Interest Rate:                 0.0167%                                                
Annualized Compound Interest Rate:            6.2774%                                                

Trade Statistics                                                                                     
Trades Processed:                             6090                                                   
Trades Taken:                                 536                                                    
Partial Trades Taken:                         0                                                      
Trades Rejected:                              3950                                                   
Winning Trades:                               152 (28.36%)                                           
Losing Trades:                                384 (71.64%)                                           
Breakeven Trades:                             0 (0.00%)                                              

Largest Winning Trade/(Date):                 $22,974.58 (14/04/2000)                                
Largest Losing Trade/(Date):                  -$5,427.05 (16/09/2005)                                
Average Winning Trade:                        $1,783.14                                              
Average Losing Trade:                         -$535.50                                               
Average Win/Average Loss:                     3.3299                                                 

Trade Breakdown                               Long and Short Trades     Long Trades    Short Trades  
Normal Exit:                                  482 (89.93%)              482 (89.93%)   0 (0.00%)     
Protective Stop:                              42 (7.84%)                42 (7.84%)     0 (0.00%)     
Open Trade:                                   12 (2.24%)                12 (2.24%)     0 (0.00%)     

Total Trades:                                 536 (100.00%)             536 (100.00%)  0 (0.00%)     

Trade Duration Statistics                     Winning and Losing Trades Winning Trades Losing Trades 
Maximum Trade Duration:                       301 (days)                301 (days)     168 (days)    
Minimum Trade Duration:                       0 (days)                  6 (days)       0 (days)      
Average Trade Duration:                       45.94 (days)              107.74 (days)  21.48 (days)  

Consecutive Trade Statistics                                                                         
Maximum consecutive winning trades:           4                                                      
Maximum consecutive losing trades:            13                                                     
Average consecutive winning trades:           1.48                                                   
Average consecutive losing trades:            3.76                                                   

Trade Expectation Statistics                                                                         
Normalized Expectation per dollar risked:     $0.12                                                  
Maximum Reward/Risk ratio:                    23.41                                                  
Minimum Reward/Risk ratio:                    -3.90                                                  
Average Positive Reward/Risk ratio:           $1.57                                                  
Average Negative Reward/Risk ratio:           -$0.46                                                 

Relative Drawdown                                                                                    
Maximum Dollar Drawdown/(Date):               $10,849.20 (6/10/2006)                                 
Maximum Percentage Drawdown/(Date):           8.9920% (6/10/2006)                                    

Absolute (Peak-to-Valley) Dollar Drawdown                                                            
Maximum Dollar Drawdown:                      $32,987.01 (23.1000%)                                  
Capital Peak/(Date):                          $142,785.69 (1/04/2005)                                
Capital Valley/(Date):                        $109,798.68 (6/10/2006)                                

Absolute (Peak-to-Valley) Percent Drawdown                                                           
Maximum Percentage Drawdown:                  23.1000% ($32,987.01)                                  
Capital Peak/(Date):                          $142,785.69 (1/04/2005)                                
Capital Valley/(Date):                        $109,798.68 

(6/10/2006)


----------



## tech/a (9 December 2006)

*The second table is a Montecarlo report of 10000 portfolios* 

Monte Carlo Report                                                               

Trade Database Filename                                                          
C:\TradeSimData\Test 13 day simple weekly.trb                                    

Simulation Summary                                                               
Simulation Date:                                       9/12/2006                 
Simulation Time:                                       2:58:27 PM                
Simulation Duration:                                   113.57 seconds            

Trade Parameters                                                                 
Initial Capital:                                       $100,000.00               
Portfolio Limit:                                       100.00%                   
Maximum number of open positions:                      100                       
Position Size Model:                                   Fixed Percent Risk        
Percentage of capital risked per trade:                2.00%                     
Position size limit:                                   10.00%                    
Portfolio Heat:                                        100.00%                   
Pyramid profits:                                       Yes                       
Transaction cost (Trade Entry):                        $30.00                    
Transaction cost (Trade Exit):                         $30.00                    
Margin Requirement:                                    100.00%                   
Magnify Position Size(& Risk) according to Margin Req: No                        

Trade Preferences                                                                
Trading Instrument:                                    Stocks                    
Break Even Trades:                                     Process separately        
Trade Position Type:                                   Process long trades only  
Entry Order Type:                                      Default Order             
Exit Order Type:                                       Default Order             
Minimum Trade Size:                                    $0.00                     
Accept Partial Trades:                                 No                        
Volume Filter:                                         Ignore Volume Information 
Pyramid Trades:                                        No                        
Use Level Zero trades only:                            Yes                       

Simulation Stats                                                                 
Number of trade simulations:                           10000                     
Trades processed per simulation:                       6090                      
Maximum Number of Trades Executed:                     582                       
Average Number of Trades Executed:                     515                       
Minimum Number of Trades Executed:                     454                       
Standard Deviation:                                    16.76                     

Profit Stats                                                                     
Maximum Profit:                                        $613,218.49 (613.22%)     
Average Profit:                                        $156,845.06 (156.85%)     
Minimum Profit:                                        -$22,724.71 (-22.72%)     
Standard Deviation:                                    $78,483.58 (78.48%)       
*Probability of Profit:                                 99.65%                    
Probability of Loss:                                   0.35%  * 

Percent Winning Trade Stats                                                      
Maximum percentage of winning trades:                  37.76%                    
Average percentage of winning trades:                  30.81%                    
Minimum percentage of winning trades:                  22.68%                    
Standard Deviation:                                    1.79%                     

Percent Losing Trade Stats                                                       
Maximum percentage of losing trades:                   77.32%                    
Average percentage of losing Trades:                   69.19%                    
Minimum percentage of losing trades:                   62.24%                    
Standard Deviation:                                    1.79%                     

Average Relative Dollar Drawdown Stats                                           
Maximum of the Average Relative Dollar Drawdown:       $3,411.85                 
Average of the Average Relative Dollar Drawdown:       $1,681.34                 
Minimum of the Average Relative Dollar Drawdown:       $866.74                   
Standard Deviation:                                    $297.49                   

Average Relative Percent Drawdown Stats                                          
Maximum of the Average Relative Percent Drawdown:      1.8221%                   
Average of the Average Relative Percent Drawdown:      1.0840%                   
Minimum of the Average Relative Percent Drawdown:      0.6655%                   
Standard Deviation:                                    0.1374%                   

Maximum Peak-to-Valley Dollar Drawdown Stats                                     
Maximum Absolute Dollar Drawdown:                      $89,037.38                
Average Absolute Dollar Drawdown:                      $33,620.12                
Minimum Absolute Dollar Drawdown:                      $13,910.99                
Standard Deviation:                                    $9,376.44                 

Maximum Peak-to-Valley Percent Drawdown Stats                                    
Maximum Absolute Percent Drawdown:                     51.0453%                  
Average Absolute Percent Drawdown:                     19.5400%                  
Minimum Absolute Percent Drawdown:                     7.7471%                   
Standard Deviation:                                    5.2278%


----------



## nizar (9 December 2006)

From the first backtest of MA crossover:


> Maximum consecutive losing trades: 106



  

I note the 2nd backtest results (weekly) was profitable but only getting 28% winners, probably hard to handle psychologically. ie. you would only win 3 out of every 11 trades.

With this backtesting, where is the initial stop placed? and when is this adjusted? ie. to a breakeven stop or to a trailing stop?

Can you even backtest with those sorts of parameters?

Obviously in thinking that can make a huge difference to profitability.


----------



## CanOz (9 December 2006)

What was the criteria that you asked it to run? Were the losses due to mostly the exit? Sorry if i've missed something...on my way out the door to get a haircut! Can't see for the hair in my eyes.

Thank Tech, looking forward to going over this better later.

Cheers,


----------



## tech/a (9 December 2006)

Nizar.

Its not even worth fiddling with.
From the results its clear that it is whipsawed out all the time.

Just as a comparison here is T/Trader based upon 35% margin.
IE 65% leverage.

Detailed Report                                                                                    

Trade Database Filename                                                                            
C:\TradeSimData\TT Latch Master stop..trb                                                          

Simulation Summary                                                                                 
Simulation Date:                           9/12/2006                                               
Simulation Time:                           5:00:27 PM                                              
Simulation Duration:                       1.02 seconds                                            

Trade Summary                                                                                      
Earliest Entry Date in the Trade Database: 7/09/1998                                               
Latest Entry Date in the Trade Database:   2/11/2006                                               
Earliest Exit Date in the Trade Database:  28/09/1998                                              
Latest Exit Date in the Trade Database:    3/11/2006                                               

Start Trade Entry Date:                    7/09/1998                                               
Stop Trade Entry Date:                     2/11/2006                                               
First Entry Date:                          7/09/1998                                               
Last Entry Date:                           18/09/2006                                              
First Exit Date:                           28/09/1998                                              
Last Exit Date:                            3/11/2006                                               

Total Trading duration:                    2979 days                                               

Profit Summary                                                                                     
Profit Status:                             PROFITABLE                                              
Starting Capital:                          $100,000.00                                             
Finishing Capital:                         $2,175,204.97                                           
Maximum Equity/(Date):                     $2,075,204.97 (3/11/2006)                               
Minimum Equity/(Date):                     -$27,821.50 (18/04/2000)                                
Gross Trade Profit:                        $2,582,455.20 (2582.46%)                                
Gross Trade Loss:                          -$507,250.23 (-507.25%)                                 
Total Net Profit:                          $2,075,204.97 (2075.20%)                                
Average Profit per Trade:                  $6,050.16                                               
Profit Factor:                             5.0911                                                  
Profit Index:                              80.36%                                                  
Total Transaction Cost:                    $20,580.00                                              
Total Slippage:                            $0.00                                                   
Daily Compound Interest Rate:              0.1034%                                                 
Annualized Compound Interest Rate:         45.8399%                                                

Trade Statistics                                                                                   
Trades Processed:                          1902                                                    
Trades Taken:                              343                                                     
Partial Trades Taken:                      0                                                       
Trades Rejected:                           740                                                     
Winning Trades:                            118 (34.40%)                                            
Losing Trades:                             225 (65.60%)                                            
Breakeven Trades:                          0 (0.00%)                                               

Largest Winning Trade/(Date):              $334,597.39 (3/11/2006)                                 
Largest Losing Trade/(Date):               -$20,548.00 (15/05/2006)                                
Average Winning Trade:                     $21,885.21                                              
Average Losing Trade:                      -$2,254.45                                              
Average Win/Average Loss:                  9.7076                                                  

Trade Breakdown                            Long and Short Trades      Long Trades    Short Trades  
Normal Exit:                               238 (69.39%)               238 (69.39%)   0 (0.00%)     
Protective Stop:                           82 (23.91%)                82 (23.91%)    0 (0.00%)     
Open Trade:                                23 (6.71%)                 23 (6.71%)     0 (0.00%)     

Total Trades:                              343 (100.00%)              343 (100.00%)  0 (0.00%)     

Trade Duration Statistics                  Winning and Losing Trades  Winning Trades Losing Trades 
Maximum Trade Duration:                    1632 (days)                1632 (days)    222 (days)    
Minimum Trade Duration:                    1 (days)                   12 (days)      1 (days)      
Average Trade Duration:                    151.73 (days)              344.54 (days)  50.60 (days)  

Consecutive Trade Statistics                                                                       
Maximum consecutive winning trades:        20                                                      
Maximum consecutive losing trades:         13                                                      
Average consecutive winning trades:        1.79                                                    
Average consecutive losing trades:         3.41                                                    

Trade Expectation Statistics                                                                       
Normalized Expectation per dollar risked:  $1.53                                                   
Maximum Reward/Risk ratio:                 54.81                                                   
Minimum Reward/Risk ratio:                 -2.08                                                   
Average Positive Reward/Risk ratio:        $5.63                                                   
Average Negative Reward/Risk ratio:        -$0.62                                                  

Relative Drawdown                                                                                  
Maximum Dollar Drawdown/(Date):            $45,566.29 (15/08/2006)                                 
Maximum Percentage Drawdown/(Date):        14.3700% (18/04/2000)                                   

Absolute (Peak-to-Valley) Dollar Drawdown                                                          
Maximum Dollar Drawdown:                   $45,566.29 (4.1610%)                                    
Capital Peak/(Date):                       $1,095,150.64 (28/07/2006)                              
Capital Valley/(Date):                     $1,049,584.35 (15/08/2006)                              

Absolute (Peak-to-Valley) Percent Drawdown                                                         
Maximum Percentage Drawdown:               28.2400% ($28,241.50)                                   
Capital Peak/(Date):                       $100,000.00 (18000101)                                  
Capital Valley/(Date):                     $71,758.50 (18/04/2000)


----------



## tech/a (10 December 2006)

In my veiw the BIBLE for Systems developers.

*Trading Systems and Methods---Author Perry J Kaufman.
ISBN0-471-14879-2*


----------



## nizar (10 April 2007)

*Re: Backtest your way to financial security.*



MichaelD said:


> Paper trading also allows you to see if you've excessively curve fitted during backtesting (i.e. you've developed a plan which trades the backtest period(s) perfectly but which doesn't trade the same going forwards).




Great thread, just had another read.
Michael defines curve-fitting superbly.


----------



## nizar (10 April 2007)

Also some great points about how to make a system fail by stevo, tech, and rub92me, ill certainly be reading this thread again when im testing my system.

Thanks guys.


----------

