# Viable trading system statistics



## Roller_1 (15 September 2015)

Curious to know when you guys build a system, what are the stats you look for ie. returns vs drawdowns etc. What are the minimums and maximums that you would look for to make it a viable system?


----------



## VSntchr (15 September 2015)

*Re: viable trading system stats*



Roller_1 said:


> Curious to know when you guys build a system, what are the stats you look for ie. returns vs drawdowns etc. What are the minimums and maximums that you would look for to make it a viable system?




Everyone will have a different answer but some of the obvious ones:
expectancy, max win, max loss, avg win, avg loss, win %, avg trade length, max/min excursions.

Then there are things like: 
number of signals/trades, correlation with other strategies, capital efficiency etc.


----------



## Roller_1 (15 September 2015)

*Re: viable trading system stats*



VSntchr said:


> Everyone will have a different answer but some of the obvious ones:
> expectancy, max win, max loss, avg win, avg loss, win %, avg trade length, max/min excursions.
> 
> Then there are things like:
> number of signals/trades, correlation with other strategies, capital efficiency etc.




what would be the values though for yourself or your general opinion. ie. for a trend following system, win rate greater than 50%, <20% DD etc


----------



## minwa (17 September 2015)

*Re: viable trading system stats*



Roller_1 said:


> what would be the values though for yourself or your general opinion. ie. for a trend following system, win rate greater than 50%, <20% DD etc




If it's a breakout system around 30% winrate, R:R 4+. Mean reversion 40-80% R:R 1.5+. Draw down can be tailored to personal taste based on position sizing. Personally I don't like 15%+ DD but many can tolerate upto 30%.


----------



## ThirtysixD (8 October 2015)

Roller_1 said:


> Curious to know when you guys build a system, what are the stats you look for ie. returns vs drawdowns etc. What are the minimums and maximums that you would look for to make it a viable system?




Through a bull, bear and sideways market I want:
Annual returns > 15%
Drawdown < 50%
Sharpe > 1.5

Ill also remove the effect of large winners (not losers) but im really not that nit picky about statistics as long as the system is profitable and makes sense (not data mined).


----------



## sinner (8 October 2015)

Roller_1 said:


> Curious to know when you guys build a system, what are the stats you look for ie. returns vs drawdowns etc. What are the minimums and maximums that you would look for to make it a viable system?




For most systems I think it's irrelevant because the returns will largely be a result of the market regimes in which you test it.

You might think your trend following system only has a 30% maxDD but then you go and test it against the Nikkei 225 and find out it's 40. Or against the Great Depression and suddenly it's 50.

So what is the point of evaluating stuff in that perspective? There isn't one, at least my  ..

What I try to test for is how well the system captures a particular market regime. I don't need numbers to know that trend following systems are gonna eat it during choppy range or whatever. 

But I do need to know, does the Close > 200SMA capture trends better than buying the 200 day high? Or, whatever. In most cases, a simple Sharpe ratio of returns divided by volatility is good enough for me to understand.


----------



## howardbandy (9 October 2015)

Greetings --

I believe traders have a near-universal goal:  To have confidence that the signals generated by their trading system precede trades that provide reward adequate to compensate for the risk.  The key word is confidence.  The primary limitation is risk.

Every trader has a personal risk tolerance.  Mine is:  I am trading a $100,000 account and looking forward two years.  I am willing to accept a 5% risk of a 20% drawdown, measured from highest equity to date.

Every set of trades (actual, backtest, or hypothetical) defines a distribution.  Assuming the system is stationary (that the future will resemble the past), that distribution can be used to determine the risk associated with trading it for the next two years.  By adjusting position size, the risk of the trading system can be brought into alignment with the trader's risk tolerance.  That position size is called safe-f.  At safe-f, the trading system is defined to be "risk-normalized."  

The position size that normalizes risk determines the distribution of profit potential.  

All alternative uses of money can be normalized in this manner.  Trade the one that has the highest risk-normalized profit potential.

Best regards,
Howard


----------



## Gringotts Bank (9 October 2015)

howardbandy said:


> The position size that normalizes risk determines the distribution of profit potential.
> 
> All alternative uses of money can be normalized in this manner.  Trade the one that has the highest risk-normalized profit potential.
> 
> ...




Howard, could you give an outline of how dynamic position sizing works and what sort of difference it makes?

Thanks.


----------



## Roller_1 (9 October 2015)

sinner said:


> You might think your trend following system only has a 30% maxDD but then you go and test it against the Nikkei 225 and find out it's 40. Or against the Great Depression and suddenly it's 50.
> 
> So what is the point of evaluating stuff in that perspective? There isn't one, at least my  ...




That is why a system should be tested across a range of market conditions imo. for stocks the tech boom, gfc, bull markets etc so you have a better understanding of what could  happen under certain conditions.


----------



## howardbandy (9 October 2015)

Hi GB, and all --

My latest book has details and computer code.  You can read a lot of it on the book's webpage:
http://www.quantitativetechnicalanalysis.com/
And using Look Inside on the Amazon page:
http://www.amazon.com/Quantitative-...eywords=bandy+quantitative+technical+analysis

There are several videos of my recent (within the past two years) presentations that show the process in operation.  Those from the ATAA, MTA, and IFTA are good places to start:
http://www.screencast.com/t/Vli0B4oJr5
https://www.youtube.com/channel/UCcPOv0K7zMQNOkaMFpou31g

Broadly, dynamic position sizing uses Bayesian Analysis (contrast with frequentist statistical analysis) of recent trading performance to estimate the health of the system.  

The flowchart on the cover of the book, and on page 18 of the text (you can download that chapter), show the two systems that together make up a trading operation.  

The trading system is on the left.  The trading system generates buy and sell signals (but not position size) from processing the price and volume data.  

The trading management system is on the right.  The trading management system uses dynamic position sizing to analyze recent trades to determine how large a position to take on the next trade.  The goal is to increase position size when the system is working well and decrease position size when it is working poorly.  When the system is broken, the dynamic position sizing will have reduced the position size to zero before a large portion of the account has been lost.

Best,
Howard


----------



## sinner (9 October 2015)

Roller_1 said:


> That is why a system should be tested across a range of market conditions imo. for stocks the tech boom, gfc, bull markets etc so you have a better understanding of what could  happen under certain conditions.




But can you not see that the outcome is only a result of what the market did, and not any magic on the part of any system?

Which means that the returns of any system once again boils down to ones ability to predict the next market regime, or at the very least correctly classify which regime the market is currently in, and deploy an appropriate system for that regime.

I think the idea suggested by Dr Bandy is not a bad one, assuming that the calculation is undertaken continuously as new trades completed, this will provide a momentum effect to systems, as the market regime changes some systems will become safe (and can be adopted) and some will become unsafe (and can be discarded).


----------



## Gringotts Bank (9 October 2015)

howardbandy said:


> Hi GB, and all --
> 
> My latest book has details and computer code.  You can read a lot of it on the book's webpage:
> http://www.quantitativetechnicalanalysis.com/
> ...




Thanks Howard.  I might need to buy it.  I wonder if you'd mind posting some screenshots of how a system's equity curve changes with the addition of dynamic position sizing.  Just a MA cross or whatever.


----------



## tech/a (9 October 2015)

Gringotts Bank said:


> Thanks Howard.  I might need to buy it.  I wonder if you'd mind posting some screenshots of how a system's equity curve changes with the addition of dynamic position sizing.  Just a MA cross or whatever.




And if you wouldn't mind placing a few trades for me over the next 6 mths---that would be helpful.


----------



## Gringotts Bank (9 October 2015)

tech/a said:


> And if you wouldn't mind placing a few trades for me over the next 6 mths---that would be helpful.




... and some duck repellent.  Just a quick spray around the place would clean things up nicely.

Thanks Howard.


----------



## tech/a (9 October 2015)

Gringotts Bank said:


> ... and some duck repellent.  Just a quick spray around the place would clean things up nicely.
> 
> Thanks Howard.




Think about it GB.

Its minimising loss and maximising profit.
Without seeing a curve, you'd expect shallower drawdowns and steeper rises.

Some systems I know of use their equity curve as a filter.
Either turning new trades off at X downturn or exiting the whole lot at Y.
Lots of variations.

Winchester put out a duck repellent.


----------



## Gringotts Bank (9 October 2015)

tech/a said:


> Think about it GB.
> 
> Its minimising loss and maximising profit.
> Without seeing a curve, you'd expect shallower drawdowns and steeper rises.
> ...




I think for Howard selling a book, a 'before' and 'after' shot of the equity curve would be a nice selling point to see just how much difference it makes.  It looks like implementing dynamic sizing would require one to learn Python, so I'm not going to jump in without seeing the beneifts.  If it's a big difference, I'd go ahead.  If it's minimal, I wouldn't.

Also, if one's existing equity curves aren't too affected by drawdown, such an approach might even be counterproductive.  Those frequent little drawdowns of a smooth equity curve are followed by frequent little gains, and you wouldn't want to reduce position sizing just as it's ready to move back up.  You might even want to increase the size during drawdown... but I guess this would also be a possibility of his dynamic approach.


----------



## tech/a (9 October 2015)

Gringotts Bank said:


> I think for Howard selling a book, a 'before' and 'after' shot of the equity curve would be a nice selling point to see just how much difference it makes.  It looks like implementing dynamic sizing would require one to learn Python, so I'm not going to jump in without seeing the beneifts.  If it's a big difference, I'd go ahead.  If it's minimal, I wouldn't.
> 
> Also, if one's existing equity curves aren't too affected by drawdown, such an approach might even be counterproductive.  Those frequent little drawdowns of a smooth equity curve are followed by frequent little gains, and you wouldn't want to reduce position sizing just as it's ready to move back up.  You might even want to increase the size during drawdown... but I guess this would also be a possibility of his dynamic approach.




That's why you systems test.

Howards books are excellent I have them all.
Even a duck can follow them.\
I do attempt to avoid Python--'s


----------



## Trembling Hand (9 October 2015)

Gringotts Bank said:


> It looks like implementing dynamic sizing would require one to learn Python, so I'm not going to jump in without seeing the beneifts.  If it's a big difference, I'd go ahead.  If it's minimal, I wouldn't.




GB you are an interesting character. Seems you will go to any lengths not to do extra work or increase your general knowledge. I personally wonder how an approach would work in a field that requires self learning and a broad skill set? just my .


----------



## sinner (9 October 2015)

Trembling Hand said:


> GB you are an interesting character. Seems you will go to any lengths not to do extra work or increase your general knowledge. I personally wonder how an approach would work in a field that requires self learning and a broad skill set? just my .




It's also wrong, can be done in any computer tool that supports basic statistics and mathematics functions, even Excel.

I use R to do all of this stuff now.


----------



## Gringotts Bank (9 October 2015)

Trembling Hand said:


> GB you are an interesting character. Seems you will go to any lengths not to do extra work or increase your general knowledge. I personally wonder how an approach would work in a field that requires self learning and a broad skill set? just my .




Close.  I will go to any lengths to avoid _unecessary _work.  I have done far too much learning of unecessary crap in other fields outside of trading (stuff fed to me by experts, lecturers, academics) only to find out it's all bull****.  The only way I found out it's bull**** is by questioning everyhting like crazy.  That's my nature, but it can be exhausting.  You get to see who's really on the ball.  

So I'm extremely wary of time spent on things that won't be likely to yield a result.  The result doesnt have to be money, it could be something else.  Aside from that, life is extremely short and we'll all be dead before you know what's up, so I sure as hell am not going to learn just for the sake of it.  Leave that for the academics and geeks.


----------



## skyQuake (9 October 2015)

Gringotts Bank said:


> Close.  I will go to any lengths to avoid _unecessary _work.  I have done far too much learning of unecessary crap in other fields outside of trading (stuff fed to me by experts, lecturers, academics) only to find out it's all bull****.  So I'm extremely wary of time spent on things that won't be likely to yield a result.  The result doesnt have to be money, it could be something else.  Aside from that, life is extremely short and we'll all be dead before you know what's up, so I sure as hell am not going to learn just for the sake of it.  Leave that for the academics and geeks.




How will you know whats unnecessary and whats necessary? That's only in retrospect.

Charlie Munger on different mental models

...that 2hr boring online agriculture course will certainly be useful in the post apocalyptic wasteland


----------



## Gringotts Bank (9 October 2015)

skyQuake said:


> How will you know whats unnecessary and whats necessary? That's only in retrospect.
> 
> Charlie Munger on different mental models
> 
> ...that 2hr boring online agriculture course will certainly be useful in the post apocalyptic wasteland




By taking what information is offered, pulling it apart and investigating it.  The investigation might involve thought experiments, real experiments, social experiments, formal experiments, informal testing, philosophizing, reality testing and so on. Then going back to the person who offered it and query them with the most direct, honest and incisive questions you can generate.  

Most fold at this point.  They realize the game of playing 'expert' is over.  Of those who don't fold, some will be offended and decide to hit back.  That's just embarrassing - leave them alone.  And the remainder will recognize the amount of thought put into your questions and then you have something and someone to collborate with.  It's possible to cut out a heap rubbish without going to extreme amounts of work.


----------



## Roller_1 (9 October 2015)

sinner said:


> But can you not see that the outcome is only a result of what the market did, and not any magic on the part of any system?
> 
> Which means that the returns of any system once again boils down to ones ability to predict the next market regime, or at the very least correctly classify which regime the market is currently in, and deploy an appropriate system for that regime.
> .




can anyone truly predict what the market is going to do though? why not have a system that works reasonably well in most conditions and maybe an index filter to shut the system down when the market conditions don't suit the system.


----------



## Trembling Hand (9 October 2015)

Roller_1 said:


> can anyone truly predict what the market is going to do though? why not have a system that works reasonably well in most conditions *and maybe an index filter to shut the system down* when the market conditions don't suit the system.




That is a prediction about what the market is doing.


----------



## howardbandy (10 October 2015)

Roller_1 said:


> can anyone truly predict what the market is going to do though? why not have a system that works reasonably well in most conditions and maybe an index filter to shut the system down when the market conditions don't suit the system.




Greetings --

When we develop trading systems we are predicting what the market is going to do.  We are looking for signals in the historical data, writing rules that recognize them, and predicting / relying on the data continuing doing what it did. 

A very large difficulty is that systems do not work well in multiple or changing conditions.  

Envision an overly simplified system that trades on the cross of two moving averages.  If it was developed using a period when the two moving average lengths were 5 and 20, it will work well as long as the data continues to cycle at 5 and 20.  When conditions change to 8 and 29, the signals still come when the 5 and 20 cross, but they are no longer profitable.  

As you say, we can recognize that the 5 and 20 system is not working and take it offline.

Best,
Howard


----------



## Gringotts Bank (14 October 2015)

Gringotts Bank said:


> I think for Howard selling a book, a 'before' and 'after' shot of the equity curve would be a nice selling point to see just how much difference it makes.  It looks like implementing dynamic sizing would require one to learn Python, so I'm not going to jump in without seeing the beneifts.




Howard probably rolled is eyes at this comment of mine, since it was contained in the screencast link he gave.    A really nice before and after example of an equity curve is given.  And the Python stuff is for machine learning, not the dynamic position sizing, afaik.


----------



## 11bblandin (19 November 2015)

Greetings all 

First off, brand new to the forum and can already tell there's loads of good info on here and some top notch contributors. I recently finished reading Dr. Bandy's latest book, and the universal objective function described therein is probably the most logical I've come across. However, it isn't readily calculated in most off-the-shelf trading system development platforms. 

Back to the original topic of the thread.. My current objective function of choice combines the expectancy of the system, the dispersion of trade returns, and the number of completed trades in a 1-year period. My systems are optimized according to this statistic, and then the out-of-sample results are fed into a Monte Carlo analysis to determine how much profit potential there is given my risk level. I use ulcer index rather than Max Drawdown, as it takes into account magnitude, frequency, and length of drawdowns.


----------



## howardbandy (24 November 2015)

Greetings --

The limitations of the development platform sometimes restrict the development.  I recommend that the developer have an objective function that ranks alternative systems into the same order he or she would rank them if the reports from the individual runs were spread out on the office floor and judges subjectively.  

This does several very positive things.
1.  The dissonance between the intent of the trader and the implementation of the trading system is removed.  When alternatives are ranked according to your preference, you already know you agree that one ranked as best is one you subjectively prefer.  
2.  Whenever automatic selection of system alternatives takes place, the alternative that is ranked at the top of the list is the one used.  At that time, there is no opportunity to consider alternatives that are lower in the list.  The developer / trader must have confidence that the alternative ranked best is in fact the best -- or at least have confidence that the alternative ranked as best is acceptable.

If the development platform restricts the developer to a set of objective functions that have been pre-programmed, the opportunity to create and automatically use a custom objective function is removed.  

Note that the most common built-in objective function is the value of the trading account at the end of the simulation period.  That distorts performance for several reasons.  
1.  Using "net profit", or an equivalent, often selects a system that performs well in-sample but poorly out-of-sample.  
2.  It encourages use of compounding and inclusion of position sizing within the model, which should be avoided.
3.  It rewards lucky trade sequences -- that cannot be expected to reoccur in the future.

Developing systems that trade profitably require that the distribution of trades in the future is similar to the distribution of trades in the out-of-sample test period.  While the distribution must be similar, we cannot assume that the sequence will be identical.  I recommend the following, including using an objective function that rewards these features:
1.  Base analysis of test runs on individual trades -- just percentage or points gained or lost per trade -- no compounding, no position sizing.
2.  Trade frequently.
3.  Trade accurately.
4.  Hold a short period.
5.  Avoid large losses.

Following development runs, perform an analysis similar to that I describe in my Modeling book or in my YouTube presentation:
https://www.youtube.com/watch?v=Vw7mseQ_Tmc

The presentation on stationarity might be valuable as well:
https://www.youtube.com/watch?v=iBhrZKErJ6A

Best regards,
Howard


----------

