# Howard Bandy "Quantitative Trading Systems" & systems design Q&A



## tech/a

Howard.

Just recieved your book a couple of days ago.
Firstly send my Thanks to your packaging staff.
Tremendous job.

From a brief look at your book, there is a great deal of effort in it.
I lookforward to having a read.
With the influence of amibroker I can see I'll have to yet again add to my software.

Thought I'd start this thread in the hope that questions on systems testing either related to your book or in general can find a home here,and that you would be kind enough from time to time to drop by and make comment.

Howard.
There is very little on who you are.
In which University were you a Dean?

Thanks for your input so far and hopefully into the future. I like the idea that the book is purely mathamatically based.


----------



## nizar

Well i second that.

Packaging was great.
I received mine about 1 week after payment was made.

Having read the first 50 pages or so and i can say its very well written.
Looking forward to reading the rest.


----------



## howardbandy

Hi Tech and Nizar --

Thanks for the kind words about the book.

About my academic background --

I have degrees in math, physics, engineering, and computer science.  I did a master's thesis on artificial intelligence, a master's thesis on statistics, and doctoral work that combined modeling, simulation, and communications in computer networks.  My Ph. D. is in Computer Science from Case Western Reserve University in Cleveland, Ohio.

I founded the Computer Science program at the University of Nebraska's Omaha campus, was Professor of Computer Science at the University of Wisconsin's Platteville campus, and was Professor of Mathematics and Computer Science and Dean at Pacific Lutheran University in Tacoma, Washington.  

About the book --

As I say in the preface to Quantitative Trading Systems, I have often been frustrated by books and articles that purport to describe fully functional trading systems, only to be sadly disappointed.  One of the goals of my book is to be as complete and precise as possible in describing the concepts.  Everyone who reads my book should be able to replicate everything I have done.  

And, to the extend possible when working with financial time series, the methods used are based on sound principles of modeling, simulation, and statistical analysis.  

About AmiBroker --

Throughout many years I have worked with many trading systems and trading system development platforms, including some that were proprietary to trading companies and others I had programmed myself.  When I came across AmiBroker, I knew I had found the best platform I would ever be able to afford.  Tomasz Janeczko has done a masterful job of designing and programming AmiBroker.  It is complete, fast, and very affordable.  He is continuing to expand it, particularly in areas that I think are important -- tools for validating systems and tools for automated trading.

Thanks,
Howard


----------



## Chorlton

Hi Howard,

I've also just received your book, and would like to echo the comments of both Tech and Nizar.

The book was well packaged and I received it within a very short period of time of placing my order!!!!   

Flicking through it briefly, it seems to be well written, without unwanted padding and consequently I look forward to dedicating some time to reading it without other distractions....

All the best,

Chorlton


----------



## nizar

Howard,

I must say, that newsletter scam analogy is your book was gold!

I think I might even use it one day. LOL


----------



## Chorlton

Hello Howard,

Just re-reading the chapter on Walk-Through Testing.

On page 272, you compare the straightness of the Out-of-Sample Equity Curve. Even though this is probably an "excel" question, is it possible to explain how to do this?

In additional what importance do you place on measuring the straightness on the Equity Curve in this manner compared to say simply looking at the overall MaxDD on the system?


Thanks in advance,

Chorlton


----------



## howardbandy

Hi Chorlton --

The latest AmiBroker (V 5.05 and later) has built-in walk forward testing, which saves me hours every day.

It is unusual for the out-of-sample equity curve to be as steep or as smooth as the in-sample equity curve.  If you find one that is, and you have not cheated in the testing and validation phases, your trading system is a potential gold mine.

There are several metrics built-in to AmiBroker that measure both the steepness of the equity curve and the smoothness of the equity curve.  Try KRatio, RRR, CAR/MDD, RAR/MDD, or Ulcer Performance Index.  Any of these can be used as the metric by which the Best alternative is chosen during the walk forward process.  Or -- you can design and program your own.

Thanks,
Howard


----------



## grantk

howardbandy said:


> Hi Chorlton --
> 
> The latest AmiBroker (V 5.05 and later) has built-in walk forward testing, which saves me hours every day.




Hi Howard

I read this in the release notes for the latest AB version, however I don't understand it's use.  Perhaps I have the walk-forward concept mixed up - to me this means testing your formula which has been optimized on in-sample data, on out-of-sample data.  So in AB if I have created and optimized my formula on data from 2006-2007, walk forward testing would be to run the formula on data from 2007-2008 for example.  This currently takes two seconds to implement (just changing the date).

What am I missing in walk forward testing that AB now provides?

Cheers.


----------



## howardbandy

Hi Grant --

I think your understanding is correct.  The advantage of using the automated walk forward is that it makes the process completely objective and you get to see several in-sample to out-of-sample transitions with one click of the mouse.

1.  Define an objective function that fits your trading style.
2.  Design a trading system.
3.  Fool around with the system and find out how long the in-sample period should be for it to learn the signal without becoming overfit to the noise.

When all of that is done, and your preliminary results indicate that the system looks promising, perform the walk forward tests.

Say the in-sample period the system likes is 2 years, and you want to test the system on a ticker that has data beginning 1/1/2000.

In the Settings > Walk Forward:

Set up the walk forward with the in-sample period beginning 1/1/2000 and with a length of 2 years.  Set the Last date to 1/1/2008.

Set the out-of-sample period to anything reasonable -- say 1 year.  
(The length of the out-of-sample period is used for two things--
1.  It is the period of time between re-optimizations.
2.  It is the period of time reported for the out-of-sample results.)

I recommend Not using Anchored, which has the in-sample initial period stay fixed and the in-sample length grows longer with each step, but rather keep the in-sample length constant.

The window at the bottom will show you what dates it plans to use.  If you have selected Easy Mode, all the IS and OOS dates will be properly set.  If you selected Advanced Mode, check this window to be sure the dates are consistent. 

On the Automatic Analysis window, pull down the Optimize menu and click Walk Forward.

The walk forward process will make 7 steps -- each with a 2 year in-sample and 1 year out-of-sample.  The first IS period is 1/1/2000 to 1/1/2002.  The final IS period is 1/1/2006 to 1/1/2008.  The first OOS period is 1/1/2002 to 1/1/2003.  The final OOS period is 1/1/2008 to 4/7/2008 -- the final OOS period is whatever data is available for 2008.

AmiBroker will give a report and show the charts for the in-sample and out-of-sample results.  If the out-of-sample results look good, you have confidence that your system is recognizing the patterns you hoped it would, that those patterns persist beyond the in-sample period, and that they are profitably tradable for one year beyond.

What you have gained by doing this is seeing 7 transitions from in-sample testing to out-of-sample trading, all without any subjective judgment.  This should raise your level of confidence that the next time you reoptimize and begin trading with the latest values, you have a good idea of what to expect in the actual trades.

As always, there are no guarantees.  The best we can hope for is a high level of confidence.  And we get that confidence by observing the out-of-sample performance following each in-sample period.

Thanks,
Howard


----------



## glenn_r

Hi Howard,

Firstly I appreciate your effort in publishing QTS it is a great book and it has helped me a lot to get a handle on system testing using AB.

I have been looking at a few mean reversions systems to trade the ASX20, as an example I ran your "Stoch of RSI" system published in QTS through the walk forward optimiser function in AB on the ASX20 and it produced the following report.

I was wondering if you could offer your comments on the results, firstly to see if  I have the process correct and what should I be looking for, as I'm currently looking for a good profit factor, low draw down and around a 50/50 win/loss ratio.

If you have any comments or pointers on using the walk forward optimiser please pass them on.

I'm also testing your K-Score reversion system (as published in Active Trader) and a value index reversion system, both are also looking promising on early results.

Cheers
Glenn


----------



## Chorlton

howardbandy said:


> Hi Chorlton --
> 
> The latest AmiBroker (V 5.05 and later) has built-in walk forward testing, which saves me hours every day.
> 
> It is unusual for the out-of-sample equity curve to be as steep or as smooth as the in-sample equity curve.  If you find one that is, and you have not cheated in the testing and validation phases, your trading system is a potential gold mine.
> 
> There are several metrics built-in to AmiBroker that measure both the steepness of the equity curve and the smoothness of the equity curve.  Try KRatio, RRR, CAR/MDD, RAR/MDD, or Ulcer Performance Index.  Any of these can be used as the metric by which the Best alternative is chosen during the walk forward process.  Or -- you can design and program your own.
> 
> Thanks,
> Howard




Thanks for the reply Howard. I'll take a closer look at the metrics your mentioned.


----------



## josh_in_a_box

Just recieved the book today, cant wait till I get into it, 2 more chapters on my current book to go!


----------



## howardbandy

Hi Glenn --

First a caution to everyone -- the systems in my book, Quantitative Trading Systems, and in my articles and workshops, including those in Active Trader Magazine, are meant to be educational.  Use them to get ideas or as starting points.  When you apply them to tickers of your own choice and evaluate them using objective functions of your own choosing, then you can develop them into full trading systems of your own.

In my experience, it is easier to validate systems that have shorter holding periods and more frequent trades than systems that have longer holding periods and infrequent trades.  So many of my examples will trade actively.

A comment on the walk forward results posted.  I note a 4 year in-sample period and 1 year out-of-sample.  The latest two out-of-sample periods are not profitable, so I would not expect 2008 to be profitable.  I note exposure is very high.  Is this a single ticker traded long and short, or a portfolio that almost always has something to trade?  If it is a single ticker, separate the long and short results and see how they look -- you might want too different systems for the two different directions.  If it is a portfolio, try tightening the portfolio by allowing few simultaneous positions -- that will take trades only in those tickers with higher positionscore values.  Or try different positionscore computations.

Thanks,
Howard


----------



## glenn_r

Hi Howard

Thanks for you comments, I only used your "Stoch of RSI" as an example and it was setup for long trades only, the ASX20 is a portfolio of the top 20 stocks on the ASX.

My testing so far on the other systems that I'm working on has guided me to the fact that you need separate system's to trade the long and short sides, plus there are so many variables that it can become hard to maintain my aim, which is to create a simple, robust and profitable system.

Glenn


----------



## chops_a_must

Finished the book over the weekend Howard. Fantastic really. A couple of typos,  but nothing compared to other publications in the field I've read.

Thought the explanations petered out towards the end, but maybe that's because the issues no longer seemed as complex, and I didn't need to read as closely. 

Would loved to have seen another chapter devoted completely towards position sizing, and a longer section explaining why stops are bad for systems. But, that's probably because I like stirring and stirrers. 

I also can't see a superficial reason why you shouldn't test an entire data set if you were looking purely for statistics (even if that may be used to develop a trading system off). Given you may not develop an acceptable system, before you run out of "purely" out of sample data periods, I can't see what difference that makes. Though I do see the need to test in out of sample data.

Also, if you were looking at coding a previously discretionary strategy, and testing over a period you actually had traded in a similar method, personally I would incorporate that as in sample data (or at least try to).

I think I have some kind of grip on the code now as well. I'll just "hatchet" the code from your book, and from my indicators etc. to get things working. But at least now, thanks to your book, I have the confidence to do so. 

Cheers.


----------



## Kauri

chops_a_must said:


> Would loved to have seen another chapter devoted completely towards position sizing, and a longer section explaining why stops are bad for systems. But, that's probably because I like stirring and stirrers.
> 
> 
> Cheers.



 Hi Chops,
             I haven't read the book and may be missing something, but if you don't have an initial stop at least then what do you base your position sizing/risk on??
Cheers
...........Kauri


----------



## tech/a

While stop losses maybe detrimental to the very best possible return you could achieve from a system.

*They serve very important roles.*
They will limit your chances of ruin.
They impart a form of discipline.
For discretionary traders they tell you where your CURRENT analysis is wrong.
They guide you to position sizing.
They give you an opportunity to determine Expectancy.(Albeit in that single trade).
They often save you from being trapped in a trade ranging between your entry and below.
They give more opportunity for maximum use of your capital.
They increase trade frequency---needed for short term methods.


----------



## nizar

Kauri said:


> Hi Chops,
> I haven't read the book and may be missing something, but if you don't have an initial stop at least then what do you base your position sizing/risk on??
> Cheers
> ...........Kauri




There are other ways to determine position sizing.
eg. Fixed dollar amounts (10k parcels out of 100k capital).

In my testing, I went through several initial stops and none of them improved CAR/max.DD.

So in the end I made the initial stop = the trailing stop because I wanted to use a fixed percent risk model for position sizing.


----------



## howardbandy

tech/a said:


> While stop losses maybe detrimental to the very best possible return you could achieve from a system.
> 
> *They serve very important roles.*
> They will limit your chances of ruin.
> They impart a form of discipline.
> For discretionary traders they tell you where your CURRENT analysis is wrong.
> They guide you to position sizing.
> They give you an opportunity to determine Expectancy.(Albeit in that single trade).
> They often save you from being trapped in a trade ranging between your entry and below.
> They give more opportunity for maximum use of your capital.
> They increase trade frequency---needed for short term methods.




Hi Tech, and all --

I may not have been clear enough in my comments about stops.  My concern is with systems that regularly exit by the price falling (for long positions) enough to hit a maximum loss stop without having first triggered some other exit.  

My recommendation is that exits come primarily from signal logic.  In the case of systems that hold only a short period of time, timed exits (two days and out) and profit targets (two percent and out) can also work well.  In the case of systems that hold longer and are looking for a trend, then trailing stops (such as parabolic or chandelier) work well, while profit targets and timed exits work poorly.  I do not consider these trailing stops to be stops, but rather trailing exits.  The last resort is the maximum loss stop ($2000 loss and out) -- this is the one I am referring to.  A system that regularly exits by hitting the $2000 loss stop is poorly designed.  

I agree that knowing the maximum adverse excursion (MAE) is essential in analyzing risk, establishing position size, etc.  The MAE calculations come from examining all trades, including those made through signals, timed exits, profit targets, trailing stops, and maximum loss stops -- not just the maximum loss stop.

Sorry for any confusion.

Thanks,
Howard


----------



## howardbandy

chops_a_must said:


> Finished the book over the weekend Howard. Fantastic really. A couple of typos,  but nothing compared to other publications in the field I've read.
> 
> Would loved to have seen another chapter devoted completely towards position sizing, and a longer section explaining why stops are bad for systems. But, that's probably because I like stirring and stirrers.
> 
> 
> Cheers.




Hi Chops, and all --

Thanks for the kind words.

There is an "errata" page on the book's web site that has the known list of typos.  If you have found others, please let me know and I'll bring that page up to date.

I did have plans to include more on risk analysis, position sizing, portfolio management, walk forward analysis, Monte Carlo analysis, and related topics.  I ran out of time and pages.  And AmiBroker was and is evolving -- as we see with the recent release of Version 5 which includes automated walk forward.

There is still time to write about those topics -- either in a new book, or a revised edition of Quantitative Trading Systems after Introduction to AmiBroker is finished.

Thanks,
Howard


----------



## howardbandy

Greetings all --

Some of you may be interested in reading the PowerPoint presentation of a talk I will be giving in Irvine, California, next Wednesday, May 7.  "How to Build an Effective Trading System."

Download it here:
http://www.blueowlpress.com/NAAIM.html

I'll be happy to receive constructive comments.  And I will discuss the points -- but probably not until after returning from the conference.

Thanks,
Howard


----------



## skittlebup

I ordered my copy on Thursday and it arrived the following Wednesday - awesome, as they say.

Howard, you note that Turtle Trading or Donchian channels are no longer successful as their use has now plugged the market inefficiency they exploited. Later you note that a particular system may no longer work because _another system has 'plugged' an inefficiency.

The market surely neither knows nor cares what method the individual trader is using, and one sees and reads daily examples of 'irrational' market reaction to either breaking news, company reports or dividend announcements. Data charts of today appear to look identical to the charts printed in Schabacker or in Edwards and Magee - surely the same methods will continue to work - with the proviso that the trading opportunity will be much shorter because of computerised analysis??

On a different tack, your bibliography does not refer to Stridsman's book on Trading Systems and Money Management, which I found enormously illuminating when I first read it. Do you know it??

I am almost halfway through your text - very practical. And it is particularly helpful that you have posted the afl material on your website, thus eliminating the need for the poor reader to transcibe all the formulae into AmiBroker!

Your book is going to be one of my basic texts - thanks again!_


----------



## howardbandy

Hi Skittle --

Thanks for the kind words.

I agree with you (if I understood you correctly) that systems that have failed when applied to longer holding periods may work for shorter holding periods.

And, you are correct in interpreting my thoughts that it does not matter whether it is your own trading of a system, or someone else trading of a system that is based on recognizing and removing the same inefficiency as yours -- the more either of those systems is traded profitably, the more difficult it will be for them to continue to be profitable.

Thanks for reminding me that I should reread Stridsman's work.

Thanks,
Howard


----------



## kesso

Hi, I am trying to build a system that I am comfortable with, no luck so far, but I am slowly learning. With walk forwarding I sort of understand the concepts bit not all of Amibroker such as CAS/MDD etc. What has me is what is No. after Begin and End. In all of the OOS it is 1. Maybe Howards new book will help. Any comments on the report would also be welcomed. Thanks


----------



## howardbandy

kesso said:


> Hi, I am trying to build a system that I am comfortable with, no luck so far, but I am slowly learning. With walk forwarding I sort of understand the concepts bit not all of Amibroker such as CAS/MDD etc. What has me is what is No. after Begin and End. In all of the OOS it is 1. Maybe Howards new book will help. Any comments on the report would also be welcomed. Thanks




Hi Kesso --

There seem to be a couple of questions in your posting.

1.  Is it CAR/MDD?  CAR is Compound Annual Return.  MDD is Maximum DrawDown.  CAR/MDD is a good choice for many people to use as an objective function / fitness function / optimization target.  It rewards equity growth while penalizing drawdown.  Other good built-in metrics to use are RAR/MDD, RRR, Ulcer Performance Index, KRatio (which is almost the same as RRR).

2.  The dates?  Are you referring to the dates to enter in the walk forward setup?  To get to that screen, use the Analysis menu, select Automatic Analysis, click the Settings button, and the Walk Forward tab.

If you are just starting, click Easy Mode EOD if you are using end-of-day data.  Click Easy Mode Intraday if you are using bars shorter in time duration than a full trading day.
Start is the first day of the first in-sample period.
End is the last day of the first in-sample period.
Last is the last day of the last in-sample period.  Or you can click Use today.
Step, and the pull down menu to its right, let you select the length of time between reoptimization periods. This is also the length of the out-of-sample period.
Now look down at the table that lists the dates that AmiBroker will use for each in-sample and out-of-sample period. You can see the organization.  If you change Start, End, Last, or Step, you will see changes in the table.
Pick an Optimization Target from the pull-down menu.  I recommend one of CAR/MDD, RAR/MDD, or RRR if you are uncertain.  Those of you who have read my book, Quantitative Trading Systems, know how important the choice of objective function is, and that that choice should be made before serious testing begins.
Click OK to leave the Setup.
Use the pull-down menu next to the Optimize button and select Walk Forward Optimization.  The walk forward runs will start.

For each of the steps, there will be two lines in the report.
The first line shows the results for the in-sample period, using the best set of parameter values when all of the runs are ranked using the objective function you chose.
The second line shows the results for the out-of-sample period that immediately follows that in-sample period.  The same set of parameter values is used as in the line above it.

Incidentally, this is well explained, including screen capture images and worked out examples in my new book, Introduction to AmiBroker. 

Hope this helps,
Howard


----------



## CamKawa

Hi Howard,

I'm enjoying reading your book and think its fantastic being able to contact an author of book through a forum.

I'm wondering if you can help me. When I run Figure 9.1 MA Cross System on ASX code GOLD, the equity line rises from July 2005, when a buy signal is issued, right up to the sell signal in August 2006. However, if you look at period from about September 2007 (the green vertical line) to May 2008 (the red vertical line) the equity line remains flat as the price of GOLD rises. Any ideas why that may be? I'm using AmiBroker 5.18.

Cheers,

CamKawa


----------



## howardbandy

CamKawa said:


> Hi Howard,
> 
> I'm enjoying reading your book and think its fantastic being able to contact an author of book through a forum.
> 
> I'm wondering if you can help me. When I run Figure 9.1 MA Cross System on ASX code GOLD, the equity line rises from July 2005, when a buy signal is issued, right up to the sell signal in August 2006. However, if you look at period from about September 2007 (the green vertical line) to May 2008 (the red vertical line) the equity line remains flat as the price of GOLD rises. Any ideas why that may be? I'm using AmiBroker 5.18.
> 
> Cheers,
> 
> CamKawa




Hi CamKawa --

I can't tell from this distance.    

If your program has an ApplyStop that was triggered, that would cause the long position to exit but would not give an arrow on the chart.

If that is not it, either post the code here or e-mail it to me in a private message and I'll look at it.  

I'll be out of the the office for a few days.  I am the opening speaker at a conference of the NAAIM (National Association of Active Investment Managers) in Atlanta, Georgia.  You can download the PowerPoint of my talk.  Go to www.blueowlpress.com and click on the NAAIM link.

Thanks,
Howard


----------



## CamKawa

howardbandy said:


> If your program has an ApplyStop that was triggered, that would cause the long position to exit but would not give an arrow on the chart.



That's a good thought but I'm not using any stops.



howardbandy said:


> If that is not it, either post the code here or e-mail it to me in a private message and I'll look at it.



The code I'm using is a copy and paste straight out of your book with no modifications. Figure 9.1, MACrossSystem.afl, page 128.

// MACrossSystem.afl
//
// The classic Moving Average crossover
// using simple Moving Averages
//
// Set up the lengths for the moving averages
Length1 = Param("Length1",6,1,81,2);
Length2 = Param("Length2",35,2,200,2);

// The Moving Average calculations
MA1 = MA(*C*,Length1);
MA2 = MA(*C*,Length2);

// The Buy and Sell logic
// Buy when MA1 crosses from below MA2 to above MA2.
*Buy* = Cross(MA1,MA2);
*Sell* = Cross(MA2,MA1);

// Compute the equity for the single ticker
e = Equity();
Maxe = LastValue(Highest(e));
Plot( *Close*, "Price", *colorBlack*, *styleCandle* );

// Plot the MA lines.
Plot(MA1,"MA1",*colorGreen*,*styleLine*);
Plot(MA2,"MA2",*colorBlue*,*styleLine*);

// Plot the Buy and Sell arrows.
shape = *Buy* * *shapeUpArrow* + *Sell* * *shapeDownArrow*;
PlotShapes(shape, IIf(*Buy*,*colorGreen*,*colorRed*), 0, IIf(*Buy*,*Low*,*High*));

// Plot the equity curve
Plot(e,"Equity",*colorBlue*,*styleLine*|*styleOwnScale*,0,Maxe);
Plot(10000,"",*colorBlue*,*styleLine*|*styleOwnScale*,0,Maxe);
*GraphXSpace* = 5;

//Figure 9.1 MA Cross System

I copied and pasted this into the formula editor, saved it, then dragged and dropped it straight onto the chart.



howardbandy said:


> You can download the PowerPoint of my talk. Go to www.blueowlpress.com and click on the NAAIM link.



Thanks.


----------



## howardbandy

CamKawa said:


> That's a good thought but I'm not using any stops.
> 
> 
> Thanks.




Hi CamKawa --

Keep reading from page 128 on.  

The code is an example of the traditional wisdom about buying when the fast moving average is above the slow moving average is usually wrong.  

With almost any ticker, this system with these parameters loses money.  The equity line stops moving because the equity dropped too low to take any more trades.  

Look at page 131 where you change the PARAM to OPTIMIZE.  The values that it finds to be somewhat profitable have the relative speed of the two moving averages reversed.  Buy when the fast moving average crosses from above to below the slow moving average.  

In other words, a moving average crossover system is often best used as a mean reversion system, rather than as a trend following system.

Thanks,
Howard


----------



## bingk6

Hi Howard,

In your book, you mentioned several potential candidates for the objective function using the existing AB metrics. My question relate to the statistical measure of the smoothness and/or choppiness of the equity curve when its turned parabolic. After all, even for a fairly modest system that returns 5-10%pa will turn parabolic if you give it enough time purely just from compounding alone.

Howard, in your opinion, are there any metrics (AB provided or otherwise) which are more suited to the evaluation of a parabolic equity curve than others ??


----------



## howardbandy

Hi Bing --

Using percentage changes rather than price changes -- that is, using a log scale rather than a linear scale -- changes the parabolic equity line to a straight line.

Alternatively, do not compound the equity.  Set the amount committed to each trade to a fixed dollar amount rather than a percentage amount.  The resulting equity line will be linear, not parabolic.

But, for most testing, even a parabolic curve is linear in the region being examined.

For still better control, at greater effort, process the trade results yourself.  You may be able to just load the ~~~Equity and process it, or you may have to use Custom Backtester.

Thanks,
Howard


----------



## bingk6

howardbandy said:


> Hi Bing --
> 
> Using percentage changes rather than price changes -- that is, using a log scale rather than a linear scale -- changes the parabolic equity line to a straight line.
> 
> Alternatively, do not compound the equity.  Set the amount committed to each trade to a fixed dollar amount rather than a percentage amount.  The resulting equity line will be linear, not parabolic.
> 
> But, for most testing, even a parabolic curve is linear in the region being examined.
> 
> For still better control, at greater effort, process the trade results yourself.  You may be able to just load the ~~~Equity and process it, or you may have to use Custom Backtester.
> 
> Thanks,
> Howard




Many thanks Howard, I'll try out some of your suggestions.


----------



## howardbandy

Greetings all --

We are in the process of reorganizing and freshening our websites. There will probably be periods when some of the pages will be unavailable. To ease your inconvenience, both of our books, "Quantitative Trading Systems" and "Introduction to AmiBroker" have their prices reduced by $10.00 for a short time.

Thanks,
Howard
www.quantitativetradingsystems.com
www.introductiontoamibroker.com


----------



## CamKawa

Howard, have you got any plans to write another book? If so what can you tell me about it?


----------



## howardbandy

Greetings --

Yes, I am writing now.  The title will be Advanced AmiBroker.  It will focus on portfolio management, risk assessment, position sizing, and similar topics.  Implementation code will be AmiBroker.  Some advanced programming concepts in AmiBroker, such as Custom BackTester, will be covered.  The contents are still a little fluid.  You can see the details as they are posted on hte book's website:
www.advancedamibroker.com

Thanks,
Howard


----------



## howardbandy

howardbandy said:


> Greetings all --
> 
> We are in the process of reorganizing and freshening our websites. There will probably be periods when some of the pages will be unavailable. To ease your inconvenience, both of our books, "Quantitative Trading Systems" and "Introduction to AmiBroker" have their prices reduced by $10.00 for a short time.
> 
> Thanks,
> Howard
> www.quantitativetradingsystems.com
> www.introductiontoamibroker.com




Greetings --

Quantitative Trading Systems -- the book -- has been updated to reflect additions that have been made to AmiBroker since the book's publication.

The information in QTS related to the design, testing, and validation of trading systems has not changed. But, since the publication of QTS in April 2007, some significant and powerful features and tools have been added to AmiBroker. This addendum brings the Quantitative Trading Systems book up to date with respect to those features.

Specifically, the ones addressed in the addendum are:
1. The Mersenne Twister random number function.
2. Automatic walk forward testing.
3. Non-exhaustive optimization.

The errata file has also been brought up to date.

Anyone who wishes to can download a copy of the addendum. It is a 10 page long, 1.3 MB, pdf file. It can be found at:
HTTP://www.quantitativetradingsystems.com/book.html
Click on the link labeled: Addendum and Errata

Beginning today, all orders for the QTS book will have a printed copy of this file included with the book.

The websites for Blue Owl Press, Quantitative Trading Systems, Introduction to AmiBroker, and Advanced AmiBroker have been rewritten and reposted. When we started refreshing the web sites a few weeks ago, we put both the Introduction to AmiBroker book and the Quantitative Trading Systems book on sale at $10 off. Many of you have taken advantage of that sale -- thank you. That sale will end on Monday, June 8.

There may still be some bugs in the web sites. If you find any, let me know at howard at blueowlpress dot com.

Thanks,
Howard


----------



## chesl73

I've started to look at backtesting some basic ideas in Amibroker. However, I've no real idea what I should be looking for in all the results, which ratios or values in the reports are more important than others (and what the ratios actually mean), which one's I should be paying particular attention to and also what 'typical' values I should hope to be getting for a system that's considered to be 'reasonable'. 
Would Howard's book address my issues above? Or maybe his new 'Introduction to Amibroker' might be more appropriate? What do people think who have read the book(s)? (Or Howard if he's around).
Thanks


----------



## howardbandy

Hi Chesl --

Have you read Quantitative Trading Systems?  It explains in detail the importance of selecting an objective function that fits your trading.  Once you have done that, decide what to trade, when to trade, what kinds of orders to use.  Then work on entries and exits.  After that you will be ready to do some testing, backtesting, optimization, and validation.  

The process starts with the objective function.

There is a little in Introduction to AmiBroker about trading system design, but much more in Quantitative Trading Systems.

Thanks,
Howard


----------



## chesl73

Thanks Howard. Expect an order for the book. 
In fact, I believe you're in Melbourne next week.
I'm not going to the workshops/conference but do you have your
books available for sale in a lunch break or something? It would be
good to pop over and get it and save on postage.


----------



## howardbandy

Hi Chesl --

I am in Melbourne.  The ATAA Conference wrapped up today, and I am catching up on emails.  I did not see your posting in time to respond before the conference started.  We did have books for sale at the conference, and they have all sold out.  I hope you were able to stop in and pick up a copy.  If not, the best way to get a copy is order it from the book's website and we will ship it when we return to the US about November 13.

Thanks,
Howard


----------



## bullbear39

howardbandy said:


> ...He is continuing to expand it, particularly in areas that I think are important -- tools for validating systems and tools for automated trading.




Howard,
Your obviously the man to seek for guidance on this. Is the automated trading available yet in AB? I am new to it (5.30.4) but I can't see it anywhere yet. 

What I want to do is backtest my systems - which I can do now - but then once confirmed with a positive equity curve, switch it over to automated trading through my IB account. Do you know if this is possible now with AB (a plugin maybe?) or would NinjaTrader be better for this as it already has automated trading I believe?
Thanks very much.


----------



## howardbandy

Hi BullBear --

There are interfaces between AmiBroker and brokerages.  Most of the work has been done with Interactive Brokers.  There are several AmiBroker forums on Yahoo -- one of them is Automated Trading.  Here is a link to that forum:
http://finance.groups.yahoo.com/group/AmiBroker-at/

I have been personally involved with systems that trade automatically and recommend that automatic trading is only practical when the trader has a large enough organization to use it safely.  That means duplicate everything -- computers, data feeds, power sources, communications facilities, etc -- staffing so that there is a person at the terminal at all times (we had two traders around the clock), and a good enough relationship with the brokerage firm that they will break trades that the automated system entered that should not have been entered.  If a system is trading automatically, it cannot be safely left alone for even five minutes.  

Thanks for listening,
Howard


----------



## bullbear39

howardbandy said:


> ...If a system is trading automatically, it cannot be safely left alone for even five minutes.




Hi Howard,

Well that's rained on my parade... but then I may have been confusing the term 'mechanical trading system' with 'automated trading system'. Would I be correct in thinking you support the former not the latter? As in, the mechanical system would unemotionally find the trade for you based on set criteria - but it won't go ahead and place the trade. That would need to be executed by a manual process via broker or platform. 

Most of us don't have staff/traders we can place on standby around the clock as you say - so the mechanical/manual approach would no doubt miss many opportunities I'm guessing. 

Just curious - if a system; 
a) backtested well over a long period (10 years etc), and
b) position sizing was small enough (1%), and
c) exits were triggered by system conditions (not stops), and
d) it was in a constant market (few or no gaps) like Forex
would you still think it too risky to be left to automation? I respect your opinion Howard, I'm just trying to establish how much evidence would be required to sway that point of view. Or would you never trust a machine to execute no matter the results?


----------



## weird

howardbandy said:


> Hi BullBear --
> 
> There are interfaces between AmiBroker and brokerages.  Most of the work has been done with Interactive Brokers.  There are several AmiBroker forums on Yahoo -- one of them is Automated Trading.  Here is a link to that forum:
> http://finance.groups.yahoo.com/group/AmiBroker-at/
> 
> I have been personally involved with systems that trade automatically and recommend that automatic trading is only practical when the trader has a large enough organization to use it safely.  That means duplicate everything -- computers, data feeds, power sources, communications facilities, etc -- staffing so that there is a person at the terminal at all times (we had two traders around the clock), and a good enough relationship with the brokerage firm that they will break trades that the automated system entered that should not have been entered.  If a system is trading automatically, it cannot be safely left alone for even five minutes.
> 
> Thanks for listening,
> Howard




5 mins ... that sucks ... lol, are you sure you have exhausted your approach on trading.

Btw, I know of traders that have successfully traded using AT (automated trading) using AB (amibroker) with IB (interactive brokers) for years now.

Perhaps they trade different to Howard ?


----------



## weird

Howard, one thing I have always wondered, is what you trade, and what time frame ?

It would be easier to reply to your comments.

My responses for automation was EOD, but I also trade most futures (All) and stocks (US and Aussie).

What do you actually trade when you make comments ?


----------



## wayneL

weird said:


> 5 mins ... that sucks ... lol, are you sure you have exhausted your approach on trading.



That's a strange comment. Are you sure you have interpreted HB's comments correctly? 


> Btw, I know of traders that have successfully traded using AT (automated trading) using AB (amibroker) with IB (interactive brokers) for years now.



Well bully for you. 



> Perhaps they trade different to Howard ?




Your own guru mentions in his book that many profitable traders trade differently to each other. So what?


----------



## waza1960

> I have been personally involved with systems that trade automatically and recommend that automatic trading is only practical when the trader has a large enough organization to use it safely. That means duplicate everything -- computers, data feeds, power sources, communications facilities, etc -- staffing so that there is a person at the terminal at all times (we had two traders around the clock), and a good enough relationship with the brokerage firm that they will break trades that the automated system entered that should not have been entered. If a system is trading automatically, it cannot be safely left alone for even five minutes.



    I cannot agree here HB and I say that with the utmost respect for you as your teachings and books are the main influence in my trading .
There are many perspectives and styles of trading and I believe that there are trading styles that have little or none of the disadvantages you have stated.
I have used Ninja Trader for a few months running systems live 24hrs a day on FX and have not experienced any issues but I have managed possible risks such as I use a VPS which has a guaranteed reliability of over 98%.
  Also this style of trading uses multiple systems with small postition sizes and tight stops and systems are using an hourly time frame.I only checked these systems twice a day on average and they had no operational problems.


----------



## waza1960

> What I want to do is backtest my systems - which I can do now - but then once confirmed with a positive equity curve, switch it over to automated trading through my IB account. Do you know if this is possible now with AB (a plugin maybe?) or would NinjaTrader be better for this as it already has automated trading I believe?



         I have being told that you can not run systems on IB 24hrs/day  due to the fact that they reset their data daily which will stop your systems but DYOR.
           Ninja Traders Auto interface is more advanced IMO and more reliable so I would go that way for now but you would have to use other brokers for trading due to the above issue.
  I also have Amibroker and it is the best and most comprehensive tool out there ATM and if they decide to further develop their Auto Trading Interface then that would be the ultimate.


----------



## howardbandy

bullbear39 said:


> Hi Howard,
> 
> Well that's rained on my parade... but then I may have been confusing the term 'mechanical trading system' with 'automated trading system'. Would I be correct in thinking you support the former not the latter? As in, the mechanical system would unemotionally find the trade for you based on set criteria - but it won't go ahead and place the trade. That would need to be executed by a manual process via broker or platform.




Hi Bullbear --

Yes, I am strongly in favor of mechanical trading systems.  Mechanical as compared with subjective.

It is possible to use intra-day orders without using automated trading.  I regularly place limit orders to be filled when conditions that my trading system calculates occur.  But I place these myself through the brokerage interface -- I do not allow the computer to place them for me.  Depending on your broker, these orders can be quite complex, such as including one-cancels-other, so you have quite a lot of control over your trading without the need to be at the terminal throughout the day.

Thanks,
Howard


----------



## howardbandy

weird said:


> Howard, one thing I have always wondered, is what you trade, and what time frame ?
> 
> It would be easier to reply to your comments.
> 
> My responses for automation was EOD, but I also trade most futures (All) and stocks (US and Aussie).
> 
> What do you actually trade when you make comments ?




Hi Dave --

I personally trade high liquidity instruments, including ETFs, options, and occasionally futures, based on broad indexes on the US exchanges.  Holding periods range from a few minutes to a few days.

But my comments about automated trading are based on my real-life experience while I was an employee of a major commodity trading advisor.  The firm traded futures in all markets on all exchanges around the world around the clock.  We regularly had to rely on a backup to one or more of our system's components -- data feed, communications, line, power, computer, broker.  We regularly tested all of our backups -- throw the breaker on the power to the computer room, etc.

And we regularly saw the computer enter an order based on data it saw, have that order filled, then have that data point corrected or removed.  We needed a good relationship with our brokers to get out of that trade quickly -- either have it reversed or manually exit.  Whenever a trade is entered and the data that caused that entry is revised, the computer no long has a record that it is in a trade and it will never issue an exit for it.

I regularly speak with people who are trading automatically and have done so without problems.  My caution is that when problems arise, it is wise to have contingency plans in place and have tested them.  It is difficult to do this with a limited organization.

Thanks for listening,
Howard


----------



## weird

Thanks mate, although no one is listening, perhaps thanks for reading (J/J) ... Aussies are terrible jokesters.


----------



## reprpi

Howard,

I already have your "Introduction to Amibroker" book and in the process of reading it. But being new to trading, I would like to know if "Quantitative Trading System" book will explain the different types of trading systems.

Thanks in anticipation.


----------



## howardbandy

Hi reprpi --

Yes, Quantitative Trading Systems talks about different kinds of trading systems.  There is more information at the book's website:
http://www.quantitativetradingsystems.com

Thanks,
Howard


----------



## CanOz

I'm in my annual 'learn to code phase' and wanted to get into this book again. *Does anyone know if all the code in the book is error free?* I saw a complaint on antoehr forum and thought it may have been with the version of Amibroker being used. Is the code all error free, i would love to type out the code so i can get in the habit of actually constructing the code myself...


----------



## howardbandy

Hi CanOz --

1.  There was a nasty exchange on another forum from a person who had some undefined agenda and was making derogatory comments without specific and answerable points.  One of those was "I am getting errors" without quoting the line or even the program that was causing the error.  I was banned from that site for mentioning that I had published a book explaining the answer to a question asked in one of the threads.

2.  Does anyone know if all the code in the book is error free?  To the best of my knowledge, all programs execute correctly in the context of the example being explained in the book.  If you try something and it does not work as anticipated, ask for clarification.   Please identify:
A.  Which book.  (There are five books.)
B.  Which program -- by figure number or page number.
C.  What you expected the program to provide and how what you obtained was different.

Each book has an "errata" file.  The url is www.<booktitle>.com/book.  Corrections to text are noted in the errata file.  Corrections to programs are both noted in the file and reposted to the downloadable code in corrected form.  Read that file.  

I will do my best to clarify.  

Best,
Howard


----------



## CanOz

howardbandy said:


> Hi CanOz --
> 
> 1.  There was a nasty exchange on another forum from a person who had some undefined agenda and was making derogatory comments without specific and answerable points.  One of those was "I am getting errors" without quoting the line or even the program that was causing the error.  I was banned from that site for mentioning that I had published a book explaining the answer to a question asked in one of the threads.
> 
> 2.  Does anyone know if all the code in the book is error free?  To the best of my knowledge, all programs execute correctly in the context of the example being explained in the book.  If you try something and it does not work as anticipated, ask for clarification.   Please identify:
> A.  Which book.  (There are five books.)
> B.  Which program -- by figure number or page number.
> C.  What you expected the program to provide and how what you obtained was different.
> 
> Each book has an "errata" file.  The url is www.<booktitle>.com/book.  Corrections to text are noted in the errata file.  Corrections to programs are both noted in the file and reposted to the downloadable code in corrected form.  Read that file.
> 
> I will do my best to clarify.
> 
> Best,
> Howard




Howard that's great, thanks. Can't imagine why anyone would want to ban you, that's just silly.

I need a course to learn this stuff. I have a short attention span, books are difficult for me to learn from. I really wish i could just work my way through a course.

Howard, will you offering any of your books in eBook form?


----------



## captain black

CanOz said:


> I need a course to learn this stuff.




CanOz, I think Nick (Radge) has an Amibroker mentoring course at trial stage at the moment, not sure how much of it is coding orientated.


----------



## Gringotts Bank

CanOz said:


> Howard that's great, thanks. Can't imagine why anyone would want to ban you, that's just silly.
> 
> I need a course to learn this stuff. I have a short attention span, books are difficult for me to learn from. I really wish i could just work my way through a course.
> 
> Howard, will you offering any of your books in eBook form?




You've probably progressed beyond this already, but might be worth a look.  Plenty there.  http://www.asxmarketwatch.com/2012/06/lets-learn-amibroker-how-to-make-a-watchlist/


----------



## CanOz

Gringotts Bank said:


> You've probably progressed beyond this already, but might be worth a look.  Plenty there.  http://www.asxmarketwatch.com/2012/06/lets-learn-amibroker-how-to-make-a-watchlist/




Thanks, i've been reading through that stuff the last few days GB, he's pretty good. Just the YT access is a pain from here...


----------



## Wysiwyg

Gringotts Bank said:


> You've probably progressed beyond this already, but might be worth a look.  Plenty there.  http://www.asxmarketwatch.com/2012/06/lets-learn-amibroker-how-to-make-a-watchlist/



That is the start of the backtest merry-go-round.  Backtesting stocks that survived to present is a con.


----------



## Trembling Hand

Wysiwyg said:


> That is the start of the backtest merry-go-round.  Backtesting stocks that survived to present is a con.




I would be genuinely interested in your ideas that you back tested?


----------



## Gringotts Bank

Wysiwyg said:


> That is the start of the backtest merry-go-round.  Backtesting stocks that survived to present is a con.




There's two ways around survivorship bias, and I think you know what they are, yes?

The systems on that site are more useful for getting into the swing of coding, then you start to do your own thing.


----------



## CanOz

My data base includes de-listed stocks as well....even one that still has my money...


----------



## Wysiwyg

Gringotts Bank said:


> There's two ways around survivorship bias, and I think you know what they are, yes?



Yes, test using delisted stocks also for that period or test over recent periods where most stocks are still in the present list.

Some other considerations are 

1) stock volume actually available at entry/exit time
2) commission round trip
3) maximum consecutive losers
4) maximum system drawdown


----------



## Gringotts Bank

Wysiwyg said:


> Yes, test using delisted stocks also for that period or test over recent periods where most stocks are still in the present list.




Using much shorter time frames will reduce the over-estimation of profits that a long-only backtest might provide.  And yeh, using survivorship bias-free data in your testing.


----------



## Wysiwyg

Gringotts Bank said:


> Using much shorter time frames will reduce the over-estimation of profits that a long-only backtest might provide.  And yeh, using survivorship bias-free data in your testing.



There are also stocks that are not delisted but are removed from an Index list for market capitalisation reasons or other.


----------



## CanOz

I was particularly interested in testing different index filters for my little twist, on the flipper. After a full day here's my best in and out of sample results.

I wanted to run this system weekly. So i can have more time to trade my other stuff.


----------



## howardbandy

Greetings --

I have posted four hour-length videos from recent presentations on YouTube.  I recommend watching them -- particularly the ones on Stationarity and Risk.  The two on Development are also worth while.  The first uses traditional trading system development platform, as in AmiBroker.  The second uses machine learning.  

Please note that backtesting is a necessary step in trading system development, but backtesting alone is not sufficient.  The only way to obtain a reasonably unbiased estimate of future performance of a system is validation through applying the system to data that was not used in development.  The gold standard of validation is walk forward testing, where the previously unseen data is the same data stream(s) as those used in development, but more recent in time.  I cannot say this too strongly -- in-sample test results have no value is estimating future profitability.  Out-of-sample validation is a requirement.  If it is not done in the lab, it will be done by the market.

There are many sources of bias that will interfere with development.  There are no favorable biases.  If the results are biased, future performance will be worse than expected -- higher risk, lower profit.  Here is a short list of some of the major ones:
..Survivorship / membership.
..Assuming backtesting is sufficient validation.
..Assuming independence when there is correlation.  
..Metrics that include sequence dependent terms.  
..Multiple uses of out-of-sample test data.  
..Ignoring intra-trade drawdown.
..Using single metrics rather than distributions.
..Assuming stationarity.
..Future leaks.
Try very hard to avoid all these.   

YouTube video links:
The Four Faces of Risk:
https://www.youtube.com/watch?v=Vw7mseQ_Tmc
The Importance of Being Stationary:
https://www.youtube.com/watch?v=iBhrZKErJ6A
Indicator Based Trading System Development:
https://www.youtube.com/watch?v=W-gSsqHORJE
Machine Learning Based Trading System Development:
https://www.youtube.com/watch?v=v729evhMpYk

A Google search of "Howard Bandy YouTube" will show a dozen or so additional videos.  Some by me, but most recordings made by others of interviews of me and presentations by me.  Among them are several from my presentations at the ATAA Conferences.

Best,
Howard


----------



## chops_a_must

Hi all.

Long time no see. I'm going back through and retesting some code and whatnot.

I seem to be having particular trouble with Howard Bandy's seasonality codes.

For instance, this code:

//	SeasonalityDayOfMonth_Testing.afl
//
//	Test for seasonality based on the day of the month.
//	This version does the day-by-day testing.
//
SetTradeDelays(0,0,0,0);
BuyPrice = Close;
SellPrice = Close;

DayToBuy = Optimize("DTB",1,1,31,1);
Buy = DayToBuy == Day();
Sell = BarsSince(Buy) >=1;
//Figure 11.1 Seasonality Day of Month - Testing

only generates a buy on the 1st day of the month. I can alter it, so that as a parameter, it can do each single day individually, but not all at once.

I'm sure there is a setting I have bumped that is preventing this, but I can't work it out. 

Cheers,
Chops.


----------



## Habakkuk

It works fine for me.
You're probably running "backtest" rather than  "optimize". You should get a list of 31 backtests.


----------



## chops_a_must

Habakkuk said:


> It works fine for me.
> You're probably running "backtest" rather than  "optimize". You should get a list of 31 backtests.




Haha.

Got it.


----------



## CanOz

chops_a_must said:


> Haha.
> 
> Got it.





Chops!!!

How are ya man!


----------



## Wysiwyg

chops_a_must said:


> Hi all.
> 
> Long time no see.
> Cheers,
> Chops.



I remember you. Happy to see ye posting again. The 2006 join ups.


----------



## chops_a_must

CanOz said:


> Chops!!!
> 
> How are ya man!




I'm good. After a few years of moving around and stuff, I've actually been able to settle and hopefully get back into some active trading.


----------



## howardbandy

Greetings --

Use caution with seasonality-based systems.  It is very easy to overfit the model in-sample, then find poor performance out-of-sample.

For US stocks and equity ETFs, there have been roughly two cycles per month.  Each with a period of about 3 to 5 days of price rise and 3 to 5 days of price drop.  One revolves around month-end, the other around third-Friday options expiration.  Both have been "discovered" and both are losing their edge.

The seasonality based on October being the poorest performing month was badly wrong this year. 

IMO, essentially all of the rise in the stock, bond, and real estate markets has been due to the US government manipulating interest rates and forcing people who want income from their funds to take long positions in those asset classes.  As either the government or the financial world lifts US interest rates, I expect all those markets to return to their 2008-2009 lows.  If it happens, this effect will swamp seasonality, including the traditional "January" effect.  

Best,
Howard


----------



## chops_a_must

Thanks Howard.

That's why I'm retesting these ideas.

I've never used the seasonality findings apart from on the ASX. Where the seasonality is a bit different to the book. I suspect this is because of superannuation flows and influence on our market.

I'm sure blogs such as overnight edges and the like, which many years ago were selling daily seasonality odds for positions, was probably the beginning of the end for a lot of seasonality systems.

Cheers,
Chops.


----------



## 11bblandin

Howard, 

I have a question that pertains not to Quantitative Trading Systems but rather your latest book Quantitative Technical Analysis. You use each day's mark-to-market equity as a closed trade, and then go on to sample from these "closed trades" as part of your monte carlo analysis. Is it possible to do so without eliminating the effect of serial correlation that comes from being in a position? Or sampling two consecutive points that occurred in different market regimes? Surely there are entry opportunities where multiple profitable days follow? Otherwise the concept makes a lot of sense from the standpoint of trade management.

Best,
Brian


----------



## howardbandy

Hi 11bblandin --

Marking to market daily gives an opportunity to both measure and manage daily.  My view of trading systems is that the trader must have a period of time during which he or she is willing to wait.  The assumption I make in the book and in my presentation is that period is a single day.  That is, I depend that my model will cause an exit whenever conditions are such that I would have entered an order to subjectively exit the trade.  If there is a sequence of days where the same position (say, long) is held throughout, there is no need to charge commission or slippage for the continuation days, and the result of a multi-day marked-to-market-daily trade where there is a state reported every day is identical with those of a multi-day trade delimited by impulse buy and sell signals.  

If there is in fact serial correlation for the days within a trade, you should treat the entire multiday trade as a single data point.  Test this as part of the development.  The sequence of intra-trade percentage gain per day may not be serially correlated.  If there is no serial correlation, you can treat each day as an independent data point.  

Analysis of multiple trades, as in a portfolio, is complicated when there are days with multiple holdings.  If the issues being traded are equities, there is probably strong correlation between the gains for two or more issues.  If so, they should be treated as a single "daily result" data point.  Think ahead to the Monte Carlo simulation that will be run.  The trades drawn in the MC process are assumed to be independent.  If they are not independent, the results will be biased toward over-optimism.

I hope I am answering the question you were asking,
Best,
Howard


----------



## 11bblandin

I am focused mainly on systems that trade a single issue long/flat and agree with the concept of assessing each day whether to be long or be flat (whether that means entering, exiting, or continuing to be long/flat). However, when I move on to the next step of determining risk through MC simulation, the sequences of trades that generate the largest drawdowns may likely be the result of several consecutive trades (days) when large losses hit. A system that incorporates a trend filter or similar mechanism to stay out of the market when things start to get turbulent would (hopefully) generate a signal to be flat. 

As an example, a system may encounter several very negative days while long  over the course of its test period, which drops the price below a threshold (say, a 200-day moving average) and the system dictates to be flat until more favorable conditions return. A MC simulation may string these big losers together, creating a large drawdown and indicating a high level of risk. I understand the concept of tail risk and why it's important in development, but I also try to design systems where I wouldn't continue to be long day after day as the market plummets. Hopefully I'm being clear.. I can't think of a better way, just curious if you've run into these issues in your testing and how you dealt with them


----------



## howardbandy

Greetings --

A filter such as taking long positions only when the price is above the 200 day moving average is popular in the literature.  The price comparison that rule creates is between the average of the 200 days -- the midpoint of which is 100 days ago -- and the current price.  Whether that rule helps or hurts depends on how persistent the price trends of that issue are and what the entry and exit rules are.  Perform your own tests before accepting it as good practice.

My recommendation of marking to market every day goes along with managing the trade every day.  In order to becoming stuck in a bad trade, the rules must include an exit to get out of it.  Some rules that can work in some cases include:  maximum holding periods, profit targets, maximum loss. Test to determine which, if any, help.  

Note that maximum loss exits are usually entered as "stop" orders.  There is a change coming to rules regarding which types of orders are permitted for US markets.  It is likely that stop orders will be restricted, and maybe prohibited.  Good-until-canceled stop orders are almost certain to be prohibited.

Trading systems that trade infrequently, hold a long period of time, and hope to catch big price moves are most susceptible to getting caught in a large losing position.  If your personality permits it, consider using an objective function that rewards systems that: trade frequently, trade accurately, hold a short period of time.  Systems with these characteristics often have the best performance (for long/flat positions) when the price is below its long term moving average.  In those cases, setting a long term moving average filter removes a high proportion of the profitable trades.

You are correct that a Monte Carlo analysis can choose trade sequences where losing trades follow each other closely and create large drawdowns.  The purpose of the Monte Carlo analysis is precisely that -- to estimate the risk if / when that happens in real trading.  Do not ignore the tail risk.

Best,
Howard


----------



## 11bblandin

I guess my question pertains more to trading systems driven by machine learning, but can also apply to rules-based systems that use chart patterns as setups. Since all past information must be contained in each data point if you wish to include indicator values or candle characteristics from prior bars, won't this affect the accuracy of a monte carlo analysis? 

If (as in the book) you use RSI values from the three most recent bars as inputs for a model, should we attempt to retain a similar pattern in a MC simulation? If the prediction generated by the model depends on all of these inputs, then I would think there is inherently some serial correlation in signals to be short or be flat. For instance, if a model determines that a sequence of three days with rising (or falling) RSI values is predictive of a price increase or decrease, shouldn't that be taken into account? A MC simulator will likely sample three consecutive days that don't display the characteristics that would have generated a long or flat signal in the first place.

I still think that MC analysis is extremely useful in examining confidence levels of future outcomes, just wondering if anyone has proposed a way to overcome this, or if it's even significant. I haven't found anything


----------



## howardbandy

Greetings --

There is a fundamental difference in the approach the developer takes when using traditional trading system development versus using machine learning development.  

When using a traditional trading system development platform to build a formula-based trading system, the indicators and rules are written and interpreted first, these generate buy and sell signals, resulting in trades, which are analyzed.  In short -- examine an indicator, see what happens later.

When using machine learning to build trading systems, a set of data that includes both indicator values (predictor variables) and buy and sell signals (target variables) is created, with the buy and sell signals formed so that the resulting trades are the ones the developer would like to take.  All of this data is passed to the machine learning / pattern identification toolkit where a set of rules or equations are formed that relate the indicator data to the signals.  In short -- desirable trades first, then see what happened earlier.

The machine learning toolkit, such as scikit-learn, has a large number of model templates that can be used to fit the indicator data to the signals.  One of those models is decision tree.  Others include support vector machine, linear regression, neural network, and many more.  Traditional trading system development platforms (almost always) use the decision tree template for the model.  Decision trees are tree-like sequences of if-then-else statements guiding the computations to "leafs" that have values of Buy or Sell.  They have the advantage of being easy to understand.  As evidenced by the profitability of traditional systems, decision trees can produce excellent trading systems.  One of their weaknesses is a tendency to overfit during training / optimization.  This results in them being overly fit to the in-sample data with poor performance out-of-sample.

The best validation for traditional platforms is walk forward.  The out-of-sample test data is more recent values of the same data streams used to develop the system.  The indicators and rules used are those ranked as "best" by objective function value of all alternatives tested over the in-sample period.

Monte Carlo techniques are typically not used in model development when using a traditional platform.  They may be used in guiding a non-exhaustive search for the best solution, but that is only to shorten the search and avoid an exhaustive search.

Walk forward validation is also used for systems developed using machine learning, and in precisely the same way.  There are three data sets used for machine learning development.  In additional to the out-of-sample validation data, during the learning process, and before the validation step, there is a division of the in-sample data into two sets -- training and testing.  The training set is extensively examined to determine the formulas or patterns.  The testing set is used to guide the training.  It is common to use random selection of data elements to divide the whole set of learning data into training and testing.  That can be thought of as a Monte Carlo operation.  There can also be guidance to the search using non-exhaustive methods, similar to those used for the traditional platform.

--------------

The most important use of Monte Carlo analysis -- whether the system has been developed using traditional platform or machine learning -- is estimation of risk and profit potential.  This occurs after the system has been validated.  It is a separate step that uses the out-of-sample trades as its input data, and produces risk and profit metrics as its output.

-------------

It is true that data points passed to the machine learning process must be self contained and independent of each other.  That means that historical data that the developer thinks might be valuable as a pattern variable, such as yesterday's RSI value, must be explicitly computed, stored, and included as a variable.  (Monte Carlo is not used in this operation.)  

A traditional system might appear to have just a few variables -- RSI lookback length, buy level, sell level.  While the equivalent data file for the machine learning process probably has more -- RSI today, RSI yesterday, etc.  They are called predictor variables.  The traditional development implicitly uses previous data through builtin functions that refer to yesterday's value, such as Cross.  It can do this because the time series data continues to be available during model development.   

It is tempting to compute and include a large number of predictor variables, hoping that the machine learning routine can sort out which are important.  That is usually an unreasonable expectation.  A smaller number of predictor variables (perhaps a maximum of ten or so) should be determined either by the developer using his or her experience (his or her "domain knowledge" -- in the jargon), or by a using a subset chosen by a search.

---------------

Best regards,
Howard


----------



## James Woods

Hi Howard,

Big fan of your books, I find them very insightful and they have dramatically changed my approach to trading.

I'm not sure if this is specifically something you can help with, I'm using Amibroker along with FTSE350 EOD equity data to run some swing trading systems that I use on Australian & U.S. equities - I want to make sure they are robust enough to work across global markets. 

Unfortunately I cannot find a data provider that covers Europe in the same way that Norgate covers the U.S. & Australia. I have all the historical data and assuming I can obtain a list of which equities were included in the index in a particular year, is there a way you can recommend in terms of creating your own watch lists to check whether a specific equity was included as an index constituent at a specific time.

I am also aware that this would be a very burdensome process to account for ticker changes & mergers etc, unfortunately until I find a data provider that has a paid services covering this I feel it's my only option. 

Many thanks,

James


----------



## tech/a

Clearly I'm not Howard.
But we get our data from here.

https://www.tickdata.com/


----------



## James Woods

tech/a said:


> Clearly I'm not Howard.
> But we get our data from here.
> 
> https://www.tickdata.com/




Hi Tech/a

Thank you for you quick reply, I've had a look at tickdata. It certainly has more information than I need (only looking for EOD data) and the cost is too high unfortunately for me to justify given my account size. 

Appreciate the suggestion though,

James


----------



## howardbandy

Hi James --

Thanks for the kind words.

I am not familiar with data for European markets.  

I'll second Tech/a's recommendation of tick data, providing it has the markets you want.  My experience with them is dated, but has always been good.

I am a big fan of Norgate.  Although they do not provide the data you need, they might be able to point you to a vendor who does.  Or they may have a product available in the future.  You might check with Richard Dale, one of the principals of Norgate.
http://www.premiumdata.net/

Another possibility is Quandl.  
https://www.quandl.com/collections/markets/global-markets-overview

Best regards,
Howard


----------



## James Woods

howardbandy said:


> Hi James --
> 
> Thanks for the kind words.
> 
> I am not familiar with data for European markets.
> 
> I'll second Tech/a's recommendation of tick data, providing it has the markets you want.  My experience with them is dated, but has always been good.
> 
> I am a big fan of Norgate.  Although they do not provide the data you need, they might be able to point you to a vendor who does.  Or they may have a product available in the future.  You might check with Richard Dale, one of the principals of Norgate.
> http://www.premiumdata.net/
> 
> Another possibility is Quandl.
> https://www.quandl.com/collections/markets/global-markets-overview
> 
> Best regards,
> Howard




Hi Howard,

Thank for your response, I'll get in touch with Norgate and find out - if there are any positive developments I'll be sure to let everyone know.

Many thanks,

James


----------



## obiwanchernobyl

Hi Howard

Long time reader; first time poster. Thoroughly appreciate all the work you've put into your books.

My question is around opportunity cost of trading (i.e. is it worth the effort). I was wondering if you may give some indication of return levels achieved through your methods? perhaps a range, either by you or people you have consulted with that adhere to your methods.

Or is the answer as simple as saying that if you can continually develop systems with a CAR25 of 5% for 1-year Monte Carlo risk-normalised simulations, then after 10 years you may expect at least 7.5 of those years to return greater than 5%? or to put more frank, if CAR90 was continually 50%, you may expect 1 of those years in 10 to return 50% or more?

I'm just trying to justify the effort to myself of what I know will be hard work and was hoping for a little motivation—preferably evidence based.

Thanks
Calen


----------



## obiwanchernobyl

Apologies—just found the cdf's on page 11 of the _Tools Being Used By Your Trading Competition_ thread, with the ML example showing CAR25 of 11.1%. This appears to be a reasonable answer to my question.

Do you have any examples of cdf's for non-ML systems that have been, or are actually, traded? (not just the examples in the books).

Again, just after motivation of what's possible with some (more) hard work.

Thanks
Calen


----------

