Developing a mechanical system from scratch

[jjbinks]

Recent returns on my multi system portfolio, last 100 days in most cases. In a couple of cases where the systems trade less frequently there are no trades at all.

Overall, certainly worth continued testing. These results are just the back tests on recent unseen data, not live results or live results on simulated accounts.

how much starting capital do you have in your system canoz?

Also what data do you use?

thanks!

 

[CanOz]

how much starting capital do you have in your system canoz?

Also what data do you use?

thanks!

It's not live yet, but I'll be using my super. It has roughly 150k in it.

I'll use cqg data but execute through IB.

 

[Quant]

With regards to Building Systems i'm curious to know what traders filter or measure to define parameters of what will give a signal . Given the many states that the market operates in are traders building separate systems for each individual type of market conditions ? I have spent many years developing a system that operates in all market conditions , obviously it's a compromise trying to build a system that deals with the entire spectrum of market conditions . I primarily trade indice and have based my rules and parameters on the individual style of price action attributable to index . The whole premise of building a 1 system does all is intricate and complex with many sets of rules to switch between various market conditions. I am curious to know whether there are other Traders they have gone down this road . I have written many of my own indicators to achieve accurate filters to determine the signals I require . Looking for feedback from other systems developers . Eventually i plan building an automated system around my core strategy , given the complexities and rules of my system writing the code is above my pay grade , at some stage I need to hire or collaborate with someone that has advanced code writing abilities . Anyway i'm hopeful that we can get positive discourse going on this subject . Looking forwards to any useful commentary

 

[Quant]

Whilst developing systems I have found it constructive to break each element down and work on one element at a time . These elements can consist of several things . Given that the best bang for buck trades are with the trend I find that making trend following trades the primary source a profit , so once we have defined a trend we need to ascertain places in the trend to join trend . When we have a ranging market the profit potential is equal in both directions therefore the parameters will be different to those of a trending market and a different set of rules needs to be applied . In regards 2 trading Indice tops and bottoms have distinctly different styles and parameters therefore the rules to trade a top distinctly different for those involved a bottom . Index swing highs slow and bottoms fast therefore different parameters are required to identify if market is topping or bottoming . This is where the complexity of rules come into play as you just can't use an inverse set . So breaking it down a set of rules is required for each condition and the change of conditions . This is where a list of bullet points comes in handy so you can work on one at a time and optimise the rules pertaining 2 each condition. This is where it's obvious that it becomes a complex task . Identifying the price action that defines conditions and change of conditions needs to be turned into a maths based equation ( an indicator ) so as to systemize .

 

[Quant]

I been playing with daily systems to trade indice , this one straight of the mould with ZERO risk management applied just buy and sell predefined signals/ parameters with custom indicators , also long only ( at this stage ) adding the short side will smooth the curve and reduce the drawdown ( i think ) . also flat position size as well so dynamic position sizing will likely improve returns . Such interesting concepts

 

[Quant]

The thing with these systems to stand the test of time a GFC type event must be dealt with and so I have done some work on the short side aspect , obviously long setups dismal over gfc type event but the short side setup absolutely creams it with massive returns , will work on combining the 2 into one system to evaluate the curve even thought its easier to run 2 systems concurrently ... this stuff is so exciting

 

[Quant]

Exact same setup applied to my intra time frames , obviously gets killed after swing highs with zero risk management but pretty easy to apply a filter to prevent longs in declines and limit loss sizes , and same thing adding shorts smooth it right out . OBVIOUSLY drawdowns not acceptable but i think thats an easy hurdle to overcome

Worst 10 trades not hard at all to iron these out to something much more palatable with a trend filter and some sort of active stop loss

 

[CanOz]

Great work Quant, this is a system that I'm walking forward at the moment (simulated) and was inspired by your 'descretionary systematic' approach and uses volume profile for context. Its a work in progress and something i hope to fully automate some day. Stops are wide and entries not so important, don't require all the order flow tools....Its trading GC, CL, FDAX, FGBL, 6B

 

[rb250660]

Given the many states that the market operates in are traders building separate systems for each individual type of market conditions?

I have been trading a swing type model for the last 12 months. I just finished coding a large cap trend following model a few days ago that I will let loose in April after I finished validating and tweaking it. They are two distinctly different systems and will be run as such. I don't use regime switching in the one model, too complex on so many levels.

In regards 2 trading Indice tops and bottoms have distinctly different styles and parameters therefore the rules to trade a top distinctly different for those involved a bottom. Index swing highs slow and bottoms fast therefore different parameters are required to identify if market is topping or bottoming.

Right on. In fact I would set up 2 different systems to manage this.

The thing with these systems to stand the test of time a GFC type event must be dealt with and so I have done some work on the short side aspect , obviously long setups dismal over gfc type event but the short side setup absolutely creams it with massive returns , will work on combining the 2 into one system to evaluate the curve even thought its easier to run 2 systems concurrently ... this stuff is so exciting

Make sure that in these types of events you can actually short the stocks you want to short. I am under the impression a ban was placed on shorting many stocks during the GFC.

 

[Quant]

I have been trading a swing type model for the last 12 months. I just finished coding a large cap trend following model a few days ago that I will let loose in April after I finished validating and tweaking it. They are two distinctly different systems and will be run as such. I don't use regime switching in the one model, too complex on so many levels.



Right on. In fact I would set up 2 different systems to manage this.



Make sure that in these types of events you can actually short the stocks you want to short. I am under the impression a ban was placed on shorting many stocks during the GFC.

Financial stock shorts were banned after lehmans collapse , easy enough to leverage shorts on indice , if you were short banks pre ban you were entitled to keep/hold them .

Re regime switching , sure its reasonably complex but all totally actionable and not overlly difficult , plenty of lines of code but tedious more than anything . I know a guy who has algos with 6000 lines of code , the decision trees are all logic its just the rules involved that create the dynamic algorythym , maths skillsets required but we arent doing quantum physics are we .

Re the large cap system , i am working on a similar thing , just working on the bullet points atm and should have a workable idea soon . I am doing the whole technical/fundamental combo . Pretty unique stuff i believe and should produce some industry leading returns . Mean reversion is easiest on large caps due to the solid and reasonably predictable balance sheets , over shoots either bull/bear all very actionable and predictable . Interested to see your results , need any encouragement i can find . Biggest issue for me is i have more ideas than time . One day hope to have some staff , this year the plan . Just need to trade up the capital

 

[rb250660]

Interested to see your results , need any encouragement i can find . Biggest issue for me is i have more ideas than time . One day hope to have some staff , this year the plan . Just need to trade up the capital

I have the luxury of not working so I get to spend a lot of time on model development and testing. I write all my own validation and monitoring tools too. I believe being able to code and having sound maths and statistics skills are necessary to take the quantitative road. My day job was in engineering and I have found that experience to be invaluable.

I also think too many people make decisions on single run back tests. On top of that they load position sizing into the model too which is inappropriate in almost all cases. Elements of risk should be assessed outside the model after every trade. Looking at discrete/average figures is so misleading. I see everything in trading (and life for that matter) as a distribution of probabilities. Before I enter any trade I know my stats. A lot of people don't know their stats so that's probably why they don't stick to their model, they can't cope with the uncertainty.

My swing model I actually finished developing in December 2013 but I was too busy with work to trade it up until about 12 months ago. The out of sample performance from 2014 has been really good so that confirms my model works and it still works. I made a few minor changes to it this week after observing it trade real money for a year and getting some screen time. The issues were all related and concerned liquidity, spread and volatility.

My trend model was an old idea I had back in 2013 as well. I couldn't get the code working back then and gave up too easily. 2 weeks ago I was cleaning up some files in my code folder and looked at it again and saw the problem right away. I've spent 2 weeks solid working on it and now it's essentially finished. I'm just waiting for some capital to come out of a term deposit next month and I'm set to go.

Single back test results below. I know I just **** canned that approach but the distribution of results is very tight so these averages are a pretty good representation with the exception of the draw downs. The most frequent occurrences of the draw downs are much lower than the maximums stated (about a 1/6):
Swing model:
Win: 74% of trades, 5.6% avg. gain
Loss: 26% of trades, -4.2% avg. loss
CAGR 39%, MaxDD 12%
115 trades a year

Trend model:
Win: 60% of trades, 28.4% avg. gain
Loss: 40% of trades, -6.2% avg. loss
CAGR 37%, MaxDD 27%
30 trades a year

I have capped position size on the swing model due to the relationship between liquidity and the amount of trading opportunity I am looking for (I limit my position size to around 7% of the days liquidity). The Trend model doesn't suffer from this (large cap stocks only taken) and position size is uncapped. This is an often overlooked element in those great looking back tests everyone has.

Both models are very simple and have very few 'moving parts'.

The correlation between each system's daily returns is 0.13

 

[rb250660]

Here's an observation I have made in my time testing systems and it's regarding the index filter. People seem to include it 'just because everyone else does' to getting their mechanical systems to spit back a decent back test. Presumably to limit draw downs.

I don't agree with the principle and don't use it in any of my trading. There are more intelligent ways to manage the issues that an index filter is supposed to manage. It might be the case that not all your open positions are correlated to the index in minor corrections. Too often the index filter will pull you out of quality positions just to have you re-enter soon after at a higher price. Yes, during events like the GFC correlation goes to 1 pretty much across the board but if you deal with this intelligently it's not a problem.

Anecdotally, the day traders around here know that on the days when the market has gone to **** and there are one or two stocks running everyone is flocking to them. These are the easy money and often make for a great day of trading. That's how I like to think of it but on a longer time frame.

Have a good reason to get out of a trade such as you were just plain wrong and it decided to move against you. Don't release good positions just because the XAO decided to move below its own 75SMA.

So think about it another way, I'll leave it up to you to consider. Don't just include it every time you code a new system because it's in your template or because 'everyone else' does it.

 

[howardbandy]

Re regime switching , sure its reasonably complex but all totally actionable and not overlly difficult , plenty of lines of code but tedious more than anything. I know a guy who has algos with 6000 lines of code , the decision trees are all logic its just the rules involved that create the dynamic algorythym, maths skillsets required but we arent doing quantum physics are we .

Regime switching has become easier. There is a working example in AmiBroker in my "Mean Reversion" book, and a much different example in Python in my "Quantitative Technical Analysis" book.

The AmiBroker algorithm is about 150 lines long, of which over half are comments, system settings, and lines listed separately for clarity. In essence, there are multiple (two in the code shown) separate models in the file. The trading result of each is tracked and the buy and sell signals come from the one that is best. The listing of the afl is on page 192 and is easily understood. This works, and the chart shown on the cover of the book is the out-of-sample result of this program.

The Python algorithm uses the CAR25 universal objective function along with sequential Bayesian analysis to track and switch among any number of alternatives. CAR25 is the estimate of risk-normalized profit potential. It is a Dominant metric -- which means that the best use of funds is to trade the single system that has the highest CAR25, switching to another when the ranking changes. There is no need for a portfolio. In fact, forming a portfolio means that a portion of the funds are being used sub-optimally. Risk-normalization, distributions of results, CAR25, and dynamic position sizing are all developed and explained in the QTA book. The Python code that implements dynamic position sizing is about 200 lines of code. See page 412.

For more information, and to read portions of each book, begin here:
http://www.blueowlpress.com/123-2

In always, perform your own validation before considering using either technique.

Best regards, Howard

 

[tech/a]

Howard.

You have a long "association" with Amibroker. Written many books on systems development.

But my question is
Is it adequate for the development of trading systems particularly shorter term as you have alluded to in your replies over the last few months.
Or
Do you really need to in corporate python with its greater capability? Is the software required not here yet?

Would you combine the two---could you?
Or stand alone python?

Is there enough out there for the general punter to design and monitor a profitable trading system----in your opinion?

Do you think the general punter without sound mathematical skills beyond basic high school maths and sound computer programming and literacy skill has the tools available to invent and implement a profitable trading system?

 

[howardbandy]

Greetings --

Traditional trading system development platforms, such as AmiBroker, TradeStation, and NinjaTrader, have the benefits of features specific to traditional technical analysis with its roots in chart interpretation. Analysts and traders who find value in those techniques should stay with those platforms. But skip to and read my final comments before leaving this message.

More recently, traditional platforms added capabilities to develop and test rule-based trading systems. Trading system specific languages such as TradeStation's Easy Language and AmiBroker's afl are examples. There are two primary limitations to traditional platform trading systems.
1A. They, with very few exceptions, use the "decision tree" model. Decision trees have several advantages, including rapid testing of alternative sets of rules and ease of understanding the resulting model. One of the major disadvantages of decision tree models built using traditional platforms is that they are easily and often overfit -- that is, it is easy to adjust the rules so that the fit to the in-sample data is excellent and perhaps too good.
1B. There are model development techniques that can be used to test whether a model has been overfit, and to test whether it has identified general characteristics of the data. One such technique is crossvalidation. It is regularly and widely used in almost all areas of application -- but trading systems seems to be the most important area that is missing. The model development literature describes crossvalidation well.
1C. The primary purpose of systems we develop is prediction. We want the system to identify conditions that precede profitable trading opportunities. That is, we want the model to have learned to identify some persistent signal within the data, so that it can continue to identify similar signals in data that was not used during development. The only way to know whether a model has "learned" something general or "memorized" something specific is rigorous validation. Traditional platforms are weak in validation.
1D. Model development for prediction is guided by an objective function. Traditional platforms have very limited capabilities with regard to the definition of objective functions and evaluation of models. Almost always the objective function is a single value -- such as the final equity or smoothness of an equity curve. A more thorough development makes use of a distribution of results, of which the single value is one example. Traditional platforms are weak in capabilities for analysis of the distribution of results.
2A. Decision tree models are only one of many alternative model techniques. Others include support vector machines, boosted models, nearest neighbors, neural networks, forests of individual models (including decision trees) -- there are several dozen well documented and professionally implemented alternative models available.
2B. Individual decision tree models are seldom the best of the alternatives. Unfortunately, there is no way to determine in advance of extensive model development which model technique will be best for a given set of data. Fortunately, the models available from the scikit-learn library have a common interface and can be tested, fitted, compared, and validated in a single computer run.

Machine learning is the label often given to the broad technique of developing models that learn from data. (In a very limited sense, traditional trading system development is machine learning, but with a very restricted set of applications and capabilities.)

To address your specific questions:

----------------

The two techniques -- traditional platforms and machine learning -- are equivalent with regard to bar length. Either can use data bars of any length. The pandas library -- developed by Wes McKinney while he was an analyst for Cliff Asness' AQR Capital hedge fund -- makes data manipulations (such as dealing with missing data, changing periodicity, standardizing time zones, adding columns of intermarket data, etc) easy.

There is a fundamental difference in point of view between the two techniques.
With traditional platform development, many decision are made in advance either by the publisher of the platform (in terms of what techniques and tools are available) or by the developer during development. Within those constraints, the focus is: "Compute an indicator, then see what happened later."
Contrast with machine learning, where the focus is: "Identify a useful pattern, then see what preceded it."

Holding period is related to this difference. But even that comes after frequency of evaluation. I have written extensively about the difference between impulse signals and state signals.

Traditional platforms primarily (but not exclusively) use impulse signals (Buy, Sell, Short, Cover) to mark the beginning and end of trades. State signals indicate the position to be held from the current time to the time of the next evaluation. If you are using daily bars and updating after every close, the evaluation period is daily. If you are using impulse signals (and strictly following the rules of your model), no action is taken until a new signal is issued. The holding period is the time between the Buy and the Sell. That might be limited by one of the rules, such as a minimum or maximum holding period exit. But there is no inter-signal action.

State signals are generated by the model at every evaluation. If the model is examining daily data, it is searching for patterns daily, and it will issue its recommendation for the trade for the next day. That will be one of the states you want your trades to include -- "beLong," "beFlat," or "beShort." The accounting for a multi-day hold is identical whether it was due to two impulse signals some days apart, or to several beLong state signals in succession.

With a traditional platform, decisions made by the the developer determine ahead of analysis of the data what the holding period will be. With machine learning, the model looks for the signal within the data and reports it as it occurs.

-------------

You could combine the two techniques. Traditional platforms are specialists at reading data from providers such as Yahoo, Google, and Norgate. And they have builtin functions to compute indicators (which may or may not be a good thing). You can, and I have demonstrated code that does this, read data, compute indicators, and write a csv file to disk using a tradition platform; then use a machine learning program to read that csv file, build and validate the model.

You will probably not be able to call the machine learning libraries or completed model directly from the traditional platform. So signal generation will always be a two step process involving both tools.

Python is equally as good as reading financial data. Yahoo, Google, and Quandl provide free data. Quandl provides high quality subscription data with over 4000 data series. I have had several conversations with Richard Dale of Norgate encouraging him to support a Python API.

-------------

I am not certain about what a general punter can do, or wants to do.

The profession of trading has already moved to bigger firms, shorter time periods, more sophisticated systems, and stiffer competition; and the markets are becoming more efficient and profits more difficult to come by. Machine learning is already dominant, and will become more so in the future. Being successful is not just being best among a small group of non-professional traders, but rather among the best in the entire arena. There are no handicaps given to newbies, no do-overs. David Shaw and Goldman Sachs will vacuum up all wrong trades and bankrupt all mediocre traders without a second glance.

In my opinion, every trading system developer needs to know that his or her systems can compete profitably.

-------------

Some general suggestions. All have already been published in my books and presentations.

1. Use state signals. Even if you are staying with a traditional platform. Evaluate every day. Be willing to change position every day.

2. The sweet spot is: trade frequently, trade accurately, hold a short period, avoid serious losses.

3. Use CAR25 as your objective function. It is a Dominant metric. It can be used to compare any two or more alternative uses of funds. Trade the single best, switching to the next best when CAR25 values change.

4. Trade a single issue -- the single issue with the best CAR25. Forming a portfolio only ensures that some of the funds are being used suboptimally.

5. Keep position sizing out of the signal generation model. The pattern recognition portion has no way of knowing whether a distribution drift is taking place. And -- if position sizing is not in the trading management system, then there is no "knob to turn" to adapt to changes in system health.

-------------

In summary.

My association with AmiBroker began over 15 years ago. At that time, I was in recovery from a very bad business experience where a trading system platform I had designed, built, and was beginning to market was stolen from me by a disreputable business partner. I was in the process of writing a new platform when a colleague introduced me to AmiBroker. It had most of the features I was planning to put into my new platform. Since I wanted to be a trading system developer, not a platform supporter, I gladly bought an AmiBroker license and dropped my own. Over time, AmiBroker was expanded to include most of the pieces I though were missing. I continue to support and recommend AmiBroker as the traditional platform of choice. It is inexpensive, well documented, fast, well supported, etc.

Move to machine learning-based development if you can. There is a learning curve. There is math. A few of my consulting clients are continuing to use AmiBroker. I have moved my own development and trading to machine learning. I recommend Python -- although those who already have substantial experience with, or support for, R might consider it instead.

If you decide to stay with traditional systems, at very least take position sizing out of the trading system and implement a separate trading management system. Reduce position size during drawdowns.

My macro view. It is not scientific. That is, it is not based on evaluation of in-sample data followed by validation using out-of-sample data. That said, I believe we are in the biggest bubble yet seen in all financial classes -- equities, bonds, real estate. I expect we will revisit the 2009 lows, and probably go lower, in all financial classes. I have no way to predict when this will happen, but the trading management system will recognize it when it happens and save my trading account. Holding through a deep drawdown is an act of faith -- not, in my opinion, a good thing.

Best regards,
And thanks for listening, Howard

 

[tech/a]

Thanks Howard
Excellent.

 

[rb250660]

Howard, any suggestions on learning resources for Python and the libraries mentioned? I plan to begin studying this from mid-March. I know your book has teaching resources but I'd like a few resources at my disposal.

Cheers
rb

 

[howardbandy]

Thanks for the kind words.

As I was writing a response, I thought it might deserve its own thread. Please look for "Getting Started in Machine Learning for Trading"
https://www.aussiestockforums.com/t...chine-learning-for-trading.32963/#post-939573

Best, Howard

 

[CanOz]

My very first self coded breakout strategy.....not life altering but it was fun! I think i'm going to enjoy this, its creative...

No slippage or commision and 3 years worth of data....strictly for fun at this stage.

 

[tech/a]

Wow wait until your serious!