Testing a mechanical plan

[stink]

Good morning All,

I have been reading through these forums trying to find some information in regards to testing various trading strategies. I did find quite a few posts by Tech and some others where they talk about testing your methods over 100's or 1000's of trades? This seems to me like papertrading on roids no?

So yeah i would love to hear some ways that you can backtest a plan to gain more confidence in it, i have not gone into the market yet with real $ just papermoney so far but i havent been able to find a way to backtest my plan that i have confidence in. I mean i can get the old data but really in my head i know where the stock is going to end up so its not the same.

Any advice would be greatly appreciated

Cheers Stink

 

[GreatPig]

You can get stock charting software (I use AmiBroker which is very good for the price) which includes backtesting facilities. It requires getting some experience with writing scripts though, so that you can express your system as a mechanical script, and then run that script over historic data.

Helps if you have done some computer programming, but anyone can learn it with a bit of effort. One of the main things to be wary of is that you don't reference forward data in your system, so that you're not cheating by looking into the future. While that might sound obvious, it can sometimes be a little subtle and easy to overlook.

Other popular software for doing this is Metastock, but I gather that's somewhat more expensive than AmiBroker.

Cheers,
GP

 

[stink]

Hi GP,

Thanks for the reply mate, i will have a look into those two you mention. I assume there is no free methods available.

The scripting wont be to much of a problem as i come from an IT background and have played a bit with that sort of thing.

Cheers Stink

 

[mit]

Definitely get some backtesting software. I also use Amibroker and it was a great investment. Then the fun begins because it is extremely easy to curve fit a system to past results.

I think that there are some free ones around but if you were a programmer Amibroker is the go as it is extremely flexible.

MIT

 

[Milk Man]

Exactly what MIT said. I use amibroker and have found it great to use and is the best value for money IMO. Also beware curve fitting or "data mining" as some like to call it.

Good luck. :alcohol:

 

[Magdoran]

Hello stink,


While I wouldn’t want to dissuade you from embarking on a “mechanical/back testing” path if this is your preference, (and you believe this is the direction you want to go in), it may be worth taking a step back conceptually to consider alternative paradigms including analysis styles, and trading psychology, or at least consciously recognise this choice.

Here’s what I’m on about –anyone involved in the market has a tendency to impose their perspective/bias on the limited information they have access too in their analysis. Hence understanding your own bias can be a subtle yet critical element right from the core strategic approach to investing/trading.

So, going down the mechanical/back testing route in effect excludes a whole range of alternative approaches. Many that do this have not considered the trade off they are making. Certainly if you are convinced this is the best approach for you, please, by all means proceed with confidence, people like tech/a have demonstrated that this approach can work consistently and effectively (I stress “can”).

Here’s my controversial perspective on technical analysis: I think that oscillators/moving averages actually obscure the individual’s capacity to really perceive the way markets trend. I would argue that by focussing on formulas and algorithms that a trader becomes reliant on automation, and may cease to really study charts and how markets trade.

I also think that there are discernable market patterns through the ages, and looking at interrelated markets (e.g. currencies and related commodities/indexes) is vital to developing a holistic approach to investment/trading.

My understanding of the back testing regime is that you are assuming that if you run enough numbers through a select piece of history, that this scenario of the past can be usefully extrapolated into the future, and that a positive expectancy result can be achieved by incorporating an algorithm into a mechanical system telling you where to enter and where to exit.

This approach does have merit, and can be quite effective in certain market conditions, particularly in a sustained trending market. Also, if you do go down this path exclusively, mit’s suggestion not to “curve fit” is excellent advice.

There are tradeoffs adopting the mechanical/back test style though. In a way a trader going down this path is surrendering their own inherent direct analytical skills to a “black box (although the better mechanical people at least understand what is in the “black box”).

The problem I see is that the approach is only as good as the backward oriented analysis of a time in the past, and if the trader/investor is looking into the future while imposing their bias on the market (either by machine or by limiting their ability to perceive psychologically, or both), then they are resigning themselves to a limited ability to “see” the market. eddievanhalen made an excellent point on another thread which I fully agree with:

Originally posted by eddievanhalen. Thread: Which one do you use? Technical or fundamental analysis. Post:152

the best traders I know are "discretionary" and have a system based upon experience and some concrete principles which cannot be punched into a system tester and "proven" - they are based upon a huge amount of observation , experience and hard work.

I am in agreement with this philosophy.

So I’m suggesting a path where you retain an awareness of these pitfalls, and consider actively seeking out alternative perspectives so that your future development is not cramped (much like in learning music – incorporate the “theory”, but also nurture the ability to “improvise” and create original music).

Hope this is of interest.


Regards


Magdoran

 

[stink]

HI Magdoran,

Thanks for your reply, you raise some interesting points. I would like to think that i could trade mechanically but still use my head if something just doesnt seem right.

Then again that totally conflicts with the reasons for having a mechanical system

From the research and the training i have done so far the mechanical trading concepts or strategies are more comfortable for me at this stage. I somehow do not feel confident basing my trades purely on fundamentals and psychology i think they definately have their place and are not to be ignored but i would not make decisions soley on this type of information.

What i have noticed so far and have been able to pick up on quite consistently is that the charts dont lie, its no surprise to me that prices drop and come back visiting old support and res etc etc. Sometimes the fundamentals have simply not made sense, there has been no correlation between price pattern and company announcements etc.

Its almost like the market remembers, i understand that this is no way to base trading decisions on soley. For me however it does give me the confidence to make an educated decision knowing time and the odds may be ever so slightly in my favour.

I really appreciate your input and i suppose when it comes down to it, its just a personal choice and you do what you are comfortable with.

Have a nice weekend

Stink

By the way keep the ideas and thoughts coming

 

[mit]

I agree that discretionary traders can make the most money. But why is it cutting out any other trading method?

To me the problem with discretionary trading for a beginner is how do they know that they are a good discretionary trader? Are those string of losses a normal drawdown or is their method crap? I'm sure that prior to May there were a lot of people who thought they were terrific traders.

A mechanical system can show a beginner what works (and can show the crap ideas that sound good to a beginner but never work).

When you run a mechanical system through a period like we have had since May you can compare the draw down against historical drawdowns and keep going if the trading plan is still good. You may not have traded for a long time but a well backtested system gives you a history as though you have traded for 10 years. We have had a 10% drop but there have been many meaner and deeper drops.

My opinion on the best thing for a beginner would be to start a mechanical system, but look at the charts and maybe paper trade your opinion against the systems opinion as confidence builds then start to actually trade in another account.

 

[Magdoran]

Hello mit,


I respect that there are many different schools of thought on any aspect of the market you care to speak of. There are many established debates on questions like fundamental vs technical, whether to use derivatives, which market to trade (sector, country, asset classes – Real estate, equities, commodities, Forex etc, etc).

Then there are countless debates within the technical camp about which approach is best, and where newer players should start. Look, this area is not simple, and making the “right choice” is not easy.

I tend to argue that each individual has their own path to explore, and their innate abilities and preferences should be a guide as to how they learn, but in the context of offering real choices, and being mindful of allowing the potential of innovating with a smorgasbord of concepts in the future.

Where I offer an alternative perspective is by taking issue with the notion that new technical analysts should necessarily start with the mechanical approach. I’m suggesting that this approach can actually cramp future development, much like imposing too rigorous an approach to learning to play music where the creative side is stifled, and a rote learned pattern is imposed.

mit, do you play a musical instrument for example? If you do, can you improvise a melody on the spot? – Many musicians can’t “jam” if they were trained to rote learn pieces, and just sight read – and many of these people have also been rendered creatively impotent to composing music of their own. They are in effect slaves to the stave.

In much the same way, many traders have become slaves to the black box, and have lost the ability to develop creatively beyond their confined “prison of the mind”, and grow beyond the limits of a necessarily finite system.

My argument stems in part from observation of how people learn and develop. There is a mental divide in the way people process information – the creative side of the brain and the analytical side.

I think the best traders I know use their creative side to reflect the subliminal possibilities of the future, and once this is done, then engage their analytical side to do the practical calculations. In effect inspiration provides the impetus to see an opportunity, then the perspiration is in actually planning and executing the trade – two different mental processes aligned in harmony (based on the idea one tenth inspiration, nine tenths perspiration – but you need the inspiration).

My belief (reads my personal opinion, hence not an objective fact) is that many technical analysts put the proverbial cart before the horse by using their analytical side solely for making forecasts, when they may find that the imagination of what the future could look like (and the subliminal ability to recognise complex patterns) is effectively severed (I call this “technical castration”).

The fallacy is that a machine can see the market better than a human can. Maybe this is true when the probabilities are finite (like the super powered chess computers for instance), but I’ve yet to see a PC be anything near a human when it comes to free thinking and creativity in the wider context – that day may come, and when/if it does, it’s game over for any of these discussions.

By sacrificing the creative ability to really interpret charts (which I would have thought is what a “Technical Analyst” is supposed to be in the pure sense); I think this is akin to the way newspeak was imposed in George Orwell’s book “1984”.

In essence, I think that a narrow diet of “mechanical/back testing” has a potential to mortgage the future of an aspiring technical analyst, conditioning them to think inside the square, with little chance of ever “spreading their wings” to achieve their potential. But this is not always so, and some traders thrive using the mechanical approach, and I respect their choice. What I object to is when this is foisted on new technical analysts as the only alternative.

As for paper trading, mit, excellent point, I’m all for it. I think it takes at good 3 years to become adept at technical analysis, and small positions is the way to go until a new trader/investor has had the chance to study the market enough to make some informed financial decisions.


Regards,


Magdoran

 

[Magdoran]

Mit,


By the way, don’t get me wrong, you raised some really good points…

It is very difficult to become a good technical analyst per se, discretionary or mechanical.

You raised a very relevant point about how is a new trader to learn, and where we agree is by studying the past (although I’d say look at the charts with a range of different theories, and not just with a back testing formula – actually looking physically at the charts - daily, weekly, monthly, in as many different markets as possible, right back to early data record up to the present).

The other point you make is about the actual trading plan approach about entries and exits (money management), and this is a very relevant component in effectively trading, but an area I was not addressing in the last post - this was specifically about interpreting charts, not about the planning and execution, which are of course critical in themselves.

As for the May pull back, using technical analysis, many people identified that the trend was at risk, and had exited their longs by the end of April (I certainly did). The more aggressive players actually went short, or entered a sideways strategy… that’s the advantage of not using lagging indicators.

I also agree that using the back test approach as one of several methods can quickly unearth poor strategies for beginners as you suggest. This can be valuable, as can any examination of events and how different strategies and approaches may have fared.

But re discretionary trading, you may not be aware, but in this camp there are many divisions of thinking too, just like technical vs fundamental, and mechanical vs discretionary.

What I’m suggesting is almost a step back from this, which is about how each person views what they see, and how to manage your own market bias. I hope the subtlety of the point isn’t lost in the text…


Warm Regards


Magdoran

 

[Magdoran]

Hello stink,


Spot on, there is a potential contradiction, and I hope my posts above flesh this out a bit for you.

As for lines of support and resistance, I do think markets can trend in discernable patterns which can allow for forecasting in time and price. Beware though, have a look at Mark Douglas’ book “Trading in the Zone” which will really make you think about an overall approach, probably the sooner the better really.

The stage you are at though is trying to sift through all the alternative technical analysis schools, and later the intra school refinements as you develop.

Consider reading through all the technical analysis threads you can find, get all the books you can and read them too, discuss aspects with as many traders as you can, and keep your mind open, but also be discerning, and learn to trust your own abilities (in essence nurture these).

There are many schools of technical thought around – please do explore tech/a’s approach, look at the Elliott Schools (Prechter and Frost have some interesting interpretations), there’s Stan Weinstein, Larry Williams, Jack Schwager, Stephen Bigalow, Martin Pring, W.D. Gann, R.N. Elliott, the list is endless… I’m sure that there are many more names that other “techies” can throw in – these are off the top of my head, and I personally both agree and disagree with many aspects of these “big names”.

But here’s the biggest advice I can give you. Be patient with yourself, and invest in your own knowledge. Accept that this is a very complex and difficult field, and expect to fail initially while you learn and grow. You will make mistakes, but what is most important is how you will deal with them. George Soros talks about this in “Soros on Soros” - he didn’t worry about making mistakes, - he saw this as normal, what he did was to focus on recognising and correcting them.

Good luck!


Regards


Magdoran

 

[MichaelD]

There are some very interesting points of view here, so I'll add mine into the mix as a mechanical trader.

I mechanically trade because many, many of my preconceptions of what I thought would work in the marketplace have been proven incorrect with backtesting and thus unprofitable/harmful to profitability. Trading mechanically ensures that I am exposed to the market via a profitable methodology whilst I figure out "what's really going on".

I do not see it as limiting my growth at all, but merely ensuring my survival in the meantime.

Or to put this another way, I see discretionary trading well as the ultimate goal of a trader's journey;

Start of journey: Discretionary trading - unconscious incompetence. Buy now and wait for the one way elevator. Money is lost, but there is no conception of why money is lost. Most fail here.

Next step: Mechanical/system trading - conscious incompentence to conscious competence. There's more to it than just buying anything and everything. Survival is learnt and money begins to be made. This allows the trader to survive until they reach the final step. Most profitable traders do not progress beyond this point.

Final step: Discretionary trading again - unconscious competence. The trader is in tune with what the market is saying and trades accordingly. Only the elite traders get to this stage.

 

[mit]

*** this was to your first reply. Just noticed a few posts since. I'll leave the body as is ***

Magdoran,

I think you have a very narrow view of mechanical trading. Admitedly there are people who can trade from day one. However, the failure rate from trading is often quoted at between 90% to 98%. There are those who never should trade. There are those that I think that can be successful but give up after a string of losses. Now the string of losses could be part of a trading plan that has positive expectancy, but as a beginner unless you have somebody holding your hand, a string of losses is very hard to take and it is tempting to give up.

Mechanical trading is not "black box". Black Box is being handed a trading plan and told to trade it. With backtesting you can see the effect of widening or shrinking stops. The effect of different position sizing methods. The effects of taking profits or using trailing stop. You can also test a lot of things you see in books which don't work.

Apart from just systems it can tell you such things as whether in general it is better to buy/sell on open/close. Sell at a stop or at EOD. The cost of trading CFDs against margin. In fact it gives you a bag of tools handy for a discretionary trader.

My point that it could get the number of failures down. It allows somebody to try things without blowing up their account a few times. I think it would only narrow your view if you were that kind of person already. I think we see enough of that kind of person here and other forums for all trading styles.

MIT

ps. I do play piano and guitar. I learn music by rote and no I can't improvise. But it wouldn't have any difference as I couldn't improvise before. Lack of talent unfortunately.

 

[mit]

Magdoran,

I see your points better now and agree. I still greedily read everything I can on different styles of trading and investing. Learning never stops. Also, nothing annoys me more than some threads you see where people say that method x does not work but their method y does work. It's surprising how people who have traded for years fall into the trap of a very narrow view (although the fundie/tech wars are always fun).

MichaelD,

We are on the same wavelength. I'm looking to make enough capital to leave work and study this stuff full time.

MIT

 

[Magdoran]

Hi mit,

Originally posted by mit

ps. I do play piano and guitar. I learn music by rote and no I can't improvise. But it wouldn't have any difference as I couldn't improvise before. Lack of talent unfortunately.

Really, so you play, that’s great! I used to play guitar in a few bands years ago, and even taught a bit too… I did a lot of improvising, and helped quite a few musicians to learn how to “jam”… a bit rusty now though, and probably going slowly deaf… what’s that??? You know, you actually might have some hidden talent. Can you hear pieces in your head for example? Anyway, back to the main topic.

The problem with this field is that we don’t have any universal benchmarks for terminology, and where there are some, there are still polemics raging about the finer points.

Maybe our definitions of what mechanical trading is are different – good point.

Originally posted by mit

With backtesting you can see the effect of widening or shrinking stops. The effect of different position sizing methods. The effects of taking profits or using trailing stop. You can also test a lot of things you see in books which don't work.

Oh, I agree with this approach, very worthwhile, but I’d class this as developing money management parameters (position sizing), so maybe our labels are different…


Ok, let’s try to clear up the semantics – would you mind illustrating what you know the term “mechanical” to mean please? Sorry for the 50 questions, but this will help me understand where you are coming from, if you don’t mind of course…

What triggers your entry/exit?

Do you forecast price for entries and exits?

Do you forecast time for entries and exits?

Do you interpret bar/volume in you process?

Do you use any kind of wave structure analysis in your mechanical system?

Does your system define the stop for you, or do you actually use a method of interpretation of the chart to do this (i.e. use a technical stop)?

How do you determine the trend?

I could go on, but don’t want to burden you too much… you might be surprised by the range of ways you can interpret a chart.


Regards


Magdoran

 

[Magdoran]

Hi MichaelD,


I like the way you illustrated the technical analyst’s journey. It really puts the whole experience in perspective, doesn’t it?

Examining the past and assessing the best approaches to certain conditions makes a lot of sense to me, in fact I’d say it is vital.

The key point is that you’re actively feeling forward from the brink (ala Bronowski)… as am I.


I agree, this is the “way of the trader”.


Kind Regards


Magdoran

 

[mit]

Hi mit,




Really, so you play, that’s great! I used to play guitar in a few bands years ago, and even taught a bit too… I did a lot of improvising, and helped quite a few musicians to learn how to “jam”… a bit rusty now though, and probably going slowly deaf… what’s that??? You know, you actually might have some hidden talent. Can you hear pieces in your head for example?

Yes, guitar especially. If I can't hear it in my head it is almost impossible to play. I also find it hard to keep time without hearing it in my head

For below. I am describing one mechanical system and the golden rule with mechanical systems is to keep it simple. I do have a very small account with about 5% of my capital where I trade discretionary.

What triggers your entry/exit?

If you mean the system then:
Entry - Distance from Dividend, amount of Dividend and currently bullish
Exit - Ex Div Day

If you mean how I get a trigger:
I use a spreadsheet and load data in from Amibroker, Don's Dividend sheet and live prices.

Do you forecast price for entries and exits?

No

Do you forecast time for entries and exits?

exit at exdiv unless stopped out.

Do you interpret bar/volume in you process?

Liquidity has to be high enough. I find anything averaging below $400k turnover per day not worth the hassle and will usually give me slippage between the backtested system and what actually happens.

I don't put stops in the market and will look at the day's volume if a stop is hit and the share recovers (This is the only discretionary element, but it is very rare that I'll override but you do get kangaroo tails)

Do you use any kind of wave structure analysis in your mechanical system?

Pretty hard to code. Kind of thing I'd like to try to code when I retire. More because I'd like to try and code it not that I'd think it would add anything, I think these kind of stuff plus support and resistance, 1-2-3 patterns and fibs work better straight off charts. You might be able to code the patterns but there are a lot of little judgement things here.

Does your system define the stop for you, or do you actually use a method of interpretation of the chart to do this (i.e. use a technical stop)?

Mechanical stop based on an ATR multiple. Trail it up based on the High of the day. After last October I positioned sized based on the ATR multiple but I used the most recent low as an actual stop if the low was higher than ATR stop. Back testing over October it didn't make much difference actually.

How do you determine the trend?

For this system I use a fast and slow EMA (10/20) and just look for the fast EMA to be higher than the slow EMA.

 

[stink]

Ha what a great Forum,

I seem to have hit a nerve that i in my infancy of trading am yet to develop

You guys really seem to know your **** and hopefully one day i can revisit this post and add my two cents.

Back to the topic though, the best way to backtest a mechanical plan is by using something like the software that has been suggested.

The way i see it,
I am a noob to trading so the only way i can verify if my trading plan (mechanical at this stage) has some potential, is to use some sort of backtest solution. I totally agree as stated earlier that there are probably hundreds of valid trading methods but for now in my position i see a mchanical backtest the only way?

penny for your thoughts!

Cheers Stink

 

[Mr Hurricane]

Hi Stink,
I created an Fx system using the back testing facilities incorporated in Trade Station. I had traded for some time and was aware of a good setup for a trade but it only happened about once a week. That is a long time to sit and wait for the right trade. So I automated the idea and back tested it over 3 years to see how good the idea was. It worked well and I would recommend back testing your ideas to see how they stand up over long periods.

Mr Hurricane

 

[It's Snake Pliskin]

Magdoran,

Hence understanding your own bias can be a subtle yet critical element right from the core strategic approach to investing/trading.

So, going down the mechanical/back testing route in effect excludes a whole range of alternative approaches. Many that do this have not considered the trade off they are making.

Here’s my controversial perspective on technical analysis: I think that oscillators/moving averages actually obscure the individual’s capacity to really perceive the way markets trend. I would argue that by focussing on formulas and algorithms that a trader becomes reliant on automation, and may cease to really study charts and how markets trade.

I also think that there are discernable market patterns through the ages, and looking at interrelated markets (e.g. currencies and related commodities/indexes) is vital to developing a holistic approach to investment/trading.

There are tradeoffs adopting the mechanical/back test style though. In a way a trader going down this path is surrendering their own inherent direct analytical skills to a “black box (although the better mechanical people at least understand what is in the “black box”).

The problem I see is that the approach is only as good as the backward oriented analysis of a time in the past, and if the trader/investor is looking into the future while imposing their bias on the market (either by machine or by limiting their ability to perceive psychologically, or both), then they are resigning themselves to a limited ability to “see” the market. eddievanhalen made an excellent point on another thread which I fully agree with:

I am in agreement with this philosophy.

Good stuff! and Eddie said it well.

Mit,

To me the problem with discretionary trading for a beginner is how do they know that they are a good discretionary trader? Are those string of losses a normal drawdown or is their method crap? I'm sure that prior to May there were a lot of people who thought they were terrific traders.

Results and psychology!

Yes bullmarket champions.

Stink,

You guys really seem to know your **** and hopefully one day i can revisit this post and add my two cents.

Always keep a sceptical mind to what you read or hear - in general.

I am a noob to trading so the only way i can verify if my trading plan (mechanical at this stage) has some potential, is to use some sort of backtest solution. I totally agree as stated earlier that there are probably hundreds of valid trading methods but for now in my position i see a mchanical backtest the only way

I see mechanical trading better for people who don`t have the time or psychology to deal with discretionary trading. I`m with Magdoran and the latter though if you have the time to make a killing.

Doctors go to university to study for years and they never stop. If one wants to trade should just reading one or two books be enough? NO!

Snake