# Inspired For Backtesting



## chops_a_must (6 December 2006)

I have decided to heed the advice of good old Rummy,

"Plan backwards as well as forward. Set objectives and trace back to see how to achieve them. You may find that no path can get you there. Plan forward to see where your steps will take you, which may not be clear or intuitive."

However, I'm hoping you can make things seem a little clearer for me than poor Donald did, and if you can offer me tips/ free software for backtesting, that would be great.

Cheers in advance,
Chops.


----------



## tech/a (6 December 2006)

Best value for money is Amibroker.($400 approx)
I use Metastock coupled with Tradesim ($2,500 approx).
There is also Wealthlab ($900 approx).

Then look forward to around a year to master and a year to design test and implement a good trading methodology.While the path is long the learning curve will be vertical.


----------



## stevo (6 December 2006)

Chops
I don't know of any decent free software for portfolio backtesting - don't waste your time. EOD Amibroker is $149 US, and as tech mentions is very good. 

If you are serious about backtesting I would go for AmiBroker. I also use Amibroker combined with TradeSim. Tradesim is excellent. You can do Monte Carlo analysis with Amibroker and Excel as well even though it's a little slow. If you don't already know how to code in Amibroker then you will need to spend some time understanding the software and language by running through the tutorials etc

Some suggestions and ideas to get you started;
1. For starters select a universe of stocks to test on, but don't start with too few - at least 400 to 500 stocks. I use the All Ords stocks for basic testing. You should expand/ alter the universe if you think you have a good system.

2. Don't get bogged down in one time frame. Do some simple tests on daily, weekly and monthly time frames to see which one is easiest to work with. Stick to trading long strategies for starters - you will get confused if you try testing shorting strategies initially.

3. Test the sorts of things that are pushed in trading books all the time. Try out MACD, RSI, Stochastics, Bollinger Bands Donchian channels etc using the methods outlined in trading books. Try the different time frames and see what you think of the results. 

4. Try OB /OS style indictators but also reverse the strategy - buy when overbought, sell when oversold. As an example take bollinger bands in a weekly timeframe using the traditional OB/ OS approach then reverse it. Add a trailing stop.

5. Settle on a timeframe (daily / weekly / monthly) that you are most comfortable with for your first system.

6. Take some of the ideas that you have obtained from testing the standard indicators and start experimenting with trailing stops. Try moving averages, donchian channels, ATR trailing stops and anything else you want to throw into the pot.

7. Stick with simple position sizing strategies for starters - 10% of equity or $10,000 per position on $100,000 capital. You can expand on these strategies later.

8. Ideally keep track of the results obtained in a simple spreadsheet or even hardcopy printouts - giving enough detail so you can repeat the tests in the future.

9. Don't worry about walk-forward testing until you have something that you think has a chance. 

10. The same goes for optimisation. Just stick to the usual periods touted for indicators, or if you want to change them use something like known number sequences for convenience - 1, 3, 5, 8, 13, 21, 34, 55 etc. Use any sequence you like - they are not magical.

11. Always delay your trades. For EOD trade the next day, not on the close of the current day, EOW trade in the next week not the close of the week, etc.

This will probably take around 6 months maybe longer depending  on your skills with Amibroker. I really don't have any comprehensive references on backtesting other than maybe the TradeSim manual. Leon Wilson's book - Breakthrough Trading could be useful. Obviously the online help for Amibroker is essential and there is a lot of stuff on the Amibroker Yahoo site.

Also - get decent data! Free data is useless for serious portfolio backtesting.
regards


----------



## Bobby (8 December 2006)

stevo said:
			
		

> Chops
> I don't know of any decent free software for portfolio backtesting - don't waste your time. EOD Amibroker is $149 US, and as tech mentions is very good.
> 
> If you are serious about backtesting I would go for AmiBroker. I also use Amibroker combined with TradeSim. Tradesim is excellent. You can do Monte Carlo analysis with Amibroker and Excel as well even though it's a little slow. If you don't already know how to code in Amibroker then you will need to spend some time understanding the software and language by running through the tutorials etc
> ...



Stevo an excellent  post with  premise.

Regards Bob.


----------



## It's Snake Pliskin (8 December 2006)

chops_a_must said:
			
		

> I have decided to heed the advice of good old Rummy,
> 
> "Plan backwards as well as forward. Set objectives and trace back to see how to achieve them. You may find that no path can get you there. Plan forward to see where your steps will take you, which may not be clear or intuitive."
> 
> ...


----------



## tech/a (9 December 2006)

Just to add to some of Stevo's excellent post from a slightly different angle.

(1) *What is the purpose of developing a System of trading?*

Its to have a set of parameters which when applied to a universe of stock or even singular instruments over a period of time returns a positive expectancy.
IE you return more than you lose---consistently.

(2) *I'm confused Does the method have to work on ALL of the trades I take?*

No infact you wont know at the time that you take a trade wether that trade will end profitably.The purpose of trading a system which is known to have a positive expectancy is that over a great number of trades the winners- the losers will return a profit.

(3) *Do I have to take EVERY single entry triggered by my system?*

No ----Montecarlo analysis will show you that a profitable method will work on ANY of the trades triggered. All systems I know of will trigger 10-50 times MORE trades than you can possibly take. Montecarlo testing is simply testing 1000s of combinations of triggered trades and returning the net results of the groups of stocks traded over X time.

Its like me designing a system then calling 20000 people giving them each my capital base (Say $100,000 each) the rules of the system and asking them all to trade it over the next X years---record the results then come back to me with them.

You would find (with a great system---one I would trade) that all would come back with varying positive results.There will be a low extreme and a high extreme and 1000s in the middle (or Mean).

With a bad system (One I wouldnt trade) you would find perhaps many winning participants and some losers.---back to the drawing board.

This is the problem with developing a system that you can only test in singularity.You just dont know wether you would have the same profits or even a loss had you started the system on a different date OR with a different set of trades--OR both. Montecarlo simulation removes this doubt.

In summary a great system will be profitable over time regardless of the stock traded OR when you trade it.

(4) *But testing a system over PAST data---how can I have confidence that in FUTURE it will perform the same as it has in the past?*

Bottom line is you dont. Walk forward testing will help,but is very slow and tedious (try doing that with 20000 portfolios!!).
The way I have answered this question *SO FAR * is that all systems will have a "Blueprint" or a set of NUMBERS that are characteristic of that method.
They have max and minimum numbers.If your trading remains within the Blueprint then you WILL recieve the returns that the System has returned in its blueprint.

If it trades outside of the blueprint then you would stop trading the system as market conditions would be such that they are *outside ANY TESTED conditions and as such are UNKNOWN.*

(5) *Should my system work in ALL markets---Bull or Bear*

If its designed for both then YES. If its designed for bull markets then NO.
All my trend following systems perform poorly in Bearish conditions and very well in bullish---thats what they are designed for. I am currently looking a "Switches" for turning on and off systems in times of obvious market conditions which are not "ideal"---results are mixed at this point but encouraging enough to pursue.


In the next post I'll just add to some of Steves comments and discuss some things I have found.


----------



## tech/a (9 December 2006)

stevo said:
			
		

> Chops
> I don't know of any decent free software for portfolio backtesting - don't waste your time. EOD Amibroker is $149 US, and as tech mentions is very good.
> 
> If you are serious about backtesting I would go for AmiBroker. I also use Amibroker combined with TradeSim. Tradesim is excellent. You can do Monte Carlo analysis with Amibroker and Excel as well even though it's a little slow. If you don't already know how to code in Amibroker then you will need to spend some time understanding the software and language by running through the tutorials etc




Best I have seen so far however still lacking in many areas which one day maybe overcome. IE If I switch a method off today and re enter exactly the same portfolio in 3 mths taking taxation in consideration---then how will it perform?--there are many more but this is just one example.

*Some suggestions and ideas to get you started;*



> 1. For starters select a universe of stocks to test on, but don't start with too few - at least 400 to 500 stocks. I use the All Ords stocks for basic testing. You should expand/ alter the universe if you think you have a good system.




We differ a little here. I feel its much easier to design a system on a SINGULAR stock or entity---IE and index or future.
However the system would be specifically designed around IT or the Universe you select.T/T is based upon the BT margin list and performs worse on the whole of the ASX,it performs best on the Hong Kong exchange!!?



> 2. Don't get bogged down in one time frame. Do some simple tests on daily, weekly and monthly time frames to see which one is easiest to work with. Stick to trading long strategies for starters - you will get confused if you try testing shorting strategies initially.




Strongly agree.



> 3. Test the sorts of things that are pushed in trading books all the time. Try out MACD, RSI, Stochastics, Bollinger Bands Donchian channels etc using the methods outlined in trading books. Try the different time frames and see what you think of the results.




This is the best place for Oscillators in my veiw,however I personally am not a great fan of them I prefer to use price action as much as possible. 



> 4. Try OB /OS style indictators but also reverse the strategy - buy when overbought, sell when oversold. As an example take bollinger bands in a weekly timeframe using the traditional OB/ OS approach then reverse it. Add a trailing stop.




I have found that simply reversing a trading method from long to short when designed for long side trading to be less than successful. Stock tends to act very differently when falling,than climbing. My veiw is that short side should be handled ALONE.  But as I dont trade short,Ive not looked deeply into this way of trading. The old 80/20 rule--- the market is 80% bullish so thats where I'll trade---a personal thing only.



> 5. Settle on a timeframe (daily / weekly / monthly) that you are most comfortable with for your first system.




While I trade systems in Daily timeframe Steves system of Weekly is more profitable and *easier in my veiw to design*.I think weekly is the highest timeframe that I would use.



> 6. Take some of the ideas that you have obtained from testing the standard indicators and start experimenting with trailing stops. Try moving averages, donchian channels, ATR trailing stops and anything else you want to throw into the pot.




Ive not yet been able to code throwing a trailing stop in if a move of X is achieved over Y period IE a very quick positive rise of 50% over a few days to a week---thats the only time Id use a trailing stop because if you use it any sooner it becomes the EXIT!,



> 7. Stick with simple position sizing strategies for starters - 10% of equity or $10,000 per position on $100,000 capital. You can expand on these strategies later.




Agree however slight cahnges can make dramatic changes to result.
IE I have found 8 positions to be open at the one time in a portfolio to be the optimum---or 12.5% of capital. I have found stops over 10% to weaken the equity curve---IE it isnt as smooth. The optimum stop regardles of how its placed appears to be 8-12% from the purchase price.
So If you designed a mechanical stop which was consistantly inside or ourside these parameters than I would suggest it could be having a negative impact on result.



> 8. Ideally keep track of the results obtained in a simple spreadsheet or even hardcopy printouts - giving enough detail so you can repeat the tests in the future.




Great advise---Im slack I dont do it as well as I should.--time factor and obvious profit is my weak excuse!



> 9. Don't worry about walk-forward testing until you have something that you think has a chance.




I agree here but is covered above and in particular under the info on Montecarlo testing---without it I strongly believe you could well have a system that breaksdown 5% of the time and NOT know it no matter how often you walk forward test. 



> 10. The same goes for optimisation. Just stick to the usual periods touted for indicators, or if you want to change them use something like known number sequences for convenience - 1, 3, 5, 8, 13, 21, 34, 55 etc. Use any sequence you like - they are not magical.




My veiw is that optimisation has its place with singular entities and perhaps very very small universes which are in someway correlated. To optimise un correlated universes will be a pointless exercise.To prove this to yourself run the optimisation 6 mths ago and now---they will be entirely DIFFERENT.
Not what you want.



> 11. Always delay your trades. For EOD trade the next day, not on the close of the current day, EOW trade in the next week not the close of the week, etc.




This is the BIGGEST mistake newbie systems testers make.Makes enourmious differences to results.often turning profitable systems into disasters.



> This will probably take around 6 months maybe longer depending  on your skills with Amibroker. I really don't have any comprehensive references on backtesting other than maybe the TradeSim manual. Leon Wilson's book - Breakthrough Trading could be useful. Obviously the online help for Amibroker is essential and there is a lot of stuff on the Amibroker Yahoo site.




Longer I think---but maybe Im slow on the up take after all Im only a Duck!

Ill post the name of my systems testing BIBLE when I get home,a grasp of Mathematics is a great help. You dont have to be a genius---Im a duck who builds things---cant be that hard!



> Also - get decent data! Free data is useless for serious portfolio backtesting.
> regards




Crapp data crapp results. Pay for good software and good data---oneday you expect to make a motzza---if your undercapitalised and cant afford the above---what are you doing trading????????


----------



## CanOz (9 December 2006)

chops_a_must said:
			
		

> I have decided to heed the advice of good old Rummy,
> 
> "Plan backwards as well as forward. Set objectives and trace back to see how to achieve them. You may find that no path can get you there. Plan forward to see where your steps will take you, which may not be clear or intuitive."
> 
> ...





Me too Chops, i posted some stuff on the other thread on backtesting started by snake, maybe the moderators would move my comments to this thread as its all related. Nizar is quite interested as well.

Cheers,


----------



## nizar (9 December 2006)

tech and stevo thanks for taking the time.
Excellent discussion here.


----------



## nizar (9 December 2006)

CanOz said:
			
		

> Me too Chops, i posted some stuff on the other thread on backtesting started by snake, maybe the moderators would move my comments to this thread as its all related. Nizar is quite interested as well.
> 
> Cheers,




Yes im very interested.

tech and stevo - TradeSim there are standard ($200), professional ($480) and enterprise editions ($1,500) available. WHich one do you guys use?

so tradesim enhances the backtesting capabilities of metastock/amibroker?
Looks good.

Also - What live data feed do you use?


----------



## tech/a (9 December 2006)

We both have Enterprise edition.

Live feed is Marketcast for me. I cannot use Marketcast with any of the systems software.
To do that I would have to have Metastock pro.


----------



## tech/a (9 December 2006)

More backtest info here.

https://www.aussiestockforums.com/forums/showthread.php?t=5157


----------



## chops_a_must (9 December 2006)

It's Snake Pliskin said:
			
		

>






> If it trades outside of the blueprint then you would stop trading the system as market conditions would be such that they are outside ANY TESTED conditions and as such are UNKNOWN.




Well, the philosophy behind my backtesting is about limiting this because:

"Reports that say that something hasn't happened are always interesting to me because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know."

It's all about reducing the known unknowns, and the unknown unknowns.


----------



## tech/a (9 December 2006)

So then really until the new unknown becomes the now known it is best to suspend the known when it is in unknown territory.

Thats something which I have yet to know as the known is and has stayed within the known.


----------



## Bobby (9 December 2006)

tech/a said:
			
		

> So then really until the new unknown becomes the now known it is best to suspend the known when it is in unknown territory.
> 
> Thats something which I have yet to know as the known is and has stayed within the known.



*Yes!*  Now I know what you mean


----------



## chops_a_must (10 December 2006)

tech/a said:
			
		

> So then really until the new unknown becomes the now known it is best to suspend the known when it is in unknown territory.
> 
> Thats something which I have yet to know as the known is and has stayed within the known.



Bahaha!

Now you're getting it!


----------



## Bin57again (19 December 2006)

Tech/Stevo
Good advice. Thanks.
I'm struggling with system development at the moment. My biggest problem is getting comfortable with the system I'm trading. I have 2 specific problems. Can you guys give me a hand please?:

1. Entry signals
Because MA crossovers or oscillator systems or other well known mechanical set-ups throw up so many signals, when I review the trade reports, I think to myself "how do I eliminate those losing trades?". I remind myself to focus on expectancy not on being right but the pull of improving entries is always with me. You know what I'm going to say next - off I go to the Holy Land to find the Holy Grail. I never find a way to improve the entries. Do you guys just accept that on some days you'll look at a signal and think "gees, I'm not sure about this but anyway, my rules say enter so I'll enter?"

2. Exits/stops, etc
Mark Douglas, Van Tharp et al all say that systems are judged by their exits, etc. Seems good advice. How can you argue when such authorities say "know your risk, know when to take profits, do it consistently, it's just a numbers game?" You'd be mad not to take their advice. But then, there you are with your trend following system with 50 trades in a year, with a boat load of commissions, heavy tax bill and only parts of a move, when Jimmy FinReviewMan next door who knows F All except what Kohler says has his ANZ, BHP, etc for a year and nets 30%+. Sure, OneTel and TLS, but a well diversified portfolio of quality ASX stocks looks great.
I want to be a trader so bad but I'm not getting something. What am I missing? 

Best regards
Bin


----------



## MichaelD (19 December 2006)

Bin57again said:
			
		

> 1. Entry signals
> Some days you'll look at a signal and think "gees, I'm not sure about this but anyway, my rules say enter so I'll enter?"
> 
> 2. Exits/stops, etc
> There you are with your trend following system with 50 trades in a year, with a boat load of commissions, heavy tax bill and only parts of a move, when Jimmy FinReviewMan next door who knows F All except what Kohler says has his ANZ, BHP, etc for a year and nets 30%+.



1. Rules are rules for a good reason. It's very difficult to improve on a system by using discretionary trade filters. Most of my biggest/best trades were the crappiest looking charts on entry. Conversely, the "can't lose" charts usually do lose. One added entry filter which seems to help many long systems (but test your system with and without it) is CLOSE>OPEN.

2. Because a. that's not what happens every year, and b. buy/hold forgoes considerable opportunity cost. However, if your system is not beating buy/hold it is not yet ready to be traded.

(ps systems based on MA crossovers or oscillators generally perform very poorly due to whipsawing.)


----------



## nizar (19 December 2006)

Bin57again said:
			
		

> Tech/Stevo
> Good advice. Thanks.
> I'm struggling with system development at the moment. My biggest problem is getting comfortable with the system I'm trading. I have 2 specific problems. Can you guys give me a hand please?:
> 
> ...




Just adding to what Michael has already covered. If, after commissions, taxes and other costs, you cannot beat buy and hold, which in some cases will be 30%+, then your system needs to be modifying.

Another interesting point id like to make is that when you have a system with a positive expectancy over a large number of trades, by backtesting, trading, or even papertrading, you have to make sure *you take enough trades * for that expectancy to hold true.

Just like how the chances of getting heads or tails is 50/50. Over 10 flips of the coin you may well get 7 or 8 heads. But over 1000 flips, likely to give around 50% of each.

I read an interesting analogy in Van Tharps book. About a game where you have a 60% of doubling your money (or 40% chance of losing it. Double or nothing). 40 participants were given $1000 each, and told to play the game as many times as they liked over a given time frame. At the end of it they all lost money. Why? Position size was too big. Typically, they would bet $100. Loss. And then 2 or 3 more losses.. Damn down to 600 or 700 now, due for a win, so LETS WIN BIG. Then they put 300 or 400. Another loss and they are pretty much gone. Very slim chances of winning now. $10 position sizes and they would easily make money. (Of course these guys didnt pay brokerage). And it seems to be the beginner trader (or should i say gambler/punter) mentality that your chances of winning increases after each successive loss. Ie. 3 losses in a row, good chance the next one is winner! 

Money management is so so key to successful trading.


----------



## Bin57again (19 December 2006)

Nizar
How long did it take you to develop systems which perform better than 30%? My edge is pretty poor at around 11 or 12%. What made a difference? Wider market filters or exits, etc? Could you recommend any books on systems development? I read a couple by Lars Kestner and all the usual ones by Tharp et al. 
Thanks, Bin


----------



## MichaelD (19 December 2006)

Bin57again said:
			
		

> How long did it take you to develop systems which perform better than 30%?



There's more to trading profitably as a business than just the basic entry/exit system. There's also the money managment side of the business, and the use of other people's money (later) to boost returns. It's also fallacious to look at system return as a % as the return will vary depending on market conditions. It's more important to consider system drawdown when designing a system.

It took me 6 months of fairly solid system development before I settled on my current robust long term trend following system.

I'm now 3 months into developing a short term breakout system and am just starting to see robust results from this effort.



			
				Bin57again said:
			
		

> My edge is pretty poor at around 11 or 12%. What made a difference? Wider market filters or exits, etc?



Depends on your time frame but if you target long term trend following as the best place to start developing a system, then you should start with Random Entry and a wide trailing stop (eg wide ATR stop or long term moving average stop) and go from there.

Once you've established an exit with random entry that outperforms buy and hold then you can refine the system from there. Don't be too quick to whack all sorts of entry filters into a system - they'll often do more harm than good.


----------



## weird (19 December 2006)

Michael, do you see any issues with a "Random Entry", and comparing backtesting results ?


----------



## nizar (20 December 2006)

The fact that random entry works speaks volumes about what is required to develop a profitable trading system.

Money management ie. position sizing, risk management ie. proficient use of stops, and exits, are the most important components of a trading system. Entry probably the least important, yet everyone focuses on the perfect entry.

Notice how the most used trading cliche: cut your losses fast and let your winners run, is all to do with the exit.

Bin57 - Im still developing a trading system. Started really about a month ago. Last months results were 38% winners over 29 trades. R/R=2.43. Still alot of work to do. My main weakness at this stage (one of several) is not really knowing when to move up my stop and to what level. This part is too discretionary for my liking, but maybe it has to be. Because of this flaw, At times im allowing for far too much profit giveback. Like i said, still alot of work to be done.

As for books, tech has mentioned a couple:
*The trading game: Ryan Jones (1999)
*Stock patterns for daytrading: Barry Rudd (1999)
*Trading systems and methods: Perry Kaufman (1998)


----------



## MichaelD (20 December 2006)

weird said:
			
		

> Michael, do you see any issues with a "Random Entry", and comparing backtesting results ?



No, there are no issues with a random entry. It should be considered the "gold standard" when developing a system. MOST systems will not perform as well as a system based on random entry, a powerful indicator of where system development effort should be concentrated. If you can improve upon random entry, you are doing well, but you need to have these results as a comparison baseline to know if you are doing good or harm with your extra filters.

A couple of points worth mentioning, however, are;

1. With random entry you'll get more entry signals than you have capital to trade, so it is essential to apply Monte Carlo analysis to get a distribution of results rather than a single figure for a system.

2. When comparing system results, you should not focus on % profit, but % drawdown as the important comparator. i.e. if two systems have much the same profit % but one has significantly less drawdown %, then the lower drawdown system is superior. One very useful technique is to normalize all system results to the same drawdown % and then compare profit % to get an idea of which is the best system.


----------



## weird (20 December 2006)

I would think with a random entry, it could be difficult to determine a sample size required for significance in statistical results.

I would be interested if anyone wishes to calculate the possible entry combinations when using a random entry , picking any stock from the ASX300, for a testing period of 5 years, using daily bars?


----------



## stevo (20 December 2006)

Bin
Good questions. I can give some ideas but really it is a matter of testing anything that appears to make sense, and some things that don't. 

Some ideas;
1. Have you looked at different time frames? 
2. Try using an index to determine times when it could be good to stand aside.
3. As an exercise look at a monthly time frame. Try something like enter when the stochastic goes above 50 and exit when it drops below 50. Add some things and see what happens. Try it with RSI, Bollinger bands, moving averages. Do the same in weekly and daily!
4. Look at ATR trailing stop on a weekly time frame.
5. Consider what sort of filter could weed out the slow movers. If a stock has little channce of moving at least 20 to 30% in 3 to 6 months then why trade it?
6. Consider position sizing. I am going to get much better test  on a decent system if I use 10% of capital position sizing (ie pyramiding profits) versus a flat rate position sizing strategy - like $100,000 a trade. Consider fixed risk position sizing (1% or 2%) - this is what I use. A 10% return in one position sizing strategy might be a 20% plus return using another strategy.
7. To reduce trades look at increasing time frame.
8. I am not trying to beat buy and hold. I am trying to get a suitable return with reduced risk. Buy and hold is, for me, a very risky strategy.  If I buy and hold the wrong stocks I am in serious trouble since I have no exit strategy.

I think entries are very important. Sure random entry can work with a suitable exit and position sizing, but a good entry will give the portfolio a bit more zing. But it is good to have a benchmark to beat.  I did do some work with random exits and found that a good entry with a random exit beat a random entry system with a moving average exit.

http://drawdown.blogspot.com/2005/08/random-entries-and-exits.html
http://drawdown.blogspot.com/2005/08/random-entry.html


----------



## nizar (20 December 2006)

stevo said:
			
		

> Bin
> Good questions. I can give some ideas but really it is a matter of testing anything that appears to make sense, and some things that don't.
> 
> Some ideas;
> ...




Great advice here from someone who really dominates the markets. Thanks Stevo.


----------



## MichaelD (20 December 2006)

weird said:
			
		

> I would think with a random entry, it could be difficult to determine a sample size required for significance in statistical results.



Excellent question.

My limited knowledge of stats leads me to believe that if you look at the averages of a Monto Carlo analysis and consider 3 standard deviations away from the averages that you'll have something like a 99% probability of results falling within these parameters.


----------



## Bin57again (21 December 2006)

Stevo 
Yes, I have a good handle on those pointers but I like your attitude on point 8. 
Michael - agree that drawdown is important. From this, can I ask you, Stevo, Nizar another question on drawdown please? Let's say I have a trend following system with annual return of 20%. Starting capital is $100,000 and max. drawdown for the year is $15,000. Standard error is $2000. Now, the important part - I'm comfortable with a max.drawdown of $20,000. Is this enough of a buffer? Should I really only trade a system that has a max.drawdown of 50-75% of my tolerance? Could anyone recommend some monte carlo software? I'm not aware Amibroker has such a feature (is there anything which can be used in conjunction)?


----------



## retroaugogo (21 December 2006)

Hi Bin,

Here's some software that could interest you.

Rather than think of a system and then test to see if it works Stratasearch finds the best system for the data it is given.
Also has monte carlo simulations.

Haven't used it live yet but it's given me some nice numbers.

They have a 30 day trial.

If you end up using the program let me know.
Maybe we can swap notes.


----------



## stevo (21 December 2006)

Bin57again said:
			
		

> Stevo
> I'm comfortable with a max.drawdown of $20,000. Is this enough of a buffer? Should I really only trade a system that has a max.drawdown of 50-75% of my tolerance? Could anyone recommend some monte carlo software? I'm not aware Amibroker has such a feature (is there anything which can be used in conjunction)?



Only you can answer what your tolerance is and how much buffer you want just in case. Obviously we want to maximize profit and minimise DD - but we can only do what a system / market can supply.

You can do Monte Carlo in AB; http://drawdown.blogspot.com/2005/08/monte-carlo-with-ab.html

It's not as fast or flexible as TradeSim but with some basic Excel skills it can be done.


----------



## MichaelD (21 December 2006)

Bin57again said:
			
		

> Let's say I have a trend following system with annual return of 20%. Starting capital is $100,000 and max. drawdown for the year is $15,000. Standard error is $2000. Now, the important part - I'm comfortable with a max.drawdown of $20,000. Is this enough of a buffer? Could anyone recommend some monte carlo software?



As Stevo has said, the max drawdown you are comfortable with is something only you know, however a suggestion - devise a plan with 1/2 the drawdown you anticipate is your maximum tolerated. It's easy to say you can tolerate a drawdown of $20,000, but when it actually starts to get close to that it's pretty hard to keep trading, and that's exactly the time when you should keep to the plan.

I personally use MetaStock and TradeSim for backtesting.


----------



## Bin57again (21 December 2006)

> You can do Monte Carlo in AB; http://drawdown.blogspot.com/2005/08/monte-carlo-with-ab.html
> 
> It's not as fast or flexible as TradeSim but with some basic Excel skills it can be done.



Stevo
Did you create the charts in Excel as well?



			
				retroaugogo said:
			
		

> Hi Bin,
> 
> Here's some software that could interest you.
> 
> ...




Retro
Thanks for that. I'm always interested in these kind of products - even if they turn out to be more glitz than profit value. I'm short of time right now but I'll try the trial period in the New Year and tell you what I think.
By the way, I'm sure most of the guys on here would warn you of parting with your cash on these kind of products (not that you're in any way naive) but maybe ask around - does anyone on this thread know/use this product?


----------



## stevo (22 December 2006)

Bin
The frequency charts were created in Excel using the Frequency statement.  I used MIN and MAX to set up the bins. The slow part about using Amibroker and Excel is generating the portfolios in AB to transfer to excel.


----------



## CanOz (10 April 2007)

stevo said:


> Chops
> I don't know of any decent free software for portfolio backtesting - don't waste your time. EOD Amibroker is $149 US, and as tech mentions is very good.
> 
> If you are serious about backtesting I would go for AmiBroker. I also use Amibroker combined with TradeSim. Tradesim is excellent. You can do Monte Carlo analysis with Amibroker and Excel as well even though it's a little slow. If you don't already know how to code in Amibroker then you will need to spend some time understanding the software and language by running through the tutorials etc
> ...




Ahhh, found what i was looking for, after i found my old post!

Thanks T/A and Stevo!

Cheers,


Steve


----------



## nizar (10 April 2007)

CanOz said:


> Thanks T/A and Stevo!




I second that.
Some of the posts in this thread are GOLD.
Really PRICELESS


----------



## CanOz (10 April 2007)

nizar said:


> I second that.
> Some of the posts in this thread are GOLD.
> Really PRICELESS




Nizar, I'm clearly getting the impression that that there is allot of trial and error in developing the entry signals, just a combination of things and when it backtests well in terms of accuracy, then apply an even better system of money management to it. The exits could be, as MD says discretionary EOD.

I think its time i get myself Amibroker. 

Cheers,


----------



## nizar (10 April 2007)

CanOz said:


> Nizar, I'm clearly getting the impression that that there is allot of trial and error in developing the entry signals, just a combination of things and when it backtests well in terms of accuracy, then apply an even better system of money management to it. The exits could be, as MD says discretionary EOD.
> 
> I think its time i get myself Amibroker.
> 
> Cheers,




Can.

I think the best thing to do initially is to test a system using random entry. Iv seen random entry do 30% a year over a long period of time. Then you can test for exits and money management. THen you can test through random exits, to find a good entry.


----------

