Australian (ASX) Stock Market Forum

Amibroker Beginnings

Jump to new
K

kolonel

Joined
1 January 2008
Posts
61
Reactions
0
Absorbing bit of software Amibroker. Have been running some backtesting based on TechTrader formula and have come up with some profitable results, and was wondering if i can draw on some experience to see where i can fine tune it.

Yes, a little bit impetuous, but its quite interesting what can be done with a few changes.

Initial Capital - $15000
Comms - $15 per trade
Market - ASX
Timeframe - 01/10/92 to 19/03/08
Maximum Stop Loss - 2%
Profit Target - 50%
Periodicity - Weekly
Positions - Long


All trades Long trades Short trades
Initial capital 15000.00 15000.00 15000.00
Ending capital 165403.56 165403.56 40621.58
Net Profit 150403.56 150403.56 25621.58
Net Profit % 1002.69 % 1002.69 % 170.81 %
Exposure % 79.84 % 79.84 % 0.00 %
Net Risk Adjusted Return % 1255.83 % 1255.83 % N/A
Annual Return % 16.78 % 16.78 % 6.65 %
Risk Adjusted Return % 21.02 % 21.02 % N/A

--------------------------------------------------------------------------------

All trades 544 544 (100.00 %) 0 (0.00 %)
Avg. Profit/Loss 262.14 262.14 N/A
Avg. Profit/Loss % 4.85 % 4.85 % N/A
Avg. Bars Held 58.68 58.68 N/A

--------------------------------------------------------------------------------

Winners 105 (19.30 %) 105 (19.30 %) 0 (0.00 %)
Total Profit 326473.98 326473.98 0.00
Avg. Profit 3109.28 3109.28 N/A
Avg. Profit % 56.39 % 56.39 % N/A
Avg. Bars Held 159.31 159.31 N/A
Max. Consecutive 4 4 0
Largest win 15282.79 15282.79 0.00
# bars in largest win 6 6 0

--------------------------------------------------------------------------------

Losers 439 (80.70 %) 439 (80.70 %) 0 (0.00 %)
Total Loss -183870.38 -183870.38 0.00
Avg. Loss -418.84 -418.84 N/A
Avg. Loss % -7.47 % -7.47 % N/A
Avg. Bars Held 34.61 34.61 N/A
Max. Consecutive 28 28 0
Largest loss -4247.56 -4247.56 0.00
# bars in largest loss 5 5 0

--------------------------------------------------------------------------------

Max. trade drawdown -8264.17 -8264.17 0.00
Max. trade % drawdown -56.96 % -56.96 % 0.00 %
Max. system drawdown -20841.64 -20841.64 0.00
Max. system % drawdown -21.58 % -21.58 % 0.00 %
Recovery Factor 7.22 7.22 N/A
CAR/MaxDD 0.78 0.78 N/A
RAR/MaxDD 0.97 0.97 N/A
Profit Factor 1.78 1.78 N/A
Payoff Ratio 7.42 7.42 N/A
Standard Error 19182.59 19182.59 936.38
Risk-Reward Ratio 0.42 0.42 1.74
Ulcer Index 7.95 7.95 0.00
Ulcer Performance Index 1.43 1.43 N/A
Sharpe Ratio of trades 0.28 0.28 0.00
K-Ratio 0.0299 0.0299 0.1242

Have run it from the start of this year to current and curerntly sitting at profit of $2365.00 with W/L ratio of 4/8.

Regards
kolonel
 
Hi Kolonel --

The first question to ask is -- are these results from in-sample testing? I assume the answer is yes because of the length of time being reported and the fact that it includes today.

The in-sample data is that data used to develop the system. It is always a good idea to reserve some data that follows the in-sample data, but is not used at all during the development of the system, to see how the system performs on data it has never seen before. This is the out-of-sample data.

After all, tomorrow is always out-of-sample, and we want to get some idea of what is likely to happen when we place a trade tomorrow.

Results from in-sample tests have no value in terms of estimating the likelihood of profitable trading. No value.

I note that this system has 20% winning trades, 80% losing trades. Personally, I would have trouble trading a system with such a low winning percentage.

AmiBroker has some very nice features that allow you, as the system developer, to design your own custom objective function. The objective function is the metric used to measure the "goodness" of each run, or of each alternative when you are performing optimizations.

I recommend that you investigate some of the metrics that already exist in AmiBroker that simultaneously reward equity growth and penalize drawdown. These include: K-Ratio, Ulcer Performance Index, CAR/MDD, RAR/MDD, and RRR.

Thanks for listening,
Howard
 
Thank you for the informative response Howard.

I was a little disappointed with the W/L ratio, but at the moment am restricted to using a modified TechTrader system. I am considering the book, 'Quantitative Trading Systems' for a bit of guidance designing my own system, but i feel i have a bit of reading still to do.

Another option, is 'Building a Profitable Trading Plan using T/A' by Nick Radge, but not sure if my knowledge (very little) of the whole trading market is there yet.

I am resisting the temptation to just jump in and trade, as i feel that would be a financial disaster, so i will continue to refine what Amibroker has to offer until i feel confidant.

Regards
kolonel
 
kolonel said:
Absorbing bit of software Amibroker. Have been running some backtesting based on TechTrader formula and have come up with some profitable results, and was wondering if i can draw on some experience to see where i can fine tune it.

Yes, a little bit impetuous, but its quite interesting what can be done with a few changes.

Initial Capital - $15000
Comms - $15 per trade
Market - ASX
Timeframe - 01/10/92 to 19/03/08
Maximum Stop Loss - 2%
Profit Target - 50%
Periodicity - Weekly
Positions - Long


All trades Long trades Short trades
Initial capital 15000.00 15000.00 15000.00
Ending capital 165403.56 165403.56 40621.58
Net Profit 150403.56 150403.56 25621.58
Net Profit % 1002.69 % 1002.69 % 170.81 %
Exposure % 79.84 % 79.84 % 0.00 %
Net Risk Adjusted Return % 1255.83 % 1255.83 % N/A
Annual Return % 16.78 % 16.78 % 6.65 %
Risk Adjusted Return % 21.02 % 21.02 % N/A

--------------------------------------------------------------------------------

All trades 544 544 (100.00 %) 0 (0.00 %)
Avg. Profit/Loss 262.14 262.14 N/A
Avg. Profit/Loss % 4.85 % 4.85 % N/A
Avg. Bars Held 58.68 58.68 N/A

--------------------------------------------------------------------------------

Winners 105 (19.30 %) 105 (19.30 %) 0 (0.00 %)
Total Profit 326473.98 326473.98 0.00
Avg. Profit 3109.28 3109.28 N/A
Avg. Profit % 56.39 % 56.39 % N/A
Avg. Bars Held 159.31 159.31 N/A
Max. Consecutive 4 4 0
Largest win 15282.79 15282.79 0.00
# bars in largest win 6 6 0

--------------------------------------------------------------------------------

Losers 439 (80.70 %) 439 (80.70 %) 0 (0.00 %)
Total Loss -183870.38 -183870.38 0.00
Avg. Loss -418.84 -418.84 N/A
Avg. Loss % -7.47 % -7.47 % N/A
Avg. Bars Held 34.61 34.61 N/A
Max. Consecutive 28 28 0
Largest loss -4247.56 -4247.56 0.00
# bars in largest loss 5 5 0

--------------------------------------------------------------------------------

Max. trade drawdown -8264.17 -8264.17 0.00
Max. trade % drawdown -56.96 % -56.96 % 0.00 %
Max. system drawdown -20841.64 -20841.64 0.00
Max. system % drawdown -21.58 % -21.58 % 0.00 %
Recovery Factor 7.22 7.22 N/A
CAR/MaxDD 0.78 0.78 N/A
RAR/MaxDD 0.97 0.97 N/A
Profit Factor 1.78 1.78 N/A
Payoff Ratio 7.42 7.42 N/A
Standard Error 19182.59 19182.59 936.38
Risk-Reward Ratio 0.42 0.42 1.74
Ulcer Index 7.95 7.95 0.00
Ulcer Performance Index 1.43 1.43 N/A
Sharpe Ratio of trades 0.28 0.28 0.00
K-Ratio 0.0299 0.0299 0.1242

Have run it from the start of this year to current and curerntly sitting at profit of $2365.00 with W/L ratio of 4/8.

Regards
kolonel
Click to expand...

Hi Kolonel,

Given that your winning trades is only 20%, can I ask why are using a Profit Target?

Surely, by using one you are restricting the possibility to increase the returns on your winning trades? IMO Better to let the Mrkt give you as much as possible whenever you do get a winning trade.....

Chorlton
 
Chorlton said:
Hi Kolonel,

Given that your winning trades is only 20%, can I ask why are using a Profit Target?
Click to expand...

Gday Chorlton,

Have had some mixed results runing the system from the start of the year. Maybe i am not setting it up correctly, but get some interesting results :

Profit Target % - Trailing Stop % - Profit $ - W/L ratio (all figures with 2% stop loss)

70/70/1289/3-9
70/60/1289/3-9
70/50/1289/3-9
70/40/1289/3-9
70/30/1129/3-9
70/20/1640/5-7
70/10/1693/4-8

60/60/2365/4-8
60/50/2365/4-8
60/40/2365/4-8
60/30/2205/4-8
60/20/1640/5-7
60/10/1693/4-8

50/50/2365/4-8
50/40/2365/4-8
50/30/2205/4-8
50/20/1640/5-7
50/10/1693/4-8

40/40/1358/4-8
40/30/1907/4-8
40/20/1640/5-7
40/10/1693/4-8

30/30/1907/4-8
30/20/1640/5-7
30/10/1693/4-8
30/5/1693/4-8

20/20/1819/5-7
20/15/1693/4-8
20/10/1693/4-8
20/5/1693/4-8

15/15/1693/4-8
15/10/1693/4-8
15/5/1693/4-8

10/10/1693/4-8
10/5/1693/4-8

5/5/1693/4-8

Is there an easier way to test stops without manually going back into settings on Amibroker ? Also, what timeframe can be used for a bear market we have had in the past to test on ?

Regards
kolonel
 
Also need to refine position. Ran a 1 year test based on highest profit and had the following :

Profit - $10423
W/L - 14/51
profit/trail/stop - 60/disabled/2

From 13/04/07 - 13/07/07 (Max open pos 10) = 3 months with signals but no trade.

Regards
kolonel
 
Thanks Cap,

I added these lines

/* max profit stop optimization */

ApplyStop(stopTypeProfit, stopModePercent, Optimize( "max. profit stop level", 10, 5, 80, 5 ), False );

So i am to understand it as the following :

I am applying a stop based on profit, at a percentage, optimized with a default of 10%, min of 5%, max of 80% at 5% increments ?

Does this code overide anything i put in settings ?

Regards
kolonel
 
To be honest, I am still fairly new to both AB and System Design but I'll still try to offer some help where I can. Maybe others more experienced can also comment.

Looking at your system, would you agree that it is a long-term Trend-Following system? If so, from what I understand the majority of profit is derived from the few large hitters. By using a Target Stop, I believe you may be limiting this.

Out of interest, what happens when you rely on your trailing stop to exit trades?

On the subject of Bear Mkts, unfortunately my experience is with the UK Mkts, so am unable to comment.
 
kolonel said:
I am applying a stop based on profit, at a percentage, optimized with a default of 10%, min of 5%, max of 80% at 5% increments ?
Click to expand...

That's correct.

kolonel said:
Does this code overide anything i put in settings ?
Click to expand...

Yes, formula based settings overide the settings window. This makes it much easier to backtest/scan/explore different systems without having to change things manually in the settings window.

This guide to "setoption" is handy to keep close by when coding:

http://www.amibroker.com/guide/afl/afl_view.php?id=201
 
kolonel said:
Also need to refine position. Ran a 1 year test based on highest profit and had the following :

Profit - $10423
W/L - 14/51
profit/trail/stop - 60/disabled/2

From 13/04/07 - 13/07/07 (Max open pos 10) = 3 months with signals but no trade.

Regards
kolonel
Click to expand...

It was suggested to me a little while ago that when developing a weekly system, using a backtest period of 1 year was not enough..... maybe something to bear in mind....
 
Chorlton said:
Looking at your system, would you agree that it is a long-term Trend-Following system? If so, from what I understand the majority of profit is derived from the few large hitters. By using a Target Stop, I believe you may be limiting this.
Click to expand...

Long term trend following systems like Techtrader work well when profitable trades are left to run their course. Generally speaking a profit stop will hurt such a system.

I trade a short term system (10 day avg hold) that uses a profit stop and an index based filter. Both these tend to inhibit the performance of long term systems.
 
Greetings all --

I was hoping that my post yesterday would be sufficient, but I think another comment or two might help.

The system holds each position a fairly long time -- on average 28 days for losers, 158 for winners. Systems that hold a long time should not use profit targets to exit. Profit targets are appropriate for systems that hold a much shorter time -- intraday, or a few days, or a week or two at the longest -- and there should be a high proportion of exits caused by the profit target being reached. When a profit target exit is used with a system that holds a long time, it is inevitable that the magnitude of the profit being targeted will be curve-fit to the specific data that was used for the in-sample period, and the out-of-sample performance is certain to be poor.

Note the number of consecutive losers -- 28. The ratio of winning trades to losing trades is 105 to 439 -- 80.6% losers. Personally, if I saw a system go 27 trades in a row with a loss each time, I could not place the 28th or 29th trade. In fact, the probability of 14 consecutive losing trades is 0.0488. When the 14th trade in a row lost (and the probability of that happening is <0.05), the system is telling you that it is broken and should not be traded any more.

There is a maximum trade drawdown of 56%, which is on the high side for many people. I see a note that there is a 2% maximum stop loss -- that must not have been active in the code when this test was run. But maximum loss exits are the worst kind of exit anyway. Try to exit from a signal, just like the entry was made from a signal. In systems that hold a long time, a trailing stop such as the chandelier or parabolic is often a reasonable exit to both capture the long-term gain and protect the profit.

Again -- all of these stats are taken from in-sample results. In-sample results Always Over-estimate the profitability. This system should definitely not be traded with real money.

Thanks for listening,
Howard
 
howardbandy said:
Greetings all --

I was hoping that my post yesterday would be sufficient, but I think another comment or two might help.

The system holds each position a fairly long time -- on average 28 days for losers, 158 for winners. Systems that hold a long time should not use profit targets to exit. Profit targets are appropriate for systems that hold a much shorter time -- intraday, or a few days, or a week or two at the longest -- and there should be a high proportion of exits caused by the profit target being reached. When a profit target exit is used with a system that holds a long time, it is inevitable that the magnitude of the profit being targeted will be curve-fit to the specific data that was used for the in-sample period, and the out-of-sample performance is certain to be poor.

Note the number of consecutive losers -- 28. The ratio of winning trades to losing trades is 105 to 439 -- 80.6% losers. Personally, if I saw a system go 27 trades in a row with a loss each time, I could not place the 28th or 29th trade. In fact, the probability of 14 consecutive losing trades is 0.0488. When the 14th trade in a row lost (and the probability of that happening is <0.05), the system is telling you that it is broken and should not be traded any more.

There is a maximum trade drawdown of 56%, which is on the high side for many people. I see a note that there is a 2% maximum stop loss -- that must not have been active in the code when this test was run. But maximum loss exits are the worst kind of exit anyway. Try to exit from a signal, just like the entry was made from a signal. In systems that hold a long time, a trailing stop such as the chandelier or parabolic is often a reasonable exit to both capture the long-term gain and protect the profit.

Again -- all of these stats are taken from in-sample results. In-sample results Always Over-estimate the profitability. This system should definitely not be traded with real money.

Thanks for listening,
Howard
Click to expand...


Hello Howard,

With regard to the total number of consecutive losers identifed from any system after backtesting, at which point would it be wise to "turn" the system off?

For example, assuming you were happy with say "9" consecutive losers and in real time this happened, would one abandon the system if 1 additional (ie. 10) consecutive losers appeared?

Thanks,

Chorlton
 
Hi Chorlton --

The results are from in-sample testing, which have no value as predictors of out-of-sample performance. But, assume for this discussion that those results were from out-of-sample tests achieved without polluting the out-of-sampleness of the data.

I noted that there 80.6% losing trades. Assuming that the trades and their results are independent of each other, then we can use the calculations for the binomial distribution and the probability of n losing trades in a row is (0.806)^n.

In your question, you ask whether we should worry if there are nine losers in a row. (0.806)^9 == 0.143. That implies that any sequence of nine trades, taken at random from the system that produced the results we are evaluating, will all be losing trades about 14% of the time.

People working with experiments and with quality control often measure "statistical significance" by asking how likely it is that the observed results could have happened by chance. And they often say that something is significant "at the 0.05 level" if it could have happened by chance less than 5% of the time. If random selection produces the observed results less than 5% of the time, they say that the results are probably not random. There is no way to know for certain, but by repeating the experiments the confidence can be increased.

The experiment in this case is drawing 9 trades from the distribution of trades that has 80.6% losers and 19.4% winners. Since the probability that all 9 can be losers is about 14%, and 14% is greater than 5%, we cannot conclude that the system is broken.

As a comparison ---

Assume that there is a different trading system that has 50% winning trades and 50% losing trades. The probability of n losers in a row is (0.50)^n. The probability of 9 losing trades in a row is (0.50)^9 == 0.00195. Since 0.00195 is less than 0.05, we can be confident that those results (9 losers in a row) did not come from the distribution we originally measured (50% winners), so the system has changed and is giving results that follow a new distribution that is almost certainly different than the original one. We conclude that the system is broken.

Thanks,
Howard
 
howardbandy said:
Hi Chorlton --

The results are from in-sample testing, which have no value as predictors of out-of-sample performance. But, assume for this discussion that those results were from out-of-sample tests achieved without polluting the out-of-sampleness of the data.

I noted that there 80.6% losing trades. Assuming that the trades and their results are independent of each other, then we can use the calculations for the binomial distribution and the probability of n losing trades in a row is (0.806)^n.

In your question, you ask whether we should worry if there are nine losers in a row. (0.806)^9 == 0.143. That implies that any sequence of nine trades, taken at random from the system that produced the results we are evaluating, will all be losing trades about 14% of the time.

People working with experiments and with quality control often measure "statistical significance" by asking how likely it is that the observed results could have happened by chance. And they often say that something is significant "at the 0.05 level" if it could have happened by chance less than 5% of the time. If random selection produces the observed results less than 5% of the time, they say that the results are probably not random. There is no way to know for certain, but by repeating the experiments the confidence can be increased.

The experiment in this case is drawing 9 trades from the distribution of trades that has 80.6% losers and 19.4% winners. Since the probability that all 9 can be losers is about 14%, and 14% is greater than 5%, we cannot conclude that the system is broken.

As a comparison ---

Assume that there is a different trading system that has 50% winning trades and 50% losing trades. The probability of n losers in a row is (0.50)^n. The probability of 9 losing trades in a row is (0.50)^9 == 0.00195. Since 0.00195 is less than 0.05, we can be confident that those results (9 losers in a row) did not come from the distribution we originally measured (50% winners), so the system has changed and is giving results that follow a new distribution that is almost certainly different than the original one. We conclude that the system is broken.

Thanks,
Howard
Click to expand...

Hi Howard,

As always, Many Thanks for such a detail reply!!!

Just for my understanding, once my system is ready for trading, the final step will be to backtest it on my out-of-sample data. From this backtesting, I will then know the % of losing trades. (for this example lets assume 40%)

Then, using your formula (0.4)^n, I should keep increasing the value of 'n' until the result is the first one below 0.05 which at that point will give me the number of losing trades at which I should seriously considering switching the system off.

Is this correct?

Kind Regards,

Chorlton
 
Chorlton said:
Hi Howard,

As always, Many Thanks for such a detail reply!!!

Just for my understanding, once my system is ready for trading, the final step will be to backtest it on my out-of-sample data. From this backtesting, I will then know the % of losing trades. (for this example lets assume 40%)

Then, using your formula (0.4)^n, I should keep increasing the value of 'n' until the result is the first one below 0.05 which at that point will give me the number of losing trades at which I should seriously considering switching the system off.

Is this correct?

Kind Regards,

Chorlton
Click to expand...

Hi Chorlton --

The process of walking forward -- optimizing over the in-sample data, choosing the best alternative, using the settings of that best alternative to simulate trading the out-of-sample data that follows -- is designed to verify that the system is recognizing the buy and sell points in the historical data. By looking at the concatenation of all the out-of-sample results from all the walk-forward simulated trades, you have an indication of how the system is likely to trade when you place real trades with real money. By performing several, or many, walk forward steps, you gain confidence every time the system enters a new out-of-sample period.

The walk forward gives you two very valuable things --
One, an indication of how the system performs in out-of-sample testing.
Two, confidence that you know what to expect, in terms of trading the next few days or weeks, after you have performed an optimization and chosen the newest best settings.

If you are satisfied with the performance as represented by the out-of-sample simulated trades, you are done developing and are ready to trade.

It is not necessary (or wise) to perform more testing on the out-of-sample periods. Each of the previously out-of-sample periods will become part of the in-sample data as the walk forward process walks forward. That happens automatically. Running an optimization over the latest out-of-sample period will just cause confusion. The results (settings for the best values of parameters and resulting trades taken) will almost certainly be different than those resulting from the out-of-sample test done by the walk forward process -- and it would be a mistake to use those values when placing real trades.

Be very cautious about examining the out-of-sample results and changing your system to produce better out-of-sample results. Of course you will be doing that to some degree, but beware. The more the out-of-sample results are used to make decisions, the more their "out-of-sampleness" is compromised and the less those results can be relied on to indicate the likely future performance.

Thanks,
Howard
 
Chorlton said:
Hi Howard,

As always, Many Thanks for such a detail reply!!!

Just for my understanding, once my system is ready for trading, the final step will be to backtest it on my out-of-sample data. From this backtesting, I will then know the % of losing trades. (for this example lets assume 40%)

Then, using your formula (0.4)^n, I should keep increasing the value of 'n' until the result is the first one below 0.05 which at that point will give me the number of losing trades at which I should seriously considering switching the system off.

Is this correct?

Kind Regards,

Chorlton
Click to expand...

Hi Chorlton --

Part 2.

Your understanding of the use of the binomial distribution is correct.

Assume that your unbiased out-of-sample results show a system that has 60% winning trades and 40% losing trades.

The probability of any one trade being a losing trade is 0.40.

The probability of two losing trades in a row is (0.40)^2 = 0.16. Something that happen 16% of the time is pretty common.

The probability of three losing trades in a row is (0.40)^3 = 0.064. Something that happens about 6% of the time is unusual and should be watched carefully.

The probability of four losing trades in a row is (0.40)^4 = 0.0256. Something that happens about 2 percent of the time is rare. I would take this as an indication that either the system and underlying market it is modeling are out of synchronization or the system is broken. I would stop trading the system, but continue to monitor trades that are signaled. Wait for the trades signaled but not taken to return to normal, then start trading it again.

If you have gone through the walk forward process, you have a good idea of what happens during the first few days or weeks of the new out-of-sample period after each reoptimization. You may reoptimize any time you wish. You do not need to wait for the expiration of the out-of-sample time period. Reoptimizing may resynchronize the model with the market.

So, there is an alternative to just waiting. Reoptimize. Note the new settings and code them into your trading system. Follow the new signals, but do not trade them until you are satisfied that the system was just out-of-sync and not broken. If the new values start to produce profitable trades, trade it. If they continue to be losing trades, wait longer -- reoptimize periodically if you wish.

Thanks,
Howard
 

Similar threads

W
Newbie... portfolio backtesting results
2 3
Replies
52
Views
14K
DeepState
DeepState
Top