System Robustness

[GreatPig]

If the Sum((FinalCapital - InitialCapital) for all portfolios)>0 then you should trade it.

I think I would want it better than that.

That's the same as saying if the average is > 0, but if it's too close to zero then that means you've roughly a 50-50 chance of making a profit (if the average is close to the median).

I'd be looking at least one standard deviation down to be greater than zero to be a bit more on the safe side - two standard deviations preferably. In fact, I'd rather want all backtest results to be greater than zero.

Cheers,
GP

 

[nizar]

I think I would want it better than that.

That's the same as saying if the average is > 0, but if it's too close to zero then that means you've roughly a 50-50 chance of making a profit (if the average is close to the median).

I'd be looking at least one standard deviation down to be greater than zero to be a bit more on the safe side - two standard deviations preferably. In fact, I'd rather want all backtest results to be greater than zero.

Cheers,
GP

Hi GP.

I do look for all backtest results to be greater than zero as well.

Its nice if the extreme negative outlier is substantially in profit, but just to see him in the green is enough for me.

Of course, thats as long as its an outlier and the average result over the several 000s of simulations is much much higher and something that I would be very pleased with.

 

[tech/a]

Something to aim at perhaps.

Monte Carlo Report

Trade Database Filename
C:\TradeSimData\Weekly 01.trb

Simulation Summary
Simulation Date: 17/08/2007
Simulation Time: 4:43:24 AM
Simulation Duration: 54.74 seconds

Trade Parameters
Initial Capital: $100,000.00
Portfolio Limit: 100.00%
Maximum number of open positions: 100
Position Size Model: Fixed Percent Risk
Percentage of capital risked per trade: 2.00%
Position size limit: 14.50%
Portfolio Heat: 100.00%
Pyramid profits: Yes
Transaction cost (Trade Entry): $30.00
Transaction cost (Trade Exit): $30.00
Margin Requirement: 50.00%
Magnify Position Size(& Risk) according to Margin Req: Yes

Trade Preferences
Trading Instrument: Stocks
Break Even Trades: Process separately
Trade Position Type: Process long trades only
Entry Order Type: Default Order
Exit Order Type: Default Order
Minimum Trade Size: $0.00
Accept Partial Trades: No
Volume Filter: Ignore Volume Information
Pyramid Trades: No
Use Level Zero trades only: Yes

Simulation Stats
Number of trade simulations: 10000
Trades processed per simulation: 3171
Maximum Number of Trades Executed: 166
Average Number of Trades Executed: 150
Minimum Number of Trades Executed: 132
Standard Deviation: 4.70

Profit Stats
Maximum Profit: $21,609,425.47 (21609.43%)
Average Profit: $7,856,738.16 (7856.74%)
Minimum Profit: $1,860,311.35 (1860.31%)
Standard Deviation: $2,673,221.18 (2673.22%)
Probability of Profit: 100.00%
Probability of Loss: 0.00%

Percent Winning Trade Stats
Maximum percentage of winning trades: 59.44%
Average percentage of winning trades: 50.39%
Minimum percentage of winning trades: 41.51%
Standard Deviation: 2.35%

Percent Losing Trade Stats
Maximum percentage of losing trades: 58.49%
Average percentage of losing Trades: 49.61%
Minimum percentage of losing trades: 40.56%
Standard Deviation: 2.35%

Average Relative Dollar Drawdown Stats
Maximum of the Average Relative Dollar Drawdown: $97,590.10
Average of the Average Relative Dollar Drawdown: $28,208.25
Minimum of the Average Relative Dollar Drawdown: $9,935.73
Standard Deviation: $8,859.15

Average Relative Percent Drawdown Stats
Maximum of the Average Relative Percent Drawdown: 4.7030%
Average of the Average Relative Percent Drawdown: 2.9768%
Minimum of the Average Relative Percent Drawdown: 1.9259%
Standard Deviation: 0.3768%

Maximum Peak-to-Valley Dollar Drawdown Stats
Maximum Absolute Dollar Drawdown: $1,804,845.84
Average Absolute Dollar Drawdown: $461,552.77
Minimum Absolute Dollar Drawdown: $109,716.28
Standard Deviation: $175,929.42

Maximum Peak-to-Valley Percent Drawdown Stats
Maximum Absolute Percent Drawdown: 45.9054%
Average Absolute Percent Drawdown: 27.5437%
Minimum Absolute Percent Drawdown: 21.9219%
Standard Deviation: 3.8668%

 

[R0n1n]

Here is mine. Nearly there, just need to fix that Profit percentage lol...

But seriously guys what area should I concentrate at ?

The below report is thanx to the help received from Sir Burr in converting AB it to Tradesim, thanx mate.

Trade Parameters
Initial Capital:                                           $100,000.00
Portfolio Limit:                                               100.00%
Maximum number of open positions:                         100
Position Size Model:                                Equal Dollar Units
Trade Size ($ value):                                       $10,000.00
Pyramid profits:                                                    No
Transaction cost (Trade Entry):                                 $10.00
Transaction cost (Trade Exit):                                  $10.00
Margin Requirement:                                            100.00%

Trade Preferences
Trading Instrument:                                             Stocks
Break Even Trades:                                  Process separately
Trade Position Type:                                Process all trades
Entry Order Type:                                        Default Order
Exit Order Type:                                         Default Order
Minimum Trade Size:                                              $0.00
Accept Partial Trades:                                              No
Volume Filter:                               Ignore Volume Information
Pyramid Trades:                                                     No
Use Level Zero trades only:                                        Yes

Simulation Stats
Number of trade simulations:                                     20000
Trades processed per simulation:                                 12619
Maximum Number of Trades Executed:                                1286
Average Number of Trades Executed:                                1121
Minimum Number of Trades Executed:                                 533
Standard Deviation:                                                      22.29

Profit Stats
Maximum Profit:                                  $187,217.86 (187.22%)
Average Profit:                                    $85,839.44 (85.84%)
Minimum Profit:                                    -$7,854.81 (-7.85%)
Standard Deviation:                                $23,692.78 (23.69%)
Probability of Profit:                                99.97%
Probability of Loss:                                 0.03%

Percent Winning Trade Stats
Maximum percentage of winning trades:                           41.38%
Average percentage of winning trades:                           37.97%
Minimum percentage of winning trades:                           34.48%
Standard Deviation:                                                    0.81%

Percent Losing Trade Stats
Maximum percentage of losing trades:                            65.52%
Average percentage of losing Trades:                            62.03%
Minimum percentage of losing trades:                            58.62%
Standard Deviation:                                                   0.81%

Average Relative Dollar Drawdown Stats
Maximum of the Average Relative Dollar Drawdown:             $1,260.98
Average of the Average Relative Dollar Drawdown:               $969.36
Minimum of the Average Relative Dollar Drawdown:               $793.81
Standard Deviation:                                                         $58.51

Average Relative Percent Drawdown Stats
Maximum of the Average Relative Percent Drawdown:              1.4579%
Average of the Average Relative Percent Drawdown:              0.8324%
Minimum of the Average Relative Percent Drawdown:              0.5581%
Standard Deviation:                                                          0.0983%

Maximum Peak-to-Valley Dollar Drawdown Stats
Maximum Absolute Dollar Drawdown:                           $50,662.59
Average Absolute Dollar Drawdown:                           $28,027.32
Minimum Absolute Dollar Drawdown:                           $17,345.20
Standard Deviation:                                                $4,301.58

Maximum Peak-to-Valley Percent Drawdown Stats
Maximum Absolute Percent Drawdown:                            47.4975%
Average Absolute Percent Drawdown:                            23.3832%
Minimum Absolute Percent Drawdown:                            13.6915%
Standard Deviation:                                                    3.6614%

 

[nizar]

Very impressive tech.
I like the worst portfolio and max max.DD.
Looks the goods.

WHat is the testing period for these results, i notice only 150 trades were executed?

50% winners is nice.

 

[tech/a]

Its a longterm weekly method.
10 yrs test period.
150 trades indicates that your in trades for quite a while when they go.
I'll send a sim through of a single portfolio event for other figures for you to have a sqizz over the weekend as its home not at the office.

 

[nizar]

R0nin.
Just a few points.
1. Put a realistic brokerage fee in there. I tend to overestimate mine at $44 each way. $10 i dont think you will get that anywhere for $10k parcels.
2. You will find that the profit statistics will increase significantly if you use an alternative position size method. Either a % of total capital (eg. 10%) OR you fix your risk per trade at 1-2%.
3. Choose to pyramid.

Tech.
Id imagine R/R for that system would be phenomenal.

 

[tech/a]

$10 i dont think you will get that anywhere for $10k parcels.

CFD's

 

[R0n1n]

good point guys, can u put a percentage for brokerage fees ?

Also where do I get the cheapest Historical data for ASX and US markets which is clean and fully adjusted. It used to be free somewhere on the net but I can't seem to find it. thanx.

Ronin.

 

[nizar]

good point guys, can u put a percentage for brokerage fees ?

Also where do I get the cheapest Historical data for ASX and US markets which is clean and fully adjusted. It used to be free somewhere on the net but I can't seem to find it. thanx.

Ronin.

I dont know about cheapest, but i got my data for ASX and US markets from premiumdata.net.

I paid i think around au$200.
Though annoyingly they dont yet have the list of delisted stocks for US markets. They told me its coming soon though.

And yes in tradesim you can put in a % for brokerage.

 

[_ExY_]

I think I would want it better than that.

That's the same as saying if the average is > 0, but if it's too close to zero then that means you've roughly a 50-50 chance of making a profit (if the average is close to the median).

I'd be looking at least one standard deviation down to be greater than zero to be a bit more on the safe side - two standard deviations preferably. In fact, I'd rather want all backtest results to be greater than zero.

Cheers,
GP

Of course, but perhaps from a conceptual point of view basically any system that has an average more than zero will on average make more than zero. Now it comes down to user preference on how far above zero they actually want the system to be and what kind of portfolio vs return curve they want. But nonetheless from a conceptual point of view there would be nothing rationally wrong with investing in a system that has a portfolio average above zero.

 

[R0n1n]

R0nin.
Just a few points.
1. Put a realistic brokerage fee in there. I tend to overestimate mine at $44 each way. $10 i dont think you will get that anywhere for $10k parcels.
2. You will find that the profit statistics will increase significantly if you use an alternative position size method. Either a % of total capital (eg. 10%) OR you fix your risk per trade at 1-2%.
3. Choose to pyramid.

Nizar,

where abouts would those changes go ?

 

[nizar]

Nizar,

where abouts would those changes go ?

R0n1n,

Ok, it seems your "Fixed dollar risk" and "Fixed percent risk" options are greyed out (top left of the trade parameters screen). This is likely to be because you have no initial stop coded into the metastock exploration.

You can still select "Equal percent dollar units".

For pyramiding, you need to code this into your metastock exploration.

Have a look at the TradeSim thread, tech/a has his code there and this has pyramiding and also an initial stop.

Also, if you uncheck "Favour pyramid trade" in the Preferences screen, results would be better.

For brokerage as a % i thought you could do it but i mustve been mistaken.

 

[GreatPig]

But nonetheless from a conceptual point of view there would be nothing rationally wrong with investing in a system that has a portfolio average above zero.

To me there would be if it was too close to zero.

Remember an average is just an average. If that average of just above zero is from testing over the last 20 years, then you have nearly a 50% chance of never making any money during the next 20 years.

I certainly wouldn't be investing in that scheme.

Cheers,
GP

 

[rnr]

For brokerage as a % i thought you could do it but i mustve been mistaken.

This can be done in the Preferences screen by choosing "fractional costs" from the Transaction Cost section.

Refer to TS screen shot in the post below.

Quote:
Originally Posted by nizar
R0nin.
Just a few points.
1. Put a realistic brokerage fee in there. I tend to overestimate mine at $44 each way. $10 i dont think you will get that anywhere for $10k parcels.
2. You will find that the profit statistics will increase significantly if you use an alternative position size method. Either a % of total capital (eg. 10%) OR you fix your risk per trade at 1-2%.
3. Choose to pyramid.


Nizar,

where abouts would those changes go ?

 

[_ExY_]

To me there would be if it was too close to zero.

That is personal preference (I acknowledge that I wouldn’t invest in such a system) but what im trying to say is that a rational machine would look at these results see that on average you should make money and recognise that if you didn’t make money then that is simply bad luck(given that the test itself isn’t faulty).

I certainly wouldn't be investing in that scheme.

But you would if this was the only scheme. Without introducing other possible systems(and this definition should be extended to other investment opportunites), then this system would appear to be a suitable solution so looking at the average is simply a relative measure. Like I said, it is user-preference on what type of portfolio vs. return curve should be considered as a tradeable system. Designers will take into account their own personal circumstances will looking at the effect of possible outcomes by the system such as failure. However, these details are considerations for the trader and not something concerned for the system, if you start considering these details as something for the system then your start to extend your definition of system. These details are what define this relative measure amongst systems so that the designer can make a decision about what kind of system is suitable for them.

I acknowledge what I’m saying isn’t really anything new and common-sense. I’m just trying to point out the grey areas when comes to measurements and how they should be interpreted. The rest is either personal experience or personal preference, both of which are invaluable.

 

[tech/a]

The rest is either personal experience or personal preference,

What your saying is correct.
The ability to taylor a system to your own preferences is the goal of most retail traders (interested in systems trading).
Those preferences are of course developed through experience of what is acceptable to the individual and what is possible through design.
The whole idea is to eliminate for the individual those grey areas.
Each will have a different perception of grey.

 

[theasxgorilla]

To me there would be if it was too close to zero.

Remember an average is just an average. If that average of just above zero is from testing over the last 20 years, then you have nearly a 50% chance of never making any money during the next 20 years.

I certainly wouldn't be investing in that scheme.

A random entry/exit system that I coded up in Amibroker, after 2500 runs using 10 years of data from 1/1/97 until 31/12/06, showed a 95% chance of a CAGR > 7.7% and a 95% chance of experiencing a Max DD no worse than 33.3%. No single run had a negative CAGR or a Max DD greater than 55%. This system used nothing more sophisticated than 10x10% position sizing and a 10% stoploss. There was no leverage.

As mentioned, entry and exit were random, holding period was between 1 and 6 months, market exposure was 76%, average trades per run was 449...brokerage was included at $33 each way. Average CAGR and Max DD was 12.2% and 27.4% respectively. Dividends were not included, you might conservatively estimate an extra 2.5% CAGR if you include dividends.

By adding a simple momentum filter (and I mean SIMPLE), these results were improved to become 14.7% CAGR (still without dividends) and 25.6% Max DD. Market exposure was reduced to 61%. No single run produced a CAGR of less than 2.5% or a Max DD greater than 50%.

My belief is that if you can't design a system that beats this system using the same data then you don't have a system with a sufficient edge to beat the market.

 

[GreatPig]

AG,

Do you mind posting your code for that system?

I just tried a random entry system based on the sort of time and parameters you mention and always get negative returns when run over all stocks (or most stocks).

I can get results similar to yours though over the current ASX200 or ASX300, but it seems to be very sensitive to trade price.

Cheers,
GP

 

[tech/a]

My belief is that if you can't design a system that beats this system using the same data then you don't have a system with a sufficient edge to beat the market.

Your results are to be expected.
They simply mimick the market.
Nothing special there.
The equity curve would look like
a chart of a composite of all those stocks
in your universe.