Amibroker V6 Monte Carlo

[get better]

Amibroker version 6 has come out with a new Monte Carlo function (late last year).

Has anyone used this to validate their system statistically yet? Have you found it to be useful at all in assessing your mechanical trading system?

I'm starting to fall into the trap of trying to develop systems with positive returns for 85% of simulations run in the Monte Carlo analysis to minimise losses in markets it they don't operate well under and maximise performance in ideal markets (have to be very careful about curve fitting). I'm curious to see how others are using this tool to their advantage/suggestions on how else it could be used to make trading systems 'better'.

 

[Gringotts Bank]

Yes, if you can get it looking roughly like this (closely packed runs), it will give you confidence to trade it.

Remember to do a walkforward test if you have the time and computing power. Or even just a brief version of one to get an idea of robustness.

 

[howardbandy]

Greetings --

There are several areas of trading system development where Monte Carlo analysis is a useful tool. Be careful to understand the assumptions and conditions where you plan to use it.

For analysis of risk and reward:
The data is trades (or daily equity changes). Use the least biased set of trades available. In order of preference, actual trades are best, followed by paper trades. Both will give good estimates of future performance. Truly out-of-sample trades are next best. Any set of trades that come from in-sample runs, or contaminated out-of-sample runs (where the model was modified following examination of previous OOS results) are suspect. They will underestimate risk and overestimate profit. In-sample results have no value in estimating future performance.

All of that said, the Monte Carlo tools in AmiBroker are a welcome addition to a high quality trading system development platform. They are powerful. Used incorrectly (naively or intentionally), they can create a deceptive illusion of validation of system performance. Use them with care.

My video presentation might be helpful:
https://www.youtube.com/watch?v=7k0uYRrXawA&feature=youtu.be

Best regards,
Howard

 

[howardbandy]

Yes, if you can get it looking roughly like this (closely packed runs), it will give you confidence to trade it.

Remember to do a walkforward test if you have the time and computing power. Or even just a brief version of one to get an idea of robustness.

View attachment 66026

Wow!!

That is impressive. Your fans will appreciate as much more information as you care to share.

Best,
Howard

 

[Gringotts Bank]

Thanks Howard. It was more for illustration purposes, since no slippage included. The slippage chart isn't as pretty.

 

[Roller_1]

Thanks Howard. It was more for illustration purposes, since no slippage included. The slippage chart isn't as pretty.

What figures do you use for slippage in your testing?

 

[Gringotts Bank]

What figures do you use for slippage in your testing?

I just include .25% in the brokerage, so it gets included that way. I don't really know if that's too little. The right way to calculate it would probably be by comparing actual results to backtested results.

 

[Roller_1]

I just include .25% in the brokerage, so it gets included that way. I don't really know if that's too little. The right way to calculate it would probably be by comparing actual results to backtested results.

Yeah ok, depends what market and order types the strategy uses i guess.

 

[Gringotts Bank]

Yeah ok, depends what market and order types the strategy uses i guess.

Desired position size versus available liquidity. Very hard to determine with equities, especially those outside the top 20/50. Even when trading top 20's, the spread will rarely be 1c.

 

[Wysiwyg]

Yes, if you can get it looking roughly like this (closely packed runs), it will give you confidence to trade it.

Remember to do a walkforward test if you have the time and computing power. Or even just a brief version of one to get an idea of robustness.

I refuse to back test any further than a few years for the XAO (which is rebalanced yearly every March and quarterly for the XJO) because of the survivorship. I have asked Premium Data and there isn't a chance of getting all the constituents that were delisted on the ASX. It really is a fruitless exercise testing a system otherwise. Gotta have those failures in the mix.

 

[get better]

Yes, if you can get it looking roughly like this (closely packed runs), it will give you confidence to trade it.

Remember to do a walkforward test if you have the time and computing power. Or even just a brief version of one to get an idea of robustness.

View attachment 66026

Wow, thanks GB! That chart is excellent - that pretty much shows the ideal result of a Monte Carlo analysis.

 

[Roller_1]

I refuse to back test any further than a few years for the XAO (which is rebalanced yearly every March and quarterly for the XJO) because of the survivorship. I have asked Premium Data and there isn't a chance of getting all the constituents that were delisted on the ASX. It really is a fruitless exercise testing a system otherwise. Gotta have those failures in the mix.

You can get them, its a separate database

 

[Richard Dale]

Just to clarify - the ASX database that has been fully researched to eliminate survivorship bias is currently in testing and is not a currently released product. Expect a release this year.

We are accepting testers who have an existing subscription to our ASX data updating service and data history upon specific request to become a tester.

 

[JJZ]

Just to clarify - the ASX database that has been fully researched to eliminate survivorship bias is currently in testing and is not a currently released product. Expect a release this year.

We are accepting testers who have an existing subscription to our ASX data updating service and data history upon specific request to become a tester.

I love to become a tester, what the best way to put my myself forward? Whats involved, I am already a subscriber.

 

[Richard Dale]

Just request to become a tester by email to support@premiumdata.net - we'll require you to acknowledge that the software is in testing and may encounter issues from time-to-time but then you'll be able to obtain the new software and run it in parallel with your existing environment.