This is a mobile optimized page that loads fast, if you want to load the real page, click this text.

Would you trade this system?

Jump to new
kaleon said:
Sorry Nizar,

I noticed you had transaction costs but no slippage. I think slippage would eat away a large portion of potential profit. Did you used fixed dollar units of fixed % risk in your simulations?
Click to expand...

Yes, you are correct.
Looks like your system is much better than mine!
Well done.

Seems like I have a long way to go, but thats to be expected as Iv just started on this journey.

Its gonna be alot of work to get a 2001 ASX300 list. Thanks for the list.
 

Well the answer to the part in bold is YES!
We have to do this in order to test the robustness of the system and to validate the system.

There is alot of discussion about walk-forward analysis in one of the other threads i started, titled "System Robustness".
 


Hi Nizar,

I would have thought that it would be the other way round. Futures are based on indices (whose value are constructed from the SPs of many individual stocks). Therefore the only way to fade the futures and have any sort of impact on its value would be to impact all its constitutent stocks, which makes it far less controllable than individual stocks.

Interesting topic all the same regarding system decay. I guess once performance does not measure up to expectations that it would be time to reoptimize and try again.
 
Hi Gorilla --

You asked about this statement in my response to a Monte Carlo question:
"Another is to first create a statistical distribution from the observed actual trades, then make repeated random selections from that distribution. "

This technique begins with a number of observations, with the intent of generating several other sets of observations that have the same statistical characteristics, but are not necessarily the same values, and not necessarily in the same time order.

The following example uses 15 because I can list 15 without filling up too much space. I would probably have many more than 15.

Say I have run a trading system simulation and my out-of-sample results show 15 closed trades. The percent gain for them, as they occur in time order, is:
4
-1
3
2
-1
-1
1
0
3
1
2
-3
0
-2
6

I want to simulate what the equity curve would look like if I had other sequences of trades that came from the same distribution, but not necessarily those exact same numbers and not necessarily in that order.

I begin by creating a Cumulative Distribution Function using those observations. (If that statement did not make sense, go to either my book, Quantitative Trading Systems, Chapter 22, or a text for statistics, econometrics, or simulation.) If desired, I smooth the curve.

To simulate a single equity curve, I make 15 random draws from this distribution (with replacement, for those who care), each draw representing a closed trade, and create the equity curve for those 15.

I do this many times -- say 100 times -- and evaluate the results. I can ask questions such as:
What is the probability that this system has a drawdown greater than 10% at any time?
What is the probability that there will be a string of 3 or more losing trades in a row?
What is the average terminal relative wealth?
What is the standard deviation terminal relative wealth?

Terminal relative wealth, trw, is the ratio of the final account balance to the initial account balance. If the account begins with a balance of $10,000 and ends with a balance of $25,000, then terminal relative wealth is 2.5.

If I have been compounding, that is investing 100% of the account in every trade, then there is a simple relationship between the average percentage gain per trade, p, and the number of trades, n.

trw = (1+p) ^ n

For example, a system that makes 2 trades in a year and gains 10% on each has trw of (1.10)^2 or 1.210 -- a gain of 21.0%.
A system that makes 10 trades a year and gains 2% on each has trw of (1.02)^10 or 1.219 -- a gain of 21.9%.

Thanks,
Howard
 

Hi debono --

My short answer is "no."

First, there are no historical theoretical return for comparison.

More importantly, the in-sample results have no value in predicting the future profitability or performance of a trading system. Only out-of-sample results count.

There has been a lot of discussion of this in this thread and in the thread on Robustness.

Thanks,
Howard
 

I wasnt referring to only indices, but commodities as well.
 
Sir Burr said:
Tech, a question that I have been pondering:

Why is Monte Carlo Analysis better than Optimising/Walk Forward?

We've been lead down the Monte Carlo path with Tradesim but is it the better way to go?

What's the difference?

Cheers SB
Click to expand...


S/B its just another analytic tool.
Infact Montecarlo analysis as it is used by the software most have is far from ideal.true Montecarlo looks at EVERY possible variable.
There is a great deal of difference and each can and should co exist.The opinions on Optimising/Forward testing and Walk Forward testing are in some part different to mine.

My own opinion and research differ somewhat to that expressed here.

While I have attempted to express and question the thinking of others I'm afraid it falls on deaf ears answers are obtuse or questions completely ignored.
Fortunately I'm working on systems design with my son who is a Doctor of Physics and has powerful systems design and modelling software and analytical capability.I have my own tutor.
As he says "I can make any mathamatical arguement tell you what ever you want".
 
tech/a said:
Howard.
Are you suggesting optimisation for all trading systems?

Or only single entities--Futures/Commodities/Indexes?
Click to expand...

Hi Tech/a --

Yes, optimize everything. Optimize just means make a thorough search -- nothing more than that. If I am going to choose from among two or three or twelve alternatives, why stop there? I'll look at thousands. You read my post using the "marbles" analogy, yes? Post Number 34 in this thread. I explain why in it.

Thanks,
Howard
 

Hi Tech/a --

If there is question that you have asked me that I have not answered, or an answer that I have given that is obtuse, please ask again.

Thanks,
Howard
 


For Starters.
 
tech/a said:
For Starters.
Click to expand...

Hi Tech/a --

I'll start at the top. I read that post and got confused -- the questions asked seemed to be rhetorical, leading up to your statement at the end of the quoted text. May I suggest one question per post? And if you specifically want my comments, please ask me by name.

About anchored or rolling periods --

In my opinion, rolling. If anchored is used, the in-sample period for the last walk forward step during development, and then for all the "bring up to date" steps during actual use of the system, is "all the data." The system is being given more and more data representing more and more market conditions and is less and less likely to be tuned to the current conditions. One of the benefits of using the walk forward process is being able to tune the system to the current conditions.

Thanks,
Howard
 
Hi Tech/a --

I do not understand this one:

How do you evelute the period ahead that you should use in the evaluation?

Thanks,
Howard
 
Hi Tech/a --

You wrote: "If optimising at what point do you re calibrate?"

Assuming that you mean re-optimize --

I'll define re-optimize to mean the following:
1. Move the in-sample period forward in time so that it includes the most recent data, or nearly so.
2. Run an optimization to determine the values for the arguments that give the highest objective function score for that in-sample period.
3. Begin using the new values of the arguments to test the out-of-sample data or to begin trading using the new values.

There are a couple of possibilities:

1. Always re-optimize after a set number of out-of-sample bars. This is what would have been done during the walk forward phase of development of the model, so it makes sense to continue doing that.

2. Re-optimize whenever the out-of-sample results, or trading results, indicate that the system is performing at a lower level than anticipated. To answer this question requires some statistical analysis. The implications of re-optimizing after a variable number of out-of-sample bars is that you might continue to trade a system beyond the normal walk-forward period. If the system is performing well, feel free to do that.

There is no single answer. It depends on the characteristics of the market being modeled and the model.

Thanks,
Howard
 
Hi Tech/a --

You wrote: "How many out of series test periods are in your veiw enough to give confidence and at what point do you gain that confidence?"

There is no one-size-fits-all answer to this question.

The goal of the system development process -- whether you are using methods similar to those I describe or some other method -- is to develop confidence that the system will trade profitable in the future. The emphasis is on "develop confidence." The characteristics of a trading system that give you confidence may not be the same as the characteristics that give me confidence.

I believe that the "goodness" of a trading system is defined by each individual (or trading company) well before system testing begins -- at the time when the objective function was being defined.

If the combined out-of-sample results of a walk forward process produce results that are satisfactory in terms of the definition of the objective function, then that should give the level of confidence needed.

Thanks,
Howard
 
Hi Tech/a --

You wrote: "How do you do your out of sample testing,what software do you use?"

I use AmiBroker for most of my development work. In chapter 20 of my book, I describe using AmiBroker to perform a walk forward analysis.

At present, it is necessary to set the dates for the in-sample and out-of-sample periods "manually," but I would not be surprised to see AmiBroker extended to have automatic walk forward capabilities in the near future.

Thanks,
Howard
 
Hi Tech/a --

You wrote: "Or are you simply walking forward through a data set?"

I do not understand the question.

Thanks,
Howard
 
Hi Tech/a --

You wrote: "How do you extrapolate single special events that fall within the out of sample test period/s."

I do not understand the question.

Thanks,
Howard
 
Hi Howard,

Amibroker has been around for about 5 years i think.
You mention having written a paper in 1969 and so I assume you mustve already been a top trader for a number of years before amibroker was released?

What did you use before amibroker was around?
 
You must log in or register to reply here.

Similar threads

February 2025
2 3 4
Replies
60
Views
6K
ducati916
System optimisation: IS training vs. OOS testing, and Paper Trading
Replies
10
Views
2K
Garpal Gumnut
A
Curve fitting theory and data samples
Replies
6
Views
5K
AMSH
A
Testing a moving average crossover system
2
Replies
34
Views
15K
TradeSim
T
R
The Financial Endgame
Replies
18
Views
9K
qldfrog
Menu
Log in
Register
Cookies are required to use this site. You must accept them to continue using the site. Learn more...