Viable trading system statistics

Gringotts Bank said:

Close. I will go to any lengths to avoid unecessary work. I have done far too much learning of unecessary crap in other fields outside of trading (stuff fed to me by experts, lecturers, academics) only to find out it's all bull****. So I'm extremely wary of time spent on things that won't be likely to yield a result. The result doesnt have to be money, it could be something else. Aside from that, life is extremely short and we'll all be dead before you know what's up, so I sure as hell am not going to learn just for the sake of it. Leave that for the academics and geeks.

Click to expand...

How will you know whats unnecessary and whats necessary? That's only in retrospect.

Charlie Munger on different mental models

...that 2hr boring online agriculture course will certainly be useful in the post apocalyptic wasteland

skyQuake said:

How will you know whats unnecessary and whats necessary? That's only in retrospect.

Charlie Munger on different mental models

...that 2hr boring online agriculture course will certainly be useful in the post apocalyptic wasteland

Click to expand...

By taking what information is offered, pulling it apart and investigating it. The investigation might involve thought experiments, real experiments, social experiments, formal experiments, informal testing, philosophizing, reality testing and so on. Then going back to the person who offered it and query them with the most direct, honest and incisive questions you can generate.

Most fold at this point. They realize the game of playing 'expert' is over. Of those who don't fold, some will be offended and decide to hit back. That's just embarrassing - leave them alone. And the remainder will recognize the amount of thought put into your questions and then you have something and someone to collborate with. It's possible to cut out a heap rubbish without going to extreme amounts of work.

sinner said:

But can you not see that the outcome is only a result of what the market did, and not any magic on the part of any system?

Which means that the returns of any system once again boils down to ones ability to predict the next market regime, or at the very least correctly classify which regime the market is currently in, and deploy an appropriate system for that regime.
.

Click to expand...

can anyone truly predict what the market is going to do though? why not have a system that works reasonably well in most conditions and maybe an index filter to shut the system down when the market conditions don't suit the system.

Roller_1 said:

can anyone truly predict what the market is going to do though? why not have a system that works reasonably well in most conditions and maybe an index filter to shut the system down when the market conditions don't suit the system.

Click to expand...

That is a prediction about what the market is doing.

Roller_1 said:

can anyone truly predict what the market is going to do though? why not have a system that works reasonably well in most conditions and maybe an index filter to shut the system down when the market conditions don't suit the system.

Click to expand...

Greetings --

When we develop trading systems we are predicting what the market is going to do. We are looking for signals in the historical data, writing rules that recognize them, and predicting / relying on the data continuing doing what it did.

A very large difficulty is that systems do not work well in multiple or changing conditions.

Envision an overly simplified system that trades on the cross of two moving averages. If it was developed using a period when the two moving average lengths were 5 and 20, it will work well as long as the data continues to cycle at 5 and 20. When conditions change to 8 and 29, the signals still come when the 5 and 20 cross, but they are no longer profitable.

As you say, we can recognize that the 5 and 20 system is not working and take it offline.

Best,
Howard

Gringotts Bank said:

I think for Howard selling a book, a 'before' and 'after' shot of the equity curve would be a nice selling point to see just how much difference it makes. It looks like implementing dynamic sizing would require one to learn Python, so I'm not going to jump in without seeing the beneifts.

Click to expand...

Howard probably rolled is eyes at this comment of mine, since it was contained in the screencast link he gave. A really nice before and after example of an equity curve is given. And the Python stuff is for machine learning, not the dynamic position sizing, afaik.

Greetings all

First off, brand new to the forum and can already tell there's loads of good info on here and some top notch contributors. I recently finished reading Dr. Bandy's latest book, and the universal objective function described therein is probably the most logical I've come across. However, it isn't readily calculated in most off-the-shelf trading system development platforms.

Back to the original topic of the thread.. My current objective function of choice combines the expectancy of the system, the dispersion of trade returns, and the number of completed trades in a 1-year period. My systems are optimized according to this statistic, and then the out-of-sample results are fed into a Monte Carlo analysis to determine how much profit potential there is given my risk level. I use ulcer index rather than Max Drawdown, as it takes into account magnitude, frequency, and length of drawdowns.

Greetings --

The limitations of the development platform sometimes restrict the development. I recommend that the developer have an objective function that ranks alternative systems into the same order he or she would rank them if the reports from the individual runs were spread out on the office floor and judges subjectively.

This does several very positive things.
1. The dissonance between the intent of the trader and the implementation of the trading system is removed. When alternatives are ranked according to your preference, you already know you agree that one ranked as best is one you subjectively prefer.
2. Whenever automatic selection of system alternatives takes place, the alternative that is ranked at the top of the list is the one used. At that time, there is no opportunity to consider alternatives that are lower in the list. The developer / trader must have confidence that the alternative ranked best is in fact the best -- or at least have confidence that the alternative ranked as best is acceptable.

If the development platform restricts the developer to a set of objective functions that have been pre-programmed, the opportunity to create and automatically use a custom objective function is removed.

Note that the most common built-in objective function is the value of the trading account at the end of the simulation period. That distorts performance for several reasons.
1. Using "net profit", or an equivalent, often selects a system that performs well in-sample but poorly out-of-sample.
2. It encourages use of compounding and inclusion of position sizing within the model, which should be avoided.
3. It rewards lucky trade sequences -- that cannot be expected to reoccur in the future.

Developing systems that trade profitably require that the distribution of trades in the future is similar to the distribution of trades in the out-of-sample test period. While the distribution must be similar, we cannot assume that the sequence will be identical. I recommend the following, including using an objective function that rewards these features:
1. Base analysis of test runs on individual trades -- just percentage or points gained or lost per trade -- no compounding, no position sizing.
2. Trade frequently.
3. Trade accurately.
4. Hold a short period.
5. Avoid large losses.

Following development runs, perform an analysis similar to that I describe in my Modeling book or in my YouTube presentation:
https://www.youtube.com/watch?v=Vw7mseQ_Tmc

The presentation on stationarity might be valuable as well:
https://www.youtube.com/watch?v=iBhrZKErJ6A

Best regards,
Howard