Trading Model Metrics

[Willzy]

Hi RB and Howard,

First of all Gentlemen let me say how nice it is to find a forum where serious discussion, advice and debate can be had without people getting nasty about it!

RB, from your last post, I think I can conclude that your signal model is far superior to mine!

I'm curious as to what you mean by "I filter for downside volatility", is this before or after a trade entry? I have always found that higher volatility is a good thing for mean reversion systems and if I filter the entries for too deep of a pullback I miss many of the most profitable trades.

Due to my occupation, I am unable provide regular supervision of my trading system. Thus my systems must be completely autonomous, placing and managing trades in the portfolio is all done remotely and autonomously. The only monitoring I can provide my system is to check the system results against my original design.

With my current system, I have tried using a sliding window for calculation of safeF with various lengths of lookBack and weighting technique. I found at least for the system that I am currently trading that there was practically no advantage.

The simulation was extensive and computationally very expensive with MATLAB calculating safeF and CAR25 over the sliding window iterated with every additional trade. I found the response in safeF either too choppy or too slow to respond often leaving me out of the market during the most profitable time (usually just after the drawDown period had ended)

I do not doubt either yours or Howard's work but for my system I couldn't get it to help. I will revisit this area again in the coming weeks and see if I can find a set of hyper-parameters that provide an advantage. (I will follow up with my results and hopefully attach some images from the simulation)

Howard, Thanks for your analogy re the stationary environment. I suspect that my system fails to adapt well with dynamic position sizing possibly because of the methodology I have used in my system design.

It is interesting that even though the trading environment is not stationary, I have observed that the larger the data set the more stationary it appears. Increasing the dataset by trading a portfolio of stocks and backTesting over a large timeFrame has provided me with a trade distribution that seems relatively stationary (although of course it is not)

Unfortunately I cant post a PDF report from Matlab on here so a snapshot will have to do...
Below is a snapshot of my live trade distribution against the backTest, shown as an inverse CDF plot where Delta is the percentage profit or loss of each trade. The blue dots are my actual trades.


Note that Expectancy in this case is actually avgTrade/avgLoss (making it a unit-less metric). It is interesting that when I run a safeF monteCarlo analysis on the live trades only, it tells me to increase my current positionSize. The problem is my simulations showed that the system drawDowns give no warning and if I want them to be limited to my current tolerance, then I can't increase my positionSize.

It is also interesting that although dynamic position sizing and adjusting for non-stationarity seems to make sense, my research has taken me in somewhat the other direction... alas probably to my detriment... time will tell! If I'm wrong Im sure the market will tell me.... lol

RB, I like your idea of examining retained MFE and MAE I'll have a look at that as a metric, to perhaps incorporate it into an objectiveFuncion.

RB, when you have said that, "my position sizing algorithm is quick to react and reduce size. Conversely, if performance picks up it is quick to stack on size" can you elaborate on that method, are you using monteCarlo at safeF over X days or Y trades or some other method? Secondly how have you determined X and or Y hyper-parameters? Are you doing it somehow within your Amibroker backTest or outside in another program like Matlab?

Kind Regards
Matt

 

[rb250660]

Matt,

The filtering ensures that conditions are ideal and is made up of a few components. This does add significantly to achieving my desired metrics/characteristics so I don't want to give away my secret sauce, sorry. However, I will say that it is very simple and there is nothing new about the components themselves. The trick is in the way they are used in combination. The old adage applies here I guess, look at what everyone else is doing, then do the opposite.

For the sliding window I took the following approach:
- I say that 30 recent trades are 'statistically significant',
- Based on adopting a triangular distribution I double this number to 60,
- From this baseline, do some testing.

I did a bit of brute force experimenting in MATLAB to make sure the responsiveness was reasonable. I looked at the equity curve and picked times when the model accelerated into draw down and also good performance to do the investigation, a bit unrefined I know. I found a window of 30 to be a bit too responsive and ended up somewhere just beyond it. I'm sure Howard has this one sorted using a more elegant method in Python to arrive at a more optimum solution.

My thoughts are that maybe it's appropriate to consider it in the context of the optimization schedule. Say your model trades 100 times a year for arguments sake and you re-optimize every 6 months (about 50 trades assuming even spacing) - use 50 trades as your window. Maybe this is a good starting point.

Regarding the MAE/MFE thing. I don't used it as a fitness function (optimization target) and probably wouldn't recommend it. I do look at it in the very early stages of development when assessing entry and exit methodology to make sure I am not leaving too much on the table so to speak.

 

[Willzy]

RB,

No worries regarding the filter, I often feel the same way about some of my ingredients so that's totally understandable.

As for the sliding window, your procedure was basically the same as the one I attempted for my system, except I ran mine through 2005 to 2015 in one go. It's good to know it works for some systems but unfortunately it didn't have the same effect with mine :-(

Thanks regarding the MAE / MFE thing its tagged on the very long list of things to evaluate but still there!

As for automation of your ASX system, I should have clarified my question a little better... Are you still placing the trades manually? ie (sitting in front of the computer screen) or have you found a way to automate trading of ASX shares?

I only ask as I am yet to find a viable method for this.

Cheers
Matt

 

[rb250660]

Matt you have a PM

 

[howardbandy]

Greetings --

Willzy asked RB about this statement: "I'm curious as to what you mean by "I filter for downside volatility", is this before or after a trade entry? I have always found that higher volatility is a good thing for mean reversion systems and if I filter the entries for too deep of a pullback I miss many of the most profitable trades."

My research shows clearly that removing losing trades is more important than finding winning trades.

It is typical that a distribution of trades will have some losers and some winners. Given an opportunity to either remove the 5% worst trades or add an additional 5% that are copies of the best trades, removing the worst trades is preferred. It can be measured. The risk-normalized profit potential rises when bad trades are removed, allowing a higher percentage of funds to be used and increasing final equity.

I agree that mean reversion systems work well in periods of high volatility. So we want to do two things:
... Identify issues and / or time periods when volatility is high so the mean reversion system can earn profits.
... Avoid "toxic" losing trades that increase risk and lower profit potential.

I prefer filtering to avoid entering losing trades, rather than trying to manage trades that have been entered and are losing.

Systems based on mean reversion entry hold for short periods of time. Maximum loss exits seriously degrade the performance of this type of trade. Techniques such as trailing exit prices can work, but require more effort and active management. If a mean reversion trade is going to be complete in two or three days, there is not enough time for the trailing price (based on daily bars) to catch up to the trading price (based on daily bars) to give protection. It does work when the trailing price is adjusted intra-day, say with hourly bars. But that requires intra-day to use for development, intra-trade action while the trade is live, and issuing cancel-and-replace orders with the broker.

Best, Howard

 

[Gringotts Bank]

So we want to do two things:
... Identify issues and / or time periods when volatility is high so the mean reversion system can earn profits.
... Avoid "toxic" losing trades that increase risk and lower profit potential.

Hi Howard,

Do you have a preferred 'broad brush' method for avoid these losing trades in a reversion system?

My broad brush method is ROC(C,40)>0. Interested in others' thoughts on this too.

Thanks.

 

[howardbandy]

Hi Howard,

Do you have a preferred 'broad brush' method for avoid these losing trades in a reversion system?

My broad brush method is ROC(C,40)>0. Interested in others' thoughts on this too.

Thanks.

Greetings --

The answer is in the tests. Code the filter in, run the backtest, optimization, and validation. Compute CAR25 both with and without the filter.

What often happens when using filters that block out large periods -- such as being long only when the price is above a 200 day moving average -- is the lost opportunity. At the ATAA Conference in Melbourne about three years ago, Gary Stone gave a presentation showing how often large gains over short periods of time were available. There were as many during the period a broad filter would block trades as when it permitted trades.

When trades are blocked for extended periods of time, those days contribute "zeros" into the best estimate set. No loss, which is important, but no contribution to profit either.

That said, real loss and lost opportunity are different. We all experience lost opportunity every day when we think of "if only" and "what might have been." Experiencing real loss is much more painful.

Best, Howard