How to trade the Unholy Grails system?

[tech/a]

How does 3.33% risk per trade relates to the % risk per trade used in testing?

I don't know how Nick position sized the system in the book.
Ill ask him.

 

[qldfrog]

Thanks for the discussion enjoyed it.

And thanks for your input

 

[Lone Wolf]

Last try.
I actually have 2 visual filters in my Techtrader method.
(1) The stock cannot be in a clearly long term range.
(2) The stock must be either breaking out of a longer term down trend or be In a clear up trend.

Here is where we differ.
If they were added filters then certain stocks would not be in the scans satisfying the criteria for the system.
The same criteria used to test ALL stocks selected by the system rules.

My arguement is that the MonteCarlo results give me a deviation from the mean results using ALL stocks selected
Of say 20% low end----32% mean ----42% high end.
Picking ANY of those selected in a scan will see my long term results ( according to testing ) fall somewher between 20 and 42%. Knowing that even if I used a dart on those selected ---- unless market conditions differed markedly from those used in testing--- then that's where my profit is likely to fall.

My eyeball screen I could not code so used the above logic for justifying it's use.
Over the years of live testing the system was returning results above the mean which was 32% and the mean
27% from memory.
Frankly I have no idea whether the eyeball filter or exceptional bullish conditions tilted the returns above the mean.
I suspect the latter! But I do know that compounding/margin and pyramiding saw returns on capital invested that were and are phenomenal.

Over 6 years $30k to $360k that's over 1000% on money invested.
Many lose sight of this--- infact don't even see it!

I think/ hope this explanation helps our cause Lone Wolf.
Good discussion should be encouraged and I have enjoyed this one.

Tech,

I just wanted to say thanks for this post. It clarifies in my mind what you're getting at.

I agree with what you've said. I still believe that your two visual filters did alter the probability of where in the tested range you would fall.

If you and I were to run the same system on the same market with the same starting capital. I choose which candidates to take by random selection. You choose which to take by using your knowledge and experience of the market. I am quite confident that over a long enough time period your results will outperform mine. (The idea is based on you being a good discretionary trader who will provide an additional edge to the system).

But the important point is that it doesn't really matter, since both of us will still fall within the tested range.

 

[tech/a]

I agree with what you've said. I still believe that your two visual filters did alter the probability of where in the tested range you would fall.

That was the plan.
Keeping in trades that move and hopefully putting me in the top of the range.
As it turned out I was.
Luck or design---don't know.

But the important point is that it doesn't really matter, since both of us will still fall within the tested range.

Well that's my view your eyeballing a screened stock from screened stocks.
You could also take the first or last one screened in each batch----

 

[craft]

But the important point is that it doesn't really matter, since both of us will still fall within the tested range.

This is not necessarily correct and it is a similar problem to the serial correlation for trending systems I pointed to earlier.

Monte Carlo simulations imply randomness. The likelihood of seeing something as correlated as the results of a visual filter within a Monte Carlo range is ridiculously small if you have only run a few thousand iterations.

Keeping in mind your inherent knowledge of the odds of picking 6 of 45 numbers to win Tatts, try working out the random probability of your actual trades taken appearing from the full range of system possibilities – not forgetting that Monte-Carlo returns every ball for another chance of selection.

Randomly picking the results of your visual filter selections in a few thousand iterations would be akin to winning tatts.

 

[tech/a]

This is not necessarily correct and it is a similar problem to the serial correlation for trending systems I pointed to earlier.

Monte Carlo simulations imply randomness. The likelihood of seeing something as correlated as the results of a visual filter within a Monte Carlo range is ridiculously small if you have only run a few thousand iterations.

Keeping in mind your inherent knowledge of the odds of picking 6 of 45 numbers to win Tatts, try working out the random probability of your actual trades taken appearing from the full range of system possibilities – not forgetting that Monte-Carlo returns every ball for another chance of selection.

Randomly picking the results of your visual filter selections in a few thousand iterations would be akin to winning tatts.

Not understanding?

Your saying that it could happen that selections (Visual filter) could fall OUTSIDE of the range?
Your still convinced that this visual filter is altering the system?

 

[Ves]

Not understanding?

I think there are numerous discussions about this type of thing in Taleb's book Fooled by Randomness. He puts it pretty elegantly. It sounds like craft is discussing the issue along the same lines.

 

[craft]

Not understanding?

Your saying that it could happen that selections (Visual filter) could fall OUTSIDE of the range?
Your still convinced that this visual filter is altering the system?

The math is saying that it will Probably fall outside the range - unless of course you visual filter results are no better/worse then random.

 

[Lone Wolf]

Monte Carlo simulations imply randomness. The likelihood of seeing something as correlated as the results of a visual filter within a Monte Carlo range is ridiculously small if you have only run a few thousand iterations.

It depends on how long you run your Monte Carlo simulation for. The idea is to cover every possible combination of candidates produced by your system. You probably won't do that with only a "few thousand" runs. (It obviously depends on how selective your system is).

If you test 100% of the combinations then it doesn't matter how they were chosen. If you only test 10% of the possible combinations, then your testing was incomplete and who knows what could possibly happen.

But this is another reason to understand the system and know the backtested stats. Stop trading your system if it starts behaving outside the expected ranges and work out why its happening.

I once tested a mean reversion system I found on the net somewhere, it returned good results when I ran a quick test of 100 runs. But whenever I ran it over 1000 runs the worst case scenario was a blown account. That is the danger of limited testing/understanding.

 

[tech/a]

I think there are numerous discussions about this type of thing in Taleb's book Fooled by Randomness. He puts it pretty elegantly. It sounds like craft is discussing the issue along the same lines.

Love Teleb.Viewed many lectures and read his works.

The math is saying that it will Probably fall outside the range - unless of course you visual filter results are no better/worse then random.

OK
Explain how that's going to happen given this.

The system finds prospects (any system ) by searching a given set of parameters with a given set of variables.
The search will return a number of prospects.
In testing it will randomly select a group of these prospects and label it a portfolio and over time will return a result. ALL of these possible combinations of possible portfolios will return their own set of results.
These results will fall between a high and low range.

So I could select the first one found in the scan
or only those starting in B
Or those which display a triple bi pass pattern.
Every one of those will return a set of numbers WITHIN THE RANGE of tested results.
They cant possibly return a result OUTSIDE OF THE RANGE.
because the visual selection is FROM THOSE IN THE RANGE ONLY.

The visual selection ISN'T A CONDITION of the system.
Its a self imposed condition of which one out of the prospects selected are chosen to trade.
No different to choosing ones starting in "B"

I AGREE That there is absolutely NO correlation as to results
achieved by adding this filter. It did give me a feel good feeling that I "Thought" I maybe placing myself in trades which were more likely to perform. I was "lucky" to the extent that the traded portfolio both mine and on the forum ---were at the out performance end of the results in testing.

 

[craft]

Love Teleb.Viewed many lectures and read his works.



OK
Explain how that's going to happen given this.

The system finds prospects (any system ) by searching a given set of parameters with a given set of variables.
The search will return a number of prospects.
In testing it will randomly select a group of these prospects and label it a portfolio and over time will return a result. ALL of these possible combinations of possible portfolios will return their own set of results.
These results will fall between a high and low range.

So I could select the first one found in the scan
or only those starting in B
Or those which display a triple bi pass pattern.
Every one of those will return a set of numbers WITHIN THE RANGE of tested results. Not necessarily - because they aren't random selections
They cant possibly return a result OUTSIDE OF THE RANGE.Yes they can because they are not random and if the numbers of iterations are not large enough in relation to the number of possible permutations - they probably will fall outside.
because the visual selection is FROM THOSE IN THE RANGE ONLY.

The visual selection ISN'T A CONDITION of the system.
Its a self imposed condition of which one out of the prospects selected are chosen to trade.
No different to choosing ones starting in "B"

I AGREE That there is absolutely NO correlation as to results
achieved by adding this filter.You can't possible have any idea if your results are different to random because you didn't test it. It did give me a feel good feeling that I "Thought" I maybe placing myself in trades which were more likely to perform. this is a contridiction to saying No correlation. I was "lucky" to the extent that the traded portfolio both mine and on the forum ---were at the out performance end of the results in testing.

How many possible system entries was there?

How many did you take?

What is the number of possible permutations based on those two numbers?

What is the probability that your Non Random permutation will be picked up in 1000 random iterations?

ps.

I'm not your math teacher.

 

[tech/a]

I'm not your math teacher.

True.

I have a pretty good one with a PHD in Physics.
Ill point the thread out to him and get his input.

Ill post up the reply.

 

[CanOz]

How many possible system entries was there?

How many did you take?

What is the number of possible permutations based on those two numbers?

What is the probability that your Non Random permutation will be picked up in 1000 random iterations?

ps.

I'm not your math teacher.

Craft, the whole thing is documented trade by trade on Reefcap...Have you even had a look?

CanOz

 

[tech/a]

test

 

[CanOz]

test

Hmmm....

 

[tech/a]

Cant get a post to appear.

Will try again.

Here is the reply from my Maths guru.

I think you're both right and that you agree with each other to an extent.


I think the argument is one of Practical vs. Academic.



Technically, the choice of a subset of the prospects does introduce another variable (or variables) into the system. The question is whether these additional variables greatly influence the performance of the specific portfolios that can be chosen from these prospects, e.g., does choosing all prospects starting with A perform significantly better or worse than those starting with B?


In other words: do you care about the variability in performance between the possible portfolios chosen from the found set of prospects? If most of the possible portfolios chosen from this set perform about the expected rate of return, then that's all you care about. Anything else becomes academic.


Yes, if the shortlisted set of prospects are more numerous than places you have in your portfolio, there will be a (possibly large) number of permutations possible in choosing that portfolio and each portfolio will behave differently. If the system thus far works, however, then their performance should be similar to within the statistical variability of the system. Some will perform well, some worse, but they should always fall within a (preferably tight) distribution about the parameter set that was optimised.


Take home message: if the system thus far worked as it was designed, the extra degree(s) of freedom in choosing a final portfolio shouldn't significantly affect returns.


Really, it's an embarrassment of riches! What a luxury to be able to choose from a shortlist in the knowledge that your final portfolio should perform within certain bounds.


Some people wouldn't like this arbitrariness since it forces discretion - I'm probably one of them! Fair enough. In that case, just run a filter of your choosing over the shortlist and choose the single portfolio that results. It's almost arbitrary if all you care about is the performance range dictated by the initial system. Then the problem is: what filter is best? Which parameters should be optimised and to what bounds? This step itself would require tuning during the system optimisation process - it indeed introduces free parameters, but these are parameters that shouldn't greatly affect the bounds within which the chosen portfolio will perform.


I think the point is: if the simpler system can guarantee (within statistical margins) that any portfolio chosen from the prospects will perform within desired bounds, then who cares? If it concerns you, just tighten up the selection process with an algorithm and optimise it if you want. It may very well be that adding an optimised selection algorithm to the end of the system will find the one true portfolio to rule them all, but it could present a significant optimisation problem itself. It's a matter of preference and practicality. If it works to the specifications you want, then it works. End of story.


I think I've just reiterated everything you said...



Kris

 

[tech/a]

By the way the system was never optimized.

 

[craft]

Tech, your Maths Guru hasn’t addressed how Monte Carlo simulation which implies randomness doesn’t represent well anything that has correlation.

Trend systems simulation suffers because there is correlation in the data set which you destroy when you select a sample period of 1 for the simulation and force an implied randomness.

Choosing subsets with any sort of additional filter/routine etc also introduces non-randomness which is not modelled by the Monte Carlo simulation.

You can say its all academic v’s practical but unaccounted for correlation can take you outside of your expected range from Monte Carlo testing very easily and quickly.

Taking things to an extreme may help you see the issue – Imagine your visual filter is really good and you pick only the 50 best trades out of 100 available. What are your odds that a Monte Carlo simulation would randomly select those 50 trades with just a few thousand iterations – less than winning lotto?

The only way your visual filter is not a problem for staying with-in the probable test ranges from a Monte Carlo Sim is if your filter does no better or worse than Random – hence creating no selection correlation.

I would suggest you seek a second opinion from somebody like Howard Bandy.

 

[craft]

Craft, the whole thing is documented trade by trade on Reefcap...Have you even had a look?

CanOz

Can Oz

This is not really about Tech Trader – That’s only an example of the principle, Which is Monte Carlo simulation does not model serial correlations.

Serial correlation can be introduced in the actual data set itself or by any systematic action that is not part of the tested system parameters.

If you want a real world example read Nick Radges latest article on the Growth portfolio. – The Performance Illusion – what’s happening there is real life correlation moving things away from the modelled results – not beyond the range but significant all the same.

I thought you system guys would be all over this. It’s not academic to what you do –Its very important.

My effort to raise the issue ends here – people can dismiss it out of hand or research it – I don’t care either way.

 

[CanOz]

Can Oz

This is not really about Tech Trader – That’s only an example of the principle, Which is Monte Carlo simulation does not model serial correlations.

Serial correlation can be introduced in the actual data set itself or by any systematic action that is not part of the tested system parameters.

If you want a real world example read Nick Radges latest article on the Growth portfolio. – The Performance Illusion – what’s happening there is real life correlation moving things away from the modelled results – not beyond the range but significant all the same.

I thought you system guys would be all over this. It’s not academic to what you do –Its very important.

My effort to raise the issue ends here – people can dismiss it out of hand or research it – I don’t care either way.

lol, I've been too busy with all things unimaginable to notice...but I'll ask Nick for a copy of the article...thanks!

CanOz