Backtested Systems, Lies, Damned Lies, & Statistics

[professor_frink]

kave there is some great stuff here and I dont think that it is arguementative other than the Professor looking for a bit of a stouch.

Was it that obvious
ducati wants a period where prices decline 80% before it's a valid test- what a joke!
Why is it funny to me? If there is an 80% decline in stock prices, I personally won't care too much if my system breaks and sends me broke because we'll already be in the next great depression, there will probably be another world war coming soon, so I'll just enlist in the army to keep myself fed and clothed
Or I'll just sit on the corner with all my professor buddies who will also be out of work and we'll busk for food stamps.
Although I do agree with you tech/a- there has been alot of good discussion on here- I just couldn't help myself with that one

 

[ducati916]

tech/a

Duc.
Why is it necessary for a method designed to trade long/long trends be expected to perform as well in a bear market as a bull market?Why is it that people like yourself seem to require a method designed to do only one thing---need to be "robust" in that it should trade ALL markets at ALL times?

A methodology that can return positive numbers in all and any conditions fulfills my criteria for wealth creation. Wealth creation is where I am coming from.

I dont agree my numbers have been meaningful for 4 yrs others who started 2 yrs ago have had them for 2 yrs and those that start today will have them meaningful "In their" method for X period.

If a methodology can only produce in a specific environment, you are immediately limited to finding those conditions, and accurately recognising them early enough to monetize a significant portion thereof.

The inflection points are where even the top guys struggle.
It is at the inflection points that instigating an entry of a methodological system that relies on specific market conditions is fraught with peril.

From what youve written due to data problems both Technical and Fundamental your basically saying that its not possible to trade a plan giving meaningful numbers.

Not at all.
As you well know I have recommended TT to people.
Although I jibe you about bear markets, I actually think that it may work in a bear market, not quite as well, but you should get through.

My issue with the numbers, are several. They are essentially promulgated as statistically significant. Plainly this is nonsense.
Whats the big deal about statistical significance?
Statistical significance is about probabilities, and is about a predivtive future result, within the defined test results.

The results generated by the software, would seem not to be dealing in probabilities (as defined statistically) but dealing within a DETERMINISTIC context this is a completely different paradigm.

It is negligent to suggest otherwise. Those new to trading, (and here I will generalise a little) are looking for the fast buck, are lazy, misinformed, and eager to snap-up anything that vaguely suggests easy money for no work.

Pumping in computer (market data) pushing a button and having the answer spew back out has great appeal. I did say I was generalising, but for every tech/a & stevo, you will get ten who fit my description.

mit

I think that Duc is inventing problems that aren't significant. True there are dividends and mergers and takeovers but still all of the systems I have run over the years have been pretty true to the original backtesting for expectancy.

To a certain extent it will depend on the market that you choose.
If the ASX, then there has been no significant bear market, and this will introduce a serious bias to any results.

As for the limited period of data. I agree that this is an issue. There has been no real bear market for the last couple of decades. However, I got daily Dow data back to the 1930s and had a look at what we can expect. It is true that a bear market can move down a significant percentage but what I found is that the actual down movement is relatively quick. From a couple of months in 1987 to at most a couple of years. Most seem to be less than 12 months. After this the market moves sideways. However, there is enough movement up and down that a short term swing trader could make money. Certainly any person who made money 2000-2003 would make money in the sideway bits.

Your data, what exactly does it consist of?
Please don't say a chart.

In the US, there have been the following secular bear markets
1901-1920
1929-1932
1937-1941
1966-1981
2000-current?

Part of the problem with Bear markets is the gross increase in volatility.
In Bull markets, volatility reduces, thus, trend following has greater success.
In a Bear, the increased volatility punishes stoploss based methodologies due to the whipsaw of volatility.

Unless your *testing* has survived high, protracted periods of volatility, then you are in for possibly a very rude awakening when a Bear market turns up.

jog on
d998

 

[ducati916]

Actually this link is quite interesting as regards volatility in the US markets;

http://biz.yahoo.com/tm/060313/14038.html

jog on
d998

 

[tech/a]

tech/a

A methodology that can return positive numbers in all and any conditions fulfills my criteria for wealth creation. Wealth creation is where I am coming from.

AHHHH.Then Duc you should take a great deal of notice of the following.
True wealth creation will come from those periods of outperformance,it certaintly wont come from those periods of average and underperformance.True wealth creation wont come from one source although it is possible for one source to outperform long enough for you to recognise the opportunity and create that wealth.

In 1996 the plans for the Southern Expressway fell on my desk for tendering of the Reataining Walls.I knew from experience that areas needed a REASON to increase in value---better access was a proven one,so Wife and I bought 2 houses,as equity grew we compounded and bought another 2-----bank stopped us at 10---as this is happening along comes the housing boom. While this is happening T/T and 2 other methods are doing their thing,but without the input of the magnitude of the housing investments.

I'm selling houses now but is it necessary to be right all the time? Of course not you need only to be right ONCE if your able to recognise an opportunity and get some luck your way.(Initial capital helps a great deal)

If a methodology can only produce in a specific environment, you are immediately limited to finding those conditions, and accurately recognising them early enough to monetize a significant portion thereof.

Thats where the money will be it wont be anywhere else!!

The inflection points are where even the top guys struggle.
It is at the inflection points that instigating an entry of a methodological system that relies on specific market conditions is fraught with peril.

Its about minimising Loss,maximising Opportunity, and having the Kahunas to exploit that opportunity

My issue with the numbers, are several. They are essentially promulgated as statistically significant. Plainly this is nonsense.
Whats the big deal about statistical significance?
Statistical significance is about probabilities, and is about a predivtive future result, within the defined test results.

They dont have to be statistically significant only able to point to opportunity,this they do and are doing very well,the day they stop is the day the serious opportunist will preserve as much of his earnings as he can.

The results generated by the software, would seem not to be dealing in probabilities (as defined statistically) but dealing within a DETERMINISTIC context this is a completely different paradigm.

It is negligent to suggest otherwise. Those new to trading, (and here I will generalise a little) are looking for the fast buck, are lazy, misinformed, and eager to snap-up anything that vaguely suggests easy money for no work.

Pumping in computer (market data) pushing a button and having the answer spew back out has great appeal. I did say I was generalising, but for every tech/a & stevo, you will get ten who fit my description.

Yes you will and those who learn may well move to the next level.

Duc while you attempt to find the statistically significant way to be 110% sure you wont lose your hard earned opportunity looks you in the face and you dont see it.Nor do you know how to take advantage of opportunity without serious risk to your capital (You hate Stops) If the US market tanked tommorow your Fundamental selections would as well!
The Technical approach (well mine) would mean that as of now I would be out of 4 of your selections at a 10% stop ,so tanking today means I have preserved my funds more efficiently.I have 4 less stocks and I have already 10% less loss than without a stop.

I certaintly find I'm learning more about the Ducsta everyday.
In all seriousness Duc and as constructive comment,I feel much of what you write are manifestations of your weaknesses.

Its NOT about being RIGHT its about being IN THE POSITION to be PROVEN your right.(Not necesserily to others but more to your won judgement).


As my property example above hopefully illustrates

 

[ducati916]

tech/a

AHHHH.Then Duc you should take a great deal of notice of the following.
True wealth creation will come from those periods of outperformance,it certaintly wont come from those periods of average and underperformance.True wealth creation wont come from one source although it is possible for one source to outperform long enough for you to recognise the opportunity and create that wealth.

Or have a methodology that outperforms in all market conditions

In 1996 the plans for the Southern Expressway fell on my desk for tendering of the Reataining Walls.I knew from experience that areas needed a REASON to increase in value---better access was a proven one,

Now, did you consult a chart?
Did you wait for a breakout to new highs in the property prices?
Experience, always a vital ingredient.
A fundamental driver within the property market......being access, you recognized it, and acted upon it from an analysis of the information that became available to you.

This is a very common finding.
Property investment analysis tends towards the fundamentals far more than does Stock market investment analysis. There are several reasons for this, volatility, and the ability to analyze financial statements are but two.

I'm selling houses now but is it necessary to be right all the time? Of course not you need only to be right ONCE if your able to recognise an opportunity and get some luck your way.(Initial capital helps a great deal)

For me, I like to be right all of the time.

Its about minimising Loss,maximising Opportunity, and having the Kahunas to exploit that opportunity

Indeed.
Minimising loss is the key. That is exactly where I focus my attention.

Duc while you attempt to find the statistically significant way to be 110% sure you wont lose your hard earned

I have a statistically significant methodology.

opportunity looks you in the face and you dont see it.Nor do you know how to take advantage of opportunity without serious risk to your capital (You hate Stops)

On what evidence do you base this?
If it is simply an issue of *stoplosses* they are a requirement for technical based methodologies, as of course the whole concept of profitability is based on taking losses.

While it is possible that I may take a loss, based on a statistical probability, I will take ZERO losses, and return circa 30% compounded. That is the basis of the portfolio that I am currently running.

The Technical approach (well mine) would mean that as of now I would be out of 4 of your selections at a 10% stop ,

Exactly.
And the stocks that replace them may also be stopped out, ad infinitum.
This is exactly the exercise in futility that I wish to avoid.

I certaintly find I'm learning more about the Ducsta everyday.
In all seriousness Duc and as constructive comment,I feel much of what you write are manifestations of your weaknesses.

Now you are venturing into my favorite area......psychology.
You must expand upon my manifested weaknesses, who knows you may in point of fact be quite correct.

jog on
d998

 

[stevo]

Duc

"A methodology that can return positive numbers in all and any conditions fulfills my criteria for wealth creation. Wealth creation is where I am coming from."

Ah - so it's the Holy Grail that you seek! Or have you found it?

"For me, I like to be right all of the time."

Definitely - the Holy Grail has been found in Duc's backyard. :dance:
Nobody likes to admit that they get it wrong from time to time. But if the method is right then losing occasionally is not being wrong, it's just part of the process.

"Minimising loss is the key. That is exactly where I focus my attention."

If you are right all the time is there any need to minimise loss?

"I have a statistically significant methodology."

Statistically significant on what basis?

regards

stevo

 

[tech/a]

Now, did you consult a chart?
Did you wait for a breakout to new highs in the property prices?
Experience, always a vital ingredient.
A fundamental driver within the property market......being access, you recognized it, and acted upon it from an analysis of the information that became available to you.

This is a very common finding.
Property investment analysis tends towards the fundamentals far more than does Stock market investment analysis. There are several reasons for this, volatility, and the ability to analyze financial statements are but two.

I actually looked to see if I could find one.(There is a ready made business poeple would pay for that sort of info graphically reported)

After 10 yrs of mediocrity blind freddy would have seen a monumental breakout from a chart of property had there been one.
not only that but a massive increase in volume and monthly increases in range and continuous new highs.
Now we would see consolidation and pullbacks in some areas
If ALL areas (postcodes)were represented by charts I'll bet that there would be a great deal more accuracy available for property buffs.

You must expand upon my manifested weaknesses

I dont have to,youve written pages on the topic.

 

[ducati916]

tech/a & stevo

Guys, I am disappointed.
I had higher expectations. Looks like I can be wrong after all.

Ah - so it's the Holy Grail that you seek! Or have you found it?

Found it.
Arbitrage........100% profit...........0% losses = holy grail in my book

Nobody likes to admit that they get it wrong from time to time. But if the method is right then losing occasionally is not being wrong, it's just part of the process.

True.
But that way doesn't interest me. I want my cake, and to enjoy each mouthful in addition, of course that's why I'm a fat bastard and need to jog so much.

If you are right all the time is there any need to minimise loss?

There are no losses, because that is where I focus my attention. I have yet to master achieving zero loss with zero effort

Statistically significant on what basis?

On the basis that I have been outlining on this thread.

I dont have to,youve written pages on the topic.

Precisely as I thought, mere opinion.
When it really boils down to providing some reasoned argument, supported with some evidence..................oh dear, it all heads south very quickly.
If the evidence is available, from my posted material it should be relatively easy to document my manifested weaknesses.

jog on
d998

 

[tech/a]

Duc.

Its pretty simple.

The overwhelming need to be right is the single thing that holds you back more than anything.

Accepting that you'll be wrong more often in life than correct means that being able to handle being wrong becomes more efficient.

I'm wrong all the time in business and personal life. But Ive made a decision and hence will be proven wrong or correct. Of 1000s of decisions about 10 have proven to be life altering. A few have nearly caused me bankruptcy---but let me assure you this that if you or anyone else faced bankruptcy a call to the "Duck" would be well worth the 50c---Being wrong was the best teacher I could ever wish for!

You'll learn more being wrong (as opposed to AVOIDING being wrong)than being right.

 

[professor_frink]

well said tech/a.
Not only do you learn more when you're wrong, you learn it very quickly too!

 

[tech/a]

well said tech/a.
Not only do you learn more when you're wrong, you learn it very quickly too!

If your in a Singapore,Bali jail then its never quick enough!
In cases like those I'm with DUC just dont ever ever get it that wrong!!!

Theres a difference in being Wrong and Stupid.
However many seem to have the ability to get them both RIGHT!

 

[professor_frink]

well that's a whole other level of stupidity. I won't even get started on that one

 

[ducati916]

tech/a

Accepting that you'll be wrong more often in life than correct means that being able to handle being wrong becomes more efficient.

I see where you are coming from.
*Life* in general. I am referring to the markets.
So simply, yes I can agree that you must learn from life's mistakes, and by not committing any mistakes, you probably haven't lived. Somewhat akin to deciding to brake from 290kph @ the 50m marker, rather than the 75m marker

Now of course in that context, you are accusing me of having led a boring and sheltered life, and being spoonfed with a platinum set of cutlery.
I shall definitely visit you in Adelaide this winter, and you can decide for yourself.

The overwhelming need to be right is the single thing that holds you back more than anything.

Here I disagree.
You see I am unwilling to accept at face value that there is not a better way, or that the way proposed is in point of fact not a total crock of s***e
Therefore I shall worry at a problem until I have solved it. In the process I invariably find things that I definitely didn't know, or understand previously.
You see if I had accepted the common concensus of the expectancy calculation, I would never have realised just how misleading it actually is.

Theres a difference in being Wrong and Stupid.

Indeed there is.
It is when you are both, that time is ticking.

jog on
d998

 

[It's Snake Pliskin]

Duc,

Guys, I am disappointed.
I had higher expectations. Looks like I can be wrong after all.

Actually Stevo and Tech have had some really insightful stuff to say.

Found it. Arbitrage........100% profit...........0% losses = holy grail in my book

All forms of investing or trading are arbitrage of some sort - looking for inefficiencies in the market.

Pure arbitrage has a window of opportunity to contend with. Are you always finding that window of opportunity? :headshake Time is of the essence, as you would know.

 

[ducati916]

Snake

All forms of investing or trading are arbitrage of some sort - looking for inefficiencies in the market.

Nonsense. As of course not all inefficiencies are risk free.

Pure arbitrage has a window of opportunity to contend with. Are you always finding that window of opportunity? Time is of the essence, as you would know.

No, not always.
Competition for risk free returns are after all keen.

Actually Stevo and Tech have had some really insightful stuff to say.

Of course, these examples just weren't some of their better offerings however.

jog on
d998

 

[It's Snake Pliskin]

Duc,

Found it. Arbitrage........100% profit...........0% losses = holy grail in my book

...and then..

No, not always. Competition for risk free returns are after all keen.

So you haven`t found the HOLY GRAIL then.

 

[ducati916]

Snake

So you haven`t found the HOLY GRAIL then.

Every now and then I encounter incisive thought, so penetrating, so profound, that it almost leaves me unable to respond in a manner that behooves the genius of the observation.

Now is one of those moments.
jog on
d998

 

[wayneL]

So you haven`t found the HOLY GRAIL then.

I once thought I had found it, but alas I didn't look carefully enough

 

[It's Snake Pliskin]

Snake

Every now and then I encounter incisive thought, so penetrating, so profound, that it almost leaves me unable to respond in a manner that behooves the genius of the observation.

Now is one of those moments.
jog on
d998

The mission of arbitrage is to correct inefficiencies. People who do this pick apart strategies, concepts etc, not limited to trading/investing systems though, until they find an inefficiency or a series of inefficiencies. A bit like investing in undervalued stocks, or not?

But, inefficiencies are not always found.

Therefore, move on to the next strategy, concept etc.etc. and start again. The window of opportunity is extremely important, because you are competing with other people doing the same, looking for inefficiencies. If they beat you to it you can miss out.

So how can arbitrage be the holy grail if time is not an issue? :alien2:

 

[Shane Baker]

Those interested in confidence intervals for their trading results may wish to look at this software.

http://www.adaptrade.com/sigtest.htm.

Amazingly this thread has a strong sense of deja vu (see Reefcap).

The great thing about this software is that Duc will be able to arrive at his own confidence interval about whether he is speaking out of his "fundamental" orifice or not!