Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypotheses

[sinner]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Nevertheless its still a crude system that fundies often say works. If such ideas are good there are more ideas out there that should be better considering how dumb it actually is.

It's not necessarily "good" as a trading system, but due to the possibility of everyone holding, that makes it a good benchmark.

For example, there is not much point benchmarking a $100bn manager with someone who invests in mining juniors on the ASX, and by the same token it's not fair to benchmark small investors against their ability to hedge credit risk with CDS...so the capweight is a good benchmark for everyone big and small.

(keep in mind that I am referring to a portfolio that consists of both equity and credit here, not just capweight stocks).

Now, the funny thing about benchmarks is that most people (including hedge fund managers etc) mistake the benchmark as being a "baseline" whereas, over large enough sample of traders and trades, you will find the benchmark is actually a lot closer to the "ceiling" than you may have originally considered, especially when using leverage (yay exogenous shocks). This holds even more true for small investors without access to volume brokerage, tax advantages, etc.

This leads to the misconception that some "tweaks" will add value and allow one to outperform the benchmark (although it has been demonstrated that some tweaks actually do) and through these tweaks actually lose alpha.

Coupled with behavioural issues, the chance of them "adding value" to an extent that would generate statistically significant alpha for sustained periods across market regimes is quite low (if they even survive that long).

You may be mistaking recognition of these facts and subsequent recommendation that would-be investors take advantage of index funds to mitigate most of these issues, and thereby outperform the "average investor" (although some might be forgiven for thinkings so, this doesn't actually equal making profits), as a recommendation that these strategies are "good".

Slight difference I think.

 

[Trembling Hand]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

LOL Sinner.......

 

[sinner]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

LOL Sinner.......

I know, I know...you are probably thinking "gee, considering a rant like that, what are you doing thinking you can outperform the market?"

Well...like I said, some "tweaks" have been shown to add value! Plus, not everyones goal is to outperform the index, or beat the average investor (who's return will have a slight to moderate drag vs the benchmark, and which may be deeply negative at points in time), or even to make it big.

My goals have shifted over time and I care much less about getting rich or whatever compared to the stress levels it caused me, so at least for the portion of capital I allocate to things I deem as "risk" assets, I'd be happy to nominally double the capital base every 10 years or so, at the lowest volatility and drag on geometric returns as possible. That's only ~7% p.a. after tax, not such a crazy expectation, although I'm seeing it will be more difficult in the next 5y than the last.

Just as an example of goals...this sort of equity curve is not that hard to achieve as it turns out...

 

[Trembling Hand]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

I know, I know...you are probably thinking "gee, considering a rant like that, what are you doing thinking you can outperform the market?"

Sinner I was just trying to point out something that index tracking by the nature of indexes, that is they generally reward winners and cull losers, is a system. A blunt TA system, yet its mostly the fundies who champion it. I was wondering if they see it as similar to what TA'ers do because it looks like a trend following system based on quarterly bars.

 

[sinner]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Sinner I was just trying to point out something that index tracking by the nature of indexes, that is they generally reward winners and cull losers, is a system.

I'm not actually disagreeing, more just pointing out that it is, out of the entire basket of systems, a unique system.

A blunt TA system, yet its mostly the fundies who champion it. I was wondering if they see it as similar to what TA'ers do because it looks like a trend following system based on quarterly bars.

It's obviously not a blunt TA system though, because that of how the universe of stocks is created. It has a few of the attributes of a TA system, in that it does reward winners while culling losers, but the effect isn't perfect or intentional, since the correlation between returns and market capitalisation is not perfect. A 200% return in your share value is not going to guarantee you entry to the index, you actually have to be in the 80% market capitalisation bracket. There is a hysteretic entry point, but it's fundamentally based.

Nor are winning stocks always rewarded while losing stocks always punished.
Sometimes stocks in the index get punished out of the index simply because they no longer represent the makeup of the economy.
Sometimes stocks are rewarded into the index because they are a growing sector of the economy.

 

[systematic]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Apologies if it's too late in the thread to ask. tech/a, I need some clarifcation. Are you looking for what different people count as sufficient evidence (in an imperfect world) for a strategy to be considered empirically robust?

That seems to be slightly different to the wording of your original post, but I think (from more recent replies) that it is what you are after.

If I have this correct (and I don't know whether I do or not)...can I ask what your interest is? Meaning; are you looking to broaden the ideas base of how to assess a system idea, for example? The reason I ask is because you'd already be familiar with the way several traders (e.g. Nick Radge) evaluate robustness. So, is it that you see any flaws in the type of stats that trading software (such as Amibroker - which I'm not familiar with) spit out? Or are you curious about other ways of assessing robustness?

Or have a misread you entirely, and it's not, "how to evaluate the robustness of a strategy" that you are talking about?

The reason I need the clarification is that - as I mentioned above - you originally asked simply for examples of papers with rigorously tested results. There are many examples of empirical papers...with various levels of rigor applied to the tests. It's no different in academia than it is in the world of practioners.

I have just been listening to the podcast interviews done by Better System Traders. There have been some on there that have said things that have made me thought, "I could never trade one of their systems". Doesn't mean it doesn't work for them. I just wouldn't consider it 'rigorous' enough testing to give me the confidence to trade.

The academic world and the world of practioners is no different - there are simply different levels of quality in testing...and in interpreting the results.

One thing has come through from nearly all the interviews (if not all) and I wholeheartedly agree with it. The whole point is to be 'rigorous enough' (to use your word) to give you the confidence needed...to keep trading the thing...especially when times are bad. Well, that's the psychological part.
Of course, the initial stage is simply that we want to ensure (as much as is possible) that what we are looking at is real, and not a mirage. That's a good thing, and I think it's what you are now talking about in this thread.

This is a question in itself, debated in academia (actually some good papers in recent times about "how we should test / interpret results"). There's no 'perfect' answer here, either. But I think we can get close. Close enough to be able to place a trade, at least.

In my opinion, there is the statistical aspect of interpreting result - which is a field of its own.
However, there are also qualitative factors to consider, at least for me. I've mentioned my own shopping list before.

Stuff like:
The idea must have been around before the current age of computer testing (with evidence in literature). Therefore it must work across time.
There simply must be international evidence that the thing works (too much is worked on with US data to be confident if that's all it's been tested on). Therefore it must work across geography.
For me, I'm happy if the thing works across asset classes.

I want an economic theory (intuitive reason) why the thing works. I'll read the risk based reasons for interest (yawn), but I - since I began - have found the behavioural reasons most compelling / intuitive. It goes to the core of who I am / my philosophy as a trader. So, I guess that's my bias.

And of course...back to where we started...there has to be a strong empirical result. For me, I just like the kitchen sink approach. All the usual stats...high t stats etc; whilst appreciating that backtesting can overfit insignificant factors...which is why a result must prove significant on its own. I want to see multiple sub-sample results (time periods / market states). I want to see parameter changes not change the result. For that matter I want to see an alternative measurement of the thing not change the conclusion. I want to see value weighted results not change the conclusion gained by the no doubt better results gained by equal weight. I want to see the data / methodology utilise all the usual practices (delisted stocks accounted for, financial information known at the time, sufficient liquidity to trade in real life etc). As mentioned already, I want to see the thing work elsewhere. I also want to see the thing tested to work after being published by academia and known about by practioners.

"I want it all, and I want it now" is going through my head, now. That's why I stick to some real basics, and tweak within those. I do not have the confidence / chutzpah to invest life savings on the latest gimmick in the factor zoo (there's a good paper by that name...one of the current crop of papers addressing this very question). At the same time, as mentioned, it's good to remember that we live in an imperfect world and that a search for absolute certainty in these matters will lead only to frustration.

Everyone's got to work out what they are going to be happy with, I guess. For example, my list is probably quite different to Howard Bandy's list (also in this thread, on page 1). I respect his ideas but don't agree with some of them. That's what makes it necessary to eventually have your own opinion (aside from the fact that some things are just true and false, in basic maths etc). As always, that's what makes the market.

 

[tech/a]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Apologies if it's too late in the thread to ask. tech/a, I need some clarifcation. Are you looking for what different people count as sufficient evidence (in an imperfect world) for a strategy to be considered empirically robust?

Yes this is a part of it.

That seems to be slightly different to the wording of your original post, but I think (from more recent replies) that it is what you are after.

If I have this correct (and I don't know whether I do or not)...can I ask what your interest is? Meaning; are you looking to broaden the ideas base of how to assess a system idea, for example? The reason I ask is because you'd already be familiar with the way several traders (e.g. Nick Radge) evaluate robustness. So, is it that you see any flaws in the type of stats that trading software (such as Amibroker - which I'm not familiar with) spit out? Or are you curious about other ways of assessing robustness?

Both and more.

Or have a misread you entirely, and it's not, "how to evaluate the robustness of a strategy" that you are talking about?

The reason I need the clarification is that - as I mentioned above - you originally asked simply for examples of papers with rigorously tested results. There are many examples of empirical papers...with various levels of rigor applied to the tests. It's no different in academia than it is in the world of practioners.

I have just been listening to the podcast interviews done by Better System Traders. There have been some on there that have said things that have made me thought, "I could never trade one of their systems". Doesn't mean it doesn't work for them. I just wouldn't consider it 'rigorous' enough testing to give me the confidence to trade.

Same page here.

To elaborate.

Over the last 20 yrs I have read countless books on a great variety of ideas.
In the last 10 yrs I get to the end of a book and find myself asking for Evidence to support the contents of the book/s.
There is volumes of hindsight evidence (Look here is the signal and here is how it panned out).
There are even pages of system test results with the mandatory waiver that past results cant guarantee future results.

Even some evidence of how failed trades are handled.

This leads me to where I am now.

(1) In an age where technology is advancing faster than a bushfire there "seems" to be a void in what is available
to people like me---Joe Average builder not a PHD in Computer Science or Maths. It seems to be rudimentary from what I can gather from those that do have PHD's

(2) Then there is the question of the ability of the user to actually design a method and know what he is looking for.
To know how to evaluate and to continually manage their method.

(3) To even be able to evaluate a software and its capabilities let alone put it to work. Are there software's available that CAN explore even the simple question "Should Support and Resistance play a role in a trading method?"'

(4) I've seen many arguments on Monte Carlo Analysis----which one is correct? Monte Carlo can be a powerful tool
from what I understand but unless I have confidence in the analysis available how can I have confidence when using it.

I want to know what I need to know---and above all I want to come to that conclusion myself!

 

[craft]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

but unless I have confidence in the analysis available how can I have confidence when using it.

What are you really looking for.

Now that the vale of misguided certainty has been lifted, I’m 'certain' you should embrace the 'uncertainty'. As much as you might want to give your resident quant a job and build another vale of certainty, I reckon the best eye – trade/investment skills are developed under the full glare of realistic uncertainty.

Do you reckon the guys in your tennis analogy know some secret about tennis or have some prior certainty about the right shot at the right time? Or are they just better players, better reactions - more practiced in the competitive conditions.

 

[tech/a]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

They would have

(1) Better coaches
(2) Better coaching Facilities.
(3) They would know what they need to know to beat an opponent.
(4) A lot better motivation as they reap the rewards of being an expert.

CRAFT

I hear where your coming from and most including myself do recognise those faces
in the crowd and trade them.
But even a simple question like.
Should I consider volume in my trading. To my mind needs an answer one I can be confident of.

I want to be able to evaluate what I'm doing.
This will give me the confidence when trading serious amounts.
Maybe its just a personal thing and I'm in the minority.

 

[craft]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

This will give me the confidence when trading serious amounts.
Maybe its just a personal thing and I'm in the minority.

No I reckon that puts you in the majority!!!!!

Confidence in yourself in the face of [known] uncertainty - that's probably a minority.

 

[tech/a]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

No I reckon that puts you in the majority!!!!!

Confidence in yourself in the face of [known] uncertainty - that's probably a minority.

Well if that's the case I know a few of these guys
No longer where they expected to be.
After 50-60 yrs they are looking back and they KNOW why they aren't where they could have been.
3 I'm thinking of directly .

(1) He under valued his under value of his 2 x $1.5 million Apartments he bought and is now left with 3 mill debt and $1.8 mill of current asset.----He was supremely confident in his purchase.

(2) He over valued his place in his market and when he bought a new business to combine with
his own he couldn't keep clients---He was and still is very confident in his ability to read his industry.

(3) He undervalued the sale of his business and the new owner saw great potential doubling his business
in 2 yrs.----He was super confident his business was about to fail.

Sure there are no hard and fast guarantees in anything in this world.
But I for one are glad that a more rigorous approach is used in Medicine/Product Quality Control/Building Codes/Professional training.

I'm glad I don't come up against to many who have confidence in themselves in the face of uncertainty, when my Safety/Lively Hood or Money are concerned.
There are plenty of investors in Melbourne and Sydney who are buying off plan apartments from developers who are at the extreme of your minority.
I'm sure there are lots of Trading packages/Newsletters/methods and Systems who also fit in your group.

Ill stay with the majority thanks---its served me well.

 

[craft]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Well if that's the case I know a few of these guys
No longer where they expected to be.
After 50-60 yrs they are looking back and they KNOW why they aren't where they could have been.
3 I'm thinking of directly .

(1) He under valued his under value of his 2 x $1.5 million Apartments he bought and is now left with 3 mill debt and $1.8 mill of current asset.----He was supremely confident in his purchase.

(2) He over valued his place in his market and when he bought a new business to combine with
his own he couldn't keep clients---He was and still is very confident in his ability to read his industry.

(3) He undervalued the sale of his business and the new owner saw great potential doubling his business
in 2 yrs.----He was super confident his business was about to fail.

Sure there are no hard and fast guarantees in anything in this world.
But I for one are glad that a more rigorous approach is used in Medicine/Product Quality Control/Building Codes/Professional training.

I'm glad I don't come up against to many who have confidence in themselves in the face of uncertainty, when my Safety/Lively Hood or Money are concerned.
There are plenty of investors in Melbourne and Sydney who are buying off plan apartments from developers who are at the extreme of your minority.
I'm sure there are lots of Trading packages/Newsletters/methods and Systems who also fit in your group.

Ill stay with the majority thanks---its served me well.

Very easy to derail what I’m trying to say into your argument of the perils of misplaced confidence in yourself and that is a risk. But if you fully embrace and understand the uncertainties you are more likely to guard against doing dumb things. Sizing up without doing dumb things, the answer is within, not in something you will find outside of yourself.

Anyrate your journey, you have to come to your own conclusions.

 

[tech/a]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

No conclusions yet only suspicions
I may eventually agree with you.

Seems I'm in a majority so not alone.

 

[skc]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Tech/a, I am still a bit uncleared on what you are ultimately looking for.

Can you offer a hypothetical example of what would satisfy your "evidence based results"?

Are you looking for something as iron-clad as "Water boils @ 100 degree'?

Or is something like "Monday opening gaps are closed before Friday on the SPI 72% of the time" good enough?

 

[tech/a]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Tech/a, I am still a bit uncleared on what you are ultimately looking for.

Can you offer a hypothetical example of what would satisfy your "evidence based results"?

Are you looking for something as iron-clad as "Water boils @ 100 degree'?

Or is something like "Monday opening gaps are closed before Friday on the SPI 72% of the time" good enough?

I'm actually looking at rather than For

In layman's terms (mine I'm a builder)

{Relevance
Data
Software used
Language and logic
Rational
Parameters
Sophistication
Topic
Result---

Ease of replication.}

all encompassing Evidence

 

[ThingyMajiggy]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

I'm actually looking at rather than For

In layman's terms (mine I'm a builder)

{Relevance
Data
Software used
Language and logic
Rational
Parameters
Sophistication
Topic
Result---

Ease of replication.}

all encompassing Evidence

I don't think there is such a thing? Wouldn't that be like trying to make a fully automatic trading system, at some point it will probably blow up because of so many changing conditions in the market. Would be similar trying to test the actual trading system itself and get the evidence it worked, because they are all so different and it might prove certain aspects of the method/system, but not a simple goto answer of YES or NO it doesn't work.

 

[tech/a]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

I don't think there is such a thing? Wouldn't that be like trying to make a fully automatic trading system, at some point it will probably blow up because of so many changing conditions in the market. Would be similar trying to test the actual trading system itself and get the evidence it worked, because they are all so different and it might prove certain aspects of the method/system, but not a simple goto answer of YES or NO it doesn't work.

Not on the same page here.

 

[ThingyMajiggy]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Not on the same page here.

Well it seems that you're wondering if there's like some certain formula or process that can be waved over a trading system/method and it spits back out whether that system would work or not? Or I may be way off, that's just how I'm understanding it.

But I was just thinking trying to come up with that ^^ would be like trying to make a trading system at all in the first place, there's just too many variables and different conditions to get a long-lasting solid method/system(hence why most fail).

It would be hard to come up with what I said in the first paragraph because there are that many different systems/methods that surely there just isn't such a process/formula that would work on ALL methods/systems as a hard and fast goto formula that spits out whether it works or not and why(That is....other than an equity curve over time). The parallel is that there are too many conditions/variables in either scenario for there to be a long-lasting definite method, trading system or trading-system-evaluator/evidence-producer.

I'm just thinking out loud here, all of the above might actually be possible, I dunno.

 

[Ves]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

Just as an example of goals...this sort of equity curve is not that hard to achieve as it turns out...
View attachment 63508

Sinner, been enjoying reading some of your posts since you returned.

The possible strategies behind the equity curve you've linked.... where on the spectrum of active to passive do they fit? What kind of scale is required before transaction costs become too significant etc. I guess what I am really asking, is how suitable are they do Joe Bloggs who doesn't have a big capital base, but has plenty of compounding time, and a basic grasp of finance (and capability / willingness to learn more).

If not relevant to thread, probably a suitable thread on here elsewhere.

 

[Trembling Hand]

Re: Looking for papers on Evidence Based Results for T/A and F/A methods/ideas/hypoth

The possible strategies behind the equity curve you've linked.... where on the spectrum of active to passive do they fit? What kind of scale is required before transaction costs become too significant etc. I guess what I am really asking, is how suitable are they do Joe Bloggs who doesn't have a big capital base, but has plenty of compounding time, and a basic grasp of finance (and capability / willingness to learn more).

If not relevant to thread, probably a suitable thread on here elsewhere.

http://wealthmanagement.com/etfs/can-we-count-absolute-returns