Newbie... portfolio backtesting results

[DeepState]

My ignorance and luck has served me well.
As I have clearly in correctly designed and traded systems profitably
Very profitably.

Your solution then is not systems developement and trading them.

What is?

So having a platform and getting some backtest that showed, you know: low drawdown, high ratios etc. and trading it is sufficient? Apparently....that's all there is to it. My bad!

 

[tech/a]

So having a platform and getting some backtest that showed, you know: low drawdown, high ratios etc. and trading it is sufficient? Apparently....that's all there is to it. My bad!

Well you have to trade it!

It has been in my case
obviously an outlier.

Still I'm interested in your alternative

 

[DeepState]

Well you have to trade it!

It has been in my case
obviously an outlier.

Still I'm interested in your alternative

Well, in order to win, you have to play. So having to trade it is necessary for investment success. Of course, trading often doesn't lead to success.

The missing pieces are finding something in the patterns that has a shot of actually being sustainable and predictive, re-hashing that to produce something which can be implemented, risk managing and carefully implementing it. Just those minor details. Of the number of backtests which can be produced that show a good outcome, the proportion of which contains any decent predictive ability is vanishingly small.

Something which fails backtests can reasonably be included.

---

Here's an example to just give this a tad of flavor.

We smash the database with a billion algo structures (binary, neural net, linear, n-dimensional state-dependent, CART, HORSE...) and a gazillion parameters involving a truck load of expensive data. Amazingly, millions of combinations produce very favourable outcomes. That's because we stopped the run at 5 million outstanding backtests. We could keep going and produce a billion. We can vary universes on a time dependent basis. Some profitable signals include things like: if a stock is up by 10% on Tuesday for the last three weeks in combination with no more than one being negative, then double down on Thursday following the Grand Final. It works even better if I add a Fourier transform to vary the signal strength..... really? It's even better if we adopt formulae designed to model the breeding patterns for rabbits and adapt that to market support levels. Do this and some will win. Some of those will publish, some of those books will sell. Yet, strangely, a lot of them don't seem to translate to the promises made when adopted by the masses in search of wealth which is rightfully theirs. The profitable ones are, strangely, a similar proportion to that which would be expected from chance alone.

In the same dataset, we find that stocks which have been re-rated by the analyst community generally move in the direction of re-rating. Wow...no Fourier, Tuesday, Grand Final... So, if the people who influence the market via their opinions or because their opinions give reasons for HNW advisors to call their clients with a story actually impact the market via observable channels (you can see that the analyst changes his mind, calls clients and hear them changing their minds and they trade etc..) you have a visible cause and effect which is observable and testable (what if only two analysts move of ten, and one only by a bit?). I might give a little more likelihood to this signal actually being worth something relative to our friend above. I would give it a lot of validity if I knew I could be amongst the very first who got a hint that such a change of mind were forthcoming if these were generally followed with an actual change which was broadly announced.

People who do this for real, in the detail, tend not to publish their formulae until after the idea has been exhausted or to put off others from the scent via a false publication. The book sales are just nothing compared to the value of the formulation. If I did leak formulae still in use and published for kudos, for example, I would have been totally cratered and deservedly so if this were not authorized.

Still....it's not enough. You also need a free set of steak knives. You've got to check that it can be implemented. You might wish to verify it in other markets etc. The stuff that goes on here is somewhat special and we are hitting walls on Monte Carlo definitions.

Then you produce some sort of decision rule to give exposure to it (along with strategies B through Z) and wrap it up in some decent risk management.

---

Roughly speaking, producing a backtest which works is basically very close to worthless. Producing one that works for reasons you can see and whose transmission channels are understood and viable for ongoing value creation is worth somewhat more. Getting value out of it is yet another challenge. Not blowing up along the way to getting rich off these ideas is another one on top of that.

If a $399 professional edition pack from Amibroker was all it took to go from newbie to hero, why aren't yacht sales higher than they are and why are clearly super-smart guys like WYSIWYG and KTP here talking about their backtest experience and how it varies from the actual outcomes they experienced?

If it were as easy as producing a platform and coding up something which found a favourable set of stats... yet, it can be...if you are lucky. So, I guess, it is reasonable to limit your endeavours to finding a backtest formula that works and just trading it for substantive profit...so long as you are also lucky. Do you feel lucky.. Duck?

 

[fiftyeight]

So let me join in on this game I have 5 secs.

I may regret entering this debate and I assume someone will make me regret referencing Wikipedia but I am on site and tired....

This is how I understand what is being debated.

So I have just bought a system that promises I can get rich in just 6 months, I decide to run some tests

Monte Carlo methods vary, but tend to follow a particular pattern:

1.Define a domain of possible inputs.
2.Generate inputs randomly from a probability distribution over the domain
3.Perform a deterministic computation on the inputs.
4.Aggregate the results.

1.I want to test my new fancy system and I want to see how it preformed over 6 month periods within the last 20 years. My system buys/sells and can hold for seconds to weeks and can be switched on at any time. We therefore have very large number of 6 month periods to test which would be impracticable

2. So I generate some inputs randomly from a probability distribution over the domain

3. I run my new fancy get rich in 6 months System on the inputs from above

4. I realise I have been scammed.

Is this a Monte Carlo Test?

Maybe my system will give better results if I run it for a longer period?

We scale up, make our period say 10 years, still giving a larger number 10 year periods to test. Then to 19 year periods, still a larger number initial inputs but.....

We have now reduced our initial conditions to those present in year 1. We do not have inputs randomly generated from the entire domain, rather only those present in the first year.

So we are no longer running a true Monte Carlo Test but an approximation of one using restrictions on the initial conditions.....

That took much longer than 5 secs

 

[fiftyeight]

If it were as easy as producing a platform and coding up something which found a favourable set of stats... yet, it can be...if you are lucky. So, I guess, it is reasonable to limit your endeavours to finding a backtest formula that works and just trading it for substantive profit...so long as you are also lucky. Do you feel lucky.. Duck?

You are hard to keep up with DS

To make things short and sweet are you saying that Techs system success is ONLY due to luck?

 

[AlterEgo]

So having a platform and getting some backtest that showed, you know: low drawdown, high ratios etc. and trading it is sufficient? Apparently....that's all there is to it. My bad!

So you think it's not possible? I beg to differ. See attached plot of the profit of the system I developed in Amibroker. This is my real world trades with real money, NOT a backtest. There is some leverage involved in achieving this, however the real world results have been very consistent and in line with what I was getting in backtesting.

 

[AlterEgo]

Here's an example to just give this a tad of flavor.

We smash the database with a billion algo structures (binary, neural net, linear, n-dimensional state-dependent, CART, HORSE...) and a gazillion parameters involving a truck load of expensive data. Amazingly, millions of combinations produce very favourable outcomes. That's because we stopped the run at 5 million outstanding backtests. We could keep going and produce a billion. We can vary universes on a time dependent basis. Some profitable signals include things like: if a stock is up by 10% on Tuesday for the last three weeks in combination with no more than one being negative, then double down on Thursday following the Grand Final. It works even better if I add a Fourier transform to vary the signal strength..... really? It's even better if we adopt formulae designed to model the breeding patterns for rabbits and adapt that to market support levels. Do this and some will win. Some of those will publish, some of those books will sell. Yet, strangely, a lot of them don't seem to translate to the promises made when adopted by the masses in search of wealth which is rightfully theirs. The profitable ones are, strangely, a similar proportion to that which would be expected from chance alone.

No, you don't do anything of the sort. That is curve fitting.

 

[DeepState]

So you think it's not possible? I beg to differ. See attached plot of the profit of the system I developed in Amibroker. This is my real world trades with real money, NOT a backtest. There is some leverage involved in achieving this, however the real world results have been very consistent and in line with what I was getting in backtesting.

I think you'll find that I did not say it was impossible. In fact, my implications were that it was most defininitely possible. Much as it is possible for a flipped coin to deliver profit...as an example of how pure luck can lead to profit. It would be very very hard to argue that a flipped coin could not develop a profit under any circumstance. I am, most certainly, not making such a statement.

Congratulations on developing a profit.

 

[DeepState]

No, you don't do anything of the sort. That is curve fitting.

Well, I don't. From your posts, it's pretty clear you don't either. But, given that warnings about over-fitting exist, it seems rather likely that the activity takes place somewhere and in some form. From some of what is posted or in discussions I have otherwise had elsewhere, it is pretty clear that pattern search is seen to be largely sufficient by at least some....

 

[DeepState]

You are hard to keep up with DS

To make things short and sweet are you saying that Techs system success is ONLY due to luck?

Don't know.

 

[DeepState]

I may regret entering this debate....

2. So I generate some inputs randomly from a probability distribution over the domain

Is this a Monte Carlo Test?

I am getting that feeling now...

Pls expand on 2. Exactly how is this done?

From this, I should be able to ascertain whether this is Monte Carlo or something else.

 

[tech/a]

So, I guess, it is reasonable to limit your endeavours to finding a backtest formula that works and just trading it for substantive profit...so long as you are also lucky. Do you feel lucky.. Duck?

Yeh. Made a lot of errors in life but made a lot of lucky calls as well.
Haven't needed a Doctorate in Economics
Nor have I pondered a balance sheet.

Done a lot of things that my more educated friends thought at the time
was/are lunacy. Strangely when invited to the Caribbean none can join us?

I'm convinced that education is not a pre requisite to financial security.
Can even be a hindrance.

Luck comes and goes.
Always has and always will.
I agree that my very biggest and very best gains in everything
Business
Property and
Trading
Have been Right place/Right time/Right decision---Luck if you like.
DAX trading lately has been amazing luck.

To all those who haven't a degree and not likely to get one.

"Limit Risk and take advantage of quantified opportunity."

 

[DeepState]

Yeh. Made a lot of errors in life but made a lot of lucky calls as well.
Haven't needed a Doctorate in Economics
Nor have I pondered a balance sheet.

Done a lot of things that my more educated friends thought at the time
was/are lunacy. Strangely when invited to the Caribbean none can join us?

I'm convinced that education is not a pre requisite to financial security.
Can even be a hindrance.

Luck comes and goes.
Always has and always will.
I agree that my very biggest and very best gains in everything
Business
Property and
Trading
Have been Right place/Right time/Right decision---Luck if you like.
DAX trading lately has been amazing luck.

To all those who haven't a degree and not likely to get one.

"Limit Risk and take advantage of quantified opportunity."

Cheers to that Tech/A. Many roads to Rome/Barbados. Might as well take the interesting route. If we don't make it, at least the journey was interesting.

I also think your advice to the not formally tertiary educated has wider scope....