Skate, nice post here along with your previous posts, you raise some interesting questions!
I admire the energy you bring to this forum, long may it continue!
Let's start with the comment of a legend:
“It is remarkable how much long-term advantage people like us have gotten by trying to be
consistently not stupid, instead of trying to be very intelligent.” - Charlie Munger
How can we follow this advice, instead of trying to be too smart just try to be not stupid?
If a farmer is trying to sell off his geese that lay golden eggs are we stupid enough to buy them?
Even if the farmer is a top bloke: will he just sell us the eggs?
Where to start?
Stock prices are data presented to us in a time-series format.
From this data, and maybe combined with some other data, we want to make money.
What is the character of the time series data, this sequence of numbers we pore over?
Some people say the data alone are just random numbers and any attempt to make predictions using such a sequence of random numbers is futile: if you can predict random numbers just a little you would be the scourge of the casinos!
(Note: some people believe stock data is 'fractal', eg Nick Radge. Any discussion of random data could also include fractal data.)
If share prices data are randomly generated or a fractal then any mathematical operations applied to such time-series data such as addition, subtraction differencing etc etc will not be of any help to make the series.
Much of traditional statistics lore cannot be applied since the share prices are autoregressive ie a given value is highly correlated to the value at the previous time period.
Others say if the data are random how do you explain the obvious trends in the data that we see?
This question regarding randomness has been asked and also answered (sort of) here:
Consider following inputs: 1,1,2,3,5,8 - it isn't random 2,4,8,16,32 - this neither 4,1,2,11,5,9- this one looks like random-sequence I would like to ask if is there such algorithm to prove if ...
Basically, one can never prove a sequence of numbers is randomly drawn or not but one can get somewhere:
Just when one thinks we are on top of things:
Start of quote:
The seven states of randomness in probability theory, fractals and risk analysis are extensions of the concept of randomness as modeled by the normal distribution.
These seven states were first introduced by Benoît Mandelbrot in his 1997 book Fractals and Scaling in Finance, which applied fractal analysis to the study of risk and randomness.
This classification builds upon the three main states of randomness: mild, slow, and wild.
The importance of seven states of randomness classification for mathematical finance is that methods such as Markowitz mean variance portfolio and Black–Scholes model may be invalidated as the tails of the distribution of returns are fattened: the former relies on finite standard deviation (volatility) and stability of correlation, while the latter is constructed upon Brownian motion.
These seven states build on earlier work of Mandelbrot in 1963: "The variations of certain speculative prices" and "New methods in statistical economics" in which he argued that most statistical models approached only a first stage of dealing with indeterminism in science, and that they ignored many aspects of real world turbulence, in particular, most cases of financial modeling.
Wild randomness has applications outside financial markets, e.g. it has been used in the analysis of turbulent situations such as wild forest fires.
Using elements of this distinction, in March 2006, a year before the Financial crisis of 2007–2010, and four years before the Flash crash of May 2010, during which the Dow Jones Industrial Average had a 1,000 point intraday swing within minutes, Mandelbrot and Nassim Taleb published an article in the Financial Times arguing that the traditional "bell curves" that have been in use for over a century are inadequate for measuring risk in financial markets, given that such curves disregard the possibility of sharp jumps or discontinuities.
Contrasting this approach with the traditional approaches based on random walks, they stated:
We live in a world primarily driven by random jumps, and tools designed for random walks address the wrong problem.
Mandelbrot and Taleb pointed out that although one can assume that the odds of finding a person who is several miles tall are extremely low, similar excessive observations can not be excluded in other areas of application.
They argued that while traditional bell curves may provide a satisfactory representation of height and weight in the population, they do not provide a suitable modeling mechanism for market risks or returns, where just ten trading days represent 63 per cent of the returns of the past 50 years.
The difference ten days make:
By removing the ten biggest one-day moves from the S&P 500 over the past 50 years, we see a huge difference in returns.
And yet conventional finance treats these one-day jumps as mere anomalies.
End of quote.
Skate, your comments re failure of backtesting are valid in light of the last sentence!
I have attached the aforementioned pdf.