# Market Statistics - Finding a statistical Edge in your market



## CanOz (30 May 2016)

I'm keen to know some basic facts about a market that i might trade. I've had some time so carrying on from the Inter'l thread, here is some data on the FESX regarding when the session high / low is made. This may surprise a few people....

It is during the opening few minutes of the pre-cash open that the session high or low is made, obviously on a gap up or down. You can see the shift in daylight savings on the chart as well.

There is also another concentration of highs and lows around the US open, likely a bit spread out for news.


----------



## skc (30 May 2016)

CanOz said:


> I'm keen to know some basic facts about a market that i might trade. I've had some time so carrying on from the Inter'l thread, here is some data on the FESX regarding when the session high / low is made. This may surprise a few people....
> 
> It is during the opening few minutes of the pre-cash open that the session high or low is made, obviously on a gap up or down. You can see the shift in daylight savings on the chart as well.
> 
> There is also another concentration of highs and lows around the US open, likely a bit spread out for news.




May I make a few suggestions to your chart.

1. Shift the data with daylight saving by an hour. So your X-axis should become hours from cash pre-open.

2. Plot in 5 or 10 minute blocks. I can't tell what your interval is but 5 minute blocks should provide a clearer chart.

3. Plot vertical axis in percentage of occurrences. Whilst there is a concentration of highs/lows being made in the first few minutes of open, it's hard to tell whether it's 20% or 50% of the time.

4. Plot same data by day of week and see if there are any major differences.


----------



## pixel (30 May 2016)

Thanks, CanOz, for sharing that study.
Somebody spent a lot of effort to compile and chart that data. Would be interesting to see a similar graph for ordinary stocks - individual ones as well as groups or entire Exchanges.

Especially for the small, speccie end of the ASX, I get the impression something similar may occur. And it's easy to explain too: Usually, "surprise" announcements are published before Open, and then it's the eager beavers, aka "Amateurs", that open the trading day full of exuberance or despair. And then things settle down and price drifts back towards a mean.

Have others observes the same?


----------



## CanOz (30 May 2016)

SKC, will try and do some more work tomorrow or give it to an Excel wizard that can manipulate some more.

In the meantime here is a webinar that explains the point of using market statisitcs, i think this is quite relevant and may help some understand what we're trying to do.

Any excel wizards, if you would like to help out, i can provide reams of intra-day and EOD data.

If some has ideas, i can also get an Amibroker coder to help on the EOD data.


----------



## McLovin (30 May 2016)

Interesting chart, can. Thanks for posting.



pixel said:


> Thanks, CanOz, for sharing that study.
> Somebody spent a lot of effort to compile and chart that data. Would be interesting to see a similar graph for ordinary stocks - individual ones as well as groups or entire Exchanges.
> 
> Especially for the small, speccie end of the ASX, I get the impression something similar may occur. And it's easy to explain too: Usually, "surprise" announcements are published before Open, and then it's the eager beavers, aka "Amateurs", that open the trading day full of exuberance or despair. And then things settle down and price drifts back towards a mean.
> ...




I have observed similar. I've also observed, but have in now way tested this that they run hard out of the blocks then retrace then slowly rise again. At the speccie end announcements are usually poorly interpreted (as an example in biotechs regulatory approval is a really misunderstood concept), and companies often use announcements as a marketing exercise not having much to say, but putting in a few positive phrases that keep the sp ticking along. I think in smaller less well followed (researched) companies, the amateurs are often trading with each other and that keeps things rolling along (fomo) much longer than it does in larger mc companies. I'm pretty sure you could build a nice little living by  buying stocks with the most threads/posts on the other forum.


----------



## kid hustlr (30 May 2016)

Can,

Any chance you want to also do this for the FTSE 

Working off the assumption the FESX has follows the FTSE quite closely, I 'm surprised there's not more of a 'U' shape, I wouldn't expect many new highs and lows in the junk period after the first say 2-3 hours.

You should split this into highs and lows as well imo


----------



## CanOz (30 May 2016)

kid hustlr said:


> Can,
> 
> Any chance you want to also do this for the FTSE
> 
> ...




We'll it actually is spilt, but as skc suggested some work on the interval needs to be done to allow that visibility....


----------



## Roller_1 (30 May 2016)

Plot the high above and lows below the axis


----------



## fiftyeight (30 May 2016)

CanOz gave me some hourly data for the SPI for the last 3 years, including ETH.

This shows how many times the weekly high or low occurred on a particular day.

Next step is to find out if changing the start day of the week has any significance.

We can then look at if the market are trending or range bound shows anything interesting.

Hopefully look at some other markets as well


----------



## CanOz (30 May 2016)

Roller_1 said:


> Plot the high above and lows below the axis




Yeah that's a good idea....


----------



## Wysiwyg (31 May 2016)

If statistics prove that price moves greater than a 50/50 bet, i.e. 51/49 or better, does this make it an edge? If there is another metric other than ratio/percentage (edge?) then please enlighten.


----------



## DaveDaGr8 (31 May 2016)

It's win ratio * win percentage.
given your 51/49 split

IF your average win % was 10% and average loss % is 12% and your trade is $1000

51 times you make $100      $5100
and 49 times you lose $120  $5880

You would have a losing system.


----------



## CanOz (31 May 2016)

Watch the video....


----------



## DaveDaGr8 (31 May 2016)

Hey Canoz,

I'd be happy to have a look. I can do excel and matlab if i need to.


----------



## Wysiwyg (31 May 2016)

DaveDaGr8 said:


> It's win ratio * win percentage.
> given your 51/49 split
> 
> IF your average win % was 10% and average loss % is 12% and your trade is $1000
> ...



That is a position sizing and trade management scenario and I suppose, over a number of trades, would be a statistical result. Not sure if that is the aim of the discussion.


----------



## Wysiwyg (31 May 2016)

CanOz said:


> Watch the video....




"It's because all those results that you're being given are of a statistical nature and they do not describe what might happen to a trade when it (ahh) actually goes off." 52.45 minutes onward.  Big Mike.


----------



## CanOz (31 May 2016)

Wysiwyg said:


> "It's because all those results that you're being given are of a statistical nature and they do not describe what might happen to a trade when it (ahh) actually goes off." 52.45 minutes onward.  Big Mike.




That wasn't big Mike....that was morad speaking. What do you want wysiwyg? A silver bullet?


----------



## Wysiwyg (31 May 2016)

CanOz said:


> That wasn't big Mike....that was morad speaking. What do you want wysiwyg? A silver bullet?



I have gone deeper and deeper into the rabbit hole and found nothing more than one thing. Following the big dollars. Get in when they're in, get out when they get out. It is only the weight of money that move prices no matter how under valued or how crazy a valuation. is.


----------



## cynic (31 May 2016)

CanOz said:


> That wasn't big Mike....that was morad speaking. What do you want wysiwyg? A silver bullet?




A lead one will do! We all know how to transmute that stuff into gold, don't we?


----------



## cynic (31 May 2016)

And even if we don't,  lead bullets certainly have their uses!


----------



## CanOz (31 May 2016)

People might misunderstand the purpose of knowing your statistics....it is as much for keeping you out of a trade as it is for getting you in one. If the market has been one time framing it and it's only put in half its daily average range, one would have zero chance at a successful fade even if it's at a solid level....as an example


----------



## cynic (1 June 2016)

fiftyeight said:


> ....
> Next step is to find out if changing the start day of the week has any significance.
> ...
> 
> Hopefully look at some other markets as well




Ewe write two bead out full!

Buy inn tent sieve eggs acute shun off algae rhythmic comb pewter proses, eye fine alley no watt two eggs pecked.

Weak bee kin old tars fine all rear salt!



Pea erse:7500 daze SP500 dart tar eggs salmon neigh shun.


----------



## ThingyMajiggy (1 June 2016)

cynic said:


> Ewe write two bead out full!
> 
> Buy inn tent sieve eggs acute shun off algae rhythmic comb pewter proses, eye fine alley no watt two eggs pecked.
> 
> ...


----------



## CanOz (1 June 2016)

Not sure what you were drink'in last night Cynic but let me know where ya bought it...

Looking at the *HHI *now that i have some intraday data...

I've always suspected that the first 15 minutes prior to the cash open might be significant. 

We can test a range breakout here with a half range target and see what % of the time the system would achive a half range target after breaking out of the pre-cash range.

A 69.64% win rate tells us that a break of the range will result in continuation to a target of at least half the range. 

58% of the time the market will achieve 100% of this pre-cash opening range upon breakout.

51% of the time the market will achieve a target of 2 x pre-cash period.

Now while i have no intention on trading this as a system it is almost trade-able as a system on its own. it has a very low draw-down


----------



## CanOz (1 June 2016)

So how does this help the discretionary trader? 

1. Don't try and fade the opening range

2. Look for the potential of the move, what obstacles are in front of a move out and past the range? Set reasonable targets? Some of things i'm thinking of in order to put this in the book as a play.

3. Because i used a filter as well, the trade depends on an expansion of volatility that is setup before hand. I need to be patient for this trade to work the best.


----------



## CanOz (1 June 2016)

Also, consider how a breakout of the range, either long or short fits in with the overall context, technical, fundamental/news.

Does the market open inside or outside of its current bracket or balance area?

Imbalanced opens then to be more volatile.


----------



## ThingyMajiggy (1 June 2016)

CanOz said:


> Not sure what you were drink'in last night Cynic but let me know where ya bought it...
> 
> Looking at the *HHI *now that i have some intraday data...
> 
> ...




Good stuff Can  going to be a gem of a thread if it keeps going. 

You done anything similar on the Eurex products from their hour pre-cash session? How much intra-day historic data for FESX have you got?


----------



## CanOz (1 June 2016)

ThingyMajiggy said:


> Good stuff Can  going to be a gem of a thread if it keeps going.
> 
> You done anything similar on the Eurex products from their hour pre-cash session? How much intra-day historic data for FESX have you got?




I had yes, i think there is some data here somewhere on those tests...i'll have a search.


----------



## CanOz (1 June 2016)

CanOz said:


> I had yes, i think there is some data here somewhere on those tests...i'll have a search.




I can't find that other thread at the moment but that strategy that i tested has a filter that only would trade the opening range (pre-cash) if there was a gap up or down, so the market needed to be imbalanced. It worked quite well from memory but i don't have that code anymore. My current code uses a filter as well, but its not a gap. It doesn't test very well on the 60m pre-cash open at all, or any other duration for that matter. With or without the filter. If anything you'd be better off fading the range...not sure why that is.


----------



## CanOz (1 June 2016)

Hers the other thread where I was testing the gap orb strategy https://www.aussiestockforums.com/forums/showthread.php?t=7046&page=20&highlight=Scratch


----------



## fiftyeight (2 June 2016)

Ahhhhhh, why in a modern 24 hour world, with 24 hour news and markets with ETH, why are the stats showing what they are showing haha

Donated by a very helpful fellow ASF member

CL



NKY



FDAX


----------



## CanOz (2 June 2016)

fiftyeight said:


> Ahhhhhh, why in a modern 24 hour world, with 24 hour news and markets with ETH, why are the stats showing what they are showing haha
> 
> Donated by a very helpful fellow ASF member




Awesome, this is great. 

So now we know the probability for a high or low for the week and the time of day, or did we do those markets?


----------



## ThingyMajiggy (2 June 2016)

Interesting, so basically more often than not, every week plays out like the shape of a tick, Monday more likely to be the low of the week, pushing higher through the week to a more than likely high on Friday? Is this because we've been in a bullish market? Wonder if it's the opposite for bearish periods.


----------



## fiftyeight (2 June 2016)

This data goes back to 2009. It should not be too hard to retrospectively go back and run it for only bear markets during this period or for ranging periods.


----------



## fiftyeight (2 June 2016)

CanOz said:


> Awesome, this is great.
> 
> So now we know the probability for a high or low for the week and the time of day, or did we do those markets?




My additional charts are from daily driver. The only hourly data I have is from the SPI that I got from AOTB


----------



## fiftyeight (2 June 2016)

Really quickly while I eat.

Zoomed out the NK is ranging from 2009 - 2013, however the shape of the most probable high remains very consistent.


----------



## CanOz (3 June 2016)

Today I'm going to do some research on the question "Are the Thursday and Friday at the first of the month rotational days before the announcements (ECB/NFP). If it is true then we should be able to know that we can safely apply 'balance' trading rules, i.e. fade the range extremes.

To do this i'll pull up a couple of longer term FESX charts and simply look for rotational price action prior to the announcement times. Rotational being highs and lows but no real trend, price basically unchanged into the announcements. I'll start a spread sheet to help quantify the results. Now this will not be enough info to make a decision on, but perhaps enough to go further with the research.

I'm also going to answer another question as well "Did the market open in balance or out of balance?"


----------



## CanOz (3 June 2016)

So, what is the price action like on the first two days of the month prior to the big announcements, ECB and NFP?


Just going over the period from June 2015 to yesterday, you can see that in almost all cases, if the market opens in balance, then it will rotate around until the announcement, on both days. To explain 'in balance' means opening in the prior days range basically. If you have volume profile then it should be the prior days value area. There are times when the market can open outside the prior days balance area but still behave as a balanced market, usually because it opens in a prior balance area, which all makes sense when you think of the market in terms of accepting value.

Also, there was one day that ended up being a trend day, but there was some jawboning on that day regarding the ECB's QE program.

So this tells us what most veteran traders would likely already know anyway:

Don't be looking for big trendy moves, fade the value area highs and lows for trades back to value and the other high or low without the fear of getting run over. Its good context for the day. 

I'll probably go on and do some research around these two days and a couple of others, for example...Monday's, if they open in balance do they trade rotationally until the US opens? If they Gap open,do they always trend?

I likely won't post all the results but its just to give you an idea of ways to look for plays and context for those particular days. 

Hopefully, others can post some research of their own...


----------



## fiftyeight (3 June 2016)

CanOz said:


> So, what is the price action like on the first two days of the month prior to the big announcements, ECB and NFP?





Great post CanOz. 

These kinds of questions are a great way to gain some perspective on how things are moving


----------



## CanOz (3 June 2016)

Actually the ECB only reports 8 times per year on rate decisions....so the two days is mostly waiting for the NFP. There 3 occasions, including this week that the ECB reported the day before the NFP report. 

I'm trying to look for trend days after they report, but not seeing much in it yet.


----------



## fiftyeight (3 June 2016)

CanOz said:


> Actually the ECB only reports 8 times per year on rate decisions....so the two days is mostly waiting for the NFP. There 3 occasions, including this week that the ECB reported the day before the NFP report.
> 
> I'm trying to look for trend days after they report, but not seeing much in it yet.




Are you just eyeballing an hourly chart on specific days?


----------



## CanOz (3 June 2016)

fiftyeight said:


> Are you just eyeballing an hourly chart on specific days?




Yup, we'll I was eyeballing a TPO / VP chart, a 5m and a 240 minute with the global cursor.... But I want to use that information to see if it's worth while going a step further, like getting a mean reversion strategy coded...then I can gather more stats.


----------



## CanOz (3 June 2016)

Seems a fitting night for one of these.....


----------



## cynic (4 June 2016)

ThingyMajiggy said:


> View attachment 66917




Sins ewe knead tool urn mooring leash, mite eye sir jest ewe ewes ditch atoll know take err four trance lay shun!

Ore reef knot, reed allowed!



Rear salts forded axe egg salmon neigh shun ova too honey dread ant fore demons dart are:








Pea erse: Sum won comb planed eye ewes tomb anise ill  labels.


----------



## Quant (4 June 2016)

""  Market Statistics - Finding a statistical Edge in your market ""

A thread with a purpose , watching with interest . Ultimately success in this endeavour is as close to the holy grail you will find , Mean reversion systems on indice are likely (definitely) the place to look . I believe a volatility filter is an essential part of this process ( hint hint )  .

Certain days are definitely statistically superior for highs and lows as had been discussed , certain days are statistically superior on being trending days . Intraday reversals similar traits timewise . There are a plethora of edges to be found , developing systems to exploit said edges is the hard part


 Hope this thread gets traction without getting the typical derail  ... carry on  

In the Wise Words of Howard Bandy whom we are very lucky to have on ASF , he has the keys to finding the answers , pursue them with vigour


----------



## Quant (4 June 2016)

This podcast definitely worth a listen for those heading down this road 


https://chatwithtraders.com/ep-070-jeff-davis/


----------



## get better (5 June 2016)

Quant said:


> This podcast definitely worth a listen for those heading down this road
> 
> 
> https://chatwithtraders.com/ep-070-jeff-davis/




Thanks for the link!


----------



## fiftyeight (15 July 2016)

I was trying to replicate some stuff I found on TH’s old blog (Re-reading it). I was looking at gaps and found this. 

Has something structurally changed in the NKY that could cause the fall in the %gap or is something going on with my data?

This posed an interesting thought for me. If you do identify something while playing with stats, how do you make sure it is actually something interesting and not a change in data or some other structural change?


----------



## History Repeats (15 July 2016)

Look at time period of your chart and the index


----------



## fiftyeight (15 July 2016)

History Repeats said:


> Look at time period of your chart and the index




Yeah it was range bound during this period so I had a look at the below.

The high low range although more volatile, remains relatively unchanged. Unlike %Gaps


----------



## fiftyeight (22 July 2016)

I am thinking about trading the close of the es-mini. I have some daily data so thought I would have a look at how far off the lows does the close finish on a down day.

There have been 810 down days since 27/01/2009. 

About 50% of the time the market recovers 0-27% off the lows. 18% of the time it recovers 10% of the range or less. I think this probably falls in lines with people's general perception of the markets so not that fruitful but thought I would post anyway.


----------



## cynic (22 July 2016)

fiftyeight said:


> I am thinking about trading the close of the es-mini. I have some daily data so thought I would have a look at how far off the lows does the close finish on a down day.
> 
> There have been 810 down days since 27/01/2009.
> 
> ...




Interesting analysis, just the same.

One puzzling thought does spring to mind though. Until the market closes for the day, how do you determine that the daily low is already in place?


----------



## Gringotts Bank (22 July 2016)

cynic said:


> Interesting analysis, just the same.
> 
> One puzzling thought does spring to mind though. Until the market closes for the day, how do you determine that the daily low is already in place?




Yeh there's look ahead bias built in.  Not tradable.  

The only thing you could do is use the (high-open) stats from the previous days, if today is likely to be a down day, and trade around the open.  Or vice versa.


----------



## cynic (22 July 2016)

Gringotts Bank said:


> Yeh there's look ahead bias built in.  Not tradable.
> 
> The only thing you could do is use the (high-open) stats from the previous days, if today is likely to be a down day, and trade around the open.  Or vice versa.




Following on from your logic, instead of using EOD, one could use intraday data to obtain an interim daily low (at an earlier than close cut off time) and trade the residual period.


----------



## fiftyeight (23 July 2016)

cynic said:


> One puzzling thought does spring to mind though. Until the market closes for the day, how do you determine that the daily low is already in place?




Yeah I did think of this, if I find anything of note I will have to investigate further. 

I did find this interesting though.

After a recovery or a sell off back to the open, there is an almost 17% range where the es has not closed in 1854 trading days. (could also be a very tight range, but then I would not expect this gap)

Not sure if it my excel skills or data or something interesting yet? Further investigating required


----------



## Kryzz (24 August 2016)

I'm really surprised this thread hasn't gained a significant amount of attention. After listening/watching through the podcasts and videos in this thread, it is really interesting stuff. 

From anyone's experience, is there a resource where such statistics are readily available? Or do most people generate these themselves?

I've been doing a lot more basic stuff with some major FX pairs, with an EOD approach.

As mentioned earlier in the thread, it's a not a trading system but definitely can provide a market bias. Using the example below, looking at the past 20 years of data there's a 70%+ chance that April will be a positive month for the AUDUSD.

In my mind this takes a lot of the guess work out, you would more inclined to keep your eyes peeled for long only setups. 

I will be going through a lot of the major FX pairs and recording the statistics, if anyone is interested I will post up the spreadsheet accordingly.


----------



## Gringotts Bank (24 August 2016)

Kryzz said:


> From anyone's experience, is there a resource where such statistics are readily available? Or do most people generate these themselves?




There's plenty of blogs around.  Generally such information is not presented as a table of statistics, but rather as a complete trading system in a backtest.  

Say you use AMIbroker, you can for example select/optimize buy conditions based on the month/week/day/hourofday/minuteofday in a single line of code, then match it with a corresponding set of sell parameters.  Statistics are a good way to find potential edges, but a complete system requires much more.  Knowing April will have a 70% of being positive is a very long way from being something you can trade, but it's possibly a useful piece of knowledge.  You can accumulate such chunks of knowledge and try to put them into a tradable system.

Everyone will be interested in your stats - post 'em up.


----------



## Wysiwyg (24 August 2016)

Kryzz said:


> I'm really surprised this thread hasn't gained a significant amount of attention. After listening/watching through the podcasts and videos in this thread, it is really interesting stuff.



A securities market statistician might be able to prove me wrong, but from my experience, statistics are bound to the data from which they are gathered. Extrapolating a guaranteed winning strategy from the gathered statistics is not a given.


----------



## fiftyeight (24 August 2016)

Wysiwyg said:


> A securities market statistician might be able to prove me wrong, but from my experience, statistics are bound to the data from which they are gathered. Extrapolating a *guaranteed winning strategy *from the gathered statistics is not a given.




This is not why I like looking at stats at all.

The reason I use stats is because I do not have 20 years experience in any market. I have no idea of how a market tends to behave under certain conditions.

Even if I did have 20 years experience, the human memory is so fallible that it is not the best source of stats.

Simply trying to have a better understanding of how they "normally" move


----------



## Kryzz (24 August 2016)

fiftyeight said:


> This is not why I like looking at stats at all.
> 
> The reason I use stats is because I do not have 20 years experience in any market. I have no idea of how a market tends to behave under certain conditions.
> 
> ...




Agree with the above, I would be looking to use statistics to gain an insight/bias toward market behaviour not necessarily the basis of a trading system in its own right.


----------



## CanOz (25 August 2016)

Great posts guys....once I get back I hope to be able to contribute some more to this with some back test results as well....


----------



## Wysiwyg (3 December 2017)

*statistic *
*noun*
1A fact or piece of data obtained from a study of a large quantity of numerical data.



Wysiwyg said:


> I look at the bottom line - profit/loss. The reasons why for either are in my head because in my experience the market isn't statistical and stats. tell me nothing of use.





tech/a said:


> That’s because you don’t know what to look for and what it is you need to find
> And why it would be useful to you.



Open to assess any statistic you deem useful regarding trading.


----------



## tech/a (3 December 2017)

I’m interested in any statistic you feel meaningless ( of no use ) 

Because I don’t think you know what to look for
What it is you need to find and how you go about finding it.

You have demonstrated over the posts I read of yours that you don’t have a structure 
The only stat your interested in is profit

How do you know — without structure—— 
That your trading even has a chance of being profitable.
Nothing that you have demonstrated over the months indicates 
That you do trade profitably.


----------



## Roller_1 (3 December 2017)

What do you mean, everyone knows if you trade trends and don't double your money then you're not doing it right...


----------



## tech/a (3 December 2017)

Roller_1 said:


> What do you mean, everyone knows if you trade trends and don't double your money then you're not doing it right...




Thing is that if you don’t then data analysis will tell you why and what you need to do.

So if “ Everyone knows “ enlighten me on what it would be that I’m not doing right?


----------



## Wysiwyg (4 December 2017)

Wysiwyg said:


> Incredible number of stocks trending up. After the quick buck chasing novelty has blown up several times and the search for an edge has been exhausted, one eventually comes around to the time proven EOD trend following. *It can be slow, it can be boring but after a year if your account hasn't doubled you're not doing it right.*



My above post is bugging you so I will put you out of your misery.



Roller_1 said:


> What do you mean, *everyone knows* if you trade trends and don't double your money then you're not doing it right...



Mate, look at the sentence end and what do you see? It is toungue-in-cheek.
"everyone knows" (reading glasses are essential).


----------



## Wysiwyg (4 December 2017)

tech/a said:


> I’m interested in any statistic you feel meaningless ( of no use ).
> 
> Because I don’t think you know what to look for
> What it is you need to find and how you go about finding it.



All of them but willing to be shown some statistical use.


> How do you know — without structure——
> That your trading even has a chance of being profitable.
> Nothing that you have demonstrated over the months indicates
> That you do trade profitably.



As I have posted twice before, I BOUGHT A HOUSE (5 years ago now) and my earnings go into mortgage repayments and living expenses. However it is convenient to trade stocks in my superannuation account which I do so EOD using the experiences I have gathered. I can show you what stocks positions I have open at the moment and can assure you the account is profitable. Back to the statistics stuff now and if you have something useful for trading then please, enlighten us.


----------



## rb250660 (4 December 2017)

And this is why these forums are ****...


----------

