# TH's data, conniptions and brain dumps



## Trembling Hand (26 September 2014)

I made a comment on Monday about the way I expected the markets to react based on experience,


Trembling Hand said:


> Nasty Monday,
> They normally fail to follow through. I think I've said it a few 100 times mostly they are a buy but when they do keep on going they make for really bad weeks ahead.




So with another comment I made here,


Trembling Hand said:


> TA doesn't work. *Test it.*




I thought I would hold myself to the same scrutiny that I was dumping on the H&S and technical analysis guys and *test it.*

I grabbed the index data for XJO, HSI, Nikkei, Dax, FTSE and S&P500.

I tested each index over the last 10 years. What I was looking for was the return by the close 5 days later (Being Friday if there was no hols) if Monday fell more than 1% lower than the previous Fridays close. 

Then compared that to the 5 day return after any day except Monday were we get greater than 1% fall. The reason for the comparison is to see if there is any particular edge buying Monday drops compared to other days.







Results? Seems legit  except for the Aussie market


----------



## barney (26 September 2014)

Appreciate the work there TH.  I assume you are running the XJO data off the 4.30 close?  Any chance of running the same test off the sycom close at 24.00 hrs .... be interesting if it skewed the numbers back into the same ballpark as the others?


----------



## tech/a (26 September 2014)

You must be bored.
Stuffed your shoulder again?


----------



## fiftyeight (26 September 2014)

Selection bias, testing one comment????

I have been pondering a few comments you have made over many posts but



> TA doesn't work. Test it.




and



> Yep the pleasures of trying to make a buck hey!! In spite of having a good start and the right plan for a gap fill I just got spanked on god knows what other than a good trap when the HSI closed the ON gap and keep going.....




Some it up.

Is TA just looking for repeating patterns and trading them accordingly? (Profitably or not)

If I called the above a "Gap Fill", have you not just used your own kind of TA setup? (Profitably or not)

Not trying to argue semantics, just wondering if I have missed something obvious?


----------



## Trembling Hand (26 September 2014)

fiftyeight said:


> Selection bias, testing one comment????




 Hmmmmmmm you wonder why anyone would bother. I was just trying to show how to add hard numbers to trading ideas away from the feel good fuzzy BS newbs waste their time with.

Here is a question for you. What is the chance of a short working late Friday morning after a week of falls? How would you find that out? You could trade a "TA Pattern" that looks like a short setup but tell me.......

What are the stats on this TA Pattern you would use?

Ask yourself this - have you got 1 hard stat on a single pattern you think you trade? Either through any type of backtest or forward test?


----------



## fiftyeight (26 September 2014)

HAHA WOW 

TH thought you of all people would of picked up on a little sarcasm...

Selection bias again destroying one trade I posted??? (Sarcasm, it was a horrible trade and I know it)

After my great trading tonight (sarcasm) I went and had a look at the weekly chart and noticed not many bars seem to finish on their lows. I was going to ask for advise on how I could look into this further and plot some distributions (not sarcasm).

This seems like it may turn into a great thread that may answer a lot of the questions I am coming up with as I begin trading more.

My original question was leggit though, I am not sure if I am talking semantics or I have missed something really obvious??


----------



## burglar (26 September 2014)

fiftyeight said:


> ... Is TA just looking for repeating patterns and trading them accordingly? (Profitably or not) ...




Surely, if the patterns* are repeating* ... and one is *trading them accordingly* ... one must be making a profit.
Unless I am missing something?@!!


----------



## Trembling Hand (27 September 2014)

fiftyeight said:


> After my great trading tonight (sarcasm) I went and had a look at the weekly chart and noticed not many bars seem to finish on their lows. I was going to ask for advise on how I could look into this further and plot some distributions (not sarcasm).




I've also noticed that. And tested it here,

http://tremblinghandtrader.typepad....2010/03/where-will-the-weekly-highlow-be.html
And 
http://tremblinghandtrader.typepad....high-and-low-for-the-hang-seng-index-hsi.html

Data,




The lows and highs are distributed to Mondays and Fridays which is a great bit of info to set up trading throughout the week.

This was done in 2010 and I might do it again with recent data but read the links and you get the idea.


----------



## Trembling Hand (27 September 2014)

burglar said:


> Surely, if the patterns* are repeating* .
> 
> 
> Unless I am missing something?@!!




Thats just it. *Where are these patterns?* Head and shoulder? Break outs/downs? Elliot wave? whateever? You will not find a nice constant repeatable pattern to shoehorn the market into. If you did you wouldn't need stoplosses, money management and have win rates up around the 90% rate.

Punters waste time looking for the correct line up of magic tea leafs but thats no what makes a trader profitable. Their decisions are far more dynamic that.


----------



## lesm (27 September 2014)

burglar said:


> ..... and one is *trading them accordingly* ... one must be making a profit.
> Unless I am missing something?@!!




Define 'trading them accordingly'

Will a 'repeatable pattern' be profitable in all scenarios?

The post implies a level of determinism, which may not be valid, with a potential caveat that appears to become one of trading them accordingly.

It would still be possible to slowly bleed to death. Otherwise, as TH states 90% of traders would be successful.

If the repeatable patterns have a high probability of being successful regardless of the instrument they are being applied to, then why are there not more algorithmic systems in use, even at the retail trader level?

Maybe it depends on what's in the line up of magical tea leaves


----------



## John Swift (30 September 2014)

Trembling Hand said:


> I've also noticed that. And tested it here,
> 
> http://tremblinghandtrader.typepad....2010/03/where-will-the-weekly-highlow-be.html
> And
> ...




Hmmm. So based on those blogs...

Would a possible trade setup be to buy on a break of the Monday high with a stop at the Monday low and holding until the close on Friday?


----------



## Trembling Hand (30 September 2014)

John Swift said:


> Hmmm. So based on those blogs...
> 
> Would a possible trade setup be to buy on a break of the Monday high with a stop at the Monday low and holding until the close on Friday?




That is not what I think of when I see those stats but we all view markets differently. I run stats like this to help me understand market moves. Frame trade ideas but not necessarily trigger or produce trades.


----------



## CanOz (30 September 2014)

Trembling Hand said:


> That is not what I think of when I see those stats but we all view markets differently. I run stats like this to help me understand market moves. Frame trade ideas but not necessarily trigger or produce trades.




Sort of stuff to use in context later?


----------



## ThingyMajiggy (30 September 2014)

Great thread TH, more more more!  

Hardest part I find is getting decent data and working with Excel, spend 90% of the time rooting around trying to reverse a column or figuring out some formula or some crap


----------



## Trembling Hand (30 September 2014)

CanOz said:


> Sort of stuff to use in context later?




Yeah, apparently we are meant to trade with the larger time frame trend . Its Wednesday morning and we have gapped lower Monday, flip flop around Tuesday and now tested Mondays lows and bounced back to the highs, knowing these two data points (Monday gap downs usually fail and weeks usually trend) you would lean towards a week trend higher and be cautious shorting breaks to new highs.


----------



## ThingyMajiggy (5 October 2014)

Been working on trying to find some stats all weekend on the DAX(as well as learning Excel!) and have got this so far, nothing really jumping out at me, other than seems to close it's gap a hell of a lot of the time at some point during the day and if we get ranges towards the 100 ticks, it's likely to keep going...I think lol. My brain is starting to hurt keeping track of what I've done and all the formulas 

Not sure what sort of stuff to be looking for, wouldn't mind figuring out % of time it closes the gap within the first hour etc. 

Do you have certain things you look for TH or just keep fiddling away until you find an edge somewhere? 




P.S - Some of this might not be hugely accurate as I might have made a few formula mistakes in there, so don't go basing any trades off my findings whatever you do, need to run over it all again to make sure it's all good. Just showing how hard I'm working on it for not much reward haha


----------



## tech/a (6 October 2014)

I would think that GAP information is pretty handy


----------



## Trembling Hand (6 October 2014)

ThingyMajiggy said:


> Been working on trying to find some stats all weekend on the DAX(as well as learning Excel!)
> 
> 
> Not sure what sort of stuff to be looking for, wouldn't mind figuring out % of time it closes the gap within the first hour etc.
> ...




Good stuff. Once you learn how to test for 'something' more things start to become possible. I really just use it to add weight to ideas or nagging questions. I think you find the most useful info by sticking to what makes sense from your observations. Like how quick a gap does close. I might have a look at that one later today.


----------



## Wysiwyg (6 October 2014)

tech/a said:


> I would think that GAP information is pretty handy



I think the gap 99% is meant to be open > ref(close,-1) or open < ref(close,-1). There certainly isn't daily gapping on the DAX 98% of the time.


----------



## tech/a (6 October 2014)

Wysiwyg said:


> I think the gap 99% is meant to be open > close or open < close. .




Agree.
So would be handy I'd have thought


----------



## ThingyMajiggy (6 October 2014)

Yeah I've made some stuff ups, the gap figures don't make sense to me. I have done if open > or < previous close, but need to modify it more as I think I've done the wrong parameters, as I said before this is just the stuff I'm TRYING to work on so don't go off my figures/findings, but more of it is learning excel and thinking straight about what I'm trying to calculate, can turn into a bit of Inception sometimes


----------



## CanOz (6 October 2014)

ThingyMajiggy said:


> Yeah I've made some stuff ups, the gap figures don't make sense to me. I have done if open > or < previous close, but need to modify it more as I think I've done the wrong parameters, as I said before this is just the stuff I'm TRYING to work on so don't go off my figures/findings, but more of it is learning excel and thinking straight about what I'm trying to calculate, can turn into a bit of Inception sometimes




If you get sick of that i have an eLancer that is an absolute ace with Excel and he's very economically priced as well.


----------



## ThingyMajiggy (15 October 2014)

Have been fiddling around with some more stats for the DAX, I'm also doing some tutorials on Excel to hopefully make things a little easier and so I can do more stuff.

I went over the last 12 months hourly data and found that we had 249 gap opens, and that it only closes the gap in the first hour 31% of the time, which obviously means that 70% of the time it doesn't close the gap in the first hour, which lines up with what I've seen, tends to have a push the opposite way on open and just mess about until cash opens then it has 2 or 3 angry swings, settles and _attempts_ to head toward gap close. This took forever, went through it all manually, had to find all the first hours bar data on thousands of rows, probably an easier way but I got there in the end! 

Now I wouldn't mind finding out what happens in the first hour after a down/up day, say we have a nice trend days or strong moves at the ends of sessions, what tends to happen initially on the open the next day(short covering, profit taking etc.).


----------



## Weatsop (15 October 2014)

John Swift said:


> Hmmm. So based on those blogs...
> 
> Would a possible trade setup be to buy on a break of the Monday high with a stop at the Monday low and holding until the close on Friday?




Monday (by those figures) is the low-of-the-week one third of the time (146 out of 428). The other two thirds, the low is on another day. I doubt you could get a programmable system out of this (you'd need to look at the amount you'd get or lose on each week).

...but you could drill down into those 146 Mondays and see if there's more to see.

...and you could run the rest of the week with a slight bias towards staying above the Monday low. If it's gone up from Monday, each individual day has a fairly low chance of breaching Monday's low - even if the week is fairly likely to eventually. It gives you a sort of mini-trend to consider.

Don't look for a win, look for an edge.


----------



## ThingyMajiggy (15 October 2014)

Weatsop said:


> Don't look for a win, look for an edge.




How did you find yours? 

I wonder how much of an edge the likes of volume is, I know tech trades off volume a lot but I'm not sure if he's ever statistically tested it somehow. 

Like I wonder if I kept watching and trading the markets, replaying it back at 10x speed and giving it everything, if I would eventually just naturally have an edge, or more "feel" for it and would be profitable, or if trades that are made need to be based on hard evidence(like stats etc.).


----------



## Weatsop (15 October 2014)

ThingyMajiggy said:


> How did you find yours?
> 
> I wonder how much of an edge the likes of volume is, I know tech trades off volume a lot but I'm not sure if he's ever statistically tested it somehow.
> 
> Like I wonder if I kept watching and trading the markets, replaying it back at 10x speed and giving it everything, if I would eventually just naturally have an edge, or more "feel" for it and would be profitable, or if trades that are made need to be based on hard evidence(like stats etc.).




I'm not going to hijack TH's thread with my guff! I am literally the worst trader you'll ever talk to. That is, I have traded profitably, and expect to again, but right now I only have paper pips to boast about, and I'm really frighteningly ignorant about anything but my own little corner.

...but regarding voodoo indicators (like the numerology ones) - I like scalping down in the one-minutes because you don't need to put them on your chart. If the price does pile up against a line, you'll SEE it. Who gives a **** why it did, or if some old dood predicted it a thousand years ago? THERE IT IS. Piling up against a line. Why do you need an old dood to show it to you?

Tell me you can't see the line right across the top there:




(Not the dotted line, that's just the daily open - which, yeah, might be the reason it's boinging off - but who cares why?)

Edit: And what did it do?




Easy four pips. Someone better definitely could have got more.

I'll enter short a few pips down past the old top, or wait for a channel above that line and bracket it. Easy.

Edit 2: ok, well that was too perfect:




See how it slouched down to the line, then when it went through, it went hard. It's tried to get up past that line a few times and failed, now it's gone through but turned, so where is it going now?

Easy.

Screw indicators. Look at the price.


Way I see it, the price ignores the "technical" lines most of the time. But whether they do or not, you'll see it on the chart when it happens.


----------



## Weatsop (15 October 2014)

*Sorry, TH. Said I wouldn't gunk up the thread, then did. Shutting up.*

*(Also, for the record, I didn't actually enter the trades, since I was typing over here and I'm still dicking about with brokers. In reality I might have just been too chicken the whole move. Don't believe people on the internet!)


----------



## fiftyeight (16 October 2014)

Enjoying the posts Thingy.

I thought people did this kind of testing with amibroker? Is there a reason why you and TH use excel?


----------



## ThingyMajiggy (16 October 2014)

fiftyeight said:


> Enjoying the posts Thingy.
> 
> I thought people did this kind of testing with amibroker? Is there a reason why you and TH use excel?




Thanks, would be good to have more discussion on edges and finding them, as I don't really know poo from clay, but I'm trying. 

As far as why Excel, it opens everything, is easy to use, plenty of help out there and most platforms export to it with ease. Plus Amibroker annoys me, I sometimes use Python to programmatically find stuff. I know TH has found a lot of useful stuff in Excel, so that was another reason I just went with Excel, it works and it's easy...ish  Pretty handy program to be sufficient in.


----------



## Trembling Hand (16 October 2014)

fiftyeight said:


> Enjoying the posts Thingy.
> 
> I thought people did this kind of testing with amibroker? Is there a reason why you and TH use excel?




Its very easy to use to find simple stats. You can bang together a spreadsheet in 5 minutes that only has a few simple formulas so is robust and not likely to get you false data. Actually I also use MS Access a fair bit when I go to intraday bars as I find it easy to put together queries that are time based rather than a huge spreadsheet than gets boggy.


----------



## fiftyeight (17 October 2014)

Good stuff, might as well learn on something I have access to rather than spending more $$$

I have a few simple ideas that I want to play with, so when I get from holidays ill see if I can finally contribute something to ASF instead of all take take take


----------



## Trembling Hand (20 October 2014)

Everyone should read this,

http://traderfeed.blogspot.com.au/2014/10/the-real-source-of-trading-success.html



> Of the traders I worked with a decade ago, fewer than 5% are currently trading and experiencing success.  In each case, they are doing something very different from what the standard trading books describe.  They have found sources of "edge" in markets that they have made their own, and they have been consistent in exploiting those edges.


----------



## rb250660 (21 October 2014)

Trembling Hand said:


> Everyone should read this




Great little article. Most people are going to struggle with achieving the concept.


----------



## John Swift (21 October 2014)

Trembling Hand said:


> Everyone should read this,
> 
> http://traderfeed.blogspot.com.au/2014/10/the-real-source-of-trading-success.html




Thanks for the article. Very interesting. How would you personally charaacterise or describe the edge you have in the market TH?


----------



## Trembling Hand (21 October 2014)

John Swift said:


> Thanks for the article. Very interesting. How would you personally charaacterise or describe the edge you have in the market TH?




Probably this,


> doing something very different from what the standard trading books describe.


----------



## dlineinvestor (21 October 2014)

"Everyone should read this,
http://traderfeed.blogspot.com.au/20...g-success.html"  TY TH


"Rb250660
Great little article. Most people are going to struggle with achieving the concept"
"You think ?"

This is the best part:
One trader, for example, came up with an ingenious method for identifying when trading in a particular asset was becoming highly crowded.  He then looked for indications of loss of price momentum and took the other side of the crowded trade, benefiting from the herd running for the doors.  A big part of his edge was that, if he didn't find the right patterns of crowding, he did not trade.  He only played the game when the odds were on his side.  

So ..... for any retail trader, they are doomed .... unless they find / develop that edge ! 
The question is: "How are they going to find and develop this edge when they do not have the ability themselves?" They don't have the ability because they come from another world the "trading book world. A world where they do not have financial trading experience.

The real and only hurdle; they need to find the right person or group or anything outside what they are currently doing to get that edge. 

Trading especially futures is a deeply emotional and solitude _trade_ ... a complex chess game with high stakes.
That's why any of the top 5% that have made wont just share with you. Can you blame them, he has paid dearly for it. Years of stress, failures, doubt, self sacrifice, thousands lost. Yet he still managed to persevere and inch toward his goal of becoming consistent. I know OF such a person, he is "good he is "consistent, but he won't give me many clues and he won't give me straight answers to my questions.

The push continues.


----------



## tech/a (21 October 2014)

> I know OF such a person, he is "good he is "consistent, but he won't give me many clues and he won't give me straight answers to my questions.




Could be T/H and it could be a few others here.

Truth is there are no straight answers to the same question raised time and time again.
That's why answers appear cryptic.
When you find your own setups and see your own faces in the crowd you cant show someone how you trade them to a consistent profit, purely because each time may have subtle differences. 
The experience of seeing something building from past experience cant be just passed on.

But T/H has said many times that its screen time.
AND That's what it is.


----------



## dlineinvestor (21 October 2014)

Hey Tech .. 
Not talking about anyone on forums, someone I've known over 4 years purely through trading.
Cheers,


----------



## tech/a (21 October 2014)

dlineinvestor said:


> Hey Tech ..
> Not talking about anyone on forums, someone I've known over 4 years purely through trading.
> Cheers,




Nor am I.
I saw it as a generic "Quote" so gave a generic reply.

This bit anyway.


> Could be T/H and it could be a few others here


----------



## fiftyeight (3 March 2015)

Any ideas on where I can the get data to do this stuff? Are you getting it off of TWS or NT? I have tried googleing but I just keep coming up with indicators and back testing a system


----------



## tech/a (3 March 2015)

fiftyeight said:


> Any ideas on where I can the get data to do this stuff? Are you getting it off of TWS or NT? I have tried googleing but I just keep coming up with indicators and back testing a system




You can download from TWS
But will take awhile to get a library.
Never done it.

I bought tick data history from tickdata.com
You'll then need Amibroker or similar to run it through.
Well that's what we do.
It was cheap really thought it would be a lot more.


----------



## fiftyeight (4 March 2015)

I was hoping to avoid AB. 

All I am wanting to do initially is some small investigations like;

What is the distribution of say the next 5 bars after volume has exceeded 2 s.t.d

No idea if 2 s.t.d or 5 bars or volume will be significant, this is what I want to investigate. But with no intention for a future indicator or system just better understanding of how things work


----------



## ThingyMajiggy (4 March 2015)

Best way is via NT, it has its own data export feature but I prefer to do what this guy does, can add whatever columns you like then and it just goes off your chart data. 

Do as he does.


----------

