# Using investor sentiment in trading



## adamim1 (25 January 2012)

Hi,

IG markets just released a new tool on their trading platform called "Insights".

It has a few nice little tools, one of them being "client Sentiment". This shows you how many open longs and shorts their clients have on stocks/commodities/indices.

How do you use sentiment in trading? E.g. if 98% of clients are long the ASX200, should I be looking for a shorting opportunity, or maybe going with the flow? Or does it really depend on the other indicators as well as fundamentals?Obviously I would be looking at other indicators in addition to sentiment, but just want to know what I should be looking out for.

Thanks.


----------



## Gringotts Bank (25 January 2012)

Can you post a picture of their sentiment indicator?


----------



## adamim1 (25 January 2012)

Sure,

This is the EUR/USD sentiment in IG markets.


----------



## peter2 (25 January 2012)

This new sentiment indicator means nothing unless you can access and research it's past history. You might like to record the values at the same time everyday and keep a record yourself. If you find an edge then your results will be exclusively yours to profit by. There might be of worth to record the values just prior to each fx session (Asian, UK, US). 

Let me suggest an acronym for your notes, The COAT (Commitment of Amateur Traders) report.


----------



## Gringotts Bank (26 January 2012)

Could be very handy, but as peter says, it would be a good idea to have some stats to work with.

If you know how to screen scrape, set your PC up to take hourly or daily shots of the ratio and the price, then plot it up.  This is just a guess, but I would expect large swings taking the ratio from below to above 50% (and vice versa) would be worth looking at.  Also extreme one-sidedness in the ratio might signal turning points.

What would really be interesting to see is how much money is on each side of the book.  IG won't give you that info in a hurry!


----------



## sinner (26 January 2012)

Gringotts Bank said:


> Could be very handy, but as peter says, it would be a good idea to have some stats to work with.
> 
> If you know how to screen scrape, set your PC up to take hourly or daily shots of the ratio and the price, then plot it up.  This is just a guess, but I would expect large swings taking the ratio from below to above 50% (and vice versa) would be worth looking at.  Also extreme one-sidedness in the ratio might signal turning points.
> 
> What would really be interesting to see is how much money is on each side of the book.  IG won't give you that info in a hurry!




I use 

http://www.dukascopy.com/swiss/english/marketwatch/sentiment/

Also a good example is Guy Lerner

http://www.safehaven.com/author/93/guy-lerner

(look at the articles titled Investor Sentiment)


----------



## adamim1 (27 January 2012)

Gringotts Bank said:


> Could be very handy, but as peter says, it would be a good idea to have some stats to work with.
> 
> If you know how to screen scrape, set your PC up to take hourly or daily shots of the ratio and the price, then plot it up.  This is just a guess, but I would expect large swings taking the ratio from below to above 50% (and vice versa) would be worth looking at.  Also extreme one-sidedness in the ratio might signal turning points.
> 
> What would really be interesting to see is how much money is on each side of the book.  IG won't give you that info in a hurry!




Nope, its only based upon accounts I believe.

so 80% of accounts could be short silver and 20% are going long. But theres 25 million dollars in the long position and only 1 million going short.

So i guess it can be misleading.

I'll get some stats over time and post them up. I'll do it upon the different markets opening up.


----------



## Gringotts Bank (11 June 2012)

Extraordinary claims:

from Reuters.com
"Much of the excitement around Twitter trading stems from a paper by academics Johan Bollen and Huina Mao of Indiana University, and Xiao-Jun Zeng of the University of Manchester. The report found that gauging the investing public's mood can be a startlingly predictive mechanism for the stock market. "We find an accuracy of 87.6 percent in predicting the daily up and down changes in the closing values of the Dow Jones industrial average," the authors wrote."

87% ?????  If this is true, it's a license to print money... as much as you want.

http://www.reuters.com/article/2012/02/16/us-twitter-stockpredictions-idUSTRE81F21I20120216
http://venturebeat.com/2011/03/17/study-social-media-popularity-can-predict-stock-prices/


----------



## Gringotts Bank (11 June 2012)

A potential problem would be if it becomes too successful, someone will decide they can open 100 million new Twitter and Facebook accounts laced with 1000's of emotional keywords and direct traffic.  Wouldn't be hard.


----------



## skc (11 June 2012)

Gringotts Bank said:


> Extraordinary claims:
> 
> from Reuters.com
> "Much of the excitement around Twitter trading stems from a paper by academics Johan Bollen and Huina Mao of Indiana University, and Xiao-Jun Zeng of the University of Manchester. The report found that gauging the investing public's mood can be a startlingly predictive mechanism for the stock market. "We find an accuracy of 87.6 percent in predicting the daily up and down changes in the closing values of the Dow Jones industrial average," the authors wrote."
> ...




High volatility = high mean reversion. 87% correct prediction is great. However I'd like to see the actual expectancy.


----------



## Gringotts Bank (11 June 2012)

I don't get what you're saying here.  The average up day would equal the average down day in terms of size of movement, so 87% accuracy would mean you couldn't lose, like some sort of money tree.

This is the abstract:
Behavioral economics tells us that emotions can profoundly affect individual behavior and decision-making. Does this also apply to societies at large, i.e., can societies experience mood states that affect their collective decision making? By extension is the public mood correlated or even predictive of economic indicators? Here we investigate whether measurements of collective mood states derived from large-scale Twitter feeds are correlated to the value of the Dow Jones Industrial Average (DJIA) over time. We analyze the text content of daily Twitter feeds by two mood tracking tools, namely OpinionFinder that measures positive vs. negative mood and Google-Profile of Mood States (GPOMS) that measures mood in terms of 6 dimensions (Calm, Alert, Sure, Vital, Kind, and Happy). We cross-validate the resulting mood time series by comparing their ability to detect the public's response to the presidential election and Thanksgiving day in 2008. A Granger causality analysis and a Self-Organizing Fuzzy Neural Network are then used to investigate the hypothesis that public mood states, as measured by the OpinionFinder and GPOMS mood time series, are predictive of changes in DJIA closing values. Our results indicate that the accuracy of DJIA predictions can be significantly improved by the inclusion of specific public mood dimensions but not others. We find an accuracy of 87.6% in predicting the daily up and down changes in the closing values of the DJIA and a reduction of the Mean Average Percentage Error by more than 6%.


----------



## skc (11 June 2012)

Gringotts Bank said:


> I don't get what you're saying here.  The average up day would equal the average down day in terms of size of movement, so 87% accuracy would mean you couldn't lose, like some sort of money tree.




Well you know the expectancy formula... E = Win% x avg win - Loss% x avg loss.

They are not trading every day - they are only trading when sentiments are extreme - like using overbought / oversold type indicators. So the avg win and avg loss will not equal movements of average up / down day.

And like those overbought/oversold indicators, they can stay "over" for sustained periods of time. If they have no stop and catch a big move in the wrong direction, the avg loss can be large. 

So without knowing the expectancy it's hard to call it a money tree.

I have no idea what the "mean average percentage error" means...


----------



## Gringotts Bank (11 June 2012)

skc said:


> They are not trading every day.




Right, didn't think of that.  Still a nice hit rate.


----------



## sinner (12 June 2012)

skc said:


> Well you know the expectancy formula... E = Win% x avg win - Loss% x avg loss.
> 
> They are not trading every day - they are only trading when sentiments are extreme - like using overbought / oversold type indicators. So the avg win and avg loss will not equal movements of average up / down day.
> 
> ...




Are you sure they aren't trading every day? I'd say from the abstract GB posted, they are.

EDIT: My personal experience is sentiment is a very very useful tool, wouldn't use a NN to interpret it though, nor would it be the 'core' of any trading decision.


----------



## skc (12 June 2012)

sinner said:


> Are you sure they aren't trading every day? I'd say from the abstract GB posted, they are.
> 
> EDIT: My personal experience is sentiment is a very very useful tool, wouldn't use a NN to interpret it though, nor would it be the 'core' of any trading decision.




No I am not sure... haven't read the whole paper. It just seems to make sense for predictions to be more accurate at points of extreme sentiments.

I guess the my point was really thinking out loud how to translate a high win % into a high/decent expectancy... and I don't know if that's something those academics have had a good think about.


----------



## sinner (12 June 2012)

skc said:


> No I am not sure... haven't read the whole paper. It just seems to make sense for predictions to be more accurate at points of extreme sentiments.




Yerp sure, I agree, but that doesn't mean they aren't predicting every day. Just that of that set of predictions, those made at sentiment extremes are more likely to be accurate.



> I guess the my point was really thinking out loud how to translate a high win % into a high/decent expectancy... and I don't know if that's something those academics have had a good think about.




Yep. This is the pre-eminent problem with 'machine learning/classification' combined with trading. Obviously it's present in all classification problems however the 'impact' on equity curve from false negatives and false positives needs to be evaluated (this is pretty much what you're stating) carefully. 

Most classifiers supply a 'regression' mode so you don't have to just predict +1 (up day) or -1 (down day) but actually the magnitude of the prediction also (e.g. 2.35% up from yesterdays close). Depending on how you define and then trade a given classification, this can help or hinder. Probably the most 'correct' technique would be % exposure style, so if sentiment regression is 10% net long for tomorrow then you take a FullPositionSize*0.1 position long in the underlying. Transaction costs can be a bitch in this regard, so running the model only when the market is likely to be predictable makes a lot of sense.

Lots of academics writing papers on 'predicting' time-series using SVM or similar tool actually do take it into account, unfortunately they rarely choose to provide in-depth results, rather as an addendum in their papers, e.g. "this thing has a 90% prediction rate tested 8 ways to Sunday and oh yeah if you traded it the Sharpe would be 0.9" sort of thing.


----------



## Gringotts Bank (12 June 2012)

http://www.marketpsychdata.com/tools/dashboards

worth a look


----------

