# An Experiment: can crowds predict market movements?



## BrianEastlake (5 February 2015)

Hi,

I'm Brian Eastlake, new to the forums.

I was talking with a friend of mine a few weeks ago about trading. He doesn’t think that future price action can be assigned a probability based upon past price action. ie, he doesn’t believe in technical analysis whatsoever, and is somewhat suspicious of fundamental analysis too. Basically, he thinks price action is random.

I disagreed. I think both fundamental and technical analysis have their place - fundamental forces drive the higher timeframes, and I can clearly see technical patterns in price action, I’m just not always right about which pattern is dominating at any particular point in time. 

My friend - Will - and I are both also interested in psychology, and we’ve previously discussed the phenomenon of how crowd aggregated predictions tend to be better than the predictions made by individuals (even experts).

You may have heard about the jellybeans-in-a-jar contest, where you have to guess the number of beans and the person with the closest guess wins the jar. If so, you may also have heard that the average of all the guesses is *remarkably* accurate, and consistently so.

Will pointed out that if price action wasn’t random, and future price action could be assigned a probability based upon past price action and/or fundamental analysis, that crowds should be able to consistently predict future price action better than most individuals.

So, we created a site to find out, and we’re now looking for participants in our experiment.

To be clear: we're not providing investment advice or selling a service, we're just trying to settle this question.

Participants in the experiment will make weekly predictions on EURUSD movements, which we will then average and compare to the actual market. If it works, we may get an idea of whether the market is likely to go up or down during the following week, as well as the likely size of the move. Initially the experiment will run for eight weeks, and we will of course be sharing the weekly predictions with whoever participates.

If you want to join us, all the details can be found on the website: https://www.marketcrowdwise.com/details. Or just ask me any questions you have here and I’ll do my best to answer them.

Cheers
-Brian


----------



## tech/a (5 February 2015)

> can crowds predict market movements?




No but you can anticipate what crowds will do.
Anticipate NOT predict.


----------



## Wysiwyg (5 February 2015)

BrianEastlake said:


> Will pointed out that if price action wasn’t random, and future price action could be assigned a probability based upon past price action and/or fundamental analysis, that crowds should be able to consistently predict future price action better than most individuals.



Let's face it. Everyone has a different agenda. The dude/s creating resistance at one level are content with their profits right there. There are others who believe (belief via perception or inner circle) that price will or should rise higher and once a majority (of funds) is formed, this belief becomes reality.


----------



## rimtas (5 February 2015)

Crowd can not predict anything, because it is herding.
 But some individuals from that same herd can predict where the herd is going to move. For this he needs just the knowledge of crowd behaviour. 
Individually we are all different and have an intelect, an ability to think logical. In a crowd we are all emotionaly driven sheeps with no brains what so ever. The same like herd of cows, flamingos and birds. If you think cows can predict anything, think twice.

Jump out of the herd, put the knowledge of herd behaviour at work and you will have the best tool available for forecasting markets. 
Looking at historical price action is the key of determining where the crowd is located. Knowing how it tends to herd and the current location you can forecast a future crowd behaviour quite accurately.
 Markets are not random, markets are not rational, markets are not logical. Forecasts using those perceptions are always failing.


----------



## DeepState (5 February 2015)

This experiment has essentially been done a lot of times.  Any finding from your experiment would need to be pooled with the others to make useful inference and will, thus, converge to their sample outcomes.  Take a look at retail flows, earnings estimate levels and economist forecasts for key variables.  There is basically no predictive ability in an aggregate sense for directional expectation.  This study has been done and done.  The sell side uses surveys of buy-side expectations as a contrarian signal.  The buy-side does the same for the sell-side.  Each thinks the other is an idiot. 

Crowds are generally better predictors than any given, randomly selected, individual for things where the subject of prediction is not impacted by the guesses/estimates.  Where is the submarine at the bottom of the ocean?  What is the weight of the cow?  How many jelly beans in the container?  None of these is impacted by the estimates given by individuals or the crowd itself.  Individual, unbiased, estimate errors wash out due to the presence of some independence.  The bogey doesn't move.  If it is capable of estimation, the crowd outdoes the great bulk of individuals reliably.

When it comes to markets, the prices are determined by the crowd.  What the crowd thinks moves prices.  In order for the crowd to predict market movements, it has to know where it is heading.  If it knew where it was heading, it would just go there in the first place without taking a detour...the logic basically collapses.  What you end up with is fair compensation for risk over time as the crowd determines it to be (this could actually be zero, or even negative).  This is the gross outcome.  There are micro-structure things which shave off this bulk explanation at the edges, but that is not really about crowd-based prediction of an endogenous system like the broad financial markets.  You will find exceptions in some special, niche, cases.  A crowd of inside information traders will outdo the broader crowd of investors, for example.

This is very different to saying that you cannot ascribe any probability to any outcome in financial markets.  You absolutely can.  Trees do not go to infinity and markets cannot go below total loss.  Distributions are hardly so unstable that nothing can be estimated even slightly.  The market is not entirely chaotic and fully uncertain.  If it were, it would eventually collapse on itself.  Investment requires an ability to estimate in a valuable way or otherwise an IPO is simply a request for money and that's it....how long do you think this would last?  Without a primary market, there is no secondary market.

Within the crowd, the action is different.  In here, your interest in psychology can pay dividends (and capital gains).  Or maybe not.  The crowd will determine that too.


----------



## BrianEastlake (6 February 2015)

First of all, thank you so much for your insightful comments! Much appreciated. 



DeepState said:


> This experiment has essentially been done a lot of times.  Any finding from your experiment would need to be pooled with the others to make useful inference and will, thus, converge to their sample outcomes.  Take a look at retail flows, earnings estimate levels and economist forecasts for key variables.  There is basically no predictive ability in an aggregate sense for directional expectation.  This study has been done and done.




I perhaps should have come here and asked the question first, maybe could have saved myself some effort. ;-)

Still, though, I did do some research beforehand to try to find out whether this type of experiment had been done before, and didn't find anything that qualified as similar enough to satisfy me.

One of the closest I could find was this:
http://stocktwits.com/research/Viability-of-StockTwits-and-Google-Trends-Loughlin_Harnisch.pdf

Their conclusion was effectively: "StockTwits data was significant in predicting Apple, Google, and Microsoft stock returns". But it was for stocks (which I think are more prone to bubbles and irrational price movements than forex), it's being used as an indicator, and there was no attempt to get a "smart" crowd, as defined by James Surowiecki.

I went looking just now for details about retail flows, earnings estimate levels and economist forecasts for key variables, but am struggling to find something that would suggest this type of experiment has already been done. If you have any specific experiments or research data that you've looked into, can you share them with me? I'd be very interested to read about them (and I know Will is keen too).



DeepState said:


> When it comes to markets, the prices are determined by the crowd.  What the crowd thinks moves prices.  In order for the crowd to predict market movements, it has to know where it is heading.




If I understand you correctly, you're saying that a weekly bar can't be predicted in advance because the shape of the bar unfolds based upon how it starts to unfold. ie, so the data from Monday affects what happens on Tuesday, and so on, right?

I totally agree. But I also think it might be possible to identify probable outcomes (or at least probabilities for the possible outcomes), for the next bar - particularly on the more stable higher timeframes - and by setting the timeframe parameters of the experiment to weekly, that's effectively what we're asking people to try to do: identify those probabilities.



DeepState said:


> This is very different to saying that you cannot ascribe any probability to any outcome in financial markets.  You absolutely can.




Right. I believe so too. What Will is trying to argue, is that if you can ascribe a probability to an outcome, then it is that probability that a smart crowd should be able to identify/uncover. So that although the aggregated prediction of the crowd wouldn't necessarily be correct, it would - over enough samples - be more accurate than most individuals.

Let's say, based upon technical and fundamental analysis, you think the market has a 70% probability of going up and a 30% probability of going down, over the course of the following week. And imagine that your trading strategy was to enter the market at the start of the week, and exit at the end. For the sake of argument, we'll imagine there are no brokerage fees. If your assessment of the probabilities is correct, and you went long at the start of the week, you'd expect a 7 out of 10 chance to have made money. If we create the right environment for a "smart crowd", and if those probabilities are indeed there, I don't see why it would be unreasonable to think that the crowd could uncover those probabilities?

Cheers
-Brian


----------



## BrianEastlake (6 February 2015)

Wysiwyg said:


> Let's face it. Everyone has a different agenda. The dude/s creating resistance at one level are content with their profits right there. There are others who believe (belief via perception or inner circle) that price will or should rise higher and once a majority (of funds) is formed, this belief becomes reality.




Yep, absolutely. Zooming out a bit, you could think of price action as the aggregated prediction of market participants.

So if the aggregated prediction of all market participants is equal to the price action, how closely would the aggregated prediction of _some_ market participants be? (Via a "smart" crowd of course... as distinct from a regular crowd. The two are most definitely not the same thing.)


----------



## BrianEastlake (6 February 2015)

rimtas said:


> Crowd can not predict anything, because it is herding.
> But some individuals from that same herd can predict where the herd is going to move. For this he needs just the knowledge of crowd behaviour.
> Individually we are all different and have an intelect, an ability to think logical. In a crowd we are all emotionaly driven sheeps with no brains what so ever. The same like herd of cows, flamingos and birds. If you think cows can predict anything, think twice.




That is definitely the most customary view on crowd behaviour. And I think that's exactly what would happen if we ran the experiment without satisfying the criteria for creating a smart (or "wise") crowd.

You can't just cobble together any group of people and expect to get good/quality/accurate answers. There are a specific set of things that you need to do in order to get a smart crowd. 

James Surowiecki identified 4 of them, and we think we'll be able to create a smart crowd based upon those criteria. If you're interested, all the information is on the details page of the site.

There's also some information about it in this Wikipedia entry about James' book:
http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds

Of course, the book itself is a fascinating read, and well worth the time.

Cheers
-Brian


----------



## DeepState (7 February 2015)

BrianEastlake said:


> If you have any specific experiments or research data




Kinnel M, 2014, "Mind the Gap", Morningstar information piece.  Shows retail punters mistime markets badly. This is a crowd survey of the retail market weighted by dollars.

Goedhart M et al, 2010, “Equity Analysts: Still too bullish”, McKinsey information piece. Shows large bias is aggregate analyst estimates for the US (and, I can tell you, other markets).  Observe the start and end levels and through time levels of the estimates and you will find that analysts forecast history quite accurately.  This is a broad survey of the sell-side equity analyst universe.  You already have stats on the aggregate performance of buy-side vs index, I assume. Check out Bogle, if not.

Agar P et al, 2009, “The accuracy and efficiency of the Consensus Forecasts: A further
application and extension of the pooled approach”, Journal of International Forecasting.  Analysed forecast performance from Consensus Economics which is the most comprehensive survey of economists globally.  Key take-out “information efficiency has to be rejected in most cases”.  That was for the decade leading up to the GFC…. It hasn’t exactly become better since then.

Basically, there is virtually no value in large sample survey forecasts of levels of economic variables, aggregate earnings or market levels for meaningful timeframes like one or two years out.  



BrianEastlake said:


> If I understand you correctly, you're saying that a weekly bar can't be predicted in advance because the shape of the bar unfolds based upon how it starts to unfold. ie, so the data from Monday affects what happens on Tuesday, and so on, right?




For micro-structure reasons, certain types of price patterns can exist.  This is one of those “at the edges” issues that is less to do with forecasts in aggregate, than trying to implement them.  When the scalpers on this forum look for levels and such over 10-minute frames, no concept of a one-year outcome generally features.  You can extend the logic.  After a while, long term reality of the type which anchors security prices in aggregate asserts itself more than liquidity.



BrianEastlake said:


> I also think it might be possible to identify probable outcomes (or at least probabilities for the possible outcomes), for the next bar - particularly on the more stable higher timeframes - and by setting the timeframe parameters of the experiment to weekly, that's effectively what we're asking people to try to do: identify those probabilities.




By estimating probabilities, you will get some notion of the distribution of outcomes and, very importantly, the expected return figure.  

What you want is for the crowd estimates on the expected return to actually be useful and to vary materially around a long term risk premium (straight buy and hold) in a way that you can actually get set and still make (more) money (than buy and hold).  

The distribution around the mean is quite stable. Creating a valuable conditional mean estimate is the juice.  Otherwise you are ‘only’ risk managing or, worse, outright punting.

The forecasts of average returns for just about anything in a macro level aggregate sense for economic variables and other endogenous dependents is not meaningfully distinguished from a metaphorical stopped clock.  The shape of the distribution around whatever mean happens to come out in future is quite stable in most cases.  The present ‘crowd’ estimate of this is visible in the options market.



BrianEastlake said:


> What Will is trying to argue, is that if you can ascribe a probability to an outcome, then it is that probability that a smart crowd should be able to identify/uncover. So that although the aggregated prediction of the crowd wouldn't necessarily be correct, it would - over enough samples - be more accurate than most individuals.




Crowd based forecasts would have less error in an individual forecast than for most individuals.  That is a given and there is absolutely no need to test it unless examining fairly extreme/absurd scenarios.  However, what matters for money making is that it can actually forecast in a useful way directionally.  A bigger crowd can forecast with less error and still be utterly useless.  A collection of 100 coin flippers determining EURUSD movement next week around the carry profit is not exactly going to produce a better outcome than a subset of 5.  However, it will produce a more stable average estimate…progressively tighter towards zero as crowd size increases. This estimate will vary proportionately less than the outcome of any single coin flipper. The crowd is more accurate than the individual, yet both are useless at forecasting. The findings indicate that the crowd forecasts for aggregate economic variables aren’t too different from coin flippers as a whole.



BrianEastlake said:


> Let's say, based upon technical and fundamental analysis, you think the market has a 70% probability of going up and a 30% probability of going down, over the course of the following week. And imagine that your trading strategy was to enter the market at the start of the week, and exit at the end. For the sake of argument, we'll imagine there are no brokerage fees. If your assessment of the probabilities is correct, and you went long at the start of the week, you'd expect a 7 out of 10 chance to have made money. If we create the right environment for a "smart crowd", and if those probabilities are indeed there, I don't see why it would be unreasonable to think that the crowd could uncover those probabilities?




So now what you are arguing is that it is not so much the wisdom of crowds that you are looking for to forecast the market, but obtaining/creating access to the wisdom of a subset that knows where the crowd is going to go.   The ‘smart’ analysts. If you do create that environment you are seeking and can actually elicit true thoughts from those in the circle of foresight, you have just become an investor/trader with an edge.  

In order to capitalize that edge, the market/rest-of-crowd has to come to realise that what the high priest thought in the first place was right. So, yes, this only works if your insight is not so profound that it escapes the market’s attention forever.  It requires that they cotton onto your idea eventually.  

In aggregate, the crowd follows the smart money.  It does so because the smart money knows where the crowd is going and the crowd wants to be smart.  In that sense, the crowd is forecasting with some accuracy after the insight is revealed to them. However, each new insight identified by the smart analysts earlier is really a surprise to that part of the crowd which is not in the inner sanctum.  That inner sanctum is actually small at the outset for any decent idea.  The larger it gets, the more crowd-like it becomes.  You really can’t front run yourself and expect to make money.  At any time, the crowd as a whole, taken as a straw poll, pretty much doesn’t actually know where it is going next.  You can mount all sorts of extreme outlier examples to dispute this, but it is broadly correct on average, through time.

As soon as this source of extraordinary wisdom generation is gleaned by others and knowledge spreads, those probabilities become hard to distinguish from chance as the edge gets priced out.  All that is happening in there is wealth transfers within the crowd for a period that the edge existed. Because this process is so tumultuous, the crowd actually has a hard time figuring out who the hot investor of the minute is and doesn’t tend to react that much in an aggregate market sense. 

If you can find a subset that really knows what they are doing and what the rest will do, great.  Go with that.  This can never be a large portion of the crowd and will change all the time.  For the rest of the crowd, there is really no need to know what they think at all if you are looking for forecasts relevant to investing beyond buy and hold.  It’s already pretty much in the price give or take some random figure.  What they think can be useful to those in the know because the great ones can then see where the crowd is wrong and judge whether they will ever come to their senses in this lifetime.  The rest of the crowd has no idea where it is wrong and will only find out later….or really doesn’t care that much.

Your survey of crowds with or without the smart money will look virtually identical most of the time.  That’s if you even knew who the smart money are at any time and could segment your crowd appropriately.  Overall, there is no forecasting power in a useful sense.  If there were, the market is somehow materially and voluntarily informationally inefficient in aggregate or has structures like the really smart people are numerous and yet have virtually no money to invest.  Neither seems a sensible proposition on which to rest a thesis for money management.

I think you might be interested in Lo A, 2004, “The Adaptive Markets Hypothesis: Market Efficiency from an Evolutionary Perspective”.

In general, this wisdom of crowds concept is not applicable to financial markets because the crowd members are not independent of each other or the variable they are trying to forecast.  This violates critical foundations espoused by Surowiecki.  This concept is not applicable to endogenous systems like financial markets despite often being quoted in the same context.  If you are interested in how largely endogeonous systems (like capital markets and the broader political economy) work, then the Sante Fe Institute for Complexity Systems is a good place to start.  They drop that critical assumption which invalidates the use of Surowiecki’s thesis in financial markets.


----------



## qldfrog (8 February 2015)

eur/usd is quite specialised for an aussie site and so your survey will not use a random population, but a few highly focused specialists (if you find them)
If worth carrrying such an experiment, i would use asx or all ord estimate for end of year
The forum has a thread for this and you could then compare the results to the forecasts  with the merit of available history for the past few years


----------



## BrianEastlake (8 February 2015)

qldfrog said:


> eur/usd is quite specialised for an aussie site and so your survey will not use a random population, but a few highly focused specialists (if you find them)
> If worth carrrying such an experiment, i would use asx or all ord estimate for end of year
> The forum has a thread for this and you could then compare the results to the forecasts  with the merit of available history for the past few years




That's a very good point. Our intention was to cast a broad net, so that we'd have people with a range of trading experience - from the absolute beginner to the advanced - and with a broad geographic distribution. We were actually quite keen to include people that were not actively participating in the EURUSD market. That is, we weren't trying to find those few highly focussed specialists. They are of course welcome, but we would not expect that they make up the majority of the crowd.

I am Australian myself, which is of course why I thought of extending the invitation here. 

But I hadn't realised that my message didn't come across clearly. (I'll have to work on that...) So let me clear it up right now: you don't need to be a EURUSD participant to join!  In fact, in some ways it'd be preferable if you weren't, because then you'd be less likely to be influenced by the plethora of EURUSD discussions that are taking place on forums / twitter / blogs / facebook / etc.

We selected the EURUSD market quite deliberately. The problem with using the All Ords is that it only has value in one direction, and is more prone to bubbles and irrational investor behaviour than forex. ie, with the EURUSD, if you're buying EUR then you're selling USD, and if you're buying USD then you're selling EUR. There are two enormous economies behind either side of the transaction. Now, admittedly, most traders don't use it that way, but the market wasn't created for those traders. The EURUSD market is really the interface for international trade between the European and US markets, and is by far the most actively traded forex market.


----------



## BrianEastlake (8 February 2015)

DeepState said:


> Kinnel M, 2014, "Mind the Gap", Morningstar information piece.  Shows retail punters mistime markets badly. This is a crowd survey of the retail market weighted by dollars.
> 
> Goedhart M et al, 2010, “Equity Analysts: Still too bullish”, McKinsey information piece. Shows large bias is aggregate analyst estimates for the US (and, I can tell you, other markets).  Observe the start and end levels and through time levels of the estimates and you will find that analysts forecast history quite accurately.  This is a broad survey of the sell-side equity analyst universe.  You already have stats on the aggregate performance of buy-side vs index, I assume. Check out Bogle, if not.
> 
> ...




Awesome, thanks so much for taking the time to compile those references. It'll take me (and Will) a while to properly digest the information.

The only one I was already familiar with is Kinnel's "Mind the Gap" ... Bogle (yep, I'm also familiar with his work) eloquently argues the same thing, that investors routinely mistime the market. But bear in mind that the price action in the market is the aggregated prediction of the market participants. From that perspective: the crowd is always right. If you took a crowd, toward one of the peaks, and aggregate the crowd's prediction for the market, I wonder whether it would also have gone with the retail investors and done the "wrong" thing? Or if it would have pointed toward what the larger crowd (aka, "the market") was about to do..?



DeepState said:


> Basically, there is virtually no value in large sample survey forecasts of levels of economic variables, aggregate earnings or market levels for meaningful timeframes like one or two years out.




Ok, that may well be the case. However, the experimental parameters we have set up do not fall into those categories, and I'm still left with the sense that the experiment, as we're trying to conduct it, hasn't been done before.

Similar experiments have been conducted, for sure. But, similar enough to be absolutely sure it won't work even without trying it? I'm not that absolutely sure of anything. I'd rather conduct the experiment and see what happens. After all, one of the best tools for science and discovery is keeping an open mind. ;-)



DeepState said:


> For micro-structure reasons, certain types of price patterns can exist.  This is one of those “at the edges” issues that is less to do with forecasts in aggregate, than trying to implement them.  When the scalpers on this forum look for levels and such over 10-minute frames, no concept of a one-year outcome generally features.  You can extend the logic.  After a while, long term reality of the type which anchors security prices in aggregate asserts itself more than liquidity.




Right, and this is one of the reasons we chose to go with weekly. It isn't so zoomed-in that it will be impacted by the kinds of price movements that scalpers are interested in. (Though, how the scalpers compete with the HFT's without the benefit of seeing the order flow is beyond me.)

Weekly data doesn't have the same "artefacts" that minute / 5-minute / 10-minute timeframes have. Yet it isn't so far in the future (monthly or yearly) that the movements are driven purely by fundamental data - I believe there are still technical patterns discernable in weekly data and know that some forex traders are using technical analysis on weekly data to make trading decisions.

Daily could also be suitable, but there are big challenges in implementing the experiment on daily data in forex because the market only closes over the weekend.



DeepState said:


> What you want is for the crowd estimates on the expected return to actually be useful and to vary materially around a long term risk premium (straight buy and hold) in a way that you can actually get set and still make (more) money (than buy and hold).
> 
> The distribution around the mean is quite stable. Creating a valuable conditional mean estimate is the juice.  Otherwise you are ‘only’ risk managing or, worse, outright punting.




I think you're referring to indexing here, but there isn't quite the same concept in forex markets.



DeepState said:


> Crowd based forecasts would have less error in an individual forecast than for most individuals.  That is a given and there is absolutely no need to test it unless examining fairly extreme/absurd scenarios. However, what matters for money making is that it can actually forecast in a useful way directionally.  A bigger crowd can forecast with less error and still be utterly useless.  A collection of 100 coin flippers determining EURUSD movement next week around the carry profit is not exactly going to produce a better outcome than a subset of 5.  However, it will produce a more stable average estimate…progressively tighter towards zero as crowd size increases. This estimate will vary proportionately less than the outcome of any single coin flipper. The crowd is more accurate than the individual, yet both are useless at forecasting. The findings indicate that the crowd forecasts for aggregate economic variables aren’t too different from coin flippers as a whole.




Hmm... flipping coins isn't the same as having a smart crowd predict price movements. If a smart crowd can uncover the probabilities of the possible future price action (and, if you don't believe those probabilities exist, then you aren't trading, you're gambling), and your smart crowd has a 60% chance of being correct, then you have a clearly tradeable edge.

The crowd doesn't need to be able to forecast economic variables to do it. (At least, not on a weekly timescale.)



DeepState said:


> So now what you are arguing is that it is not so much the wisdom of crowds that you are looking for to forecast the market, but obtaining/creating access to the wisdom of a subset that knows where the crowd is going to go.   The ‘smart’ analysts. If you do create that environment you are seeking and can actually elicit true thoughts from those in the circle of foresight, you have just become an investor/trader with an edge.




No. That's definitely not what we want. We want to know whether a smart crowd (so, filled with a range of people, not necessarily smart individuals or experts) can predict market movements - essentially by uncovering the probabilities for future price movement, not necessarily by being correct all the time.

My favourite example of this comes from 1906, at a country fair in Plymouth, England, there was a contest held for people to guess the weight of an ox. There were 787 valid entries to the competition, and as you might expect there was a wide range of estimates. After the competition, a statistician - Sir Francis Galton - tabulated the results and noticed something truly remarkable: the average estimate was astonishingly accurate. At 1207 pounds, it was only 0.8% away from the correct value of 1198 pounds. More accurate, in fact, than the estimates made by most cattle experts at the fair.

This phenomenon isn't restricted to guessing the weight of oxen, or even the more commonly used example of jellybeans in a jar. It seems that, given the right conditions, if you ask enough people the same question then collectively they tend to produce a better answer than the experts.

I realise you're saying that while it may be true for estimating things like jellybeans, it doesn't hold true for predicting market movements or any economic variables.

I haven't seen clear evidence to suggest it's impossible, and Will insists it would have to be possible, if it is at all possible to ascribe probabilities to future price movements.

To be clear: I'm not saying it is possible, and I'm not saying it isn't possible. I'm saying I don't know, and that it's an interesting experiment to conduct.



DeepState said:


> In general, this wisdom of crowds concept is not applicable to financial markets because the crowd members are not independent of each other or the variable they are trying to forecast.




This is a really awesome point, and it's given us food for thought. We'd already listed it as one of the risks for the experiment on the details page of the site, having realised that a weakness of the experimental model is that we are reliant upon the participants not discussing their predictions with others. But, I'm wondering how effective that will be, given that EURUSD market participants will invariably be influenced by the thoughts and opinions of others around them on the internet. I'm starting to wonder whether we should restrict the experiment to only those traders who *don't* trade and/or follow the EURUSD market.

In its current form, you sound like you think the experiment has no merit. Would you find the experiment more interesting if the crowd was made up only of people who didn't trade EURUSD?

Cheers
-Brian


----------



## Wysiwyg (8 February 2015)

I think the experiment would need to define "crowd". For example if the crowd is all market participants then the range of crowd behaviour would stretch from the most risk averse (sydboy007 for example ) to the least risk averse (darkhorse70 for example ). Within that crowd range, factions that have the similar risk profile will join the market movement at different stages.

So what defines the "crowd" without being bias toward one part of a crowd?


----------



## Wysiwyg (8 February 2015)

Whatever the crowd is defined as, there is increasing influence from 'bot trades in the market place. Does more of  the crowd need to go automated? Trade wise, I would like to know when this company had peaks in trading volumes.




> CHICAGO, ILLINOIS — (Marketwired) — 02/26/14 – Deep Value, developer of high performance trading algorithms, reached a new milestone executing 481,401,795 shares or *5.26 per cent of total US-wide stock market trading volume* on September 20, 2013.


----------



## Wysiwyg (8 February 2015)

Another thing to consider is the type of traders that are attracted to certain stocks. Take Fortescue Metals for example. Well established down trend but it attracts near the largest number of trades and volume on the ASX.  Telstra is in a well established up trend and it also attracts large volume and number of trades. This would suggest the FMG crowd can't predict price movement.


----------



## DeepState (8 February 2015)

Hi Brian



BrianEastlake said:


> From that perspective: the crowd is always right.



Which is why the market price represents what the crowd thinks.  If the crowd is always right, you will not get a better estimate of what they think than the price.  It is your survey outcome, weighted by money, with a full population sample size – the ultimate unbiased sample.



BrianEastlake said:


> I think you're referring to indexing here, but there isn't quite the same concept in forex markets.



I was referring to markets in general, which includes FX.



BrianEastlake said:


> Hmm... flipping coins isn't the same as having a smart crowd predict price movements. If a smart crowd can uncover the probabilities of the possible future price action (and, if you don't believe those probabilities exist, then you aren't trading, you're gambling), and your smart crowd has a 60% chance of being correct, then you have a clearly tradeable edge.



Then it comes down to finding that smart crowd.  That smart crowd is a subset of the crowd.  It is not the crowd.

What matters is accurate probabilities.  Of those, to make money, you need to have an accurate sense of the central tendency.  Anyone can forecast.  Anyone can assign probabilities to alien life on Pluto.  What you want is accuracy sufficient to make money above and beyond buy and hold.  A crowd as a whole will not do so reliably/valuably for you in terms of return expectation. You can see the probability distribution in the options market. It is always right in the sense that it has no real idea what it is going to do next, or would already have done it, for the most part.

Probabilities exist.  However, that doesn’t mean you know what they are.  It’s even harder when your efforts to discover them impact the outcome.  If you like, check out the Heisenberg Principle.  Once you find out where the particle is…it’s gone. Same deal.



BrianEastlake said:


> I realise you're saying that while it may be true for estimating things like jellybeans, it doesn't hold true for predicting market movements or any economic variables.
> 
> I haven't seen clear evidence to suggest it's impossible, and Will insists it would have to be possible, if it is at all possible to ascribe probabilities to future price movements.
> 
> To be clear: I'm not saying it is possible, and I'm not saying it isn't possible. I'm saying I don't know, and that it's an interesting experiment to conduct.



Finding proof of impossibility is somewhat esoteric. You will never get it.  I am absolutely sure of that given you cannot actually be sure of the absence of something.  Absence of proof, as they say, is not proof of absence. So even if you find nothing, we will still be left with a possibility that whatever you were looking for still exists.  

I look forward to learning of how Will progresses his flow of logic.  Strictly speaking, he is correct.  It is possible.  That’s because it actually isn’t impossible for an occasional survey outcome to be correct.  Sometimes, the next tick in the EURUSD is unchanged.  Correct prediction from the market.  It’s when it changes that you get problems.  Change is what we are interested in.

Surowiecki has plainly stated that his stuff doesn’t work in these situations.  I think he means that it is not reliable. That’s what I mean. Even throwing darts gets the right answer from time to time.  You can calculate probabilities from a dart board quite reliably and map them onto the options market distribution where probabilities are explicitly assigned. That doesn’t make it a worthy signal for making money.



BrianEastlake said:


> In its current form, you sound like you think the experiment has no merit. Would you find the experiment more interesting if the crowd was made up only of people who didn't trade EURUSD?



1.	That implies you are trying to get answers from people who aren’t really motivated to find out what the answer should be.
2.	Even if they were motivated but were not direct participants, they will be affected by visible variables and interact on that basis.  This is how most information contagion takes place.  It is still endogenous.  It violates Surowiecki’s requirements.
3.	If you then want to isolate them in a separate room with no allowable contact and deprive them of information that might influence them in aggregate, the useful stuff, and pay them for accurate outcomes (hence: motivation, diversity, independence, a variable capable of estimation, decentralization)….you will still get probability estimates.  And you know what to do with those.

When you survey the market for an opinion, you pretty much get the price.  In illiquid markets, you can get delays.  If you regard the EURUSD as illiquid, it’s probably a parameter you should reconsider.

By all means pursue your passion on this.  I am just one survey outcome.  Further, I might be foxing...or, maybe, retail.  

All the best with it.


----------



## BrianEastlake (9 February 2015)

Wysiwyg said:


> I think the experiment would need to define "crowd". For example if the crowd is all market participants then the range of crowd behaviour would stretch from the most risk averse (sydboy007 for example ) to the least risk averse (darkhorse70 for example ). Within that crowd range, factions that have the similar risk profile will join the market movement at different stages.
> 
> So what defines the "crowd" without being bias toward one part of a crowd?




In the case of the experiment, the crowd is made up of whoever is willing to participate and make weekly predictions.

The more diverse opinions we can get, the better for the experiment. But, it hopefully won't matter what the risk tolerance of the participants is, because someone can think that the market will go up without being sure enough about it to be willing to risk capital by going long. ;-)

It is important however that we cast as wide a net as possible. If, for example, we only had participants from a single country, then we run a greater risk of not gathering a smart crowd.

We ask participants to tell us which country/timezoe they are in and their level of trading experience. It's optional, but if enough people fill it in that should help us to identify whether or not we have a broad range of traders.

Cheers
-Brian


----------



## BrianEastlake (9 February 2015)

DeepState said:


> Probabilities exist.  However, that doesn’t mean you know what they are.  It’s even harder when your efforts to discover them impact the outcome.  If you like, check out the Heisenberg Principle.  Once you find out where the particle is…it’s gone. Same deal.




And yet, there is an entire industry built around the idea that not only is price action non-random, it's predictable enough that you can apply analytical tools to it and determine those probabilities with enough certainty to make money.

Every trader that believes in technical analysis must also believe it is possible to predict future price movements.

If those probabilities exist, and if individuals are able to identify them, then I don't see why a (smart) crowd couldn't identify them with even greater accuracy.



DeepState said:


> I look forward to learning of how Will progresses his flow of logic.  Strictly speaking, he is correct.  It is possible.  That’s because it actually isn’t impossible for an occasional survey outcome to be correct.  Sometimes, the next tick in the EURUSD is unchanged.  Correct prediction from the market.  It’s when it changes that you get problems.  Change is what we are interested in.




I think if the crowd-sourced predictions are wildly wrong, Will is going to take that to mean price action is effectively random - ie, that despite what technical analysis may lead traders to believe, the patterns we're looking at are all in our minds and/or applied after-the-fact. That's what he currently thinks, so it would reinforce his opinion.

He does concede however that if they're wildly wrong it could also indicate a problem with the crowd itself, that it wasn't sufficiently diversified or was influencing itself (not via the price - because we don't believe that is an issue when using a fixed weekly timeframe and entering predictions when the market is closed - but via forums/twitter/etc). We have some ideas on how to try to identify whether those scenarios have occurred.



DeepState said:


> 1.	That implies you are trying to get answers from people who aren’t really motivated to find out what the answer should be.




That presupposes nobody is interested in the experiment itself, only in what they could potentially gain from it. Money is a powerful motivator, but so is curiosity, and there would have to be a certain level of motivation for the participants to join in the first place.



DeepState said:


> 2.	Even if they were motivated but were not direct participants, they will be affected by visible variables and interact on that basis.  This is how most information contagion takes place.  It is still endogenous.  It violates Surowiecki’s requirements.




Yes, we realise that's a risk. In conducting this experiment, we have to rely upon the integrity of the participants not to discuss their prediction.



DeepState said:


> 3.	If you then want to isolate them in a separate room with no allowable contact and deprive them of information that might influence them in aggregate, the useful stuff, and pay them for accurate outcomes (hence: motivation, diversity, independence, a variable capable of estimation, decentralization)….you will still get probability estimates.  And you know what to do with those.




But, probability estimates are exactly what we want, and I don't think we have to go to quite those extremes to get them. 

If we get a statistically significant result, Will is going to concede that price action isn't random, that trading isn't simply gambling, and that traders aren't deluding themselves about market analysis. If it further turns out that the probabilities identified by the crowd have a tradeable edge, well, that's icing on the cake.



DeepState said:


> All the best with it.




Thanks. I'll be sure to update this thread with the results when we start getting some data.

Cheers
-Brian


----------



## tech/a (9 February 2015)

Many have been before you.

http://works.bepress.com/bruce_vanstone/subject_areas.html


----------



## tech/a (9 February 2015)

This is excellent

http://riverpublishers.com/journal/journal_articles/RP_Journal_2245-456X_211.pdf


----------



## BrianEastlake (10 February 2015)

tech/a said:


> Many have been before you.
> 
> http://works.bepress.com/bruce_vanstone/subject_areas.html




Wow, that's a real treasure-trove of papers! Thanks for the link. 
But, I don't really see any there that relate to crowd-sourcing predictions?


----------



## BrianEastlake (10 February 2015)

tech/a said:


> This is excellent
> 
> http://riverpublishers.com/journal/journal_articles/RP_Journal_2245-456X_211.pdf




Thanks for the tip. It looks interesting, I'll probably have to dedicate a weekend to studying it properly. I'll also pass it on to Will - he'll have a better shot at understanding the maths in the paper.

It seems that these researchers have arrived at the conclusion that future price action can be predicted (in certain scenarios at least).

Cheers
-Brian


----------



## DeepState (10 February 2015)

Hi Brian

My last entry on this thread. 

Key Points:
+ In something like financial markets, the participants as a whole cannot sustainably, meaningfully, outforecast their own subsequent actions if there exists motivation to profit from this forecast. In capital markets, that motivation is apparent.
+ It follows that a representative sample of this crowd will not do so either.  Creating independence of the type that Surowiecki requires in an endogenous system sees it collapse on itself as a forecasting method. It requires a crowd of forecasters to stand completely apart from the crowd being forecast. That’s not practically feasible when they look at the same things and think/act in similar ways if the info is useful and there is logic to its use.  They also live in the same political economy. If these points do not matter, there is no forecasting power anyway.
+ A crowd in which some are truly insightful and are better than others on average at forecasting is not any better at forecasting its overall actions for the presence of a few with insight.  It does mean that those with insight will profit more than those without.  Some degree of heterogeneity in ability is required for the healthy function of capital markets.  It is true that, if you can identity those with insight, a survey of a crowd of those talented few will likely produce a more stable outcome than any particular individual within it chosen at random. The critical issue is to survey those only.  Not the whole crowd or a representative sample of it.
+ There is motivation to game the preference provided by participants which you will not be able to reliably uncover.  You will not be able to rely on the kindness of strangers, even if they are just curious.
+ The only type of crowd-based survey that matters for prediction of a type that aims to make money directly is one which surveys the current thoughts of those with real insight before it is known to the rest of the crowd.  An historical example is given.
+ Will is wrong.  He should pay-up now or leave the market…in which case he should pay-up now because that action demonstrates why he is wrong so long as a single person in the market cares about making money. I feel comfortable with that assumption. He may argue otherwise.  If he protests, who cares, his thoughts about the market are random and can be disregarded as noise allowing others to find advantage for his absence of effort…..Hi Will.
+ An experimental statistical outline is suggested.  The behavior of the EURUSD is such that, for short term forecasts like that suggested, a reasonable bogey is simply a forecast that next week’s exchange rate is today’s.  If you get significant results, you’ll need to explain how a smart crowd managed to outsmart itself and intends to keep doing so when it is motivated to ensure that this does not occur.  This will require the kind of arithmetic gymnastics that would be needed in explaining how a table of poker players managed to outguess itself as a whole.

------



BrianEastlake said:


> And yet, there is an entire industry built around the idea that not only is price action non-random, it's predictable enough that you can apply analytical tools to it and determine those probabilities with enough certainty to make money.



The great bulk of this industry is about competition from within.  If the industry as a whole were satisfied with just getting buy-hold returns, most of it would not exist.  It exists because most people aren’t satisfied with that and want more.  Overall, they will fail after frictions and fees are considered.  On average, they will persist with that endeavor anyway. Welcome to human behavior.  If you are interested in psychology of investing, this aspect bears a lot of worthy examination.  

In reality, you need at least some of this behavior to keep capital markets functioning, but having a lot of it doesn’t mean that the whole is more predictive. If you need a reference on this, try Grossman-Stiglitz.

As a result, the only survey that makes sense for making super-normal returns is one where the truly insightful ones are surveyed and the result compared with the rest of the crowd (whose outcome is sufficiently expressed in the prevailing price so as to make no practical differerence for a broad market like EURUSD).  Surveying the crowd as a whole, which can be comprised of very smart people on average, but not all of whom can be above average, will yield nothing more than what the market is showing for the most part.



BrianEastlake said:


> If those probabilities exist, and if individuals are able to identify them, then I don't see why a (smart) crowd couldn't identify them with even greater accuracy.



The concept of regarding a crowd that learns off the smart ones as a smart predictor of its own behavior is erroneous.  A crowd where elements are capable of some prediction (as must exist for a sustainably functioning capital market) consists of those who are insightful and those who are not. Insight is a relative concept in this world. Even the smart ones are surprised by what they find when they find it. The others are surprised when they find out a little later.  On average, they are consistently being surprised.  This is what makes aggregate survey outcomes non-predictive, but why, within a crowd, superior prediction leading to excess money-making can exist for a subset. Whilst individuals within this subset make predictions which are accurate, it is true that a wider survey of them will yield more stable outcomes….but these other participants also have to be insightful.  The crowd as a whole cannot be more insightful than itself. That is why a survey of the whole population trying to predict itself is not a strong idea.

Except for pocket solutions for smaller markets than the type we are talking about, surveying the crowd will get nowhere sustainably.  You need to survey the insightful.  The rest, forget about it.  It’s already visible in the price.

-----


BrianEastlake said:


> He does concede however that if they're wildly wrong it could also indicate a problem with the crowd itself, that it wasn't sufficiently diversified or was influencing itself (not via the price - because we don't believe that is an issue when using a fixed weekly timeframe and entering predictions when the market is closed - but via forums/twitter/etc). We have some ideas on how to try to identify whether those scenarios have occurred.



If your crowd experiment doesn’t work, making it even more diverse to the point that you survey the entire populace of earth isn’t going to make it better.  What will improve it is narrowing the survey to an extremely unrepresentative sample of those who truly know better than the crowd overall.



BrianEastlake said:


> That presupposes nobody is interested in the experiment itself, only in what they could potentially gain from it. Money is a powerful motivator, but so is curiosity, and there would have to be a certain level of motivation for the participants to join in the first place.



This is an extract from a report which talks about the kind of survey which actually works in real life.  The idea was created by an Australian a long time ago although the news item refers to the US and a firm called BlackRock, currently the world’s largest money manager.  The idea was so good that, in the end, the umpires effectively changed the rules or, alternatively, chose to enforce the rules differently.  It is a survey of influential people in the crowd whose opinions are listened to, cared about, and thus influence the crowd.  Know what they are thinking, and you have a good notion of what the wider crowd will be thinking.

http://www.bloomberg.com/news/artic...agrees-with-n-y-to-end-analyst-survey-program

You will note that people who work extremely hard to become influential in the financial market and know a lot about it tend to work for money, even if they are curious about it.  They seek to monetize their views.  That said, central bankers and policy officials are not particularly well paid and, yet, are influential.  However, if they were to release information to your survey before it becomes public and are found to do so, they will be fired.  If you knew you were receiving information from them and used it, you would also be imprisoned if discovered.  If you were offering kickbacks to them for participation beyond curiosity, they would also be imprisoned. If I used the survey outcome knowing what you did, I would be imprisoned. I hope that’s not your underlying purpose here…

As for the rest of the possibilities: those who have no particular influence because their opinion is not known or cared for by others with money in the market ahead of the survey release; are not creating policy directly that influences outcomes; but yet are very insightful (and not malicious voters); are also keen to share those insights to an essentially private survey before otherwise releasing it elsewhere for curiosity; and for these outcomes not to be used in any way that influences the market and allows it to adapt….leads me to wonder about this element of your experimental design and how incentives for participation like platonic curiosity of the type you need can be expected to produce a sustainably useful outcome. It would certainly not be a representative sample of the crowd or market participants which is the underlying proposition about smart crowds and prediction in endogenous settings. 

Some reading on game theory might be called for.  I was only half-kidding (less, actually) when I talked about foxing in my prior sign-off.  



BrianEastlake said:


> Yes, we realise that's a risk. In conducting this experiment, we have to rely upon the integrity of the participants not to discuss their prediction.



They don’t have to discuss it.  All they need to do is read the news and look at economic variables and also know the price series.  It is through these issues that forecasters lose independence when the system is endogenous. This is not like being able to estimate the dimensions of an ox and having a sense of the mass per unit volume.  It is the influence of these common variables that help cause the madness of crowds.  I do not personally know the guy from Barclays Fixed Income Trading in the Square Mile, but I am reading some of the same stuff.  If this stuff is useful for prediction, there will be correlation amongst those with predictive ability.  This violates Surowiecki.  If it doesn’t the experiment fails before it starts anyway and any positive statistical outcome which might arise is just an outlier.



BrianEastlake said:


> But, probability estimates are exactly what we want, and I don't think we have to go to quite those extremes to get them.



In order to meet Surowiecki’s requirements within an endogenous system, you will essentially need to create an exogenous system within an otherwise endogenous one.  This is pretty much the only way to reconcile them. Not only can they not talk to each other or even refer to past survey results, but they can’t even look at the price or have any information that might coordinate their estimates. Once again, the wisdom of crowds concept does not apply to financial markets unless it is an insightful subset that you are managing to survey.  Integrity about not discussing their predictions with anyone doesn’t even matter except in fairly extreme situations like publishing their views as editor of the Financial Times.  Survey participants can be mute, unable to type a word (but can type their estimate for next week’s EURUSD level or otherwise communicate it), not visible to one another, deaf and pious.  If they use the same info, they will correlate unless that info is useless and/or they all don’t know anything about the info available and apply it randomly.

If you try to force this survey design to meet the Surowiecki requirments, this design is the natural end point. All participants will be independent, somehow motivated, but none insightful (at least as referenced by usual powers of deduction) beyond individual idiosyncratic data not available to any other. A crowd standing fully independently of another crowd whose actions they are trying to estimate. A group of curious people in separate, (figurative) sub-terrainian chambers, estimating what the latest fashion is on the surface of the planet. Trying hard to emulate a situation like counting jellybeans from outside the jar or weighing oxen. Except this is not like that. Is this design issue not evident?  

You will still get a probability distribution just for asking, as I would get when asking my dice via rolling them.  It just won’t bear much resemblance to anything that might be useful beyond curiosity.




BrianEastlake said:


> If we get a statistically significant result, Will is going to concede that price action isn't random, that trading isn't simply gambling, and that traders aren't deluding themselves about market analysis. If it further turns out that the probabilities identified by the crowd have a tradeable edge, well, that's icing on the cake.



For a market to exist, price action cannot be fully random all the time. Grossman-Stiglitz explains this if not immediately evident. Price action exists in a strange and unstable space of near-randomness. Will can concede anytime he wants and pay up in any manner that you have pre-agreed. If he doesn’t, it doesn’t matter because he can sit out in that belief (for to participate is worse than gambling) and really ceases to have any relevance to the market. By doing so, he reduces its randomness as if fait accompli.  If it were otherwise, there is no incentive for price discovery and there would be no capital market.  A certain order exists in the markets which would not if it were fully uncertain. It’s not even vaguely close to that. 

Nonetheless, one scenario is that Will demands evidence that it is not fully random. He can claim that a P-value of 0.00001 still left room for doubt or find some wrinkle as to why the survey was flawed and ask for another one with some new feature. You can counter with a request to provide evidence that it is random and show evidence which rejects complete uncertainty to high confidence but is not absolute. If you are both motivated from curiosity, please start a thread and attempt to claim this grey space between the two viewpoints!  You will be inundated….with those saying it is not random (because ASF is a biased sample by headcount…something for you to consider in your experimental design from the outset).  A crowd survey by straw poll, drawing on their collective wisdom, will slay Will’s hypothesis.  That’s unless, knowing we are running an experiment on crowds, they try to spite us and vote in favour of Will out of mirth and curiosity (for there is no material monetary incentive on this outcome)…which will it be? …

In any case, the EURUSD weekly return as measured by log adjusted returns over the last 15 years (pretty much to the birth of the Euro although pre-dating the minting of coins etc.) suggests that there is no short term momentum in there at the basic level although someone out there will torture the heck out of it with some algo with an acronym and publish a statistically significant result, disavowing the other hundred they tried.  Hence, for the most part (though not strictly speaking), the current level of the EURUSD now is pretty much the best estimate of where it will be in a week from now for the crowd as a whole given what it knows in aggregate (this is not true over longer periods).  That can serve as your null hypothesis.

To be useful for money making and to obtain the upside kind of curiosity, the sample crowd outcome will need to be statistically better than the bogey above and also statistically positive in a net of t-cost P&L sense.  That’s at least two hurdles. For implementation, experienced traders would consider a lot more before making something like this operational.  Still, let’s start there.

If you manage to obtain some result which surpasses an arbitrary hurdle like a 5% confidence interval (note P&L will almost certainly be non-normal and require a different stat test than t-stat or anything based on student’s t or normal distribution assumptions. You are moving into non-parametric stuff here), you will still need to overcome an explanation of why the crowd managed to outsmart itself and kept on doing it.  

Importantly, you will need to explain why a crowd which can outsmart itself intends to keep doing so and yet be serious enough to continue to regard itself as smart! Capital markets are driven by money and it does not seem like a good experimental assumption that those who reveal their preference out of curiosity, do so without malice, do not otherwise participate in the market, are independent in data, are so independent that they do not contact each other or share meaningful common links, yet are as smart or more predictive than the insightful ones with who have incentives which go beyond curiosity. That is a large set of compounding assumptions. It does not make it impossible.  It falls just a little bit short of that. 

A smart crowd in this setting learns until it cannot beat itself. Anything else is not smart or consistent with the presence of insight and the motivation to actually use it. A table of poker players does not beat itself even if all the players are just curious, play for toothpicks and possibly bragging rights, and don’t talk to each other. Neither can a group of curious people who cannot interact with the table or each other in any way.  There is a whole world of HFT/Algo/Fundamental/Tech/Astrology/SÃ©ance working hard every day to make it hard for it to happen sustainably. Some of these efforts are outlined in Tech/A’s attachments.  His do not specifically relate to crowd-sourced data.  You might, however, be interested in this piece of fairly full-bore data-mining which is massively crowd sourced and goes beyond the use of simple, period ahead estimates, as single dot points.  It is an effort at crowd-sourced artificial intelligence.  'Supervised learning':




In contrast to surveying a crowd, a survey of those with real insight can and, routinely, does violate the Surowiecki requirements and yet reveal their superior wisdom in prediction. However, that is very different to the proposition that Surowiecki is talking about when he refers to wisdom of crowds as it relates to oxen.  It is also a very different proposition to what is being presented here.  What you want in this situation is the wisdom of the wisest in the crowd. Don’t bother with surveying the crowd as a whole, or a representative sub-sample consisting of the curious but otherwise uninterested/uninvolved.  The hardest part is finding the most insightful/predictive part of the seething mass and motivating them in whatever way needed to give you their views ahead of general distribution.  The next hardest is to use it without becoming incarcerated.  



BrianEastlake said:


> Thanks. I'll be sure to update this thread with the results when we start getting some data.



I’ll sign off here and look forward to how this rolls out if you choose to pursue it and report it.  Before discovery, curiosity usually featured somewhere.  Enjoy your journey.


----------

