Tools being used by your trading competition

[DeepState]

The ultimate idea to is to find an edge that works well enough to make money on average through time...or, at least, be entertaining enough to keep your interest.

To this end, "Quant", with this broad meaning encapsulating:
+ data sciences which allow the gathering and analysis of vast amounts of information;
+ statistics which allows an investor to utilise another means to test their thinking;
+ quantitative finance which allows for increasingly rigorous and sophisticated concepts to be applied to pricing, portfolio optimisation and trading (for example); and
+ technology to harness information for investment/trading, organise this information rapidly and acccurately (as coded) and place ideas into the market quickly and efficiently...

can bring tools which are helpful to some investors. So it is with the power of speech and the ability to write, for example. It is possibly helpful but can be misapplied or abused accidently or purposefully.

Not all who can read do so with equal value. Not all who can speak have much with is worth saying. Not all who can codify a random forest can see the wood for the trees. Running a neural net over a single time series for prediction of a share price may not very smart. Still, I'd rather be able to speak and write than not.

Anyone can have an opinion. Very few opinions in investment withstand even the slightest quant analysis. Quant analysis helps with finding the truth. It will sometimes fail in this endeavour or give unreasonable confidence in its assertions.

"Individual results may vary."

I feel reasonably confident that, on the whole, speech and reading have added to human capacity. So it is with quant.

 

[tech/a]

Yes understanding quantitative analysis is a useful tool to have at your disposal.
Yes computers are becoming more and more advanced.
Yes it can help with the $$

It is a mammoth task to learn...

I agree if your like myself. I don't think I need a PHD to know what I need to know. I can use the skills of a quant to answer questions and guide me in what I want/need to know and I can be guided to understand what I need to know---I don't have to know it ALL---that's exciting.

Steady steady, hedge my bets

Money/trade management part of the equation.

I just disagree with the current established view that something has to be quantified by numbers.
That only by quantifying it with advanced qunatitiave analysis is it then acceptable.
That being a quant is the only way and that without numbers you will be a mammoth

That's fine you can certainly hold that view, but every time I hop on a plane, drive over a bridge, take a
prescription drug, listen to my Linn Sondec speakers, or drive my fPace, Im glad quants were involved.
Personally if I'm placing large wads for $$s in the market then I (Personally again) want every opportunity available to me involved and explored.

The people who caused the GFC quantified insurance and housing loans....

Then the models failed and they asked the people for a bailout...

Grab a hold of the book
"The Crisis of Crowding."
The quants were warning the market way before the Crap hit the fan.
Even Buffett the champion of "every man' very nearly became a catastrophic statistic having made a bid for LTCM (Long Term Capital Management) in 1998---fortunately the bid failed.

I am pretty sure a rational person would have seen the loans were unreasonable if they just went and interviewed a couple of borrowers.

Human nature caused the crash and PHD's in suits legitimised it with numbers.

There was a sad lack of rational people leading up to the crash. Loans given with no collateral to people with no jobs like free chips at a casino. Human nature played a part in the crash, over leveraging of worthless paper caused the catastrophic domino effect which if you think is never capable of happening again---think again---its still being in wound!

But herd all followed the quants numbers and crashed the world economy...

People trusted the numbers and quant analysis

I am just being the devils advocate to prove a point again!

Your not correct in this sweeping statement. Far more involved than what your alluding to.
Quants were and are on both sides of the equation. Read the book if you can get it.

Still waiting for actual evidence where price action is different. Proof. The truth is price action has not changed and the only thing "unprecedented" is the hype.

Price action changes daily and weekly and monthly.
Today more than ever we have to tools to help us regardless of timeframe.
Would you rather dig a hole for a swimming pool with a shovel or an excavator.
Both will do the job!

Have you looked at a chart yourself or simply going on what Cap Cook is saying ? Chart of the S&P..most would not agree with you that long periods of trending are now not happening..quite the opposite actually the trends seem better than 01-08 do you not think ?

View attachment 69459

Another sweeping statement, there are just as many charts supporting alternate views.
The trick of course is being able to place yourself with some confidence in front of a possible trend before it happens. Howards point I think is that he can do that far more often over a 3 day period than longer. You maybe able to do way better.

So statistics and optimization ? They've been around long before modern computing powers.

So were fingers and toes and the abacas. Have advancements in computers made life easier or harder?

And to point (1) Does it/they work. - if you need a computer to answer that you really shouldn't be trading.

Complacency and arrogance have bought down many in business. Those who fail to grow with technology can be eventually taken over by it. Sure there are those who will survive. Youll survive---but could you be better? Those that do tend to be in niche areas. Nothing wrong with that. But if I want to have multiple ways of profiting from the same pool would I rather have the power of computing behind me -- or if I wanted to see how I could improve what I'm doing would I welcome some quant in put?
For me yes---for others maybe you--- no.

Amibroker ? Been around since 1995. I'm sure other alternatives too. Yes now data churning are faster but so is pulling up charts for a discretionary trader/filtering criteria on balance sheets for a fundamentalist. Things have gotten more efficient - they have NOT EVOLVED to the point where having the latest computing power and packages gives you an edge for the retail trader.

Its very good on a retail level but on a quant level it is very restrictive. You cant do with it what you can do with python. (I'm not a quant but I'm working with (Paying). them and that's what they all say).

Howard deems them outdated and over in his first post. I commend you that you love the challenge and the mental stimulation. Keeps your mind sharp from the mental workout. Personally bottom line I'm here to make a living and if there was a unchallenging way to do it I would.

WysiView attachment 69447

I too--- work for myself. I'm for ever on the look out to find ways of being way way better than my competitors. One of the biggest ways I've found is being surrounded by people they don't have.
If I'm not an expert in a field I need to be (If I'm making a living) Then Ill find the best I can.
If I can have some equipment they haven't got------ then if it puts me even further out of their reach as a competitor Ill either buy it or better still lease it!.

I search out challenge. Challenge is the engine room. Without it you'll learn very little of worth.

Just saw Deep states Reply which crossed mine.
Very well said. Your writing a speaking are beyond my own skill set.

 

[minwa]

Another sweeping statement, there are just as many charts supporting alternate views.

Please post some, I'd really like to see how you concluded that markets have changed as a result of quants now on major markets that most people trade. Howard made that conclusion based on the system that he trades. Does he system cover every system ? I doubt it..Also his fear that a meltdown is coming to reduce time frame held. That's an opinion that hasn't happened yet.

 

[tech/a]

Please post some, I'd really like to see how you concluded that markets have changed as a result of quants now on major markets that most people trade. Howard made that conclusion based on the system that he trades. Does he system cover every system ? I doubt it..Also his fear that a meltdown is coming to reduce time frame held. That's an opinion that hasn't happened yet.

I'll grab some when I get a little more time.
But I don't know where you think the markets have changed because of quants.
My only point is that in my view what they have to offer should be included in
Market research.

 

[minwa]

Clearly this may differ but longer timeframes are becoming harder to trade and "Predict ' Going forward than they were in years past.
Long periods of trending EG 2001-2008 are now not happening.

OK I was under the impression you think that quants have changed the market, if not quants then what do you propose is changing the longer timeframes and that they are harder to predict than before ?

 

[tech/a]

OK I was under the impression you think that quants have changed the market, if not quants then what do you propose is changing the longer timeframes and that they are harder to predict than before ?

Economic world events.
Eg the end of the China boom
GFC not only in the US but also through Europe.
This has had massive impact on growth.

Just a couple of examples.
I'm sure you and others can think of many more Macro economic
Events which have caused un certainty

 

[minwa]

Economic world events.
Eg the end of the China boom
GFC not only in the US but also through Europe.
This has had massive impact on growth.

Just a couple of examples.
I'm sure you and others can think of many more Macro economic
Events which have caused un certainty

OK, that's the reason you think that market has changed it's "trendiness". I just fail to see that they have actually changed looking at the 2 major markets stocks and bonds.

 

[tech/a]

If you've been trading them and as a trend follower using longer holding times
I can understand your point of view.

 

[Trendnomics]

Wow this thread is terrible!

 

[OmegaTrader]

Wow this thread is terrible!


Funny pic
lol
But was that sarcasm???

 

[craft]

Edge is identification of risk which on a probabilistic basis is mispriced In conjunction with a system to exploit the mispricing.

Using data to sift out the possible existence of an edge and know it by no more than a posative expectancy calculation on historical data is fraught with danger. Most are mirages conjured by people snooping around in deserts of data destined to disappear as soon as you walk towards them.

Those oasis that may actually be real and stationary enough to be found can never sustain the throngs trying to survive off them for long. So why play such a stupid game unless you are indeed the fastest draw Quant in the game?

If you can't beat 'em, join them would have to be the dumbest advice in trading that I have ever heard.yet that is what the gurus on this thread seem to be saying. You have to get Quant to have a chance. I say rubish. I reckon they are just trying to baffle with bull**** either to hide the fact they are still stuck in the beginners cycle looking for new tech to give them the elusive answer to what eludes them or they are trying/preparing to sell something, even though of course they don't need to because this new thing is the holy grail and money can be made at will if only they desired to do so.

When I see people who I suspect are trying to fudge it with complexity totally misconstrue Buffett as part of their bedazzlement they wander into territory Where I know they have no idea what they are talking about and confirm my suspicions.

If you want a true Buffett direction in relation to this thread try this quote.
"I don't look to jump over 7-foot bars: I look around for 1-foot bars that I can step over."

The 7 foot bar is finding your edge by reverse engineering data. It's not robust and the guy with the most money can buy the most toys and the most toys wins – at least until nobody wants to play with you any more. Even if you win you lose and you have to go find another group who's mummies have unwittingly made you lunch.

The 1 foot hurdles is understanding you edge qualitatively and putting in the elbow grease to effectively and safely exploit it on a probabilistic basis. Maybe you might even find that in an arena of physical economic business production – maybe a business that makes lunches, mmmmmm lunches forever without the hassle of who you going to take the next one off.

Data analyses can be a tool to gain knowledge and judgement. However plenty who can analyse data efficiently will never see the forest for the trees. Whilst the better you qualatativelly understand the cause of mispriced risk you seek to exploit the less reliant on historical data you will be.

 

[Trendnomics]

^^^^^^^^^^

 

[tech/a]

There will always be diversity in opinion and view.
That's the key to innovation.

Exciting!

 

[barney]

Data analyses can be a tool to gain knowledge and judgement. However plenty who can analyse data efficiently will never see the forest for the trees. Whilst the better you qualatativelly understand the cause of mispriced risk you seek to exploit the less reliant on historical data you will be.

Great post Craft.

Been an interesting thread ... my 2 bobs worth.

The failure of "Long Term Capital Management" in the late 1990's is a good example of quantitative analysis going off the rails and why perhaps this new breed of super Quants who are supposedly going to take all us little trading guys to the cleaners might be just a bit exaggerated.

LTCM had some of the biggest swinging Q's of the era and killed the market for 3 years before blowing $4.5 Billion (plus change) in less than 6 months!!

I think tech/a used the analogy of digging a hole with an excavator or a shovel which is probably a totally reasonable analogy if you are a big honcho in the trading world ..... no doubt the excavator will do thousands of times more work in the same period, however the cost of either buying the excavator or paying the dude who knows how to drive it is still relatively priced compared to using my Bunnings shovel, plus I'm a lot more likely to see that sewer pipe I'm about to crunch if I'm up close with my shovel

If you run a hedge fund and have your wages paid by multitudes of punters who don't want the hassle of trading for themselves I'm sure you need Quants to compete with the other companies who are doing the same thing. At this stage I just can't see a lot of relative value or disadvantage for us little guys.

 

[jjbinks]

I think important thing is to be open to using/trying the different tools.

If you are on this site you are probably reading/practising and learning to improve your trading. Statistics/programming are just another tool or area to learn about which will help you improve or even find an edge.

 

[minwa]

If you run a hedge fund and have your wages paid by multitudes of punters who don't want the hassle of trading for themselves I'm sure you need Quants to compete with the other companies who are doing the same thing. At this stage I just can't see a lot of relative value or disadvantage for us little guys.

Even with all the resources and brainpower at the hedge fund level, it's massive competition that has hit a brick wall years ago. http://www.businessinsider.com.au/top-goldman-quant-quant-trading-is-dead-2009-12

If Goldman is not confident I don't see how any of us tiny flies can be with our resources. Only select few have thrived like Renaissance - they have resources and technology we probably don't even have the vocabulary for.

 

[Gringotts Bank]

... they have resources and technology we probably don't even have the vocabulary for.

I'd be investigating how the evening news bulletins, social media and the major finance dailys can be manipulated to drive the overnight or next day's moves. Such things are already monitored, but afaik, no one has been caught profiting from the manipulation of sentiment. It's absolutely possible to do such things. The Russians tried to influence the US elections this way but as far as I can see their efforts were brute force, unsophisticated and ineffective.

Here's someone who knows how to do it...

 

[tech/a]

I think important thing is to be open to using/trying the different tools.

If you are on this site you are probably reading/practising and learning to improve your trading. Statistics/programming are just another tool or area to learn about which will help you improve or even find an edge.

Exactly
Another tool
Not Armageddon by Quant!!

Jjgets it.

 

[ThingyMajiggy]

Right, so let's see something in action, Howard, tech, anyone? I think it's clear where everyone stands so we might as well start having something useful in this thread. Show us some Quant tool, or ML predictions, don't have to show any code, just the output if you like.

 

[howardbandy]

Greetings --

This thread has more diversity in responses than I expected it would. If I may, let me refocus on what I believe are some of the critical issues we all face, and how we might prepare for them.

Some of the underlying assumptions and definitions:

1. Trading is buying and selling financial instruments with the intent of selling at a higher price than buying. Investing is buying and never selling. My comments apply to trading.

2. We are trading in order to increase our net worth.

3. Trading is a zero sum game, slightly biased in alignment with changes in GDP, workforce, and productivity.

4. Each trade has a buyer and seller. For that trade, the net is zero sum -- less commissions that reduce the profit on both sides.

5. Repeating a well known management tenet -- "It is not possible to manage a process without being able to measure that process."

6. Trading is competitive.

7. Traders stop trading when the drawdown of their trading account exceeds their personal risk tolerance.

Given those, and probably some others that list members will add, there are rational techniques that traders can use to help them be among the winners rather than the losers.

1. Every set of trades can be decomposed into a set of one-day holdings. When a trade is held multiple days, there are no transactions between days, but there is accounting and -- importantly -- opportunity for management each day.

2. Each equity curve is a single data point representing one ordering of the trades that make it up. The risk of drawdown we are exposed to when we trade is determined by the distribution of that set of trades. The very best we can hope for in terms of future performance is that the distribution does not change. We cannot expect the order of trades to be repeated.

3. Given a set of trades and a forecast horizon, we can create many equally likely trade sequences, measure the drawdown associated with each, and compute the distribution of drawdown.

4. Given the distribution of drawdown and the trader's personal risk tolerance, we can compute the maximum safe position size associated with that set of trades. I call it "safe-f" in the presentations I make. Money management is critically important in maximizing profit. Safe-f is the position size for those trades that maximizes account wealth while holding drawdown to the trader's tolerance.

5. After adjusting the position size, the final equity after a period of trading where trades come from that same distribution can be computed. Pick a point on the distribution as the metric by which alternatives will be measured. I recommend the 25th percentile, and call the metric "CAR25" -- the Compound Annual Rate of Return at the 25th percentile of the estimated profit from risk-normalized trades.

6. Every set of trades has its own CAR25. A rational trader will develop several trading systems, each with its own set of trades, and compute CAR25 for each of them. The one that should be traded is the one with the highest CAR25. It is the Dominant system. Given a specified level of risk, it is the best system -- the one that will provide the highest return.

Like it or not, everything I have stated about the distribution of trades and risk up to this point exists. It is not a theory. Wishing it were not so is as futile as wishing the value of pi was 3. Computations would be so much simpler than the unwieldy 3.14159...

Whatever technique you use, the system has a CAR25. Ignore that at risk to your trading account.

We can use our knowledge to give ourselves an advantage over at least some of our competition. We can manage our trading to keep ourselves maximally profitable while avoiding account-destroying drawdowns.

7. CAR25 computed for the set of trades was static. As the system is traded, new trades will be added to the distribution. Risk will change. CAR25 will change. If we ignore these changes we will either lose opportunity for profit or realize drawdown beyond out tolerance. A trading management system based on sequential learning and Bayesian analysis monitors trade-by-trade results and adjusts position size as necessary to keep risk normalized and maximize CAR25.

So ---- we need a technique for doing all of this. For developing a system, gathering the set of trades that are the best estimate of future performance, estimating risk, computing safe-f, estimating CAR25, and managing the trading of the result.

Our alternatives for trading system development are:

A. Subjective trade selection. In which I include all techniques that do not have explicit rules that signal trade entry and trade exit. A set of subjectively chosen trades can be analyzed using the techniques I describe. Trader's who are successful with subjective techniques should continue to use them and ignore me.

B. Objective trade selection. A set of rules coded into the language of a computer-based trading system development platform that specify all entries and all exits. I recommend using state signals which are evaluated every day, giving management opportunities that coincide with accounting measurement.

All of the above material is preliminary to the original posting in this thread. All of it is explained in much more detail in my books, including downloadable computer code, and in the videos I have posted to YouTube.

We have two broad techniques for objective trading system development:

B1. Traditional platforms such as AmiBroker, TradeStation, NinjaTrader, etc

B2. Machine learning.

(Trading system management, including computation of trade-by-trade position size, is done in a separate computer program and should be a separate discussion topic. See the brief note below.)

------------------------------

My original post suggested that some of the biggest and best of our trading counterparties -- our competition -- is using machine learning. And we might consider doing so ourselves if we want to be among the winners.

To briefly reiterate the differences between B1 and B2 --

B1. Traditional platforms build models that are primarily decision trees. These are sets of if-then rules that play "20 questions" with the data, following branches of logic and eventually arriving at a leaf of the tree with a label Buy or Sell. In terms of state signals, the state for the next day-to-day holding advice at the leaf would be beLong, beShort, or beFlat.

Decision trees have advantages of being easy to build, fast to build, tolerant of outliers, and easy to interpret.

They suffer from the disadvantages of using decision rules that are parallel to axes, being sensitive to branching, and too easily overfit.

Traditional platforms use the paradigm of "compute an indicator, then see what happens after."

B2. Machine learning expands model types. Decision trees are still available. Others include neural networks, support vector machines, genetic algorithms, ensembles, and dozens more.

Machine learning uses the paradigm of "identify a profitable trading opportunity, then see what happened earlier."

-----------------------------

Note that the trading management system is separate from trading system development. Since it uses only the sequence of recent trades, it is itself independent of the method used to issue the signals. It does require a computer program to carry out its calculations. The example of such a program I recommend is written in Python and published in my materials.

-----------------------------

I understand that some readers have different points of view. Some are on topic and appreciated. Others considerably off topic, some picking at nits that are not relevant to the main discussion, some denying facts, some wishful thinking.

The intention of my original post was, and with this post continues to be, merely to let the readers of Aussie Stock Forum know what tools are being used by some of the best traders and how our members might use similar tools to their own advantage.

Thanks for listening,
Howard