Resources for Systematic/Algorithmic Trading

Chipp · 10 August 2024

Right up my alley , I really like the Bandy books but have read many systematic style books from most authors . Not a fan of all of them and a couple on Richards list are part of that segment , i will leave that right there though . Better system trader a very good resource for podcasts with a great range of styles and experience . As a quant type trader i am right into what i call market telemetry and collect/measure things that matter to me . I virtually use no generic indicators outside ATR and W%R .

I hope this thread goes in a good direction but its a world where thats just a dream in the main . I miss the days of stockcentral a good 20 years ago , run by Austin Hui ? from memory , some very talented and knowledgable people came out of there . I sort of doubt this is the place where we will see much traction tbh . Nevertheless i will hope for the best . I read a thread from in here with Mr Howard Bandy and he got run out of dodge by a few rusted on bully ' locals ' . The guys who really know what they speak about will have to break a few myths and spit some truth to a few that just aint going to accept it .

A book i highly reccomend is

Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals

Evidence , the cornerstone of any systematic approach , seems to be lacking a bit for some . Interested to see the discussions that may develop in here . Will drop in here when i see things of interest when i have the time . #1 i am a trader so thats a time consuming pursuit .

Richard Dale · 10 August 2024

@Chipp I have lot of respect for David Aronson's book "Evidence-Based Technical Analysis" and also have his book here too. My initial list was just a brief look back at my bookshelf to mention a few names I could recall off-the-top-of-my-head - I'll be adding a lot more and Aronson is the first one I'll be adding. Great book.

I quite enjoy reading academic papers related to predictions on index movements, cross-sectionals, biases, pre/post events, etc. and seeing if there's anything I can statistically correlate to current markets.

ducati916 · 10 August 2024

From @Chipp

So I was intrigued by your recommendation:

Screen Shot 2024-08-10 at 6.45.18 AM.png

Screen Shot 2024-08-10 at 6.45.32 AM.png

And a couple of reviews:

Top reviews from the United States

jjq
5.0 out of 5 stars A Good Place to Start If Traditional TA Is Letting You Down
Reviewed in the United States on December 9, 2009
Verified Purchase
UPDATE: 10/1/21

15 years later- Still the BEST BOOK on security trading you can own.

IMAGINE—

That in one hand you held a bag full of returns from YOUR technical trading system....

.... In the other hand you held a bag full of RANDOM returns from the market over the same period.

NOW IMAGINE—

That there was a 500 year old scientific tool which would allow you to COMPARE each bag of returns to determine if YOUR trading system actually works better (makes more $$$) than a system driven by pure LUCK.

The tool actually TELLS YOU IF YOU ARE “WINNING” SIMPLY BECAUSE OF LUCK or YOUR TRADING SYSTEM IS ADDING ANY VALUE, ANY VALUE AT ALL, TO YOUR TRADES.

Aronson’s book explains, in thorough detail, THAT scientific method.

###########. ORIGINAL REVIEW OF 12 YEARS AGO ############

What you take-away from a reading of this book really depends on where you're coming from.

For STATITICIANS with an interest in trading markets--You'll likely walk away with the feeling: "Yeah, that's what I've been thinking for years, nice to see someone took the time to debunk the TA myth."

For TECHNICIANS (traders) with an interest in statistics--You'll likely walk away thinking "You gotta be kidding. There are a hundred good books which can show you how to use TA to make money. This book sucks."

Aronson suggests that the truth does NOT lie in between-- He is firmly in the camp of the Statistician.

But a close reading of this powerful book does not "close the door" on profitable TA, it simply confirms what every first-year MBA learns:

"No OBJECTIVE black-box trading strategy CONSISTENTLY beats the market AVERAGES over the LONG TERM."

But hard-core TA fans take heart. You will find something interesting in this book also. I'm certain Aronson would agree with the following:

"Sure you can add COMPLEX rules to the black-box, and PERIODICALLY find runs of profitability with TA. If EXCESS returns exist only in the SHORT TERM, hey, that's good enough for me."

For those not willing to take the time to digest the painstakingly presented statistical concepts, there will be little value in this book-- This is a serious study with lots of math. Its not hard math. But math best understood after fully internalizing a college level stats class.

Even for those with a Stats or Econometrics degree statistics are tough--both computing and interpreting statistical data takes a little work. To complicate the issue, market-related statistics are fraught with half-truths, mind-bending math, and wall-street lore.

This book goes a long way to put bogus TA lore to rest by presenting a clear, scientifically sound procedure to test Technical rules.

For those seriously considering buying this book let me suggest that you find Aronson's website [...] and download and read Dr. Timothy Masters' .pdf "Monte-Carlo Evaluation of Trading Systems." The document, both in tone, and sophistication, mirrors Aronson's book.

If you like Masters' 43 page doc--you will love Aronson's 500+ page book.

The Review

I break the book up into four parts, each with various degrees of usefulness depending on your background--ie: Technician or Statistician. Below, I'll simply give what I thought was the "money-quote" from each part, plus a couple of observations for those considering buying the book.

***** Part 1: Chapter 1 - 31 pages
Objective Rules and Their Evaluation

"The isolated fact that a rule earned 10 percent rate of return in a back test is meaningless. If many other rules earned over 30 percent on the same data, 10 percent would indicate inferiority, whereas if all other rules were barely profitable, 10 percent might indicate superiority." - Aronson, page 23

Constructing Rules - Intro to bi-modal rule construction and trigger thresholds
Data Transformation - Nice review of position-bias, log-differences and testing biases
Benchmarking Rules - Good review of why "Relative-Benchmarking" is important
Beating the Benchmark - Why a profitable back test is not conclusive proof of good rule

***** Part 2: Chapters 2-3 - 130 pages
The Illusory Validity of Subjective Technical Analysis
The Scientific Method and Technical Analysis

"Statistician Harry Roberts said that technical analysts fall victim to illusion of patters and trends for two possible reasons. First, the "usual method of graphing stock prices gives a picture of successive (price) levels rather than of price changes and levels can give an artificial appearance of pattern or trend. Second, chance behavior itself produces patterns that invite spurious interpretations""-- Aronson, page 83

The Eye Deceives - Charting a random process and the representativeness heuristic
Subjective vs. Objective -- Why its important to be able to "hard-code" a TA rule
The Role of Logic - Why "Falsification" is more important than "Affirmation" in TA
Astrology vs Astronomy - Pushing the TA boundaries from pseudo- to science

***** Part 3: Chapter 4-7 - 230 pages
Statistical Analysis
Hypothesis Tests and Confidence Intervals
Data-Mining Bias: The Fool's Gold of Objective TA
Theories of Non-Random Price Motion

"Informal data analysis is simply not up to the task of extracting valid knowledge from financial markets. The data blossoms with illusionary patterns whereas valid patterns are veiled by noise and complexity. Rigorous statistical analysis is far better suited to this difficult task." - Aronson, page 172

Hypothesis Testing--Good review of probability and statistical inference
The Traditional Solution - Actually put your college-level stats knowledge to use
The Monte-Carlo Solution - Putting computer randomization and re-sampling to work
The Data-Mining Problem -- Why traditional MC solutions don't work
Inefficient Markets - How, where and why profitable TA rules should STILL exist

***** Part 4: Chapter 8-9 - 100 pages
Case Study of Rule Data Mining for the S&P 500
Case Study Results and the Future of TA

"Few rule studies in popular TA apply significance tests of any sort. Thus, they do not address the possibility that rule profits may be due to ordinary sampling error. This is a serious omission, which is easily corrected by applying ordinary hypothesis tests." - Aronson. page 449

The Operators - Reviews: channel-break-outs, moving averages, channel-normalization
The Indicators -- Reviews: price, volume, breadth, spreads, yields
The Rules - Reviews: trends, inverse trends, reversions, divergence
The Results - Analysis of why 0 of the 6,402 tested rules produced no significant results

The Bottom Line

Aronson's book reminds me of that masked-magician on TV who has given away the secrets to all the best stage illusions.

Novice magicians and apprentice conjurers will undoubtedly be "pissed-off."

But true professionals are liberated.

The best in the field can focus on new and potentially MORE exciting illusions--not the same old tricks.
Read less
45 people found this helpful

af7-bb7b-9af39c158370._CR62%2C0%2C375%2C375_SX460_.jpg

Scott C. Locklin
4.0 out of 5 stars Possibly the most important book you'll read on trading
Reviewed in the United States on August 30, 2010
Verified Purchase
This is a tough book to review. The material covered is simply not covered anywhere else I've found, and it is absolutely crucial in building a scientific approach to building trading systems. As such, you pretty much have to read this book if you want to trade and not lose your shirt. On the other hand, it's got some fairly serious flaws.

The author seems to be a "seat of his pants" proprietary trader who eventually got science-religion, and became a scientific trader. As such, it is probably more or less oriented towards people like him; people who may not have been exposed to ideas like "standard deviation" or "statistical distribution" before they read this book. I'm not sure it succeeds in explaining this issues. I found the explanations to be excellent and extremely clear; but I have a Ph.D. in physics, and have been thinking in these terms since I was a teenager. Will some 40 year old knuckle-dragger who has never heard of the Student-T distribution get anything from this? I don't know. I kind of suspect he won't. Can I think of a better way to explain these concepts to an older student coming to the ideas for the first time? Nope; certainly not. I'd probably just give them this book and hope for the best. The other flaw is also kind of a strength: the author "talks" too much. This book is over 500 pages long. The crucial material in it; the explanation of White's reality check and the Monte-Carlo analogue by Tim Masters is really only a couple of pages. Most of the other text is interesting and well written as the author is a learned and experienced man, but, well, Aronson could use an editor. I believe Ambrose Bierce once reviewed a book with, "The covers of this book are too far apart." This is unfortunately sort of true here.

I'll say it again: this book is the only one I know of which deals seriously with the issue of data mining bias. This is what separates the men from the boys. It's easy to build signal processing techniques which find real signal in financial time series (and yes, they work a lot better than the lame TA signals the author uses), but more difficult to find out when these techniques are lying. I'm planning on giving away a piece of software you can use to find some kinds of signal fairly painlessly: I probably won't give away the "reality check" stuff, because that's the hard part.

What would I have liked in the ideal world? Maybe a little less Popper and bad history of science, drop the specific test he did and add more technical stuff on the various forms of reality check. For example, the reality checks described here deal exclusively with simple entry points: how do you deal with more complex entries and exits and money management? There are ways of doing this for certain, but this book is only the beginning in figuring them out. What do you do about signals which have the Markov property, or, for example, what do you do with signal-finding algorithms which have the bootstrap property baked into them already? What about a data mining reality check for Sharpe ratio? What do you do when you have a signal with varying probability of being true? By this, I mean, you may have a signal you have determined has a 51% chance of being correct, and in some cases, you may have a signal which you know has a 54% chance of being correct (you probably will never have a 99% correct signal; not in finance anyway); what you do with such signals is different. Sometimes you have a signal where you have no idea what the probability of success is: these need to also be handled differently. There are also issues with correlations between trading systems, bet sizing ... I supposed there are lots of issues like this which I would have liked to see addressed in a book like this, but until someone writes such a book, we have to make due with this book, Grinold and Kahn and SSRN. Speaking of Grinold and Kahn, while this is probably outside the author's field of expertise, application of these ideas to classical macro/microeconomic models used in the "alpha plus" investment funds would have been incredibly awesome. Those fields use plain old regression to build their accounting based models. G & K's book doesn't mention much beyond Student-T tests for backtesting (Elton and Gruber does mention the bootstrap without telling much on how to use one). Applying the machinery of White's reality check to this "arbitrage pricing model" sort of thing would have been a huge win: far more interesting than using it on various technical analysis methods as he does in the second to last chapter. Anyway, that's how I would have rolled.
Read less
36 people found this helpful

So have now purchased the book.

I'll report back in due course after I have read it.

jog on
duc

Skate · 10 August 2024

ducati916 said:
I'll say it again: this book is the only one I know of which deals seriously with the issue of data mining bias. This is what separates the men from the boys. It's easy to build signal processing techniques which find real signal in financial time series (and yes, they work a lot better than the lame TA signals the author uses), but more difficult to find out when these techniques are lying. "Sure you can add COMPLEX rules to the black-box, and PERIODICALLY find runs of profitability with TA. If EXCESS returns exist only in the SHORT TERM, hey, that's good enough for me."

@ducati916, your post struck a chord with me. It feels like I’ve absorbed the essence of Aronson’s entire book just by reading your insights. @Richard Dale thread couldn’t have come at a better time. The concept of “Believing and Achieving” is something I’d love to explore further in my thread.

Believing that I’ve covered all the bases
Backtesting, optimisations, in and out-of-sample testing, exporting results into Excel for additional data analysis and plotting scatter charts and equity curves to better understand the metrics have given me the confidence to start live trading. Confidence comes from trusting the rigorous personal process of developing strategies.

Skate.

Chipp · 10 August 2024

Richard Dale said:
My initial list was just a brief look back at my bookshelf to mention a few names I could recall off-the-top-of-my-head - I'll be adding a lot more and Aronson is the first one I'll be adding. Great book.

I am sure you will be adding quite a few more books to the list in time as various subjects attached to systematic trading are breached . Eventually a definitive list of essential reading would be a fantastic resource . I certainly have a couple more to add to the list in time

Richard Dale said:
I quite enjoy reading academic papers related to predictions on index movements, cross-sectionals, biases, pre/post events, etc. and seeing if there's anything I can statistically correlate to current markets

I also pursue the types of thing you describe here . Over the years just for practice i have attempted to prove/disprove many market myths as well has turned into a very enlightening process , From simple thing like golden/death crosses to the extreme other end . Not all myths are myths are myths but many are . Much of what i code is testing to exploit price imbalances , this is where a lot of ' gold ' exists . Mostly based around logical patterns created by events .

This is a deep rabbit hole and there is an incredible amount of ground to cover , maybe a set of bullet points with books pertaining to each point type of list . Systematic traders are essentially building a manual to trade something like an engineer would . There are so many decisions that need to be made to start this journey . Algos are simply ( not suggesting its simple for a second) a set of questions asked in a particular order like a flow chart that will eventually come out with a binary result or a series of binary results . The Decision Tree , you basically should be applying the same concept to objectives in becoming systematic . I could go on for hours but i dont have the time atm .

ducati916 · 11 August 2024

So I will attach 3 of the papers that are associated with the book 'Evidence Based Technical Analysis'.

Having read the papers, then in a highly simplified definition: what is being sought is a statistically significant edge. To differentiate out of the mass of data a causation as opposed to random luck.

Obviously I haven't even received the book yet, never mind read it, so that definition may become more nuanced moving forward.

From Mr Chipp's post abovr:

Much of what i code is testing to exploit price imbalances , this is where a lot of ' gold ' exists . Mostly based around logical patterns created by events .

So these two sentences drive the following thoughts:

(i) Arbitrage is 1 instrument with 2 different prices existing at the same time. Sell 1 buy 1 and lock in the profit. Buy or produce physical gold against a futures contract for gold. This was the basis of the futures markets.

(ii) Risk Arbitrage 1: An example: An off the run 10yr bond and the newly issued 10yr bond. now technically, these are 2 different instruments, with slightly different durations. However both will pay par at maturation, or $10,000 face value. The price differential is usually measured in basis points. The risk is the differential in time to maturity.

Obviously, this is the trade that (amongst others) that blew up LTCM due to taking massive leverage to increase the dollar returns

I would classify this as Quant.

(ii) Risk Arbitrage 2: where Company A is going to buy Company B and tenders an offer for outstanding shares above the market price. Buy shares of B and wait for completion. The risk is that it never closes.

So returning to the quote above:

Price imbalance: yes in our 2 examples.
Event: yes in our 2 examples.

Now on a purely technical analysis type of scenario we have some examples:

(i) Dogs of the Dow,
(ii) The January effect,
(iii) Closing imbalances between SPX Futures and SPY ETF

This I would classify as the more fertile ground for what we are discussing re. backtesting and systematic trading.

Again we do have:

(i) Price imbalances,
(ii) Event

Returning to the LTCM example: the issues were that because of the 2 different instruments involved, prices were not locked together, ie as one went down, the other went down and the massive leverage involved meant that as the spread opened the losses magnified past the risk capital held.

Hedge Funds are still using variants of the LTCM example: The Yen Carry trade is simply a variation of the above, which went into meltdown last Monday.

My boss (technically) had his hands full last Sunday night/Monday morning as a cadre of SPX traders in the firm were over-leveraged and just about blew themselves up. Lucky escape for them. They lost a ton of money but are still alive...just.

The reason I mention it is as a firm, many traders follow a simplified market quant system. I post it most weeks. I personally don't, but apparently it has been backtested and is profitable.

So I'll post them here just in case anyone wants to test them:

Screen Shot 2024-08-10 at 11.09.32 AM.png

Screen Shot 2024-08-11 at 6.48.42 AM.png

Screen Shot 2024-08-11 at 6.48.07 AM.png

Screen Shot 2024-08-10 at 11.11.16 AM.png

So nice and simple.

This is the 'edge'.

Which 'should' be in excess of random luck.

jog on
duc

ducati916 · 11 August 2024

jog on
duc

Chipp · 11 August 2024

ducati916 said:
jog on
duc

I got to be honest i am stoked when 75% of investors disagree with me . Thanks for that , i enjoyed it

Richard Dale · 11 August 2024

I'm gradually adding more resources to this list, in no particular order.

Add your (considered) ideas (but not discussion about it) here.

Chipp · 12 August 2024

A good source for formulas of various technical indicators would be TA-AZ . There is a free copy somewhere on the web , will try and track it down later . Coding has a lot of maths in it and some of these formulas are a great starting point , ultimately if you go very deep you will be writing your own but existing code is a great place to start , even to ' hack '

Richard Dale · 13 August 2024

Skate, I have Rich Brennan's blog noted above - no need to reinterpet/repost articles from it here. This is a thread with links/references to various resources, not the actual resources themselves.

If you wish to discuss his blog post, do it in a different thread.

Richard Dale · 13 August 2024

Chipp said:
A good source for formulas of various technical indicators would be TA-AZ . There is a free copy somewhere on the web , will try and track it down later . Coding has a lot of maths in it and some of these formulas are a great starting point , ultimately if you go very deep you will be writing your own but existing code is a great place to start , even to ' hack '

Hi Chipp,

I'm not sure this resource is exactly systematic/algorithimic trading.... sure it's a building block for certain algorithmic trading, but not really a core resource. If you agree, I'll get the posts removed on this (but feel free to put it into other threads about TA).

I've used this book many times over the years.

Resources for Systematic/Algorithmic Trading

Chipp