# Historical Index Constituents



## nizar (13 August 2007)

I was very disappointed to get this email from Richard Dale from Premium Data this morning:



> We don’t provide historical constituents for any index.
> No other data providers have this information – although I’m sure Standard & Poors will provide this to you for a price.




I thought I can backtest on the XAO index with ease going back 10 or 20 years. Unfortunately not.

Does anybody know how I can overcome this problem?

Thanks.


----------



## CanOz (13 August 2007)

nizar said:


> I was very disappointed to get this email from Richard Dale from Premium Data this morning:
> 
> 
> 
> ...




So you want to back test the cash index? Curious why?

Cheers,


----------



## nizar (13 August 2007)

CanOz said:


> So you want to back test the cash index? Curious why?
> 
> Cheers,




No.
Its constituents.
Say if I want to run a backtest using my universe as the XAO.


----------



## CanOz (13 August 2007)

nizar said:


> No.
> Its constituents.
> Say if I want to run a backtest using my universe as the XAO.




Ahhh, sorry i get ya now. But all you need is a watchlist of all of those stock codes right? Someone must have a MS list of those surely.

Cheers,


----------



## nizar (13 August 2007)

CanOz said:


> Ahhh, sorry i get ya now. But all you need is a watchlist of all of those stock codes right? Someone must have a MS list of those surely.
> 
> Cheers,




Yeh the All Ords is getting updated every few months though.
So the 1992 All Ords for example, is much different to what it is now.
Also an issue of survivorship bias is there as well.

LOL yeh somebody must....

Anybody here care to lend a hand, please......

Thanks


----------



## happytrader (13 August 2007)

Hi Nizar

www.bigcharts.com has xao and constituents from as far back as 1988 if thats of any use to you.

Cheers
Happytrader


----------



## nizar (13 August 2007)

happytrader said:


> Hi Nizar
> 
> www.bigcharts.com has xao and constituents from as far back as 1988 if thats of any use to you.
> 
> ...




Hi Happy.

Thanks for trying to help but not really what i was after.
I need to know what was IN the all ords Index from 1992 all the way until today.


----------



## Sir Burr (13 August 2007)

nizar said:


> I need to know what was IN the all ords Index from 1992 all the way until today.




What reason is there to use it's constituents. Might be difficult in practice to backtest with them while changing every 3 months (or whatever).


----------



## tech/a (13 August 2007)

Nizar.

I know what your after and why.
You wont find it.
It doesnt matter---as Radge says its WHY your system is profitable thats important,not WHAT you test it against.

These are NOT cryptic clues nor less than straight answers.When you understand them all will be clear and like myself and many others you'll have your Ahhhh---Haaaa moment.

When you least expect it.


----------



## nizar (13 August 2007)

Sirr Burr and tech/a.
You are both seasoned system designers so you know your stuff.
Thanks for your thoughts.

I thought coz I see alot of backtesting done on forums and blogs, and often the universe used is the ASX300 or the s&p500. So I tried to do something similar. 

I thought it was a popular practice in systems testing and design -- turns out not.

Liquidity filters will have to do me for now.


----------



## theasxgorilla (14 August 2007)

Hi Niz,

We have to rely on the assumption that even though stocks delist due to both failure and M&A activity that even those stocks that eventually fail may have once been trend following candidates.  Consider the attached chart from Chemeq...where it began, where might have become a trend following candidate, where a system might have sold it, and whether it might eventually become delisted.

If you use a list like the XAO or ASX300 going forward, you're pretty much guaranteed that over the next 10 years your universe will have included much more than just 500 or 300 shares.  The consensus of the experts seems to be that with the right filters in place (momentum/price??) the larger the universe the better the results.  So you could conclude that a system which targets those stocks that stand the greatest chance of being delisted should perform better in reality than in testing.

ASX.G


----------



## nizar (14 August 2007)

theasxgorilla said:


> Hi Niz,
> 
> We have to rely on the assumption that even though stocks delist due to both failure and M&A activity that even those stocks that eventually fail may have once been trend following candidates.  Consider the attached chart from Chemeq...where it began, where might have become a trend following candidate, where a system might have sold it, and whether it might eventually become delisted.
> 
> ...




ASX.G

Thanks for your post.
Yeh i realise that now -- after the replies here and elsewhere, and after doing some of my own testing.

And i agree about what your saying -- All the systems Iv been experimenting with so far have performed significantly better when delisted stocks were included in the backtest.

So i guess at least some of these stocks were champions at some stage.


----------



## Tanaka (11 February 2011)

I’m looking for the constituents of the ASX 300 for the past 20 years. I have spent an hour trying to find them on the S&P website and have been waiting for a reply from them as well. I understand they change every couple of months and will be a pain to load into TradeSim, but it is needed for my backtesting.

This thread died a while ago. Anyone have this information or know where I can get it?

Thank you,

Tanaka


----------



## Tanaka (12 February 2011)

Tanaka said:


> I’m looking for the constituents of the ASX 300 for the past 20 years. I have spent an hour trying to find them on the S&P website and have been waiting for a reply from them as well. I understand they change every couple of months and will be a pain to load into TradeSim, but it is needed for my backtesting.




S&P replied to my above request, they advised me that it would cost $1000 for this information 

I don't believe this info. is worth a $1000. It used to be free on their website


----------



## holyLemon (28 September 2011)

Hey guys thanks for a very informative thread!

I was wondering - (apologies in advance if this is a dumb question!) - if your universe of stocks is now expanded to simply include everything (including the since-delisted ones), how do you know what the total universe of stocks (say everything that was traded on the ASX in 2003) was?

You would also then need information on which stocks were added(floated) and delisted from the exchange on an ongoing basis, right?

I would like to do a ranking system backtest which starts at 2006, uses 3 years' worth of historical data (ie back to 2003) ...

Any advice would be greatly appreciated !

HL


----------



## holyLemon (4 October 2011)

So was this a dumb question? 

Any experienced backtesters using EOD ASX data, any advice would be greatly appreciated !!

Anyone using Yahoo finance for free historical financial data (of a big portfolio of ever-changing stocks)?

HL



holyLemon said:


> Hey guys thanks for a very informative thread!
> 
> I was wondering - (apologies in advance if this is a dumb question!) - if your universe of stocks is now expanded to simply include everything (including the since-delisted ones), how do you know what the total universe of stocks (say everything that was traded on the ASX in 2003) was?
> 
> ...


----------



## tech/a (4 October 2011)

holyLemon said:


> So was this a dumb question?
> 
> Any experienced backtesters using EOD ASX data, any advice would be greatly appreciated !!
> 
> ...




Have seen your post and will answer when I have a little more time.


----------



## Billyb (4 October 2011)

holyLemon said:


> So was this a dumb question?
> 
> Any experienced backtesters using EOD ASX data, any advice would be greatly appreciated !!
> 
> ...




Don't bother with Yahoo ASX data if you are backtesting a system that you eventually plan to use, it's fine for practising systems development but that's about it. There are a lot of errors on there once you leave the top 100-200 stocks. Look at Norgate Premium Data I think it's great. They will keep the database up to date for you. They DO have all the delisted ASX data but it's not organized and so it could be difficult to incorporate into your testing unless you are a real hardcore and don't mind the extra work

It's hard to get around this issue of survivorship bias. But if your system works on lots of different sets of data then chances are it will work in real life even if there was survivorship bias.


----------



## cudderbean (4 October 2011)

Here are a list of codes from the All Ords in 1994 I found on an old puter. Not sure why there are only 172 of them. 

I used them once to backtest a buy in gloom and sell in boom theory in memory of a Mr Womack, a Texas pig farmer, who bought a range of quality dividend paying stocks whenever he saw on the *first *item of the main news (not financial news) in the evening “Stock market 3 year low’ etc, went back to his pig farm and thought it wonderful that he got a dividend cheque every 6 months without having to do any work. Then a few years later, one night on *the main news item *he heard “Stock market record high”. Then he went to his broker and sold the lot. He never picked the exact bottom nor the exact top. But he didn’t care... he never lost.

I did this myself in the 1980s with some excellent results. Then I bought my first computer, got greedy, and tried to pick the precise reversal points.... disaster.

I back tested the codes with various entry and exit points, even investing at the worst possible moments. I think it produced a capital return of from 6-10% pa from memory plus all your divs. 

I remember I got criticised on the old Stock Central forum for "past post betting"... it’s only the surviving codes that I could test all the way back. The moderator (Daryl Guppy.. all of whose books I own and value!) said many of the failed companies had been dropped from the XAO for non performance. Another poster said yes, but many companies had also been takeover targets and some such as BHP had been split several times, so I was actually being conservative about the theory’s success.

Must dig out all my research sometime from the old puter for folks to mull over here.

Anyway, I hope this list of codes from 1994 helps you.

ABC
ADB
AFI
AGL
AGR
AJO
AJR
AMC
ANI
ANZ
AOG
APF
APN
ARG
ARL
ARS
ASH
AUD
AUG
AWA
AZL
BEN
BHP
BIL
BKL
BOC
BOR
BPC
BQD
BRL
BRY
BTA
BTE
BTG
BTP
CBA
CCL
CEP
CEQ
CIM
CMC
CMF
CML
CMX
CNG
CPL
CPY
CRG
CRS
CSR
CTR
CTX
CTY
CYG
DGD
DOM
EDI
EEC
ELT
EML
EMP
ENE
ERG
FAF
FAI
FBG
FCL
FHF
FIF
FLL
FMH
FOA
FXJ
GCM
GIO
GMF
GOW
GPT
GSP
GWT
HAH
HCO
HIH
HLX
HRW
ICT
JFC
JJS
JUP
KGM
KYC
LAC
LEI
LLC
LNN
MAH
MAY
MCP
MIM
MMC
MMF
MRV
NAB
NBH
NCM
NCP
NDY
NFD
NFM
NML
NMP
OEC
OPS
ORI
OSH
PAS
PBB
PDP
PEM
PIP
PMM
PMP
PNI
PPT
PTD
PTZ
QBE
QMC
QNI
QRL
QUR
RGS
RIC
RSG
RSM
RTH
RUP
SBM
SCH
SCP
SEV
SGB
SGP
SGW
SID
SMI
SMS
SPP
SPS
SPT
SRP
STO
SVR
SWD
TEH
TOR
TYA
VRL
VWD
WAN
WBC
WES
WFT
WHW
WMC
WOW
WPL
WPT
WSF
WSL
WYL
YTS


----------



## cudderbean (5 October 2011)

nizar said:


> I was very disappointed to get this email from Richard Dale from Premium Data this morning:
> 
> 
> 
> ...




I wrote a little scan for the current XAO (could do it for any group or whole mkt) to discover how far back Bullcharts data goes on its standard software, which is all downloadable into a spreadsheet - one of the features I like about BC... or testable on Tradesim. 

Also in an archive search for Tradesim... the backtesting module for Bullcharts I noticed this..
https://www.aussiestockforums.com/forums/archive/index.php/t-6709.html


TradeSim8th-January-2010, 03:07 PM
One other nice feature that people may not be aware of is the fact that you can view price charts of all delisted and expired securities. These are securities that are no longer current and are not downloaded onto your machine but the Bull Charts server can slurp up the data to you at any time.

For example you can view NCP before it was changed to NWS and the NWS chart shows both the adjusted and non adjusted data in different colors.

I don't know any other data supplier that supplies this information 

BC's first eod for the current XAO goes back to... (had to resubmit ..text too long, so only included XAO codes that go back prior to 1997)


ANZ	ANZ Banking Grp Ltd	1/10/1986
AWC	Alumina Limited	1/10/1986
BHP	BHP Billiton Limited	1/10/1986
CSR	CSR Limited	1/10/1986
FGL	Foster's Group Limited	1/10/1986
NAB	National Australia Bank Limited	1/10/1986
NWS	News Corporation	1/10/1986
STO	Santos Limited	1/10/1986
WBC	Westpac Banking Corporation	1/10/1986
WPL	Woodside Petroleum Limited	1/10/1986
AGK	Agl Energy Limited	19/11/1986
AMC	Amcor Limited	19/11/1986
BKW	Brickworks Limited	19/11/1986
BOC	Bougainville Copper Limited	19/11/1986
BXB	Brambles Limited	19/11/1986
CTX	Caltex Australia Limited	19/11/1986
ELD	Elders Limited	19/11/1986
HIL	Hills Holdings Limited	19/11/1986
JHX	James Hardie Industries Se	19/11/1986
LEI	Leighton Holdings Limited	19/11/1986
MAH	Macmahon Holdings Limited	19/11/1986
MCP	Mcpherson's Limited	19/11/1986
ORI	Orica Limited	19/11/1986
OSH	Oil Search Limited	19/11/1986
SBM	ST Barbara Limited	19/11/1986
SSN	Samson Oil & Gas Limited	19/11/1986
SST	Steamships Trading Company Limited	19/11/1986
CNA	Coal & Allied Industries Limited	20/11/1986
HZN	Horizon Oil Limited	20/11/1986
REH	Reece Australia Limited	20/11/1986
RSG	Resolute Mining Limited	20/11/1986
WTP	Watpac Limited	20/11/1986
ABC	Adelaide Brighton Limited	21/11/1986
BKL	Blackmores Limited	21/11/1986
BTA	Biota Holdings Limited	21/11/1986
SPT	Spotless Group Limited	21/11/1986
WES	Wesfarmers Limited	21/11/1986
ALS	Alesco Corporation Limited	24/11/1986
SOL	Soul Pattinson (W.H)	25/11/1986
SXY	Senex Energy Limited	25/11/1986
GUD	G.U.D. Holdings Limited	2/12/1986
VRL	Village Roadshow Limited	26/02/1987
TRY	Troy Resources NL	5/03/1987
PPT	Perpetual Limited	6/04/1987
SGP	Stockland Units/Ord Stapled	27/04/1987
ORL	Orotongroup Limited	18/06/1987
ARP	Arb Corporation Limited	26/06/1987
CTY	Country Road Limited	2/07/1987
DLS	Drillsearch Energy Limited	2/07/1987
PEM	Perilya Limited	23/07/1987
PMV	Premier Investments Limited	6/08/1987
HVN	Harvey Norman Holdings Limited	3/09/1987
LLC	Lend Lease Group Unit/Ord Stapled	8/11/1988
GPT	GPT Group Stapled Sec.	29/11/1988
CGX	Cga Mining Limited	11/12/1989
AHD	Amalgamated Holdings Limited	1/10/1990
ALK	Alkane Resources Limited	1/10/1990
CCL	Coca-Cola Amatil Limited	1/10/1990
CPB	Campbell Brothers Limited	1/10/1990
DYL	Deep Yellow Limited	1/10/1990
ERA	Energy Resources Of Australia Limited	1/10/1990
GOW	Gowing Bros Limited	1/10/1990
MPO	Molopo Energy Limited	1/10/1990
QBE	QBE Insurance Group Limited	1/10/1990
RHL	Ruralco Holdings Limited	1/10/1990
ANG	Austin Engineering Limited	2/10/1990
BPT	Beach Energy Limited	2/10/1990
EQT	Equity Trustees Limited	2/10/1990
IMD	Imdex Limited	2/10/1990
TRU	The Trust Company Limited	3/10/1990
IXR	Imx Resources Limited	4/10/1990
RRS	Range Resources Limited	4/10/1990
MSF	MSF Sugar Limited	5/10/1990
RRL	Regis Resources Limited	5/10/1990
RIC	Ridley Corporation Limited	8/10/1990
COF	Coffey International Limited	10/10/1990
EWC	Energy World Corporation Ltd	10/10/1990
ARH	Australasian Resources Limited	17/10/1990
MCO	Morning Star Gold N.L.	1/11/1990
FWD	Fleetwood Corporation Limited	23/11/1990
APN	APN News & Media Limited	2/01/1991
FXJ	Fairfax Media Limited	2/01/1991
KCN	Kingsgate Consolidated Limited.	2/01/1991
MND	Monadelphous Group Limited	2/01/1991
NCM	Newcrest Mining Limited	2/01/1991
PMP	PMP Limited	2/01/1991
PRT	Prime Media Group Limited	2/01/1991
SGM	Sims Metal Management Limited	2/01/1991
TEL	Telecom Corporation	2/01/1991
UOS	United Overseas Australia Limited	2/01/1991
APE	Ap Eagers Limited	28/06/1991
BEN	Bendigo And Adelaide Bank Limited	28/06/1991
CVN	Carnarvon Petroleum Limited	28/06/1991
GWA	GWA Group Limited.	28/06/1991
CBA	Commonwealth Bank Of Australia.	12/09/1991
SWM	Seven West Media Limited	9/01/1992
WOW	Woolworths Limited	12/07/1993
TOL	Toll Holdings Limited	12/10/1993
DVN	Devine Limited	25/11/1993
GNS	Gunns Limited	16/12/1993
CNX	Carbon Energy Limited	22/12/1993
ASL	Ausdrill Limited	6/01/1994
PDN	Paladin Energy Ltd	29/03/1994
CFX	CFS Retail Property Units	7/04/1994
SMM	Summit Resources Limited	7/04/1994
GBG	Gindalbie Metals Ltd	12/04/1994
PEN	Peninsula Energy Limited	26/05/1994
CPU	Computershare Limited.	27/05/1994
CSL	CSL Limited	8/06/1994
THG	Thakral Holdings Grp Ordinary/Unit	17/06/1994
HGO	Hillgrove Resources Limited	22/07/1994
SKE	Skilled Group Limited	27/07/1994
TAH	Tabcorp Holdings Limited	26/08/1994
CWP	Cedar Woods Properties Limited	31/08/1994
WBB	Wide Bay Australia Ltd	19/09/1994
AGS	Alliance Resources Limited	19/10/1994
ATI	Atlantic Limited	19/12/1994
TAM	Tanami Gold NL	21/12/1994
FRI	Finbar Group Limited	13/02/1995
SDG	Sunland Group Limited	28/02/1995
QAN	Qantas Airways Limited	31/07/1995
PNA	Panaust Limited	20/11/1995
DJS	David Jones Limited	27/11/1995
FLT	Flight Centre Limited	1/12/1995
COH	Cochlear Limited	4/12/1995
SHL	Sonic Healthcare Limited	7/12/1995
CPL	Coalspur Mines Limited	5/01/1996
AVJ	Avjennings Limited	12/01/1996
CQT	Conquest Mining Limited	29/01/1996
TCL	Transurban Group Triple Stapled Sec.	15/03/1996
CUE	CUE Energy Resources Limited	25/03/1996
OZL	Oz Minerals Limited	30/04/1996
NCR	Nucoal Resources NL	31/05/1996
ALL	Aristocrat Leisure Limited	9/07/1996
CCC	Continental Coal Limited	19/07/1996
MQG	Macquarie Group Limited	29/07/1996
TAP	TAP Oil Limited	17/09/1996
SMX	SMS Management & Technology Limited.	14/10/1996
RED	RED 5 Limited	16/10/1996
SHV	Select Harvests Limited	16/10/1996
SGN	STW Communications Group Limited	1/11/1996
RCR	RCR Tomlinson Limited	15/11/1996


----------



## tech/a (5 October 2011)

This is a problem which those of us without the resources of a small country face when confronted with he survivorship question.

It's a problem when using any sub index of stock in a portfolio system.
As already pointed out there are many factors including membership of the sub group which are missed on many stocks particularly over the longer term--- years.

I have pondered over the dilemma often.
My own personal conclusion is the following.
(1) The occurrences of delisting are rare and to be long in a stock at the time of delisting whilst not impossible is very rare.
(2) Portfolio trading commonly has 10 or more open trades in the portfolio at one time.delisting would result in around 10 % or less loss in any one occurrence .
(3) Splits have a similar effect and will show an exit normally as the split looks like a severe drop in price.
(4) Take overs and changes of code show one trade left open until closed by the summing of all open profit at the end of testing or by code which looks for in activity.

I currently can't do anything about survivorship.
From the many systems I have tested and traded I've not had one which trades outside of it's blue print of Montecarlo testing.
The only time I have seen this occur and it happens to them all ---- when market conditions are outside of those used in testing.


----------



## holyLemon (31 October 2011)

Hi guys,

I was wondering tech/a if you could recommend a good data provider I basically I just need some EOD data spanning back 8 years for the ASX 200. Ideally if it has some sort of indication of split points (along with their ratios, as not all splits are 2:1) that would be really great and also if the constituent list is historically accurate (as you pointed out not too crucial but for backtesting my system it would be good to always have 200 securities to test)

I'd like to download the data and import into a locally-running mysql (or sql server) database for backtesting with. 

This is ONLY for past data, I don't need live data.

Quite happy and willing to pay for it.

I'm building a pair-trading type system that will rank the ASX200 daily and select the best and worst for pairs-trading candidates.

Regards
HL



tech/a said:


> This is a problem which those of us without the resources of a small country face when confronted with he survivorship question.
> 
> It's a problem when using any sub index of stock in a portfolio system.
> As already pointed out there are many factors including membership of the sub group which are missed on many stocks particularly over the longer term--- years.
> ...


----------



## Punta (31 October 2011)

I am new to this, but I am finding eoddata.com to be pretty good.  I think they have historical indexes as well.

As with most, the history does not include stocks that no longer trade, so it may be impossible to construct your own historical ASX200 etc


----------



## tech/a (31 October 2011)

holyLemon said:


> Hi guys,
> 
> I was wondering tech/a if you could recommend a good data provider I basically I just need some EOD data spanning back 8 years for the ASX 200. Ideally if it has some sort of indication of split points (along with their ratios, as not all splits are 2:1) that would be really great and also if the constituent list is historically accurate (as you pointed out not too crucial but for backtesting my system it would be good to always have 200 securities to test)
> 
> ...




http://www.premiumdata.net/

Call Richard Dale directly he is as up on data as anyone


----------



## holyLemon (1 November 2011)

Many thanks guys, have contacted them.

HL



tech/a said:


> http://www.premiumdata.net/
> 
> Call Richard Dale directly he is as up on data as anyone


----------



## Richard Dale (3 October 2018)

Just updating this old thread with new information as it still comes up on various search engines.  Our Norgate Data service allows you to backtest on historical index constituents.


----------



## beeg20 (4 June 2020)

I have recently subscribed to Norgate Data with historical index constituents. Previously i was using premium data, backtesting on all ords + delisted stocks. My backtesting results with all ords historical index constituents is vastly different. I am wondering if anyone has compared it and had a similar experience?


----------



## Richard Dale (4 June 2020)

That's expected - it's a significantly different universe.  

Plus, you might also be using a different adjustment method too.

If you're using AmiBroker, you can check the Detailed Trade Log to view which trades each is taking and then work back to the individual stocks to figure out why.  You might want to make use of the plottable indicator Charts -> Norgate Data -> Index Constituent too.

Cheers,
Richard.


----------

