Best accurate historical data source

[DingoCC]

Hi,

asxhistoricaldata.com goes back to 1997. Yahoo historical provides daily from about 1999. For the purpose of backtesting which is better to use in regards to adjustments (splits, mergers, etc) and completeness. A very quick spot check shows Yahoo to be missing some days in 1999-2000.

Regards,
Colin.

 

[lesm]

Hi,

asxhistoricaldata.com goes back to 1997. Yahoo historical provides daily from about 1999. For the purpose of backtesting which is better to use in regards to adjustments (splits, mergers, etc) and completeness. A very quick spot check shows Yahoo to be missing some days in 1999-2000.

Regards,
Colin.

Hi Colin

Part of it comes down to whether you want to pay for it or you want it for free.

The caveat emptor for free data, is that you get what you pay for. There are no guarantees as to the level of data cleansing that has been performed on free data or if other than possibly some relatively basic cleansing may been undertaken. Would you really be prepared to put a high level of your capital at risk based on this type of data.

The alternative is to purchase your data, where you should be in a position expect that an acceptable level of data cleansing has been performed, inclusive of those you mention above, with no missing days. It would be expected that a level of support would be required where you have any queries related to the data, which may require resolution. In this regard, a couple of potential candidates that come to mind are Premium Data and Just Data. There are others, but you also need to consider, which markets/instruments that you intend to trade in, as well as whether you may just require EOD data or intraday data.

In general terms, to ensure that you have good quality of data available for back testing and/or forward testing, which results/statistics would you place a higher level of reliance on free data or paid data. Garbage in, Garbage Out (GIGO)

From memory, at one time Dukascopy used to make intraday data available and intraday data can be downloaded via Meta Trader.

The proper cleansing of data is not a minor activity, as besides the obvious events that may occur there are instances of dirty data, which need to be identified and corrected.

Disclosure:

I use Just Data as one of my sources of data.

 

[Wysiwyg]

Remember when you use a group of stocks to backtest such as the All Ords, the present day group contains stocks that have survived the past time you backtest over as well as new additions along the way. The list would be much different 10 years ago than it is today. I use Premium Data and backtest no further than 2 years and use the service mainly for T/A and to explore for stocks in present time.
Sick of backtesting really.

 

[howardbandy]

Hi,

asxhistoricaldata.com goes back to 1997. Yahoo historical provides daily from about 1999. For the purpose of backtesting which is better to use in regards to adjustments (splits, mergers, etc) and completeness. A very quick spot check shows Yahoo to be missing some days in 1999-2000.

Regards,
Colin.

Hi Dingo --

Consider whether the conditions reflected in the data have changed over the historical period being tested to develop your trading system. Your trading system will be profitable in the future only if the model can learn patterns from the past that persist into the future. Stale data -- data containing patterns that no longer occur -- will lead your development toward rules that cannot be profitable in the future. Stale data may be worse than no data. Depending on your model, you may not need, or even want, the older data.

Best,
Howard

 

[DingoCC]

Hi Dingo --

Consider whether the conditions reflected in the data have changed over the historical period being tested to develop your trading system. Your trading system will be profitable in the future only if the model can learn patterns from the past that persist into the future. Stale data -- data containing patterns that no longer occur -- will lead your development toward rules that cannot be profitable in the future. Stale data may be worse than no data. Depending on your model, you may not need, or even want, the older data.

Best,
Howard

Hello Howard,

Who's to say next month will be like last month or more like sometime in 2001? I'm a little perplexed at the idea of 'stale'. Is it not reasonable to expect any pattern in the past to occur again, regardless of how 'old' it is? Markets are cyclic on many scales right. In a Curtis Faith book I finished this week he says something similar in that back-test results from recent years have more weight than prior years. What's the implication here; limit backtesting to the last 3 years and chop it up into small windows for walk-forward testing?

My model is a long only medium term trend follower that I intend to trade for a good few years.

Dingo.