Data - Where from and how much?

[nizar]

Just wanted to know where every1 gets their data and how much do you pay?

Obviously im after quality data that would include spilts, adjustments and delisted stocks (for historical data for backtesting purposes).

Any1 use just data or premium data?
Any feedback about those guys?

Is their data compatible with an amibroker/tradesim set-up ?

Thanks

 

[tech/a]

Re: Data

Just data.

I like it and was one of those who got in early and bought lifetime supply of 7 bourses for $660---thats now gone and its a monthly amount.Google them.

Its powerful and very handy to be able to makeup my own indexes.
Just reciently the index I have for those in my portfolio diverged against the XJO telling me that there was selling into the strenght of the general market.

Pretty helpful.
You can also make up composites of advancers and decliners.

Support is excellent emails answered next day.
data problems rare. got my first only a few days ago.
Phone them and support will talk you through a problem.

Datas clean everything done for you.
Yeh I'm happy.

 

[nizar]

Re: Data

Just data.

I like it and was one of those who got in early and bought lifetime supply of 7 bourses for $660---thats now gone and its a monthly amount.Google them.

Its powerful and very handy to be able to makeup my own indexes.
Just reciently the index I have for those in my portfolio diverged against the XJO telling me that there was selling into the strenght of the general market.

Pretty helpful.
You can also make up composites of advancers and decliners.

Support is excellent emails answered next day.
data problems rare. got my first only a few days ago.
Phone them and support will talk you through a problem.

Datas clean everything done for you.
Yeh I'm happy.

Thanks for your feedback tech.
Yeh just data charge about $100 for historical and $30/month for ASX historical end of day data.
Looks good.

This may sound like a stupid question but would the historical data, if i buy it today, does it cover everything up until yesterdaY?

I would expect so....

 

[Kauri]

Re: Data

Thanks for your feedback tech.
Yeh just data charge about $100 for historical and $30/month for ASX historical end of day data.
Looks good.

This may sound like a stupid question but would the historical data, if i buy it today, does it cover everything up until yesterdaY?

I would expect so....

Historical data is on a CD and covers up to the end of the last month.. their download server holds about 6 months, so they overlap.

 

[pirx]

Re: Data - naive question - why not from yahoo?

what is the advantage of getting the (daily) stock data from the sources mentioned in this thread?

http://finance.yahoo.com/ has the daily data going years back, both for major indexes and for shares, including ASX. Also, ASX data (daily and historical) are available for free from the comsec site.

so unless one wants to have intraday data, why any of these sources would be preferable?

 

[mrWoodo]

Re: Data

Pirx, I use PremiumData for AmiBroker (using MetaStock plugin) Check out this page as an example of how it breaks the securities down into the asx/gics structures. This is why I'm happy to pay the $30/mth for eod data. Their homepage blurb sums it up,

You need to ask yourself whether you want to be a trader or a database manager.

It's a biggish hit upfront, but when I think about it as a dollar a day, it's insignificant

 

[nizar]

Re: Data - naive question - why not from yahoo?

what is the advantage of getting the (daily) stock data from the sources mentioned in this thread?

http://finance.yahoo.com/ has the daily data going years back, both for major indexes and for shares, including ASX. Also, ASX data (daily and historical) are available for free from the comsec site.

so unless one wants to have intraday data, why any of these sources would be preferable?

Data is clean and adjusted for splits, consolidation, etc.
And it includes delisted stocks.

This is essential as its very possible that your system will buy stocks that become delisted in real-time.

And as for consolidations, check out CUO. Went from 3.8c to 38c in one day if you look at the ASX charts, but actually it was a consolidation.

You dont want your system to spit out awesome numbers, if they are meaningless because they are based on cheap data.

 

[nizar]

Re: Data

How far back is data required for backtesting?
Premiumdata offers 1992 onwards for ASX.
Just data from 1980 onwards.

Id really like to see how my system would perform from 1988-1992, pretty much a flat market, but damn, 1993 was a great year, 33% up, much more than anything during the 2003-current bull run.

But is 1988-1992 just as similar as 1999-2002 ??

 

[kaveman]

Re: Data

I use premiumdata and find the download service really good. I think if you buy the history CD it contains history back to early-mid 80's for ASX market.
To be honest I do not think working systems backtests over data much older than 1990 would be of much benefit as the market dynamics seems to have changed since then (just my impression)

 

[nizar]

Re: Data

To be honest I do not think working systems backtests over data much older than 1990 would be of much benefit as the market dynamics seems to have changed since then (just my impression)

Agree. Good point.

 

[MichaelD]

Re: Data

Just data.

I like it and was one of those who got in early and bought lifetime supply of 7 bourses for $660.

Now THAT's a bargain - a real rarity in this game.

How far back is data required for backtesting?

I don't think there's a hard and fast answer here - perhaps a statistical answer would be related to your maximum hold time for a position and multiplying that by (eg) 3.

I'm personally happy that my data includes 1987, as that means that I can run my systems through that time period and study how they might have behaved back then.

would the historical data, if i buy it today, does it cover everything up until yesterdaY?

You'll get data up to a certain date on the historical CD and extra downloads for the first month to get the data up to current.

 

[Peakey]

Re: Data

I also use PremiumData for AmiBroker. Very happy.... it's a no brainer, all you do is hit the update button within the PremiumData software and it takes care of everything for you...... It works.... Ties in nicely with AmiBroker.

Peakey

 

[nizar]

Re: Data

To be honest I do not think working systems backtests over data much older than 1990 would be of much benefit as the market dynamics seems to have changed since then (just my impression)

Michael, tech, (and other systems traders), it would be great if i could have your opinion on this comment.

 

[tech/a]

Re: Data

The more data you cah test over the more likley you are to have statistical significance.
There are a zillion ways to develope a system.

If you were a Futures trader you would possibly be only looking at a few instruments.Then its about bars available to test. Same really with stocks.
But the case of survivorship then comes into play.In 20 + years many stocks have been delisted and the list you have today only has stocks which are trading now---many which traded back then will just not be on todays boards.
So you have the problem of either finding them or keeping all issues which have been delisted.I have 7 yrs worth.
If your trading a stock which is delisted then that can have a big impact on your bottom line.

The question then becomes how accurate does you system methodology need be.Does it matter.
Well that depends on how your setting up the Money management side of things and a topic within it self.

EG if your trading CFD's at 10:1 leverage (And you can design systems with thissort of leverage) and you are trading say only 5 positions then a delisting could wipe you out.
However if your trading the ASX100 10 positions un leveraged the impact and possibility is far less.
This is a Super long journey and one which really never ends.

The Bible on systems design is
"Trading Systems and methods" by Perry Kaufman.
Mind you you'll learn a lot from David Samborsky's Tradesim Manual.
I print out hard copies as they are easier to reference.

 

[tech/a]

Re: Data

Nizar.

Timeframe is also important.

Longerterm methods particularly those which hold positions for years need as much historical data as possible.

T/Trader was designed with only 8 yrs historical data and is seen by myself as one of its weaknesses,as it is highly likley that market conditions outside those used for testing will one day raise their head.Hence the importance of keeping an eye on the Numbers or "blueprint" returned in testing.

Shorter term methods are more inclined to need less data to find likely parameters or conditions in which you'll be trading,particularly if the method designed is for both long and short.

Very short or day trading would use even less and this is likely to be very short term minute type data.

 

[>Apocalypto<]

Re: Data

Nizar.

Timeframe is also important.

Longerterm methods particularly those which hold positions for years need as much historical data as possible.

T/Trader was designed with only 8 yrs historical data and is seen by myself as one of its weaknesses,as it is highly likley that market conditions outside those used for testing will one day raise their head.Hence the importance of keeping an eye on the Numbers or "blueprint" returned in testing.

Shorter term methods are more inclined to need less data to find likely parameters or conditions in which you'll be trading,particularly if the method designed is for both long and short.

Very short or day trading would use even less and this is likely to be very short term minute type data.

There is much Wisdom in that Tech,

I read Gann used to back test systems up to hundreds of years back.

 

[tech/a]

Re: Data

There is much Wisdom in that Tech,

I read Gann used to back test systems up to hundreds of years back.

Dont think there was a stock market around back in the 1700s if it was "100s of years of back data".
All Ganns testing would have been by hand and all hand calculations as well---not even a calculator in sight.

And we think we have it hard!!

 

[nizar]

Re: Data

The more data you cah test over the more likley you are to have statistical significance.
There are a zillion ways to develope a system.

If you were a Futures trader you would possibly be only looking at a few instruments.Then its about bars available to test. Same really with stocks.
But the case of survivorship then comes into play.In 20 + years many stocks have been delisted and the list you have today only has stocks which are trading now---many which traded back then will just not be on todays boards.
So you have the problem of either finding them or keeping all issues which have been delisted.I have 7 yrs worth.
If your trading a stock which is delisted then that can have a big impact on your bottom line.

The question then becomes how accurate does you system methodology need be.Does it matter.
Well that depends on how your setting up the Money management side of things and a topic within it self.

EG if your trading CFD's at 10:1 leverage (And you can design systems with thissort of leverage) and you are trading say only 5 positions then a delisting could wipe you out.
However if your trading the ASX100 10 positions un leveraged the impact and possibility is far less.
This is a Super long journey and one which really never ends.

The Bible on systems design is
"Trading Systems and methods" by Perry Kaufman.
Mind you you'll learn a lot from David Samborsky's Tradesim Manual.
I print out hard copies as they are easier to reference.

Thanks for that tech, a very useful post.

Im gonna use premium data which includes all delisted stocks going back until 1992. I think thats essential since its a very real possibility that as you can say will affect your bottom line.

Im interested in trading longer term trends, something like what Stevo has been doing with his weekly system (although obviously it will take several years to get to that level of pro-ness).

Stevo if you are around id be very keen to hear your thoughts on the matter.

 

[tech/a]

Re: Data

Nizar.
For what its worth---and one of the reasons I'm looking into shorter term and some discretionary trading.

The chances of having the sort of performance enjoyed over the last 4 yrs in the near future---are indeed slim.

You maybe better served by including some of the above in your systems developement investigations---just a thought.

 

[nizar]

Re: Data

Nizar.
For what its worth---and one of the reasons I'm looking into shorter term and some discretionary trading.

The chances of having the sort of performance enjoyed over the last 4 yrs in the near future---are indeed slim.

You maybe better served by including some of the above in your systems developement investigations---just a thought.

Tech
Considering you developed techtrader before this bullmarket began, what sort of returns were you expecting after you finished its design?

Im guessing the figures were substantially less than the actual performance - not that thats a bad thing

I will take the above into account of course, thanks for your continuous advice and support - much appreciated.