Software for Fundamental data backtesting - your opinion is needed

[KnowThePast]

As some of you may know, I've been writing some software to backtest on fundamental data among other things. A few people on and off the forum have expressed interest in me making it publicly available. I would be very greatful if everyone could voice their opinion to help me decide whether it is worth it to proceed with the effort.

I have previously wrote with some examples of what my software does here (post 200 and 201):
https://www.aussiestockforums.com/forums/showthread.php?t=26890&page=10

and here (post 3):
https://www.aussiestockforums.com/forums/showthread.php?t=29175

There's a simple demonstration of it on youtube:
https://www.youtube.com/watch?feature=player_embedded&v=a85bD6iXBVc

It is primarily based on fundamental data and investing, there isn't much for a price/momentum based investor other than the basics.


The plan is to create a website that allows you to do things such as:

- See detailed financial information about all ASX stocks.

- Backtest based on this data. For instance, buy when PE<7 AND PB < 1; Sell when PE>15. This will be very flexible and people will be able to build their own criteria. In addition to specifying backtest criteria for buy and sell, it will be possible to also control many portfolio management strategies, such as stop losses, sell losers, position sizing, cash interest, averaging up/down, selling when doubles, etc.

As an example, I can run a test for the last 10 years, with the following configurable parameters:
Buy if EV/EBIT is less than 5, Price/Sales less than 0.3 and Price/Book less than 1.0.
Sell when EV/EBIT is greater than 15.
Sell if in red after 3 years.
Portfolio size $50,000. Position size: 4%.
Evaluate monthly.
Brokerage: $30
Buy more if price drops 30%, but only if the original Buy criteria still applies.
Sell a third if price doubles.
4% interest rate on cash balances.
Includes dividends.

The software than produces a list of all trades that would have been triggered over the last 10 years, and calculates total return, hit rate and other statistics.

- Advanced filters/search. The same way you can build complex criteria for backtests, you'll be able to create criteria to search current stocks for ones that make the cut.

- Advanced charting. It can chart any of the company's financial data. It will be possible to aggregate into one chart the numbers from one/all/or selected companies. For example, it will be possibly to chart revenue and profit growth of all companies in one industry in comparison to their market cap.

While I have all this fully working on my computer, it will take some effort to build a website that hooks into the existing engine and presents things nicely. So I would really appreciate if people could give me some feedback on this, either as a reply to this thread or a private message.

1. How interested would you be in this kind of information/service?
a. Useless.
b. I would look at it out of curiosity.
c. I think this could help my investing/trading.
d. This is exactly what I've been looking for.
e. I think it is useful, but only for large funds.

2. What do you think is a fair price to pay for this service?
a. Zero
b. $20/month
c. $100/month
d. $500/month

3. How much would you pay for this service?
a. Zero
b. $20/month
c. $100/month
d. $500/month

4. The fundamental data I have is entered by me, not sourced from Reuters or other agencies. There will be a disclaimer there that no responsibility will be taken if any of it is incorrect. Will this impact your decision on whether to subscribe or not.
a. Not at all.
b. Maybe
c. I will not be buying unless the data comes from a verified data source.
d. Unsure.

5. If the service was not a website, but an application that would need to be downloaded and installed only on PC, would that influence your decision to buy?

Any other comments or suggestions?


Thank you very much for taking the time to read and hopefully respond.

To encourage participation, I will randomly select 10 respondents and give away a month of free service once it is ready.

KTP


P.S. Joe, I hope it is appropriate to post something like this, please remove it if not.

 

[galumay]

1. How interested would you be in this kind of information/service?
a. Useless.
b. I would look at it out of curiosity.
c. I think this could help my investing/trading.
d. This is exactly what I've been looking for.
e. I think it is useful, but only for large funds.

I admire the amount of effort you are putting into your software, its not for me because I dont believe back testing adds anything to fundamental analysis. (and please, dont others drag this off topic into a debate about this point.)

I am sure that for those that were interested in such an approach a subscription would be an acceptable way to charge.

If you are not going to use a web based service then i think that any application should be system agnostic, the days of just writing content for Windows based systems is long gone, you simply cant afford to ignore a significant and growing sector of the market that is using OSX or other OS's.

 

[KnowThePast]

I admire the amount of effort you are putting into your software, its not for me because I dont believe back testing adds anything to fundamental analysis. (and please, dont others drag this off topic into a debate about this point.)

I am sure that for those that were interested in such an approach a subscription would be an acceptable way to charge.

If you are not going to use a web based service then i think that any application should be system agnostic, the days of just writing content for Windows based systems is long gone, you simply cant afford to ignore a significant and growing sector of the market that is using OSX or other OS's.

Thanks for the feedback.

I definitely do agree that it will ideally support all platforms.

But as I already have everything working on PC, I could release that first without doing as much work.

 

[g-topa-g]

Thanks for the feedback.

I definitely do agree that it will ideally support all platforms.

But as I already have everything working on PC, I could release that first without doing as much work.

I had a look at the you tube video...no audio and video quality not clear.

From what I could see it looks like you have put a great deal of effort into it so congrats for that. When it comes to backtesting I don't think you will find to many supporters in this forum from my experience thus far. However I am a supporter and wish you all the best with it. Perhaps you can provide some audio and better video quality. Let me know when you do and I'll happily have another look at it.

All the best!

 

[ThirtysixD]

1. How interested would you be in this kind of information/service?
d. This is exactly what I've been looking for.

There is a gap in the market and ASX is under-serviced

2. What do you think is a fair price to pay for this service?
b. $20/month
It all depends on how much detail you go into.

3. How much would you pay for this service?
b. $20/month
Would need to be free at first (or limited)

4. The fundamental data I have is entered by me, not sourced from Reuters or other agencies. There will be a disclaimer there that no responsibility will be taken if any of it is incorrect. Will this impact your decision on whether to subscribe or not.
d. Unsure.
Would have to see data first (clarify accuracy)
I am assuming you are not doing this by hand and would be scraping it from somewhere?

 

[Richard Dale]

Historical fundamental data is very hard to do on a normalised basis. At Norgate, I call them "funnymentals" because of the can-of-worms that occurs when you try and do a company-to-company comparison

How do you handle:
Change of domicile
Survivorship bias
Conversion to CDI (typically with different multiplier)
Backdoor listings
Major change of business activity (eg. miner - dot com - miner etc.)
Change in structure (eg. stapling/destapling, alternative class of shares issued etc)
Reporting currency different from trading currency (eg. sales reported in USD but stock priced in AUD)
Change of fiscal end-of-year boundaries
Restating of accounts
Staggered reporting of accounts (this is mainly a US thing where headline figures are announced then detailed report comes later)
Mergers
Demergers
Renounceable issues of stock (with participation available to stockholders)
Placements of stock (with participation unavailable to stockholders)
How do you handle extraordinary items
How do you handle multiple classes of shares, with possibly different ownership/voting rights?

Lastly, it is my belief based upon others that have produced software that provides buy or sell signals (even if partially user-defined) needs an Australian Financial Services licence, no matter how many disclaimers you put in there.

 

[tech/a]

Extensive list Richard.

Just to add.
How do you get past history for back testing.

How do you know that this will create an edge.

There are buildings with floors of analysts doing this by hand all day everyday.
Few of their bosses achieve better than the index.

 

[KnowThePast]

1. How interested would you be in this kind of information/service?
d. This is exactly what I've been looking for.

There is a gap in the market and ASX is under-serviced

2. What do you think is a fair price to pay for this service?
b. $20/month
It all depends on how much detail you go into.

3. How much would you pay for this service?
b. $20/month
Would need to be free at first (or limited)

4. The fundamental data I have is entered by me, not sourced from Reuters or other agencies. There will be a disclaimer there that no responsibility will be taken if any of it is incorrect. Will this impact your decision on whether to subscribe or not.
d. Unsure.
Would have to see data first (clarify accuracy)
I am assuming you are not doing this by hand and would be scraping it from somewhere?

Thanks for the feedback ThirtysixD!

Currently, a lot of the data is scraped, but a lot of it is manually entered. I am still thinking of subscribing to a commercial data feed, but not without someone helping with the funding at this stage. Both can co-exist.

A large part of what I am doing is allowing the user to add their own data to companies, including scripts to calculate custom formulas, and allowing backtesting on that. You can write your own valuation formula, then see how it would have performed. So, a custom data format is needed to accomodate that regardless of where the initial data comes from.

 

[KnowThePast]

Historical fundamental data is very hard to do on a normalised basis. At Norgate, I call them "funnymentals" because of the can-of-worms that occurs when you try and do a company-to-company comparison

How do you handle:
Change of domicile
Survivorship bias
Conversion to CDI (typically with different multiplier)
Backdoor listings
Major change of business activity (eg. miner - dot com - miner etc.)
Change in structure (eg. stapling/destapling, alternative class of shares issued etc)
Reporting currency different from trading currency (eg. sales reported in USD but stock priced in AUD)
Change of fiscal end-of-year boundaries
Restating of accounts
Staggered reporting of accounts (this is mainly a US thing where headline figures are announced then detailed report comes later)
Mergers
Demergers
Renounceable issues of stock (with participation available to stockholders)
Placements of stock (with participation unavailable to stockholders)
How do you handle extraordinary items
How do you handle multiple classes of shares, with possibly different ownership/voting rights?

Lastly, it is my belief based upon others that have produced software that provides buy or sell signals (even if partially user-defined) needs an Australian Financial Services licence, no matter how many disclaimers you put in there.

Hi Richard,

That's an excellent, extensive list. Some of these are issues that can be worked around with proper data, but it's not always available and often hard to implement. The majority are a genuine shortcomings of backtesting that one needs to be aware of.

As you rightly pointed out, an automated company to company comparison is just about impossible. However, when evaluating a large group over many years, it is a different story. An allowance needs to be made for them, of course. My backtester allows selection of companies to include/exclude in the backtest, so ones with data issues can be disabled to see whether it makes a substantial difference to the result.

My data is only for Australia and only for revenue generating companies, which removes more than half of ASX listed companies and those tend to be the ones most likely to have these issues.

A backtest is not a black box which accepts some buy/sell criteria and spits out a strategy, at least it shouldn't be. It is another analysis tool for an investor. You start off with a theory, try different criteria, then drill into the results to see if you can spot any oddities or opportunities. It's more about exploration of opportunities then hardcoding a strategy.

Some backtests will be more effected by data issues then others. PE/PB strategies will be less effected by mergers/acquisitions/inst share issues factors then a revenue growth strategy.

An excellent point about license. I am not planning to make any recommendation, only provide historical results, but it is definitely one of the things I will be investigating if I am going to proceed.

Extensive list Richard.

Just to add.
How do you get past history for back testing.

How do you know that this will create an edge.

There are buildings with floors of analysts doing this by hand all day everyday.
Few of their bosses achieve better than the index.

Hi tech,

An extensive topic that I won't get into here fully because I am not sure it is relevant here.

How do you know that anything provides an edge until you practice it for many years?

A backtest is not a silver bullet that is guaranteed to produce a winning strategy. It is yet another analysis tool, the value of it is as much to do with the practitioner as the tool itself.

Does charting software or DCF spreadsheet create an edge? No, but it doesn't make them useless.

My tool started off with me wanting answers to questions like these, in an Australian market:
- do value strategies still work?
- is there a size/liquidity premium?
- is size/stability/low debt truly a safer investment strategy?
- are there any unexploited niches worth investigating?

There are books about it, lots of overseas studies, but few Australian ones. I found it to be much more educational to see the results for myself.

Furthermore, this doesn't need to lead to a purely automated, quant strategy. It can be just a way to find areas of the market where one applies his individual skills, or areas one avoids.