Metrics for quantifying an intraday strategy?

[Punta]

Hi All,

Another newbie question:

What are some metrics for quantifying an intraday strategy?

E.g. I have been working on an algorithm that in leave-one-year-out back testing

1) triggers a trade on 70% of trading days
2) wins on 75% of the days that it trades
3) averages a price change of 1.2% on the days that it trades (difference between close and open)
4) on the long-side, gives a increase in (un-leveraged) capital of a factor of ~9 in a year
5) produces similar statistics across different exchanges

These stats are blowing my mind, but I'm wondering how to best communicate them.

Are there other, more standard and/or powerful metrics to quantify intraday strategies?

Obviously metric #4 cannot be used for a short strategy, although it is not clear to me how to quantify capital changes with shorting strategies, because you will not actually be buying securities at the low price and selling at the high. I guess you could just pretend that you do...

 

[nomore4s]

Hi All,

Another newbie question:

What are some metrics for quantifying an intraday strategy?

E.g. I have been working on an algorithm that in leave-one-year-out back testing

1) triggers a trade on 70% of trading days
2) wins on 75% of the days that it trades
3) averages a price change of 1.2% on the days that it trades (difference between close and open)
4) on the long-side, gives a increase in (un-leveraged) capital of a factor of ~9 in a year
5) produces similar statistics across different exchanges

These stats are blowing my mind, but I'm wondering how to best communicate them.

Are there other, more standard and/or powerful metrics to quantify intraday strategies?

Obviously metric #4 cannot be used for a short strategy, although it is not clear to me how to quantify capital changes with shorting strategies, because you will not actually be buying securities at the low price and selling at the high. I guess you could just pretend that you do...

My first question would be - what market are you testing it on and where is the data sourced from?

 

[prawn_86]

My first thoughts are what is the maximum draw down and average trade size as a % of portfolio size?

 

[Punta]

My first question would be - what market are you testing it on and where is the data sourced from?

I bought the data from eoddata, and so far have tested NYSE and ASX.

I am getting around to other exchanges, although I am also getting quite excited about putting energy into the implementation side. Or at least talking to people who know these exchanges and can provide some advice on what to do...

 

[Punta]

My first thoughts are what is the maximum draw down and average trade size as a % of portfolio size?

Thanks, these are good/important metrics.

My first thoughts are what is the maximum draw down

By maximum draw down do you mean biggest daily loss? In the leave-one-year-out back testing, the largest daily loss is 8.7%, which came on a day that only one security was bought/sold. I guess this is the draw-down, as all trades are intraday...

average trade size as a % of portfolio size?

I simply split the portfolio evenly between all stocks that are to be bought/sold that day. I guess capital could be apportioned in a more sophisticated way, but this method realises the fractional change in each stock in a "fair" way.

Depending on which exchange it's run on, let's say the algorithm averages 25 buys/sells a day, although the pdf of this number is not normal - e.g. the single buy/sell referenced in the draw down.

 

[prawn_86]

Any chance of posting up an equity curve/graph?

So at all times you are fully invested is that correct?

Where are your stops? IE - whats the maximum loss you would/could be exposed to in a day?

 

[nomore4s]

I bought the data from eoddata, and so far have tested NYSE and ASX.

I am getting around to other exchanges, although I am also getting quite excited about putting energy into the implementation side. Or at least talking to people who know these exchanges and can provide some advice on what to do...

When you say ASX, what actual product is the data from?

My first thoughts are that it won't actually be able to be traded. IE you need to test the SPI futures open and close if you actually want to trade something.

 

[Punta]

Any chance of posting up an equity curve/graph?

See attached - the time is one calendar year (around 250 trading days). This is ASX data, finishing in Oct 2011. The test data plotted were not used in training the algorithm...

So at all times you are fully invested is that correct?

Where are your stops? IE - whats the maximum loss you would/could be exposed to in a day?

Yes, always fully invested on this test.

There are no stops - when the buy/sell is triggered, the entire daily change is taken on the chin, win or lose. I guess I could easily tell the algo to exit the position when the intraday losses exceed X percent, although this would be more difficult to backtest - I would have to use high frequency data, not just the EOD I use at the moment.

It is possible that such a stop-loss would reduce profits, if the intraday curves are very volatile, and a stop-loss causes the algo to exit trades early in the day, that turn out to be profitable over the entire day.

Either way, the returns curve looks outrageously good to me as is...

 

[Punta]

When you say ASX, what actual product is the data from?

My first thoughts are that it won't actually be able to be traded. IE you need to test the SPI futures open and close if you actually want to trade something.

Yeah there's obviously something I'm missing - there's no way that returns like this are realisable.

The actual data I am looking at is the open and close data for each stock. On the long side, the fractional change in my capital is (close-open)/open.

 

[Gringotts Bank]

Are you saying your system returns 900% pa? Make sure your algorithm isn't referencing future data. It looks too good to be true.

No wonder you're concerned about brokers piggybacking.

I'd be attempting to trade it manually for a week (if possible). You'll find out a lot of potential pitfalls that way. Obviously you won't be able to trade more than 5 or so stocks this way, but it would provide very useful feedback.

 

[skyQuake]

Yeah there's obviously something I'm missing - there's no way that returns like this are realisable.

The actual data I am looking at is the open and close data for each stock. On the long side, the fractional change in my capital is (close-open)/open.

Takeee itttt

 

[prawn_86]

There are no stops - when the buy/sell is triggered, the entire daily change is taken on the chin, win or lose. I guess I could easily tell the algo to exit the position when the intraday losses exceed X percent, although this would be more difficult to backtest - I would have to use high frequency data, not just the EOD I use at the moment.

There is the first issue i can note. EOD prices are based on the auction that takes place after 4pm, essentially a clearing auction, so you are not always going to be sell at the final EOD prices. If possible i would backtest it with selling at say 3:45pm instead of EOD and see what result that brings (might be better or worse i have no idea)

So as far as i understand it at the moment:
You enter a stock based on a certain criteria and then only sell at EOD, not at any other point?

What does the equity curve (which is very very nice at this stage) look like if you include the 1 yr you have left out?

 

[Gringotts Bank]

By the way, I've never heard of the term ''leave-one-year-out''.

Does this mean you are training your algo on in-sample data (one years worth), then the running the resulting parameters on the out-of-sample data?

 

[nomore4s]

Yeah there's obviously something I'm missing - there's no way that returns like this are realisable.

The actual data I am looking at is the open and close data for each stock. On the long side, the fractional change in my capital is (close-open)/open.

I'm a bit confused but will try to put this together.

So you are saying your data is of every stocks open & close and the results are from taking a position in every stock on the day in question?

Or is your data of the ASX index open and close?

Either way neither system is able to be traded in the real world.

I would also like to know roughly what your entry trigger is before each days trading as it may turn out that might not be actually trade-able either.

 

[nomore4s]

What does the equity curve (which is very very nice at this stage) look like if you include the 1 yr you have left out?

It doesn't matter what the equity curve looks like if you can't actually trade what you are testing.

 

[prawn_86]

It doesn't matter what the equity curve looks like if you can't actually trade what you are testing.

True

The way i read the first post is that he has back-tested say 10 yrs, but the results are only like this if he leaves a certain year out (im guessing 2008).

If i am reading it correctly, how does Punta know what years to enter and what to sit out? Hence why i want to see the equity curve to prove my suspicions

 

[Punta]

There is the first issue i can note. EOD prices are based on the auction that takes place after 4pm, essentially a clearing auction, so you are not always going to be sell at the final EOD prices. If possible i would backtest it with selling at say 3:45pm instead of EOD and see what result that brings (might be better or worse i have no idea)

Yeah I was thinking that. I realise (through talking to people here) that I can get filled at open, but not at close. I have had a brief look at prices at 3.45 versus close, and didn't find any systematic differences.

What does the equity curve (which is very very nice at this stage) look like if you include the 1 yr you have left out?

I don't quite know what you mean here - I don't want to include the "test year" in the training time - the fact that a technical analysis of historical data works on the data it is trained on is no guarantee that it will work when you give it new data.

 

[Punta]

By the way, I've never heard of the term ''leave-one-year-out''.

Does this mean you are training your algo on in-sample data (one years worth), then the running the resulting parameters on the out-of-sample data?

Yeah basically - I'm training the algo on a few years of historical data, and then testing it on the most recent year of available data.

It's a common technique in statistical weather/climate modelling. You wouldn't necessarily have to leave out the most recent year for testing, although in the markets, where patterns change with time, it is probably wisest to choose the most recent as your test year, and older data for training...

 

[Wysiwyg]

Metrics for quantifying an intraday strategy?

Include brokerage for each trade.
Test stocks that are liquid so you can actually buy & sell.
Be realistic with yourself.

 

[Punta]

So you are saying your data is of every stocks open & close and the results are from taking a position in every stock on the day in question?

Or is your data of the ASX index open and close?

Either way neither system is able to be traded in the real world.

I would also like to know roughly what your entry trigger is before each days trading as it may turn out that might not be actually trade-able either.

The EOD data is for every stock's open and close. I take positions in a small fraction of these, based on criteria over the preceding days.

The entry trigger is okay, insofar as that it can be known before the decision to buy/short needs to be made.