# Metrics for quantifying an intraday strategy?



## Punta (31 October 2011)

Hi All,

Another newbie question:

What are some metrics for quantifying an intraday strategy?

E.g. I have been working on an algorithm that in leave-one-year-out back testing

1) triggers a trade on 70% of trading days 
2) wins on 75% of the days that it trades
3) averages a price change of 1.2% on the days that it trades (difference between close and open)
4) on the long-side, gives a increase in (un-leveraged) capital of a factor of ~9 in a year  
5) produces similar statistics across different exchanges

These stats are blowing my mind, but I'm wondering how to best communicate them.  

Are there other, more standard and/or powerful metrics to quantify intraday strategies?  

Obviously metric #4 cannot be used for a short strategy, although it is not clear to me how to quantify capital changes with shorting strategies, because you will not actually be buying securities at the low price and selling at the high.  I guess you could just pretend that you do...


----------



## nomore4s (31 October 2011)

Punta said:


> Hi All,
> 
> Another newbie question:
> 
> ...




My first question would be - what market are you testing it on and where is the data sourced from?


----------



## prawn_86 (31 October 2011)

My first thoughts are what is the maximum draw down and average trade size as a % of portfolio size?


----------



## Punta (31 October 2011)

nomore4s said:


> My first question would be - what market are you testing it on and where is the data sourced from?




I bought the data from eoddata, and so far have tested NYSE and ASX.  

I am getting around to other exchanges, although I am also getting quite excited about putting energy into the implementation side.  Or at least talking to people who know these exchanges and can provide some advice on what to do...


----------



## Punta (31 October 2011)

prawn_86 said:


> My first thoughts are what is the maximum draw down and average trade size as a % of portfolio size?




Thanks, these are good/important metrics. 



prawn_86 said:


> My first thoughts are what is the maximum draw down




By maximum draw down do you mean biggest daily loss?  In the leave-one-year-out back testing, the largest daily loss is 8.7%, which came on a day that only one security was bought/sold.  I guess this is the draw-down, as all trades are intraday...



prawn_86 said:


> average trade size as a % of portfolio size?




I simply split the portfolio evenly between all stocks that are to be bought/sold that day.  I guess capital could be apportioned in a more sophisticated way, but this method realises the fractional change in each stock in a "fair" way.

Depending on which exchange it's run on, let's say the algorithm averages 25 buys/sells a day, although the pdf of this number is not normal - e.g. the single buy/sell referenced in the draw down.


----------



## prawn_86 (31 October 2011)

Any chance of posting up an equity curve/graph?

So at all times you are fully invested is that correct? 

Where are your stops? IE - whats the maximum loss you would/could be exposed to in a day?


----------



## nomore4s (31 October 2011)

Punta said:


> I bought the data from eoddata, and so far have tested NYSE and ASX.
> 
> I am getting around to other exchanges, although I am also getting quite excited about putting energy into the implementation side.  Or at least talking to people who know these exchanges and can provide some advice on what to do...




When you say ASX, what actual product is the data from?

My first thoughts are that it won't actually be able to be traded. IE you need to test the SPI futures open and close if you actually want to trade something.


----------



## Punta (31 October 2011)

prawn_86 said:


> Any chance of posting up an equity curve/graph?




See attached - the time is one calendar year (around 250 trading days).  This is ASX data, finishing in Oct 2011.  The test data plotted were not used in training the algorithm...






prawn_86 said:


> So at all times you are fully invested is that correct?
> 
> Where are your stops? IE - whats the maximum loss you would/could be exposed to in a day?




Yes, always fully invested on this test.

There are no stops - when the buy/sell is triggered, the entire daily change is taken on the chin, win or lose.  I guess I could easily tell the algo to exit the position when the intraday losses exceed X percent, although this would be more difficult to backtest - I would have to use high frequency data, not just the EOD I use at the moment.  

It is possible that such a stop-loss would reduce profits, if the intraday curves are very volatile, and a stop-loss causes the algo to exit trades early in the day, that turn out to be profitable over the entire day.

Either way, the returns curve looks outrageously good to me as is...


----------



## Punta (31 October 2011)

nomore4s said:


> When you say ASX, what actual product is the data from?
> 
> My first thoughts are that it won't actually be able to be traded. IE you need to test the SPI futures open and close if you actually want to trade something.




Yeah there's obviously something I'm missing - there's no way that returns like this are realisable.

The actual data I am looking at is the open and close data for each stock.  On the long side, the fractional change in my capital is (close-open)/open.


----------



## Gringotts Bank (31 October 2011)

Are you saying your system returns 900% pa?    Make sure your algorithm isn't referencing future data.  It looks too good to be true.

No wonder you're concerned about brokers piggybacking.

I'd be attempting to trade it manually for a week (if possible).  You'll find out a lot of potential pitfalls that way.  Obviously you won't be able to trade more than 5 or so stocks this way, but it would provide very useful feedback.


----------



## skyQuake (31 October 2011)

Punta said:


> Yeah there's obviously something I'm missing - there's no way that returns like this are realisable.
> 
> The actual data I am looking at is the open and close data for each stock.  On the long side, the fractional change in my capital is (close-open)/open.




Takeee itttt


----------



## prawn_86 (31 October 2011)

Punta said:


> There are no stops - when the buy/sell is triggered, the entire daily change is taken on the chin, win or lose.  I guess I could easily tell the algo to exit the position when the intraday losses exceed X percent, although this would be more difficult to backtest - I would have to use high frequency data, not just the EOD I use at the moment.




There is the first issue i can note. EOD prices are based on the auction that takes place after 4pm, essentially a clearing auction, so you are not always going to be sell at the final EOD prices. If possible i would backtest it with selling at say 3:45pm instead of EOD and see what result that brings (might be better or worse i have no idea)

So as far as i understand it at the moment:
You enter a stock based on a certain criteria and then only sell at EOD, not at any other point?

What does the equity curve (which is very very nice at this stage) look like if you include the 1 yr you have left out?


----------



## Gringotts Bank (31 October 2011)

By the way, I've never heard of the term ''leave-one-year-out''.

Does this mean you are training your algo on in-sample data (one years worth), then the running the resulting parameters on the out-of-sample data?


----------



## nomore4s (31 October 2011)

Punta said:


> Yeah there's obviously something I'm missing - there's no way that returns like this are realisable.
> 
> The actual data I am looking at is the open and close data for each stock.  On the long side, the fractional change in my capital is (close-open)/open.




I'm a bit confused but will try to put this together.

So you are saying your data is of every stocks open & close and the results are from taking a position in every stock on the day in question?

Or is your data of the ASX index open and close?

Either way neither system is able to be traded in the real world.

I would also like to know roughly what your entry trigger is before each days trading as it may turn out that might not be actually trade-able either.


----------



## nomore4s (31 October 2011)

prawn_86 said:


> What does the equity curve (which is very very nice at this stage) look like if you include the 1 yr you have left out?




It doesn't matter what the equity curve looks like if you can't actually trade what you are testing.


----------



## prawn_86 (31 October 2011)

nomore4s said:


> It doesn't matter what the equity curve looks like if you can't actually trade what you are testing.




True

The way i read the first post is that he has back-tested say 10 yrs, but the results are only like this if he leaves a certain year out (im guessing 2008). 

If i am reading it correctly, how does Punta know what years to enter and what to sit out? Hence why i want to see the equity curve to prove my suspicions


----------



## Punta (31 October 2011)

prawn_86 said:


> There is the first issue i can note. EOD prices are based on the auction that takes place after 4pm, essentially a clearing auction, so you are not always going to be sell at the final EOD prices. If possible i would backtest it with selling at say 3:45pm instead of EOD and see what result that brings (might be better or worse i have no idea)




Yeah I was thinking that.  I realise (through talking to people here) that I can get filled at open, but not at close.  I have had a brief look at prices at 3.45 versus close, and didn't find any systematic differences.



prawn_86 said:


> What does the equity curve (which is very very nice at this stage) look like if you include the 1 yr you have left out?




I don't quite know what you mean here - I don't want to include the "test year" in the training time - the fact that a technical analysis of historical data works on the data it is trained on is no guarantee that it will work when you give it new data.


----------



## Punta (31 October 2011)

Gringotts Bank said:


> By the way, I've never heard of the term ''leave-one-year-out''.
> 
> Does this mean you are training your algo on in-sample data (one years worth), then the running the resulting parameters on the out-of-sample data?




Yeah basically - I'm training the algo on a few years of historical data, and then testing it on the most recent year of available data.

It's a common technique in statistical weather/climate modelling.  You wouldn't necessarily have to leave out the most recent year for testing, although in the markets, where patterns change with time, it is probably wisest to choose the most recent as your test year, and older data for training...


----------



## Wysiwyg (31 October 2011)

> Metrics for quantifying an intraday strategy?



Include brokerage for each trade.
Test stocks that are liquid so you can actually buy & sell.
Be realistic with yourself.


----------



## Punta (31 October 2011)

nomore4s said:


> So you are saying your data is of every stocks open & close and the results are from taking a position in every stock on the day in question?
> 
> Or is your data of the ASX index open and close?
> 
> ...




The EOD data is for every stock's open and close.  I take positions in a small fraction of these, based on criteria over the preceding days.

The entry trigger is okay, insofar as that it can be known before the decision to buy/short needs to be made.


----------



## Punta (31 October 2011)

prawn_86 said:


> True
> 
> The way i read the first post is that he has back-tested say 10 yrs, but the results are only like this if he leaves a certain year out (im guessing 2008).
> 
> If i am reading it correctly, how does Punta know what years to enter and what to sit out? Hence why i want to see the equity curve to prove my suspicions




Oh no, that's not it - by leave-one-year-out, I simply mean I'm testing the algo on data that it was not trained on.  This period could be at the beginning, middle, or end of the available dataset, but obviously it makes sense to chose the most recent period, if you plan on implementing it into the future.


----------



## Wysiwyg (31 October 2011)

Punta said:


> See attached - the time is one calendar year (around 250 trading days).  This is ASX data, finishing in Oct 2011.  The test data plotted were not used in training the algorithm...



The ASX has been predominantly down daily so that graph is another sign of b.s..


----------



## Punta (31 October 2011)

Wysiwyg said:


> The ASX has been predominantly down daily so that graph is another sign of b.s..




The graph was actually plotted from a "long-only" strategy.


----------



## nomore4s (31 October 2011)

Wysiwyg said:


> The ASX has been predominantly down daily so that graph is another sign of b.s..




Doesn't mean it's B.S if all the down moves happen over night and during the day prices move to fill the overnight gaps - improbable but not impossible.


----------



## Punta (31 October 2011)

nomore4s said:


> Doesn't mean it's B.S if all the down moves happen over night and during the day prices move to fill the overnight gaps - improbable but not impossible.




The strategy is based on indicators over the last few days.  If 60% of the positions you enter are successful, you will make money (assuming no skew in the up/downside), and like I said (and the graph shows), this strategy actually makes money on 75% of the days it trades.

As people have said, maybe the brokerage fees on lots of small trades will kill this (but then again it is possible that fewer larger positions will work equally well).


----------



## Wysiwyg (31 October 2011)

Punta said:


> The graph was actually plotted from a "long-only" strategy.



You're right mate.


----------



## skyQuake (31 October 2011)

Punta said:


> Yeah I was thinking that.  I realise (through talking to people here) that I can get filled at open, but not at close.  I have had a brief look at prices at 3.45 versus close, and didn't find any systematic differences.




Think you have it the other way around. Liquidity at the close is massive. Open is pretty crap. You can definitely use the close price, but treat the open with a grain of salt.

You might want to put in some liquidity filters. Eg, you can only do 10% of a day's volume. So that'll filter out small stocks with large moves ticked on a few shares.


----------

