My current systems - walking forward

[CanOz]

I thought i would make use of my newly reacquired access to ASF and post some results from three or four of my systems that I'm currently testing on the simulator.

I developed these systems with a Genetic Program called Adaptrade Builder. I run them on MultiCharts.
I also have an NT system or two that i may post as well.

There is much discussion on these types of genetic programs and i thought it may be interesting to get some system discussion going again, there is much to learn from many on ASF on this topic.

Since i just returned from a short break i thought the timing seemed right to start the thread.

Fingers crossed that i can continue this thread, i certainly have the time available now...i just hope the access continues.

Cheers,


CanOz

 

[CanOz]

The first results are from a DAX 15 minute system.

This is OOS data from July 2011 to the last contract expiry.

I'm a little worried for the Profit factor, and the W/L ratio. Its testing forward better than the first and second OOS runs.

It has not yet been optimized.

CanOz

 

[Trembling Hand]

Got a scatter chart of trades?

 

[CanOz]

Scatter gram...

 

[Trembling Hand]

I liked it better before I looked at that

 

[CanOz]

I liked it better before I looked at that

Yeah, for having such a wide MM stop, it doesn't produce enough big winners. My worry about the PF and the W/L Ratio.

I can try and optimise the MM stop, or i could try to build in a trailing stop.

OR i could try and filter...but i think the GP should have found the optimum filter already.

CanOz

 

[waza1960]

Thanks for the Thread CanOz
Questions: What time period are you conducting in sample testing ?
What kind of Simulator are you using ?
Do you have any concerns that the Adaptrade approach is isolating you
from the Strategy development at all and if so are you confortable with
this?
Thoughts :Win/Loss ratio and PF is a problem IMO .The metric that I have been giving most weight to for Optimisation and observing in Backtests is avg win/avg loss ratio and then % profitable trades .I have actually written my own metric within Ninja Trader to optimise on which links these two functions.
A more sophisicated stop (Trailing/ATR)would be beneficial I don't use normal stops anymore.

 

[CanOz]

Thanks for the Thread CanOz
Questions: What time period are you conducting in sample testing ?
What kind of Simulator are you using ?
Do you have any concerns that the Adaptrade approach is isolating you
from the Strategy development at all and if so are you confortable with
this?
Thoughts :Win/Loss ratio and PF is a problem IMO .The metric that I have been giving most weight to for Optimisation and observing in Backtests is avg win/avg loss ratio and then % profitable trades .I have actually written my own metric within Ninja Trader to optimise on which links these two functions.
A more sophisicated stop (Trailing/ATR)would be beneficial I don't use normal stops anymore.

Hiya Waza, glad to hear from you...

I also used 15 minutes to develop the strategy over 1 year of data from 2009-2010. Then i ran the top 100 over the next year and chose a strategy that had a consistent metric profile.

I'm comfortable with using the Builder (its allot of work to find a decent algorithm), but still going through the learning curve with Easy Language. I have more ideas than i could possibly test in a lifetime but I'll do my best to get through them!

Love to exchange some ideas sometime.

Cheers,


CanOz

 

[Punta]

Interesting thread. What is a simulator? Does this mean you've been paper trading it for a year? Or is this a way of taking historical data, and "trading" it, so as to try and factor slippage into your testing?

 

[CanOz]

For these systems i am using Interactive Brokers simulator (paper trader) which is almost a direct copy of their TWS. I can even copy the settings from the Sim TWS to the live account once I'm confident all of the settings are working as hoped.

There is no need to paper trade the system for a year, but it needs to be 'walked forward' to ensure it performs similar on new data as it did on past data.

The paper trading just ensures that the automation works ok, the orders get executed properly etc. before going live.

Cheers,


CanOz

 

[Punta]

Yeah, I didn't mean to specify a year, but have you been paper trading that since July 2011? Seems like a long time. If my paper trading is matching up with the backtesting system, I would go live after a much shorter "walking forward" period.

 

[CanOz]

Yeah, I didn't mean to specify a year, but have you been paper trading that since July 2011? Seems like a long time. If my paper trading is matching up with the backtesting system, I would go live after a much shorter "walking forward" period.

These systems were developed in early March, so i've got a few months of testing left to go. I would think i'll be going through the summer, hopefully ready for last couple of quarters of the year.

For me to be confident to trade them i think i'll need at least two months of WF. If they can't convince me after that then i'll start over.

These are my first systems to WF, so its new to me.

Cheers,


CanOz

 

[CanOz]

This my current HSI system. I have optimized the MM Stops, and the ATR stops.

It doesn't win as often, but it wins bigger.

An interesting metric that Sunny Harris (author trader etc.) uses if the PF x W/L Ratio x %profitable and it must be above 1.2. None of my systems exhibit this on WF testing. This one is .993

What i would like is .5 x 1.6 x 1.6 = 1.28 up to .55 x 1.7 x 1.7 = 1.58

CanOz

 

[CanOz]

Lots of issues left to resolve...like this sell stop on the DAX...the algo put up the order well before price even got near it, but for whatever reason it didn't fill, on the simulator

CanOz

 

[tech/a]

Lots of issues left to resolve...like this sell stop on the DAX...the algo put up the order well before price even got near it, but for whatever reason it didn't fill, on the simulator

CanOz

Had that happen on the odd occasion R/T.
If a trade doesn't occur at the exact same price your stop is at you won't get done.
That can and does happen in a quickly moving market.
The DAX is often really quick gapping over prices.

I also have commoners on your systems I'll have more time as the weekend goes on.

 

[waza1960]

Yeah, I didn't mean to specify a year, but have you been paper trading that since July 2011? Seems like a long time. If my paper trading is matching up with the backtesting system, I would go live after a much shorter "walking forward" period.

I tend to agree with this my testing is indicating that shorter time frames (less than 30 min) need to have considerably shorter testing periods or rather testing periods closer to current price action at least with regards to FX anyway .

 

[CanOz]

I tend to agree with this my testing is indicating that shorter time frames (less than 30 min) need to have considerably shorter testing periods or rather testing periods closer to current price action at least with regards to FX anyway .

There is a whole debate around this and i also tend to agree that more recent data tend to reflect the current market behavior...think participants too.

I use 1 year to construct and 1 year as my first OOS period at the moment, but today i will try and construct over shorter time frames with many more generations (using a GP to reverse engineer) and see if i get something that performs closer to the construct period on OOS1 and OOS2.

Mike Bryant from Adaptrade (literally a rocket scientist) is also in favor of frequent optimizations.

Again ,there is a huge raging debate among quants on how much to optimize...i stay out of it.....no place for me to jump in on a bunch of math and physic Phd's duking it out on a forum:headshake

Great topic.


CanOz

 

[CanOz]

Had that happen on the odd occasion R/T.
If a trade doesn't occur at the exact same price your stop is at you won't get done.
That can and does happen in a quickly moving market.
The DAX is often really quick gapping over prices.

I also have commoners on your systems I'll have more time as the weekend goes on.

Would you believe it was my fault. With MultiCharts you need to have the chart loaded with IB data in order for the orders to execute...makes sense...with NinjaTrader you can use eSignal data and still execute through IB, which i am used to....my bad...doh!

Too many platforms...

CanOz

 

[sinner]

Mike Bryant from Adaptrade (literally a rocket scientist) is also in favor of frequent optimizations.

Again ,there is a huge raging debate among quants on how much to optimize...i stay out of it.....no place for me to jump in on a bunch of math and physic Phd's duking it out on a forum

Hi Canoz,

I think this website is an important read for you

http://www.adaptivetradingsystems.com/blog/

The main thing I took away from the blog is that it's much much better to 'swarm' rather than have a single adaptive bot and take their aggregate signal as the actual signal.

The blog finds that trading a single adaptive bot is not a very good way to trade, the most common problem is adapting to any optimal solution too late. However there are other issues which can't be simply overcome.

My testing concurs with these findings, although I admittedly don't use 'regular' methods which most use like genetic algorithms or ANN or similar.

As for your shorter time period for testing/training, my advice is to just stay the F* out of the forex markets for this type of trading, stick to the index futs.

 

[CanOz]

Hi Canoz,

I think this website is an important read for you

http://www.adaptivetradingsystems.com/blog/

The main thing I took away from the blog is that it's much much better to 'swarm' rather than have a single adaptive bot and take their aggregate signal as the actual signal.

The blog finds that trading a single adaptive bot is not a very good way to trade, the most common problem is adapting to any optimal solution too late. However there are other issues which can't be simply overcome.

My testing concurs with these findings, although I admittedly don't use 'regular' methods which most use like genetic algorithms or ANN or similar.

As for your shorter time period for testing/training, my advice is to just stay the F* out of the forex markets for this type of trading, stick to the index futs.

Thanks Sinner, i can't access Blogs. Can you give me an idea of what the discussion is?

There is allot of discussion on Mike Bryant's google group as well here...

Agree on Forex, but I've got other ideas for that, they don't involve using the GP to develop though....Actually the time of day/session is one of the most critical factors in FX trading according to my testing so far...not allot of research done on it yet though...save it for a rainy day.

Again, appreciate the feedback.

CanOz