Dump it Here

MovingAverage · 24 June 2022

Happy Friday Y'All,

Well I bang on about making sure you use a statistically relevant number of trades before placing any faith in your simulation results--not enough trades and your results are worthless and too many trades can be a waste of CPU cycles, your time and unlikely to give you any further insight.

I was recently doing an analysis of a system and was reviewing some simulation charts and thought the below chart highlighted the importance of understanding what the relevant number of trades for your system simulations is. What you see below is the winning percentage of trades over time and I'm just using this as a simple example. This chart was generated using TradeSim and while I would like the X axis to show trade count and not date unfortunately TradeSim only allows me to show date. Nonetheless this is not a big deal as I explain below.

The important thing to note with the chart below is the date range of June to Oct 2018. Prior to this date the simulation has executed less than about 300 trades and obviously after that date the system has executed more than 300 trades. You can see that up to the 300 trade mark the winning trade % goes up than then slowly down to what is referred to as the "system steady state" (300 trade mark around June / Oct 2018) where is settles at around the 58% winning trade level.

What's the big deal I hear you say--well simulating this system with less then approximately 300 trades is a complete and utter waste of time and will only lead to misleading simulation results. Simulating this system with 1000 trades will be a complete waste of time and unlikely to yield any additional confidence in your simulation results. Personally, I think simulating this system with around 500 trades would be adequate to give you a reasonable level of confidence that the system has reached steady state and the simulation results are reasonably insightful.

Please do not make the mistake of saying using only 500 trades would not span different market conditions and therefor you must use 20 years of data--wrong! This is where walk-forward testing on OOS data is important so you can understand how your system performs across all market conditions and how robust (or not, as the case may be) your system is and what market conditions it performs poorly in.

Stay classy ASF.

Skate · 24 June 2022

MovingAverage said:
before placing any faith in your simulation results

@MovingAverage I've cherry-picked 8 words out of your post (which was great by the way) to explain an anomaly that I find hard to accept. Let me explain, a trend trading system can produce a variety of results depending on filters & parameters used in your buy condition. For a second let's assume the backtests results were unfavourable. Now backtest the strategy with the same buy condition coded completely opposite as before. All logic would indicate that reversing the signals that generated a loss would now generate a win, but it's never the case.

Skate.

MovingAverage · 24 June 2022

Skate said:
@MovingAverage I've cherry-picked 8 words out of your post (which was great by the way) to explain an anomaly that I find hard to accept. Let me explain, a trend trading system can produce a variety of results depending on filters & parameters used in your buy condition. For a second let's assume the backtests results were unfavourable. Now backtest the strategy with the same buy condition coded completely opposite as before. All logic would indicate that reversing the signals that generated a loss would now generate a win, but it's never the case.

Skate.

Hi @Skate: sorry, any chance you can elaborate a bit more--not sure I understand the point correctly.

Skate · 24 June 2022

MovingAverage said:
Hi @Skate: sorry, any chance you can elaborate a bit more--not sure I understand the point correctly.

@MovingAverage let me explain further. I'll use a Simple Moving Average to better explain my point. I coded a strategy quickly to demonstrate the point I was trying to make. Nevertheless, if you reverse your buy condition (to one one you just used) meaning, the condition to be completely opposite it doesn't mean your results will be opposite. Both below are poor examples but you will be able to see the reversal of buy conditions being reversed but the results are not reversed. Just try reversing the buy condition of your strategy & see what the difference is.

The two examples
The first example is the backtest of (strategy 3) versus (strategy 3a)
The second example is the backtest of (strategy 3) versus (strategy 4)

Example 3
Buy = C > MA(C,200); Greater Than
Sell= C < MA(C,50);

# Reverse the buy condition

Example 3a
Buy = C < MA(C,200); Less Than
Sell= C < MA(C,50);

The Backtest results (Strategy 3 versus Strategy 3a)
The backtest capture for example 3 produces a profit where you would expect example 3a to display a loss.

Example 4
Buy = C > MA(C,200); Greater Than
Sell= C < MA(C,50);

# Reverse the buy condition with the sell parameters & vice versa

Example 3a
Buy = C > MA(C,50); Greater than
Sell= C < MA(C,200);;

The Backtest results (Strategy 3 versus Strategy 4)
The backtest capture for example 3 produces a profit where you would expect example 4 to display something totally different.

Summary
The coded examples don't display what was intended. I was trying to say if one strategy was a disappointment changing the buy condition to the opposite doesn't display what was expected. Let me think of another post that explains why you need to be able to think on your feet.

Skate.

Skate · 24 June 2022

Skate said:
why you need to be able to think on your feet.

A lost dog strays into a jungle
A lion sees this from a distance and says to himself with caution, “This guy looks edible, never seen his kind before.”

So the lion starts rushing towards the dog with menace
The dog notices and starts to panic but as he’s about to run he sees some bones next to him and gets an idea. He says loudly, “Mmm…that was delicious lion meat!”

The lion abruptly stops and says
“Woah! - this guy must be tougher than he looks, I better leave while I can.”

Over by the tree top, a monkey witnessed everything
Evidently, the monkey realises that he can benefit from this situation by telling the lion and getting something in return. So the monkey proceeds to tell the lion what really happened and the lion says angrily, “Get on my back, we’ll go get him together.”

So they start rushing back to the dog
The dog sees them and realized what happened and starts to panic even more. He then gets another idea and shouts, “Where the hell is that monkey?! - I told him to bring me another lion an hour ago!!”

Skate.

MovingAverage · 24 June 2022

Skate said:
if you reverse your buy condition (to one one you just used) meaning, the condition to be completely opposite it doesn't mean your results will be opposite.

I now understand your point, but to be honest I'm still missing the connection between my last post on appropriate trade count for system testing and using an opposite buy condition for entries--they call me an idiot for good reasons

Back to your point--I wouldn't expect that using a completely opposite entry condition would necessarily lead to a completely opposite result. If I reversed my entry conditions it might do well for shorting

Skate · 24 June 2022

MovingAverage said:
I now understand your point, but to be honest I'm still missing the connection between my last post on appropriate trade count for system testing and using an opposite buy condition for entries--they call me an idiot for good reasons

Back to your point--I wouldn't expect that using a completely opposite entry condition would necessarily lead to a completely opposite result. If I reversed my entry conditions it might do well for shorting

@MovingAverage, your post invoked a memory for me (that wasn't directly related to your post ) that allowed me to make a post out of eight words that you had used. Explaining what I have found doing a backtest was for those who like to play (fiddle) with their code looking for improvements. Meaning if this buy condition didn't work, doing exactly the opposite should, which is not the case.

I was trying to think on my feet when you said "please explain"
Well, as I was going from memory I couldn't code up something quickly enough that would explain what I was trying to get across. All it did was to "invoke another memory" about why it's so important to think quickly on your feet.

Mike Tyson once said
"Everyone has a plan till they get punched in the face". With boxing, if you don't think quickly on your feet you will be knocked off them.

Thinking on your feet
After making my post, it Invoked another memory of the joke about thinking quickly to get yourself out of a sticky situation. Posts are light on these days in the "Dump it here" thread so posting about "this & that" at least gives others something to read, keeping the thread active.

Skate.

Skate · 24 June 2022

MovingAverage said:
If I reversed my entry conditions it might do well for shorting

There is the point I was trying to make
If the buy condition doesn't work one way it must work the other way, which is certainly not the case. I know your comment was "a facetious remark" making light humour that shouldn't be taken seriously.

Skate.

MovingAverage · 24 June 2022

Skate said:
@MovingAverage, your post invoked a memory for me (that wasn't directly related to your post ) that allowed me to make a post out of eight words that you had used. Explaining what I have found doing a backtest was for those who like to play (fiddle) with their code looking for improvements. Meaning if this buy condition didn't work, doing exactly the opposite should, which is not the case.

I was trying to think on my feet when you said "please explain"
Well, as I was going from memory I couldn't code up something quickly enough that would explain what I was trying to get across. All it did was to "invoke another memory" about why it's so important to think quickly on your feet.

Mike Tyson once said
"Everyone has a plan till they get punched in the face". With boxing, if you don't think quickly on your feet you will be knocked off them.

Thinking on your feet
After making my post, it Invoked another memory of the joke about thinking quickly to get yourself out of a sticky situation. Posts are light on these days in the "Dump it here" thread so posting about "this & that" at least gives others something to read, keeping the thread active.

Skate.

Oh...got it. Was thinking what the hell am I missing in your post--was scratching my heading wondering what on earth a lion, a monkey and a dog could teach me about sample size

DaveTrade · 24 June 2022

MovingAverage said:
Happy Friday Y'All,

Well I bang on about making sure you use a statistically relevant number of trades before placing any faith in your simulation results--not enough trades and your results are worthless and too many trades can be a waste of CPU cycles, your time and unlikely to give you any further insight.

I was recently doing an analysis of a system and was reviewing some simulation charts and thought the below chart highlighted the importance of understanding what the relevant number of trades for your system simulations is. What you see below is the winning percentage of trades over time and I'm just using this as a simple example. This chart was generated using TradeSim and while I would like the X axis to show trade count and not date unfortunately TradeSim only allows me to show date. Nonetheless this is not a big deal as I explain below.

The important thing to note with the chart below is the date range of June to Oct 2018. Prior to this date the simulation has executed less than about 300 trades and obviously after that date the system has executed more than 300 trades. You can see that up to the 300 trade mark the winning trade % goes up than then slowly down to what is referred to as the "system steady state" (300 trade mark around June / Oct 2018) where is settles at around the 58% winning trade level.

What's the big deal I hear you say--well simulating this system with less then approximately 300 trades is a complete and utter waste of time and will only lead to misleading simulation results. Simulating this system with 1000 trades will be a complete waste of time and unlikely to yield any additional confidence in your simulation results. Personally, I think simulating this system with around 500 trades would be adequate to give you a reasonable level of confidence that the system has reached steady state and the simulation results are reasonably insightful.

Please do not make the mistake of saying using only 500 trades would not span different market conditions and therefor you must use 20 years of data--wrong! This is where walk-forward testing on OOS data is important so you can understand how your system performs across all market conditions and how robust (or not, as the case may be) your system is and what market conditions it performs poorly in.

Stay classy ASF.

View attachment 143242

An informative post, thanks for giving us the benefit of your work.

MovingAverage · 24 June 2022

DaveTrade said:
An informative post, thanks for giving us the benefit of your work.

All part of the service

I spend a lot of time doing this and don't give it a second thought. Trying to make a concerted effort to stop and think, some of this stuff might make an interesting post for some.

MovingAverage · 24 June 2022

MovingAverage said:
Happy Friday Y'All,

Well I bang on about making sure you use a statistically relevant number of trades before placing any faith in your simulation results--not enough trades and your results are worthless and too many trades can be a waste of CPU cycles, your time and unlikely to give you any further insight.

I was recently doing an analysis of a system and was reviewing some simulation charts and thought the below chart highlighted the importance of understanding what the relevant number of trades for your system simulations is. What you see below is the winning percentage of trades over time and I'm just using this as a simple example. This chart was generated using TradeSim and while I would like the X axis to show trade count and not date unfortunately TradeSim only allows me to show date. Nonetheless this is not a big deal as I explain below.

The important thing to note with the chart below is the date range of June to Oct 2018. Prior to this date the simulation has executed less than about 300 trades and obviously after that date the system has executed more than 300 trades. You can see that up to the 300 trade mark the winning trade % goes up than then slowly down to what is referred to as the "system steady state" (300 trade mark around June / Oct 2018) where is settles at around the 58% winning trade level.

What's the big deal I hear you say--well simulating this system with less then approximately 300 trades is a complete and utter waste of time and will only lead to misleading simulation results. Simulating this system with 1000 trades will be a complete waste of time and unlikely to yield any additional confidence in your simulation results. Personally, I think simulating this system with around 500 trades would be adequate to give you a reasonable level of confidence that the system has reached steady state and the simulation results are reasonably insightful.

Please do not make the mistake of saying using only 500 trades would not span different market conditions and therefor you must use 20 years of data--wrong! This is where walk-forward testing on OOS data is important so you can understand how your system performs across all market conditions and how robust (or not, as the case may be) your system is and what market conditions it performs poorly in.

Stay classy ASF.

View attachment 143242

Worth noting: this is why I don't have the stomach to trade monthly systems despite really wanting to. I've used 20 or so years of historic ASX data and I have never been able to develop a reasonable monthly system that will take enough trades to get to the steady state for my sims.

MovingAverage · 24 June 2022

Any one use an overbought filter for entries on their breakout systems?

I've been reviewing a number of my live trades over the past 18 months for my breakout system. When looking at the charts for those trades I noticed a number of those positions (not all) that closed out at a loss and they entered the position when most common overbought/oversold indicators clearly indicated those entries occurred when the stock was overbought.

Thought I'd have a look at the impact of adding an overbought filter to the entry conditions of a breakout system so run a few sims to get some better insight--the results look promising and definitely worth further review. What you're looking at below is the sim results of a breakout system with and without a breakout entry filter.

Some key initial observations from me about the difference:

- big improvement in net profit with the filter.
- less trades with the filter (much more efficient).
- increase in winning trades with the filter, approaching 60% with the filter which is pretty good for a breakout system.
- reduction in losing trades with the filter.
- if sharpe ratio is your thing (lot of system traders are obsessed with this, but me I'm on the fence about this metric) with the filter exceeds the "good" level of 1 and is a notable improvement on the no filter system.

The second image below is for the AB MC charts. Most notable thing about the MC charts is that without the filter the average is below 0.8M and with the filter the average is above 0.8M. Visually the with filter look tighter so possibility a better SD.

So overall this looks positive and definitely worth further investigation.

It's worth noting that I looked at a range of different overbought/oversold indicators and all seemed to perform better than without a filter, but some overbought/oversold indicators performed better than others so take a look for yourselves as see what you prefer.

Anyway, it is wine o'clock for me so I'll say "over and out" so I can ponder what a lion, dog and monkey can teach me about trading.

Stay classy ASF

Skate · 24 June 2022

MovingAverage said:
Anyway, it is wine o'clock for me so I'll say "over and out" so I can ponder what a lion, dog and monkey can teach me about trading.

@MovingAverage it was a light-hearted way to demonstate why quick-thinking is an art, not a science. When you are placed in a bind there is always an alternative.

It's worth repeating
I've said multiple times "Don't React, RESPOND to every situation". Responding gives you time to think.

Skate.

MovingAverage · 24 June 2022

Skate said:
@MovingAverage it was a light-hearted way to demonstate why quick-thinking is an art, not a science.

Skate.

and my reference was intended only as friendly banter and light-hearted break from the very dry and unfunny topic of system testing--nothing more and nothing less

Skate · 24 June 2022

MovingAverage said:
Any one use an overbought filter for entries on their breakout systems?

No joy for me
I would say most system traders have tried many different varieties using this indicator. As you have started using the over brought side of the indicator I've added to the buy condition. I've not found it to be helpful & most times a hindrance. I should also say I trade many strategies & none use this indicator.

For the Backtest example
The comparison two-year backtest capture from 24/6/2020 to 24/6/2022 using this buy condition (BuyThisMA = C > MA(C,200) with & without using the filter. It's at odd to your findings. The example was using a simple moving average with a combination of filters.

Skate.

MovingAverage · 24 June 2022

Skate said:
No joy for me
I would say most system traders have tried many different varieties using this indicator. As you have started using the over brought side of the indicator I've added to the buy condition. I've not found it to be helpful & most times a hindrance. I should also say I trade many strategies & none use this indicator.

what indicator did you use?

Skate · 24 June 2022

MovingAverage said:
what indicator did you use?

The “Over brought indicator”.

The better question would have been “how did you use the indicator?”.

@MovingAverage how did you use the indicator?

Skate.

MovingAverage · 25 June 2022

Skate said:
The “Over brought indicator”.

The better question would have been “how did you use the indicator?”.

@MovingAverage how did you use the indicator?

Skate.

Not rocket science and kept it simple. Kept all the system buy conditions the same but would only take a buy if the overbought indicator was less than a fixed threshold— below the threshold the stock was not overbought so trade taken above the threshold the stock was overbought so trade rejected.

MovingAverage · 25 June 2022

Skate said:
The “Over brought indicator”.

The better question would have been “how did you use the indicator?”.

@MovingAverage how did you use the indicator?

Skate.

I’m missing the point here—brought bought ?

Dump it Here

MovingAverage

Just a retail hack

Skate

MovingAverage

Just a retail hack

Skate

Skate

MovingAverage

Just a retail hack

Skate

Skate

MovingAverage

Just a retail hack

DaveTrade

MovingAverage

Just a retail hack

MovingAverage

Just a retail hack

MovingAverage

Just a retail hack

Skate

MovingAverage

Just a retail hack

Skate

MovingAverage

Just a retail hack

Skate

MovingAverage

Just a retail hack

MovingAverage

Just a retail hack

Similar threads

Dump it Here

Just a retail hack

Just a retail hack

Just a retail hack

Just a retail hack

Just a retail hack

Just a retail hack

Just a retail hack

Just a retail hack

Just a retail hack

Just a retail hack

Just a retail hack

Similar threads

Privacy & Transparency

Privacy & Transparency