Dump it Here

[Skate]

As traders, we always have a tendency of fooling ourselves
In hindsight, we all learn how to make up a story of just went wrong & then foolishly believe it. I'm yet to find a trader who can successfully predict what the market is going to do next. But that's ok we don't need to predict what the market will do next to make money.

Trend following
Money can be made by just following the herd by jumping on a trend & riding that trend right till the end. Finding those trends becomes the mission. When others talk about what they believe the market will do next is simply a matter of guesswork & at times this guesswork can come from experience but even so, it holds little value to a system trader. One look at the monthly stock picking competition proves my point.

Skate.

 

[Skate]

The markets are unpredictable & irrational
When you can accept this unpredictability you'll suddenly find sticking with your trading plan is so much easier. Accepting losses will also allow you to let your winning trades run. Fiddling with your strategy by overriding it "believing" what is going to happen next will inevitably "white ant" you every turn. For a position to go up it needs breathing room & trading with a "gut feeling" is really trading in such a way that's unquantifiable.

Chart patterns need rules
Chart patterns are meaningless without a set of rules to take advantage of those patterns. Rules decide when to trade, how much to trade
& where to take profits, or where to crystalise a loss. Once you have your rules in place you need to be able to test them against a sample of historical data, allowing evaluation & validation of those rules.

Skate.

 

[Skate]

The benefits of having a defined set of rules should not be underestimated
The peace of mind @MovingAverage indicated in a previous post is also not to be underestimated. Knowing when to trade, the price to enter & exit will be the key driver in determining how quickly you will improve as a trader. If you don't have rules you can't test the validity of your strategy. It's also prudent for me to mention at this stage "you are trading against the smartest minds around".

Skate.

 

[Skate]

Gut trading
The problem with overriding a trading strategy is that you'll never be sure if it's the rules or the overriding of the rules was the driver of a losing streak. Poor trading can be attributed to poor execution. The more "inexperienced" traders will bend the rules to suit their mood & wonder why live trading fails to replicate their backtest results.

Jumping ship
The last few posts go somewhat in explaining why new traders constantly jump from one strategy to another. Jumping ship is normally an emotional decision rather than simple logic that all strategies take time to develop.

I'll leave the summary up to @peter2
I've made a series of posts when one would have sufficed.

You first goal is to learn as much as you can about the markets you prefer to trade. Your second goal is to formulate a trading plan that will allow you to trade consistently and more importantly to keep your starting capital intact for as long as it takes for you to develop the skills to trade profitably.

Skate.

 

[ducati916]

Morning Mr Skate,

Traders in Aus. should probably have been paying attention to US Small Caps. The good news, obviously, is that the chop seems to have resolved to the upside.



If you are not already, stocks should be trading higher soon.

jog on
duc

 

[Greynomad99]

Gut trading
The problem with overriding a trading strategy is that you'll never be sure if it's the rules or the overriding of the rules was the driver of a losing streak. Poor trading can be attributed to poor execution. The more "inexperienced" traders will bend the rules to suit their mood & wonder why live trading fails to replicate their backtest results.

Jumping ship
The last few posts go somewhat in explaining why new traders constantly jump from one strategy to another. Jumping ship is normally an emotional decision rather than simple logic that all strategies take time to develop.

I'll leave the summary up to @peter2
I've made a series of posts when one would have sufficed.



Skate.

Well said Skate.
For the first few years of trading I was overtrading and the bank was making more in brokerage than I was earning from trading.
Second, I was following the trading text book and baling out when prices broke my stop losses only to find the next day they recovered and yet again I'd sold at the bottom. This when to hold and when to fold is the single hardest thing to get right for short/medium term traders and while I get it right much more often than wrong these days, there isn't any formula I've found that tells me how far to hold past a stop loss. And holding past a stop loss is something I and the text books tell new traders to never, ever do - it is the single most dangerous thing to do as a trader. If you don't adhere strictly to a stop loss then you are not trading, you are gambling. So, yes, a bit of gambling on my part by fudging stop losses - and certainly a case of do as I say, not do as I do for anyone without say 3 - 5 years of high level T/A and active trading under their belt.
What do I do these days? Well if my T/A says price is going to turn up about now (bearing in mind that T/A is only right about 75% of the time at best) and I get to the point of thinking ' that's it, had enough I'll close this one out', I'll hold another day or two (subject to said stock price not acting wierdly). I figure Mr Murphy is waiting in the wings for me to sell and if I decide to, but then don't, I might beat him at his own game! Not great logic, but it seems to work more often than it doesn't.
IFM is one where I got caught on its plummet in price when its CEO walked. Based on T/A I expected price to recover and with little to lose, I held rather than bail (my stop loss was lying bleeding on the floor after the price fall). Price rose but then fell back to retest the lows and actually made a lower low (another signal to get out of town). It has jumped about 5% in the last 2 days and while it lost a bit off its highs yesterday, I'm hoping I've beaten Murphy again. IFM might take a long time to get back to its previous value but I'll be happy to take a small loss rather than a big loss if it proves the recovery is too drawn out for me.
PS: IFM stopped 2 cents shy of where I thought it would thereafter be a truly lost cause, so I won't be suprised if the price yet again tests its lows. So far thise week price has made a candle that looks like one that suggests reversal (as in higher prices after reversing the prior downtrend). Still a work in progess that could see my plans dashed and broken on the jagged rocks at the bottom of the abyss I'm peering over the edge at.

 

[Greynomad99]

Morning Mr Skate,

Traders in Aus. should probably have been paying attention to US Small Caps. The good news, obviously, is that the chop seems to have resolved to the upside.

View attachment 132419

If you are not already, stocks should be trading higher soon.

jog on
duc

I hope you are right. While I don't follow the US market apart from a morning summary of the major indicies (and I don't trade small caps unless I've had too much to drink - and at my age that's something I try to avoid these days!), that parabolic curve suggests to me that price could be rolling over and more falls are ahead. All markets are at all-time highs and while I feel some (like the XAO) may return to their all-time highs in the next few weeks, I can't see sustained new all-time highs - so I am really cautious about potential 'sucker rallies'. However, as traders, while the sun is shining need to see what we can ring out of the currently rising prices (while constantly looking overo ur shoulder).

 

[MovingAverage]

I hope you are right. While I don't follow the US market apart from a morning summary of the major indicies (and I don't trade small caps unless I've had too much to drink - and at my age that's something I try to avoid these days!),

That's why I like trading the ASX--it is closed when I do my drinking

 

[Lone Wolf]

In summary
Amibroker includes the position as a buy (using either padding system) whereas in reality the position was never purchased in the first place. Amibroker doesn't allow adjustments to be made to the Portfolio Manager to accurately display only the physical positions in the portfolio. Using my custom Exploration Analysis solves this dilemma (for me) as those signals can be manipulated.

Hi Skate,

I thought about your issue but I don't have much to add. If it's the backtest validity you're concerned about, the only way I can see of fixing it would be to identify weeks where the first day of the week is a padded day and include that as a forward looking condition in your backtest. I'm sure this could be done, but you already have a similar issue where price opens above your buy limit and runs higher. Your backtest still buys on open when in reality you miss the trade. Last I saw you weren't interested in correcting for this as backtests don't mean jack anyway.

I suspect the percentage of trades that are incorrectly shown in the backtester due to the stock being in a trading halt on Monday would be quite few. For a robust system you can check that the performance isn't bing carried by a very small number of outliers. Or, if by design the system is carried by a small number of outliers, at least manually check that those outliers are valid entries.

I am a big fan of having the backtest match reality wherever possible. But it's worth reminding ourselves occasionally that our objective here is to make money, not write code. Sometimes you need to choose your battles and decide how much extra value you get out of writing more code.

 

[Skate]

If you are not already, stocks should be trading higher soon

All markets are at all-time highs and while I feel some (like the XAO) may return to their all-time highs in the next few weeks

On the same page
@ducati916 & @Greynomad99 respective posts on the first reading seem to be at odds with each other whereas, in reality, they hold the same overall view. Both respective views come from experience. When seasons traders speak it is our job to listen & digest the information being presented. Skim reading without critical thinking could very well lead us down the wrong path into making a poor decision. Why? because our "perception" of the information consumed drives our reaction.

When is it safe to trade, that's the question
Well, that's the dilemma we all face. There is a strong consensus on the "safest time" to trade is when the market is going up. What metric you decide to use is up to you. The most common metric is an Index Filter. A crude "Index Filter" often discussed in this forum is a "Simple Moving Average" of an Index. If the close of the Index is above a "Simple Moving Average" the market is deemed to be up as a "Buy Filter".

Skate.

 

[qldfrog]

Hi Skate,

I thought about your issue but I don't have much to add. If it's the backtest validity you're concerned about, the only way I can see of fixing it would be to identify weeks where the first day of the week is a padded day and include that as a forward looking condition in your backtest. I'm sure this could be done, but you already have a similar issue where price opens above your buy limit and runs higher. Your backtest still buys on open when in reality you miss the trade. Last I saw you weren't interested in correcting for this as backtests don't mean jack anyway.

I suspect the percentage of trades that are incorrectly shown in the backtester due to the stock being in a trading halt on Monday would be quite few. For a robust system you can check that the performance isn't bing carried by a very small number of outliers. Or, if by design the system is carried by a small number of outliers, at least manually check that those outliers are valid entries.

I am a big fan of having the backtest match reality wherever possible. But it's worth reminding ourselves occasionally that our objective here is to make money, not write code. Sometimes you need to choose your battles and decide how much extra value you get out of writing more code.

For green feets like me:
we had a whole set of exchanges about this issue;
my trader bell Direct limits the % above close that I can set in my orders before Open;
And i miss a lot of ,probably the best, entries
One way I do check my strategy is add the following:

//Random entries
rd = Random();
buySignal=IIf (rd>0.2,True,False);
Buy=Buy AND buySignal AND Optimize("Run",1,1,1000,1);

Note: I do not own the idea:thanks to numerous mentors here,


backtests miss a percentage of entries at random(here 20% are missed); similar to what happens when my buy orders are missed:
Perform 1 thousand run if you do an optimise run on AB;
I then take these 1000 runs into excel and do a very crude distribution of these results;
as long as it is heavily concentrated and not too far from my expected single run backtest without random misses; it should be fair to say it will be robust enough;
I do not do the same on sale, but could probably add a reducing factor on sale price
Yes real trade will NOT match the backtest but should be near enough...not always...and you should be able to trace back the missed buy or lower sell price (should always).
A bit of work but worthwhile after a first few ticks of real trades

 

[Skate]

If it's the backtest validity you're concerned about

@Lone Wolf the validity of the backtest reports is not the concern it's more to do with Amibroker Portfolio Manager using the Backtest feature. "Detail Log" in the Analysis Setting generates these buy positions. The issue I have is the signals in the "Detailed Log" report can't be adjusted or modified. Meaning if a position is not taken that's displayed in the "Detail Log" (for whatever reason) can't be corrected.

Amibroker handles your strategy - you handle your trading plan, it's up to you to execute your plan, there is a distinction.

Backtest results "mean jack" (to me)
Backtesting a strategy is straight forward but backtesting a "trading plan" with all its variables becomes another kettle of fish. But in saying this Backtesting a strategy allows you to compare & contrast between conditions, filters & parameters to take the strategy to the next level.

Skate.

 

[Skate]

What's the procedure to get buy & sell signals in Amibroker
There are a few ways, the first is using the Backtest "Detail Log" to generate a report. The other way is to use the "Exploration Analysis" to generate an alternative report. the "Detail Log" is a portfolio manager whereas the exploration analysis simply displays all the raw signals.

Which method should I use?
I prefer using the "exploration analysis" as the raw signal can be manipulated within the strategy to display other trade information. Using the "Detail Log" of a backtest report doesn't allow "user" intervention.

Trade Signals
If you prefer to generate buy & sell using the backtest report the analysis settings should have the "Detail Log" radio button selected. The "Portfolio" analysis setting should have "Add artificial future bar" ticked to allow you to see tomorrow's trades.




This is the report style when the "Detail Log" is selected
The example below displays the backtest signals of the portfolio manager. The date selection is critical to the signals being displayed. As the report is for a 10 position strategy - the report will only show the buy & sell signals to this PositionSize.



Skate.

 

[Skate]

I often get asked: "are my Analysis Settings" correct?
The analysis setting seems easy to understand but at times selecting the correct setting can be a nightmare. Most of these settings can be hardcoded into your strategy where others can't. I'll have attached an afl file to display which settings can & can't be programmed into a strategy.

Boiler Plate
The "Boiler Plate" attached is old but still relevant. All credit must go to "BruceR" for the original afl with additional thanks to @trash for additional comments & options. The purpose of this afl file is to initialize all backtesting/optimization "factors" to a default value. It also serves as a "reference check" to make sure that all of the factors have been accounted for.

These factors fall into 3 categories
1. Factors that are in the Amibroker Analysis Backtest Settings.
2. Factors that are NOT in Settings
3. Factors that are in settings but can NOT be set programmatically

The bottom line
There is no way to guarantee in one place that all factors are accounted for. Theoretically, to cover all options, a combination of the "Analysis Settings" & "AFL Settings" should be used. At times Amibroker users are reluctant to fiddle with the original settings for the fear of messing it up.

But there is an alternative
Select as many settings as possible & code them into your strategy. The afl attached explains the settings that can "only" be set in the analysis settings. The Boiler Plate lists the settings that can & can't be programmed into the strategy.

For those who don't have Amibroker
All afl's can be viewed in Microsoft's "Note Pad" for those who don't have Amibroker. The list below can't be set "programmatically" set & should be carried out manually.

1. Pad and align
2. Reference symbol
3. Risk-free rate Sharpe
4. Risk-free rate UPI
5. Add artificial future bar
6. Limit trade size %
7. Disable trade size limit
8. Walk forward mode and data parameters
9. Optimization target

Skate.

 

[Lone Wolf]

For green feets like me:
we had a whole set of exchanges about this issue;
my trader bell Direct limits the % above close that I can set in my orders before Open;
And i miss a lot of ,probably the best, entries
One way I do check my strategy is add the following:

//Random entries
rd = Random();
buySignal=IIf (rd>0.2,True,False);
Buy=Buy AND buySignal AND Optimize("Run",1,1,1000,1);

Note: I do not own the idea:thanks to numerous mentors here,


backtests miss a percentage of entries at random(here 20% are missed); similar to what happens when my buy orders are missed:

I thought I remember you implementing a buy condition that looks froward a day and rejects the signal if the low on entry day is above the previous close plus %3 (or whatever your limit is). Does this not achieve what you want, or do you dislike that Amibroker will forever complain that your code has a future leak?

A couple of things to consider (which you probably already have, but for anyone newer who might be following along) - You are simulating missing some trades by randomly missing 20% of entry signals. Something to be careful of is that in reality you aren't missing random trades, you are missing a very specific type of trade. The trades that gap up on open and don't look back. It's worth checking how many of your winners start this way.

Another slight difference is that when you miss a trade in reality you are a position short and not fully invested. In your simulation that randomly skips signals you will still get allocated the full number of positions, just not the ones you ideally wanted (assuming you have enough remaining signals that is). I've never used the custom backtester, I have no idea if it's possible to leave positions unfilled if a signal is skipped.

 

[Lone Wolf]

@Lone Wolf the validity of the backtest reports is not the concern it's more to do with Amibroker Portfolio Manager using the Backtest feature. "Detail Log" in the Analysis Setting generates these buy positions. The issue I have is the signals in the "Detailed Log" report can't be adjusted or modified. Meaning if a position is not taken that's displayed in the "Detail Log" (for whatever reason) can't be corrected.

I see, but still can't help. For me backtest is for backtesting only. Explore is for finding signals. I don't make good use of the portfolio manager.

 

[Skate]

I don't make good use of the portfolio manager.

@Lone Wolf there are advantages & disadvantages to using either method to generate signals.

The "advantage" of using a backtest "Detail Log" for signals
Only the signals to be actioned are displayed in the portfolio. Guesswork or "additional checks" are not required.

The "disadvantage" of using a backtest "Detail Log" for signals
Amibroker has full control over which signals make it into the portfolio & fudges (re-calculates) the buy price as well as the number of shares purchased after the market has opened. Once a position is in the portfolio (Detailed Log) it "restricts user intervention" to correct the records. Positions may not have been taken for a variety of reasons being the opening price exceeded my buy price limit (gap ups) or because the position didn't trade on Monday because of a "Trading Halt" or the position fell fowl of the Trading Plan. It should be noted that a trading plan overrides a trading strategy.

Backtest Detail Log (today at 1 pm)
This report displays "only the signals" for inclusion into the 10-position portfolio. At this stage, the share price & the number of shares to buy are sourced from the opening of the previous bar (one week old). The price for the calculations is not reflective of the closing price at 2 pm today.



The "advantage" of using the Exploration Analysis for signals
The Exploration Analysis can produce any display of your choice as your exploration can be coded making metric more "visual" thus making trading easier. The analysis can be colour coded & sorted into columns making the process a breeze.

The "disadvantage" of using the Exploration Analysis for signals (today at 1 pm)
Unfortunately, it displays all raw signals & "user intervention" is required when it comes to selecting only the required number of buy positions to fill the "PositionSize" of your portfolio. Also, the explorational analysis displays all the raw sell signals that need to be manually evaluated in relation to their inclusion in the portfolio. Using the exploration Analysis for signals is similar to being at a cafeteria - it's up to you to pick & choose the signals that are applicable to the holding in your portfolio.



Skate.

 

[Skate]

Moreover
Using the "Exploration Analysis" as @Lone Wolf has indicated allows you to code additional information to make trading decisions much easier. The format of the report is only limited by your imagination. I've captured the same exploration analysis but using updated data at (2 pm).


Report format #1




Report format #2




Report format #3



Skate.

 

[qldfrog]

I thought I remember you implementing a buy condition that looks froward a day and rejects the signal if the low on entry day is above the previous close plus %3 (or whatever your limit is). Does this not achieve what you want, or do you dislike that Amibroker will forever complain that your code has a future leak?

A couple of things to consider (which you probably already have, but for anyone newer who might be following along) - You are simulating missing some trades by randomly missing 20% of entry signals. Something to be careful of is that in reality you aren't missing random trades, you are missing a very specific type of trade. The trades that gap up on open and don't look back. It's worth checking how many of your winners start this way.

Another slight difference is that when you miss a trade in reality you are a position short and not fully invested. In your simulation that randomly skips signals you will still get allocated the full number of positions, just not the ones you ideally wanted (assuming you have enough remaining signals that is). I've never used the custom backtester, I have no idea if it's possible to leave positions unfilled if a signal is skipped.

All true, and i replace these up gaping good trades by some at the end of my ranking
I did try indeed to emulate the reality but gave up as the rejected trades are not following a given rule i can code, so why bother going into a code design war if i will ultimately not be matching anyway:
This rough element of random i introduce is enough for me to label my results as robust enough or not.
The next step is a live vs backtest forensic search to confirm any difference is expected and explained..
Basically you record buy sell as they come on paper daily/weekly and compare vs a backtest after a decent time period
That works for me..meaning i am confident enough to run my systems.

Does not guarantee profit

 

[qldfrog]

@Lone Wolf there are advantages & disadvantages to using either method to generate signals.

The "advantage" of using a backtest "Detail Log" for signals
Only the signals to be actioned are displayed in the portfolio. Guesswork or "additional checks" are not required.

The "disadvantage" of using a backtest "Detail Log" for signals
Amibroker has full control over which signals make it into the portfolio & fudges (re-calculates) the buy price as well as the number of shares purchased after the market has opened. Once a position is in the portfolio (Detailed Log) it "restricts user intervention" to correct the records. Positions may not have been taken for a variety of reasons being the opening price exceeded my buy price limit (gap ups) or because the position didn't trade on Monday because of a "Trading Halt" or the position fell fowl of the Trading Plan. It should be noted that a trading plan overrides a trading strategy.

Backtest Detail Log (today at 1 pm)
This report displays "only the signals" for inclusion into the 10-position portfolio. At this stage, the share price & the number of shares to buy are sourced from the opening of the previous bar (one week old). The price for the calculations is not reflective of the closing price at 2 pm today.

View attachment 132437

The "advantage" of using the Exploration Analysis for signals
The Exploration Analysis can produce any display of your choice as your exploration can be coded making metric more "visual" thus making trading easier. The analysis can be colour coded & sorted into columns making the process a breeze.

The "disadvantage" of using the Exploration Analysis for signals (today at 1 pm)
Unfortunately, it displays all raw signals & "user intervention" is required when it comes to selecting only the required number of buy positions to fill the "PositionSize" of your portfolio. Also, the explorational analysis displays all the raw sell signals that need to be manually evaluated in relation to their inclusion in the portfolio. Using the exploration Analysis for signals is similar to being at a cafeteria - it's up to you to pick & choose the signals that are applicable to the holding in your portfolio.

View attachment 132439

Skate.

As an exploration user, it is true that you can sometimes miss a sell order...
Human error
And that sell instruction will not popup anymore
as a result i do an explore, enter the trade then check graphically that none of the remaining portfolio has a buy attached
Not ideal, time consuming