This is a mobile optimized page that loads fast, if you want to load the real page, click this text.

Amibroker System Backtesting

Jump to new
the only point I can add and maybe of value is:
do you truly believe the market react in any way nowadays as it did 10 or 20y ago, before heavy internet trading qants etc;
Out of this then: be careful of pre 2010/2015 backtests/ optimisation;
Just my thoughts :thanks to the latest crash we have a variety of recent periods to use and check system behaviours with a market more reasonably close to what we can expect in the near future
 

A question to those forum members interested in system design.

I am just chasing your thoughts concerning the legitimacy of coding a stop, for back-testing purposes, which intentionally references a future value?
Whilst this question could relate to any software (as in not only Amibroker) that may be applicable to the task I have included it here as Trav has referenced "back-testing trade ideas".

Cheers,
Rob
 

So I have settled on the following setup which will hopefully reduce the human error in having incorrect settings applied or at at least inconsistent setting.

In the made code I have inserted the following line ( you will have to replace the **** with your folder structure)

#include_once "C:\*****\*****\*****\Formulas\Include\BackTest_Settings.afl"
​
I also created a new AFL file and stored it in the following folder with the file name as shown

\Formulas\Include\BackTest_Settings.afl​

That AFL file contains the above quoted settings and whatever you want to remain consistent across all your systems when you include the backtest settings file which does open up a few interesting ideas.
 
Willzy said:
2018 - 2020 could be a bit narrow, do you get similar results say 2000 - 2018?
Click to expand...

@Willzy I hadn't forgotten your suggestion but I am having trouble with the $XVI data as my database for the $XVI starts at 2008.

I have the 20 year package and all the other data comes in fine, so I am just investigating this issues.

Cheers
 
Hey Trav, Did you have some info somewhere on the different regimes you're testing? Think i've missed something...
 
@Roller_1 yeah sorry mate, I started this discussion in the Amibroker Tips and Trick thread which I managed to derail a bit and got this new thread created so I can understand the confusion.

https://www.aussiestockforums.com/posts/1076776/

The link should take you to the start of the discussion about regime filters which should hopefully link everything together for you.

Cheers
 
Link below is for the Amibroker guide that briefly explains how to read the backtest report.

https://www.amibroker.com/guide/h_report.html

I also like this point as previously I was unsure about the colouring as I normally just had green

Some of the metrics in the backtest report are color-coded. Blue means "neutral", Green means "good", Red means "bad". Metrics that are not colorized are always black.
This color coding is of course arbitrary and should be used as guideance only. Treat 'red' as a warning flag and advice to check the value in detail.

As of now the following metrics are colorized:
Net Profit, Net Profit % - bad < 0, good > 0
Annual Profit %, bad < 0, neutral betwen 0 and 10, good > 10
RAR % bad < 0, good > (10 / Exposure)
Avg. Profit/Loss all trades (Expectancy $) - bad < 0, good > 0
Avg Profit/Loss % all trades (Expectancy %) - bad < 0, good > 0
Max. system % drawdown - bad: dd worse than -30%, neutral: dd between -30 and -10%, good - -10% to 0%
CAR/MaxDD, RAR/MaxDD - bad < 1, neutral between 1 and 2, good > 2
Recovery factor - bad < 1, neutral between 1 and 2, good > 2
Payoff ratio - bad < 1, neutral between 1 and 2, good > 2​
 
For those testing systems on a weekly periodicity and trade delays set to 1

As per below post from the AmiBroker forum for a trade on Monday (next daily bar) do you run backtest in Daily periodicity and use TimeFrameCompress ??

This is something that I didn't know about in a weekly system. I have used TrimeFrameCompress for bringing in Foreign data for a filter but I would have thought that AmiBroker backtest would have taken in account that most people would have traded on the first opportunity being the Monday (next daily bar). I will need to test and see what this does to the performance ( good or bad ?)

https://forum.amibroker.com/t/periodicity-weekly-cbi/2783

 
Just a quick test run below.

Same code used in the body of each system tested but I have included the following lines in the test system that we are trying to be a weekly system run on the analysis settings of Daily Periodicity

TimeFrameSet(inWeekly);
//*******CODE
TimeFrameRestore ();
TimeFrameExpand( C, inWeekly );

Results with trade delays set to 1

Normal code run in backtest with the periodicity set WEEKLY..........Modified code with the TimeFrameSet code added and run on DAILY periodicity



So we can see a couple of lines of code can dramatically change your backtest results, in this cause negatively.

Does anyone have experience in this?

Did I run the test correctly or use the code incorrectly?

Happy to hear your thoughts or even see what differences you have come across.

Cheers
 
not sure I interpret/misinterpret properly: weekly system, trade delay = 1, price =open
a buy in BT will still buy at the open price on the Monday following Friday EOW..
That was my assumption, is it otherwise?
that assumption was really matching real action when I was following my systems week by week against backtest
 
look at the number of trades Trav looks like the code is doing something completely different in both tests
 
Trav. said:
I have found a paper from Howard Bandy (2012) who goes into good detail about walk forward testing and many other things

Developing Robust Trading Systems, with Implications for Position Sizing and System Health
Click to expand...

Brokers know your positions. I don't like it.
 

Very interesting @Trav. I'm curious, have you looked at the monte results for some of those system metrics? I love to know what the spread or std dev is on trade and system drawdown across the position sizes.
 
@MovingAverage no mate I have not looked at the Monte Carlo results as that is another thing that I need to get my head around.

When I get time next week I will re-run some backtests and get some data for you so we can look at the differences (good or bad)
 
When running backtest the ApplyStop function can trigger a number of stops as per post #18 above

I found this bit of code on the AmiBroker forum that plots the exit type and value on the chart which is ideal for reviewing your backtest results to ensure that the system is doing what you expect it to do. Again the AFL code for the applystop doesn't need to be checked but the way we use it can be the problem ( for example - exit intraday or next bar)

Code: 
Equity( 1 );

bi = Barindex();
fvb = FirstVisiblevalue( bi );
lvb = LastVisiblevalue( bi );
dist = 1.5*ATR(10);
sellorcovertrigger = "Regular,Stop,Profit,Trail,N-bars,Ruin"; 

for( i = fvb; i <= lvb; i++ )
{
    if( Buy[i] )
        PlotText( "Buy\n" + BuyPrice[i], i, BuyPrice[i] - dist[i], colorGreen, -1, 0 );

    if( Sell[i] )
        PlotText( StrExtract(sellorcovertrigger, Sell[i]-1) + "\n" + SellPrice[i], i, SellPrice[i] + dist[i], colorRed, -1, 0 );

    if( Short[i] )
        PlotText( "Short\n" + ShortPrice[i], i, ShortPrice[i] + dist[i], colorRed, -1, 0 );

    if( Cover[i] )
        PlotText( StrExtract(sellorcovertrigger, Cover[i]-1) + "\n" + CoverPrice[i], i, CoverPrice[i] + dist[i], colorGreen, -1, 0 );
}
 
Hi All,

I am new and have recently started using Amibroker. I have some questions about system backtest and reading the backtest report. My apologies, if this thread is not the right place for the following questions.

Q1 - I am wondering which periods do you guys use in your backtest? Currently I am building my own trading system and I have been backtesting my system twice, firstly from 1/1/2008 to 31/12/2011, and secondly 1/1/2018 to 24/06/2020.

Q2 - If possible can you guys share about which items do you focus on when reading your backtest report (items such as Annual returns, Avg W/L, MDD and etc).

Cheers,
Antiwarclan
 
Welcome to the forum @Antiwarclan

I am sure everyone has there favourite way to test as there are so many options I don't think that there is a right or wrong way.

Are we talking Weekly or Daily ?

If Daily I personally focus on the last 2 years of data for the initial review of the system the run the Walk Forward option with default settings to see how it performs in the previous years with out of sample date.

For weekly you will probably want to look at an a bigger initial time frame before the walk forward run.

Post # 20 here has a pdf that may help you interpret the backtest report and what metrics to focus on, as it is not just about the profit that the system has.

How many losses in a row can you handle?
Maximum trade draw down or system draw down levels are really dependent on you.
Avg. Bars Held is something that I look at being short term trader but if weekly does this matter?

So unfortunately the above is not much help but if you have something specific to ask about back testing then fire away as hopefully someone might be able to help.

Cheers

Trav
 

Q1: Personally, I don't focus first and foremost on a specific date range. I'm more concerned about backtesting with a statistically relevant number of trades. For example, testing over a five year period for a system that takes 5000 trades during that time is probably ok. But if you test a different system over the same period of time and it only takes 150 trades then that 5 year period is not sufficient. I also consider testing over different market conditions: trending up, sideways and tending down.

Q2: Personally, the parameters I focus most on are: system draw down; trade draw down; and consecutive wins and losers. These are the ones that mess with my head the most when I have real money in the market. And of course CAGR is the obvious one.

Good luck
 
@Trav. Thank you for the reply. Yes, I have downloaded the pdf document and reading it now.

@MovingAverage Thank you for your reply too. you were referring to "backtesting with a statistically relevant number of trades", i suppose you mean that it depends on the type of system and the rules. In that case, if the number of trades less than the relevant number or expected number then the sample period may not be sufficient. Am i interpretating it correctly?
 
You must log in or register to reply here.

Similar threads

AmiBroker - CBT Help
Replies
2
Views
1K
Trav.
System optimisation: IS training vs. OOS testing, and Paper Trading
Replies
10
Views
2K
Garpal Gumnut
XAO backtest benchmarks for 2018-2020
2
Replies
37
Views
7K
Warr87
W
My New Monthly System
2 3 4
Replies
79
Views
21K
jon the pom
XJO Trading System
Replies
7
Views
6K
volatility
Menu
Log in
Register
Cookies are required to use this site. You must accept them to continue using the site. Learn more...