This is a mobile optimized page that loads fast, if you want to load the real page, click this text.

Dump it Here


If you're using the latest version of Excel it has a great statistics plugin that has a lot of easy to use functions and analysis including histograms and important statistical analysis of those histograms. I have a colleague that uses it to generate some generalized models of his system (from the back test analysis) and then uses those generalized models in conjunction with his live trades to make sure the system is behaving itself. It's a great way to make alert yourself to system behavior that is out of the ordinary. I haven't got that far yet, but it is something I seriously want to do.
 
@Newt Here's a quick and dirty example for you. This assumes you are using the latest version of Excel. Also assumed you know how to export the results out of AB into a CSV (which you do).

Referring to the pic below, all you do is open the CSV file (from AB) in Excel and then you will see in the top right hand corner there is an "Analyze Data" button. Click it and you will be presented with the Analyze Data options on the righthand size. You can see it is even showing a pre-packaged histogram option (frequency). You just click on that and hey presto a histogram is automatically generated for you--don't need to worry about formulas etc. See the second image below. You can do this analysis on whatever column of data you want. The Analyze Data option is a fantastic tool and allows you to do a lot of statistic analysis on your data--it's an invaluable tool. More on that tomorrow.

As a related but side issue--a very important aspect of any system is positive expectancy. You can see from the second picture below that based on the 1000 runs shown in the first figure that this system has a positive expectancy, that is none of the 1000 runs returned a negative net profit. However, if you look at the third image below AB's MC analysis certainly does not show that the same system has a positive expectancy with only around 10% of the 1000 runs returning a positive return--according to AB 90% resulted in complete capital loss. This is not because AB has some amazing way of doing MC--as I mentioned previously it is flawed.





 
Sadly true, working in IT, i have seen many like him.it is sad to show such arrogance preventing meaningful improvement in the software.which is looking a bit outdated by now and also has some bugs left after more than a decade

i agree with both of you. i enjoy the RT community and the creator of RT is very quick, welcoming, and helpful with the software or strategies. pretty quick to fix issues too. if he keeps the momentum up, then i see RT eventually outpacing AB.

there are also a bunch of sample strategies that come with RT. most of which are really helpful and actually pretty good. real advice is given on the forum too. definitely something which AB lacks, which seems more about asking why you're so stupid and just didn't know lol.
 
I would think so. it's pretty versitale software IMO. it doesn't have all the bells and whistles of tones of indicators that AB has. i coded up a supertrend indicator (and marsten helped!), and if its not available you can ask marsten. he'll even help with some code to include in your script, or add it to his to-do list in the next update. i find RT to be more intuitive as well. it handles multiple strategy testing better too. even testing things like futures is much much easier in RT than AB.
 
custom charts would be one area that is difficult to compete with AB. the charts are much more simplisitic. but since i'm running algo's more than taking indepth look at charts (i still look at charts to see a visual of what i'm trying to code), if custom layouts and custom charts is important than AB would be better IMHO.
 
@Warr87 @Linus van Pelt Do you consider RealTest as being the closest competitor to AB?
I should maybe give a try,?
 
What scripting language do they use? Please tell me Python ?
 
I agree with everything @Warr87 said.

Random, stream of consciousness thoughts...

I describe the difference between RT and AB this way (it's just an analogy, it's not perfect):

AB is analogous to Python: it's big, can do a lot, requires strong programming skills, will likely take you longer to code, you can shoot yourself in the foot if you're not careful.

RT is analogous to Excel: it can't do as much as Python, requires less programming skills, you can likely code it faster, harder to shoot yourself in the foot, there are many things it does better than Python.

If all you needed was a financial spreadsheet, would you code it up in Python? You probably could, but why would you?

AB is more a programming environment, RT is more a scripting environment.

You're not going to suck in a web page, or read and write to external files (except built in functionality), create fancy charts that are beautiful eye candy (but do they make you money or just impress your friends?).

RT makes if very hard to code a future leak...you have to really try, almost do it on purpose. It's possible, just much harder than with AB.

RT makes "sense" in places that AB doesn't. One example: if you are using weekly bars, it will show your buy date as Monday, not Friday. I mean, WTF Amibroker? It's just software - can't you work out what day Monday is on a weekly bar?

In RT, you import your data from Norgate (or other providers) into a binary format. Think of it like your hiberfil.sys file when you hibernate to disk. It a memory image saved to disk. It can take a long(ish) time to do the download (like 5 mins, sometimes longer if you're importing the entire NYSE from 1990-today), but once you've done so, RT is really really fast. I actually did a walkforward in RT, something that would have taken days in AB using my laptop. But for live trading you would just import the data you need (shorter time frames). Sometimes I can run a backtest in 1-2 seconds, especially when run repeatedly - the first time caches the data formulas in memory. Think of it like static variables for the first run, then subsequent runs only update those formulas when required. In many ways it's faster than AB.

It has built in rolling backtests via optimization, so I don't need my Powershell script to do that.

You can easily combine multiple strategies in one, say a long/short hedging strategy, where the two strategies are "self contained" (hard to describe - I'd show you the samples but that probably violates the RT license, esp. if I posted it here). As Warr said, the examples are really good, not just to illustrate the many capabilities of RT but as trading systems themselves.

The documentation is clear, written by a native English speaker, and not full of typos, grammar errors, and incomplete or inconsistent examples.

One thing that is really cool is the strategy can use it's own equity curve as input. I like to describe it as being "self aware". I read in a Bandy book that you should use daily mark to market performance to make trading decisions (position size, whether to switch off, etc). With RT this is easy.

RT is written by a very successful trader, and is built for traders. AFAIK Tomasz is not a trader.

Things you have to use the CBT for you can just easily code in RT. A quote from Marsten:


RT is much "younger" than AB, so it doesn't have all of AB's bells and whistles. For now, I will still use both, esp. AB's charting capabilities to visualize price patterns that I can try to code into a system. Hopefully, when my license for AB expires (AFAIK it will still work, I just won't get updates), I will have completely transitioned to RT.

I won't miss the the toxic culture of the AB forums. The culture in the forums, like the culture in many organisations (or even recent US Presidencies) is toxic because it flows down from its leader. Its members follow the example set by its leader.
 

I'll leave that to Warr as he's a more experienced trader than me. AB is certainly more well known, while RT is just gaining traction. But I don't know about other packages, such as TradingBlox, Metastock, etc.
 
Nice review, thanks. Certainly some of the issues you raise about AB are very true. BTW, there is an easy resolution to the weekly buy day showing as Monday not Friday. Have you looked at the “Intraday” tab (yes strange this is under intraday) in the Preferences settings? I’ve just downloaded the software so will be interesting to look at—I’m very keen to see what it’s backtesting and optimisation speed is like.
 
I am biting the bullet and going your way.some of my systems have plenty of buys and i can not fault the logic /process of doing a proper MC vs what MA showed was dummy MC on AB
It is actually difficult as i might discover some unpleasant truth.
Isn't it funny how we are wired?.i am scared finding something wrong,yet my rationale side agrees that the earlier the better.
A bit like people postponing cancer check.. because it could work and detect one...
Just to say: how could i ever do discretionary trading with a brain like that?
 
I think @Linus van Pelt covered everything pretty well. One downside of RT though is that it is a one-man band. If marsten was to stop, get sick and take some time off, etc., then RT wouldn't progress any further. All one person development teams are obviously prone to this. Marsten doesn't appear to be slowing down any time soon and has many improvements (and additional software) in the works.

i was hesitant with RT but i most definitely prefer RT over AB.
 
@Linus van Pelt, thanks for sharing the info on RT. Very interesting.

@MovingAverage - thanks for showing what the latest Excel can do. I'm behind the times there, but the good old Data Analysis module in older Excel versions doesn't make it too hard to knock up graphs for columns in a spreadsheet and whack on some formatting. Below are frequency histograms for 200 runs of a TF system over many years, in "Stevo format". The only element of randomness in these 200 runs was a 1/13 chance any particular set of Buy signals would NOT be taken.

Obviously this is yet another decision to make, as to how you "randomise". Have read about people introducing noise into price itself, BUY signals (or components of the overall signal).

 
On Amibroker:

  • Its another tool, but what an amazing tool
  • Every different technique or alternative tool we employ is likely to emphase an aspect of the process that may not have been clear in AB - we should probably focus less on "what's best" and more on "what else can I learn from this tool"?
  • There are definitely pitfalls with any tool, and AB has plenty of ways to blow your foot off, along with all the other design decisions developing systems (universe, win rate, max acceptable DD, etc, etc)
  • AB MC graphs has fascinated me every since figuring out how to do broom plots. Any given run involving "randomisation" may be over-optimistic or over-pessimistic. For me a huge benefit of multiple runs (either exported to Excel or simply eye-balled equity curves in AB) is the likelihood of showing up overfitting/over-optimisation.

Here are a couple of AB MC backtest runs from the pessimistic and optimistic ends of the runs for same system in previous post for the pretty graphs. I do worry MA if there are too many "failed/neg" runs in a MC - could be suggestive of too high a risk of ruin? (a whole new realm to explore....).

Believe there are still useful aspects worth review in this plots (e.g. spread, equity curves, main page statistics), but agree caution is required not to accept as gospel.




 
That’s a great set of charts @Newt -- well done. That's exactly what I do. These charts provide some real insight into what you can expect out of your system and of course you can apply this technique to any of the metrics AB spits out.

I've read about folks adding randomness to buy and sell prices but to be honest I do not see any value in that for my personal style of trading--all my systems buy/sell on open/close, but could see some merit in that if you are an intra-day trader. While I haven't done this yet but see advantage in it, is to add an element of randomness to your buy/sell key parameters to better understand how sensitive your system is to those parameter settings. As a simple example assume you buy on a simple short MA crossing over a slow MA. Your short MA is 5 and your long MA is 20. You could add a constrained amount of randomness to the 5 and 20 values (not wide changes in the values) just to see how sensitive your system is to those parameters. The logic being that a good system should not be too sensitive to limited changes to the 5 and 20 values. You would want to see a very tight standard deviation on that.
 
Last edited:
I do worry MA if there are too many "failed/neg" runs in a MC - could be suggestive of too high a risk of ruin? (a whole new realm to explore....).
@Newt - If you are talking generally about MC results (regardless of the tool) then absolutely you are on the money--no way I'd live trade any system that shows even 1 negative run. However, I wouldn't make that decision based solely on the MC results shown in AB for the reasons I've previously described. If MC analysis in Excel or TradeSim are showing any negative returns then no way I'd live trade it.
 
Cookies are required to use this site. You must accept them to continue using the site. Learn more...