New posts

Reply to thread

Message

Hi Kolonel --

The first question to ask is -- are these results from in-sample testing? I assume the answer is yes because of the length of time being reported and the fact that it includes today.

The in-sample data is that data used to develop the system. It is always a good idea to reserve some data that follows the in-sample data, but is not used at all during the development of the system, to see how the system performs on data it has never seen before. This is the out-of-sample data.

After all, tomorrow is always out-of-sample, and we want to get some idea of what is likely to happen when we place a trade tomorrow.

Results from in-sample tests have no value in terms of estimating the likelihood of profitable trading. No value.

I note that this system has 20% winning trades, 80% losing trades. Personally, I would have trouble trading a system with such a low winning percentage.

AmiBroker has some very nice features that allow you, as the system developer, to design your own custom objective function. The objective function is the metric used to measure the "goodness" of each run, or of each alternative when you are performing optimizations.

I recommend that you investigate some of the metrics that already exist in AmiBroker that simultaneously reward equity growth and penalize drawdown. These include: K-Ratio, Ulcer Performance Index, CAR/MDD, RAR/MDD, and RRR.

Thanks for listening,
Howard

<blockquote data-quote="howardbandy" data-source="post: 273136" data-attributes="member: 9179">Hi Kolonel --The first question to ask is -- are these results from in-sample testing? I assume the answer is yes because of the length of time being reported and the fact that it includes today.The in-sample data is that data used to develop the system. It is always a good idea to reserve some data that follows the in-sample data, but is not used at all during the development of the system, to see how the system performs on data it has never seen before. This is the out-of-sample data.After all, tomorrow is always out-of-sample, and we want to get some idea of what is likely to happen when we place a trade tomorrow.Results from in-sample tests have no value in terms of estimating the likelihood of profitable trading. No value.I note that this system has 20% winning trades, 80% losing trades. Personally, I would have trouble trading a system with such a low winning percentage.AmiBroker has some very nice features that allow you, as the system developer, to design your own custom objective function. The objective function is the metric used to measure the &quot;goodness&quot; of each run, or of each alternative when you are performing optimizations.I recommend that you investigate some of the metrics that already exist in AmiBroker that simultaneously reward equity growth and penalize drawdown. These include: K-Ratio, Ulcer Performance Index, CAR/MDD, RAR/MDD, and RRR.Thanks for listening,Howard</blockquote>

[QUOTE="howardbandy, post: 273136, member: 9179"] Hi Kolonel -- The first question to ask is -- are these results from in-sample testing? I assume the answer is yes because of the length of time being reported and the fact that it includes today. The in-sample data is that data used to develop the system. It is always a good idea to reserve some data that follows the in-sample data, but is not used at all during the development of the system, to see how the system performs on data it has never seen before. This is the out-of-sample data. After all, tomorrow is always out-of-sample, and we want to get some idea of what is likely to happen when we place a trade tomorrow. Results from in-sample tests have no value in terms of estimating the likelihood of profitable trading. No value. I note that this system has 20% winning trades, 80% losing trades. Personally, I would have trouble trading a system with such a low winning percentage. AmiBroker has some very nice features that allow you, as the system developer, to design your own custom objective function. The objective function is the metric used to measure the "goodness" of each run, or of each alternative when you are performing optimizations. I recommend that you investigate some of the metrics that already exist in AmiBroker that simultaneously reward equity growth and penalize drawdown. These include: K-Ratio, Ulcer Performance Index, CAR/MDD, RAR/MDD, and RRR. Thanks for listening, Howard [/QUOTE]

Verification