Not sure what you mean by "truly", but I'm actually in the middle of backtesting a scalping method myself.
Based on tick charts though, not the market depth.
If you read TH's posts, you would see why it can't be backtested, and why charts can't be used.
Why would someone who has a map (trading plan) turn left (so called emotional trade) when the map calls for a right turn(taking a small loss) or no turn(no trade)?
I can come up with only three reasons.
1. they are mad, stupid or deluded
2. they are looking for entertainment because they are bored.
3. they have no confidence in the map because they know that they have no results or that the results are from luck. ie their map is wrong.
Maybe there is others but I think the main reason people make bad trades is that they have no proof that what they do is the right thing but are unable to admit it.
I'd say there are other reasons. If your trading has unintended consequences, like not being able to sleep, regardless of position size and success. If the markets you need to trade are messing with your ordinary life. But this is probably got more to do with what I said with matching trading plans with your own personality. Not sure if can know this fully until you start doing it however...
I cannot figure out how you went from my statement of
"you KNOW in advance when the strategy will be declared not to be working any longer"
to
"unless you believe that markets are inherently predictable by way of analysing historical data, in which case, every system once profitable will always remain profitable"
Such a belief system would be illogical, irrational and ultimately lead to blowing up. All backtesting does is show you what worked in the past and give you some idea of how it worked in the past. As such, it is an invaluable tool - if it has never worked in the past, then it is not likely to suddenly start working tomorrow. It is also able to provide you some indication of when you should say "enough - this system is not performing to expectation".
But just because it's not performing to expectation, doesn't mean it isn't working. You simply just cannot know in avance when/ if a system will stop working, regardless of its performance. In my mind anyway.
Efficient market hypothesis may suggest that something not working in the past, may well have the possibility of working in the future...
Same thing I've been trying to get through to tech, find out the origin of the word "expectation", and a lot of this will make sense.
However, I have not yet come across a strategy that cannot be backtested and walk-forward tested adequately - it may be very time consuming and labour intensive to examine some edges, but they can be tested if you really want to.
Would you like to offer an example of a strategy that could not be backtested adequately?
I doubt you could backtest options strategies adequately as well. Especially, those with a writing bias. One couldn't deny that even a conservative strategy has a great chance of increasing income, but because of volatility, the effect on the premiums, I doubt you could ever really test that. You could only test the underlying strategies you would then trade options on, but that is no guarantee you can make money from the options trades based on it.
I think you've described mechanical traders rather accurately in one word.
Cheers.