Testing and Tuning Market Trading Systems is intended for readers who have a modest statistics background (Statistics 101 is plenty), have some programming skill in any language (C++ with a strong bent toward traditional C is used in the examples here), and are interested in trading financial markets with a degree of mathematical rigor far beyond that of most traders. Here you will find a useful collection of algorithms, including sample code, that will help you tweak your ideas into trading systems that have above-average likelihood of profitability. But there are many things that you will not find in this book.
Introduction:
The following topics are covered in this book:
If your system involves optimization of parameters, and most do, you will learn how to determine whether your optimized system has captured authentic market patterns or whether it has simply learned random noise patterns that will never again appear. You will learn how to modify linear regression in a way that makes it even less susceptible to overfitting than it already is and that, as a bonus, separates predictors into those that are valuable and those that are worthless. You will also learn how to modify linear regression to enable its use in moderately nonlinear situations.
You will discover an extremely general and powerful nonlinear optimization algorithm that is applicable to both predictive-model-based trading systems and traditional algorithmic systems. All trading systems assume a degree of consistency in the market being traded; if the pattern on which your system is based has occurred regularly over recent history, we must assume that this same pattern will continue into at least the near future. Some trading systems are robust against moderate changes in market patterns, while other systems are rendered worthless by even tiny changes in market patterns. You will learn how to assess the degree to which your system is robust against such changes.
If you have designed your own proprietary indicators, you will learn how to confirm that they are reasonably stationary (a critical property for any effective indicator) or massage them into stationarity if they are not. You will also learn how to compute them so as to maximize their information content, minimize their noise, and supply them to your trading system in an effective, efficient manner so as to maximize their utility. Most trading system developers are familiar with walkforward testing. But not so many are aware that ordinary walkforward algorithms are often insufficient for the correct validation of trading system candidates and can produce dangerously optimistic results for subtle reasons. You will learn how to embed one walkforward algorithm inside a second layer of walkforward analysis or perhaps embed a layer of cross validation inside a walkforward analysis. This “validation-within-validation” scenario is often not only the best way to test a trading system but the only truly correct way.
You will learn how estimate the range of possible future profits that your system can be expected to produce. If you discover that your system has almost certain future profitability but there is a high probability that this profit will be small relative to the risk incurred, you will know that your system is not yet ready to be traded. You will learn how to estimate the probability of catastrophic drawdown, even when your system is operating “correctly.”
You will learn about rigorous statistical testing algorithms that are resistant to the occasional large wins and losses that invalidate many “traditional” validation algorithms. Many trading system developers prefer to use the “spaghetti-on-the- wall” approach to trading system development. Although frequently scorned, this is actually a legitimate approach, as long as it is done intelligently. You will learn how to determine whether the “best” of numerous competing systems is truly worthwhile.
Contents:
- Introduction
- Pre-optimization Issues
- Optimization Issues
- Post-optimization Issues
- Estimating Future Performance I: Unbiased Trade Simulation
- Estimating Future Performance II: Trade Analysis
- Permutation Tests
Testing and Tuning Market Trading Systems: Algorithms in C++ By Timothy Masters pdf