Telling bear markets via model divergence

Oct 19, 2011: Telling bear markets via model divergence

home faq articles premium contact

One of the holy grails of traders is knowing when markets turn from long-term bullish to long-term bearish.
The most common technique used in this context is the death-cross (and its opposite: the golden cross) namely the crossing of the 200 day moving average by the shorter 50 day moving average.
Ignore the arguments in favor or against using exponential moving averages (EMAs) vs simple-moving-averages (SMAs). Any of these methods is too simplistic and suffers from lag (being too late) and whipsaws (giving false signals).
My preferred method is to watch the error rate convergences of the market models I build. Bear markets are inherently different from bull markets not just in their long-term direction, but also in their characteristic volatilities, correlations and more. Ergo, if we can build a reasonable predictive market model based on, say, bullish conditions, its error rate should generally decrease the more data we feed it, modulo a relatively stable amount of noise.

Here's a chart showing such convergence. The statistical learner has been fed nearly 80,000 examples containing various technical inputs on about 300 ETFs over the course of a year from May 2010 to May 2011. The target feature the learner is trying to predict is the %change for a month after each set of inputs were observed.
The X-axis is represents time via number of examples (the learning process) and the Y-axis represents the observed error of the model (predicted value vs. actual value). The learner never peeks forward into the future, it simply tries to generalize from the examples and 'guess' the future. Later, when it sees the future it adjust the prediction model based on the error it observed.
Before starting the model knows nothing so it starts with a very high absolute error of about 45% on the left of the chart.
We can see how the error term of the model keeps going down, until it nearly plateaus at a point which I like to call the "ambient-noise level of a bull market":
The most important take-away here is that this convergence is being maintained throughout the experiment. The error term diminishes asymptotically in a monotone fashion over all the checkpoint steps. Note that the X-axis is logarithmic: the number of examples seen by the learner, doubles with every consecutive point. In other words, as long as bull market conditions persist, the model converges onto a smaller error (the lowest is about 2.10% error for the average month-ahead %change prediction).
The important point is that this particular learning method never diverges at the resolution used, where each successive iteration doubles the number of examples we try to learn from.

But the moment the regime changes, the past examples are inherently different than the new ones being seen. The 'experience' of learning from the past, is becoming a liability, and the model starts diverging instead of converging. Here's the same chart, 3 months later, on Aug 26, 2011. The first signs of a regime change are that doubling the number of examples (last 6 months out of a full year) makes the error go up. This is because 3 out of these 6 months have been inherently different than the prior 3.
To demonstrate the superiority of this method over the death cross: The blue line started showing clear divergence two weeks earlier, on Aug 14, 2011. This was well before the death-cross occurred. Moreover, the death-cross signal wrongly fired twice during the 2010 April-Sep bull-market correction while this divergence method fired only once.

Let's zoom into the last 6 points. If this was hard to notice in our 2nd chart, the zoomed-in 3rd chart makes the picture crystal clear: the 20-day average prediction error has jumped from about 2.1019%, up to 2.1579%, a nearly 3% jump for a metric that has never jumped up before, as long as bull-market inherent ambient noise was kept at relatively constant levels.

What about today? Fast forward to October 19, 2011. The divergence is alive and well. The error rate is now about 20% above its trough. Obviously the model is yet to fully unlearn what it has learned during the bull market.
What should we expect in the future? Here's my guess based on my data: at a certain point, once we have enough history of typical bear-market conditions, the error rate should level-off and approach the bear market inherent ambient error level asymptotically. At that point, since we'd have a consistent (all bear) training set, I expect the chart to go monotonically down again but stop descending at a much higher level that it did during the bull. Looking back, the "ambient error" was about 3.37% for the average month-ahead prediction, at the beginning of March 2009 (just before the end of the prior bear). Interestingly, at that point in time it has already been slightly declining as the volatility of Jan-Mar 2009 was lower than the post Lehman bankruptcy rout (Sept-Oct 2008).
As the bull returns and volatility dies down, errors should get smaller as we start descending south, back towards the typical and lower bull-market (~2%) error-rates.
To sum up:

The ambient noise inherent in bull markets is inherently smaller that the ambient noise inherent in bear markets.
A good robust model, one which doesn't tend to overfit, tends to converge as long as conditions are statistically similar.
That model will start diverging when conditions change from a bull to a bear market.
This model convergence/divergence method is more robust, promising and earlier than the common and simplistic MA crossover methods.
Dec 2011 update:
About 1 month after this article was published the errors seems to have peaked followed by a decline in volatility.
About two months after the article was published (Dec 16, 2011) the errors have mostly stabilized around a lower baseline. The 20-day prediction online training errors were back at ~2.11% and the 1-day was back at ~1.64%. Here the full spectrum of estimated errors as of Dec 16, 2011:
Prediction period mean absolute error rates
 1 days         1.64%
 2 days         2.15%
 3 days         2.24%
 5 days         2.23%
10 days         2.14%
15 days         2.18%
20 days         2.11%
While this is still higher than the errors we used to have at the height of the bull market (under 2% for the 20-day prediction), they are much lower than they were in Oct 2011. It remains to be seen whether we'll see a continuation of the decline in volatility or the market will significantly change regime again.
home faq articles premium contact

The above merely reflects my own thinking and actions at the time of writing. Every investor should make up his own decisions based on his risk tolerance, time-frames, comfort-zones, convictions, and understanding. Never investment advice.
Any feedback is welcome.
-- ariel