Measurable herding behavior precedes large market drops

9 May 2011 by Jim Fickett.

Cynics often note that when a bull market is in full swing, all news is interpreted as good news. A recent paper proposes a quantitative measure of the extent to which investors are all following each other, rather than thinking about the facts. One variant of this measure gives some advance warning of large, single-day market drops.

Bubbles and, more generally, herd behavior, are at least as important as fundamental value for the markets. Hence the value of momentum strategies. But of course running with the herd also risks running over the cliff with the herd. Momentum investing would be much more attractive if one had some objective means of determining when time is getting short.

Along these lines, Otto Ritter linked, in an April comment, to an interesting academic paper entitled, Predicting economic market crises using measures of collective panic, by Dion Harmon, Marcus A. M. de Aguiar, David D. Chinellato, Dan Braha, Irving R. Epstein, and Yaneer Bar-Yam.

The central idea of the paper is that an increase in herd behavior precedes a crash, and this can be measured by looking at the fraction of stocks moving in the same direction each day.

We are interested in the extent to which stocks move together. The extent of such co-movement may be large even when price movements are small. Indeed, even when price changes are small, we expect that co-movement itself is the collective behavior that is characteristic of panic, or panicky behavior that precedes a panic. Thus, rather than measuring volatility or correlation, we measure the fraction of stocks that move in the same direction. We find that this increases well before the market crash, and there is significant advance warning to provide a clear indicator of an impending crash. The existence of the indicator shows that market crashes are preceded by nervousness that gives rise to following behavior — increased collective behavior prior to a panic.

The way this idea is implemented gets a little complicated in the details.

Step 1. On any particular day, let k be the number of stocks that are higher at the close than at the open. Let N be the number of stocks in total, so k/N is the fraction of the market that moves up on the day.

Step 2. The predictive method looks at the distribution of k/N in moving windows of one year. So for any year-long period, we can plot, for each k/N, the number of days on which that fraction of the market rose. The authors call this frequency distribution f(k/N).

Step 3. Using a theoretical network model of the market, they introduce a family of analytical frequency distributions, parameterized by a variable U. Intuitively, U is the amount that real news matters. I.e. when U is high, investors are making decisions based on facts; when U is low they are mostly looking to see what everyone else is doing.

Step 4. Coming back to the data, for each year-long window of data, we can plot the actual frequency distribution f(k/N), and then find the value of U that provides a best fit to that data. This value of U is assigned to the final day of the year-long period from which it is calculated.

Step 5. At this point we have a value of U for each day, calculated from the year-long window of data coming up to that day. Now we look for unusually large drops in U. For each day, we first calculate a standard deviation for U in the period from two years previous to one year previous, and then calculate the change in U from one year previous to the current time, in units of that standard deviation.

Step 6. Finally we get to the predictive rule: When U falls by more than 2 standard deviations, after a period of rising, then there is likely to be a large one-day market fall within one year. For convenience, let's call such a 2 standard deviation drop a “herding signal”.

The authors look at the period from 1985 to 2010 and calculate U as above, using the data on stocks in the Russell 3000. They show that each of the eight largest one-day percentage drops in the Dow Jones Industrial Average come in the year following a herding signal, and each herding signal is followed by one of the eight largest drops, within one year.

Here is a figure showing the main result (reproduced with permission):

The top panel shows U itself, along with estimated sampling error bounds. The bottom panel shows the year-over-year change in U measured in standard deviation units (blue line). A herding signal occurs when the blue line falls from above the blue dashed line to below the red dashed line. The one-year windows following the herding signals are shown by blue shading, and the largest one day drops in the DJIA are shown by vertical red lines (note, the days with large drops are clustered, and not all show up as visually distinct).

There are a number of somewhat arbitrary choices in the both the measure and its evaluation, so it is not quite clear just how general the results may really be:

• Why not use the same index, say the S&P 500, both for calculating U and measuring its ability to predict drops?
• Is there anything special about one year windows, the period from 1985 to 2010, or the eight largest one-day drops?

Still, the results are impressive, and it would be quite interesting to try to use them to warn of markets becoming more brittle.

The first challenge for the retail investor, in trying to apply the ideas in this paper, is that access to large databases of daily stock price changes is expensive. However it is possible to access, without charge, tables of daily price histories of individual stocks (for example, from Daily Finance). I would guess that you don't really need the full Russell 3000 but, rather, if one were to choose perhaps 20-50 diverse individual stocks that respond very differently to different aspects of the global economy, one might well observe the same results.

The warning of a possible crash within one year is useful, but it would be nice to come up with an investable thesis and a shorter-term warning. So if I were to pursue this I might, just for example,

• Choose a set of about 20 stocks
• Construct the measure based on a three-month window of data
• Look to see if, more generally, such a measure were correlated with volatility, say, 3 to 6 months later
• Try trading options on the VIX using the herding measure

In any case, this is a nice objective confirmation that when investors start watching each other instead of the fundamentals, it is time to batten down the hatches.