Control and the Definition of Randomness

Over fifty years ago, the leading statistician Maurice Kendall analyzed price movements in Chicago wheat futures(at the time, the world standard for a deep and liquid market). Finally, he wrote:

The series looks like a 'wandering' one, almost as if once a week the Demon of Chance drew a random number from a symmetrical population of fixed dispersion and added it to the current price to determine the next week's price. And this, we recall, is not the behavior of some back water market. The data derive from the Chicago wheat market over a period of fifty years.

It may be that the motion is genuinely random and that what looks like a purposive movement over a long period is merely a kind of economic Brownian motion. But economists -- and I cannot help sympathizing with them -- will doubtless resist any such conclusion very strongly.

Maurice Kendall, in Journal of the Royal Statistical Society (1953), p. 13.

We are interested not so much in Kendall's finding as in his reaction to it. The study of economics has changed greatly over the intervening half century, to the point that the existence of the 'Demon of Chance' has become an axiom and the study of that demon's properties is a [roughly forty] billion-dollar industry. The change, however, has not been in our understanding of how prices are constructed -- we share Kendall's beliefs about price discovery almost to the letter -- but in our definition of 'genuinely random'.

Kendall's words show a deep belief in the power of rational analysis, even in a case where the underlying dynamics are known to be highly resistant to such analysis. [Predictable market patterns, if they are discovered, tend to be obliterated by the attempts of traders to profit from them.] His faith can only have been based on a belief that he could overcome the market's resistance -- that the rational power of statisticians and economists was somehow qualitatively superior to that of mere traders.

This belief, baldly stated, seems indefensible, but it is a prevalent undercurrent below a wide swath of recent thought. Isaac Asimov's tremendously influential Foundation Trilogy, for example, was published from 1950 to 1953 (though the first Foundation stories were written in the early 1940s). [In Foundation, Asimov posits a science of 'psychohistory' which can predict the future history of a sufficiently large group.]

Kendall's difficulty is not so different from that confronting a planner in a controlled economy. To believe in the efficacy of such control, it is necessary to believe, as Asimov did and Kendall seems to have, in the superiority of the planners. Friedrich Hayek had written on this point still earlier:

The peculiar character of the problem of rational economic order is determined precisely by the fact that the knowledge of the circumstances of which we must make use never exists in concentrate or integrated form, but solely as disbursed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess. The economic problem of society is thus a problem of the utilization of knowledge not given to anyone in its totality.

The most significant fact about the price system is the economy of knowledge with which it operates, or how little the individual participants need to know in order to be able to take the right action.

Friedrich Hayek, in American Economic Review (1945), p. 526.

Hayek's thrust here is that the information whose discovery drives price changes is not discovered, but created. Command economies are doomed to inferiority because they cannot operate with this 'economy of knowledge'; and no scientific priesthood can save the day, because the information they are seeking to unearth does not in fact exist.

This is what we mean today by random motion in prices. The idea that price changes are completely random -- 'efficient markets' -- has been so successful that it is now the starting point for any study, or any attempt at proprietary trading. [Assuming that markets are not quite efficient, where can I find inefficiencies large enough to profit from?] But the greatest measure of its success is that the word 'random' carries a different meaning to us than to Kendall -- and that we are, in the main, unaware that it was ever different.

(Hat tip: Mark Rubinstein)