Illustration by Jaci Kessler
Illustration by Jaci Kessler

At the time of World War I, many meteorologists had all but given up on the idea of accurate and scientific weather forecasting. Then a physicist and ambulance driver by the name of Lewis Richardson, in spare moments between terrifying bouts rescuing the injured, undertook a momentous project.

His aim was to calculate, by simulating the actual physics, the development of the weather over a local zone of Europe over eight hours. He failed, but only because of a small arithmetical error. His basic idea was correct, and weather forecasting centers around the globe now use variations of the technique with impressive predictive success.

In Reading, England, for example, the European Centre for Medium-Range Weather Forecasts runs two supercomputers simulating a virtual atmosphere. The model forecasts the wind, the temperature, and the humidity at more than 20 million points from the earth’s surface up to a height of about 40 miles. In the U.S., the National Centers for Environmental Prediction does much the same thing.

The simulations lie behind the daily and weekly weather forecasts reported on the nightly news, as well as more specific prediction services for farmers, the airline and shipping industries, the military, and anyone else whose projects depend seriously on the weather. When an oil company charts a path for a tanker journey of several weeks, it saves tens of thousands of dollars by routing away from strong winds and storms.

Driving Forces

Forces much like those that determine the weather also drive the most important and disruptive events in economics and finance -- bubbles, debt crises, bank runs, even waves of corporate corruption. With ideas and techniques from other parts of science, it’s possible to explore market feedbacks and instabilities in detail never before possible.

In the not-too-distant future, it’s easy to imagine a U.S. or European Center for Financial Forecasting. Thousands of researchers would oversee massive simulations probing the developing network of interactions among the world’s largest financial players, following the vast web of loans, ownership stakes and other legal claims that link banks, governments, hedge funds, insurance companies and ratings companies.

The computers would test scenarios and calculate hundreds of indicators of systemic leverage, the density of interconnections, or the concentration of risk at single institutions. Experts would probe models of the financial system, looking for weak points and testing resilience, much as engineers now do with models of the electrical grid or other complex systems.

What’s currently missing, aside from the willingness of the economics profession, is data. To ensure the safety and stability of a nuclear reactor, engineers need access to every detail of its operations and the ability to examine every component and its links to others. The same should be true of any agency trying to support the stability of the financial markets.

Authorities have historically collected financial data on an institution by institution basis, being less concerned by the links between them. Such a piecemeal approach obviously makes it impossible to say anything about interconnections and the feedbacks they create.

The crisis has spurred moves to collect much greater amounts of data on financial networks. In the U.S., for example, the Dodd-Frank Act created the new Office of Financial Research to bring better financial data to policy makers. Private hedge funds will soon be obliged to report information on their funds’ exposure to different asset classes, their use of leverage, and their vulnerability to liquidity shortages.

Data Revolution

A real data revolution might go much further. Modern sensor systems -- as computerized components find their way into almost every object we use and own -- will probably gather as much data in the next 10 years as we have gathered in all of human history. One can only imagine how all these data might feed into forecasting models. If excessive optimism or pessimism drive many market crises, these collective excursions from reality almost certainly show up in the physiology of the people involved. Think of a patch worn by volunteers that gathers physiological information and uploads it directly to some database.

Of course, none of the forecasts based on such data will meet the ideal of perfect knowledge of the future. Weather forecasters don’t aim for this ideal, as they always have incomplete data on the atmosphere and can only work with approximate equations. They make a strength of this uncertainty by running thousands of simulations, changing the data randomly to reflect their ignorance and so generating thousands of possible forecasts about the future. The result is a cloud or “ensemble” of guesses about where the future will lie.

Ensemble forecasting in finance and economics might work similarly, using slightly different possibilities for how people and companies behave, also enabling those elements to have their own independent intelligence to try things and learn tricks the modeler may never see. The result would be not a single prediction but a swarm of possibilities.

In pondering this future, delicate issues loom into view. As we develop large computational systems packed with masses of data monitoring the financial and economic system and projecting its likely future, this knowledge becomes extremely valuable. It should be treated as a public good, akin to clean air and water.

Dealing with such questions is the price we pay for moving beyond the myth of a perfect self-regulating equilibrium -- the paradigm that has dominated economic thinking for the past several decades and that has done such a poor job of predicting the economic weather.

We must not replace it with a new fantasy, fooling ourselves into thinking we can build a complete theory of markets once we appreciate their similarities to other physical processes. We can’t eliminate fragility, only reduce it, and then only with persistent efforts to highlight our ignorance, our biases, our preconceptions about what can and can’t happen.

(Mark Buchanan, a theoretical physicist, is the author of “The Social Atom: Why the Rich Get Richer, Cheaters Get Caught and Your Neighbor Usually Looks Like You.” This is the last in a series of three excerpts from his new book, “Forecast: What Physics, Meteorology and the Natural Sciences Can Teach Us About Economics,” to be published March 26 by Bloomsbury. The opinions expressed are his own. Read parts one and two.)

To contact the writer of this article: Mark Buchanan at buchanan.mark@gmail.com.

To contact the editor responsible for this article: Mark Whitehouse at mwhitehouse1@bloomberg.net.