The Paradox of Prediction

Excerpt from a chapter in Zann Gill’s forthcoming book If Microbes begat Mind.

That future does not yet exist for us to observe or predict. It awaits our design.

The paradox of prediction emerges from a cyclical defining process. Prediction can be characterized in four ways: absolute, probable, possible, and uncertain. Curiously, the definition of prediction itself loops around to unpredict its “self” (Gill 1986). The four aspects of prediction participate in an iterative cycle, showing the need to move beyond linear, goal-setting processes to design environments and tools to enhance collaborative intelligence. ZannGill-ParadoxofPrediction-2-16

As a thought experiment, imagine that four degrees of prediction lie on a spectrum. Starting with absolute certainty, and progressing to complete uncertainty, that spectrum cycles back on itself to become a feedback loop. So prediction breaks the rules of logic, paradoxically contradicting its “self.”  The difference between retrodiction (the reconstruction of the past), explanation, and prediction (the construction of the future) is generally neglected. All three require completing a pattern based on partial information. But there is a critical difference: prediction points in the same direction as the arrow of time in evolution, creative thinking, and design.

Collaborative intelligence posits that a particular form of contradiction, paradox, may lie at the root of any logical attempt to analyze synthesis, which constructs feedforward loops into the future.

We cannot predict our futures. The act of predicting influences the outcome of the events predicted, creating a paradox (Gill 1986). Once we abandon a future of “objects to be predicted,” we maximize the potential for collaborative intelligence by specifying constraints and windows of opportunity through which future potential can emerge. Through pattern recognition, rebalancing at each arrival of new information, evolution integrates, building new synergies and recognizing new potential. New patterns emerge to be recognized.

Absolute prediction — hypotheses in the physical sciences

The first connotation of prediction, as absolute, is associated with testing hypotheses in the physical sciences. In the ideal case, no deviation would be allowed for in the predicted outcome. There is zero tolerance, except insofar as the granularity of the prediction allows tolerance.

Absolute prediction is passive. “Passive” implies that the event predicted will occur or not, independent of whether the prediction is made or not. Prediction typically acts independently of the context observed, exerting no influence. In the physical sciences, a theory is disproved by finding a single counter-example that contradicts the theory’s prediction, one black swan.

Probabilistic prediction — hypotheses in biological sciences

The second connotation of prediction, as probabilistic, is associated with constructing rules in the biological sciences, and is also passive. As biologist Ernst Mayr noted, seldom can biology boast the certainty of absolute laws (Mayr 1986).

Unlike the physical sciences, which aim for prediction that anticipates with absolute accuracy, in biology accuracy is generally statistical, rather than absolute. Prediction may be assigned a probability of occurrence, rather than a certainty.

A theory predicts, either absolutely or probabilistically, the behavior of phenomena in the specific instances it covers. When one instance contradicts the prediction, the theory must be abandoned or transformed to account for this larger context. Prediction is not static, an “object.” Rather, it is an evolving representation of our understanding. The two passive connotations of prediction (the upper two quadrants of the diagram) are generally taken to cover the meaning of the term.

Possibilistic prediction — self-fulfilling prophecies

The third connotation of prediction, as possibilistic, is active, in that predicting may influence the outcome of future events, becoming a “self fulfilling prophecy.” Today, with new tools to support scenario-building and simulation, this third possibilistic connotation has new meaning. The term “simulation” no longer refers only to modeling an existing object or process. Simulating the future with partial data in the present, and hypotheses about trends, invokes “the paradox of prediction.” Prediction, through simulation and forecasting, once deemed a reliable basis for action, can itself influence the future outcome of events.

Uncertain prediction — the emergence of novelty

The fourth connotation of prediction as uncertain, emergent, and active, recalls the unpredictable inferential leaps, the sudden insights that philosopher of science C.S. Peirce called abduction (Peirce 1934). This fourth connotation extends the diminishing certainty implied by probabilistic and possibilistic prediction, and the link between prediction and innovation. Nelson Goodman noted that “what has happened poses no logical restrictions on what will happen” (Goodman 1983).

In summary, in the first connotation prediction aims to reveal absolute truth: prediction and contradiction are complementary. In the second, prediction is coupled with probability, while in the third, prediction depends upon how ambiguity and incomplete data are interpreted through pattern recognition.

The second and third connotations manifest varying degrees of emergence or contingency — unpredictability. The fourth, uncertain, active connotation of prediction is akin to the inferential leap that introduces a new idea; it is emergent.

The first three connotations of prediction, described above, suggest a progression from the ideal of absolute certainty in the physical sciences, to  probability, accepted in the life sciences, to the possibilistic speculation of simulation and projection in scientific disciplines through data visualization. But such a progression is an illusion. There is no spectrum with physics sitting stolidly on one end and visualization teetering speculatively on the other. Visualization in science depends upon computation. And computation-enabling visualization of complex data depends upon researchers’ visual pattern recognition and interpretative capabilities. The scientist analyzing visualized data to recognize patterns emerging works more like a designer than ever before.

In the paradox of prediction, the definition of prediction cycles from analysis to synthesis, looping around to unpredict its “self.” But, through this circular argument, science and design emerge, not as opposed poles, but seated on the same loop, counterposing Lewis Mumford’s question about linear projections, typical of future forecasting, with accelerating feedback loops converging toward an outcome that could not be predicted in advance.


Above is a summary of Zann Gill. 1986. “The Paradox of Prediction” published in Daedalus: Journal of the Academy of Arts and Sciences 115. 3. 17 – 49. (released as a book by Academic Press.) and also developed as a chapter in her book, If Microbes begat Mind.
Charles Sanders Peirce. Peirce, C. S. Collected Papers of Charles Sanders Peirce. Cambridge, MA, Harvard University Press. 1934.
Nelson Goodman Fact, Fiction, and Forecast. 1954. Fourth Edition. Cambridge. Harvard University Press. 1983. Print.