Free will and randomness feel opposed to one another: free wil is what makes us human; randomness is the epitome of meaninglessness. But the two share a deep affinity: they both are built on breaking with the past and with contingency.
All things are contingent. No event has a single cause. But it seems to us that we have free will: the ability to make decisions, and so this seeming capacity to intervene in the world gives rise to causal questions. If I choose to eat this mushroom, will I get sick? Implicit in the question is the possibility that I might choose, or not, to eat the mushroom. Of course, our decision to intervene is itself contingent, since our minds are also part of the world. The contingency of our own minds makes it hard to infer causation: amongst patients who choose to go to the hospital outcomes are worse, but this does not mean going to the hospital makes you sicker. Or maybe I would have slept badly anyway on the nights I choose to stay on my phone until late. Our choices don’t feel very contingent; if they did, there would be no reason to ask about causation. But we know, abstractly, that our choices may produce selection bias in ways we don’t fully understand.
How, then, do we establish causation? We require a technology to breaks the chain of contingency more effectively than our own decision making processes. That technology is randomness: aleatoric machines, designed originally for gambling, which are specially constructed using symmetry and information-discarding physical processes (spinning a wheel, a coin, or a die; mixing an urn of balls; or their mathematical analogues in random number generators). The function of these aleatoric machines is to break the bonds between the past and the future more thoroughly than our own minds can. If an outcome follows from the output of an aleatoric device, it cannot have followed from anything else: if you are selected for the treatment arm of an experiment by a coin flip, the effect of the past on your outcome is broken at the moment of the coin flip, at least relative to what would have happened had the coin flip turned out otherwise.
So free will gives meaning to causation, and causation is detected by randomness. But there is an irony to this arrangement. Causation matters because we make choices, the choices we make are the place where who we are connects to the world, and the source of meaning. But the very notion of causation requires a break from the past, a break which is most perfectly created by randomness, the epitome of meaninglessness, as a process which is, by design, maximally disconnected from the rest of the world. At a deep level, the phenomena of free will and randomness are siblings: each relies on the possibility of non-contingency. But behind free will there lies a soul, and behind randomness, there is nothing (or worse, a tawdry betting game). The difference between the two is one of value, though, not of kind.
This priveleged epistemic role of aleatoric devices in establishing causation is an extremely recent invention. Often the idea is attributed to Neyman [1], though this idea is certainly in Hume (Section VIII, Part I of [2]), who writes:
“And if the definition above mentioned be admitted; liberty, when opposed to necessity, not to constraint, is the same thing with chance; which is universally allowed to have no existence.”
(I might argue that Section VIII, and not Section VI, is the right place to look in Hume for the origins of modern probabilisitic causal inference, in contrast to some other authors [2]).
But even if the origins were to be traced back to the 17th century, it is well-established that the pre-modern world did not give aleatoric reasoning such a priveleged status ([4]). To me, this opens up a range of interesting questions. Is it only the modern epistemic prominence of randomness that allows us to even conceive of causation this way? Does the close proximity between meaninglessness and free will affect our view of ourselves? When we insist, in practice, that only randomness can establish causation, do we leave behind ways of interacting with the world that do not fit into this framework? In what other ways can we imagine leaving contingency behind? How have other cultures and times done so?
[1] Neyman, Jerzy, and Karolina Iwaszkiewicz. “Statistical problems in agricultural experimentation.” Supplement to the Journal of the Royal Statistical Society 2.2 (1935): 107-180.
[2] Hume, D. (1748). An Enquiry Concerning Human Understanding. Renascence Editions.
[3] Holland, Paul W. “Statistics and causal inference.” Journal of the American statistical Association 81.396 (1986): 945-960.
[4] Hacking, Ian. The emergence of probability: A philosophical study of early ideas about probability, induction and statistical inference. Cambridge University Press, 2006.