We’re doing a (super fun) reading group this semester, jointly with the UCB philosophy department, on the foundations of statistics. Naturally there’s a lot of talk about the benefits and shortcomings of different approaches to statistical inference. Is Bayesian statistics “more dogmatic” than frequentist statistics? Can you argue with a Bayesian about their prior? Do Bayesians care about model selection? Can a frequentist have beliefs about unknown parameters?
As we have these discussions, I find myself coming back to this quote from Roman Jakobson (and Franz Boas) (Jakobson (1944)), about languages:
“Different languages differently select those aspects of experience ‘that must be expressed.’ Such ‘obligatory aspects are expressed by means of grammatical devices,’ whereas some other aspects are taken as non-obligatory and are expressed by lexical means. And each language in its own way chooses the concepts to be expressed by single simple terms or by combinations of distinct terms, by entirely heterogeneous or by related terms.”
For example, in Russian you might say “Я еду в магазин,” whereas in English you might say “I’m going to the store.” In Russian, you have additinally specified — as you must, given the peculiarites of Russian verbs of motion — that you are going in some kind of vehicle. In English, you have (necessarily) specified “the store” (some particular store) rather than “a store” (any store), emphasizing the store rather than the going. In the respective languages you have no choice but to specify these things, but in Russian you are free add words to specify a particular store, and in English you are free to specify that you’re going in a vehicle. The languages are limited in what they must say, not what they can say.
If you think, as I do, that statistical analyses are most usefully thought of as a kind of speech, then Jakobson’s assertion about language applies equally well to statistical frameworks. A Bayesian must specify a family of full generative processes and a frequentist must consider the behavior of their procedure under new datasets. But this does not prevent Bayesians from considering the frequentist properties of their estimators (look at the Bernstein von-Mises theorem), nor does it prevent frequentists from fully specifying generative processes and integrating out uncertainty (look at the EM algorithm).
Many of us are not free in our daily lives to switch to the language that is most appropriate to the task. But unlike languages, we can happily choose a framework that is most appropriate to a particular problem. Dogmatism arises only when we refuse to do so, or perhaps when we assert that another person is incapable of doing so. I would argue that some analyses that are essentially one-off inferential problems are really best thought of in the Bayesian framework (e.g., the degree to which our data constrains the values of certain physical constants). Similary, some staistical procedures are clearly intended to be repetitive, and are not concerned with correct specification (e.g., most of machine learning). That said, solutions to each problem can be made in either framework, as long as the task and interpretation of the result is properly understood by all the parties involved.
Given this, it’s a little unfortunate that so much of the philosophical discussion appears to be one side accusing the other of being unable to perform some inferential task. It seems to me that such objections are usually straw men that don’t align with practice, as I argue above. A more fruitful discussion would be about what aspects of statistical problem we want routine practitioners to be required to think about, in certain settings, in order to avoid common errors.