More retarded bayesian jokes at Gelman’s, Yudkowsky’s (the blue boxes). And of course xkcd and xkcd.
I recently came up with an example I would use to explain the difference between bayesian and frequentist statistics to a layman. It goes like this:
I have my hand in my pocket. Imagine I asked someone to guess what number I’m doing with my digits. First, she would think something like “The pocket seems just roundish, so probably his fist is closed. Uhm but he could be holding out his pinkie below. But any other digit would be too revealing. So I bet 75 % it’s zero, 25 % it’s one.” Then, she shouts “ZERO!” hoping to guess it right. And I say “nope.” (I would have answered that in any case.) The point were she states “It’s this percent for this possibility, that for that one” is bayesian reasoning. When she shoots out a number trying to nail it, that’s frequentist.
Now I need to walk around in the little village where I’m living and find an old lady who just wishes to know more about inference.
Honestly I find it quite surprising that these common dissings between Bayesians and Frequentists usually don’t include the — I would say — major core difference between the two theories, which lies in the different interpretations of the word probability that the two theories have.
https://en.wikipedia.org/wiki/Probability_interpretations
In the given example the used reasoning is always Subjective Bayesian, since the probabilities represent the speaker epistemic uncertainties.
For an answer to be Frequentist one should be able to “repeat the experiment”, whatever this means in the example.
Bayes theorem and probability laws are just a mathematical theory.
BTW very nice blog. Also, since I’m trying to better understand the issue too, don’t be afraid to post back.
Yes, you’re right, in the example the person is always reasoning in a bayesian way, and to be frequentist you have to say what’s to be repeated: do I have to ask it again to the same person? To another random person? How random? With the same hand? With the same finger out?
However in practice you don’t repeat experiments. I’d say that the possibility of repetition is a strong idealization.
Whenever I see some convergence proof that as the number of samples $N$ goes to infinity some frequentist tool works perfectly whatever the distribution is, and that should be reassuring because you don’t have to make assumptions on the distribution, I think that you have just hidden a huge independent and identically distributed variables assumption under the carpet. (Often a weaker assumption actually, but you get the point.)
So the example tries to give a practical intuition on what you have in hand when an analysis gives you an estimator vs. a posterior distribution. Even the frequentist answer has some reasoning and assumptions under the hood.
My example could be inappropriate in another way: you could say that when the person shouts her guess, it’s not a frequentist estimate, it’s her applying bayesian decision theory to the game of guessing the digit.
Refining the statistical understanding of that simple example requires putting a lot of context around it.
In general I see any statistical analysis as some core mathematically clear steps surrounded by a mess of implicit informal assumptions and reasoning. I’d say the part that has a greater effect on the result is almost always that outside the explicit math. The more I can pull out from the reasoning carpet and put over the math rug, the more I sleep well. So I see a frequentist analysis as something that hides more things under the carpet to keep the math leaner: elegant but not good overall.