fallacy of probability is one that needs to be understood before we
even attempt to define what probability means (that's why our
definition of probability is left until the next article). It is the
fallacy that there is one and only one valid way to measure
Consider the following two statements:
"There is a 50% chance that the next toss on a
fair coin will be a Head"
"There is a 0.0000001% chance of Martians
landing on earth this year"
Each of these is a statement that attempts to quantify our uncertainty
about some unkown or future event. But, although the statements are superficially
similar there are fundamental differences between them.
Statement 1 can be explained by
a 'frequentist' arguments: if
you toss a fair coin many times it will land as a Head roughly 50% of
the times. So it makes sense to use this knowledge to inform your
Statement 2 has no such frequentist argument. We cannot 'play' this
year over and over again counting the number of times in which Martians
land. We can only provide a subjective measure of uncertainty based on
our current state of knowledge.
Some people (including even clever ones) feel comfortable with the
frequentist approach but so uncomfortable with the subjective approach
that they reject it as invalid. Their primary objections are that:
the subjective measure cannot be
different experts will give different
The problem with these
objections is that they apply just as much to
the frequentist approach. Even in the coin tossing example, if we toss
a coin 10,000 times it is almost certain that Heads will NOT come up on
exactly 5,000 occasions. Moreover, different 'experts' running
different sequences of 10,000 tosses would almost certainly
arrive at different numbers of Heads. Does that make the 50% figure
invalid? In fact, things can be much murkier. It has been demonstrated
that if you toss a fair coin in a particular way it is possible to
ensure that it does not flip over at all. This means you can ensure
that the toss is always a Head. So, if you observe what is assumed
to be a fair coin being tossed 100 times and each time the result is
Heads, what do you believe are the chances of the next coin being
Heads? The theoretical (frequentist) argument given a fair coin
wouldinsist the answer is still 50%. But surely you have to take
account of the data you have seen and even a frequentist must make some
subjective assumptions in this case (such as about the way the coin is
being tossed) in order to arrive at a rational prediction. So, it seems
that even the most definitive frequentist example (tossing a fair coin)
inevitably involves a range of assumptions and subjective judgements.
If you still believe that the frequentist approach is somehow pure
and superior then consider the following two statements which are
less extreme versions of the frequentist versus subjective examples of
statements 1 and 2 above:
3. "There is a 50.9% chance that a baby born in
the UK is a girl"
4. "There is a 5% chance of Spurs winning the FA
Cup next year"
There is no doubt that statement 3 is explained by a frequentist
argument: Over the last 100 years 50.9% of all births recorded in the
UK have been girls.
There is also no doubt that statement 4 has no such
frequentist explanation (and hence must be subjective) since there is
only one FA Cup next year and we cannot somehow play the tournament
many times in the same year and count the number of occasions on which Spurs win.
But if we dig a little deeper here, things get rather murky. The 50.9%
figure in statement 3 is actually based on many years of data that may
disguise crucial trend information. Suppose we discover that the
percentage of girls born is increasing; say a hundred years ago 48.5%
of babies were girls compared with 51.2% last year. Then surely the
probability of a randomly selected new born baby being a girl now is
higher than 50.9% (and higher than 51.2% if the figures have been
steadily increasing). And what exactly do we mean by a 'randomly'
selected baby. Surely what we are most interested in are specific
babies such as "the next baby born to Mrs Roberts of 213 White Hart
Land, London N17". In that case the frequency data may need to be
'adjusted' to take account of specific factors relevant to Mrs Roberts.
Both the general trend adjustments and the case specific adjustments
here clearly require the subjective judgment of relevant experts. But
that means, according to the frequentists, that their own approach is
no longer valid since, as we saw above:
the measure cannot be validated
different experts will give different
Now look at statement 4 in comparison. Although it is true that we
cannot play the FA Cup more than once this season, we can nevertheless
consider the number of times Spurs won the FA Cup in the last 100 years
as a key factor informing our subjective judgement. Of course past form
(especially of the distant past) is not a strong indicator of
current form, but can we say with true certainty that the situation was
any different for the past 'form' of babies born? It is not infeasible
that drastic changes in national figures could result from sudden
environmental changes. And just as Spurs might invest in the world's
greatest players to increase their chances of winning the FA Cup this
year, so a particular mother might apply a range of techniques to
dramatically increase or decrease the chances of having a girl.
Whatever anybody's objection to subjective measures, like it or not,
they are used so extensively that the fabric of modern society would
break down without them. Hence bookies will provide 'odds' on events
(such as Spurs winning the FA Cup) based on subjective measures, while
insurance companies will do the same in determining policy premiums and
governments the same when determing economic policies.
The frequentist approach for measuring uncertainty is all well and good
providing that we have been able to record accurate information about
many past instances of the event. However, most uncertain events of
interest do not have such historical databases associated with them,
and even where relevant historical data does exist it must still
usually be informed by subjective judgements before it can be used for
measuring uncertainty. Hence, generally we cannot rely on the
frequentist approach to measure uncertainty.
The subjective approach accepts unreservedly that different
people (even experts) may have vastly different beliefs about the
uncertainty of the same event. Hence Norman's belief about the chances
of Spurs winning the FA Cup this year may be very different from
Daniel's. Norman, using only his knowledge of the current team and past
achievements may rate the chances at 10%. Daniel, on the other hand,
may rate the chances as 2% based on some inside knowledge he has
about key players having to be sold in the next two months.
Hence the subjective approach is
always based on some prior body of
knowledge. In this sense subjective measures of uncertainty
are always conditional on this prior knowledge. The subjective approach
is also called the Bayesian approach, because only in the Bayesian
approach is there a rigorous way of reasoning about such conditional
knowledge, and only the Bayesian approach can provide a rational way of
revising your beliefs in the light of evidence (such as when you have
seen 100 consecutive Heads tossed on an apparently fiar coin).