Agena Company Logo

Agena Company Logo

Professor Norman Fenton’s invited video appearance at the Annual Conference of the Society for Expert Witnesses (Studley Castle, Warwickshire, 6-7 October 2006).

The Society for Expert Witnesses made a video interview with Norman Fenton for their Annual Conference at Studley Castle. The Society was keen to get a proper explanation of the so-called “Prosecutor’s fallacy” whereby lawyers make incorrect deductions about probabilities that are known to have a profound impact on juries. The Society’s Press Officer Tom Magnum had sought out a number of statisticians and probability experts but was unable to get a simple explanation that he felt could be understood by lawyers and judges. He turned to Norman Fenton after reading an article Norman had written with Martin Neil about the subject of probability fallacies in legal reasoning.

Norman explained the prosecutor’s fallacy as follows:

“Suppose a crime has been committed and that the criminal has left some physical evidence, such as some of their blood at the scene. Suppose the blood type is such that only 1 in every 1000 people has the matching type. A suspect X who matches the blood type is put on trial. The prosecutor claims that the probability that an innocent person has the matching blood type is 0.1% (1 in a 1000). Person X has the matching blood type and therefore the probability that X is innocent is just 0.1%.

But the prosecutor’s assertion, which sounds convincing and could easily sway a jury, is wrong. And it is easy to see why. Let’s suppose that the crime could only have been committed by an adult male and that in the population there are 10 million adult males. Then from this population we would actually expect a very large number of people who have the matching blood type (about 10,000). If there is no evidence other than the blood to link X to the crime then X is no more likely than any of the other 9999 matching blood type men to have committed the crime. This means that the probability X is innocent is actually 99.99% which is rather different to the 0.1% claimed by the prosecution.

So what is the source of the fallacy and why do lawyers so commonly make it? It all boils down to a basic misunderstanding about probability (a misunderstanding which many intelligent people have because this kind of basic probability is never taught at schools). The misunderstanding is to assume that the probability of A given that we know B is true  (written P(A|B)) is the same as the probability of B given that we know A is true (written P(B|A)) . In this case let A be the assertion “Person X is innocent” and let B be the assertion “person X has the matching blood type”. What we are really want to know is P(A|B) (the probability of innocence given the evidence of matching blood type)  and this is what the lawyer claims is equal to 0.1%. But in fact, what we actually know is that P(B|A) (the probability of the matching blood type given innocence) is equal to 0.1%. The lawyer has simply stated the probability P(B|A) and claimed this is actually the probability P(A|B).

The fallacy becomes especially challenging when DNA evidence is used. In such cases P(B|A) can be extremely low, such as 1 in 10 million. When the lawyer wrongly asserts that the probability of innocence is therefore 1 in 10 million it seems especially convincing. But even in this case the probability of innocence is actually very high. Assuming again a population of 10 million people who could have committed the crime. Then it turns out that the probability of more than one person having the matching DNA is still actually quite high - about 0.46. So instead of the claimed 1 in 10 million probability of innocence the real probability is not much less than 1 in 2. In such circumstances the ‘beyond reasonable doubt’ criteria can hardly be claimed to be met.”

Here is a film containing an edited version of Norman Fenton's interview (this is a Quick Time file of about 6Mb and the Fenton interview is in the middle).

Link to Fenton and Neil paper:

Fenton NE and Neil M, ''The Jury Observation Fallacy and the use of Bayesian Networks to present Probabilistic Legal Arguments'', Mathematics Today ( Bulletin of the IMA, 36(6)), 180-187, 2000.

https://www.dcs.qmul.ac.uk/~norman/papers/jury_fallacy.pdf