Bayes' theorem

In probability therory we call Bayes' theorem a relation between probability P(E|X) and a probablity of an inversly conditioned event. This relationship was published by Thomas Bayes in his article An Essay towards solving a Problem in the Doctrine of Chances in 1763.

Definition

Suppose we have two events and such that $P(E|F) \\neq 0$ and $P(F|E) \\neq 0$ , than

$P(E|F) = {{P(F|E)\\cdot P(E)} \\over {P(F)}}$

Suppose we have mutually exclusive and exhaustive events $E_{1},\\; E_{2},\\dots,\\; E_{n}$ such that $P(E_{i}) \\neq 0$ , for all , where $i \\leq 1 \\leq n$ , than

$P(E_{i}|F) = {{P(F|E_{i}) \\cdot P(E_{i})}\\over{P(F|E_{1}) \\cdot P(E_{1}) + P(F|E_{2}) \\cdot P(E_{2}) + \\cdots + P(F|E_{n}) \\cdot P(E_{n})}}$

Prove

The prove of the first equation stems from a definition of conditional probability, i.e.:

$\\begin{array}{lcr} P(E|F) = {{P(E \\cap F)} \\over {P(F)}} \\\\ P(F|E) = {{P(F \\cap E)} \\over {P(E)}} \\end{array}$

After multiplication of both equations by the denominator on their right sides we get:

$P(E|F)\\cdot P(F) = P(F|R)\\cdot P(E)$

To retrieve the desired relation, we divide the equation by P(F) .

The prove of the second relation is identical. We only have to use expression from the rule of total probability, i.e.:

$P(F) = \\sum_{i=1}^{n}P(F|E_{i})\\cdot P(E_{i})$

Example

Charles underwent X-ray screening of chest and the results came back positive for lung cancer. Determine the probability that Charles really has lung cancer, if you know that the false positive rate of the test performed is $5\\%$ and false negative rate is $40\\%$ . You also know that Charles works as an accountant and only 2 of 1000 of his colleagues have lung cancer.

$\\begin{array}{lcr} P(Test = positive | Cancer = present) = 0.6 \\\\ P(Test = positive | Cancer = absent) = 0.05 \\end{array}$

$\\begin{array}{lcr} P(present|positive) = { {P(positive|present) \\cdot P(present)} \\over {P(positive|present) \\cdot P(present) + P(positive|absent) \\cdot P(absent)}} = \\\\ = {{0.6 \\cdot 0.002} \\over {0.6 \\cdot 0.002 + 0.05 \\cdot 0.998}} = 0.023 \\end{array}$

Probability that Charles has lung cancer is only $2.3\\%$ .

Example 2

George underwent the same test as Charles (and the test also came back positive). But George worked for 20 years in a coal mine. George knows that $15\\%$ of his former colleagues have lung cancer. What is the probability that George has lung cancer too?

$\\begin{array}{lcr} P(present|positive) = {{P(positive|present)\\cdot P(present)} \\over {P(positive|present) \\cdot P(present) + P(positive|absent) \\cdot P(absent)}} = \\\\ = {{0.6 \\cdot 0.15} \\over {0.6 \\cdot 0.15 + 0.05 \\cdot 0.85}} = 0.679 \\end{array}$

Probability that George has lung cancer is almost $68\\%$ .

Sources

NEAPOLITAN, Richard E. Learning Bayesian Networks. Chicago, Illinois: Northeastern Illinois University, 2003, p. 12-29. ISBN 978-0130125347.

Definition

Prove

Example

Example 2

Sources

Place for your banner