ConceptProbability and Statistics used inMSIAI-written · not yet proofread

Bayes' Theorem

Also Bayes Theorem · Bayes' Rule · Bayes Rule · Bayes' Law · A-priori Probability · A-posteriori Probability · Satz von Bayes

Bayes' theorem P(A|B) = P(B|A) P(A) / P(B) turns the a-priori probability P(A) into the a-posteriori probability P(A|B) once the evidence B has been observed.

Know these firstConditional Probability

Properties#

Derivation: from the definition of conditional probability, $P(A, B) = P(A|B)P(B)$ and $P(A, B) = P(B|A)P(A)$ . Equating and dividing by $P(B)$ gives the theorem.
Density form: the same steps with $p(x|y) = p(x, y)/p(y)$ give $p(x|y) = \dfrac{p(y|x)\,p(x)}{p(y)}$ .
Use in MSI: updating a prior belief by evidence is the basis of Bayesian estimation (script chapter 5, “Maximum Likelihood and Bayesian Estimation”).
Named after Thomas Bayes (1701–1761).

Example#

Sources#

MSI script pp. 13–14 (§2.1, theorem and example), MSI Slides 02 > 2.1 Random Variables and Probabilities (not published)

Connections

Mentioned in inline links5

Naive Bayes ClassifierDBIS
Derivation. By Bayes’ Theorem, $\Pr(c \mid d) = \Pr(d \mid c)\,\Pr(c) / \Pr(d)$ . The denominator $\Pr(d)$ is the same for all classes and drops out of the $\arg\max$ . The naive assumption is that the words of $d$ are independent given the… §
Naive Bayes ClassificationDBIS
Prerequisites from the foundations: Bayes’ Theorem, Conditional Probability, Statistical Independence. §
Chain: Bayes’ Theorem turns $\Pr(c \mid d)$ into $\Pr(d \mid c)\Pr(c)/\Pr(d)$ ; $\Pr(d)$ is constant over classes and dropped; conditional independence of the words factorizes $\Pr(d \mid c)$ into the $p_{wc}$ . §
VL02 – Probability and Statistics in a NutshellMSI
The section introduces a Random Variable $X$ as a description of an experiment’s outcomes. $X$ has no value itself and only takes values $x$ . It then gives the rules for probabilities of events: the sum for mutually exclusive events and th… §
Bayes’ Theorem: a-priori → a-posteriori probability §
Conditional Probability
Product rule: $P(A, B) = P(A|B)\,P(B) = P(B|A)\,P(A)$ . Equating the two factorizations gives Bayes’ Theorem. §
Order matters: $P(A|B) \ne P(B|A)$ in general. In the antique store example of Bayes’ Theorem, $P(B|A) = 0.2$ but $P(A|B) = 2/3$ . §
Random Variables and Distributions
A random variable models a quantity that is not known before the experiment, such as a noisy measurement. It is described by probabilities of events, or more conveniently by a PDF. Joint, marginal and conditional densities describe several… §
Bayes’ Theorem — turning $P(B|A)$ into $P(A|B)$ : prior to posterior §

Search in these mentions →