Math · Introductory statistics · Concept

The binomial distribution and expected value

A binomial random variable counts the successes in a fixed number $n$ of independent trials that each succeed with the same probability $p$ . The chance of exactly $k$ successes is $C (n, k) p^{k} {(1 - p)}^{n - k}$ . On average there are $n p$ successes, with variance $n p (1 - p)$ , and when $n$ is large enough a normal curve with that mean and standard deviation approximates the whole distribution.

A concrete case

Three free throws

A player makes $70 %$ of her free throws, and each shot is independent of the others. She takes three. What is the chance she makes exactly two?

Write M for a make and X for a miss. Two makes can happen three ways, MMX, MXM and XMM. Each way has probability $0.7 \times 0.7 \times 0.3 = 0.147$ , because independent chances multiply. The three ways cannot happen together, so their probabilities add, giving $3 \times 0.147 = 0.441$ .

The table sorts all eight outcomes by the number of makes, and the four totals add to $1$ . Every row has the same shape, a number of arrangements times one arrangement’s probability. That shape is the binomial formula.

3 \cdot {0.7}^{2} \cdot 0.3 = 0.441

Makes in three free throws, each made with probability $0.7$
Makes	Arrangements	Each	Total
$0$	XXX	$0.027$	$0.027$
$1$	MXX, XMX, XXM	$0.063$	$0.189$
$2$	MMX, MXM, XMM	$0.147$	$0.441$
$3$	MMM	$0.343$	$0.343$

The rule

When a count is binomial

A Bernoulli trial is one attempt with two outcomes, called success and failure, where success has probability $p$ . A count $X$ of successes is binomial when four conditions hold. The number of trials $n$ is fixed in advance, each trial is a success or a failure, the trials are independent, and $p$ is the same on every trial. Then $X \sim B (n, p)$ .

Guessing on $20$ multiple-choice questions fits. Dealing $5$ cards from a deck and counting aces does not, because each card dealt changes the chances for the next.

X \sim B (n, p)

The probability of exactly $k$ successes

Choose which $k$ of the $n$ trials succeed. The number of ways is the binomial coefficient $C (n, k)$ , read “ $n$ choose $k$ ”. Each way has probability $p^{k} {(1 - p)}^{n - k}$ , and the ways are disjoint, so their probabilities add.

P (X = k) = (\begin{matrix} n \\ k \end{matrix}) p^{k} {(1 - p)}^{n - k}, (\begin{matrix} n \\ k \end{matrix}) = \frac{n!}{k! (n - k)!}

\begin{array}{c} P (X = k) = (\begin{matrix} n \\ k \end{matrix}) p^{k} {(1 - p)}^{n - k} \\ (\begin{matrix} n \\ k \end{matrix}) = \frac{n!}{k! (n - k)!} \end{array}

Expected value, variance and standard deviation

The expected value $E (X)$ is the long-run average of $X$ over many repetitions of the whole experiment. For a binomial count it is $n p$ . The variance, written $Var (X)$ or $σ^{2}$ , is $n p (1 - p)$ , and the standard deviation $SD (X)$ or $σ$ is its square root, in the same units as $X$ . The expected value need not be a possible count. Five trials with $p = 0.1$ give $0.5$ expected successes.

E (X) = n p, Var (X) = n p (1 - p), σ = \sqrt{n p (1 - p)}

\begin{array}{c} E (X) = n p \\ Var (X) = n p (1 - p) \\ σ = \sqrt{n p (1 - p)} \end{array}

Ranges and “at least one”

A probability for a range of counts is a sum of $P (X = k)$ terms, one for each count in the range. When the range covers most of the possibilities, subtract the rest from $1$ instead. “At least one success” is everything except “no successes”.

P (X \geq 1) = 1 - {(1 - p)}^{n}

The normal approximation

When $n$ is large, the binomial probabilities form a bell shape, and a normal distribution with mean $n p$ and standard deviation $\sqrt{n p (1 - p)}$ approximates them. A common rule of thumb asks for at least $10$ expected successes and $10$ expected failures, $n p \geq 10$ and $n (1 - p) \geq 10$ . Some courses accept $5$ .

A count moves in whole steps and a normal curve is continuous. Each count $k$ stands for the interval from $k - 0.5$ to $k + 0.5$ , so widen a range by half a unit. $P (X \geq 60)$ becomes the area to the right of $59.5$ . That adjustment is the continuity correction.

Why it works

Why the formula has three parts

The trials are independent, so the chance of one particular arrangement is a product of $n$ factors, $p$ for each success and $1 - p$ for each failure. Multiplication ignores order, so every arrangement with $k$ successes has the same product, $p^{k} {(1 - p)}^{n - k}$ . Counting the arrangements means choosing which $k$ of the $n$ positions hold the successes, which can be done $C (n, k)$ ways. Different arrangements cannot happen at once, so their probabilities add.

All the probabilities together add to $1$ . By the binomial theorem, the sum of $C (n, k) p^{k} {(1 - p)}^{n - k}$ over every $k$ is ${(p + (1 - p))}^{n}$ , which is $1^{n} = 1$ . The distribution takes its name from that theorem.

\sum_{k = 0}^{n} (\begin{matrix} n \\ k \end{matrix}) p^{k} {(1 - p)}^{n - k} = (p + (1 - p))^{n} = 1

\begin{array}{c} \sum_{k = 0}^{n} (\begin{matrix} n \\ k \end{matrix}) p^{k} {(1 - p)}^{n - k} \\ = (p + (1 - p))^{n} = 1 \end{array}

Why $n p$ and $n p (1 - p)$

A binomial count is a sum of $n$ small counts, one per trial, each $1$ for a success and $0$ for a failure. One trial averages $1 \times p + 0 \times (1 - p) = p$ . The average of a sum is the sum of the averages, so $n$ trials average $n p$ .

One trial’s variance is its mean square minus its squared mean, a shortcut you get by multiplying out the average of ${(X - p)}^{2}$ . Its square equals itself, because $0^{2} = 0$ and $1^{2} = 1$ , so its mean square is $p$ and its variance is $p - p^{2} = p (1 - p)$ . For independent trials the variances add too, giving $n p (1 - p)$ .

The product $p (1 - p)$ is largest at $p = 0.5$ , so a count is most spread out when success and failure are equally likely. With $n = 20$ , the variance is $1.8$ at $p = 0.1$ and $5$ at $p = 0.5$ .

The method

A binomial probability in five steps

The table finds the chance of exactly $4$ heads in $6$ fair coin flips. For a range of counts, add the terms one by one, or subtract the terms you do not want from $1$ , whichever list is shorter.

(\begin{matrix} 6 \\ 4 \end{matrix}) {(0.5)}^{4} {(0.5)}^{2} = \frac{15}{64}

Exactly $4$ heads in $6$ fair flips
Step	What to do	For $4$ heads in $6$ flips
1	Check the four conditions	$6$ flips, heads or tails, independent, $p = 0.5$ every time
2	Name $n$ , $p$ and $k$	$n = 6$ , $p = 0.5$ , $k = 4$
3	Count the arrangements	$C (6, 4)$ $= 15$
4	Multiply by $p^{k} {(1 - p)}^{n - k}$	$15 {(0.5)}^{4} {(0.5)}^{2}$ $= 15 / 64$ $\approx 0.234$
5	Check the size	$E (X) = 3$ and $4$ is close to it, so a sizable probability makes sense

The normal approximation in four steps

The table approximates the chance of $60$ or more heads in $100$ fair flips, the third worked example.

$60$ or more heads in $100$ fair flips
Step	What to do	For $60$ or more heads
1	Check $n p \geq 10$ and $n (1 - p)$ $\geq 10$	$50$ and $50$
2	Find the mean and the standard deviation	$n p = 50$ and $\sqrt{100 \cdot 0.5 \cdot 0.5}$ $= 5$
3	Widen the count by half a unit toward the region you want	$60$ or more starts at $59.5$
4	Standardize and use the normal table	$z = 1.9$ , area $1 - 0.9713$ $= 0.0287$

Where it breaks

The four conditions are where the model breaks. Drawing without replacement from a small group breaks independence. The chance of no aces in $5$ cards dealt from a deck is about $0.659$ , while $B (5, 1 / 13)$ gives about $0.670$ . When the sample is under $10 %$ of the population, the difference is usually small enough to ignore.

A count with no fixed number of trials is not binomial. Flipping until the first head counts flips, not successes in $n$ flips, and it follows a different model, the geometric distribution. A count whose $p$ drifts is not binomial either, such as a player’s free throws as she tires.

More than two outcomes can still give a binomial count, as long as the question asks about one outcome against the rest. Rolling a six, or not, is a Bernoulli trial even though a die has six faces.

The normal approximation fails when $n p$ or $n (1 - p)$ is small. $B (40, 0.05)$ has $n p = 2$ , and its probabilities pile up against $0$ with a long right tail. A bell centered at $2$ would put area below $0$ , where no count can be.

The binomial theorem and the binomial distribution are different kinds of statement. The theorem is algebra, expanding ${(a + b)}^{n}$ . The distribution is probability: its terms are that expansion with $a = p$ and $b = 1 - p$ , each read as the chance of a count.

Check your answer

A binomial table has a shape you can predict before computing a single term, and that shape tests the terms.

Every probability lies between $0$ and $1$ , and for a small $n$ the whole table adds to $1$ . In the free-throw case, $0.027$ $+ 0.189$ $+ 0.441$ $+ 0.343$ $= 1$ .

The largest probabilities sit near $n p$ . For $B (20, 0.25)$ the peak is at $5$ , the expected value, and a count far from $n p$ should have a small probability.

Find the expected value a second way for a small table. Multiply each count by its probability and add. For the free throws, $1 (0.189)$ $+ 2 (0.441)$ $+ 3 (0.343)$ $= 2.1$ , which is $3 \times 0.7$ .

To have Chalk check the work, declare the model on the first row of a Math box, $X \sim B (20, 0.25)$ . Below it, three rows read ✓ against the binomial rules. They hold the mean, $E (X)$ $= n p$ $= 20 \cdot 0.25$ $= 5$ , the variance, $Var (X)$ $= n p (1 - p)$ $= 20 \cdot 0.25 \cdot 0.75$ $= 3.75$ , and one probability, $P (X = 5)$ $= C (20, 5) {(0.25)}^{5} {(0.75)}^{15}$ . A variance written as $n p$ reads ✗ with the message “Doesn’t match Var(X) = np(1 − p) for a binomial.”

Chalk reads a fractional $p$ stacked or with a slash, $B (4, 1 / 6)$ . It reads a rounded value on a row that multiplies out the count: under $P (X = 8)$ $= C (10, 8) {(0.8)}^{8} {(0.2)}^{2}$ , the row “ $= 45 {(0.8)}^{8} {(0.2)}^{2}$ $\approx 0.3020$ ” reads ✓. It reads one on a row that still holds $C (20, 5)$ too, as in $P (X = 5)$ $= C (20, 5) {(0.25)}^{5} {(0.75)}^{15}$ $\approx 0.2023$ . A wrong rounding reads ✗. A row such as $P (X = 0)$ $= {0.95}^{40}$ $\approx 0.1285$ reads ✓ too.

Where the binomial leads

The normal approximation is a special case of the central limit theorem. A binomial count is a sum of $n$ independent 0-or-1 counts, and sums of many independent pieces are close to normal. Abraham de Moivre found this case in 1733, long before the general theorem.

When $n$ is large and $p$ is small, with $n p$ held at some value $λ$ , the binomial probabilities approach the Poisson probabilities $e^{- λ} λ^{k} / k!$ . For $B (40, 0.05)$ , $λ = 2$ gives $P (X = 0)$ $\approx e^{- 2}$ $\approx 0.1353$ , near the exact $0.1285$ . Rare events in many trials, such as typos on a page, follow this pattern.

Dividing a count by $n$ gives a sample proportion $\hat{p} = X / n$ . Its mean is $p$ and its standard deviation is $\sqrt{p (1 - p) / n}$ , which shrinks as $n$ grows. That standard deviation is the root of a poll’s margin of error, and the start of confidence intervals for a proportion.

P (X = k) \approx \frac{e^{- λ} λ^{k}}{k!}, λ = n p

\begin{array}{c} P (X = k) \approx \frac{e^{- λ} λ^{k}}{k!} \\ λ = n p \end{array}

Common mistakes

Using the binomial model when the trials are not independent. Dealing $5$ cards and counting aces is not $B (5, 1 / 13)$ , because each card dealt changes the deck. The chance of no aces is about $0.659$ , not $0.670$ .
Leaving out $C (n, k)$ . ${(0.25)}^{5} {(0.75)}^{15}$ $\approx 0.000013$ is the chance of one arrangement of $5$ right answers. There are $C (20, 5) = 15, 504$ arrangements, and the full probability is about $0.2023$ .
Reporting $n p (1 - p)$ as the standard deviation. For $B (20, 0.25)$ , $3.75$ is the variance. The standard deviation is $\sqrt{3.75} \approx 1.94$ , in the same units as the count.
Reading $n p$ as a probability. Five tickets with $p = 0.1$ give $0.5$ expected wins, but the chance of at least one win is $1 - {0.9}^{5} \approx 0.41$ . Outcomes with two or more wins count extra in the average.
Computing “at least one” as $1 - P (X = 1)$ . The opposite of at least one success is no successes, so $P (X \geq 1) = 1 - P (X = 0)$ . For the raffle that gives $0.40951$ , not $1 - 0.32805 = 0.67195$ .
Skipping the continuity correction. $P (X \geq 60)$ for $100$ fair flips starts at $59.5$ on the normal scale, giving $z = 1.9$ and $0.0287$ . Starting at $60$ gives $z = 2$ and $0.0228$ , far from the exact $0.0284$ .
Using the normal approximation when $n p$ is small. $B (40, 0.05)$ has $n p = 2$ , well under $10$ . Compute exact binomial terms instead, such as $P (X = 0)$ $= {0.95}^{40}$ $\approx 0.1285$ .

Key terms

Binomial distribution: The distribution of the number of successes $X$ in $n$ independent trials that each succeed with probability $p$ , written $X$ ~ $B (n, p)$ : $P (X = k)$ $= C (n, k) p^{k} {(1 - p)}^{n - k}$ . Its mean is $n p$ and its variance $n p (1 - p)$ .
Bernoulli trial: One random attempt with exactly two outcomes, success and failure, where success has probability $p$ . Scoring a success as $1$ and a failure as $0$ , a single trial has expected value $p$ and variance $p (1 - p)$ . A binomial count adds up $n$ independent Bernoulli trials with the same $p$ .
Expected value: The long-run average of a random variable: each possible value times its probability, added up, $E (X) = \sum x \cdot P (X = x)$ . It is linear: $E (a X + b) = a E (X) + b$ and $E (X + Y) = E (X) + E (Y)$ .
Variance: The average of the squared distances from the mean, a measure of spread in squared units. The population variance divides by $N$ , and the sample variance divides by $n - 1$ .
Standard deviation: How spread out the values are around the mean, in the data’s own units: the square root of the variance. It describes individual values, not how precise the mean is.
Combination count: The number of ways to choose $k$ objects from $n$ when order doesn’t matter: $C (n, k)$ $= n! / (k! (n - k)!)$ . For example, $2$ people can be chosen from $4$ in $6$ ways.
Normal distribution: The symmetric, bell-shaped distribution set by its mean and standard deviation. Real data can look roughly symmetric without actually being normal.

Work through an example

A quiz has $20$ multiple-choice questions with four choices each, and a student guesses every answer at random. Let $X$ be the number of correct answers. Find the expected value, variance and standard deviation of $X$ , and the probability of exactly $5$ correct.