Estimating Probabilities Using Simulation

The concept in 60 seconds

A simulation uses a random mechanism to imitate a process. Instead of repeating the real situation, we repeat its model and see how often the event of interest occurs.

For example, random digits can represent made and missed basketball attempts. If our question concerns $3$ attempts together, we must simulate the full set $\text{of }3$ before deciding whether the event happened.

$1$ trial may need several random draws. $\text{Count the event }1\text{ time per trial}$, even if there are several individual successes inside that trial.

A basketball question

In an original fictional example, a player takes $3\text{ free throws}$. Our simplified model assumes that each attempt is made with probability $0.70$ and that attempts are independent: the model’s chance of making an attempt is unchanged by earlier results.

“$\text{At least }2$” includes exactly $2$ makes and all $3$ makes. A trial ends after the $3\text{rd attempt}$. We record Yes if there are $2\text{ or }3$ makes, and No if there are $0\text{ or }1$.

The $0.70$ input describes $1$ attempt. Our target concerns a set of $3$ attempts. These are different events, so do not automatically give $0.70$ as the answer to the question.

The simulation estimates a probability for the stated model. It does not establish that a real player’s success rate stays constant or that real attempts are independent. Those assumptions must fit the situation before the model’s results are used outside this example.

Key ideas and notation

A model probability and an estimate

The model gives a $0.70$ chance for a make on an individual attempt. After simulating sets $\text{of }3$, the relative frequency of event hits estimates the chance of at least $2$ makes per set.

You may see this estimate written as $\hat{p}$, read “p-hat.” The mark reminds us that it is an estimate from a finite run. It is not a promise about the next trial.

Probabilities and estimated proportions lie between $0$ and $1$, or between $0\%$ and $100\%$. A ratio above $1$ is a signal to check what you counted.

Build a valid simulation

1. Choose an appropriate random mechanism

Generate random integers from $0$ through $9$, inclusive, with each digit equally likely. Map $7$ digits to Make and $3\text{ to Miss}$. For this independent-attempt model, generate each new digit independently; repeated digits are allowed.

Digits $0\text{-}6$ contain $7$ possible values, $\text{including }0$. Digits $7\text{-}9$ $\text{contain }3$. Thus the mechanism models Make with chance $\frac{7}{10} = 0.70$ and Miss with chance $\frac{3}{10} = 0.30$.

A fair coin mapped to Make and Miss would give $50\%$ and $50\%$, so it would model a different player. A random mechanism should match the stated probabilities, not just produce $2\text{ possible labels}$.

2. Define one complete trial

Generate $3$ digits, translate each into $\texttt{M}$ or $\texttt{X}$, and count the makes. Record whether the count is $\text{at least }2$. Stop that trial after its $3\text{rd attempt}$, then begin a new trial using the same rules.

3. Repeat and record consistently

Choose a total number of trials before running the simulation. Use enough repetitions for a useful estimate. Keep the random mechanism, assumptions, event definition and trial boundary unchanged throughout the run.

A random-number generator, a random-digit table, or an appropriate physical device can supply randomness. Reading $3$ digits per trial from a random-digit table is valid here. Picking numbers you think “look random” can introduce patterns and is not a substitute for a random mechanism.

4. Decide whether replacement belongs in the model

For the basketball model, a digit must be available again on every attempt. If using $10$ digit cards physically, return and mix the card after each draw. Removing cards would change the next attempt’s chances and create dependence.

Replacement is not a universal rule. When a real situation draws distinct objects without replacement, a faithful simulation must reproduce that dependence within a trial. To make successive whole trials independent, reset to the original setup between trials.

Read and estimate from a simulation log

The following log comes from a computer-generated example run of the basketball model. $4$ trials are shown as cards; open the full $10\text{-trial}$ log to trace every result. $\texttt{M}$ means Make and $\texttt{X}$ means Miss.

There are $7$ event hits in $10$ complete trials, so the estimate from this small run is $\frac{7}{10} = 0.70 = 70\%$. The denominator is $10$ sets, not $30$ attempts. The $\text{event was scored }1\text{ time}$ for each complete set.

This small estimate happens to equal the input make probability, $0.70$, but the $2$ numbers describe different events. Longer simulated runs need not keep that equality. In the same generated run, the event occurs in $77$ of the first $100$ trials and $791$ of the first $1{,}000$ trials, giving estimates of $77\%$ and $79.1\%$.

Cumulative totals and separate batches

The $10$-trial, $100$-trial and $1{,}000$-trial summaries are prefixes of the same run. Do not add them: the first $10$ trials are already included in the $\text{first }100$, and all of those are included in the $\text{first }1{,}000$.

When combining separate runs that use the same model and event, add event hits and add completed trials. Do not average percentages unless the run sizes are equal. Keep incomplete trials separate until they can be completed under the defined rules.

Understand the long run

The probability of an outcome or event describes its long-run relative frequency. A finite simulation gives an estimate, which varies from run to run.

The law of large numbers explains why, for independent repetitions of the same chance process, the cumulative relative frequency tends to stabilize near the event’s probability as the number of trials grows. It does not require every new estimate to be closer than the previous one.

To see this clearly, switch to a separate fair-coin experiment. Here $1$ trial is $1$ toss, the event is Heads, and the model probability is $0.50$. These trials are not the $3\text{-attempt}$ basketball sets.

In this run, the Heads share is $\frac{4}{10} = 0.40$ after $10$ tosses and $\frac{7}{20} = 0.35$ $\text{after }20$. It moved farther from $0.50$ over that interval. Later, $\frac{494}{1{,}000} = 0.494$ is close to $0.50$. That illustrates the difference between a long-run tendency and a guarantee about the next step.

More trials do not repair a wrong model

If you wrongly map $8\text{ of }10$ digits to Make, the simulated player has an $80\%$ single-attempt make probability. Running thousands of trials estimates probabilities for that incorrect model more steadily; it does not turn the model into a $70\%$ player.

Earlier results do not make a correction “due”

For independent fair-coin tosses, the next toss still has a $0.50$ chance of Heads after a long run of Heads. A coin does not remember the earlier imbalance. Its proportion can move toward $0.50$ over many tosses without forcing the next toss to be Tails.

Six worked examples