Sampling Distributions and the Central Limit Theorem
sample gives answer. Repeated samples show how that answer can vary. Learn to build a sampling distribution, read a simulation, and explain why averages can look nearly normal even when individual values do not.
By the end of this lesson, you should be able to:
- Distinguish a population distribution, a sample distribution and a sampling distribution.
- Build an exact sampling distribution in a small example.
- Design a simulation that records statistic from each random sample.
- Describe simulated distributions using shape, center and spread.
- Explain what the Central Limit Theorem says about sample means.
- Distinguish drawing new samples from randomly reassigning treatment labels.
Before you start: Know what a mean, standard deviation, parameter and statistic represent. Review Topic 2.11: The Normal Distribution for bell curves and normal areas.
First time learning this? Start with the example, then follow the café simulation.
Here to revise? Use the quick checklist, then try the practice questions before opening the solutions.
The concept in 60 seconds
Suppose you randomly choose café customers and calculate their mean wait. A different random sample of customers will probably give a different mean. That is sampling variability: a statistic changes from sample to sample, even when the population stays the same.
A sampling distribution describes the possible values of a statistic and how likely they are under a specified sampling method and fixed sample size. Think of a graph where each dot represents sample’s answer.
The key change:
A dot showing customer’s wait belongs to a distribution of individual values.
A dot showing the mean of customers’ waits belongs to a distribution of sample means.
The Central Limit Theorem (CLT) helps explain why, with suitable independent random sampling and a sufficiently large sample size, a distribution of sample means can be approximately normal. The original wait times may still be strongly right-skewed.
All numerical examples here are original fictional teaching examples. The simulated café model is chosen to make sampling variability visible; it does not describe collected customer records.
Quick check: does sample of give me sample means?
No. It gives individual observations and sample mean. To obtain sample means, take separate samples and calculate a mean for each.
Three different distributions
Values: waits for everyone in the population, or the probability model for an individual wait.
Each value: wait, such as minutes.
Values: the individual waits in sample.
Each value: sampled wait, such as minutes.
Values: a statistic from each possible sample, approximated by repeated sampling.
Each value: mean of waits, such as minutes.
The horizontal axis can have the same units in all three graphs. Read what a plotted value summarizes before deciding which distribution you are looking at.
Parameter
A number describing the population or its probability model. It is fixed for that population.
is the population mean; is the population SD.
Statistic
A number calculated from a sample. Before sampling, its value can vary.
, read “x-bar,” is a sample mean; , read “p-hat,” is a sample proportion.
Sample size
The number of observations in sample. Keep fixed when building one sampling distribution.
means waits are averaged each time.
Repetitions
The number of samples generated in a simulation. is a convenient simulation label used in these notes.
means sample means are recorded.
You can build a sampling distribution for a mean, proportion, median, difference between means, or another statistic. The CLT statement about means should not be applied automatically to every statistic. A sample minimum, for example, need not become normal as grows.
Quick check: a histogram contains the waits from sample. Which distribution is it?
A sample distribution of individual wait times. It becomes a simulated sampling distribution of means only when its plotted values are means from repeated samples.
All possible samples: an exact sampling distribution
A tiny population has people’s recorded waits: minutes. Choose a simple random sample of different people, without replacement. Each unordered pair is equally likely. There are pairs.
| Selected waits (minutes) | Sample mean (minutes) | Probability of this sample |
|---|---|---|
Each upper dot is a person. Each lower dot is of the equally likely samples. The stacked dots at retain distinct samples; both panels use the same horizontal scale.
The population mean is minutes. The equally likely sample means are minutes. Their average is also minutes.
Do not remove the second . distinct samples produce it, so . Each of the other possible means has probability . The probabilities add to .
This is an exact sampling distribution because every possible sample under the stated design has been included. In a large population, listing them all is usually impractical, so we use a model or simulation.
The sampling method matters: With replacement, pairs such as become possible. The probabilities and possible means would change. State whether you sample with or without replacement before counting outcomes.
Quick check: what is in this example?
The qualifying means are and . They arise from equally likely pairs, so . The denominator is the number of possible samples, not the people.
Build and read a sampling simulation
For our café example, assume customer’s wait follows a right-skewed exponential model with mean minutes and SD minutes. The model has many short waits and some long waits. Draw observations independently from that same model.
You do not need to learn exponential probability formulas for this lesson. It is simply a specified population model that lets us generate positive, skewed data.
Generate independent waits from the same specified café model.
Add the waits and divide by . Record that mean in minutes.
Generate a fresh sample of . Repeat until you have means.
Plot the means. Count those satisfying your event and divide by .
For this plan, raw waits lead to plotted means. Keep the model, sample size and statistic fixed throughout the run.
A complete plan for
- Define repetition: generate independent waits from the exponential model with mean minutes.
- Calculate the statistic: average those waits.
- Record result: store the sample mean, measured in minutes.
- Repeat independently: start a fresh sample of , keeping the population model and sampling rule unchanged.
- Build the distribution: graph all recorded means and describe its shape, center and spread.
- Estimate an event: count the means that satisfy the requested condition and divide by the total repetitions.
What this lesson’s actual simulation produced
We generated independent samples for each , and . Each setting used fresh draws from the same population model. The random seed was 2,122,026, making this run reproducible.
| Sample size | Observed mean of the means (min) | Observed SD of the means (min) | Theoretical SD (min) |
|---|---|---|---|
For , of sample means were at least minutes. The simulation estimate is , or . This describes groups of customers whose average wait is at least minutes; it does not describe the percentage of individual customers waiting that long.
Another seed will usually produce a slightly different fraction. Simulated proportions approximate model probabilities; they are not exact just because the software displays many decimal places.
How to reproduce the model with technology
Use a random-number tool’s exponential generator with . Some tools instead request a ; check how the tool labels its input. If only uniform random numbers are available, an equivalent wait is , where is uniform on . Regenerate if the tool returns either endpoint, or .
Arrange generated waits into rows and columns. Calculate the mean of each row, then plot the row means. Do not plot all raw waits and call that a sampling distribution of means.
Quick check: repetitions with give how many raw waits and how many plotted means?
Raw waits: . Recorded sample means: . Each group of contributes mean.
The Central Limit Theorem
For independent, identically distributed observations from a population with a finite mean and finite positive variance, the standardized sample mean approaches a standard normal distribution as increases. The standardized mean uses the SD of the sample mean; represents the mean before the sample is drawn. In practice, this lets us approximate the shape of a sample mean’s sampling distribution by a normal curve when is sufficiently large.
“Identically distributed” means each observation comes from the same population model. “Finite variance” rules out unusual mathematical models whose variability is infinite.