-Values
A sample result can differ from a claim even when the claim is true. Learn how a -value measures how unusual that result would be under the null model—and how to explain the probability clearly.
By the end of this lesson, you should be able to:
- Explain a -value as a probability calculated assuming the null hypothesis is true.
- Choose the correct extreme region from a one- or two-sided alternative.
- Find a -value from a supplied -statistic or a null simulation.
- Interpret the probability using the population claim and the observed sample result.
- Explain why a smaller -value gives stronger evidence against the specified null model.
- Avoid confusing a -value with the probability that a hypothesis is true.
Before you start: Know , , and . Topic 3.5 develops the hypothesis setup and conditions. Review Topic 3.2 if sampling distributions need a refresher.
First time learning this? Follow the school example, compare the tail pictures, then try the first worked example.
Here to revise? Use the interpretation checklist, then attempt the practice before opening answers.
The concept in 60 seconds
A school investigates whether more than of its students prefer an earlier lunch. An SRS of students from students, taken without replacement, gives yes responses. The observed sample proportion is .
Let be the proportion of all students at this school who prefer an earlier lunch. The hypotheses are and . If the population really had support, different random samples would still give different proportions. The test asks how often the null model would give a result at least as high as .
Build the school’s null model at preference.
Ask about this high result and results even higher.
For , measure the null probability to the right.
The probability concerns possible sample results under . It is not the probability that itself is true.
A -value is the probability, assuming is true, of a test statistic at least as extreme as the observed statistic in the direction specified by .
For this school’s one-sided normal -test, the -value is approximately , or . Assuming the true preference proportion is , about of random samples of this size would produce a result this high or higher, according to the approximate null model.
That is a relatively unusual result under the model, providing evidence for a larger population proportion. It does not tell us the probability that the claim is true.
All contexts are fictional teaching examples. Normal tail probabilities are approximations for the one-proportion -test. The simulation later uses generated null-model statistics, not collected school responses.
Quick check: what do we assume when calculating the school’s -value?
Assume is true, together with the justified sampling model. We evaluate sample results under that assumption; we do not calculate the probability that the assumption itself is true.
What is the probability about?
The phrase “assuming is true” is essential. It tells us which model supplies the probability. The event is a sample test statistic in the extreme region determined by .
The bar means “given” or “assuming.”
Read this in order: first assume the null model; then find the probability of the specified sample results. Reversing that order changes the question. A probability about data under a hypothesis is different from a probability about the hypothesis after observing data.
The actual whole-school preference proportion.
The fixed value used to build the null model.
A tail probability under that model, for this observed result and alternative.
The sample statistic is the observed evidence. It differs from the population parameter, the null benchmark and the -value.
| Symbol or term | Meaning | School example |
|---|---|---|
| Unknown population proportion | Proportion of all school students preferring earlier lunch | |
| Null population benchmark | ||
| Observed sample proportion | ||
| Observed standardized test statistic | Approximately | |
| Random statistic under the null approximation | Standard normal: center , SD | |
| -value | Null probability of the specified extreme region | |
| Number of simulated null statistics | in the illustrated run |
Three different uses of “”
In this unit, alone usually names the unknown population proportion. is the numerical proportion specified by . The -value is a calculated probability about the test statistic. The words “-value” keep its role clear.
In the school example, is unknown, , and the right-tail -value is approximately . These quantities answer different questions.
Do not reverse the condition: “Assuming of students prefer earlier lunch, how unusual is this sample?” is the question the -value answers. “Given this sample, what is the chance that exactly prefer earlier lunch?” needs a different framework and is not answered by this -value.
Quick check: does mean there is a chance that ?
No. The calculation assumes . The is the approximate probability of a sample result as high as the observed result or higher under that assumption.
The null model and extreme results
A null distribution describes how the test statistic would vary across repeated samples if were true. For a justified one-proportion -test, the standardized statistic is approximately standard normal under : centered at with standard deviation .
The school result corresponds to a supplied observed statistic of . A positive means the observed proportion is above the null benchmark. A negative means it is below. Calculating from the sample is developed in Topic 3.7; here we use it to select and interpret probability areas.
What makes a result “as extreme or more extreme”?
- For : results at least as far toward larger proportions count. Use .
- For : results at least as far toward smaller proportions count. Use .
- For : departures at least as large in either direction count. For this symmetric normal model, use .
The -value includes a whole region of outcomes. It is not the probability of exactly the observed value. On a continuous normal model, point has probability ; a tail area can have positive probability.
Check that the reference model is appropriate
The school uses an SRS, satisfies , and has null expected counts and , both at least . These support the usual normal approximation.
A calculation cannot repair a biased sampling design or a poor probability model. If the normal approximation is unsuitable, a suitable exact or null simulation method may give a different probability. Do not treat a normal -test’s -value as an exact binomial probability.
Quick check: does the right-tail school -value include results above ?
Yes. It includes and higher in the approximate sampling model. “At least as extreme” includes more extreme results, not just the sample result we happened to observe.
Choose the correct tail area
Let follow the standard normal null approximation, and let be the observed statistic. The sign in tells you which probability to calculate.
| Alternative | Extreme region for | Normal-model calculation |
|---|---|---|