AP Statistics Topic 3.8: Potential Errors When Performing Tests

01 · A decision can be wrong even when the method is valid

The concept in 60 seconds

A school tests whether more than $60\%$ of its students prefer an earlier lunch. Let $p$ be the proportion of all students at this school who prefer the change.

H_0:p=0.60

H_a:p>0.60

A random sample supplies evidence, but it does not reveal the exact population proportion. Sometimes a sample looks unusually supportive even when the true proportion is $0.60$ . Sometimes it fails to show convincing support even when the true proportion is above $0.60$ .

Type I error: reject a true $H_{0}$ . The test finds evidence for an increase that is not actually there at the null benchmark.

Type II error: fail to reject $H_{0}$ when the specified alternative is actually true. The test misses a real increase.

These are errors in the statistical decision, not necessarily mistakes in arithmetic. You can write the hypotheses correctly, check the conditions and calculate the $p$ -value correctly, yet still make $1$ of these errors because samples vary.

Power describes how well the test can detect a specified real departure from $H_{0}$ . Higher power means a smaller chance of missing that departure.

All school, library and factory contexts are fictional teaching examples. Numerical planning illustrations use explicitly stated models or supplied probabilities; they are not measured performance claims.

Quick check: does every rejection of

H_{0}

count as a Type I error?

No. It is a Type I error only when $H_{0}$ is actually true. Rejecting $H_{0}$ when the specified alternative is true is a correct decision. A decision alone does not tell us whether an error occurred.

02 · Cross the decision with the truth

Decision versus reality: $4$ possible outcomes

Keep two questions separate:

What did the test decide? Reject $H_{0}$ or fail to reject $H_{0}$ .
What is actually true? The null benchmark holds, or a specified departure in $H_{a}$ is real.

Visual guide 1:

4

outcomes from decision and reality

Cross the test decision with the population truth.
Test decision	Reality: $H_{0}$ true	Reality: specified $H_{a}$ true
Reject $H_{0}$	Type I error. Probability $\alpha$ .	Correct detection. Probability $\operatorname{Power}$ .
Fail to reject $H_{0}$	Correct nonrejection. Probability $1 – \alpha$ .	Type II error. Probability $\beta$ .

Each probability is conditional on its column’s assumed truth. For the school’s right-tailed question, compare the null benchmark $p = 0.60$ with a specified true increase such as $p = 0.65$ . The displayed columns are $2$ distinct hypothetical population models; they are not $2$ categories whose probabilities add across a row.

The reality columns describe different possible populations. We do not usually know which column applies in an actual study. The table explains the possible outcomes; it does not reveal the hidden truth after $1$ test.

A memory aid that keeps the logic correct

Type I: wrong rejection. We reject a null hypothesis that is true.
Type II: missed departure. We fail to reject even though the alternative’s departure is real.

“False alarm” can help with a Type I error, and “missed detection” can help with a Type II error. But an error’s meaning depends on $H_{a}$ . If the alternative claims an improvement, the false alarm is a false claim of improvement; it is not automatically an alarm about a defect.

Failure to reject is not acceptance. It says that this test did not find convincing evidence for $H_{a}$ at the chosen level. It does not establish $H_{0}$ as true.

Quick check: the alternative is real, but the test rejects

H_{0}

. Which error occurred?

No error. This is correct detection of the specified alternative. Its probability, under that true alternative, is the test’s power.

03 · Name the decision and what is really true

Describe both errors in context

A strong error description states what the test concludes or fails to detect, then states what the population is actually like. Avoid stopping at the labels “Type I” and “Type II.”

Visual guide 2: describe the error, not just its label

Type IFalse claim above $60\%$

The test finds convincing evidence for $p > 0.60$ when the school’s true $p$ is $0.60$ .

Type IIMissed real increase

The test fails to find convincing evidence for $p > 0.60$ when the school’s true $p$ is above $0.60$ .

Check the wordingDecision and actual truth

Name the preference proportion and all students at this school. Failure to reject does not assert that $p = 0.60$ .

The error description depends on $H_{a}$ . Reverse the direction for a less-than claim; include either direction for a not-equal claim.

School example: Type I error

“The test finds convincing statistical evidence that more than $60\%$ of all students at this school prefer an earlier lunch, when in fact the true preference proportion is $0.60$ .”

This is a wrong rejection of $H_{0}$ . The school could make a schedule change based on evidence that incorrectly suggests support exceeds the benchmark. The test does not guarantee any particular policy action; that is a separate decision.

School example: Type II error

“The test does not find convincing statistical evidence that more than $60\%$ of all students at this school prefer an earlier lunch, when in fact the true preference proportion is greater than $0.60$ .”

This is a missed real increase. The school could overlook a genuine preference for a change. Do not rewrite failure to reject as “the test concludes that at most $60\%$ prefer it.” That statement claims more than the test establishes.

For a less-than alternative

If a factory tests $H_{0}: p = 0.04$ against $H_{a}: p < 0.04$ , where $p$ is a batch’s defect proportion:

Type I: find convincing evidence that the batch’s defect proportion is below $4\%$ when it is actually $4\%$ .
Type II: fail to find convincing evidence of a reduction when the batch’s defect proportion is actually below $4\%$ .

For a not-equal alternative

If $H_{a}$ says $p \ne p_{0}$ , either an increase or a decrease can be a real departure. A Type I error claims a difference when $p = p_{0}$ . A Type II error misses a difference when $p$ really differs from $p_{0}$ .

Quick check: why is “Type II means concluding

H_{0}

is true when it is false” an overstatement?

The test fails to reject $H_{0}$ ; it does not prove or conclude that $H_{0}$ is true. Describe Type II as missing convincing evidence for a specified real alternative.

04 · Attach each probability to its condition

$\alpha$ , $\beta$ and power

The probabilities on this page are conditional: each assumes a particular truth about the population. The “given” condition is part of the meaning, not a detail to omit.

\alpha=P(\text{reject }H_0\mid H_0\text{ is true})

\beta=P(\text{fail to reject }H_0\mid\text{a specified alternative is true})

\operatorname{Power}=P(\text{reject }H_0\mid\text{that specified alternative is true})

\operatorname{Power}=1-\beta\quad\text{so}\quad\beta=1-\operatorname{Power}

The vertical bar $\mid$ means “given” or “assuming.” Read each probability with its condition. $\alpha$ is the significance level chosen before examining the data. $\beta$ , read “beta,” depends on the test design and the particular true alternative you want to detect.

Visual guide 3:

3

probabilities,

2

assumed truths

Null world $\alpha = 0.05$

Assume $p = 0.60$ . The nominal probability of rejecting a true null is $5\%$ .

Alternative world $\beta \approx 0.3420$

Assume $p = 0.65$ . In the illustrated plan, about $34.2\%$ of samples fail to reject.

Same alternative world $\operatorname{Power} \approx 0.6580$

Assume that same $p = 0.65$ . About $65.8\%$ of samples reject correctly.

These are the continuous school planning model’s probabilities explained in the next section. $\beta$ and power complement one another under the same true alternative. $\alpha$ belongs to a different assumed truth.

Keep the condition attached to each probability.
Term	Meaning	Illustrated school plan
$\alpha$ : significance level	Nominal $P(\text{reject }H_0\mid H_0\text{ true})$	$0.05$ under true $p = 0.60$
$\beta$ : Type II error probability	$P(\text{fail to reject }H_0\mid\text{specified }H_a\text{ true})$	Approximately $0.3420$ under true $p = 0.65$
$\operatorname{Power}$	$P(\text{reject }H_0\mid\text{specified }H_a\text{ true})$	Approximately $0.6580$ under the same true $p = 0.65$
$1 – \alpha$	Nominal probability of correct nonrejection under $H_{0}$	$0.95$ under true $p = 0.60$
$p$ -value	Observed extreme-result probability assuming the null model	Computed after observing a sample; not $\alpha$ , $\beta$ or power

Why do $\beta$ and power add to $1$ ?

Under the same specified true alternative, the test either rejects $H_{0}$ or fails to reject $H_{0}$ . Correct detection and a Type II error are complementary outcomes. For example, a supplied power of $0.80$ means $\beta = 1 – 0.80 = 0.20$ .

Why do $\alpha$ and $\beta$ not generally add to $1$ ?

They are calculated under different population truths. $\alpha$ assumes $H_{0}$ is true. $\beta$ assumes a specified alternative is true. They are not the $2$ parts of $1$ probability total. A test might have $\alpha = 0.05$ and $\beta = 0.20$ ; there is no requirement for their sum to equal $1$ .

Power is not the probability that $H_{a}$ is true

A power of $0.80$ at a specified true $p$ means that the test would reject $H_{0}$ in about $80\%$ of repetitions if that $p$ were the actual population proportion. It does not mean there is an $80\%$ chance that $H_{a}$ is true after observing data.

Different roles: $\alpha$ is a planned threshold; a $p$ -value comes from the observed sample under the null model; power is a detection probability under a specified alternative model. None is the probability that a hypothesis itself is true.

In the AP test framework, $\alpha$ represents the Type I error probability. For a discrete proportion test that uses a normal approximation, the actual rejection probability at the null benchmark can differ slightly from the nominal $\alpha$ . The visual model below makes its approximation explicit.

Quick check: if power is

0.87

at the stated true proportion, what is

\beta

$\beta = 1 – 0.87 = 0.13$ . Under that same true proportion and design, there is a $13\%$ probability of failing to reject $H_{0}$ . Do not use $1 – \alpha$ to calculate power.

05 · Change the assumed truth, keep the decision rule

See the error probabilities

Consider a planned school study with an SRS of $n = 400$ students from the school’s $10{,}000$ students, without replacement. The question remains $H_{0}: p = 0.60$ versus $H_{a}: p > 0.60$ , with $\alpha = 0.05$ selected in advance.

To examine detection, suppose the true population proportion were $p = 0.65$ . That is an assumed alternative for this planning illustration. It is not a truth established by a sample.

The sampling checks are appropriate for the illustration: an SRS is specified, $400 \le 1{,}000$ , and the expected counts are $240$ and $160$ under $p = 0.60$ , or $260$ and $140$ under $p = 0.65$ . Both distributions can be approximated by normal curves.

Visual guide 4: locate

\alpha

\beta

and power on the correct curve

      EXCELLENT  
   Based on 51 reviews 
    
 
 
 
   
 
 
       Posted on Google  
   
   0000003998 : Abdad Alam Shamim Alam 
 
 
 
 Trustindex verifies that the original source of the review is Google. This is the bestest institution I have ever come to and I love it very much.
  
 
      Posted on Google  
   
   Aliki S 
 
 
 
 Trustindex verifies that the original source of the review is Google. 
  
 
      Posted on Google  
   
   Ali 
 
 
 
 Trustindex verifies that the original source of the review is Google. Sly is him
  
 
      Posted on Google  
   
   Jitendra Kumar Kumawat 
 
 
 
 Trustindex verifies that the original source of the review is Google. 
  
 
      Posted on Google  
   
   Raahil Hasan 
 
 
 
 Trustindex verifies that the original source of the review is Google. 
  
 
      Posted on Google  
   
   Tina Mudarres 
 
 
 
 Trustindex verifies that the original source of the review is Google. The classes are amazing and my child learnt so much
  
 
      Posted on Google  
   
   alisha gadoya 
 
 
 
 Trustindex verifies that the original source of the review is Google. I have experienced a lot of good things, it has taught me so many things and from B/Cs I have gone to an A! This is wonderful and Mavish’s class is awesome.
  
 
      Posted on Google  
   
   smasher 123 
 
 
 
 Trustindex verifies that the original source of the review is Google. It’s sensational my kid went. First he only would get C now it’s all A’s really good and recommend
  
 
      Posted on Google  
   
   Mosa Al- Samaraie 
 
 
 
 Trustindex verifies that the original source of the review is Google. AMAZING PRICES GREAT TEACHERS STRAIGHT FORWARD LEARNING STEADY PACE IN TUTORING
  
 
      Posted on Google  
   
   Cael Dagnelie 
 
 
 
 Trustindex verifies that the original source of the review is Google. I had a great experience learning math with Ms. Mavish. She explains complex topics in a very clear and simple way, which made it easier for me to understand and enjoy the subject. Her patience and dedication really stood out, and she always made sure that everyone in the class was keeping up. I especially appreciated how approachable she was — I never felt afraid to ask questions, and she was always willing to help. Thanks to her teaching, my confidence in math has grown a lot. I’m really thankful for the effort she puts into every lesson!

NUM8ERS is one of finest tutoring institutes in UAE, Located in Al Barsha 1, Dubai. Close to DUBAI AMERICAN ACADEMY (DAA) & AMERICAN SCHOOL OF DUBAI (ASD).

Potential Errors When Performing Tests

By the end of this lesson, you should be able to:

The concept in 60 seconds

Decision versus reality: $4$ possible outcomes

A memory aid that keeps the logic correct