Experimental Design
To learn whether a treatment makes a difference, plan a fair comparison. Learn how to assign treatments, reduce other sources of variation, and explain what an experiment’s results can support.
By the end of this lesson, you should be able to:
- Identify experimental units, treatments, explanatory variables and responses.
- Explain comparison, random assignment, replication and direct control.
- Recognize and justify completely randomized, randomized block and matched pairs designs.
- Distinguish control groups, placebos and blinding.
- Describe a complete random assignment procedure and the limits of a conclusion.
Before you start: Know the difference between selecting people for a study and assigning treatments to people already in it. Review Topic 1.12: Potential Problems with Sampling to revisit collection problems.
First time learning this? Follow the practice-method study, then compare the three designs and work through their assignment procedures.
Here to revise? Use the design checklist, then attempt the questions before opening the solutions.
The concept in 60 seconds
In an experiment, a researcher deliberately assigns treatments and measures a response. In an observational study, the researcher records what happens without assigning the treatment being investigated. Recording which app students already use is different from assigning students to use one of apps.
A useful experiment compares treatments while making the groups as comparable as possible. Chance decides assignments; multiple experimental units receive each treatment; relevant conditions are kept consistent. These choices help separate a treatment effect from other reasons the responses might differ.
Study the same material for .
Study the same material for .
This diagram specifies a design, not a study result. No students or scores in these teaching scenarios are from a real investigation.
Two random steps answer two different questions. Random selection asks, “Who enters the study?” Random assignment asks, “Which treatment does each experimental unit receive?” An experiment can have random assignment even when its participants are volunteers.
A sound randomized experiment can support a cause-and-effect conclusion when the results provide convincing evidence of a difference. It does not guarantee a treatment works, make every observed difference meaningful, or automatically justify applying the result to everyone.
A school compares
A fictional school recruits consenting students for a study of flashcard practice versus reading practice. Each student receives method for , then completes the same -question assessment. The outcome is the number of questions answered correctly. A treatment is assigned to each individual student, so that student is the experimental unit.
Experimental unit
The smallest unit that can be assigned a treatment independently.
Here: student. In another study it might be pot, machine, plot or entire classroom.
Subject
A human experimental unit.
The students are subjects. A seedling receiving a fertilizer treatment is an experimental unit, but not a human subject.
Explanatory variable / factor
The variable whose relationship with the response is being investigated. A manipulated explanatory variable in an experiment is often called a factor.
Here: assigned practice method. Its are flashcards and reading.
Treatment
A specific experimental condition assigned to a unit.
Here: of flashcard practice, or of reading practice.
Response variable
The outcome measured to assess what happened after treatment.
Here: number correct out of , rather than the name of the practice method.
Extraneous variable
Another characteristic or condition that may affect the response but is not the explanatory variable of interest.
Prior knowledge, room noise and study time could affect the assessment score.
If a study investigates , a treatment can be a combination of their levels. For example, comparing practice methods at study durations gives . The actual combinations, rather than the factor names alone, are the conditions being assigned.
Think first: suppose a school assigns each of to method and records every student’s score. Are the individual students the independently assigned experimental units?
Check your prediction
No. The classrooms receive the independent assignments. Student scores are measurements within those classrooms. Recording many students does not turn classroom assignments into dozens of independent treatment assignments. A design and its analysis must respect the level where treatment was assigned.
Four principles of a well-designed experiment
Compare at least treatment conditions. Reading provides a reference for flashcards.
Use a chance process so prior preparation and other characteristics are not deliberately tied to a method.
Use multiple students in each group so the comparison is not based on ‘s response.
Keep study duration, material and assessment conditions the same across units.
Comparison: provide a meaningful reference
A class’s score might improve after practice because of the practice, familiarity with the test, extra attention or ordinary changes over time. Comparing appropriately treated groups gives a reference for the method under investigation. A comparison can use an established method, a placebo or no treatment, depending on the question.
A comparison group does not have to receive “nothing.” Reading practice is an active comparison condition. If the question is whether flashcards outperform reading, removing the reading group would change the question.
Random assignment: reduce systematic group differences
Do not let the teacher give flashcards only to the strongest students or let students choose the method they already prefer. Random assignment helps distribute measured and unmeasured extraneous characteristics across treatment groups. Its goal is to reduce potential confounding.
Chance does not guarantee identical groups in one allocation. One group may happen to contain more experienced students. Randomization prevents the assignment rule from systematically using those characteristics to favor treatment; it does not erase every possible imbalance.
Replication: use multiple experimental units per treatment
in each method provide replication within this experiment. Multiple units help reveal unit-to-unit variability and improve the precision of a treatment comparison.
Asking the same student questions does not create independently assigned students. Measuring plant every day likewise does not replace having several independently treated plants or pots. Repeating an entire study can be useful, but the principle here concerns multiple units receiving each treatment within the study.
Direct control: keep chosen conditions consistent
Use the same study duration, content, testing room conditions, instructions and scoring rule where feasible. That reduces variation in scores caused by those conditions. You cannot hold every student’s prior knowledge constant, so randomization and sometimes blocking address differences that cannot simply be fixed.
Direct control and a control group are different ideas. Direct control standardizes conditions. A control group supplies a comparison. A study can standardize its test conditions while comparing .
Confounding: two explanations move together
Suppose every flashcard student studies in a quiet room, while every reading student studies beside a noisy corridor. Room conditions may affect scores and are tied to assigned method. If scores differ, you cannot separate the method’s contribution from the room’s contribution using that comparison.
Both method and room condition differ from Group B.
A score difference has more than one plausible explanation.
Repair: keep room conditions comparable and randomly assign the practice methods. If different rooms must be used, give both methods within each appropriate room block, with random assignment inside each block.
An extraneous variable is not automatically a confounder. It becomes a confounding concern when its relationship with treatment makes their effects hard to distinguish. Identifying “room noise” is stronger when you explain how the rooms differ between the treatment groups.
Compare three experimental designs
1. Completely randomized design
Assign treatments at random across the full set of experimental units, without first separating those units into blocks. The -student diagram above illustrates this design: choose students at random for flashcards, and give reading to the remaining .
This is a useful starting point when there is no important measured characteristic that calls for grouping before assignment. Equal group sizes are common, but a completely randomized design can deliberately use unequal sizes, such as units receiving A and receiving B. The allocation must still use an appropriate random procedure.
2. Randomized block design
If an existing characteristic is expected to affect the response, first form groups of similar units using that characteristic. These groups are blocks. Then randomly assign treatments within every block. Each block contains all treatments being compared.
For the school study, prior knowledge could affect the assessment. Form a lower-prior-score block of students and a higher-prior-score block of . Randomly assign students to each method inside each block. Compare the methods while accounting for the block structure.
Each method receives students overall. Giving all lower-score students A and all higher-score students B would confound method with prior knowledge.
Why block? It separates response variation associated with a useful blocking variable from the treatment comparison. Students within a block are more similar in prior preparation than the full group, which can make the comparison more precise. A student ID number is usually a poor block choice unless there is a reason it relates to the response.
Blocking is not stratified sampling. Stratification groups a population before selecting a sample; blocking groups experimental units before assigning treatments. Both organize similar units, but they operate at different steps and answer different questions.
3. Matched pairs design
A matched pairs experiment compares treatments using closely related responses. It is a special randomized block design. There are two common arrangements.
Match on a relevant characteristic first. Flip a fair coin within each pair to decide which member gets A; the other gets B.
Similar seedlings paired by starting height → one fertilizer A, one fertilizer B.
Each unit provides . Compare the responses within that unit.
uses both fonts on comparable passages in a randomly assigned order.
With , randomize the treatment assignment within each pair. When each unit receives both treatments, randomize the treatment order to address practice, fatigue or period effects. Always giving A first makes treatment and order hard to separate.
Using the is appropriate only if the treatments and measurement allow it. An earlier treatment might have a lasting effect on a later response. Randomizing order does not automatically eliminate that carryover. A learning method that permanently teaches the tested material may be better compared using separate students on equivalent conditions.
| Design | Before assignment | Random step | Useful when… |
|---|---|---|---|
| Completely randomized | Use the available units as one pool. | Assign treatments across the whole pool. | No useful blocking structure is specified. |
| Randomized block | Group by a characteristic relevant to the response. | Assign all treatments within each block. | A known source of variation can be accounted for. |
| Matched pairs | , or plan measurements per unit. | Assign A/B within each pair, or randomize A/B order. | can be compared using closely related responses. |
Control groups, placebos and blinding
A control group is a comparison group
A control group supplies a reference condition for evaluating a treatment of interest. It may receive an existing treatment, an inactive substitute or no treatment. Choose the reference to match the question and what is appropriate for the experimental units.
If a company asks whether a new printer setting improves performance over its standard setting, the standard setting is the relevant comparison. An experiment does not always need a placebo, especially when the units are objects and expectations are not part of their responses.
A placebo resembles treatment without its active component
A placebo is an inactive substitute designed to resemble the treatment being investigated. It helps make experiences and expectations more comparable. “Inactive” refers to the component under investigation; receiving attention or believing a treatment might help can still influence a person’s response.
For this topic, distinguish the average response under placebo from the average response under no treatment. Their difference describes the placebo effect. A placebo group’s improvement from its own baseline does not, by itself, isolate that effect because improvement over time can have other causes.
Illustration: in a fictional audio-feature experiment, suppose comparable randomized groups report mean focus ratings of with the feature, with a similar recording lacking that feature, and with no recording. The feature-versus-placebo difference is . The placebo-versus-no-treatment difference is . These invented means illustrate two different comparisons; they are not evidence that a real audio feature works.
Blinding is about who knows the assignment
If people know which condition they receive, expectations can affect behavior or reporting. If staff know assignments, their encouragement, treatment delivery or outcome assessment can differ. Blinding, also called masking, withholds assignment information from relevant people to reduce those influences.
Participants and interacting staff know the treatment assignments.
Participants are unaware while interacting staff know, or the relevant staff are unaware while participants know. State which role is blinded.
Participants and the research-team members interacting with them do not know which treatment each participant receives.
A coordinator can hold the assignment key without interacting with participants. Double-blind does not mean that nobody anywhere can know the code.
A coded answer sheet could blind the school study’s scorer to practice method. Students can see whether they use flashcards or reading, so you should not describe that study as double-blind. When full blinding is impractical, keep other procedures consistent and describe accurately who is unaware.
Human experiments require informed consent and appropriate oversight. Do not assign harmful or inappropriate conditions to make a statistical comparison. These examples are design exercises, not instructions to conduct treatment trials.
Five worked examples
All situations, allocations and response values here are original fictional teaching examples. A possible allocation shows what a valid assignment could look like; it is not a claim that a real trial has been run.
Example 1: Describe a completely randomized design
Task: compare fertilizers A and B using similar seedlings, each growing alone in its own pot. Assign pots to each fertilizer and measure each seedling’s height gain, in centimeters, after weeks.
- Identify units and treatments: each pot containing seedling is independently assigned a fertilizer. The treatments are A and B at specified application amounts.
- Label: give the pots distinct labels .
- Randomize: use a random-number generator to select distinct labels from , without replacement. If the generator repeats a selected label, ignore that repeat and generate another.
- Assign: the selected pots receive A; all remaining pots receive B. Record the allocation before applying either fertilizer.
- Control and measure: use the same pot type, soil mixture, watering schedule and light conditions. Measure starting and final height consistently; calculate for each seedling.
- Compare: compare the mean height gains for A and B, using analysis appropriate to the randomized design.
One possible A group:
Remaining B group:
This is completely randomized: treatments are assigned across the entire pool without prior blocks. There are experimental units per fertilizer, so the design has replication. Recording height on each of days would give more measurements of those same pots, not times as many independently assigned units.
Why the procedure is complete: it names the labels, random device, duplicate rule, group sizes and treatment mapping. “Randomly split the plants” leaves those decisions unstated.
Example 2: Block on a relevant existing difference
Task: the same fertilizer comparison uses pots: contain each of soil types. Soil type is expected to affect height gain. The goal is to compare fertilizers while accounting for soil.
- Form blocks: place the pots of each soil type in its own block before assigning fertilizer.
- Label within blocks: use , and . These labels keep the soil membership clear.
- Randomize separately: within each soil block, choose distinct pot labels at random, without replacement, for A. Assign B to the remaining pots in that block.
- Keep other conditions consistent: standardize pot size, watering, application amounts and light, and measure height gains the same way.
- Respect the blocks in the comparison: compare A and B within soil types and use an analysis that accounts for the blocks.
Each block has . Overall there are . This is a randomized block design, with soil type as the blocking variable.
Justification: soil is expected to affect height gain, so grouping by soil removes its between-block differences from the within-block fertilizer comparisons. Giving A only in soil and B only in soil would confound fertilizer with soil, rather than block effectively.
Example 3: Compare paired responses and randomize order
Task: typists each type a comparable passage in font A and a comparable passage in font B. The response is completion time in seconds. Describe the assignment structure and the direction of a paired difference.
- Identify the pairing: each typist provides a time under both fonts. A typist serves as their own comparison, reducing between-person typing-speed differences.
- Randomize order: for each typist, flip a fair coin. Heads means A first, then B; tails means B first, then A. Independent flips need not produce exactly of each order.
- Standardize the tasks: use comparable passage difficulty, equipment, timing rules and breaks. Randomize which comparable passage accompanies each font so font is not tied to one easier passage.
- Define a difference consistently: calculate for each typist. A negative value means B was faster for that typist.
- Analyze paired differences: preserve each typist’s pair instead of treating the as unrelated groups.
For illustration, of the typists might have A times of , , and , and corresponding B times of , , and . Their differences are , , and , with mean . Let denote the mean of these illustrated paired differences.
For those invented pairs, B is faster on average. That is a descriptive comparison, not proof of a population effect. Randomizing order addresses possible practice and fatigue; it does not guarantee they disappear. typists provide paired units and timing measurements.
Alternative matched pairs arrangement: pair different typists by similar baseline speed. Within each pair, flip a coin to assign A to one member and B to the other. Here each person receives only font; randomize within pairs, rather than treatment order.
Example 4: Find a confounded comparison
Task: a teacher assigns the morning class to a new practice method and the afternoon class to the existing method. The teacher uses the same assessment. Time of day is believed to affect performance. Can the comparison isolate the practice method?
- Identify what changes together: method and class time. The new method always occurs in the morning; the existing method always occurs in the afternoon.
- Explain the concern: time of day could affect scores, so a difference cannot be separated into a method effect and a time-of-day effect from this allocation.
- Recognize the unit issue: assigning entire class to each method gives assigned classroom per treatment, even if each class has many students.
- Suggest a relevant repair: if individual assignment is feasible without students’ treatments interfering, randomly assign both methods within each class-time block. If treatment must be assigned to classrooms, recruit multiple comparable classrooms at each relevant time and randomize methods within time blocks.
Conclusion: this plan does not provide a sound randomized comparison isolating method. A larger number of students in the would not remove the confounding or add independent classroom assignments.
Example 5: Identify who is actually blinded
Task: volunteers are randomly assigned to test . They can see which design they use. Staff distribute identical closed outer sleeves and collect coded ratings without seeing the package designs, so they cannot identify the design assignments. A separate coordinator keeps the code key.
- Check participants: they know which visible design they receive.
- Check interacting staff: they do not know the assignment behind each neutral code.
- Classify: this is a single-blind arrangement with the interacting staff blinded, rather than double-blind.
- Explain the benefit: staff are less able to give different instructions or attention based on knowing the treatment.
The coordinator holding a key does not undo blinding of the relevant staff. To describe a double-blind arrangement, both participants and the team members interacting with them would need to be unaware of assignments. When the treatment is visibly different, participant blinding may not be feasible.
What conclusions can the experiment support?
Random assignment and random selection support different parts of a conclusion. Assignment helps address competing explanations for a treatment difference. Selection helps justify representing a target population. Check both instead of treating the word “random” as permission for every claim.
A sound study can support a treatment-effect conclusion and generalization to the sampled population, subject to evidence and study conditions.
A sound experiment can support a treatment-effect conclusion for its units and plausibly similar units. Volunteer recruitment does not establish representation of everyone.
A sound observational study may estimate a population relationship, but random selection alone does not isolate a cause.
Describe the observed group and consider how it was obtained. Broad generalization and causal claims lack these design supports.
Explain why random assignment matters
For the practice-method study: “Randomly assigning the participating students to methods helps make the groups comparable in prior knowledge and other extraneous characteristics. This reduces potential confounding, so a convincing score difference from a well-conducted experiment can be attributed to the assigned method under these conditions.”
Do not write “any difference proves causation.” Sample responses vary. Later inference methods help assess whether an observed difference is convincing evidence beyond chance variation. Here, your task is to identify whether the design permits a causal interpretation, not to declare a treatment effective before looking at results.
Explain the limit of volunteer recruitment
The school recruited willing students rather than taking a random sample of every student in the country. A successful randomized comparison does not establish that its result applies to all ages, subjects, study durations or schools. Conclusions should fit the participating students and conditions, with cautious extension to students similar to them.
Justify a choice using a relevant characteristic
Design-justification frame: “I would use [design] because [specific characteristic] is expected to affect [response]. [Explain the grouping or pairing]. Randomly assign [treatments within those groups / treatment order], then compare [the appropriate responses].”
For a randomized block design, name the blocking variable and explain its relationship with the outcome. For matched pairs, explain what makes responses comparable and specify the random step. “Blocking is more accurate” or “matched pairs is best” without a reason tied to the study is incomplete.
Good assignment does not repair later missing outcomes, inconsistent treatment delivery or biased scoring. The complete collection process still matters; use the cautions from Topic 1.12 when assessing how the experiment was carried out.
Common mistakes and how to fix them
| Mistake | Why it fails | Better approach |
|---|---|---|
| “Random sample” means treatments were randomly assigned. | Selection describes entry into the study; assignment describes receipt of treatment. | Identify both procedures separately. |
| Letting participants choose treatments. | Preference and other characteristics may differ between groups. | Use chance to assign the treatment conditions. |
| Giving each block only treatment. | Treatment is tied to the blocking variable instead of compared within blocks. | Randomly assign all treatments inside every block. |
| Using student IDs as blocks without justification. | Arbitrary grouping need not explain response variation. | Choose an existing characteristic expected to affect the outcome. |
| Calling repeated measurements independent replication. | More observations of one unit do not create more independently assigned units. | Count units receiving independent assignments. |
| Using A first for every paired subject. | Treatment and order are tied together. | Randomize order and consider practice, fatigue and carryover. |
| “A control group gets no treatment.” | The reference could be an existing treatment or a placebo. | Describe what the comparison group actually receives. |
| “.” | Blinding refers to knowledge of assignment. | Identify participants’ and interacting staff’s knowledge. |
| “Randomization guarantees identical groups.” | Chance imbalances can remain. | Explain reduction of systematic differences and potential confounding. |
| “The volunteers represent everyone because assignment was random.” | Assignment does not make volunteer recruitment representative. | Separate causal evidence from the population scope. |
Eight practice questions with hints and solutions
Describe the design in context. For a proposed procedure, include enough detail for someone else to carry it out.
1. Identify the units and treatments
A printer manufacturer independently assigns each of printers . It measures the number of pages printed without an error in . Identify the experimental unit, factor, treatments and response. How many units receive each setting if the allocation is equal?
Hint
What receives the assignment? What is measured after that assignment?
Solution and explanation
The experimental unit is printer. The factor is speed setting; its are the treatments. The response is the number of error-free pages printed in . Equal allocation gives . Pages are measured outputs, not independently assigned printers.
2. Write a complete random assignment procedure
seedlings, each in a separate pot, will receive fertilizer A or B, with pots per treatment. Describe a completely randomized assignment using a random-number generator.
Hint
Give labels, a selection rule, a rule for repeats and the treatment mapping.
Solution and explanation
Label the pots . Generate labels uniformly at random from that range until distinct labels have been selected, ignoring repeat labels. Assign A to those pots and B to all remaining pots, and record the allocation. This randomizes across the whole pool. The full experiment should also standardize appropriate growing conditions and measure a clearly defined response.
3. Repair the blocked assignment
A researcher has seedlings: of . Variety is expected to affect growth. The researcher gives every variety- seedling A and every variety- seedling B, then says the study is blocked by variety. Describe a valid blocked alternative comparing A and B.
Hint
Can A and B be compared within every variety?
Solution and explanation
First form variety blocks of seedlings. Within each block, randomly choose distinct seedlings for A and assign B to the . All varieties then receive both treatments, giving seedlings per treatment overall. Variety is relevant because it may affect growth. The original allocation ties treatment to variety and does not provide within-variety comparisons.
4. Fix a matched pairs order problem
Each of volunteers performs comparable typing tasks with . Everyone uses A first and B second. Identify the design structure, the order problem and a relevant random step.
Hint
form a pair? What else changes when the layout changes?
Solution and explanation
The same volunteer’s responses form a pair, giving a matched pairs structure. Always using B second ties layout to order, so practice or fatigue could affect the comparison. Randomly assign each volunteer an or order, such as with a fair coin. Use comparable tasks and breaks, and consider whether learning one layout affects later performance. There are volunteers and measurements; these are not independent subjects.
5. Distinguish control and blinding
In a fictional coded-product study, the comparison group receives an inactive substitute resembling the tested product. Participants do not know their assignment, but the research staff interacting with them do. Identify the comparison treatment and the blinding arrangement. What would need to change for double-blinding?
Hint
Classify what is received separately from who knows.
Solution and explanation
The inactive look-alike is a placebo, used as a control condition. The experiment is single-blind with participants blinded. For double-blinding, the relevant research-team members interacting with participants must also be unaware of individual assignments, with coding and a separate key holder as appropriate.
6. Count actual replication
uses setting A and a different machine uses setting B. Each produces items, all of which are measured. A researcher claims independently assigned units per treatment. Assess the claim and identify an additional concern.
Hint
Was a treatment assigned independently to each item, or once to each machine?
Solution and explanation
As described, each machine receives a setting, giving assigned machine per treatment. Measuring outputs does not supply independent machine assignments. Machine differences are also tied to setting, so setting and machine effects are difficult to separate. Use multiple comparable independently assigned machines per setting, or an appropriate repeated-use design if switching settings is feasible and carryover can be addressed.
7. Separate a causal claim from broad generalization
adult volunteers are randomly assigned to . The study is well conducted and its analysis gives convincing evidence of a difference. The researcher says the better method causes higher scores for every student worldwide. What is supported, and what is overstated?
Hint
Which random step is present? Which one is missing?
Solution and explanation
Random assignment supports a causal treatment comparison for the participating units under the study conditions, with cautious extension to similar units. Volunteer recruitment does not establish representation of all students worldwide. A difference in group performance also does not imply that every individual benefits. State the target outcome, tested conditions and limited scope instead of the universal claim.
8. Choose and justify a design
A lab compares coatings on seedlings. Initial height strongly affects later height, and each seedling can receive only coating. Propose a matched pairs design and justify it.
Hint
Pairs do not require giving both treatments to the same unit.
Solution and explanation
Form pairs of seedlings with similar initial height. Within each pair, flip a fair coin to choose which seedling receives A; give B to its partner. Keep other conditions consistent and compare paired responses. Matching on starting height reduces its contribution to between-treatment differences. This arrangement is appropriate because can receive the treatments separately; the same seedling need not receive both coatings.
Quick revision checklist
Quick questions students often ask
Can volunteers take part in a randomized experiment?
Yes. Recruitment can use volunteers while treatments are randomly assigned among those volunteers. Random assignment helps the treatment comparison, but does not make those volunteers a random sample of a broad population.
Does a completely randomized design require equal group sizes?
No. A planned unequal allocation can still use random assignment across all units. State the intended sizes and use a procedure that produces them. Equal sizes are common, not part of the definition.
Does blocking replace random assignment?
No. Forming blocks organizes a known source of response variation. Treatment assignments must still be randomized within every block. Blocking is also different from directly holding a condition constant for everyone.
Does a before-and-after measurement automatically make a good experiment?
No. on the same unit are related, but without a suitable comparison and assignment plan, changes can reflect time, practice or other influences. In a randomized matched pairs experiment, specify the treatment conditions and the appropriate random assignment or order.
Final understanding check
A fictional school recruits consenting volunteers to compare flashcard and reading practice. Prior knowledge is expected to affect performance. The school forms of students. Within each block, it randomly chooses distinct students for flashcards and assigns the remaining to reading. Both methods use the same content and -minute study duration, followed by the same -question assessment, scored by number correct. Students can see their method. Scorers receive coded answer sheets and do not know the method assignments.
- Name the experimental units, treatments, response and blocking variable.
- Identify the design and count the units per treatment overall.
- Explain random assignment, replication and direct control in this plan.
- State why blocking is relevant.
- Describe the blinding arrangement.
- Explain the design’s support for causal conclusions and its generalization limit.
Reveal the full solution
1. Pieces: each independently assigned student is an experimental unit. The treatments are of flashcard practice and of reading practice. The response is the number correct out of . The blocking variable is prior score.
2. Design: randomized block. Each block contains students per method, so overall there are flashcard students and reading students. Both methods occur within both prior-score blocks.
3. Principles: random choice of the flashcard students within each block assigns method by chance, helping reduce potential confounding with other characteristics. Multiple students per method provide replication. The same duration, content, assessment and scoring rules directly control selected conditions. The reading method supplies the comparison for flashcards.
4. Blocking: prior knowledge is expected to affect performance. Comparing methods within prior-score groups separates some prior-knowledge variation from the method comparison and can improve precision. This is different from assigning all high-score students method.
5. Blinding: students know their practice method, while scorers do not. The scoring is blinded; describe it as a single-blind arrangement with outcome assessors blinded, not double-blind. The plan does not state that every staff member interacting with students is unaware.
6. Conclusions: if the experiment is carried out well and analysis gives convincing evidence of a difference, random assignment supports a cause-and-effect interpretation for the tested methods under these conditions. Volunteer recruitment does not establish that participants represent all students. Limit conclusions to the participating students and appropriately similar students, rather than everyone or every possible learning setting.
Final check: a higher sample mean alone does not establish a treatment effect. The design supplies the basis for a causal comparison; statistical analysis assesses the evidence.
Ready to move on? You should be able to draw the assignment structure, write a workable randomization procedure, explain why the design fits the study, and match the conclusion to its scope.
Continue learning
Topic 2.1: Tables and Graphs for Two Categorical Variables →
Build on one-variable summaries by using two-way tables and graphs to explore how two categorical variables are related.
Experimental Design completes the topics in Unit 1. Before starting Unit 2, check that you can distinguish population, sample, treatment assignment and response measurement.
Previous: Topic 1.12 — Potential Problems with Sampling · Review the designs · Review conclusion scope · Back to the lesson overview