The Investigative Question Revisitedand Data Collection
A useful answer starts with a clear question and a suitable study. Learn what to measure, how to recognize a census, survey or experiment, and how the way data are collected limits your conclusions.
By the end of this lesson, you should be able to:
- Connect an investigative question to its variables, analysis goal and intended population.
- Distinguish a census, an experiment and an observational study.
- Identify treatments, experimental units, explanatory variables and response variables.
- Recognize prospective studies, retrospective studies, surveys and possible confounding.
- Explain the different roles of random selection and random assignment.
Before you start: Know the difference between a population and a sample, and between a variable and a summary. Review Topic 1.2 if you need a reminder.
First time learning this? Follow the Cedar High question through the lesson, then classify the four worked studies.
Here to revise? Use the study checklist, then try the practice questions before opening the solutions.
The concept in 60 seconds
Choose the study to fit the question—and the conclusion to fit the study. Before collecting numbers, identify who or what you want to learn about, what you will record, and whether you want to describe, estimate, compare or investigate a possible effect.
What information?
Identify the variable or variables, with usable definitions and units.
What analysis goal?
Decide whether you want an estimate, a comparison or evidence about a stated claim.
What conclusion?
Name the population and decide whether a causal question needs an experiment.
Two different decisions: random selection chooses units from a population. Random assignment places participating units into treatment groups. One addresses who the study represents; the other strengthens a comparison of treatment effects.
Careful arithmetic cannot fix a study that measured the wrong variable or used the wrong group. This lesson connects the question you ask to the evidence you need.
Cedar High wants to understand morning travel
In earlier lessons, we described travel times from a fictional sample of Cedar High students. Now the school wants to plan a new study. Each question below needs its own data-collection plan.
| Question | Information to collect | Goal |
|---|---|---|
| What is the mean one-way travel time of all currently enrolled Cedar High students on a specified Tuesday? | Each selected student’s travel time in minutes for that Tuesday. | Estimate a population mean. |
| Is that population mean greater than ? | The same precisely defined travel-time variable. | Investigate a directional claim about the mean. |
| Is travel method associated with travel time? | Travel method and travel time for each selected student. | Investigate an association. |
“How do you get to school?” is a question asked to one participant. “Is travel method associated with travel time among Cedar High students?” is an investigative question about a group. One participant’s answer contributes data; it does not settle the group question.
Think first: if students already choose their own travel methods and the researcher only records them, is this an experiment?
Check your prediction
No. The researcher is observing existing choices, not assigning travel methods. It is an observational study. Students’ distance from school might be associated with both their chosen method and their travel time, providing an alternative explanation for a difference.
The three components of an investigative question
A strong question tells the reader what data are needed, what analysis is intended, and where the conclusion applies. The three components do not need to appear in a fixed word order.
“For all currently enrolled Cedar High students, what range of plausible values estimates the population mean of one-way home-to-school travel time, in minutes, on Tuesday 13 October?”
Data collection
One-way travel time, minutes, specified Tuesday. Each student contributes .
Analysis choice
Estimate the population mean with a range of plausible values: a confidence-interval goal.
Conclusion scope
All currently enrolled Cedar High students. Broad generalization needs a suitable sampling plan.
Question anatomy, not collected results. The date fixes the measurement occasion; this question does not estimate every possible school day’s travel time.
Make the variable measurable
Replace “Are students busy?” with a recorded quantity such as “minutes spent on assigned homework on the specified school night.” State when the timing starts and stops. If students report the value themselves, say so; a self-reported time and an automatically recorded time need not be interchangeable.
Match the wording to the intended analysis
| Goal | Example question | Parameter or relationship |
|---|---|---|
| Estimate | What range of plausible values estimates the proportion of Cedar High students who use the school bus on that Tuesday? | Population proportion ; confidence-interval goal. |
| Test “greater than” | Is the population mean travel time greater than ? | Population mean ; direction . |
| Test “less than” | Is the population proportion arriving late below ? | Population proportion ; direction . |
| Test “different” | Does the population mean travel time differ from ? | Population mean ; direction . |
| Investigate association | Are travel method and late-arrival status associated among Cedar High students? | Relationship between categorical variables; association versus independence. |
(“mu”) represents a population mean; represents a population proportion. The numerical mean or proportion calculated from a sample is a statistic used to learn about the corresponding parameter.
This is a preview of inference language. You are identifying the question’s goal here; calculating confidence intervals, test statistics and -values comes in later units. “Estimate” and “test a claim” are different goals.
State the intended population and kind of claim
“Students at Cedar High” and “all teenagers” are different populations. A study drawn from one school does not automatically represent every teenager. A question about a cause also calls for stronger evidence than a question about a relationship: compare “Is workshop attendance associated with scores?” with “Does assigning the workshop increase scores?”
Census, experiment or observational study?
Compare the measured group with the defined population.
Every population member contributes the relevant information.
Identify how those units were selected.
Look for a researcher assigning a condition.
Record the assigned treatments and measured response.
Record existing characteristics, choices or events.
Coverage and treatment assignment describe different features. A census can use a survey; “census” and “survey” are not opposites.
Census: information from the whole population
A school records the selected Tuesday’s arrival status for every currently enrolled student. That is a census of that population. If the population is instead all students across the district, information from one school is only part of it.
A completed census eliminates sampling variability for that population and occasion. A census can still be affected by recording errors or an unclear measurement definition. Missing responses mean complete census data were not obtained. Inviting everyone to respond does not guarantee information from everyone.
Experiment: the researcher assigns treatments
A teacher assigns participating students either a worked-example revision sheet or a retrieval-practice sheet, then measures each student’s score on the same quiz. The assigned revision condition makes this an experiment. Random assignment is a further design feature; assignment need not be random for the study to be called an experiment.
| Term | Meaning in this example |
|---|---|
| Experimental unit | An individual student, because each student receives an assigned condition. Human units may be called participants or subjects. |
| Explanatory variable / factor | Revision-sheet type. |
| Levels | Worked-example sheet and retrieval-practice sheet. |
| Treatments | The assigned revision conditions. |
| Response variable | Quiz score in points, measured after revision. |
If entire classes receive the assigned condition, the class is the experimental unit, even if scores are recorded for individual students. Identify the level at which a treatment is assigned.
More than factor: treatments are combinations
Suppose both sheet type and revision duration are assigned. Sheet type has levels; duration has levels: , and . If every combination is used, there are .
| Sheet type | |||
|---|---|---|---|
| Worked examples | Worked examples; duration | Worked examples; duration | Worked examples; duration |
| Retrieval practice | Retrieval practice; duration | Retrieval practice; duration | Retrieval practice; duration |
Observational study: record without assigning treatments
A researcher records the revision method students already chose and their subsequent quiz scores. The variables may be identical to the experiment’s variables, but the researcher did not assign the methods. It is an observational study.
“Explanatory” names the variable whose relationship with the response is being investigated. It does not establish that the variable causes the response.
Time direction does not determine whether treatments were imposed. Following people into the future can still be observational.
Prospective example: select students at the start of term, record their existing revision habits, then collect quiz outcomes over the next month without assigning habits.
Retrospective example: select students now and obtain last term’s attendance and score records.
Survey example: ask selected students the same set of questions about their chosen travel method and recorded Tuesday travel time. A survey is an observational study that collects responses from people. Asking about an existing choice does not impose a treatment.
Confounding, generalization and cause-and-effect
Suppose students who choose to attend a revision workshop also have higher quiz scores. A possible confounding variable offers another explanation for this relationship and must be associated with both workshop attendance and score.
A schematic example, not measured evidence or estimated effects
On a small screen, scroll the diagram sideways to see both branches.
Explain both links. For example: better-prepared students might be more likely to choose the optional workshop, and prior preparation might also relate to higher quiz scores. If preparation is related only to scores, you have not yet explained why it would confound the workshop comparison.
Random selection: who can the sample represent?
Suppose the school uses a random mechanism to choose students from its complete enrollment list. That supports generalizing to the school population from which they were selected, provided data collection is carried out appropriately. Random selection does not guarantee a perfectly representative realized sample or fix missing responses.
A group recruited from volunteers is not a random sample of all students. Conclusions are most directly about those studied; broader application is limited to students sufficiently similar to the participants, with that similarity justified. You cannot simply relabel the volunteers as representative of the entire school.
Random assignment: can treatment effects be investigated?
Randomly assigning participating students to revision conditions helps make the groups comparable with respect to other influences. In a well-designed experiment, this supports investigating a causal effect of the assigned condition. It does not make volunteers a random sample of every student.
| Randomly selected from the target population? | Treatments randomly assigned? | What the design supports |
|---|---|---|
| Yes | Yes | Population generalization and investigation of a causal effect, with suitable design, implementation and analysis. |
| Yes | No | Population generalization of descriptions or associations. The design alone does not establish a causal effect. |
| No | Yes | Investigation of a causal effect for the participating units; extension only to sufficiently similar units, not automatically the whole target population. |
| No | No | Descriptions or associations for the studied group, with cautious application to similar units. Neither broad population representation nor a causal effect is established by the design. |
A design permits a kind of conclusion; it does not predetermine the result. Random assignment does not prove that a treatment works. You still need appropriate outcome evidence and analysis. A large sample alone does not supply random selection or remove confounding.
Four worked examples
The scenarios below are invented for practice. Focus on the described selection and assignment processes, not on whether a story sounds scientific.
Example 1: Repair a vague question
Starting question: “Do students travel too far?” Cedar High wants to estimate travel time for currently enrolled students on Tuesday 13 October.
- Define the variable: elapsed one-way home-to-school travel time, in minutes, on that Tuesday. Distance would answer a different question.
- State the goal: estimate the population mean with a range of plausible values.
- State the population: all currently enrolled Cedar High students.
Improved question: “What range of plausible values estimates the mean one-way home-to-school travel time, in minutes, on Tuesday 13 October for all currently enrolled Cedar High students?”
A suitable plan selects students using a random mechanism from the complete enrollment list, collects the defined time for each selected student, and records how missing responses are handled. You have specified an estimation goal; you have not yet calculated an interval.
Example 2: A census can also be a survey
A library’s population is its registered members on 1 October. Staff obtain a response from every member to the same question about the number of books borrowed during September.
- Coverage: all members provide information, so this is a census of that defined population.
- Collection method: a standard question is asked to people, so it is also a survey.
- Treatment assignment: no borrowing conditions are imposed; the study is observational.
- Scope: the result describes those members’ September borrowing, not every library user in the city or all future months.
If only members responded to an invitation sent to all , the completed data would not cover the whole population. Do not confuse an intended census with complete census data.
Example 3: Assignments to classes
A researcher recruits volunteer classes. classes are randomly assigned a worked-example revision sheet, and a retrieval-practice sheet. Everyone receives the same revision time and final quiz.
| Feature | Answer |
|---|---|
| Study type | Experiment: the researcher assigns revision conditions. |
| Experimental unit | A class: whole classes receive an assigned condition. |
| Factor and levels | Revision-sheet type, with levels: worked examples and retrieval practice. |
| Treatments | The assigned sheet conditions. |
| Response | Individual quiz score in points; class-level assignment must be respected in the eventual analysis. |
Conclusion reasoning: random assignment supports investigating the causal effect of the assigned sheet in these participating classes, if the design and analysis are suitable. Volunteer recruitment does not support automatic generalization to every class in the district. Application beyond participants needs a sufficiently similar setting.
Many students measured within a class do not become many independently assigned experimental units. This lesson identifies the unit; methods for analyzing grouped responses are beyond this introductory example.
Example 4: Look forward, but stay observational
A researcher randomly selects students from Cedar High’s enrollment list. At the start of a month, students report their existing revision habits. The researcher then records their quiz scores over the month and never assigns a revision method.
- Study type: observational, because students keep their existing choices.
- Time direction: prospective, because the selected students are followed into the future.
- Variables: revision habit is the explanatory variable; later quiz performance is the response.
- Generalization: random selection supports describing relationships for the enrolled Cedar High population, assuming appropriate collection and analysis.
- Causal limit: there is no random assignment of revision methods. Prior preparation might relate to both habit choice and later performance, so a habit–score association alone does not establish an effect.
Model wording: “The prospective observational study can investigate an association between revision habits and later quiz performance among Cedar High students. Random selection supports population generalization, but existing habit choices and possible confounding prevent a causal claim from this design alone.”
Explain your choices in context
A label such as “observational” is only part of an answer. Point to the action in the story that justifies it, then connect the design to the intended conclusion.
Study-type sentence: “This is a [study type] because the researcher [specific assigned or recorded action].”
Scope sentence: “The results can reasonably apply to [population or suitably similar group] because [selection evidence]. A cause-and-effect conclusion [is / is not] supported by the design because [assignment evidence and relevant limitation].”
A data-collection plan in six decisions
- Target: define the population and the observation period.
- Unit: say who or what contributes ; identify the assignment unit in an experiment.
- Variables: name each variable, its type, units or categories, and its measurement rule.
- Process: explain how units enter the study and whether treatments are imposed.
- Quality: use consistent measurement, distinguish , and record incomplete responses.
- Claim: state the estimate, comparison or relationship sought and the population to which it could apply.
For the Cedar High travel-time question, “time taken” needs a consistent start and finish. A possible definition is elapsed time from leaving home to arriving at the school entrance, including waits along the way. Ask every selected student about the same specified Tuesday.
Keep collection appropriate and practical
Collect only the information the question needs, explain how responses will be used, and protect participants’ identifying information. Participation and assigned activities should be appropriate for the setting. A causal question does not justify assigning a harmful condition.
For example, investigate students’ existing sleep patterns observationally rather than requiring sleep deprivation. An observational design can answer an association question while leaving a causal question unresolved.
Try the distinction: “How many minutes did you spend revising yesterday?” is a participant question. “What is the mean revision time among enrolled students for that day?” is the group-level investigative question. Avoid mixing a -day measurement with a claim about an entire term.
Common mistakes and how to fix them
| Mistake | Why it fails | Better approach |
|---|---|---|
| “They used a survey, so it cannot be a census.” | Survey describes collection; census describes coverage. | Ask whether every member of the defined population supplied information. |
| “They randomly picked students, so this is an experiment.” | Random selection does not impose a treatment. | Look separately for researcher-assigned conditions. |
| “They compared groups, so this is an experiment.” | Existing groups can be compared observationally. | Identify who decided the group membership. |
| “Every experiment uses random assignment.” | Treatments can be assigned nonrandomly. | Distinguish the experiment label from the strength of its design. |
| “They followed people forward, so the study proves a cause.” | Time direction does not remove alternative explanations. | State whether treatments were assigned and how confounding is addressed. |
| Naming any score-related variable as a confounder | Association with the response alone is insufficient. | Explain a plausible link to both explanatory and response variables. |
| Generalizing a volunteer study to every student | Volunteers were not randomly selected from that population. | Limit the scope; explain any application to sufficiently similar students. |
| Changing “greater than” to “different from” | It changes the claim and intended alternative direction. | Keep the parameter, benchmark and stated direction aligned. |
Eight practice questions with hints and solutions
For each scenario, name the relevant feature and justify it using a detail from the description. These are original fictional scenarios.
1. Identify the three question components
“Is the proportion of currently enrolled Cedar High students who arrive after 8:00 a.m. on Tuesday 13 October greater than ?” Identify the needed variable, analysis goal and population.
Hint
The variable belongs to each student. The proportion summarizes the population. Keep the direction in the question.
Solution and explanation
Variable: late-arrival status on that Tuesday, defined as arrival after 8:00 a.m.; categorical yes/no. Goal: test a claim about population proportion , with direction . Population: all currently enrolled Cedar High students. The question concerns a population proportion; it does not ask about a causal effect.
2. An invitation sent to everyone
A club asks all members about September volunteering hours. Only respond. Are the completed responses a census of the -member population? Is asking these questions an experiment?
Hint
Separate intended coverage, achieved coverage and treatment assignment.
Solution and explanation
The club attempted a census, but the completed responses do not cover all members. It is a survey and an observational study because hours already volunteered are recorded without assigning a volunteering condition. The respondents may differ from the nonrespondents.
3. Factor levels and treatments
Plants are individually assigned watering schedules and light conditions. Every schedule–light combination is used. Identify the experimental unit, factors and number of treatments. Plant height after is measured.
Hint
The assigned condition is a combination of level from each factor.
Solution and explanation
Unit: an individual plant. Factors: watering schedule, with levels, and light condition, with levels. Treatments: . Response: plant height after , with the measurement unit specified in the collection plan. This is an experiment because conditions are assigned.
4. Prospective or retrospective?
In October, a researcher selects students and obtains their travel-time records from the previous September. No travel conditions are assigned. Classify the study.
Hint
Relative to selection in October, are the measured events in the past or future?
Solution and explanation
It is a retrospective observational study. Students are selected in October and past September information is gathered. The absence of assigned travel conditions makes it observational. The described dates do not establish how the students were selected, so do not invent random selection.
5. Explain both confounding links
Students who choose a paid revision workshop score higher than nonattenders. A student proposes prior preparation as a possible confounding variable. What connections must the explanation address?
Hint
Connect preparation separately to workshop choice and to quiz performance.
Solution and explanation
Prior preparation must be associated with attendance choice and with quiz score. For example, well-prepared students might be more likely to choose extra revision and might also score higher regardless of that workshop. This is a plausible alternative explanation, not proof that this confounding actually occurred.
6. Random selection without random assignment
A district randomly selects enrolled students from its complete list and records their existing transport choice and Tuesday arrival status. Can this design support district-wide descriptions? Does it establish that a transport choice causes lateness?
Hint
Answer the selection question and assignment question separately.
Solution and explanation
Random selection supports generalizing descriptions or associations to the enrolled district population, with appropriate collection and analysis. The study is observational: transport choices are recorded, not assigned. It does not establish a causal effect from the design alone. Home-to-school distance could relate to both transport choice and arrival status.
7. Volunteers with random assignment
are randomly assigned revision sheets. The researcher uses the same revision time and quiz for both groups. A classmate says, “Randomization means the result applies to every teenager.” Assess the statement.
Hint
Which process was random: recruitment or treatment assignment?
Solution and explanation
The statement confuses random assignment with random selection. Random assignment supports investigating a causal sheet effect in a suitable experiment, but volunteers do not automatically represent every teenager. Generalization is limited to participants and sufficiently similar students, with similarity justified. Random assignment also does not guarantee that the outcome data will show an effect.
8. Experiment without random assignment
A teacher assigns the first class of the day a worked-example sheet and the last class a retrieval-practice sheet. Both take the same quiz. Is this an experiment? Why would a score difference not automatically identify the sheet’s effect?
Hint
Researcher assignment determines the study label. Other differences determine whether the comparison isolates a cause.
Solution and explanation
Yes: the teacher imposes different revision treatments. However, assignment is not random, and sheet type is linked to class and time of day. Prior preparation or time-of-day conditions may differ between classes and offer alternative explanations. The design alone does not isolate a causal sheet effect.
Quick revision checklist
Quick questions students often ask
Can the same study be both a census and observational?
Yes. Record an existing characteristic for every member of a defined population without assigning a treatment. Census describes coverage; observational describes the absence of an imposed treatment.
Does a prospective study always establish causation?
No. Recording habits now and outcomes later can remain observational. Confounding can still offer alternative explanations.
Is an experiment with volunteers useless?
No. With appropriate random assignment and other design features, it can investigate treatment effects. The limit concerns how broadly the participants represent other people.
Must I calculate a confidence interval in this lesson?
No. Recognize that an estimation question about a population parameter can seek a range of plausible values. The interval procedures and their interpretation are developed later.
Final understanding check
A school wants to know whether assigning a retrieval-practice sheet improves the mean quiz score compared with a worked-example sheet among its currently enrolled Year 11 students. It randomly selects students from the complete Year 11 list, then independently uses random assignment to place in each revision condition. All selected students participate, receive the same revision time and take the same quiz.
- State a clear investigative question that includes variables, an analysis goal and the population.
- Classify the study. Identify the experimental unit, factor, levels, treatments and response variable.
- Is this a census? Explain.
- State what random selection and random assignment contribute separately.
- Does the design already prove that retrieval practice improves scores? Explain.
Reveal the full solution
1. Question: “For the school’s currently enrolled Year 11 students, does assignment to the retrieval-practice sheet increase the population mean quiz score, in points, compared with assignment to the worked-example sheet under the same revision time?” The goal is a directional comparison of mean responses: retrieval-practice mean minus worked-example mean greater than . Let be the population mean quiz score under retrieval practice and the population mean under worked examples. The directional comparison is:
2. Study: an experiment, because revision conditions are imposed. Unit: individual student. Factor: sheet type. Levels/treatments: retrieval-practice sheet and worked-example sheet. Response: subsequent quiz score in points.
3. Coverage: of students are included, so it is a sample, not a census.
4. Selection: random selection supports generalization to the enrolled Year 11 students. Assignment: random assignment strengthens the causal comparison of the assigned sheets, with the stated common time and assessment helping make the conditions comparable.
5. Result: no. This design supports investigating an effect, but outcome data and suitable statistical analysis are still needed to determine what the evidence shows. An observed sample mean difference by itself is not automatic proof of a population effect.
Ready to move on? You should be able to build a question, name the study and variables, explain possible confounding, and justify exactly who and what a conclusion can describe.
Continue learning
Learn how simple random, stratified, cluster and systematic sampling select units from a population.
Previous: Topic 1.9 — Comparing distributions and z-scores · Review question construction · Review study types · Back to the lesson overview