AP® Statistics review sheet from Aim for Five (aimforfive.com/stats/units/1/1-13)
Unit 1 · Topic 1.13
1.13 Experimental Design
A well-designed experiment is the only way to show cause and effect with data. You'll learn its four principles (comparison, random assignment, replication and control), the roles of placebos and blinding, and three designs: completely randomized, randomized block and matched pairs.
Key terms
- random assignment
- control group and placebo
- single-blind / double-blind
- replication
- randomized block design
- matched pairs design
Four principles
An extraneous variable is anything that could affect the response but isn't the explanatory variable being studied. Control holds some of them fixed. Random assignment handles the rest by making the groups roughly alike on all of them, measured or not. If it works, the groups end up roughly balanced on every extraneous variable, so a difference in the response can be traced to the treatments.
- Comparison: two or more treatments to compare. Often one group is a control group that gets a placebo, the usual treatment or no treatment.
- Random assignment: use chance to decide which experimental units get which treatment.
- Replication: give each treatment to more than one experimental unit, so chance differences between units tend to even out.
- Control: keep other variables that could affect the response the same for every unit, like the amount of water each plant gets.
Placebos and blinding
A placebo is a fake treatment, such as a sugar pill. People often respond to a treatment just because they expect it to work. That's the placebo effect: the difference between how people respond to a placebo and how they'd respond to nothing at all. Giving the control group a placebo makes the comparison fair.
In a single-blind experiment, either the subjects or the people who work with them don't know who got which treatment. In a double-blind experiment, neither knows. Blinding prevents expectations from changing how subjects respond or how researchers measure results.
Three designs
Completely randomized design: assign all units to treatments completely at random. Group sizes are often equal but don't have to be.
Randomized block design: first sort units into blocks of similar units based on a variable that affects the response (the blocking variable), then randomly assign treatments within each block, so every treatment appears in every block. Blocking separates out the variation caused by the blocking variable, so treatment comparisons are more precise.
Matched pairs design: a block design with two treatments. Either pair up similar units and randomly assign one treatment to each member of the pair, or give each unit both treatments in a random order.
Conclusions and ethics
Random assignment reduces the chance that a confounding variable explains the result, which is why a well-designed experiment can support cause and effect.
Most experiments use volunteers, because randomly selecting people and making them take part would be unethical or impossible. So the conclusions apply to units like those who took part, not automatically to a whole population.
Worked examples
Try each one yourself first, then open the solution.
- Example 1Calculator allowed
Describe a completely randomized design
A coach wants to know whether a new warm-up routine improves sprint times compared with the usual warm-up. 40 team members volunteer. Describe a completely randomized design.
Show the solutionHide the solution
- Step 1: Number the 40 athletes 1 to 40.
- Step 2: Use a random number generator to pick 20 different numbers from 1 to 40 (ignoring repeats). Those athletes do the new warm-up; the other 20 do the usual one.
- Step 3: Keep everything else the same: same track, same time of day, same timing method. If possible, the timer shouldn't know which warm-up each athlete did.
- Step 4: After the warm-up, record each athlete's 40-meter sprint time and compare the mean times for the two groups.
Answer: Randomly assign 20 of the 40 numbered athletes to the new warm-up and 20 to the usual one, hold other conditions constant, measure sprint times and compare the group means.
- Example 2Calculator allowed
Why block?
Researchers test two acne creams on 60 teenage volunteers: 30 with mild acne and 30 with severe acne. Severity strongly affects how much skin improves. Describe a better design than a completely randomized one, and explain the benefit.
Show the solutionHide the solution
- Step 1: Severity affects the response, so use it as a blocking variable.
- Step 2: Form two blocks: mild (30) and severe (30).
- Step 3: Within each block, randomly assign 15 subjects to Cream A and 15 to Cream B.
- Step 4: Compare the creams within each block. Because the subjects in a block start out similar, differences due to severity don't get mixed into the comparison.
Answer: Use a randomized block design, blocking by acne severity and randomly assigning creams within each block. This removes the variation due to severity, so the comparison between creams is more precise.
- Example 3Calculator allowed
Trap: randomization without a comparison
A company randomly selects 100 customers to try a new sleep app. After a month, 70% say they sleep better. Can the company conclude the app improves sleep?
Show the solutionHide the solution
- Step 1: There's random selection but only one group, so nothing to compare against.
- Step 2: Sleep might improve for other reasons: the season, the placebo effect or simply paying attention to sleep.
- Step 3: A better design would randomly assign customers to the app or a placebo-style app and compare the groups.
Answer: No. Without a comparison group, the company can't tell whether the app caused the improvement.
Common mistakes
- Confusing random assignment (who gets which treatment) with random sampling (who is in the study).
- Blocking on something that doesn't affect the response. Blocks should be built on a variable linked to the response.
- Saying a block design randomly assigns units to blocks. Blocks are formed by a characteristic; randomization happens within each block.
- Describing replication as repeating the whole experiment. In this course it means using more than one unit per treatment.
On the exam
- Describing a randomization method is a frequent free-response task: number the units, use a random number generator (or slips of paper), handle repeats, and say how many go to each treatment.
- When asked why blocking helps, say it accounts for variation in the response due to the blocking variable, making it easier to see differences between treatments.
Connected topics
Videos
Check yourself
4 questions on 1.13 Experimental Design. Pick an answer to see if you got it, and why.
An experiment tests two fertilizers on tomato plants. Half of the plants are in a greenhouse and half are outdoors. The researcher randomly assigns the fertilizers separately within the greenhouse plants and within the outdoor plants. What is the main benefit of this design?
A shoe company wants to compare how fast two sole materials wear out. Which plan is a matched pairs design?
An experiment tests the effects of watering schedule (daily or every other day) and light level (low, medium or high) on plant growth. Each combination of watering and light is used. How many treatments are there?
In a well-designed experiment comparing two study methods, why are students randomly assigned to the methods?
0 of 4 answered