AP Statistics · Lesson 3

Sampling, bias and experiments

Know Know it

A population is every individual you want to learn about, and a sample is the part you actually measure. A number that describes a population is a parameter; a number from a sample is a statistic.

Random sampling methods: a simple random sample gives every group of the chosen size an equal chance. A stratified random sample splits the population into similar groups (strata) and takes a random sample from each. A cluster sample randomly chooses whole groups and measures everyone in them. A systematic random sample picks a random start, then every kth individual.

A sampling method is biased if it tends to miss the truth in the same direction. A convenience sample takes whoever is easy to reach. A voluntary response sample lets people choose themselves, so strong opinions are overrepresented. Undercoverage leaves some groups out of the list sampled from. Nonresponse happens when chosen people do not answer. Response bias comes from leading wording or people giving answers they think are wanted. A larger sample does not fix bias.

In an observational study, researchers measure without assigning treatments, so a confounding variable may explain any link. In an experiment, researchers assign treatments to experimental units. A good experiment uses comparison of two or more treatments (often with a control group or placebo), random assignment, control of other variables, and replication on enough units. Blinding keeps subjects or evaluators from knowing who got which treatment. A randomized block design randomly assigns treatments within groups of similar units, and a matched pairs design compares two treatments within pairs or within each subject.

Random sampling lets you generalize to the population sampled. Random assignment lets you conclude cause and effect.

Trap Random sampling and random assignment do different jobs

Random sampling decides who is in the study and supports generalizing to the population. Random assignment decides who gets which treatment and supports a cause-and-effect claim. An experiment on volunteers can show cause and effect, but only for units like those volunteers.

Worked example Easy

A radio host invites listeners to call in about a proposed city tax, and 85 percent of the callers oppose it. What kind of sample is this?

  1. A stratified random sample
  2. A cluster sample
  3. A simple random sample
  4. A voluntary response sampleAnswer

How to solve it

  1. Ask who decided who is in the sample.
  2. The listeners chose to call, so it is a voluntary response sample, which tends to overstate strong opinions.

Why each choice is right or wrong

  • A. Incorrect. A stratified random sample would split listeners into groups and pick at random from each, but no one was picked.
  • B. Incorrect. A cluster sample would randomly choose whole groups of people, but here callers chose themselves.
  • C. Incorrect. A simple random sample gives every listener the same chance to be chosen, but the callers selected themselves.
  • D. Correct. The callers decided for themselves whether to respond, which makes this a voluntary response sample.

Explain Explain it

Task verbs: explain, justify

To explain a design choice, say what the choice does and why it matters for the conclusion. For example: blocking by greenhouse puts equal numbers of each treatment in sunny and shady conditions, so a difference in sunlight cannot be mistaken for a fertilizer effect.

Worked example Medium

A gardener wants to test whether a new fertilizer increases tomato yield. She has 40 plants: 20 in a sunny greenhouse and 20 in a shady one. Which plan best keeps sunlight from confusing the results?

  1. Within each greenhouse, randomly assign 10 plants to the new fertilizer and 10 to the oldAnswer
  2. Give the new fertilizer to all 20 plants in the sunny greenhouse and the old one to the shady greenhouse
  3. Let the gardener pick the 20 healthiest-looking plants to receive the new fertilizer
  4. Give every plant the new fertilizer and compare this year's yield with last year's

How to solve it

  1. Sunlight differs between greenhouses and could affect yield.
  2. Treat each greenhouse as a block and randomly assign both fertilizers inside it, so sunlight is balanced across treatments.

Why each choice is right or wrong

  • A. Correct. This is a randomized block design: each greenhouse holds both treatments, so sunlight affects both groups equally.
  • B. Incorrect. Giving the new fertilizer only to the sunny greenhouse confounds fertilizer with sunlight, so their effects cannot be separated.
  • C. Incorrect. Letting the gardener pick the healthiest plants for the new fertilizer adds bias, since healthier plants may yield more anyway.
  • D. Incorrect. Comparing this year with last year has no control group, so weather or other changes could explain any difference.

Explain Your turn

In a fictional study, 600 office workers reported how much coffee they drink and how many sick days they took last year. Workers who drank 3 or more cups a day took fewer sick days. Explain why this study cannot show that coffee causes fewer sick days, and name one variable that could be confounded with coffee drinking, saying how it could be linked to both coffee and sick days.

Write it, then compare. Nothing you type here is saved or graded.

Compare with a model answer

A full answer does this

  • Explains that the study is observational, with no random assignment of coffee drinking
  • Names a plausible confounding variable and says how it could be linked to both coffee drinking and sick days

Model answer

The researchers only recorded what the workers already did; no one was randomly assigned to drink coffee, so this is an observational study and cannot show cause and effect. General health could be confounded with coffee: workers who already feel healthy and energetic may drink more coffee to keep up a busy schedule, and healthier workers also get sick less often, so they take fewer sick days.

Apply Apply it

Practice these topics. AP Practice questions on exactly what this lesson covers, with an explanation for every answer.

Practice these topics