Sampling Bias: Undercoverage, Nonresponse, and Response Bias
Learn how undercoverage, nonresponse, response bias, wording, and voluntary response can systematically distort a sample.
Direct answer
sampling bias: Sampling bias is systematic distortion caused by how a sample is framed, selected, contacted, or measured. Diagnose the mechanism first, then explain whether the resulting estimate is likely too high, too low, or of unknown direction.
Quick reference: Sampling Bias: Undercoverage, Nonresponse, and Response Bias
| Bias | A systematic tendency to overestimate or underestimate the target. |
|---|---|
| Nonresponse | Selected individuals do not provide data. |
| Undercoverage | Part of the population is missing from the frame. |
Sampling Bias: Undercoverage, Nonresponse, and Response Bias: complete lesson
Bias is a systematic direction, not ordinary sampling variability
A random sample will rarely reproduce a population perfectly. That ordinary sample-to-sample fluctuation is sampling variability. Sampling bias is different: the data-collection process systematically favors some outcomes, so repeated samples tend to miss the target in the same direction. AP Statistics questions often test whether a flaw is merely random noise or whether the design creates a directional distortion. A large sample can reduce random variability, but it cannot rescue a biased mechanism; a million responses from the wrong frame can be precisely wrong.
The first diagnostic question is therefore not “How many people were sampled?” but “Who had a realistic chance to enter the sample, who actually responded, and could the measurement process push answers upward or downward?” This separates frame problems, contact/participation problems, and measurement problems. The wording of a prompt matters because a conclusion such as “the estimate may be biased” is incomplete when the direction can be justified from the mechanism.
Undercoverage begins before the first invitation is sent
Undercoverage occurs when some members of the target population are missing from, or poorly represented in, the sampling frame. A city wants the proportion of all adult residents who use public transit but samples only names from a list of registered car owners. Adults without cars have little or no chance of selection, and transit use is plausibly higher among those omitted residents. The design therefore tends to underestimate transit use. Randomly selecting names from the car-owner list does not remove the undercoverage because the frame itself is incomplete.
A useful AP response names three pieces: the target population, the omitted group, and why the omitted group is likely to differ on the response variable. Without the third piece, the direction of bias is speculation. If the omitted group could reasonably have either higher or lower values, say the estimate may be biased but the direction cannot be determined from the information given.
Nonresponse is about selected units that do not provide usable data
Nonresponse happens after a sampling mechanism has selected units. The concern is not the response rate by itself; the concern is whether responders and nonresponders differ on the variable being estimated. Suppose a random sample of employees is invited to a confidential survey about weekly unpaid overtime, but employees working the longest hours are least likely to respond because they are busy. The responding sample may understate the true average. Repeated reminders can raise the response rate, but weighting or follow-up design is needed when nonresponders remain systematically different.
Do not label every missing observation “nonresponse bias.” If missingness is unrelated to the response after conditioning on the design, the loss can increase variability without necessarily creating substantial bias. AP questions reward a mechanism-based explanation: identify who is less likely to respond and connect that lower response propensity to the likely value of the measured variable.
Response bias changes what a respondent reports
Response bias arises when the recorded answer differs systematically from the respondent’s true value or opinion. Leading wording, interviewer presence, social desirability, memory limitations, order effects, and sensitive questions can all contribute. Asking “How much did you responsibly save for retirement last month?” embeds approval in the wording and may push reported saving upward. Asking students to recall exactly how many minutes they studied each day six months ago can produce recall error even when the question is neutral.
Anonymity, neutral wording, balanced response options, shorter recall periods, validated measurement instruments, and interviewer training target different mechanisms. “Make the sample larger” does not directly fix response bias because more respondents can reproduce the same systematic measurement distortion with smaller random error.
Voluntary response and convenience samples create selection mechanisms of their own
A voluntary-response sample allows people to decide whether to enter the sample after seeing the invitation. People with strong opinions, unusual experiences, or high engagement often participate at different rates. A news site poll asking readers whether a new parking fee is fair can overrepresent readers motivated enough to click. A convenience sample selects units because they are easy to reach—for example, surveying the first 100 customers entering one store on Saturday morning. Neither method supports the same population generalization as a probability sample unless additional design information justifies it.
A voluntary-response problem is not identical to nonresponse. With nonresponse, a probability mechanism selected a person and the selected person failed to respond. With voluntary response, self-selection is part of who enters the sample in the first place. That distinction is frequently tested because the remedies and inferential claims differ.
Bias, confounding, and lack of random assignment are different diagnoses
Sampling bias concerns how observations represent a target population or how responses are measured. Confounding concerns inability to separate the effect of an explanatory variable from another variable in a study. Lack of random assignment limits causal conclusions even if the sample represents the population perfectly. A national random sample comparing people who chose two diets may generalize the observed association broadly, but without random assignment the comparison is still observational and does not by itself establish that the diet caused the difference.
Conversely, a randomized experiment performed entirely on volunteers can support a causal comparison among those experimental units while still having limited generalizability to a larger population. AP Statistics questions often combine these ideas, so a complete response distinguishes the scope of inference from the internal validity of the treatment comparison.
Direction-of-bias reasoning should be conditional, not automatic
Many textbook labels do not determine direction by themselves. Undercoverage can bias high or low depending on which group is missing and how that group differs. Nonresponse can bias high or low depending on who does not answer. Response bias can move a measure in either direction depending on wording or social pressure. The defensible structure is: identify the affected group, predict how its values compare with the represented group, then state the resulting direction for the statistic.
For example, a college estimates average weekly exercise by emailing a survey during varsity-team registration. If athletes are more likely than other students to see and complete the survey and typically exercise more, the sample mean is likely biased upward. If the prompt never states or reasonably implies the relationship between participation and exercise, the safer answer is that the direction is unknown.
Prevention belongs at the design stage
A strong sampling-bias analysis proposes a remedy matched to the failure. Undercoverage calls for a broader or multiple-source frame. Nonresponse calls for repeated contacts, alternative contact modes, incentives, short questionnaires, and targeted follow-up of hard-to-reach cases. Response bias calls for neutral wording, privacy, randomization of question order when appropriate, validated measures, and reduced interviewer influence. Voluntary response calls for a probability-based selection process rather than an open invitation.
Post-survey weighting can reduce some known imbalances, but weighting is not magic. If omitted or nonresponding units differ on variables not captured by the weighting controls, bias can remain. AP-level reasoning should treat weighting as an adjustment that relies on assumptions, not as proof that the final estimate is unbiased.
Worked sampling-bias diagnoses
Worked case 1: Library internet survey
Design: A county estimates broadband access using an online-only questionnaire linked from the county website.
Bias mechanism: Residents without reliable internet are least able to see or complete the form. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: The estimate of broadband access is likely biased upward. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Provide phone/mail/in-person response modes and draw from an address-based frame. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 2: Fitness-app sleep study
Design: A university samples students who voluntarily connect a fitness app to study sleep duration.
Bias mechanism: Students who own compatible devices and are willing to share health data can differ from all students. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: The sample may not represent campus sleep patterns; the direction is not guaranteed. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Select students first, then offer multiple measurement methods or supplied devices. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 3: Restaurant receipt survey
Design: A restaurant prints a survey link only on dine-in receipts to estimate satisfaction among all customers.
Bias mechanism: Delivery and takeout customers are undercovered. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: Direction depends on whether satisfaction differs by ordering mode. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Sample across dine-in, takeout, and delivery transactions. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 4: Teacher evaluation
Design: Students complete a course evaluation during the final five minutes of the last class meeting.
Bias mechanism: Absent students cannot respond, and attendance may relate to satisfaction. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: Direction cannot be inferred without information about absent students. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Contact the selected roster outside class and follow up with nonresponders. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 5: Sensitive spending question
Design: A face-to-face interviewer asks households to report spending on gambling.
Bias mechanism: Social desirability can cause underreporting of a stigmatized behavior. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: The estimated mean is likely biased downward. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Use confidential self-administration and neutral wording. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 6: Neighborhood noise poll
Design: A local news site asks readers who feel strongly about airport noise to vote.
Bias mechanism: People most affected or most motivated self-select into the poll. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: The result cannot be treated as a probability estimate of neighborhood opinion. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Draw a probability sample of households in the defined area. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 7: Hospital follow-up
Design: A hospital mails recovery surveys; patients with complications are more likely to remain in contact and respond.
Bias mechanism: Response propensity is associated with the outcome. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: Reported complication rates can be biased upward. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Use systematic follow-up of selected patients regardless of outcome. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 8: Salary estimate from alumni
Design: A college estimates graduate salary using only alumni who update their career profile.
Bias mechanism: Employment success may affect willingness to update profiles. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: The estimated salary may be biased upward if higher earners update more often. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Sample the graduating cohort and pursue nonresponse follow-up. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 9: School lunch preference
Design: A principal surveys students standing in the cafeteria hot-lunch line about preferred menu items.
Bias mechanism: Students who bring lunch from home are missing. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: The results can overrepresent preferences of cafeteria users. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Sample from the enrollment roster, not the serving line. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 10: Commute-time recall
Design: Workers are asked to recall their exact commute time for every day of the previous year.
Bias mechanism: Long recall period creates measurement error and possible rounding/heaping. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: The direction is uncertain, but precision and validity are reduced. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Use recent diary entries or passively recorded trip data with consent. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 11: Donor satisfaction
Design: A charity emails only donors who opened the previous fundraising email.
Bias mechanism: Low-engagement donors are undercovered before the survey begins. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: Satisfaction may be overstated if engaged donors are happier. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Sample from the entire donor database and vary contact modes. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Worked case 12: Medication adherence
Design: A physician asks patients, in person, “You take every dose as prescribed, correct?”
Bias mechanism: Leading wording plus authority pressure encourages socially desirable answers. The correct label follows from where the problem enters the process rather than from the size of the sample.
Likely consequence: Adherence is likely overreported. This conclusion is justified only to the extent that the relationship between selection or reporting and the response variable is supported by the scenario.
Better design: Use neutral, private questions and objective refill data when appropriate. A useful repair changes the selection, contact, or measurement mechanism that created the distortion; increasing sample size alone does not repair the systematic problem.
Bias audit laboratory
Bias audit laboratory 1: school survey
For each study, trace the path from target population to sampling frame to selected sample to responding sample to recorded measurement. Mark the exact stage where representation can be lost, then state a repair that changes that stage rather than merely enlarging n. In this school survey, write the target in words before using notation so the calculation remains tied to the variable, population, or model actually being studied.
A high-quality solution should also include a self-check tailored to this topic: compare the sign, units, boundary, probability range, or design logic with what the scenario makes plausible. If the numerical output contradicts that check, revisit the setup before changing the conclusion. In “Bias audit laboratory 1: school survey,” apply this check to the named variables and numerical direction rather than treating it as a reusable sentence from another exercise. Tie the diagnosis to frame coverage, response propensity, or measurement behavior and state the direction only when the scenario supports it.
Finally, distinguish what the statistical method establishes from what the study design does not establish. The strongest response gives the numerical or graphical evidence and then limits the claim to the population, process, association, or legacy-enrichment scope justified by the data-generating mechanism. In “Bias audit laboratory 1: school survey,” apply this check to the named variables and numerical direction rather than treating it as a reusable sentence from another exercise. Tie the diagnosis to frame coverage, response propensity, or measurement behavior and state the direction only when the scenario supports it.
Bias audit laboratory 2: public-health study
For each study, trace the path from target population to sampling frame to selected sample to responding sample to recorded measurement. Mark the exact stage where representation can be lost, then state a repair that changes that stage rather than merely enlarging n. In this public-health study, write the target in words before using notation so the calculation remains tied to the variable, population, or model actually being studied.
A high-quality solution should also include a self-check tailored to this topic: compare the sign, units, boundary, probability range, or design logic with what the scenario makes plausible. If the numerical output contradicts that check, revisit the setup before changing the conclusion. In “Bias audit laboratory 2: public-health study,” apply this check to the named variables and numerical direction rather than treating it as a reusable sentence from another exercise. Tie the diagnosis to frame coverage, response propensity, or measurement behavior and state the direction only when the scenario supports it.
Finally, distinguish what the statistical method establishes from what the study design does not establish. The strongest response gives the numerical or graphical evidence and then limits the claim to the population, process, association, or legacy-enrichment scope justified by the data-generating mechanism. In “Bias audit laboratory 2: public-health study,” apply this check to the named variables and numerical direction rather than treating it as a reusable sentence from another exercise. Tie the diagnosis to frame coverage, response propensity, or measurement behavior and state the direction only when the scenario supports it.
Bias audit laboratory 3: manufacturing process
For each study, trace the path from target population to sampling frame to selected sample to responding sample to recorded measurement. Mark the exact stage where representation can be lost, then state a repair that changes that stage rather than merely enlarging n. In this manufacturing process, write the target in words before using notation so the calculation remains tied to the variable, population, or model actually being studied.
A high-quality solution should also include a self-check tailored to this topic: compare the sign, units, boundary, probability range, or design logic with what the scenario makes plausible. If the numerical output contradicts that check, revisit the setup before changing the conclusion. In “Bias audit laboratory 3: manufacturing process,” apply this check to the named variables and numerical direction rather than treating it as a reusable sentence from another exercise. Tie the diagnosis to frame coverage, response propensity, or measurement behavior and state the direction only when the scenario supports it.
Finally, distinguish what the statistical method establishes from what the study design does not establish. The strongest response gives the numerical or graphical evidence and then limits the claim to the population, process, association, or legacy-enrichment scope justified by the data-generating mechanism. In “Bias audit laboratory 3: manufacturing process,” apply this check to the named variables and numerical direction rather than treating it as a reusable sentence from another exercise. Tie the diagnosis to frame coverage, response propensity, or measurement behavior and state the direction only when the scenario supports it.
Bias audit laboratory 4: transportation system
For each study, trace the path from target population to sampling frame to selected sample to responding sample to recorded measurement. Mark the exact stage where representation can be lost, then state a repair that changes that stage rather than merely enlarging n. In this transportation system, write the target in words before using notation so the calculation remains tied to the variable, population, or model actually being studied.
A high-quality solution should also include a self-check tailored to this topic: compare the sign, units, boundary, probability range, or design logic with what the scenario makes plausible. If the numerical output contradicts that check, revisit the setup before changing the conclusion. In “Bias audit laboratory 4: transportation system,” apply this check to the named variables and numerical direction rather than treating it as a reusable sentence from another exercise. Tie the diagnosis to frame coverage, response propensity, or measurement behavior and state the direction only when the scenario supports it.
Finally, distinguish what the statistical method establishes from what the study design does not establish. The strongest response gives the numerical or graphical evidence and then limits the claim to the population, process, association, or legacy-enrichment scope justified by the data-generating mechanism. In “Bias audit laboratory 4: transportation system,” apply this check to the named variables and numerical direction rather than treating it as a reusable sentence from another exercise. Tie the diagnosis to frame coverage, response propensity, or measurement behavior and state the direction only when the scenario supports it.
Sampling Bias: Undercoverage, Nonresponse, and Response Bias: 22 multiple-choice questions
Each item asks you to identify a specific source of bias, infer direction only when justified, or choose a design repair matched to the mechanism.
Question 1. Sampling Bias
A regional manufacturer in Riverbend during a six-week field trial posts an open link and allows anyone who feels strongly to participate. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: A
The main problem at the regional manufacturer in Riverbend during a six-week field trial is voluntary response: the plan posts an open link and allows anyone who feels strongly to participate. This mechanism systematically distorts who is represented or what is reported about part diameter. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 2. Sampling Bias
A digital learning platform in Capital Region during a summer implementation review uses a frame that omits residents without registered email addresses. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: B
The main problem at the digital learning platform in Capital Region during a summer implementation review is undercoverage: the plan uses a frame that omits residents without registered email addresses. This mechanism systematically distorts who is represented or what is reported about lesson completion. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 3. Sampling Bias
A wildlife clinic in Coastal Plains during a regional benchmarking study surveys only the people standing nearest the office at noon. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: B
The main problem at the wildlife clinic in Coastal Plains during a regional benchmarking study is convenience: the plan surveys only the people standing nearest the office at noon. This mechanism systematically distorts who is represented or what is reported about recovery time. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 4. Sampling Bias
A housing authority in Riverbend during a community outreach cycle repeatedly contacts a random sample, but many selected people never answer. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: A
The main problem at the housing authority in Riverbend during a community outreach cycle is nonresponse: the plan repeatedly contacts a random sample, but many selected people never answer. This mechanism systematically distorts who is represented or what is reported about application processing time. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 5. Sampling Bias
A state park in Pine Ridge during a baseline measurement week asks a leading question that praises one response before asking for an opinion. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: D
The main problem at the state park in Pine Ridge during a baseline measurement week is response bias: the plan asks a leading question that praises one response before asking for an opinion. This mechanism systematically distorts who is represented or what is reported about trail-use duration. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 6. Sampling Bias
A county election office in Capital Region during a weekday operations study asks a leading question that praises one response before asking for an opinion. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: B
The main problem at the county election office in Capital Region during a weekday operations study is response bias: the plan asks a leading question that praises one response before asking for an opinion. This mechanism systematically distorts who is represented or what is reported about ballot-processing time. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 7. Sampling Bias
A recycling program in Atlantic Corridor during a yearly program evaluation uses a frame that omits residents without registered email addresses. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: D
The main problem at the recycling program in Atlantic Corridor during a yearly program evaluation is undercoverage: the plan uses a frame that omits residents without registered email addresses. This mechanism systematically distorts who is represented or what is reported about weekly material weight. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 8. Sampling Bias
A solar installer in Desert County during a regional benchmarking study uses a frame that omits residents without registered email addresses. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: B
The main problem at the solar installer in Desert County during a regional benchmarking study is undercoverage: the plan uses a frame that omits residents without registered email addresses. This mechanism systematically distorts who is represented or what is reported about daily energy output. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 9. Sampling Bias
A municipal emergency dispatch center in Pacific Northwest during a community outreach cycle surveys only the people standing nearest the office at noon. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: D
The main problem at the municipal emergency dispatch center in Pacific Northwest during a community outreach cycle is convenience: the plan surveys only the people standing nearest the office at noon. This mechanism systematically distorts who is represented or what is reported about response time. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 10. Sampling Bias
A county library in Pacific Northwest during a yearly program evaluation repeatedly contacts a random sample, but many selected people never answer. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: D
The main problem at the county library in Pacific Northwest during a yearly program evaluation is nonresponse: the plan repeatedly contacts a random sample, but many selected people never answer. This mechanism systematically distorts who is represented or what is reported about weekly program attendance. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 11. Sampling Bias
A solar installer in Great Lakes during a winter readiness review surveys only the people standing nearest the office at noon. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: B
The main problem at the solar installer in Great Lakes during a winter readiness review is convenience: the plan surveys only the people standing nearest the office at noon. This mechanism systematically distorts who is represented or what is reported about daily energy output. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 12. Sampling Bias
A food safety laboratory in Midwest consortium during a weekday operations study uses a frame that omits residents without registered email addresses. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: D
The main problem at the food safety laboratory in Midwest consortium during a weekday operations study is undercoverage: the plan uses a frame that omits residents without registered email addresses. This mechanism systematically distorts who is represented or what is reported about sample concentration. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 13. Sampling Bias
A digital learning platform in South Harbor during a semester-long cohort study surveys only the people standing nearest the office at noon. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: D
The main problem at the digital learning platform in South Harbor during a semester-long cohort study is convenience: the plan surveys only the people standing nearest the office at noon. This mechanism systematically distorts who is represented or what is reported about lesson completion. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 14. Sampling Bias
A university advising center in North Valley during a pre-exam training cycle asks a leading question that praises one response before asking for an opinion. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: A
The main problem at the university advising center in North Valley during a pre-exam training cycle is response bias: the plan asks a leading question that praises one response before asking for an opinion. This mechanism systematically distorts who is represented or what is reported about appointment wait time. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 15. Sampling Bias
A county library in Great Lakes during a pre-exam training cycle posts an open link and allows anyone who feels strongly to participate. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: B
The main problem at the county library in Great Lakes during a pre-exam training cycle is voluntary response: the plan posts an open link and allows anyone who feels strongly to participate. This mechanism systematically distorts who is represented or what is reported about weekly program attendance. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 16. Sampling Bias
A public high school in Westview during a multiweek validation study repeatedly contacts a random sample, but many selected people never answer. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: C
The main problem at the public high school in Westview during a multiweek validation study is nonresponse: the plan repeatedly contacts a random sample, but many selected people never answer. This mechanism systematically distorts who is represented or what is reported about algebra benchmark completion. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 17. Sampling Bias
A municipal water office in Desert County during a community outreach cycle surveys only the people standing nearest the office at noon. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: D
The main problem at the municipal water office in Desert County during a community outreach cycle is convenience: the plan surveys only the people standing nearest the office at noon. This mechanism systematically distorts who is represented or what is reported about monthly household use. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 18. Sampling Bias
A public high school in Lakeside district during a fall 2026 audit asks a leading question that praises one response before asking for an opinion. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: B
The main problem at the public high school in Lakeside district during a fall 2026 audit is response bias: the plan asks a leading question that praises one response before asking for an opinion. This mechanism systematically distorts who is represented or what is reported about algebra benchmark completion. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 19. Sampling Bias
A city transit agency in North Valley during a multiweek validation study repeatedly contacts a random sample, but many selected people never answer. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: C
The main problem at the city transit agency in North Valley during a multiweek validation study is nonresponse: the plan repeatedly contacts a random sample, but many selected people never answer. This mechanism systematically distorts who is represented or what is reported about on-time arrival. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 20. Sampling Bias
A municipal emergency dispatch center in Sunbelt district during a fall 2026 audit asks a leading question that praises one response before asking for an opinion. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: D
The main problem at the municipal emergency dispatch center in Sunbelt district during a fall 2026 audit is response bias: the plan asks a leading question that praises one response before asking for an opinion. This mechanism systematically distorts who is represented or what is reported about response time. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 21. Sampling Bias
A city transit agency in Metro East during a quarterly performance study uses a frame that omits residents without registered email addresses. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: A
The main problem at the city transit agency in Metro East during a quarterly performance study is undercoverage: the plan uses a frame that omits residents without registered email addresses. This mechanism systematically distorts who is represented or what is reported about on-time arrival. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Question 22. Sampling Bias
A community college in Midwest consortium during a regional benchmarking study surveys only the people standing nearest the office at noon. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
Answer: A
The main problem at the community college in Midwest consortium during a regional benchmarking study is convenience: the plan surveys only the people standing nearest the office at noon. This mechanism systematically distorts who is represented or what is reported about course completion. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Sampling Bias: Undercoverage, Nonresponse, and Response Bias: 6 free-response questions
For each free-response prompt, identify where distortion enters the survey process, explain who is over- or underrepresented or mismeasured, and propose a remedy aimed at that exact mechanism.
FRQ set 1: Sampling Bias
Scenario. A regional hospital in Metro East during a service-improvement study asks a leading question that praises one response before asking for an opinion. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
- Identify the observational units, population, treatments or sampling frame, and the design goal.
- Describe the selection or assignment mechanism precisely enough that another researcher could implement it.
- Explain how the design controls bias, confounding, or variability, and identify a remaining limitation.
- State the scope of inference: population generalization, causal conclusion, both, or neither.
Model response
The main problem at the regional hospital in Metro East during a service-improvement study is response bias: the plan asks a leading question that praises one response before asking for an opinion. This mechanism systematically distorts who is represented or what is reported about appointment completion. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
FRQ set 2: Sampling Bias
Scenario. A wildlife clinic in Cedar Grove during a semester-long cohort study asks a leading question that praises one response before asking for an opinion. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
- Identify the observational units, population, treatments or sampling frame, and the design goal.
- Describe the selection or assignment mechanism precisely enough that another researcher could implement it.
- Explain how the design controls bias, confounding, or variability, and identify a remaining limitation.
- State the scope of inference: population generalization, causal conclusion, both, or neither.
Model response
The main problem at the wildlife clinic in Cedar Grove during a semester-long cohort study is response bias: the plan asks a leading question that praises one response before asking for an opinion. This mechanism systematically distorts who is represented or what is reported about recovery time. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
FRQ set 3: Sampling Bias
Scenario. A farm cooperative in Central County during a school-year data collection asks a leading question that praises one response before asking for an opinion. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
- Identify the observational units, population, treatments or sampling frame, and the design goal.
- Describe the selection or assignment mechanism precisely enough that another researcher could implement it.
- Explain how the design controls bias, confounding, or variability, and identify a remaining limitation.
- State the scope of inference: population generalization, causal conclusion, both, or neither.
Model response
The main problem at the farm cooperative in Central County during a school-year data collection is response bias: the plan asks a leading question that praises one response before asking for an opinion. This mechanism systematically distorts who is represented or what is reported about crop yield. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
FRQ set 4: Sampling Bias
Scenario. A public high school in Prairie District during a yearly program evaluation repeatedly contacts a random sample, but many selected people never answer. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
- Identify the observational units, population, treatments or sampling frame, and the design goal.
- Describe the selection or assignment mechanism precisely enough that another researcher could implement it.
- Explain how the design controls bias, confounding, or variability, and identify a remaining limitation.
- State the scope of inference: population generalization, causal conclusion, both, or neither.
Model response
The main problem at the public high school in Prairie District during a yearly program evaluation is nonresponse: the plan repeatedly contacts a random sample, but many selected people never answer. This mechanism systematically distorts who is represented or what is reported about algebra benchmark completion. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
FRQ set 5: Sampling Bias
Scenario. A regional manufacturer in Lakeside district during a follow-up evaluation period surveys only the people standing nearest the office at noon. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
- Identify the observational units, population, treatments or sampling frame, and the design goal.
- Describe the selection or assignment mechanism precisely enough that another researcher could implement it.
- Explain how the design controls bias, confounding, or variability, and identify a remaining limitation.
- State the scope of inference: population generalization, causal conclusion, both, or neither.
Model response
The main problem at the regional manufacturer in Lakeside district during a follow-up evaluation period is convenience: the plan surveys only the people standing nearest the office at noon. This mechanism systematically distorts who is represented or what is reported about part diameter. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
FRQ set 6: Sampling Bias
Scenario. A municipal water office in Riverbend during a pre-exam training cycle repeatedly contacts a random sample, but many selected people never answer. Identify the main source of bias and explain why increasing the sample size would not necessarily remove it.
- Identify the observational units, population, treatments or sampling frame, and the design goal.
- Describe the selection or assignment mechanism precisely enough that another researcher could implement it.
- Explain how the design controls bias, confounding, or variability, and identify a remaining limitation.
- State the scope of inference: population generalization, causal conclusion, both, or neither.
Model response
The main problem at the municipal water office in Riverbend during a pre-exam training cycle is nonresponse: the plan repeatedly contacts a random sample, but many selected people never answer. This mechanism systematically distorts who is represented or what is reported about monthly household use. A larger sample produced by the same flawed mechanism can estimate the wrong target more precisely; it does not eliminate systematic bias.
Continue with the next connected AP Statistics skill
The most useful next step is to connect this topic to a neighboring method rather than repeating the same question type indefinitely.
