AI and Survey Instrument Debiasing: Spotting Leading Questions

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-research-AI-and-survey-instrument-debiasing-r13a7-creators

1. Which of the following is a task AI can reliably perform when debiasing a survey instrument?

   1. Replacing the need for any pilot testing
   2. Guaranteeing your sample will be representative
   3. Predicting exactly how your target population will respond
   4. Flagging leading or loaded language in questions

2. A survey question asks: 'How much do you agree that our excellent product should cost more?' This exemplifies what problem?

   1. A double-barreled question
   2. Balanced response options
   3. Neutral wording
   4. Loaded language pushing toward a particular response

3. What does the lesson identify as a key limitation of using AI to debias survey instruments?

   1. AI is too expensive for most research budgets
   2. AI cannot detect grammatical errors in questions
   3. AI cannot predict how actual respondents will interpret or react to questions
   4. AI requires internet access to function

4. A researcher writes: 'Do you think taxes should be raised to fund schools or should they be lowered?' This question suffers from which flaw?

   1. It is properly neutral
   2. It uses balanced response options
   3. It is double-barreled, asking about two different things
   4. It lacks any response options

5. After using AI to clean survey questions for biased language, what must researchers do before collecting final data?

   1. Submit the survey to an ethics board
   2. Delete any question AI flagged as problematic
   3. Translate the survey into another language
   4. Pilot the instrument with real respondents

6. What does the lesson mean when it states that 'subtle wording shifts move response distributions by double digits'?

   1. Only dramatic rewording affects responses
   2. Small changes in question wording can shift how 10-99% of respondents answer
   3. Question wording has minimal impact on responses
   4. AI cannot detect subtle wording differences

7. Which term describes a survey question that simultaneously asks about two different issues?

   1. Balanced
   2. Double-barreled
   3. Leading
   4. Loaded

8. A survey offers these response options: 'Strongly agree', 'Agree', 'Somewhat agree', 'Disagree', 'Strongly disagree'. What problem might this create?

   1. The options are too balanced
   2. The options use loaded language
   3. The options are double-barreled
   4. The options lack a neutral midpoint

9. Why is it insufficient to rely solely on AI when designing a survey about sensitive topics like income or political beliefs?

   1. AI cannot detect which questions might offend or confuse real respondents
   2. AI will bias the results intentionally
   3. AI will automatically delete all sensitive questions
   4. AI cannot format questions properly

10. What type of language in a survey question might cause respondents to answer in a particular direction regardless of their true beliefs?

    1. Loaded language
    2. Neutral language
    3. Plain language
    4. Technical language

11. A researcher uses AI to review a survey. What should the researcher do with AI's suggestions for 'neutral rewordings'?

    1. Discard them if they change the original meaning
    2. Use only the rewordings that make questions shorter
    3. Accept them without review since AI is always accurate
    4. Combine them with pilot testing feedback from real respondents

12. What does the lesson identify as the primary value AI adds to survey design?

    1. Detecting patterns of bias humans overlook
    2. Eliminating the need for any human oversight
    3. Predicting which questions will go viral
    4. Guaranteeing statistically significant results

13. What is a 'cognitive interview' in the context of survey methodology?

    1. A statistical analysis of response patterns
    2. An AI-powered interview with chatbots
    3. An interview conducted over Zoom
    4. A pilot test where respondents verbalize their thought process while answering

14. The lesson notes that AI can propose balanced response options. What makes a set of response options 'balanced'?

    1. Options are written in question format
    2. Options use only positive words
    3. Options fairly represent the full range of possible views
    4. Each option is the same length

15. If an AI flags a question as 'leading,' what should the researcher do next?

    1. Replace it with a neutral rewrite and pilot test with real respondents
    2. Change it to an open-ended question
    3. Submit the survey immediately since AI approval is final
    4. Delete the question entirely