AI and Survey Cognitive Pretest Prompts: Probe Question Bank

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-research-AI-and-survey-cognitive-pretest-prompts-r11a3-creators

1. A survey researcher wants to pretest a questionnaire using cognitive interviews. What is the primary role AI can play in this process?

   1. Replace the methodologist in deciding which items need modification
   2. Automatically revise ambiguous survey items based on responses
   3. Conduct live cognitive interviews with survey respondents
   4. Generate a bank of probe questions that a methodologist will use during interviews

2. Which four types of probe questions can AI generate for each survey item?

   1. Comprehension, retrieval, judgment, and response probes
   2. Open-ended, closed-ended, Likert, and demographic probes
   3. Pilot, validation, reliability, and feasibility probes
   4. Memory, attention, perception, and motivation probes

3. What is a fundamental capability that AI lacks in the cognitive pretesting workflow?

   1. Running a cognitive interview with an actual human respondent
   2. Producing multiple probe types per item
   3. Identifying items that may contain ambiguous language
   4. Generating probe questions for survey items

4. After a cognitive interview is complete, who is responsible for deciding whether to revise a survey item?

   1. The methodologist conducting the interview
   2. The AI system that generated the probes
   3. The survey respondent who was interviewed
   4. The software platform hosting the survey

5. The lesson describes cognitive interviewing as a 'craft.' What does this imply?

   1. It relies primarily on statistical analysis techniques
   2. It involves artistic design of survey questions
   3. It requires skilled human judgment and cannot be fully automated
   4. It can be learned only through formal academic coursework

6. A comprehension probe in cognitive interviewing is designed to assess whether the respondent:

   1. Can accurately recall the information needed to answer
   2. Formulates a judgment or evaluation before responding
   3. Understands the meaning of the question as intended by the researcher
   4. Knows how to record or articulate their answer

7. Judgment probes are used to understand how respondents:

   1. Interpret the meaning of complex vocabulary
   2. Recall personal experiences relevant to the question
   3. Evaluate or weigh information before giving their answer
   4. Decide which response option to select from a list

8. What does a response probe typically investigate?

   1. How the respondent constructs and articulates their answer
   2. If the respondent can remember the relevant information
   3. Whether the respondent understood the question correctly
   4. How the respondent evaluates the topic being asked about

9. Based on the lesson, how can AI help identify problematic survey items before cognitive interviewing begins?

   1. By comparing the survey to other surveys in a database
   2. By automatically deleting items that contain technical jargon
   3. By generating probes that are likely to expose ambiguity in certain items
   4. By conducting a statistical analysis of response patterns

10. A methodologist receives an AI-generated probe bank for a 20-item survey. What is the next step in the pretesting process?

    1. Use the probes to write the final survey questionnaire
    2. Send the probe bank directly to survey respondents
    3. Have the AI system analyze the probe bank for errors
    4. Use the probes to conduct cognitive interviews with sample respondents

11. Which statement best reflects the relationship between AI-generated probes and the methodologist's work?

    1. AI prepares probe questions; the methodologist conducts interviews and interprets findings
    2. AI writes the survey; the methodologist proofreads it
    3. AI and the methodologist both conduct interviews simultaneously
    4. AI evaluates interview findings; the methodologist observes

12. If an AI system generates excellent probe questions but no cognitive interviews are conducted, what is missing from the pretesting process?

    1. The computer hardware needed to process the responses
    2. The actual interaction with respondents that reveals how questions are understood
    3. The machine learning algorithm that refines the probes
    4. The statistical analysis that validates survey questions

13. What is the primary value of AI in the survey pretesting workflow?

    1. Efficiently generating a comprehensive bank of probe questions
    2. Automatically determining which questions to keep in the final survey
    3. Eliminating the need for respondent feedback entirely
    4. Replacing human methodologists in survey design

14. A survey item asks respondents to report their 'total household income from last year.' Why might AI-generated probes help identify a problem with this item?

    1. Probes would determine if respondents own a computer
    2. Probes might reveal respondents unsure whether to include all income sources or only primary income
    3. Probes would confirm that respondents understand the word 'household'
    4. Probes would automatically correct the question wording

15. The lesson emphasizes that AI probes 'prepare the methodologist.' What does this mean?

    1. AI trains the methodologist to become certified in cognitive interviewing
    2. AI schedules interview appointments for the methodologist
    3. AI generates questions the methodologist will use when interviewing respondents
    4. AI evaluates whether the methodologist is qualified to conduct interviews