AI Recommender Radicalization Audits: Trajectory Testing

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-ethics-safety-AI-and-recommender-radicalization-audit-adults

1. What is a primary capability of AI in the context of recommender system audits for detecting harmful content drift?

   1. Manually reviewing each recommended video for policy violations
   2. Predicting the exact number of views a video will receive
   3. Deciding which content should be removed from the platform
   4. Generating synthetic user personas that simulate diverse starting points in the recommender

2. According to the principles of responsible AI auditing, which task should remain with human decision-makers rather than being delegated to AI systems?

   1. Clustering trajectory endpoints against a harm taxonomy
   2. Generating diverse synthetic starting personas
   3. Simulating user click patterns through a recommender
   4. Determining what threshold of drift constitutes a policy violation

3. When designing a synthetic-persona audit harness for a recommender system, which of the following specifications would be most appropriate to include?

   1. The real names and email addresses of test users
   2. The exact timestamps when real users typically watch videos
   3. The action policy for clicking recommendations (top 1 vs top 3)
   4. The clothing preferences and favorite colors of synthetic personas

4. What is the purpose of applying a harm taxonomy to trajectory endpoints in an AI recommender audit?

   1. To categorize the exact monetary value of each recommended item
   2. To calculate the total viewing time of each persona
   3. To classify endpoints according to types and severity of potential harm
   4. To determine which users should receive premium subscriptions

5. Why is trajectory testing considered more valuable than testing individual recommendations in isolation?

   1. Trajectory testing can detect gradual harmful drift over a sequence of recommendations
   2. Trajectory testing requires less computational resources
   3. Individual recommendations are always accurate
   4. Individual recommendations are too complex to analyze

6. A company wants to use AI to automatically determine when their recommender system violates content policies. What is the fundamental limitation they will encounter?

   1. AI lacks the ability to simulate user behavior
   2. AI cannot generate sufficient test data
   3. AI cannot process video content at scale
   4. AI cannot make policy-level judgments about acceptable harm thresholds

7. What critical type of information do synthetic-persona audits fail to capture that real user studies can provide?

   1. The device type used to access content
   2. The exact genre of videos being recommended
   3. The number of recommendations served per session
   4. Social context and user intent behind viewing behavior

8. Before claiming that a recommender system is safe based on synthetic-persona audits alone, what additional evidence should be gathered?

   1. Consented opt-in studies with actual human users
   2. More computational power to run additional simulations
   3. Approval from the platform's legal team
   4. Higher-resolution video quality tests

9. In the context of recommender systems, what does the term 'rabbit hole' refer to?

   1. A testing methodology for measuring recommendation latency
   2. A technical error that causes videos to fail to load
   3. A gradual escalation path where users are recommended increasingly extreme content
   4. A feature that allows users to hide certain video categories

10. When clustering trajectory endpoints against a harm taxonomy, what is being analyzed?

    1. The final content categories and potential harms at the end of user journeys
    2. The number of likes on recommended videos
    3. The viewing duration of each video in a session
    4. The color schemes of video thumbnails

11. If an audit specifies an action policy of 'always click top 1' versus 'click top 3', what aspect of user behavior is being tested?

    1. The user's typing speed when searching
    2. The user's social network size
    3. The user's willingness to pay for premium features
    4. The depth of engagement with recommended content

12. Why must a multidisciplinary harm review accompany synthetic-persona audit results?

    1. Because harm determination requires expertise beyond what technical metrics can provide
    2. Because synthetic personas require diverse academic backgrounds
    3. Because video content requires artistic interpretation
    4. Because the audit software needs legal approval

13. When selecting the 20 starting personas for a synthetic-persona audit, what characteristic is most important to ensure meaningful results?

    1. They should represent diverse starting points and audience segments
    2. They should be named after famous scientists
    3. They should all live in the same geographic region
    4. They should all have the same viewing history

14. What does the session length parameter control in a synthetic-persona audit?

    1. The time of day when audits are run
    2. The duration of each video watched in milliseconds
    3. The number of recommendation interactions simulated per persona
    4. The physical length of video thumbnails

15. A technology company publishes a report claiming their recommender system is 'safe' based solely on synthetic-persona audit results. What is the primary weakness in this claim?

    1. Synthetic audits cannot capture real user intent and social context
    2. The audit did not include video games
    3. The company used too many personas
    4. The report was published on a Tuesday