AI product design: designing for uncertainty and recovery

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-careers-AI-product-design-adults

1. Who is responsible for deciding the appropriate confidence threshold for displaying uncertainty to users in an AI product?

   1. The engineering team based on available computational resources
   2. The legal team based on regulatory requirements for the industry
   3. The product team, based on their target audience and use case
   4. The machine learning model automatically based on its training data

2. What is the consequence of hiding model uncertainty from users to make an AI appear more capable?

   1. User engagement increases because the AI seems more reliable
   2. Regulatory compliance is automatically achieved
   3. Trust is destroyed the moment the first wrong answer is revealed
   4. The model becomes more accurate over time automatically

3. According to the design principles discussed, what is the ideal number of clicks required for a user to recover from an AI-generated suggestion they want to override?

   1. Two clicks, to prevent accidental overrides
   2. One click, to balance speed with user autonomy
   3. Three clicks, to force users to reconsider their choice
   4. Zero clicks—the AI should never suggest anything wrong

4. Which of the following tasks can AI tools currently accomplish in the context of confidence display design?

   1. Determine which users will trust AI suggestions
   2. Decide the right confidence threshold for your audience
   3. Generate microcopy variants for confidence displays
   4. Replace usability testing with model predictions

5. What is the primary purpose of microcopy displayed alongside AI suggestions?

   1. Increase conversion rates by adding persuasive language
   2. Enable the AI to learn from user feedback automatically
   3. Reduce the technical complexity of the interface
   4. Help users understand why the AI made a particular suggestion

6. In AI product design, what does the term 'edit affordance' refer to?

   1. A pricing model that allows editing after purchase
   2. The ability for users to train or fine-tune the AI model itself
   3. A visual or interactive element that signals users can modify AI-generated content
   4. A legal right to modify AI-generated outputs

7. What does 'graceful degradation' mean in the context of AI-powered products?

   1. The AI automatically improves after making mistakes
   2. The product shuts down safely when errors exceed a threshold
   3. The interface becomes simpler as users gain experience
   4. The system continues to provide value even when AI confidence is low

8. What does 'trust calibration' involve in AI product design?

   1. Calibrating the AI model's internal confidence thresholds
   2. Matching the user's trust level to the AI's confidence score
   3. Setting accurate expectations about AI capabilities and limitations
   4. Measuring how much users trust the company behind the AI

9. When designing confidence displays, what trade-off must product designers balance?

   1. Cost of development versus number of supported languages
   2. Speed of user decision-making versus understanding of AI reasoning
   3. Accuracy of AI predictions versus number of features
   4. Privacy of user data versus personalization level

10. A product team is designing a recovery flow after a user rejects an AI suggestion. What should be their primary design priority?

    1. Requiring detailed feedback to improve the AI model
    2. Enabling rapid recovery so users can proceed without delay
    3. Sending a confirmation email for audit purposes
    4. Displaying alternative suggestions from other users

11. What does a 'happy-path specification' describe in AI product design?

    1. The database schema for storing user preferences
    2. The error messages users will encounter
    3. The minimum viable product requirements
    4. The ideal user journey when everything works as expected

12. Why is it important to 'show your work' when AI makes suggestions?

    1. It reduces the computational load on servers
    2. It satisfies legal requirements for algorithmic transparency
    3. It enables users to understand and verify AI reasoning, building trust
    4. It makes the interface more visually appealing

13. In the context of this design domain, what is the strongest argument for one-click recovery?

    1. It maintains user agency without sacrificing the efficiency gains of AI assistance
    2. It reduces the need for customer support tickets
    3. It complies with accessibility standards
    4. It creates more training data for the AI model

14. A PM argues that hiding low-confidence suggestions will create a better user experience because users won't see 'wrong' answers. What is the flaw in this reasoning?

    1. This approach violates data protection regulations
    2. Hidden suggestions require more server resources to maintain
    3. Users will eventually encounter wrong suggestions anyway, and hidden uncertainty will feel like deception
    4. Users prefer to see all options regardless of quality

15. Why can AI not replace usability testing when designing confidence displays and recovery flows?

    1. Because AI generates synthetic data that is more reliable
    2. Because usability testing is legally required for all products
    3. Because real users reveal behaviors and preferences that cannot be predicted from first principles
    4. Because confidence displays are exempt from testing requirements