Using AI Vendor Due Diligence in Procurement

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-ethics-ai-procurement-due-diligence-creators

1. A procurement team is evaluating an AI vendor who claims their model is 'the most accurate in the industry.' What should the team do next?

   1. Discard the vendor since claims cannot be verified
   2. Ask the vendor to remove competing products from the market
   3. Request independent benchmarks and validation studies
   4. Accept the claim since the vendor is established

2. Which of the following is a capability of structured ethics due diligence when evaluating AI vendors?

   1. Verifying all vendor claims automatically without human input
   2. Ensuring the AI vendor will never experience any technical failures
   3. Probing training data sourcing and model evaluation transparency
   4. Replacing the need for legal review entirely

3. A procurement officer discovers an AI vendor cannot provide documentation about where their training data came from. What is the most appropriate response?

   1. Include this as a risk factor and require mitigation before contracting
   2. Assume the data is publicly available since the vendor won't share it
   3. Report the vendor to regulatory authorities immediately
   4. Proceed with the contract since the model performs well

4. Why is it insufficient to rely solely on vendor-provided performance metrics when selecting an AI system?

   1. Government regulations prohibit using vendor metrics
   2. Performance metrics are irrelevant to AI evaluation
   3. Vendor-reported metrics may reflect optimized test conditions rather than real-world performance
   4. Vendors never report accurate performance numbers

5. What does 'model evaluation transparency' refer to in AI vendor due diligence?

   1. The vendor publishes their pricing structure publicly
   2. The vendor allows customers to modify the model's code
   3. The vendor shares how models are tested, what benchmarks are used, and how performance is measured
   4. The vendor provides physical access to their servers for inspection

6. What is a fundamental limitation of AI vendor due diligence?

   1. It cannot verify vendor claims without supporting evidence
   2. It cannot be performed on software vendors
   3. It automatically detects all potential ethical violations
   4. It replaces the need for procurement teams

7. During vendor evaluation, a procurement team receives a response sheet full of impressive statistics and certifications. How should these be interpreted?

   1. As verified facts that can be trusted
   2. As marketing materials that require independent verification
   3. As irrelevant information to the procurement decision
   4. As legal documents binding the vendor

8. An AI vendor claims their facial recognition system has 99% accuracy. What evidence would satisfy proper due diligence?

   1. A press release from the vendor
   2. The vendor's marketing brochure
   3. Customer testimonials from the vendor's website
   4. A third-party benchmark evaluation using independent test data

9. Why should procurement teams include incident response capabilities in their AI vendor evaluation?

   1. To replace the vendor's customer support team
   2. To guarantee the AI system will not fail
   3. To understand how the vendor handles AI failures, data breaches, or biased outputs
   4. To ensure the vendor never experiences any incidents

10. What is the relationship between 'procurement' and 'due diligence' in AI vendor selection?

    1. Due diligence is a component of the procurement process focused on ethical and capability assessment
    2. Procurement and due diligence are unrelated processes
    3. Procurement occurs after due diligence is completely finished
    4. Due diligence replaces procurement entirely

11. A procurement manager wants to evaluate whether an AI vendor's model might produce biased outputs. Which question should be included in the vendor questionnaire?

    1. How the vendor evaluates model fairness across demographic groups
    2. How much did the AI system cost to develop
    3. Whether the vendor has ever had a server outage
    4. What geographic regions the vendor operates in

12. What distinguishes thorough AI vendor due diligence from a superficial review?

    1. Superficial review is more comprehensive
    2. There is no meaningful difference between them
    3. Thorough due diligence takes longer and requires evidence for all claims
    4. Thorough due diligence relies on vendor testimonials

13. An AI vendor provides a case study showing excellent results from a pilot program. What should procurement teams consider?

    1. The case study proves the AI will work for any use case
    2. The case study is government-certified
    3. The results may not generalize and require validation in the procurement team's specific context
    4. The vendor is obligated to replicate those exact results

14. Why is it important to ask about data governance when evaluating AI vendors?

    1. To determine if the vendor uses the cheapest data storage
    2. To verify the vendor uses a specific database technology
    3. To ensure the vendor will share all their data with the customer
    4. To understand how data is collected, used, and protected, which affects ethical compliance and risk

15. What should a procurement team do if an AI vendor refuses to answer questions about their training data sourcing?

    1. Immediately terminate all communications without explanation
    2. Consider this a significant red flag requiring risk mitigation or vendor removal
    3. Accept the refusal as standard industry practice
    4. Assume the data was ethically sourced since the vendor is reputable