Engaging Red Teams for AI Safety Testing

Red teams improve safety; engagement quality shapes outcomes.

What AI does well here

• Engage diverse red team perspectives
• Define scope clearly
• Compensate red teams fairly
• Act on findings substantively

What AI cannot do

• Get safety from red teams alone
• Substitute one-time engagement for ongoing
• Make every issue findable

What makes red team engagement actually work

Red teaming AI systems has become a standard practice at frontier labs — Anthropic, OpenAI, Google DeepMind, and major government agencies all run red team programs before major releases. But the quality of red team outcomes varies enormously based on how engagement is designed. The most common failure is narrow scope: internal teams define the attack surface based on what they already know to worry about, which systematically misses the harms they are not yet imagining. Effective red team programs use genuine outsiders — people from different professional backgrounds, lived experiences, and adversarial mindsets. Former social engineers, independent security researchers, civil society advocates, and domain experts in high-stakes fields (healthcare, legal, finance) find different things than internal ML safety engineers. Compensation matters for quality: underpaid red teamers rush. Psychologically safe disclosure processes matter for thoroughness: red teamers who fear legal blowback self-censor. Most critically, the loop must close: red team findings that are documented and then ignored erode the program's value entirely. The most mature programs track remediation rates, publish summary findings, and re-test after patches.

• Use genuine outsiders with diverse backgrounds — not just internal safety engineers
• Define scope broadly enough to capture harms you have not yet imagined
• Pay red teamers fairly and establish clear legal safe harbor for findings
• Track remediation rates and re-test after fixes — the loop must close