Debate as an Alignment Method

Two AIs argue opposite sides. A human judges the transcript. The bet: truth is easier to defend than lies, so debate surfaces signal a human alone would miss. Two Lawyers, One Judge Proposed by Irving, Christiano, and Amodei at OpenAI in 2018, AI Safety via Debate structures oversight as an adversarial game.

35 min · Reviewed 2026

Two Lawyers, One Judge

Proposed by Irving, Christiano, and Amodei at OpenAI in 2018, AI Safety via Debate structures oversight as an adversarial game. Two copies of a model take opposite positions on a question. They argue. A human reads the exchange and picks a winner. The hypothesis is that lying requires a more fragile story than telling the truth, so the liar loses over many rounds.

Why adversarial structure helps

A single model can confidently lie to a rater with no counter
In debate, the other model is motivated to expose the lie
The human does not need to know the truth directly — just which argument is stronger
Works for questions where humans can evaluate locally even if they can't reason end-to-end

Where it wobbles

Obfuscated arguments: a well-crafted lie with plausible local steps can beat a clumsy truth
Human judges get fooled by rhetoric, confidence, and length
Debate assumes both sides have equal capability — unequal models break the symmetry
Some questions don't decompose into checkable sub-claims

If the only tool we had were RLHF, we would be stuck. Debate is one attempt at a different tool.
— Geoffrey Irving, UK AI Safety Institute

The big idea: adversarial oversight is a structural idea, not a product. It may or may not scale, but the reasoning behind it — use conflict to surface signal — is worth carrying into any supervision scheme.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-safety2-debate-alignment-creators

What is the main idea of "Debate as an Alignment Method"?
1. Two AIs argue opposite sides.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Debate as an Alignment Method"?
1. scalable oversight
2. debate
3. adversarial training
4. obfuscated argument
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. A single model can confidently lie to a rater with no counter
4. Treat the AI output as automatically correct
What should a careful learner remember about "Toy success"?
1. Use AI to draft or organize ideas about debate, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. AI cannot make the human values decision for you.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about debate be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about debate.
Which action would help you apply "Debate as an Alignment Method" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. In debate, the other model is motivated to expose the lie

← Back to interactive lesson

Tendril · Creators · Ethics & Society