Weak-to-Strong Generalization

What if you have to supervise a student smarter than you? OpenAI's 2023 paper asked that question by using GPT-2 to train GPT-4. The results were surprising.

38 min · Reviewed 2026

The Inverse Tutor Problem

Human teachers will eventually be the weak party. If we want to keep supervising models that are better than us at most tasks, we need to know: does a stronger model, trained on weaker labels, stay stuck at the weak supervisor's ceiling, or can it generalize past it?

The experiment

In December 2023, OpenAI published Weak-to-Strong Generalization. They simulated the future problem today by using GPT-2 as the weak supervisor and GPT-4 as the strong student. GPT-2 generated labels, sometimes wrong ones, and GPT-4 was fine-tuned on them.

Baseline: GPT-4 with perfect labels performs at ceiling
Weak-label training: GPT-4 performance drops, but stays well above GPT-2's level
With training tweaks (auxiliary confidence loss): GPT-4 recovers much of the ceiling gap
The strong model partially 'knows better' than its teacher

Why this is not a solution

Results are uneven across tasks, especially ones requiring long reasoning
Performance gap persists for harder domains like chess endgames
The method assumes the strong model already has the right knowledge in pretraining — not strongly expected after review for future capabilities
Honesty and deception might behave differently than raw performance

Weak-to-strong is not the answer. It is evidence that the shape of the answer might exist.
— Collin Burns, paper co-author, interview (paraphrased)

The big idea: smarter students might partially teach themselves from weaker teachers. That is encouraging but far from sufficient for the real superhuman case.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-safety2-weak-to-strong-creators

What is the main idea of "Weak-to-Strong Generalization"?
1. What if you have to supervise a student smarter than you?
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Weak-to-Strong Generalization"?
1. generalization
2. weak-to-strong
3. superalignment
4. auxiliary loss
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Baseline: GPT-4 with perfect labels performs at ceiling
4. Treat the AI output as automatically correct
What should a careful learner remember about "Why this is a clue"?
1. Use AI to draft or organize ideas about weak-to-strong, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. AI cannot make the human values decision for you.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about weak-to-strong be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about weak-to-strong.
Which action would help you apply "Weak-to-Strong Generalization" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Weak-label training: GPT-4 performance drops, but stays well above GPT-2's level

← Back to interactive lesson

Tendril · Creators · Ethics & Society

Weak-to-Strong Generalization

What if you have to supervise a student smarter than you? OpenAI's 2023 paper asked that question by using GPT-2 to train GPT-4. The results were surprising.

38 min · Reviewed 2026

The Inverse Tutor Problem

The experiment

Baseline: GPT-4 with perfect labels performs at ceiling
Weak-label training: GPT-4 performance drops, but stays well above GPT-2's level
With training tweaks (auxiliary confidence loss): GPT-4 recovers much of the ceiling gap
The strong model partially 'knows better' than its teacher

Why this is not a solution

Results are uneven across tasks, especially ones requiring long reasoning
Performance gap persists for harder domains like chess endgames
The method assumes the strong model already has the right knowledge in pretraining — not strongly expected after review for future capabilities
Honesty and deception might behave differently than raw performance

Weak-to-strong is not the answer. It is evidence that the shape of the answer might exist.
— Collin Burns, paper co-author, interview (paraphrased)

The big idea: smarter students might partially teach themselves from weaker teachers. That is encouraging but far from sufficient for the real superhuman case.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-safety2-weak-to-strong-creators

What is the main idea of "Weak-to-Strong Generalization"?
1. What if you have to supervise a student smarter than you?
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Weak-to-Strong Generalization"?
1. generalization
2. weak-to-strong
3. superalignment
4. auxiliary loss
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Baseline: GPT-4 with perfect labels performs at ceiling
4. Treat the AI output as automatically correct
What should a careful learner remember about "Why this is a clue"?
1. Use AI to draft or organize ideas about weak-to-strong, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. AI cannot make the human values decision for you.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about weak-to-strong be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about weak-to-strong.
Which action would help you apply "Weak-to-Strong Generalization" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Weak-label training: GPT-4 performance drops, but stays well above GPT-2's level

← Back to interactive lesson